Legacy of the IBM System/360
As presented to the NIU Student Chapter of the ACM
Dan Greiner
dgreiner@us.ibm.com
IBM z/Server Architecture
10 October 2013, 4:00 pm
IBM Systems and Technology Group (STG)
© Copyright International Business Machines Corporation 2011-2013.
The Legal Stuff

Trademarks:
►
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
–
–
–
–
–
►

►
►
►
ESA/390
IBM
z/Architecture
z/OS
z/VM
IEEE is a trademark of the Institute of Electrical and Electronics Engineers, Inc. in the United States, other
countries, or both.
Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.
Unicode is a registered trademark of Unicode, Incorporated in the United States, other countries, or both.
Other trademarks and registered trademarks are the properties of their respective companies.
All information contained in this document is subject to change without notice. The products described in
this document are not intended for use in applications such as implantation, life support, or other
hazardous uses where malfunction could result in death, bodily injury or catastrophic property damage.
The information contained in this document does not affect or change IBM product specifications or
warranties. Nothing in this document shall operate as an express or implied license or indemnity under the
intellectual property rights of IBM or third parties. All information contained in this document was obtained
in specific environments, and is presented as an illustration. The results obtained in other operating
environments may vary.
 While the information contained herein is believed to be accurate, such information is preliminary, and
should not be relied upon for accuracy or completeness, and no representations or warranties of accuracy
or completeness are made.
 The information in contained in this document is provided on an “AS IS” basis. In no event will IBM be
liable for damages arising directly or indirectly from any use of the information contained in this document.
© Copyright International Business Machines Corporation 2011-2013. Permission is granted to Northern
Illinois University to record this presentation.
2
Topics du Jour:
 Data processing prior to the
System/360
 The System/360 Project
► Requirements
► Development
► Software
► Challenges
 System/360 Legacy
► 49½ years of industry-leading
information processing
IBM System/360 Model 65 Operator’s Console
3
Ancient History: 1952

Beginning of the “computer age”:
► 8:30 p.m. EST, 4 November 1952
► Remington Rand’s Univac-I predicted U. S. General
election results
–
Eisenhower – 438 electoral votes; Stephenson – 93
–
With only 5% of the votes counted, Univac was within 1% of
the final results!
–
CBS (Walter Cronkite’s 1st election coverage) delayed
broadcasting Univac results until after midnight because
they couldn’t believe it!

In that era, IBM was world leader in electro-mechanical
accounting machines
► A legacy of Herman Hollerith’s 1890’s punched-card
IBM 701 Data-Processing Machine
technology

IBM enters the “modern” computer era in late 1952
with the IBM 701 Electronic Data Processing Machine
► 19 units built and sold
4
Setting the Stage
 From 1952-1960:
► IBM revenues grew almost 10x
► Earnings - $305 million; Revenue – 2.6 billion
► Employees grew from 30,000 to over 125,000
 IBM computing product line in the early 1960s:
► Eight separate products
► Completely different architectures, software, & peripherals
► Migrating from one to another – or to a new product – was
difficult
5
S/360’s Ancestors

SAGE – Early Warning Air-Defense System (late ’50s – early ’60s)
► First real-time, integrated, online system over North America
► Each system weighed >250 tons, 60,000 vacuum tubes

NASA Projects (late ’50s & on)
► Real-time systems for Vanguard, Mercury, Gemini, & Apollo missions
► Key to developing multi-processing systems

SABRE – Airline Reservations System (early 1960s & on)
► Initially 1,000 reservations terminals in 60 cities
► Also handled seat selection, rental car info, crew scheduling, fuel management, stand-by
lists, aircraft maintenance info

Stretch – Supercomputer for AEC lab at Los Alamos (IBM 7030, delivered 1961)
► Advanced expertise in ferrite core memory & semiconductors
► 200 x faster than IBM 701 processors; 3 x faster than SAGE processors
► Evolved into IBM 7090 commercial system
6
1962 Environment
 Competition was nipping at IBM’s heels
► Burroughs, Control Data Corp., GE, Honeywell, NCR, RCA, & Univac
 IBM had eight separate, totally incompatible products
► Data-Systems Division (Poughkeepsie, NY):
– Systems that leased for more then $10,000 / month
– IBM 7000-series systems (proposed new 8000-series)
► General-Products Division
– Burlington, VT, Endicott, NY, Rochester, NY, & San Jose, CA
– Systems that leased for less than $10,000 / month
– IBM 1400-series systems (1st computer to sell > 10,000 units);
IBM 1600-series in development
► World-Trade Corporation
– Developing small scientific computer (SCAMP)
► Intense inter-divisional rivalry (and secrecy)
7
1962 Technology
 Solid-state technology was relatively
new
► IBM’s Standard-Modular System
(SMS) contained a handful of
transistors on a 2.5” x 4.5” card
► Solid-Logic Technology (SLT)
IBM Standard-Modular-System Card
– 3 to 6 times the density of SMS cards
– 100 times more reliable
► Integrated circuits
– 100 times the density of SMS cards
– Leading edge, very little background
available
– But … being used by RCA in
competitive products
IBM Solid-Logic-Technology Module
Dual-InlinePackage (DIP)
Integrated Circuit
8
1962 – A Momentous Decision !
 Customers were fed up with incompatible systems
 The SPREAD (Systems, Programming, Research,
Engineering & Development) team
► 11-member team from across divisions
 SPREAD Report:
► Recommended future development on all mainframe
projects be scrapped
► New product line (NPL) would be developed:
– Initially planned on five models (ended up with six)
– Largest system to be 200x faster than smallest
– Performance range accomplished using SLT technology of
differing speed, data-path-width differences, &c.
– Each CPU to have high-speed memory using permanentlystored control information
9
1962 – A Momentous Decision !!
 SPREAD Report:
► Software will be compatible on all levels of the system
– upward and downward!
► Each processor will be economically competitive in its
own marketplace
► Standard I/O interfaces will be used across the entire
product line
– Required development of dozens of new peripherals:
disk drives, tape drives, printers, card readers, magnetic
& optical character readers, communications adapters,
and terminals
► “Since such processors must have capabilities not
now present in any IBM processor product, the new
family of products will not be compatible with our
existing processors.”
10
1962 – 1964: Development
 Estimated total cost in 1962 was $675 million
 Actual development cost closer to $5 billion
► ~ $35 billion in 2013 dollars, adjusted for inflation
► Accustomed to “swimming in cash”, IBM came close
to not being able to meet payroll
► $750 million invested in engineering
► $4.5 billion invested in factories, equipment, and the
product itself
► Fortune called it, “IBM’s $5,000,000,000 Gamble”
11
1962 – 1964: Development
 Honeywell’s H-200 computer (announced 1963):
► “Liberator” program allowed translation of IBM 1401
programs on the Honeywell platform
► IBM lost nearly 200 customers within 2 months after
announcement.
– Overall computer demand up 15% … IBM grew only 7%
– IBM head of sales: “Help, I’m being slaughtered.”
– Potential that the entire NPL would be scrapped
 Emulator option:
► With relatively small microcode investment, NPL could
run existing (1401 & 7090) customer applications on
NPL faster than on existing systems
► Salvaged many customers ready to jump ship
12
System/360 – the Name
 Initially plans for NPL – IBM 500
► Not particularly attractive … but we
needed a number
► 360 represented the number of degrees
on a compass
– Indicated the processor was suited to
any application
– Stylized compass rose chosen as an
icon
– “System” and the slash added “because
it looked nice.”
 Much corporate churn about the name
► President & VP of data-products
division, VP of communications battled
it out
► System/360 won by default when some
clever entrepreneur painted it on the
nose of the corporate aircraft
13
7 April 1964
 IBM System/360 announced to the
world
► Announcement efforts planned for
over a year in advance
► Press conferences in 165 cities & 14
countries
► Estimated 100,000 customers &
prospects
► Special train from NYC to
Poughkeepsie
“A new generation of electronic
computing equipment was introduced
by the IBM Corporation
IBM Board Chairman Thomas J. Watson,
Jr. called the event the most important
announcement in the company’s
history.
The new equipment is known as the IBM
System/360”
14
Post-Announcement Reaction
 Within 4 weeks of the announcement, orders for
1,000 systems were received
► Within the second 4 weeks, 1,000 more orders
 T. J. Watson memoirs:
► [System/360] was the biggest, riskiest decision I ever
made, and I agonized about it for weeks, but deep
down, I believed there was nothing IBM couldn’t do.
► Within IBM there was a great feeling of celebration
because a new era was opening up, … But when I
looked at those new products, I didn’t feel as confident
as I’d have liked. Not all the equipment on display was
real; some units were just mockups made of wood …
an uncomfortable reminder to me of how far we had to
go before we could call the program a success.
15
The Next 18 Months – Major Problem #1
 Technology problems with Solid
Logic Technology (SLT) modules
► Volume requirements would exceed
capacity of vendors
► Pressure for production causing
yield failures
– End of 1965 … 25% SLT failure
► In-house manufacturing facilities
developed:
– East Fishkill, NY plant SLT module
output: 1963 – ½ million; 1964 – 12
million; 1965 – 28 million; 1966 – 90
million
– Plants in Burlington, VT & Essonnes,
France – 26 million each
16
The Next 18 Months – Major Problem #2
 Logistics Problems
► Component delays
– Running out of circuit breakers halted manufacturing
► Software delays
► Competitor announcements
– Marketing pressure to change System/360 to address
competition
► Personality problems
– Friction between engineering and manufacturing (led by
Dick Watson – T.J.’s younger brother) and sales led to
Dick Watson being replaced.
17
The Next 18 Months – Major Problem #3
 Software Problems
► Upward- & downward-compatibility requirements.
► Multiprogramming challenges
► Developing interim products to counter competitive
challenges
► Moving target – constantly changing due to dynamic
marketing requirements
► SW-development, led by Fred Brooks, grew to over 1,000
people
– Brook’s Law: Adding manpower to a late software project
makes it later.
► Some teams working 60-hour weeks
► Initially, over 1,000,000 lines of code (LOC)
– Grew to 10,000,000 LOC
– Budgeted $30-40 million; actual ~ $500 million
18
The Next 18 Months & Onwards
 By December 1966:
► Over 7,000 systems installed
► $4 billion in new revenue
– $1 billion in pre-tax profits
► IBM hired 25,000 new employees in 1966
► Added 3 million square feet of manufacturing space
► Producing 1,000 System/360 units per month
► SLT modules reached projected 33-million-hour MTBF
 By 1970:
► Revenues went from $3.2 billion to $7.5 billion
► Earnings went from $431 million to over $ 1 billion
► Employees grew nearly 120,000 … to 269,000.
19
The System/360 Architectural Legacy
IBM System/360 Model 44
20
In the Beginning … System/360 (1964)

CPU Architecture
► 32-bit arithmetic
► 16 general-purpose registers
► 24-bit addressing (16,777,216 bytes max.)
–
More than a few megabytes was quite rare
► Real addressing only! No virtual memory
► Approximately 142 instructions total
► Some features were optional

–
Decimal instructions (in-storage only)
–
Floating point (with 4 floating-point registers)
–
Direct control (specialty I/O for check sorters, &c.)
–
Protection feature (i.e., storage keys)
I/O architecture
► Maximum of 7 channels
–
One byte-multiplexor channel (printers, card
readers, &c)
–
Up to seven selector channels (disks, tape)
IBM System/360 Model 50
► Maximum of 256 devices per channel
► Most machines had far fewer channels &
devices
21
In the Beginning … System/360 (1964)
 Storage technology
► Ferrite core storage
– Each toroid “donut” represented one
bit
► Architectural maximum: 16 megabytes
– Reality: Most customers had no more
than 1-2 megabytes
► Increasing density … the donut-hole
IBM System/360 Model 50
test:
– New product’s core toroid fit through
IBM System/360 Core-Memory Panels (approx. 32K)
the donut hole of the previous
product’s core
22
System/360 Software
 Operating Systems
 Languages
► Basic Operating System
(BOS)
► ALGOL
► Tape Operating System (TOS)
► Basic
► Disk Operating System (DOS)
► COBOL
► Operating System / Multiple
Fixed Tasks (OS/MFT)
► Fortran
► Operating System / Multiple
Variable Tasks (OS/MVT)
► RPG
► SABRE (Airline Reservations)
► Assembler
► PL/1
 Online Transaction Processing
► Time-Sharing System (TSS)
► Customer Information
Control System (CICS)
► Control Program / 67 (CP/67)
with the Cambridge Monitor
System (CMS)
► Conversational Programming
System (CPS)
 Numerous independentsoftware-vendor packages
23
System 370 (1970)

Virtual addressing
► One or multiple 24-bit virtual spaces
► Ability to over-commit real storage
► Ability to segregate data according to
subsystem and user

Real storage still limited to 24-bit
addressing
► Most CPUs still had (far) less than 16 Mbytes
► Monolithic solid-state memory … no more
core


Introduced new 16 control registers
Introduced 13 new instructions
► Load/store control registers
► Compare / move long streams of characters
► Compare / insert / store character under
mask

I/O subsystem expanded to (up to) 16
IBM System/370 Model 138
channels
► 32 channel option on some later machines

Hardware-assisted debugging (PER)
24
System/370 Software
 Operating Systems
► Disk Operating System / Virtual
Storage Extended (DOS/VSE)
► Single Virtual Storage (SVS)
► Multiple Virtual Storage (MVS)
► Transaction Processing Facility
(TPF, successor to SABRE)
► Virtual Machine / 370 (VM/370)
 Database Systems
► Information Management System
(IMS)
IBM System/370 Model 148
► DB/2
► System Query Language (SQL)
25
System 370 Enhancements (1978-1982)
 26-bit real addressing (up to 64 MB)
► Single virtual address space still
limited to 24 bits (16 MB)
 Various specialty facilities
► Operating-system assist
instructions for obtaining /
releasing locks, tracing, &c.
► Mathematical assists
(transcendental functions, &c.)
► High-accuracy arithmetic
 Dual-address space
► Primary & secondary spaces
► New instructions:
IBM 3031 Processor Complex
– PROGRAM CALL / PROGRAM
RETURN
– INSERT / SET ADDRESS SPACE
CONTROL
– MOVE TO PRIMARY / SECONDARY
26
370 Extended Architecture (1983)
 31-bit virtual addressing
► Single address space providing up to 2
G-bytes
► Bimodal addressing (1 bit of the 32-bit
address stolen to designate 24- or 31-bit
addressing mode)
 31-bit real addressing (2 gigabytes max.)
► Most customers still had much less
memory
 Entirely new I/O subsystem
► Up to 256 I/O channels
► Up to 65,536 I/O devices
 New instructions for I/O, storage key
manipulation, and program linkage
 Hardware tracing
IBM 3081 Processor Complex
► Eliminated significant bottleneck in
multiprocessor tracing
27
Interpretive Execution (1984)
 Hardware virtualization
► Used by Processor Resource /
System Manager (PR/SM)
– Provides logical partitioning of
physical machine
► Used by VM/370 Operating System
► Apportions physical resources to
“guest” programs
– CPUs, I/O channels, real memory
► Allows multiple operating systems to
operate concurrently / separately
IBM 3084 Processor Complex
► Extremely fast context switch from
one guest to another
28
A Few Side Trips (1980s)
 4300-series processor models
► Three models of mid-sized, lowercost processors for departmentlevel or small business
applications
► Same CPU architecture and I/O
capabilities
IBM 4381 Processor Complex
 Vector facility
► Single-engine multiple-data (SIMD)
instruction extension
► Intended for large-array analytics:
oil research, weather prediction,
fluid-dynamics modeling
IBM 3090 Processor Complex
29
Enterprise Systems Architecture / 370 (1989)
 Introduced access-register translation
(ART)
► Provided the means by which a
program could access multiple
address spaces with minimum
overhead
► Up to 2,048 31-bit (2 G-byte) address
spaces (i.e., up to 4 T-bytes)
► New nonprivileged instructions for
manipulation of ARs
 Introduced the home address space
► Location for principal task (process)
control structures
► Where to go (i.e., who to blame) when
a task abnormally ends
 Introduced the linkage stack
IBM System 390
► Push-down stack for semi-authorized
tasks
► New instructions for manipulation of
linkage stack.
30
Enterprise Systems Architecture / 390 (1990-1999)
 Numerous new facilities:
► Arithmetic instructions with 16-bit
immediate operands
► Branch instructions with 16-bit
relative-branch location
► Binary-floating-point instructions
(IEEE standard)
– 95 new instructions and floatingpoint control register
► Compression facility
► Check-sum instruction (for TCP/IP
& others)
► Extended-translation instructions
(Unicode™ conversion)
► Sorting-assist instructions
► String-manipulation instructions
IBM Enterprise Server 9000
 Enhancements to dual-addressspace facilities
31
Mainframe Perceptions in the 1990s
 Reports of the death of the
mainframe were premature
► “I predict that the last
mainframe will be unplugged on
March 15, 1996.”
– Stewart Alsop, March 1991
► “It’s clear that corporate
customers still like to have
centrally controlled, very
predictable, reliable computing
systems—exactly the kind of
systems that IBM specializes
in.”
– Stewart Alsop, February 2002
Stuart Alsop (from IBM 2001 Annual Report)
32
z/Architecture & the zSeries z900 & z800 (2000)
 General registers grew to 64 bits
► Existing 32-bit instructions retained
– Used rightmost 32 bits of the 64-bit
registers
► Large suite of 64-bit analogues added
► 163 new instructions (139 general, 11
control, 12 floating point)
 Provides 64-bit virtual address space
► Up to 16 exabytes in a single space
– 18,446,744,073,709,551,616 bytes
► With access-register translation, up to
2,048 spaces (275 bytes)
– 37,778,931,862,957,161,709,568 bytes
► Trimodal (24-, 31-, or 64-bit)
addressing
– Ability to dynamically switch between
addressing modes
 64-bit real storage addressing
► Current models offer up to six
terabytes of real storage
IBM eServer zSeries 900
33
zSeries 64-Bit Operating Systems
 z/OS
► Successor to MFT, MVT, SVS, MVS, OS/390
► “Western civilization runs on z/OS” – Bob Rogers, IBM DE Emeritus
 z/VM
► Successor to CP/67, VM/370 and follow-on systems
► Gold standard of virtualization
 z/TPF
► Continuing support of large-scale transaction processing used by
airlines, hotels, car-rentals, &c
 zLinux
► Mainframe-class server software with attitude
34
Facilities Added to to the z900 & z800
 Extended-translation facilities
► Provides for translating, comparing, and moving single- and
double-byte character representations
 Multiply and add / subtract instructions
► Enhancements to classic hexadecimal floating point
► Key to advanced math & crypto operations
 Long-displacement facility
► Provides significant register-constraint relief
► Increases displacement in storage-accessing instructions
from 4K to 1M
35
Facilities Added in the zSeries z990 & z890 (2002)
 Message-Security Assist
► Five general instructions that
perform wide variety of HW
cryptographic operations
► DEA-64, -128, 192; SHA-1
 More extended-translation facility
enhancements
► Instructions to perform
Unicode™ conversions
 Functional enhancements to
cross-memory linkage operations
IBM eServer zSeries 990
36
Facilities Added in the z9-109 (2005)

Decimal floating point facility
► 54 new instructions based on the emerging
DFP standard

Additional instructions with extended (32bit) immediate operands
► Relieves register constraints & cache
accesses

Improved timing-facility instructions
► Allows synchronization of TOD clocks to
within nanoseconds across great distances

Unnormalized extensions for classic
hexadecimal floating point (useful for crypto
operations)

Message-security-assist enhancements #1
IBM eServer zSeries z9-109
► SHA-256, AES-128, & PRNG
37
Facilities Added in the System z10 (2008)

Instructions to provide better indication of
machine topology
► Cache cognizance / prefetching of
operands

New compare-and-branch / compare-andtrap combo instructions

Additional instructions with long immediate
operands

Additional instructions with relative
addresses



Rotate-then-***-selected bits instructions
Instructions to facilitate text parsing
Message-security-assist enhancements # 2
► AES-192, AES-256, & SHA-512


IBM Enterprise Class z10
“Large page” dynamic address translation
CPU measurement facility
38
Facilities Added in the z196 (2010)

High-word facility
►

Interlocked-access facility
►




Effectively provides 16 additional 32-bit general registers
Improved multiprocessor serialized access to storage
Load / store-on-condition facility
►
Conditional execution of an instruction based on CC
►
Significant code performance improvement
Distinct-operands facility
BFP and DFP floating-point enhancements
Message-security-assist enhancements #3 & 4
►
Provides hardware-secured encrypted keys for existing
functions
►
CFB-, OFB-, CMAC-, CC-, GC-, and XTS-modes of
encryption


Numerous other performance enhancements.
IBM zEnterprise z196
BladeCenter – direct attachment of blade-technology
servers of other architectures.
39
Facilities Added in the zEC12 (2012)

DFP zoned-conversion facility
►
Allows efficient calculation of packed / zoned values, with
minimum storage accesses

Enhanced-DAT Facility 2
►

Interlocked-access facility 2
►

2 G-byte storage frame
Simplifies multi-programming
Optimizations for Java
►
Load-and-trap facility
►
Execution-hint & branch-prediction facilities
►
Miscellaneous general instruction enhancements
►
Run-time instrumentation facility
–

Dynamically monitor and alter workloads
Transactional-execution facility
►
First generally-available processor to provide transactional
memory implementation.
►
Potential for significant multiprocessing improvements
►
Lock elision
►
Speculative execution
IBM zEnterprise EC12
40
??
S/360 Legacy Systems – Instruction Growth
1000
INSTRUCTIONS
800
System/360 – ESA/390
600
z/Architecture
400
200
0
1960
1970
1980
1990
2000
2010
2010
YEAR
41
Summary (1)
 System/360 was the evolutionary outgrowth of the IBM products
of the early 1960s
► An amalgam of existing hardware technology
► A completely new CPU architecture
 Significant corporate gamble
► Costs greatly exceeded expectations … but
► Results were successful beyond all expectations
 CPU architecture that is robust and enduring
► Upwards and downwards software compatibility across all product
models
► Application programs from 1964 can still successfully execute on
the latest z/Architecture processors
► Architectural legacy that continues to grow
42
Summary (2)
 S/360 architecture has evolved to provide numerous
enhancements:
► Additional addressing (24  31  64-bit addresses)
► Larger binary data (32  64-bit registers)
► Two additional floating-point representations (HFP + BFP & DFP)
► Advanced program-linkage operations
► Ability to access multiple address spaces simultaneously
► Much broader I/O capacity
– Over 1,000 I/O channels using fibre-channel, Ethernet, PCI, &c
► Additional instructions for compiler efficiency
–
–
–
–
Register constraint relief
Cache optimization
General performance benefit
z/Architecture now includes over 1,000 instructions !!
 All while retaining application-program compatibility with the
original S/360 instruction set
43