Server Time Protocol
Implementation Considerations
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
2
© 2010 IBM Corporation
System (Sysplex) Time task
Timing Network tab
3
© 2010 IBM Corporation
General HMC Panel Overview
ETR Feature Installed
4
ETR Feature NOT Installed
STP Feature
Installed
STP Feature
NOT Installed
STP Feature
Installed
ETR Configuration


ETR Status


Timing Network


Network
Configuration


STP Configuration


STP Status


ETS Configuration


STP Feature
NOT Installed
System
(Sysplex) Time
task is not
available
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
5
© 2010 IBM Corporation
Configuring an External Time Source (ETS)
 The ETS can be configured as:
– Dial out to a telephone time service
• NIST Automated Computer Time Service (ACTS)
• NRC Canadian Time Service (CTS)
• IEN Telephone Date Code (CTD)
– NTP server
– NTP server with pulse per second (PPS) output
6
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
7
© 2010 IBM Corporation
Configuring Dial out as ETS
 Steps include:
– Configure an HMC to dial out to a telephone time
service. At the Customize Outbound Connectivity
task on the HMC, check both:
• Enable local system as a call-home server
• Allow external time source dialing using the local
modem
– Add scheduled operations to the Preferred Time
Server (PTS) and the Backup Time Server (BTS),
if applicable, to access the ETS.
8
© 2010 IBM Corporation
HMC Setup for ETS Dial out configuration
Customize Outbound
Connectivity:
External Time Source
9
© 2010 IBM Corporation
HMC Setup for ETS Dial out configuration
Test External Time
Source Phone number
10
© 2010 IBM Corporation
Configure ETS Dial Out Schedule
These panels are specific to the
Support Element (SE) that you
are setting up for
 Need to set up for CTS
 Recommended to set up
for BTS also
11
© 2010 IBM Corporation
Setup Scheduled Operation/Repeat
12
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
13
© 2010 IBM Corporation
NTP Server on HMC as ETS
non-System z
time
synchronized
servers
NTP server
Stratum 1
System z HMC
NTP server enabled
Stratum 2 or above
July 14 14:21:00 2007 UTC
10.10.10.53
10.1.6.48
eth1
Intranet
eth0
F1
Ethernet
Switch
F2
Internet
SNTP
client
PTS / CTS
us.pool.ntp.org
NTP server
Stratum 1
14
© 2010 IBM Corporation
Configuring an NTP Server on HMC
Customize Console Date and Time – initial view
15
© 2010 IBM Corporation
Configuring an NTP Server on HMC
Add a corporate NTP server
 The server host name or the
IP address of a known NTP
server must be entered in this
window.
 This is the IP address of a
NTP server on the corporate
network
16
© 2010 IBM Corporation
Configuring an NTP Server on HMC
Enable NTP service on this console
 To turn on the NTP service,
check the “Enable NTP
service on this console”
checkbox and click OK.
 When you click OK, you
should be able to see the
console clock at the bottom
right corner get updated to
the server's time.
 Even though the time
change is instantaneous, it
still takes time for the
console's NTP code to
ensure that the HMC’s NTP
server has started properly.
17
© 2010 IBM Corporation
Configuring an NTP Server on HMC
Enable this Hardware Management Console to act as a time server
 To allow another HMC
console, TKE console, or
an STP-only CTN to use
the HMC as a NTP server,
check the “Enable this
Hardware Management
Console to act as a time
server” checkbox and click
the OK button.
18
© 2010 IBM Corporation
Configuring an NTP server on the PTS/CTS or BTS
 This is the IP address of the HMC NTP Server residing on the private network.
 It is different from the IP address of the NTP server on the corporate network.
19
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
20
© 2010 IBM Corporation
Forward Migration
DISCLAIMER
 The following example of a forward migration is
comprised of excerpts from
– System z Expo customer presentation, with the
customer’s permission
– STP Implementation Guide (SG24-7281)
 Some displays will not match the configuration
diagrams. The examples shown were not meant to
have continuity, they are used to show the logical
migration flow for learning purposes.
21
© 2010 IBM Corporation
Starting Environment
Z9CD - 2094
Z9BE - 2094
z/OS 1.8
SYSB
z/OS 1.8
SYSE
Z9G - 2096
z/OS 1.8
SYSG
ETR Timing Mode
ETR Timing Mode
z/OS 1.8
SYSC
z/OS 1.8
SYSD
CFCC
CF14
ETR Timing Mode
Site 1 Data
Center *
Z9LPC4 - 2094
ETR ID = 01
ETR
• Blue Lines
STP (Coupling Links)
• Solid Green Lines = ICB4+3
• Dash Green Lines = ISC3
* Link redundancy not shown
22
9037-002
9037-002
CFCC
ETR1
CF14
ETR Timing Mode
ETR ID = 01
ETR2
Z9OC - 2094
z/OS 1.8
SYSA
GDPS/CBU
SYSK
CFCC
CF14
Site 2 Data
Center *
ETR Timing Mode
* DWDM not shown
Site 1 and Site 2 Data Centers are
located approx X km apart.
© 2010 IBM Corporation
Preparatory Work
 General Pre-Implementation Tasks
– Hardware
•
•
•
•
Install FC 1021 to STP-enable our processors
Check hardware and HMC driver levels
Installed 9037 LIC upgrade
Check STP website to ensure that DWDMs are qualified
– Software
• All systems at a STP-supported z/OS release with latest RSUs applied.
• Enhanced PSP Bucket (FUNCTION_STP).
– www14.software.ibm.com/webapp/set2/psp/srchBroker
• CLOCKxx was updated (Note: These are the default values)
– STPMODE=YES
– STPZONE=YES
• IPL all systems*.
* Images must be IPL’ed after installing new STP maintenance and STP FC 1021.
23
© 2010 IBM Corporation
Selection of Qualified DWDM Products
 Selecting qualified WDM vendor does not mean that WDM model selected is
qualified
– Vendors do not qualify all models
 Selecting qualified WDM model does not mean that specific Release level is
qualified
– Vendors do not qualify all release levels
 Ensure that the vendor qualification letter is inspected carefully by a DWDM
technical expert.
– Letter specifies model number, release level, interface modules qualified,
protocols, application limitations, etc
 Vendors to date having qualified DWDM models:
The latest list of qualified DWDM
– Adva Optical Networking
vendor products can be found on the
Resource Link™ Web site at:
– Ciena Corporation
https://www.ibm.com/servers/resourcelink/
– Cisco Systems
They are listed under the Hardware
– Ericsson AB
Products for Servers on the Library page
– Huawei Technology
– Lucent Technologies (qualified 2002 – not current technologies)
– Nortel
24
© 2010 IBM Corporation
Implementation
 Migrate ETR Network to Mixed CTN
– Set CTN ID on all servers
– Migrate first server to STP timing mode
– Migrate next two servers to STP timing mode
 Migrate from Mixed to STP-Only CTN
25
© 2010 IBM Corporation
ATTENTION!
 Selecting the System (Sysplex)
Time icon results in this panel
 Do not be alarmed! This is
normal
 The user should read this panel
– To be aware of what is
disruptive
– Answer Yes to get to the STP
panels
26
© 2010 IBM Corporation
Set CTN ID
ETR MODE
Note: In ETR
mode, the
ETR Network
ID cannot be
changed.
Note: Enter
STP Network
ID here, then
hit Apply.
27
© 2010 IBM Corporation
Timing Network - after assigning CTN ID to FIRST server
Mixed CTN
28
© 2010 IBM Corporation
STP Status - after assigning CTN ID to FIRST server
29
© 2010 IBM Corporation
STP Status - after assigning CTN ID to SECOND server
30
© 2010 IBM Corporation
SYSLOG Message
IXC438I COORDINATED TIMING INFORMATION HAS BEEN UPDATED
FOR SYSTEM: SYSA
PREVIOUS ETR NETID:
01
CURRENT CTNID:
DDS_STP1-01
CTN ID! We are
now in a Mixed CTN
Mixed CTN
31
© 2010 IBM Corporation
“D ETR” output, Before and After
32
BEFORE
D ETR
IEA282I 17.45.06 TIMING STATUS
SYNCHRONIZATION MODE = ETR
CPC PORT 0
ACTIVE ==>
OPERATIONAL
ENABLED
ETR NET ID=01
ETR PORT=05
ETR ID=01
CPC PORT 1
OPERATIONAL
ENABLED
ETR NET ID=01
ETR PORT=05
ETR ID=02
AFTER
D ETR
IEA282I 17.46.55 TIMING STATUS
SYNCHRONIZATION MODE = ETR
CPC PORT 0
ACTIVE ==>
OPERATIONAL
ENABLED
ETR NET ID=01
ETR PORT=05
ETR ID=01
THIS SERVER IS PART OF TIMING
CPC PORT 1
OPERATIONAL
CTN ID Displayed
ENABLED
ETR NET ID=01
ETR PORT=05
ETR ID=02
NETWORK DDS_STP1-01
Mixed CTN
© 2010 IBM Corporation
“D XCF,S,ALL” output, Before and After
BEFORE
D XCF,S,ALL
IXC335I 17.44.24 DISPLAY XCF
SYSTEM
TYPE SERIAL LPAR STATUS TIME
SYSTEM STATUS
SYSA
2094 XXXX
02
07/07/2008 17:44:21 ACTIVE TM=ETR
SYSH
2094 XXXX
03
07/07/2008 17:44:23 ACTIVE TM=ETR
As Expected, Nothing Changed!
AFTER
D XCF,S,ALL
IXC335I 17.48.34 DISPLAY XCF
SYSTEM
TYPE SERIAL LPAR STATUS TIME
SYSTEM STATUS
SYSA
2094 XXXX
02
07/07/2008 17:48:34 ACTIVE TM=ETR
SYSH
2094 XXXX
03
07/07/2008 17:48:33 ACTIVE TM=ETR
33
© 2010 IBM Corporation
Environment – After CTN ID Set on ALL servers
Z9CD - 2094
Z9BE - 2094
z/OS 1.8
SYSB
z/OS 1.8
SYSE
Z9G - 2096
z/OS 1.8
SYSG
ETR Timing Mode
ETR Timing Mode
Stratum 1, DDS_STP1-01
Stratum 1, DDS_STP101
Site 1 Data
Center *
z/OS 1.8
SYSC
z/OS 1.8
SYSD
CFCC
CF15
ETR Timing Mode
Stratum 1, DDS_STP101
Z9LPC4 - 2094
9037-002
9037-002
ETR ID = 01
CFCC
ETR1
CF15
ETR Timing Mode
ETR ID = 01
ETR2
Stratum 1,
DDS_STP1-01
ETR
• Blue Lines
STP (Coupling Links)
• Solid Green Lines = ICB4+3
• Dash Green Lines = ISC3
* Link redundancy not shown
34
Z9OC - 2094
z/OS 1.8
SYSA
GDPS/CBU
SYSK
CFCC
CF15
Site 2 Data
Center *
ETR Timing Mode
Stratum 1, DDS_STP1-01
* DWDM not shown
Site 1 and Site 2 Data Centers are
located approx X km apart.
© 2010 IBM Corporation
Implementation
 Migrate ETR Network to Mixed CTN
– Set CTN ID on all servers
– Migrate first server to STP timing mode
– Migrate next two servers to STP timing mode
 Migrate from Mixed to STP-Only CTN
35
© 2010 IBM Corporation
Migrate to STP timing mode – STP Status tab Verification
Before disabling ETR ports, attached systems connectivity must be verified from STP Status tab
36
© 2010 IBM Corporation
Migrate to STP timing mode – Display ETR Ports
37
© 2010 IBM Corporation
Migrate to STP timing mode – Disable ETR Ports
Note: Ports can
be disabled at the
same time
Click Port 1 State -> Disabled radio button, then Apply.
Repeat for Port 0.
38
© 2010 IBM Corporation
Migrate to STP timing mode – SYSLOG Messages
*IEA393I ETR PORT 1 IS NOT OPERATIONAL. THIS MAY BE A CTN
CONFIGURATION CHANGE.
IEA390I TOD CLOCKS DYNAMICALLY ADJUSTED TO MAINTAIN STP
SYNCHRONISM.
*IEA393I ETR PORT 0 IS NOT OPERATIONAL. THIS MAY BE A CTN
CONFIGURATION CHANGE.
IEA380I THIS SYSTEM IS NOW OPERATING IN STP TIMING MODE.
JOHN
All systems still alive, CPR (or resume) NOT Required!
39
Mixed CTN
© 2010 IBM Corporation
Migrate to STP timing mode – STP Status tab verification
After the ports have been disabled, the timing mode can be checked using the STP Status tab.
Timing mode
Stratum level
40
© 2010 IBM Corporation
Z9OC Hardware Messages
Do not be alarmed!
These messages are normal when ETR ports are disabled.
41
© 2010 IBM Corporation
Z9OC Hardware Messages
42
© 2010 IBM Corporation
Z9OC Hardware Messages
43
© 2010 IBM Corporation
Migrate to STP timing mode – ETR Status After
44
© 2010 IBM Corporation
Z9OC D ETR Before and After
D ETR
IEA282I 07.05.37 TIMING STATUS 514
SYNCHRONIZATION MODE = ETR
CPC PORT 0
ACTIVE ==>
CPC PORT 1
OPERATIONAL
OPERATIONAL
ENABLED
ENABLED
ETR NET ID=01
ETR NET ID=01
ETR PORT=10
ETR PORT=08
ETR ID=01
ETR ID=02
THIS SERVER IS PART OF TIMING NETWORK DDS_STP1-01
D ETR
IEA386I 07.07.41 TIMING STATUS
SYNCHRONIZATION MODE = STP
THIS SERVER IS A STRATUM 2
CTN ID = DDS_STP1-1
NUMBER OF USABLE TIMING LINKS =
45
9
© 2010 IBM Corporation
Z9OC D XCF Before and After
D XCF,S,ALL
IXC335I 07.05.44 DISPLAY XCF
SYSTEM
TYPE SERIAL LPAR STATUS TIME
SYSTEM STATUS
SYSA
2094 XXXX
02
07/19/2008 07:05:43 ACTIVE TM=ETR
SYSH
2094 XXXX
03
07/19/2008 07:05:41 ACTIVE TM=ETR
D XCF,S,ALL
IXC335I 07.09.48 DISPLAY XCF
SYSTEM
TYPE SERIAL LPAR STATUS TIME
SYSTEM STATUS
SYSA
2094 XXXX
02
07/19/2008 07:09:47 ACTIVE TM=STP
SYSH
2094 XXXX
03
07/19/2008 07:09:41 ACTIVE TM=STP
46
© 2010 IBM Corporation
Site 1 / Site 2 – Mixed CTN
Diagram 1 (Z9OC to STP timing mode)
Z9CD - 2094
Z9BE - 2094
z/OS 1.8
SYSB
z/OS 1.8
SYSE
Z9G - 2096
z/OS 1.8
SYSG
ETR Timing Mode
ETR Timing Mode
Stratum 1, DDS_STP1-01
Stratum 1, DDS_STP101
Site 1 Data
Center *
z/OS 1.8
SYSC
z/OS 1.8
SYSD
CFCC
CF15
ETR Timing Mode
Stratum 1, DDS_STP101
Z9LPC4 - 2094
9037-002
9037-002
ETR ID = 01
CFCC
ETR1
CF15
ETR Timing Mode
ETR ID = 02
ETR2
Stratum 1,
DDS_STP1-01
X
X
ETR
• Blue Lines
STP (Coupling Links)
• Solid Green Lines = ICB4+3
• Dash Green Lines = ISC3
* Link redundancy not shown
47
Z9OC - 2094
z/OS 1.8
SYSA
GDPS/CBU
SYSK
CFCC
CF15
Site 2 Data
Center *
STP Timing Mode
Stratum 2, DDS_STP1-01
* DWDM not shown
Site 1 and Site 2 Data Centers are
located approx X km apart.
© 2010 IBM Corporation
Implementation
 Migrate ETR Network to Mixed CTN
– Set CTN ID on all servers
– Migrate first server to STP timing mode
– Migrate next two servers to STP timing mode
 Migrate from Mixed to STP-Only CTN
48
© 2010 IBM Corporation
Site 1 / Site 2 – Mixed CTN
Diagram 2 (Z9BE and Z9G to STP timing mode)
Z9CD - 2094
Z9BE - 2094
z/OS 1.8
SYSB
z/OS 1.8
SYSE
Z9G - 2096
z/OS 1.8
SYSG
ETR Timing Mode
STP Timing Mode
Stratum 2, DDS_STP1-01
Stratum 2, DDS_STP101
z/OS 1.8
SYSC
z/OS 1.8
SYSD
CFCC
CF15
ETR Timing Mode
Stratum 1,DDS_STP1-01
X
Site 1 Data
Center *
X
X
X
Z9LPC4 - 2094
9037-002
9037-002
ETR ID = 01
CFCC
ETR1
CF15
ETR Timing Mode
ETR ID = 02
ETR2
Stratum 1,DDS_STP101
ETR
• Blue Lines
STP (Coupling Links)
• Solid Green Lines = ICB4+3
• Dash Green Lines = ISC3
* Link redundancy not shown
49
Z9OC - 2094
z/OS 1.8
SYSA
GDPS/CBU
SYSK
CFCC
CF15
Site 2 Data
Center *
STP Timing Mode
Stratum 2, DDS_STP1-01
* DWDM not shown
Site 1 and Site 2 Data Centers are
located approx X km apart.
© 2010 IBM Corporation
Implementation
 Migrate ETR Network to Mixed CTN
– Set CTN ID on all servers
– Migrate first server to STP timing mode
– Migrate next two servers to STP timing mode
 Migrate from Mixed to STP-Only CTN
50
© 2010 IBM Corporation
Network Configuration (Where to go to Assign)
Important: Assignment of server roles must be done on the server that will
become the Current Time Server for the STP-only CTN.
51
© 2010 IBM Corporation
Assign STP Roles
52
© 2010 IBM Corporation
Z9CD SYSLOG Messages – STP-Only CTN
IEA390I TOD CLOCKS DYNAMICALLY ADJUSTED TO MAINTAIN STP SYNCHRONISM.
*IEA393I ETR PORT 0 IS NOT OPERATIONAL. THIS MAY BE A CTN
CONFIGURATION CHANGE.
*IEA393I ETR PORT 1 IS NOT OPERATIONAL. THIS MAY BE A CTN
CONFIGURATION CHANGE.
IEA380I THIS SYSTEM IS NOW OPERATING IN STP TIMING MODE.
IXC438I COORDINATED TIMING INFORMATION HAS BEEN UPDATED
FOR SYSTEM: SYSC
PREVIOUS CTNID:
DDS_STP1-01
CURRENT CTNID:
DDS_STP1
53
© 2010 IBM Corporation
Z9CD SYSLOG Messages – STP-Only CTN
BEFORE
At 00:10:30.89
*IXC439E ALL SYSTEMS IN SYSPLEX HQPLEX1 ARE NOT SYNCHRONIZED
TO THE SAME TIME REFERENCE.
SYSTEM: SYSK IS USING CTNID: DDS_STP1
SYSTEM: SYSB IS USING CTNID: DDS_STP1
SYSTEM: SYSG IS USING CTNID: DDS_STP1
SYSTEM: SYSC IS USING CTNID: DDS_STP1-01
SYSTEM: SYSD IS USING CTNID: DDS_STP1-01
SYSTEM: SYSE IS USING CTNID: DDS_STP1
AFTER
At 00:10:31.90
IXC435I ALL SYSTEMS IN SYSPLEX HQPLEX1 ARE NOW SYNCHRONIZED
TO THE SAME TIME REFERENCE.
SYSTEM: SYSK IS USING CTNID: DDS_STP1
SYSTEM: SYSB IS USING CTNID: DDS_STP1
SYSTEM: SYSG IS USING CTNID: DDS_STP1
SYSTEM: SYSC IS USING CTNID: DDS_STP1
SYSTEM: SYSD IS USING CTNID: DDS_STP1
SYSTEM: SYSE IS USING CTNID: DDS_STP1
54
© 2010 IBM Corporation
Site 1 / Site 2 – Migrate to STP CTN
Diagram 3 (STP LIC disables ETR ports on z9CD & Z9LPC4 when roles assigned)
Z9CD - 2094
Z9BE - 2094
z/OS 1.8
SYSB
z/OS 1.8
SYSE
Z9G - 2096
z/OS 1.8
Arbiter
SYSG
STP Timing Mode
STP Timing Mode
Stratum 2, DDS_STP1
Stratum 2, DDS_STP1
z/OS 1.8
SYSC
z/OS 1.8
SYSD
CFCC
CF15
STP Timing Mode
Stratum 1,DDS_STP1
X
PTS/CTS
X
Site 1 Data
Center *
Z9LPC4 - 2094
9037-002
9037-002
ETR ID = 01
CFCC
ETR1
X
CF15
STP Timing Mode
X
ETR ID = 02
ETR2
Stratum 2,DDS_STP1
ETR
• Blue Lines
STP (Coupling Links)
• Solid Green Lines = ICB4+3
• Dash Green Lines = ISC3
* Link redundancy not shown
55
Z9OC - 2094
z/OS 1.8
SYSA
GDPS/CBU
SYSK
CFCC
CF15
Site 2 Data
Center *
STP Timing Mode
Stratum 2, DDS_STP1
BTS
* DWDM not shown
Site 1 and Site 2 Data Centers are
located approx X km apart.
© 2010 IBM Corporation
Z9CD “D ETR” output, Before and After
BEFORE
D ETR
IEA282I 00.03.46 TIMING STATUS
SYNCHRONIZATION MODE = ETR
CPC PORT 0
ACTIVE ==>
OPERATIONAL
ENABLED
ETR NET ID=01
ETR PORT=08
ETR ID=01
THIS SERVER IS PART OF TIMING
CPC PORT 1
OPERATIONAL
ENABLED
ETR NET ID=01
ETR PORT=00
ETR ID=02
NETWORK DDS_STP1-01
AFTER
D ETR
IEA386I 00.11.54 TIMING STATUS
SYNCHRONIZATION MODE = STP
THIS SERVER IS A STRATUM 1
CTN ID = DDS_STP1
THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000XXXXX
THIS IS THE PREFERRED TIME SERVER
56
© 2010 IBM Corporation
Z9CD “D XCF,S,ALL” Before and After
D XCF,S,ALL
BEFORE
IXC335I 00.04.18 DISPLAY XCF 976
SYSTEM
TYPE SERIAL LPAR STATUS TIME
SYSK
2094 XXXX
04
08/03/2008 00:04:14
SYSB
2094 XXXX
01
08/03/2008 00:04:17
SYSG
2096 XXXX
01
08/03/2008 00:04:17
SYSC
2094 XXXX
01
08/03/2008 00:04:15
SYSD
2094 XXXX
02
08/03/2008 00:04:16
SYSE
2094 XXXX
02
08/03/2008 00:04:17
SYSTEM STATUS
ACTIVE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
TM=STP
TM=STP
TM=STP
TM=ETR
TM=ETR
TM=STP
AFTER
D XCF,S,ALL
IXC335I 00.10.51 DISPLAY XCF 134
SYSTEM
TYPE SERIAL LPAR STATUS TIME
SYSK
2094 XXXX
04
08/03/2008 00:10:50
SYSB
2094 XXXX
01
08/03/2008 00:10:50
SYSG
2096 XXXX
01
08/03/2008 00:10:50
SYSC
2094 XXXX
01
08/03/2008 00:10:50
SYSD
2094 XXXX
02
08/03/2008 00:10:49
SYSE
2094 XXXX
02
08/03/2008 00:10:51
57
SYSTEM STATUS
ACTIVE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
TM=STP
TM=STP
TM=STP
TM=STP
TM=STP
TM=STP
© 2010 IBM Corporation
Selected “RO *ALL,D ETR” output
Z9BE Systems
RESPONSES ------------------------------SYNCHRONIZATION MODE = STP
THIS SERVER IS A STRATUM 2
THIS IS THE ARBITER SERVER
THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ
Z9CD Systems
RESPONSES ------------------------------SYNCHRONIZATION MODE = STP
THIS SERVER IS A STRATUM 1
THIS IS THE PREFERRED TIME SERVER
THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ
Z9G Systems
RESPONSES ------------------------------SYNCHRONIZATION MODE = STP
THIS SERVER IS A STRATUM 2
THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ
Z9OC Systems
RESPONSES ------------------------------SYNCHRONIZATION MODE = STP
THIS SERVER IS A STRATUM 2
THIS IS THE BACKUP TIME SERVER
THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ
58
© 2010 IBM Corporation
STP Status – after migrating to STP-Only CTN
59
© 2010 IBM Corporation
Site 1 / Site 2 – STP only CTN
Z9CD - 2094
Z9BE - 2094
z/OS 1.8
SYSB
z/OS 1.8
SYSE
Z9G - 2096
ARB
z/OS 1.8
SYSG
STP Timing Mode
STP Timing Mode
Stratum 2, DDS_STP1
Stratum 2, DDS_STP1
z/OS 1.8
SYSC
z/OS 1.8
SYSD
CFCC
CF15
STP Timing Mode
Stratum 1,DDS_STP1
Arbiter
Site 1 Data
Center *
PTS/CTS
Z9LPC4 - 2094
CFCC
CF15
STP Timing Mode
Stratum 2,DDS_STP1
ETR
• Blue Lines
STP (Coupling Links)
• Solid Green Lines = ICB4+3
• Dash Green Lines = ISC3
* Link redundancy not shown
60
To be used as a Backup server
for outages on the PTS, BTS or
ARB servers. Manually reassign
roles…….
Z9OC - 2094
z/OS 1.8
SYSA
GDPS/CBU
SYSK
CFCC
CF15
Site 2 Data
Center *
STP Timing Mode
Stratum 2, DDS_STP1
BTS
* DWDM not shown
Site 1 and Site 2 Data Centers are
located approx X km apart.
© 2010 IBM Corporation
Setting the Time Zone Offset - Initial
61
© 2010 IBM Corporation
Set Time Zone
 Automatically adjust is selected by default when the time zone selected
supports automatic adjustment of daylight saving time. Otherwise, this button is
disabled.
 Even if automatic adjustment is supported, the user still has the option of
selecting Set standard time or Set daylight saving time.
62
© 2010 IBM Corporation
Adjust Time Zone Offset
63
© 2010 IBM Corporation
Setting the Time Zone Offset
64
© 2010 IBM Corporation
Implementation
 Migrate ETR Network to Mixed CTN
– Set CTN ID on all servers
– Migrate servers to STP timing mode
 Migrate from Mixed to STP-Only CTN
65
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
66
© 2010 IBM Corporation
Ending Environment for STP-Only CTN Example
67
© 2010 IBM Corporation
Implementation – Configuring a STP-Only CTN
 Set CTN ID
 Initialize the time
 Assign CTN Roles
68
© 2010 IBM Corporation
Set CTN ID
 The same CTN ID must be entered on every server that
will participate in the STP-only CTN.
69
© 2010 IBM Corporation
Implementation – Configuring a STP-Only CTN
 Set CTN ID
 Initialize the time
 Assign CTN Roles
70
© 2010 IBM Corporation
Initialize the time
Initialize Time button is accessed from the Network Configuration tab
Initialize Time
button enabled
ONLY IF
 CTS and PTS
roles have not
been assigned
AND
 After CTN ID
has been
defined for
server that will
become
CTS/PTS.
Important: Initializing the time must be done on the server that will become the
Current Time Server for the STP-only CTN.
71
© 2010 IBM Corporation
Set Leap Seconds
 Clicking Initialize Time on the Network Configuration tab displays
the Initialize Time window
 There are three radio buttons on the window, each representing a
task that needs to be completed before a Network Configuration
can be applied to an STP-only CTN.
72
© 2010 IBM Corporation
Set Leap Seconds
 Although the installation may not be sensitive to leap
seconds, a valid Leap Seconds Offset value needs to
be entered to complete the Initialize Time task.
 If Leap Seconds are not used, enter a zero value.
73
© 2010 IBM Corporation
Set Time Zone
 Automatically adjust is selected by default when the time zone selected
supports automatic adjustment of daylight saving time. Otherwise, this button is
disabled.
 Even if automatic adjustment is supported, the user still has the option of
selecting Set standard time or Set daylight saving time.
74
© 2010 IBM Corporation
Adjust Time Zone Offset
75
© 2010 IBM Corporation
Set date and time
 If the local date and time are to be set to specific
values, click the Set date and time button.
 If an ETS is configured, the date and time should be
initialized by selecting the option Use External Time
Source to set date and time
Regardless of the method chosen, STP uses the information to calculate the Coordinated
Server Time and set the servers’ TOD clock when the OK button is clicked.
76
© 2010 IBM Corporation
Set date and time
 At this point, the three tasks on the Initialize Time window have a
check mark in the Complete column
 The user needs to click Cancel to exit the Initialize Time task and
return to the Network Configuration tab.
 The Coordinated Server Time will be passed to other
participating servers in the CTN when the server roles and the
Current Time Server are assigned
77
© 2010 IBM Corporation
Implementation – Configuring a STP-Only CTN
 Set CTN ID
 Initialize the time
 Assign CTN Roles
78
© 2010 IBM Corporation
Assign Server Roles

Initialize Time task is
complete

Apply button is
enabled

Server roles can now
be assigned
Important: Assignment of server roles must be done on the server that will
become the Current Time Server for the STP-only CTN.
79
© 2010 IBM Corporation
Assign PTS/CTS
Because the
PTS will be the
only server
assigned in
this first step, it
has to be the
CTS
 Force configuration box must be specified when configuring a new
STP-only CTN for the first time in order to bypass connectivity
verification, as a Current Time Server does not yet exist.
80
© 2010 IBM Corporation
Assign BTS
Because the CTS already exists, the Force configuration option is not required or recommended
– allows STP to perform connectivity check between PTS and BTS.
81
© 2010 IBM Corporation
Verify on HMC – Timing Network tab
Time field of the Coordinated Server Time section identifies the current time on the server.
82
© 2010 IBM Corporation
Verify on HMC- STP Status tab
STP Timing Mode – Stratum 1
83
© 2010 IBM Corporation
Verify on HMC – STP Status tab
STP Timing Mode – Stratum 2
84
© 2010 IBM Corporation
z/OS Verification – DISPLAY ETR
Synchronization mode
Stratum level
Node ID of Stratum 1 Server
Server role (if applicable)
Number of usable links
This is the
PTS/CTS
NUMBER OF USABLE TIMING LINKS message
will not appear here because it is not applicable
for the CTS
85
© 2010 IBM Corporation
z/OS Verification – DISPLAY XCF,SYSPLEX,ALL
 DISPLAY XCF,SYSPLEX,ALL identifies the timing
mode of all sysplex members that are STP
86
© 2010 IBM Corporation
Implementation – Configuring a STP-Only CTN
 Set CTN ID
 Initialize the time
 Assign CTN Roles
87
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
88
© 2010 IBM Corporation
STP configuration and time information saved across
Power on Resets or power outages – two server CTN
 Previously
– If both servers experienced an unplanned or planned power
outage, user had to:
• Initialize the Time;
• Redefine the PTS, BTS, CTS roles
 System Management Enhancement
• STP CTN configuration (PTS/CTS, BTS) can be configured so that
– PTS/CTS, BTS roles possibly restored after IML/POR
– Initialize Time not required
89
© 2010 IBM Corporation
Prerequisites / required set up
 MCLs required:
– z9 EC and BC Bundle 44
– z10 EC and BC Bundle 26
– Function not available on D55 (z990, z890)
 If migrating from ETR network
– PTS/CTS and BTS must first be assigned using procedures outlined in
Server Time Protocol Implementation Guide, SG24-7281, section 2.5.1
 If setting up a new STP-only CTN
– PTS/CTS and BTS must first be assigned using procedures outlined in
Server Time Protocol Implementation Guide, SG24-7281 section 3.4
 STP-only CTN can only have two servers to enable this function
– If more than two servers in operational CTN,
• Remove additional servers in the CTN by assigning them to a different to a
different CTN, if possible
• Cannot proceed with this enhancement with more than two servers in CTN
90
© 2010 IBM Corporation
Enable “Save configuration” enhancement
 Launch the System (Sysplex) Time
task of the server that is the CTS
– Go to the Network Configuration
tab
– Check the box “Only allow the
server(s) specified above to be in
the CTN”
– Click Apply
– “Only allow the server(s) specified
above to be in the CTN” can be
enabled only if two servers can
communicate over coupling links
•
Cannot select “Force
Configuration” and “Only allow
the server(s) specified above to
be in the CTN” at the same time –
error message generated
Note: Recovery and re-takeover rules unchanged for a CTN with PTS and BTS assigned
91
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Power outage affecting both servers of two server CTN
– Across a Power on Reset for single server CTN
 Additional Information
92
© 2010 IBM Corporation
STP configuration and time information saved across
Power on Reset or power outage – Single server CTN
 Previously
– For an already operational single server CTN, when IML/POR done,
user was required to:
•
•
Initialize the Time
Redefine that server as the stratum 1
 System Management enhancement
– Customer selects the option to not allow any other servers to join the
single server CTN when configuring the CTN
– STP CTN configuration (PTS/CTS) will be reinstated after IML/POR
•
Initialize Time not required
Also available on z990, z890
93
© 2010 IBM Corporation
STP configuration and time information saved across
Power on Resets or power outages (continued)
Launch the System (Sysplex)
Time task of the server that is
the CTS
–Go to the Network
Configuration tab
–Check the box “Only
allow the server(s)
specified above to be in
the CTN”
–Click Apply
94
© 2010 IBM Corporation
Agenda
 General HMC Panel Overview
 Setting up a Coordinated Timing Network (CTN)
– Configuring an External Time Source
• Configure Dial Out on HMC
• Configure an NTP Server
– Migration from ETR to STP-only CTN
– Configuring an STP-only CTN
 Restoring STP configuration information
– Across a Power on Reset for single server CTN
– Power outage affecting both servers of two server CTN
 Additional Information
95
© 2010 IBM Corporation
Additional Information
 Redbooks®
– Server Time Protocol Planning Guide SG24-7280
– Server Time Protocol Implementation Guide SG24-7281
 Education
– Introduction to Server Time Protocol (STP)
• Available on Resource Link™
• www.ibm.com/servers/resourcelink/hom03010.nsf?OpenDatabase
 STP Web site
– www.ibm.com/systems/z/pso/stp.html
 Systems Assurance
– The IBM team is required to complete a Systems Assurance Review (SAPR Guide
SA06-012) and to complete the Systems Assurance Confirmation Form via
Resource Link
 Techdocs and WSC Flashes
– http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs
• Search on “STP”
96
© 2010 IBM Corporation
IBM Implementation Services for System z – Server Time Protocol (6948-J56)
•
This offering is designed to assist clients to quickly and safely implement Server
Time Protocol within their existing environments. STP provides clients with the
capability to efficiently manage time synchronization within their multi-server
infrastructure. Following best practices and using detailed planning services, IBM
helps clients identify various implementation models and engage in the appropriate
configuration required to effectively support STP for driving a more responsive
business and IT infrastructure.
•
Infrastructure Improvement; Energy Efficiency; Better performance and lower
operational cost
•
•
•
Swift and secure implementation of STP for improved availability, integrity and
performance
Improves multi-server time synchronization without interrupting operations
Enables integration with next generation of System z infrastructure
Target Audience
•
•
Primarily core, Large Enterprise customers.
Existing z midrange clients
Key Competitors
•
In house staff
Competitive
Differentiation
•
•
•
Leverages best practices with secure implementation
Short implementation time – lower risk
Provides support and facilitates knowledge sharing through IBM’s mainframe
expertise
•
Need to safely implement a reliable replacement for Sysplex Timer® while
maintaining continuous operations
Cost of providing and maintaining hardware, floor space and solution support for
additional Sysplex Timer intermediate site
Lack of in-house expertise, skills and resources for implementing Server Time
Protocol
Offering Description
Program, Play,
Industry Alignment
Client Value (enables
customers to...)
Proof Points & Claims
for Client Value /
Differentiation
Engagement Portfolio
Offering Manager
97
•
•
•
http://spimweb1.boulder.ibm.com/services/sosf/dyno.wss?oid=50423&loc=All&langc
d=en-US#1
•
Anna Lee/Southbury/IBM, 512-590-8914, T/L: 268-9318
© 2010 IBM Corporation
IBM Announces –
IBM Implementation Services for System z – Server Time
Protocol
Implementation of STP for improved availability and performance
Offering
Assist clients to quickly and safely implement
Server Time Protocol within their existing
environments. IBM helps clients identify various
implementation models and engage in the
appropriate configuration required to effectively
support STP for driving a more responsive
business and IT infrastructure
Customer Value:
- Improves multi-server time synchronization without
interrupting operations
- Enables integration with next generation of System z
infrastructure
- Swift and secure implementation of STP for improved
availability, integrity, and performance
- Reduces
hardware maintenance and power costs while
98
Leverages IBM’s
knowledge and best
practices to help
implementation of
Server Time Protocol
© 2010 IBM Corporation
Reference Material - Terminology
 APAR



















ARB
BTS
CF
CTS
CTN
DWDM
ETR
ETS
FC
HMC
HCA
ICB
IPL
ISC
LAN
LIC
LPAR
99
Authorized Program Analysis
Report
Arbiter
Backup Time Server
Coupling Facility
Current Time Server
Coordinated Timing Network
Dense Wave Division
Multiplexer
External Time Reference
External Time Source
Feature Code
Hardware Management
Console
Host Channel Adapter
Integrated Cluster Bus
Initial Program Load
InterSystem Coupling Channel
Local Area Network
Licensed Internal Code
Logically Partition
 NTP
 PR/SM
 PSIFB
Infiniband
 PTF
 PTS
 SW
and
 SE
 TPF
 UTC




zVM
zVSE
z/OS
z/VM
Network Time Protocol
Processor Resource /
Systems Manager
Parallel Sysplex
Temporary Program Fix
Preferred Time Server
Software (programs
operating systems)
Support Element
Operating System
Coordinated Universal
Time
Operating System
Operating System
Operating System
Operating System
© 2010 IBM Corporation
Questions?
100
© 2010 IBM Corporation
Thank You!
The End
101
© 2010 IBM Corporation
Trademarks
The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.
Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not
actively marketed or is not significant within its relevant market.
Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.
For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:
*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA,
WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, System z10®, BladeCenter®
The following are trademarks or registered trademarks of other companies.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.
IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.
* All other products may be trademarks or registered trademarks of their respective companies.
Notes:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will
experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.
IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual
environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without
notice. Consult your local IBM business contact for information on the product or services available in your area.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance,
compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
102
© 2010 IBM Corporation