IBM System Storage DS6000 Series: Copy Services in Open Environments Front cover

advertisement
Front cover
IBM System Storage DS6000
Series: Copy Services in
Open Environments
Configuration of Copy Services in
heterogeneous environments
Introducing TPC for Replication
Copy Services with System i
Bert Dufrasne
Gustavo Castets
Stephen Baird
Werner Bauer
Denise Brown
Jana Jamsek
ibm.com/redbooks
Wenzel Kalabza
Peter Klee
Markus Oscheka
Ying Thia
Robert Tondini
International Technical Support Organization
IBM System Storage DS6000 Series: Copy Services in
Open Environments
November 2006
SG24-6783-02
Note: Before using this information and the product it supports, read the information in “Notices” on
page xxiii.
Third Edition (November 2006)
This edition applies to features, microcode, GUI and DS CLI as announced for the DS6000 in August 2006.
© Copyright International Business Machines Corporation 2004, 2005, 2006. All rights reserved.
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
Contents
Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
Special thanks to: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii
Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxviii
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxviii
Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix
November 2006, Third Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix
Part 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Introduction to Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.1 Point-in-Time Copy (FlashCopy). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.2 Remote Mirror and Copy RMC (formerly Peer-to-Peer Remote Copy). . . . . . . . . .
3
4
4
5
Chapter 2. Copy Services architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Introduction to the Copy Services structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.1 What is a management console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 What is a Storage Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.3 What is a Storage Facility Image (SFI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.4 What is a Storage Complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 How does the new structure of Copy Services work . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.1 Remote Mirror and Copy between Storage Complexes . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Differences between the DS CLI and the DS GUI . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 System z communication path for Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Part 2. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Chapter 3. DS Storage Manager (GUI). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1 System and hardware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Supported operating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Installation modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Connecting to the DS6000 SMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Real-time and Simulated configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.2 Advantages of using the DS GUI over the DS CLI . . . . . . . . . . . . . . . . . . . . . . . .
3.3.3 Disadvantages of using the DS GUI over the DS CLI. . . . . . . . . . . . . . . . . . . . . .
3.4 Accessing the Information Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Managing the Storage Complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1 Procedure to add a Storage Complex. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
18
18
18
19
20
20
20
20
22
23
Chapter 4. DS Command-Line Interface
(DS CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1 Introduction and functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.2 Supported operating systems for the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
iii
4.3
4.4
4.5
4.6
4.7
User accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DS CLI profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Command structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Copy Services commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using the DS CLI application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7.1 Single-shot mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7.2 Script command mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7.3 Interactive mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.8 Return codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.9 User assistance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.10 Usage examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
30
31
31
33
33
34
34
35
36
37
Part 3. FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
iv
Chapter 5. FlashCopy overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1 FlashCopy operational environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.1 Full volume copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.2 Nocopy option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4 FlashCopy in combination with other Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.1 FlashCopy and Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.2 FlashCopy and Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.3 FlashCopy and Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
42
43
43
46
46
47
47
48
49
Chapter 6. FlashCopy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1 Multiple Relationship FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Consistency Group FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 FlashCopy target as a Metro Mirror or Global Copy primary. . . . . . . . . . . . . . . . . . . . .
6.4 Incremental FlashCopy to refresh target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5 Remote FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.6 Persistent FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.7 Reverse restore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.8 Fast reverse restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.9 Options and interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
52
52
53
54
56
56
56
57
57
Chapter 7. FlashCopy ordering and activation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.1 Ordering FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Activating FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2.1 Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2.2 Activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
60
61
61
63
Chapter 8. FlashCopy interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.1 FlashCopy management interfaces: Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2 DS CLI and DS SM: Commands and options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2.1 Local FlashCopy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2.2 Remote FlashCopy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.3 Local FlashCopy using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.3.1 Parameters used with local FlashCopy commands . . . . . . . . . . . . . . . . . . . . . . .
8.3.2 Local FlashCopy commands: Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.3.3 FlashCopy Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.4 Remote FlashCopy using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.4.1 Remote FlashCopy commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.4.2 Parameters used in remote FlashCopy commands . . . . . . . . . . . . . . . . . . . . . . .
65
66
66
67
67
68
68
70
79
80
80
81
IBM System Storage DS6000 Series: Copy Services in Open Environments
8.5 FlashCopy management using the DS SM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.5.1 Initiate FlashCopy using Create . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.5.2 Display properties of existing FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.5.3 Reverse existing FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.5.4 Initiate background copy for a persistent FlashCopy relationship. . . . . . . . . . . . .
8.5.5 Re-synchronize target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.5.6 Delete existing FlashCopy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82
82
85
88
89
90
91
Chapter 9. FlashCopy performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
9.1 FlashCopy performance overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
9.1.1 Distribution of the workload: Source and target volume location . . . . . . . . . . . . . 94
9.1.2 LSS/LCU versus rank considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
9.1.3 Rank geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
9.1.4 Incremental FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
9.2 FlashCopy establish phase performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
9.3 Background copy phase performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
9.4 FlashCopy impact to applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
9.5 FlashCopy options: Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
9.6 FlashCopy scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
9.6.1 Scenario #1: Backup to disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
9.6.2 Scenario #2: Backup to tape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
9.6.3 Scenario #3: FlashCopy during peak application activity . . . . . . . . . . . . . . . . . . . 99
9.6.4 Scenario #4: Ranks reserved for FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Chapter 10. FlashCopy examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.1 Create test system or integration system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.1.1 One time test system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.1.2 Multiple setups of a test system with the same content . . . . . . . . . . . . . . . . . .
10.2 Create backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.2.1 Create a FlashCopy for backup purposes without volume copy . . . . . . . . . . . .
10.2.2 Incremental FlashCopy for backup purposes . . . . . . . . . . . . . . . . . . . . . . . . . .
10.2.3 Using a target volume to restore its contents back to the source . . . . . . . . . . .
103
104
104
104
105
105
106
106
Part 4. Metro Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Chapter 11. Metro Mirror overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.1 Metro Mirror overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.2 Metro Mirror volume state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.3 Data consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.4 Rolling disaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.5 Automation and management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
111
112
113
113
114
114
Chapter 12. Metro Mirror options and configuration . . . . . . . . . . . . . . . . . . . . . . . . . .
12.1 Basic Metro Mirror operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.2 Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.3 Failover and Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.4 Consistency Group function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.4.1 Data consistency and dependent writes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.4.2 Consistency Group function: How it works . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.5 Metro Mirror paths and links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.5.1 Fibre Channel links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.5.2 Logical Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.6 Bandwidth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.7 LSS design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
115
116
117
117
118
119
119
123
124
125
126
126
Contents
v
12.8 Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.9 Symmetrical configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.10 Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12.11 Hardware requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
127
127
128
129
Chapter 13. Metro Mirror performance and scalability . . . . . . . . . . . . . . . . . . . . . . . .
13.1 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13.1.1 Initial synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13.2 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
131
132
132
133
Chapter 14. Metro Mirror interfaces and examples . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.1 Metro Mirror management interfaces overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.2 Copy Services network components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.3 DS Command-Line Interface (DS CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.4 DS Storage Manager GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.5 Set up Metro Mirror environment using DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.5.1 Preparing to work with the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.5.2 Setup of Metro Mirror configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.5.3 Determine the available Fibre Channel links. . . . . . . . . . . . . . . . . . . . . . . . . . .
14.5.4 Create Metro Mirror paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.5.5 Create Metro Mirror pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.6 Remove Metro Mirror environment using DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.6.1 Remove Metro Mirror pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.6.2 Remove paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.7 Manage the Metro Mirror environment with the DS CLI . . . . . . . . . . . . . . . . . . . . . .
14.7.1 Suspend and resume Metro Mirror data transfer . . . . . . . . . . . . . . . . . . . . . . .
14.7.2 Add and remove paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.8 Failover and Failback functions to switch sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.8.1 Metro Mirror Failover function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.8.2 Metro Mirror Failback function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9 DS Storage Manager GUI examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.1 Create paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.2 Add paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.3 Change options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.4 Delete paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.5 Create volume pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.6 Suspend volume pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.7 Resume volume pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.8 Metro Mirror Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14.9.9 Metro Mirror Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
135
136
137
138
139
139
139
140
141
142
143
144
144
146
147
147
148
149
150
154
158
158
162
166
167
169
173
174
176
178
Part 5. Global Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
vi
Chapter 15. Global Copy overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.1 Global Copy overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.2 Volume states and change logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15.3 Global Copy positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
185
186
187
188
Chapter 16. Global Copy options and configuration . . . . . . . . . . . . . . . . . . . . . . . . . .
16.1 Global Copy basic options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.1.1 Establish Global Copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.1.2 Suspend Global Copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.1.3 Resume Global Copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.1.4 Terminate Global Copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
191
192
192
192
192
193
IBM System Storage DS6000 Series: Copy Services in Open Environments
16.1.5 Convert a Global Copy pair to Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.2 Create a consistent point-in-time copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.2.1 Procedure to take a consistent point-in-time copy . . . . . . . . . . . . . . . . . . . . . .
16.2.2 Make a FlashCopy at the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.3 Hardware requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.4 DS6800 I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.5 Global Copy connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.5.1 Fibre Channel links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.5.2 Logical Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.6 Bandwidth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.7 LSS design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.8 Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16.9 DS6000 configuration at the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
193
193
194
195
196
197
198
199
199
199
200
201
202
Chapter 17. Global Copy interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.1 Global Copy management interfaces overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.2 Copy Services network components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.3 DS Command-Line Interface (DS CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.3.1 Global Copy path commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.3.2 Global Copy pair commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.4 DS Storage Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.4.1 Path panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.4.2 Metro Mirror panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
203
204
205
206
206
208
210
210
211
Chapter 18. Global Copy performance and scalability . . . . . . . . . . . . . . . . . . . . . . . .
18.1 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18.2 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18.2.1 Adding capacity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
213
214
214
214
Chapter 19. Global Copy examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1 Set up Global Copy environment using DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1.1 Preparing to work with the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1.2 Setup of Global Copy configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1.3 Determine the available Fibre Channel links. . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1.4 Create Global Copy paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.1.5 Create Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.2 Remove Global Copy environment using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . .
19.2.1 Remove Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.2.2 Remove paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.3 Maintain Global Copy environment using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . .
19.3.1 Suspend and resume Global Copy data transfer . . . . . . . . . . . . . . . . . . . . . . .
19.3.2 Change copy mode from Global Copy to Metro Mirror . . . . . . . . . . . . . . . . . . .
19.3.3 Change the copy mode from Metro Mirror to Global Copy . . . . . . . . . . . . . . . .
19.3.4 Add and remove paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.4 Periodical off-site backup procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.4.1 Initial setup for this environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.4.2 Periodical backup operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.5 DS Storage Manager examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.5.1 Establish paths with the DS Storage Manager . . . . . . . . . . . . . . . . . . . . . . . . .
19.5.2 Establish Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.5.3 Monitoring the copy status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.5.4 Convert to Metro Mirror (synchronous) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19.5.5 Suspend a pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
215
216
216
216
217
217
217
219
219
220
221
221
223
224
225
225
226
228
231
231
235
238
239
240
Contents
vii
Part 6. Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
viii
Chapter 20. Global Mirror overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.1 Synchronous and asynchronous data replication . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.1.1 Synchronous data replication and dependent writes . . . . . . . . . . . . . . . . . . . .
20.1.2 Asynchronous data replication and dependent writes. . . . . . . . . . . . . . . . . . . .
20.2 Basic concepts of Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3 Set up a Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3.1 Simple configuration to start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3.2 Establish connectivity to the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3.3 Create Global Copy relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3.4 Introduce FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3.5 Define the Global Mirror session. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.3.6 Populate the Global Mirror session with volumes . . . . . . . . . . . . . . . . . . . . . . .
20.3.7 Start the Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.4 Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.4.1 Consistency Group formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20.4.2 Consistency Group parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
243
244
244
247
251
252
253
253
254
255
256
257
258
258
259
260
Chapter 21. Global Mirror options and configuration . . . . . . . . . . . . . . . . . . . . . . . . .
21.1 Terminology used in Global Mirror environments . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.2 Create a Global Mirror environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.3 Modify a Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.3.1 Add or remove volumes to the Global Mirror session . . . . . . . . . . . . . . . . . . . .
21.3.2 Add or remove storage disk subsystems or LSSs . . . . . . . . . . . . . . . . . . . . . .
21.3.3 Modify the Global Mirror session parameters . . . . . . . . . . . . . . . . . . . . . . . . . .
21.3.4 Global Mirror environment topology changes . . . . . . . . . . . . . . . . . . . . . . . . . .
21.3.5 Remove FlashCopy relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.3.6 Remove the Global Mirror environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.4 Global Mirror with multiple storage disk subsystems . . . . . . . . . . . . . . . . . . . . . . . .
21.5 Recovery scenario after production site failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.1 Normal Global Mirror operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.2 Production site failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.3 Global Copy Failover from B to A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.4 Verify for valid Consistency Group state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.5 Set consistent data on B volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.6 Reestablish FlashCopy relationships between B and C . . . . . . . . . . . . . . . . . .
21.5.7 Restart the application at the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.8 Prepare to switch back to the local site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.9 Return to the local site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21.5.10 Conclusions of Failover/Failback example . . . . . . . . . . . . . . . . . . . . . . . . . . .
263
264
265
267
267
268
268
269
269
269
270
274
274
274
275
276
279
281
281
282
283
285
Chapter 22. Global Mirror interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22.1 Global Mirror management interfaces overview . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22.2 DS Command-Line Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22.3 DS Storage Manager GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22.4 eRCMF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
287
288
289
291
293
Chapter 23. Global Mirror performance and scalability. . . . . . . . . . . . . . . . . . . . . . . .
23.1 Performance aspects for Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23.2 Performance considerations at coordination time . . . . . . . . . . . . . . . . . . . . . . . . . . .
23.3 Consistency Group drain time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23.4 Remote storage disk subsystem configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23.5 Balancing the disk subsystem configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
295
296
298
299
299
301
IBM System Storage DS6000 Series: Copy Services in Open Environments
23.6 Growth within Global Mirror configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
Chapter 24. Global Mirror examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.1 Set up a Global Mirror environment using the DS CLI . . . . . . . . . . . . . . . . . . . . . . .
24.1.1 Preparing to work with the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.1.2 Configuration used for the environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.1.3 Setup procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.1.4 Create Global Copy relationships: A to B volumes . . . . . . . . . . . . . . . . . . . . . .
24.1.5 Create FlashCopy relationships: B to C volumes . . . . . . . . . . . . . . . . . . . . . . .
24.1.6 Start Global Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.2 Remove a Global Mirror environment with the DS CLI . . . . . . . . . . . . . . . . . . . . . . .
24.2.1 End Global Mirror processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.2.2 Remove the A volumes from the Global Mirror session . . . . . . . . . . . . . . . . . .
24.2.3 Remove the Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.2.4 Terminate FlashCopy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.2.5 Terminate Global Copy pairs and delete the paths. . . . . . . . . . . . . . . . . . . . . .
24.3 Manage the Global Mirror environment with the DS CLI. . . . . . . . . . . . . . . . . . . . . .
24.3.1 Pause and resume the Global Mirror Consistency Group formation. . . . . . . . .
24.3.2 Change the Global Mirror tuning parameters . . . . . . . . . . . . . . . . . . . . . . . . . .
24.3.3 Stop and start Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.3.4 Add and remove A volumes to the Global Mirror environment . . . . . . . . . . . . .
24.4 Recovery scenario after local site failure with the DS CLI. . . . . . . . . . . . . . . . . . . . .
24.4.1 Stop Global Mirror processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.4.2 Perform Global Copy Failover from B to A . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.4.3 Verify for valid Consistency Group state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.4.4 Reverse FlashCopy from B to C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.4.5 Reestablish the FlashCopy relationships from B to C. . . . . . . . . . . . . . . . . . . .
24.4.6 Restart the application at the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5 Return to the local site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.1 Create paths from B to A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.2 Perform Global Copy Failback from B to A . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.3 Query for the Global Copy first pass completion. . . . . . . . . . . . . . . . . . . . . . . .
24.5.4 Quiesce the application at the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.5 Query the Out Of Sync Tracks until it shows zero . . . . . . . . . . . . . . . . . . . . . .
24.5.6 Create paths from A to B if they do not exist. . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.7 Perform Global Copy Failover from A to B . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.8 Perform Global Copy Failback from A to B . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.9 Start Global Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.5.10 Start the application at the local site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6 Practice disaster recovery readiness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.1 Query the Global Mirror environment to look at the situation . . . . . . . . . . . . . .
24.6.2 Pause Global Mirror and check its completion . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.3 Pause Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.4 Perform Global Copy Failover from B to A . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.5 Create consistent data on B volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.6 Wait for the FlashCopy background copy to complete . . . . . . . . . . . . . . . . . . .
24.6.7 Reestablish the FlashCopy relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.8 Take FlashCopy from B to D. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.9 Perform the disaster recovery testing using the D volumes . . . . . . . . . . . . . . .
24.6.10 Perform Global Copy Failback from A to B . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.6.11 Wait for the Global Copy first pass to complete . . . . . . . . . . . . . . . . . . . . . . .
24.6.12 Resume Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.7 DS Storage Manager GUI examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
303
304
304
304
305
305
306
307
312
312
313
314
314
315
316
316
319
320
321
323
324
325
326
326
328
330
331
331
332
334
334
334
334
335
337
338
339
340
340
340
341
341
342
342
342
343
344
344
345
346
347
Contents
ix
24.8 Set up a Global Mirror environment with the DS GUI . . . . . . . . . . . . . . . . . . . . . . . .
24.8.1 Define paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.8.2 Create Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.8.3 Create FlashCopy relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.8.4 Create the Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.9 Manage the Global Mirror environment with the DS GUI . . . . . . . . . . . . . . . . . . . . .
24.9.1 View settings and error information of the Global MIrror session . . . . . . . . . . .
24.9.2 View information of volumes in the Global Mirror session . . . . . . . . . . . . . . . .
24.9.3 Pause a Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.9.4 Resume a Global Mirror session. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24.9.5 Modify a Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
347
347
351
355
358
360
361
362
363
364
364
Part 7. Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
Chapter 25. Data migration through double cascading. . . . . . . . . . . . . . . . . . . . . . . . 367
25.1 Data Migration with double cascading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
x
Chapter 26. Interoperability between DS6000 and DS8000 . . . . . . . . . . . . . . . . . . . . .
26.1 DS6000 and DS8000 Copy Services interoperability . . . . . . . . . . . . . . . . . . . . . . . .
26.2 Preparing the environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.1 Minimum microcode levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.2 Hardware and licensing requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.3 Network connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.4 Create matching user ids and passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.5 Updating the DS CLI profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.6 Adding the Storage Complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.7 Volume size considerations for PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.2.8 Establishment errors on newly created volumes. . . . . . . . . . . . . . . . . . . . . . . .
26.3 RMC: Establishing paths between DS6000 and DS8000 . . . . . . . . . . . . . . . . . . . . .
26.3.1 Decoding port IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.3.2 Path creation using the DS GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.3.3 Establish logical paths between DS8000 and DS6000 using the DS CLI. . . . .
26.4 Managing Metro Mirror or Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.4.1 Managing Metro Mirror or Global Copy pairs using the DS GUI . . . . . . . . . . . .
26.4.2 Managing Metro Mirror pairs using the DS CLI. . . . . . . . . . . . . . . . . . . . . . . . .
26.4.3 Managing Global Copy pairs using the DS CLI. . . . . . . . . . . . . . . . . . . . . . . . .
26.5 Managing DS6000 to DS8000 Global Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.5.1 Managing Global Mirror pairs using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . .
26.6 Managing DS6000 and DS8000 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26.6.1 Creating a remote FlashCopy on a DS6000 using the DS CLI . . . . . . . . . . . . .
371
372
372
372
372
372
373
373
374
377
379
379
379
380
380
382
382
382
383
384
384
385
385
Chapter 27. Interoperability between DS6000 and ESS 800 . . . . . . . . . . . . . . . . . . . .
27.1 DS6000 and ESS 800 Copy Services interoperability . . . . . . . . . . . . . . . . . . . . . . .
27.2 Preparing the environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.1 Minimum microcode levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.2 Hardware and licensing requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.3 Network connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.4 Create matching user IDs and passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.5 Updating the DS CLI profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.6 Adding the Copy Services Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.7 Volume size considerations for RMC (PPRC). . . . . . . . . . . . . . . . . . . . . . . . . .
27.2.8 Volume address considerations on the ESS 800 . . . . . . . . . . . . . . . . . . . . . . .
27.2.9 Establishment errors on newly created volumes. . . . . . . . . . . . . . . . . . . . . . . .
27.3 RMC: Establishing paths between DS6000 and ESS 800 . . . . . . . . . . . . . . . . . . . .
387
388
388
388
388
388
389
389
390
391
395
395
395
IBM System Storage DS6000 Series: Copy Services in Open Environments
27.3.1 Decoding port IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.3.2 Path creation using the DS GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.3.3 Establish logical paths between DS6000 and ESS 800 using the DS CLI . . . .
27.4 Managing Metro Mirror or Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.4.1 Managing Metro Mirror or Global Copy pairs using the DS GUI . . . . . . . . . . . .
27.4.2 Managing Metro Mirror pairs using the DS CLI. . . . . . . . . . . . . . . . . . . . . . . . .
27.4.3 Managing Global Copy pairs using the DS CLI. . . . . . . . . . . . . . . . . . . . . . . . .
27.5 Managing ESS 800 Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.5.1 Managing Global Mirror pairs using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . .
27.6 Managing ESS 800 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27.6.1 Creating an ESS 800 FlashCopy using DS GUI . . . . . . . . . . . . . . . . . . . . . . . .
27.6.2 Creating an ESS 800 FlashCopy using the DS CLI . . . . . . . . . . . . . . . . . . . . .
27.6.3 Creating a remote FlashCopy on an ESS 800 using the DS CLI . . . . . . . . . . .
396
396
399
402
402
406
407
407
407
408
409
410
410
Part 8. Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
Chapter 28. IBM TotalStorage Productivity Center for Replication . . . . . . . . . . . . . .
28.1 IBM TotalStorage Productivity Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.2 Where we came from . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.3 What TPC for Replication provides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.4 Copy Services terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.4.1 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.4.2 Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.4.3 Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.4.4 Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.4.5 Failover and Failback terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.5 TPC for Replication terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.5.1 TPC for Replication copy set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.5.2 TPC for Replication session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.6 TPC for Replication session types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.6.1 TPC for Replication Basic Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.6.2 TPC for Replication Advanced Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.7 TPC for Replication session states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.8 Volumes in a copy set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.8.1 Host volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.8.2 Target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.8.3 Journal volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.9 TPC for Replication and scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.10 TPC for Replication system and connectivity overview. . . . . . . . . . . . . . . . . . . . . .
28.11 TPC for Replication monitoring and freeze capability . . . . . . . . . . . . . . . . . . . . . . .
28.12 TPC for Replication heartbeat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.13 Supported platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.14 Hardware requirements for TPC for Replication servers. . . . . . . . . . . . . . . . . . . . .
28.15 TPC for Replication GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.1 Connect to the TPC for Replication GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.2 Health Overview panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.3 Sessions panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.4 Storage Subsystems panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.5 Path Management panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.6 RM Server Configuration panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.7 Advanced Tools panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.15.8 Console log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.16 Command-Line Interface to TPC for Replication . . . . . . . . . . . . . . . . . . . . . . . . . .
415
416
417
417
418
418
419
419
419
419
421
421
422
424
424
424
425
425
426
426
426
427
427
430
431
432
432
433
434
435
437
438
440
441
442
443
444
Contents
xi
xii
Chapter 29. IBM TotalStorage Rapid Data Recovery for UNIX and Windows . . . . . .
29.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29.1.1 Solution highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29.4 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
447
448
448
448
452
455
Chapter 30. IBM TotalStorage Continuous Availability for Windows . . . . . . . . . . . . .
30.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30.3.1 Service and ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
457
458
458
460
461
Appendix A. Open systems specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
AIX specifics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
AIX and FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
AIX and Remote Mirror and Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Windows and Remote Mirror and Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Copy Services with Windows volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft Volume Shadow Copy Services (VSS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft Virtual Disk Service (VDS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SUN Solaris and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy without a volume manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Remote Copy without a Volume Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Copy Services using VERITAS Volume Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hewlett Packard-Unix and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
HP-UX and FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
HP-UX with Remote Mirror and Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
VMware ESX Server and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
VMware ESX Server and FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
VMware ESX Server with Remote Mirror and Copy . . . . . . . . . . . . . . . . . . . . . . . . . . .
463
464
464
468
470
471
473
476
481
481
482
482
485
485
487
488
489
499
Appendix B. Copy Services with System i5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
System i5 functions and external storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
System i5 structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Single-level storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Input Output Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Independent Auxiliary Storage Pools (IASPs). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror for an IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Planning and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implementation and usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror for an IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Planning and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implementation and usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror for the entire disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
501
502
502
503
503
504
505
507
508
509
510
511
512
513
538
539
540
541
541
542
543
543
IBM System Storage DS6000 Series: Copy Services in Open Environments
Solution benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Planning and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implementation and usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror for the entire disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Planning and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implementation and usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy of IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Planning and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using Metro Mirror and FlashCopy of an IASP in the same scenario. . . . . . . . . . . . . .
Implementation and usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy of the entire disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Solution benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Planning and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implementation and usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
544
545
546
546
554
554
555
556
556
557
561
561
563
563
563
564
564
567
567
568
569
569
569
Appendix C. SNMP notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SNMP overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Physical connection events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Remote copy events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror-related events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
573
574
574
576
576
Appendix D. Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Authorized Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Charging example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
579
580
581
581
Appendix E. CLI migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Migrating ESS CLI to the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Review the ESS tasks to migrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convert the individual tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
583
584
584
585
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
593
593
593
594
595
595
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597
Contents
xiii
xiv
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figures
2-1
2-2
2-3
3-1
3-2
3-3
3-4
3-5
3-6
5-1
5-2
5-3
5-4
5-5
5-6
5-7
5-8
5-9
6-1
6-2
6-3
6-4
6-5
6-6
6-7
6-8
8-1
8-2
8-3
8-4
8-5
8-6
8-7
8-8
8-9
8-10
8-11
8-12
8-13
8-14
11-1
12-1
12-2
12-3
12-4
12-5
12-6
12-7
Logical view of a Storage Complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Network Interface communication structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Remote Mirror and Copy between Storage Complexes . . . . . . . . . . . . . . . . . . . . . . 12
Connecting to a DS6000 with a Web browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Help option location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
FlashCopy help Main page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
A single Storage Complex with a single Storage Unit . . . . . . . . . . . . . . . . . . . . . . . . 22
Add Storage Complex panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Successful addition of a Storage Complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
FlashCopy uses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
FlashCopy at time t0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Reads from source and target volumes and writes to source volume . . . . . . . . . . . . 44
Writes to target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Target volume after full volume FlashCopy relationship finishes. . . . . . . . . . . . . . . . 46
FlashCopy after updates to the target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
FlashCopy and Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
FlashCopy and Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
FlashCopy and Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Multiple Relationship FlashCopy possibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
FlashCopy target is Metro Mirror (or Global Copy) primary . . . . . . . . . . . . . . . . . . . . 53
Updates to the target volume caused by a refresh target FlashCopy . . . . . . . . . . . . 54
FlashCopy with Start Change Recording set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Refresh of the FlashCopy target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Remote FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Reverse restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
FlashCopy options and interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Overview of parameters used in DS CLI FlashCopy commands. . . . . . . . . . . . . . . . 69
DS CLI remote FlashCopy commands and parameters . . . . . . . . . . . . . . . . . . . . . . 81
FlashCopy Create with DS SM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Common option window for FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
FlashCopy display properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
General folder with FlashCopy properties information. . . . . . . . . . . . . . . . . . . . . . . . 86
Display out-of-synch tracks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Parameters or copy options to use with reverse action . . . . . . . . . . . . . . . . . . . . . . . 88
Initiate background copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Prompt window for background copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Resync target select option to re-synchronize FlashCopy relationship . . . . . . . . . . . 90
Prompt window to detail resync request for FlashCopy relationship . . . . . . . . . . . . . 90
Delete select option to delete FlashCopy relationship . . . . . . . . . . . . . . . . . . . . . . . . 91
Prompt window to confirm delete request for FlashCopy relationship . . . . . . . . . . . . 91
Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Metro Mirror Failover and Failback sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
Consistency Group: Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
Consistency Group: Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Consistency Group: Example 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Logical paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
DS6000 FC ports are owned by the servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Logical paths over a physical link for Metro Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . 126
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
xv
12-8
14-1
14-2
14-3
14-4
14-5
14-6
14-7
14-8
14-9
14-10
14-11
14-12
14-13
14-14
14-15
14-16
14-17
14-18
14-19
14-20
14-21
14-22
14-23
14-24
14-25
14-26
14-27
14-28
14-29
14-30
14-31
14-32
14-33
14-34
14-35
14-36
14-37
14-38
14-39
14-40
14-41
14-42
14-43
14-44
14-45
14-46
14-47
14-48
14-49
15-1
15-2
15-3
xvi
Symmetrical Metro Mirror configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Components and connectivity of the DS Storage Management Console . . . . . . . .
DS6000 configuration in the Metro Mirror setup example . . . . . . . . . . . . . . . . . . . .
Metro Mirror environment to set up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror environment for example of switching sites. . . . . . . . . . . . . . . . . . . . .
Create path: Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create paths: Select source LSS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create paths: Select target LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create paths: Select source I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create paths: Select target I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create paths: Select path options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create paths: Verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create path: See result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Select source LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Select target LSS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Select source I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Select target I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Select path options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add paths: Result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
LSS copy options: Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
LSS copy options: Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Delete paths: Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Delete paths warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Delete paths: Result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pairs: Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Volume Pairing Method . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Select source volumes . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Select target volumes (Auto pairing) . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Result after establish . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Create Metro Mirror pair: Result after sync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Suspend: Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Suspend: Suspend at source or target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Suspend: Result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Resume: Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Resume: Verify . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Resume: Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failover: Metro Mirror panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failover verify. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Result of a failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback: Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback: Verify . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback: Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback: View from the production site Storage Unit . . . . . . . . . . . . . . . . . . . . . . .
Starting failover: View on production site DS8000. . . . . . . . . . . . . . . . . . . . . . . . . .
Starting Failback on the production site DS8000. . . . . . . . . . . . . . . . . . . . . . . . . . .
Production site Storage Unit at conclusion of disaster recovery exercise . . . . . . . .
Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy and Metro Mirror state change logic . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IBM System Storage DS6000 Series: Copy Services in Open Environments
128
138
139
141
150
159
159
160
160
161
161
162
162
163
163
164
164
165
165
166
166
167
167
168
168
168
169
170
170
171
171
172
172
173
173
174
174
175
175
175
177
177
177
178
179
179
180
180
180
181
186
187
188
16-1
16-2
16-3
16-4
17-1
17-2
17-3
17-4
19-1
19-2
19-3
19-4
19-5
19-6
19-7
19-8
19-9
19-10
19-11
19-12
19-13
19-14
19-15
19-16
19-17
19-18
19-19
19-20
19-21
19-22
19-23
19-24
20-1
20-2
20-3
20-4
20-5
20-6
20-7
20-8
20-9
20-10
20-11
20-12
20-13
20-14
20-15
20-16
20-17
21-1
21-2
21-3
21-4
Create consistent data using Global Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy establishment from the production site . . . . . . . . . . . . . . . . . . . . . . . . .
DS688 I/O port numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Logical paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
LSS Copy Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DS6000 configuration in the Global Copy setup example . . . . . . . . . . . . . . . . . . . .
Global Copy environment to set up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The DS6000 environment for Global Copy off-site backup . . . . . . . . . . . . . . . . . . .
Global Copy periodical off-site backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select source LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select target LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select source I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select target I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select path options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Verification panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Volume Pairing Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select source volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select target volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy volumes in copy pending state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Properties of a Global Copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Selecting the action to convert to synchronous . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convert to synchronous confirmation panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Synchronous data replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Synchronous data replication and unnecessary recovery after local site failure . . .
Synchronous data replication, freeze, and restart without recovery required . . . . .
Asynchronous data replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy sequence of data arrival not preserved at remote site . . . . . . . . . . . .
Global Copy asynchronous data replication involving multiple disk subsystems. . .
Distributed application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror as a distributed application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Start with a simple application environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Establish Global Copy connectivity between both sites. . . . . . . . . . . . . . . . . . . . . .
Establish Global Copy volume pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introducing FlashCopy in the Global MIrror solution . . . . . . . . . . . . . . . . . . . . . . . .
Define Global Mirror session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add Global Copy source volumes to Global Mirror session. . . . . . . . . . . . . . . . . . .
Start Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Formation of a consistent set of volumes at the remote site . . . . . . . . . . . . . . . . . .
Consistency Group tuning parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror basic configuration with Master and Subordinate disk subsystems . .
Define Global Mirror session to all potentially involved storage disk subsystems . .
Decide for Master disk subsystem and start Global Mirror session . . . . . . . . . . . . .
Global Mirror paths over FCP links between source storage disk subsystems . . . .
Figures
194
195
198
199
211
211
212
212
216
217
226
228
231
232
232
233
233
234
234
235
235
236
236
237
237
238
238
239
239
239
240
240
244
245
246
248
249
249
251
252
253
253
254
255
256
257
258
259
260
266
270
271
272
xvii
21-5
21-6
21-7
21-8
21-9
21-10
21-11
21-12
21-13
21-14
21-15
21-16
21-17
21-18
22-1
22-2
22-3
23-1
23-2
23-3
23-4
23-5
24-1
24-2
24-3
24-4
24-5
24-6
24-7
24-8
24-9
24-10
24-11
24-12
24-13
24-14
24-15
24-16
24-17
24-18
24-19
24-20
24-21
24-22
24-23
24-24
24-25
24-26
24-27
24-28
24-29
24-30
24-31
xviii
Dedicated and shared links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Dedicated Global Mirror links and dedicated Global Copy links . . . . . . . . . . . . . . .
Normal Global Mirror operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Production site fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Perform Global Copy Failover from B to A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Check Consistency Group state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy Consistency Group creation process. . . . . . . . . . . . . . . . . . . . . . . . . . .
Set a consistent set of B volumes using the C volumes as source . . . . . . . . . . . . .
Restart applications at the remote site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback operation from B to A in preparation for returning to local site . . . . . . . . .
Global Copy Failover from A to B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy Failback from A to B and resync Global Copy volumes . . . . . . . . . . .
Establish Global Mirror FlashCopy relationship between B and C . . . . . . . . . . . . .
Start Global Mirror session and resume application I/O at local site . . . . . . . . . . . .
Global Mirror main panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror pull-down list - No session selected . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror pull-down list - Session selected . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy with write hit at the remote site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy with overcommitted NVS at the remote site . . . . . . . . . . . . . . . . . . . .
Coordination time: How it impacts application write I/Os . . . . . . . . . . . . . . . . . . . . .
Remote disk subsystem with all ranks containing equal numbers of volumes . . . .
Remote disk subsystem with D volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DS6000 configuration in the Global Mirror example . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror example before unplanned production site failure . . . . . . . . . . . . . . .
Site swap scenario after failoverpprc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Site swap scenario after the reverseflash command was issued . . . . . . . . . . . . . .
Site swap scenario after the completion of the background copy . . . . . . . . . . . . . .
Site swap scenario after mkflash B to C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Site swap scenario after the application start . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Site swap scenario after Global Copy Failback from B to A . . . . . . . . . . . . . . . . . .
Site swap scenario after Global Copy Failover from A to B . . . . . . . . . . . . . . . . . . .
Site swap scenario after failbackpprc A to B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Start Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Start the application at the local site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
After mkflash B to D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Perform Global Copy Failback from A to B - test scenario . . . . . . . . . . . . . . . . . . .
Global Copy paths creation: Launch the creation process . . . . . . . . . . . . . . . . . . .
Global Copy paths creation - Step 1: Select the source LSS . . . . . . . . . . . . . . . . .
Global Copy paths creation - Step 2: Select the target LSS . . . . . . . . . . . . . . . . . .
Global Copy paths creation - Step 3: Select the source I/O ports . . . . . . . . . . . . . .
Global Copy paths creation - Step 4: Select the target I/O ports . . . . . . . . . . . . . . .
Global Copy paths creation - Step 5: Select the paths options . . . . . . . . . . . . . . . .
Global Copy paths creation - Step 6: Verification . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy creation: Launch the creation process . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy creation - Step 1: Choose volume pairing method . . . . . . . . . . . . . . .
Global Copy creation - Step 2: Select the source volumes . . . . . . . . . . . . . . . . . . .
Global Copy creation - Step 3: Select the target volume 1 . . . . . . . . . . . . . . . . . . .
Global Copy creation - Step 3: Select the target volume 2 . . . . . . . . . . . . . . . . . . .
Global Copy creation - Step 4: Select the copy options. . . . . . . . . . . . . . . . . . . . . .
Global Copy creation - Step 5: Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy creation: Launch the creation process. . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy creation - Step 1: Select the relationship type . . . . . . . . . . . . . . . . . . . .
FlashCopy creation - Step 2: Select the source volumes . . . . . . . . . . . . . . . . . . . .
IBM System Storage DS6000 Series: Copy Services in Open Environments
273
273
274
275
275
276
277
280
281
282
283
283
284
284
291
292
292
297
297
298
300
300
304
324
325
327
328
329
330
332
335
337
338
339
343
344
348
348
349
349
350
350
351
351
352
352
353
353
354
354
355
355
356
24-32
24-33
24-34
24-35
24-36
24-37
24-38
24-39
24-40
24-41
24-42
24-43
24-44
24-45
24-46
25-1
25-2
26-1
26-2
26-3
27-1
27-2
27-3
27-4
27-5
27-6
27-7
27-8
27-9
27-10
27-11
27-12
27-13
27-14
27-15
27-16
27-17
27-18
27-19
27-20
28-1
28-2
28-3
28-4
28-5
28-6
28-7
28-8
28-9
28-10
28-11
28-12
28-13
FlashCopy creation - Step 3: Select the target volumes . . . . . . . . . . . . . . . . . . . . .
FlashCopy creation - Step 4: Select the common option. . . . . . . . . . . . . . . . . . . . .
FlashCopy creation - Step 5: Verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror creation: Launch the creation process . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror creation. Step 1: Select volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror creation. Step 2: Define properties. . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror creation. Step 3: Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Visualize the Global Mirror session status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
View Global Mirror session’s properties: Launch the viewing process . . . . . . . . . .
View Global Mirror session settings: General tab . . . . . . . . . . . . . . . . . . . . . . . . . .
View Global Mirror session errors information: Failures tab . . . . . . . . . . . . . . . . . .
View Global Mirror session volumes information: Launch the viewing process. . . .
Global Mirror session volumes information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Pause Global Mirror: Confirm the pause of the Global Mirror session. . . . . . . . . . .
Pause Global Mirror: View session status paused. . . . . . . . . . . . . . . . . . . . . . . . . .
Double cascading example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Copy changed to Metro Mirror and direction reversed . . . . . . . . . . . . . . . . .
Add DS6000 complex to DS8000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add DS8000 complex to DS6000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DS6000 GUI showing multiple complexes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Add 2105 Copy Services Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Viewing ESS 800 volume size using ESS Specialist GUI . . . . . . . . . . . . . . . . . . . .
Displaying block count using Copy Services server . . . . . . . . . . . . . . . . . . . . . . . .
ESS 800 I/O ports decoded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Path panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select source LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select target LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select source I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select target I/O ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Path panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using ESS 800 Specialist GUI to display the ESS 800 WWNN . . . . . . . . . . . . . . .
Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Volume Pairing Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select source volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
First Metro Mirror target volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Second Metro Mirror target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Managing Metro Mirror pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Creating ESS 800 FlashCopy using DS GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Selecting ESS 800 source volumes for FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . .
TPC software products suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Management of Copy Services Functions supported by TPC for Replication . . . . .
Failover and Failback terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TPC for Replication: Metro Mirror copy sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TPC for Replication: Global Mirror copy sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TPC for Replication: Session concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TPC for Replication: Four sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TPC for Replication system overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Replication Manager server connectivity to DS6000 and DS8000 . . . . . . . . . . . . .
TPC for Replication server freeze . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
LSS heartbeat triggers freeze when connectivity to server fails . . . . . . . . . . . . . . .
Windows 2003 Software level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
GUI URL determined during Replication Manager installation . . . . . . . . . . . . . . . .
Figures
356
357
357
358
358
359
360
360
361
361
362
362
363
363
363
368
369
375
376
376
391
392
394
396
397
397
397
398
398
399
400
402
403
403
404
405
405
406
409
410
416
418
420
421
422
423
427
428
429
430
431
432
433
xix
28-14
28-15
28-16
28-17
28-18
28-19
28-20
28-21
28-22
28-23
28-24
28-25
28-26
28-27
29-1
29-2
29-3
29-4
29-5
29-6
29-7
30-1
30-2
A-1
A-2
A-3
A-4
A-5
A-6
A-7
A-8
A-9
A-10
A-11
A-12
A-13
A-14
A-15
A-16
A-17
A-18
A-19
A-20
A-21
A-22
A-23
B-1
B-2
B-3
B-4
B-5
B-6
B-7
xx
TPC for Replication GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Launch the Replication Manager GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Health Overview panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sessions overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
About to create copy sets to session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Actions against a session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
GUI basic layout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select storage subsystem and select View/Modify Details action . . . . . . . . . . . . . .
Storage subsystem details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Path overview panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Path overview of a DS8000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Replication Management Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Advanced tools option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Console log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sample update sequence in a database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sample Rolling Disaster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Dependent write protection of database integrity. . . . . . . . . . . . . . . . . . . . . . . . . . .
eRCMF protection tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
eRCMF architecture: 2-site configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
eRCMF architecture: 3-site configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
eRCMF issues freeze command in a Disaster Recovery scenario . . . . . . . . . . . . .
TotalStorage Continuous Availability for Windows protection tier . . . . . . . . . . . . . .
Architecture of the solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
lspv output before recreating the Volume Group . . . . . . . . . . . . . . . . . . . . . . . . . . .
lspv output after recreating the Volume Group . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Target file system stanza . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Remote Mirror and Copy phantom disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Original time stamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Update source time stamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Update secondary server ODM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft VSS architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft VSS with DS6000 FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft VDS architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
QLogic SANsurfer VDS Manager startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft VDS and VSS architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Microsoft VDS and VSS with DS6000 storage subsystem . . . . . . . . . . . . . . . . . . .
Virtual machine source disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Virtual disk types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy within the same virtual machine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy within the same virtual machines: After FlashCopy . . . . . . . . . . . . . . . .
FlashCopy between virtual machines: Source virtual machine . . . . . . . . . . . . . . . .
FlashCopy between virtual machines: Target virtual machine. . . . . . . . . . . . . . . . .
FlashCopy between virtual machines: Target virtual machine after FlashCopy . . .
FlashCopy between ESX Servers: Source virtual machine . . . . . . . . . . . . . . . . . . .
FlashCopy between ESX Servers: Target virtual machine . . . . . . . . . . . . . . . . . . .
FlashCopy between ESX Servers: Target virtual machine after FlashCopy . . . . . .
System i5 partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Single-level storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Input Output Processors (IOPs). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Elements of System i5 Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IASPs in System i5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro mirror of IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Functioning of the System i Copy Services Toolkit . . . . . . . . . . . . . . . . . . . . . . . . .
IBM System Storage DS6000 Series: Copy Services in Open Environments
434
435
436
437
438
438
439
439
440
440
441
442
443
444
449
450
451
452
453
453
454
460
461
467
467
468
469
469
470
470
474
476
477
479
480
481
490
491
492
493
494
495
496
497
498
499
503
504
505
506
508
510
514
B-8
B-9
B-10
B-11
B-12
B-13
B-14
B-15
B-16
B-17
B-18
B-19
B-20
B-21
B-22
B-23
B-24
B-25
B-26
B-27
B-28
B-29
B-30
B-31
B-32
B-33
B-34
B-35
B-36
B-37
B-38
B-39
B-40
B-41
B-42
B-43
B-44
B-45
B-46
B-47
B-48
B-49
B-50
B-51
B-52
B-53
B-54
B-55
B-56
B-57
B-58
B-59
B-60
Checking status of IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
wrkcfgsts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Status of IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
dspessdta. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IASP name. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Display Toolkit options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
chkpprc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Checking PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
chkpprc completed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
viewlog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Toolkit log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Viewprof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DS CLI profiles on i5/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Display DS CLI profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Select DS CLI script on i5/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DS CLI script on i5/OS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
swpprc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parameter scheduled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Messages about progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Log of swpprc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror copy of IASP after switchover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Original IASP after switchover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
dspessdta after the switchover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
swpprcback to production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Toolkit log after successful switchback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Status of IASP on production after the switchback . . . . . . . . . . . . . . . . . . . . . . . . .
Toolkit status after the switchback to production . . . . . . . . . . . . . . . . . . . . . . . . . . .
swpprc *unscheduled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Message Waiting for reply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Display operator message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Confirming unscheduled switch by replying to the message . . . . . . . . . . . . . . . . . .
dspessdta after unscheduled switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SWPPRC *COMPLETE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Waiting for a reply to the operator message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Confirm Replicate from remote site to local site . . . . . . . . . . . . . . . . . . . . . . . . . . .
Status after switch is completed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror of IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
falovrgmir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metro Mirror of the entire disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Tagged load source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
External load source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Properties of load source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IOP for Boot from SAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
LSPPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Script for suspending Metro mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Performing the script for suspending Metro Mirror. . . . . . . . . . . . . . . . . . . . . . . . . .
Script for Failover of Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Performing script for Failover of Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Status of primary volumes after suspend and Failover . . . . . . . . . . . . . . . . . . . . . .
Status of secondary volumes after suspend and Failover . . . . . . . . . . . . . . . . . . . .
IPL partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Script for Failback of Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Performing script for Failback of Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figures
515
516
516
517
517
518
519
520
521
522
523
523
524
524
525
525
526
527
528
528
529
529
530
530
531
532
532
533
534
534
535
536
536
537
537
538
540
543
544
547
548
548
549
550
550
550
551
551
551
551
552
552
553
xxi
B-61
B-62
B-63
B-64
B-65
B-66
B-67
B-68
B-69
B-70
B-71
B-72
B-73
B-74
B-75
B-76
B-77
B-78
B-79
B-80
B-81
B-82
B-83
B-84
E-1
E-2
E-3
xxii
Pause Metro Mirror or secondary DS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failover to primary DS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback on primary DS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Global Mirror of entire disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Pause GM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Pause Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fail over Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback Global Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failover to local site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failback Global Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Resume Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fail over Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Lsflash for GM unplanned outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Reverse FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy of an IASP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
PRPIASPBCK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Executing the command rsmiaspbck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FlashCopy of entire System i5 disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Script for making FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Make FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Script for list FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
lsflash. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Script for rmflash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Remove FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ESS Copy Services GUI tasks panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ESS task information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ESS GUI FlashCopy window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IBM System Storage DS6000 Series: Copy Services in Open Environments
553
553
554
555
558
558
558
559
559
559
559
560
561
561
562
565
566
568
570
571
571
571
572
572
585
585
586
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that does
not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions
are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS
PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and
distribute these sample programs in any form without payment to IBM for the purposes of developing, using,
marketing, or distributing application programs conforming to IBM's application programming interfaces.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
xxiii
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
Redbooks (logo)
eServer™
ibm.com®
iSeries™
i5/OS®
z/OS®
zSeries®
AIX 5L™
AIX®
AS/400®
DB2®
DS4000™
DS6000™
™
DS8000™
Enterprise Storage Server®
ESCON®
FlashCopy®
FICON®
Geographically Dispersed Parallel
Sysplex™
GDPS®
IBM®
OS/400®
Parallel Sysplex®
PowerPC®
POWER4™
POWER5™
Redbooks™
S/390®
System i™
System i5™
System p™
System z™
System Storage™
System Storage DS™
Tivoli®
TotalStorage®
WebSphere®
The following terms are trademarks of other companies:
SAP, and SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other
countries.
Java, JDK, Solaris, StorageTek, Sun, and all Java-based trademarks are trademarks of Sun Microsystems,
Inc. in the United States, other countries, or both.
Internet Explorer, Microsoft, Windows NT, Windows Server, Windows, and the Windows logo are trademarks
of Microsoft Corporation in the United States, other countries, or both.
Intel, Pentium, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of
Intel Corporation or its subsidiaries in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Other company, product, and service names may be trademarks or service marks of others.
xxiv
IBM System Storage DS6000 Series: Copy Services in Open Environments
Preface
This IBM® Redbook helps you install, tailor, and configure Copy Services in open systems
environments on the IBM System Storage™ DS6000™ storage server series. It should be
read in conjunction with the IBM System Storage DS6000 Series: Architecture and
Implementation, SG24-6781.
This redbook gives you the details necessary to configure each of the Copy Services
functions in an open systems environment. There is a companion IBM Redbook that supports
the configuration of the Copy Services functions in a z/OS® environment, IBM System
Storage DS6000 Series: Copy Services with System z servers, SG24-6782.
This IBM Redbook helps you design and create a new Copy Services installation or to
migrate from an existing installation, as well as addresses design functions and changes in
terminology from other IBM Copy Services products. We include hints and tips to maximize
the effective installation.
The team that wrote this redbook
This redbook was produced by a team of specialists from around the world working for the
International Technical Support Organization, San Jose Center in IBM Mainz, Germany.
Bert Dufrasne is a Certified Consulting IT Specialist and Project Leader for IBM System
Storage products at the International Technical Support Organization, San Jose Center. He
has worked at IBM in various IT areas. Before joining the ITSO, he worked for IBM Global
Services as an Application Architect. He holds a degree in Electrical Engineering.
Gustavo Castets is a Certified IBM IT Specialist and Project Leader working for the IBM
International Technical Support Organization, San Jose Center. While in San Jose, from
2001 to 2004, Gustavo co-authored twelve redbooks and taught IBM classes worldwide in the
area of disk storage systems. Before joining the ITSO, Gustavo was based in Buenos Aires
where he worked in different technical sales support positions for more than 22 years. Today,
in addition to his work with the ITSO, Gustavo is working for IBM Global Delivery in Argentina,
as a Storage Specialist giving support to accounts from the US and Europe.
Stephen Baird is an IT Specialist with IBM Global Services. He joined IBM in 1999, working
in open systems server performance management and capacity planning. Since 2002, he has
worked in Storage Area Network and disk storage subsystem support and has gained
experience with Brocade, Cisco, and McData Fiber Channel switches and directors as well as
DS8000™, DS4000™, and ESS series hardware. He holds a degree in Mechanical
Engineering from the Massachusetts Institute of Technology, in Cambridge, MA.
Werner Bauer is a certified consulting IT specialist in Germany. He has 26 years of
experience in storage software and hardware, as well with S/390® and z/OS. His areas of
expertise include disaster recovery solutions in enterprises utilizing the unique capabilities
and features of the IBM disk storage servers, ESS and DS6000/DS8000. He has written
extensively in various redbooks, including Transactional VSAM, DS6000/DS8000 Concepts
and Architecture, and DS6000/DS8000 Copy Services. He holds a degree in Economics from
the University of Heidelberg and in Mechanical Engineering from FH Heilbronn.
Denise Brown is a Software Engineer at the IBM Open Systems Level Test Lab in Tucson,
Arizona. She has been working with IBM Storage for the past four years, with experience in
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
xxv
storage software and hardware in an open systems environment. Her current areas of focus
include Copy Services solutions in Metro/Global Mirror and Incremental Re-synchronization
for the DS8000. She holds a degree in Engineering Mathematics.
Jana Jamsek is an IT specialist in IBM Slovenia. She works in Storage Advanced Technical
Support for Europe as a specialist for IBM Storage Systems and i5/OS® systems. Jana has
eight years of experience in the System i™ and AS/400® area and six years experience in
Storage. She holds a masters degree in computer science and a degree in mathematics from
University of Ljubljana, Slovenia. She was among the authors of the IBM Redpaper, The LTO
Ultrium Primer for IBM eServer iSeries Customers, the IBM Redbook, iSeries in Storage Area
Networks, the IBM Redbook, iSeries and IBM System Storage: A Guide to Implementing
External Disk on IBM eServer i5, and the IBM Redpaper, Multipath for IBM eServer iSeries.
Wenzel Kalabza is an IT specialist in IBM Germany. He started in 1998 as a field quality
engineer for IBM hard disk drives and was technical lead in HDD robustness and rotational
vibration testing. He has three years of experience supporting IBM storage products. As a
member of the DASD EMEA back office, he initially supported the ESS, and is now a product
field engineer for the DS6000, specializing in Copy Services.
Peter Klee is working as an IT Specialist with ATS EMEA, located in Mainz, Germany. He
has 10 years of experience in data centers. He worked for a large bank in Germany and was
responsible for the architecture and the implementation of the disk storage environment using
EMC Symmetrix, HDS Lightning, and ESS Model 800. He joined IBM in 2003, where he was
working for Strategic Outsourcing. Since June 2004, he has worked in the ATS System
Storage team in Mainz. His main focus is Copy Services in the open systems environment
with DS8000 and DS6000, especially Global Mirror and Metro/Global Mirror.
Markus Oscheka is an IT Specialist for Proof of Concepts and Benchmarks in the ATS
Customer Solutions team in Mainz, Germany. His areas of expertise include setup and
demonstration of IBM System Storage products and solutions in various environments
including AIX®, Linux®, Windows®, HP-UX, and Solaris™. He has worked at IBM for five
years. He has performed many Proofs of Concept with Copy Services on DS6000/DS8000,
as well as Performance-Benchmarks with DS4000/DS6000/DS8000.
Ying Thia is an Advisory IT Specialist based in IBM Singapore, providing storage technical
support. She has 14 years of experience in the S/390 and storage environment. Before
joining the IBM Singapore Storage team, she worked in IBM Global Services where her
responsibilities include technical support and services delivery. Her areas of expertise include
IBM high-end disk and tape storage subsystems and disaster recovery solutions using the
capabilities and features of IBM storage products. She co-authored a previous redbook and
workshop for zSeries® Copy Services.
Robert Tondini is a Senior IT specialist based in IBM Australia, providing storage technical
support. He has 12 years of experience in the S/390 and storage environment. His areas of
expertise include IBM high-end disk and tape storage subsystems and disaster recovery
solutions using the capabilities and features of IBM storage products. He co-authored several
redbooks and workshop for zSeries Copy Services.
xxvi
IBM System Storage DS6000 Series: Copy Services in Open Environments
The team: Gustavo, Robert, Wenzel, Jana, Peter, Markus, Denise, Werner, Ying, Stephen, Bertrand
Special thanks to:
John Bynum and Robert Moon - Technical Support Marketing Leads
IBM US
We want to thank Michael Eggloff and Peter Klee for hosting us at the European Storage
Competency Center in Mainz, Germany. They were able to supply us with the needed
hardware, conference room, and all of the many things needed to run a successful residency.
Gunter Schmitt, Uwe Schweikhard, and Edgar Strubel (ATS - IBM Mainz) for their help in
reserving and preparing the equipment we used.
Monika Baier, Susanne Balzer, Marion Barlen, Marion Hartmann, Andrea Witkowski, and
Gertrud Kramer - IBM Mainz, for their help with administrative tasks.
Many thanks to the authors of the previous edition of this IBM Redbook:
Peter Kimmel, Jukka Myyrlainen, Lu Nguyen, Gero Schmidt, Shin Takata, Anthony
Vandewerdt, and Bjoern Wesselbaum
We also would like to thank:
Selwyn Dickey, Timothy Klubertanz, Vess Natchev, James McCord, and Chuck Stupca
IBM Rochester - System i Client Technology Center
Guido Ihlein, Marcus Dupuis, and Wilfried Kleemann
IBM Germany
Bob Bartfai, Jennifer Mason, James Davison, Steve Van Gundy, Richard Ripberger, Bill
Raywinkle, Christopher O’Toole, Jim Sedgwick, Henry Sautter, Craig Gordon, Rosemary
McCutchen, Lee La Frese, Alan McClure, Rachel Mikolajewski, Gail Spear, Leann Vaterlaus,
David V Valverde, Sonny Williams, and Bob Moon.
Brian Sherman
IBM Canada
Nick Clayton
IBM UK
Preface
xxvii
Many thanks to Emma Jacobs and Leslie Parham.
Become a published author
Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with
specific products or solutions, while getting hands-on experience with leading-edge
technologies. You'll team with IBM technical professionals, Business Partners, and/or
customers.
Your efforts will help increase product acceptance and customer satisfaction. As a bonus,
you'll develop a network of contacts in IBM development labs, and increase your productivity
and marketability.
Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our Redbooks™ to be as helpful as possible. Send us your comments about this or
other Redbooks in one of the following ways:
򐂰 Use the online Contact us review redbook form found at:
ibm.com/redbooks
򐂰 Send your comments in an e-mail to:
[email protected]
򐂰 Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400
xxviii
IBM System Storage DS6000 Series: Copy Services in Open Environments
Summary of changes
This section describes the technical changes made in this edition of the book and in previous
editions. This edition might also include minor corrections and editorial changes that are not
identified.
Summary of Changes
for SG24-6783-02
for IBM System Storage DS6000 Series: Copy Services in Open Environments
as created or updated on November 28, 2006.
November 2006, Third Edition
This revision reflects the addition, deletion, or modification of new and changed information
described below.
New information
򐂰
򐂰
򐂰
򐂰
Data migration through double cascading
IBM TotalStorage® Productivity Center for Replication
Host operating system considerations for System i
New DS6000 model 1750-522
Changed information
򐂰 Updated Copy Services examples
򐂰 Updated licensing information
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
xxix
xxx
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 1
Part
1
Overview
In this part, we describe the various Advanced Copy Services offerings for the DS6000
series, and how they relate to previous Copy Services offerings available on the Enterprise
Storage Server® (ESS). This part also shows you how existing Copy Services functions from
the ESS can or cannot coexist with the DS6000 series Copy Services. Similarly, we discuss
their use with the DS8000 series Copy Services.
With the announcement of the DS6000 series and of Advanced Copy Services for open
systems on the DS6000 series, we introduced some new terminology. We also introduced
these terms for Advanced Copy Services Version 2 for the ESS. These are detailed in the
following IBM Redbook, IBM System Storage DS6000 Series: Architecture and
Implementation, SG24-6781, and are summarized in Table 1. This table includes the zSeries
unique Copy Services for completeness.
Table 1 Reference chart for DS Copy Services on DS6000
DS6000 function
ESS 800 Version 2 function
Formerly known as
FlashCopy®
FlashCopy
FlashCopy
Global Mirror
Global Mirror
Asynchronous PPRC
Metro Mirror
Metro Mirror
Synchronous PPRC
Global Copy
Global Copy
PPRC Extended Distance
z/OS Global Mirror, as an XRC
target only
z/OS Global Mirror
Extended Remote Copy (XRC)
z/OS Metro/Global Mirror for
zSeries, three site solution as
an XRC target only
z/OS Metro/Global Mirror
Three site solution using Sync
PPRC and XRC
Metro/Global Copy
Metro/Global Copy
Two or three site Async
cascading PPRC
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
1
The Copy Services configuration is created using either the IBM System Storage DS™
Storage Command-Line Interface, DS CLI, or the IBM System Storage DS Storage Manager
Copy Services Graphical User Interface, DS GUI.
Also note:
򐂰 All DS6000 installations require at least an Operating Equipment License (OEL) key to
operate. However, Copy Services functions can only be used where the appropriate
License Key is installed.
򐂰 The key Copy Services license types are: OEL (Operating Equipment License), RMC
(Remote Mirror and Copy or PPRC), and PTC (Point-in-Time Copy or FlashCopy).
򐂰 The DS CLI replaces both the ESS CLI and the ESS Copy Services CLI.
򐂰 The DS CLI can also be used for ESS 800 Copy Services, but not ESS configuration. For
ESS configuration, you need to continue to use the ESS Web Specialist or ESS CLI.
򐂰 The DS CLI can invoke Remote Mirror relationships with ESS 800, if the ESS 800 is at
code level LIC 2.4.3.65 or above.
򐂰 The DS CLI can be used in reusable scripts and provides a consistent interface for current
and planned IBM System Storage products.
򐂰 The DS CLI now invokes a Copy Services function directly rather than invoking a saved
task.
򐂰 The DS GUI can only be used for one-time execution of a Copy Services operation; it
cannot save tasks.
2
IBM System Storage DS6000 Series: Copy Services in Open Environments
1
Chapter 1.
Introduction
This chapter provides a brief summary of the various Copy Services functions available on
the DS6000 series. These services are very similar to the existing Copy Services for the IBM
Enterprise Storage Server (ESS), and some models of the ESS are interoperable with the
DS8000 as well as the DS6000.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
3
1.1 Introduction to Copy Services
We briefly describe each function of Copy Services in this section. Copy Services are a
collection of functions that provide for Disaster Recovery, data migration, and data duplication
functions. There are two primary types of Copy Services functions: Point-in-Time Copy and
Remote Mirror and Copy. Generally, the Point-in-Time Copy function is used for data
duplication and the Remote Mirror and Copy functions are used for data migration and
Disaster Recovery.
With the Copy Services functions, for example, you can create backup data with little or no
disruption to your application, and you can back up your application data to the remote site for
disaster recovery.
Copy Services run on the DS6000 Storage Unit and support open systems and zSeries
environments. These functions are supported also on the previous generation of storage
systems, the IBM TotalStorage Enterprise Storage Server (ESS).
Many design characteristics of the DS6000 and the data copying and mirroring capabilities of
the Copy Services features contribute to the protection of your data, 24 hours a day and 7
days a week (24x7). The licensed features included in Copy Services are as follows:
򐂰 FlashCopy, which is a Point-in-Time Copy function
򐂰 Remote Mirror and Copy functions, previously known as Peer-to-Peer Remote Copy or
PPRC, which include:
– Metro Mirror, previously known as Synchronous PPRC
– Global Copy, previously known as PPRC Extended Distance
– Global Mirror, previously known as Asynchronous PPRC
We explain these functions in more detail in the next section.
You can manage the Copy Services functions through a Command-Line Interface (DS CLI)
and a new Web-based interface (DS Storage Manager). You can also manage the Copy
Services functions through the open application programming interface (DS Open API).
When you manage the Copy Services through these interfaces, these interfaces invoke Copy
Services functions using the Ethernet network.
We explain these interfaces in Part 2, “Interfaces” on page 15.
1.1.1 Point-in-Time Copy (FlashCopy)
The Point-in-Time Copy feature, which includes FlashCopy, enables you to create full volume
copies of data in a Storage Unit. To use it, you must purchase the Point-in-Time Copy
License feature #5200 PTC, for the required DS6800 storage server machine type 1750.
When you set up a FlashCopy operation, a relationship is established between the source
and target volumes, and a bitmap of the source volume is created. Once this relationship and
bitmap are created, the target volume can be accessed as though all the data had been
physically copied. While a relationship between the source and target volume exists,
optionally, a background process copies the tracks from the source to the target volume.
When a FlashCopy operation is invoked, it takes only a few seconds to complete the process
of establishing the FlashCopy pair and creating the necessary control bitmaps. Thereafter,
you have access to a Point-in-Time Copy of the source volume. As soon as the pair has been
established, you can read and write to both the source and target volumes.
4
IBM System Storage DS6000 Series: Copy Services in Open Environments
After creating the bitmap, a background process begins to copy the real data from the source
to the target volumes. If you access the source or the target volumes during the background
copy, FlashCopy manages these I/O requests, and facilitates both reading from and writing to
both the source and target copies. When all the data has been copied to the target, the
FlashCopy relationship is ended.
1.1.2 Remote Mirror and Copy RMC (formerly Peer-to-Peer Remote Copy)
The new Remote Mirror and Copy license authorization enables several Copy Services
functions depending on the configuration and settings you use.
The Remote Mirror and Copy feature or RMC, (formerly called Peer-to-Peer Remote Copy or
PPRC), is a flexible data mirroring technology that allows replication between volumes on two
or more disk storage systems. You can also use this feature for data backup and Disaster
Recovery.
Remote Mirror and Copy is an optional function. To use it, you must purchase the Remote
Mirror and Copy function authorization code #5300 RMC for the required DS6800 storage
server machine type 1750.
DS6000 Storage Units can participate in Remote Mirror and Copy solutions with another
DS6000, or with the ESS Model 750, ESS Model 800, and DS8000 Storage Units. To
establish an RMC (formerly PPRC) relationship between the DS6000 and the ESS, the ESS
needs to have licensed internal code (LIC) Version 2.4.3.65 or later.
The Remote Mirror and Copy feature can operate in the following modes.
Metro Mirror (formerly Synchronous PPRC)
Metro Mirror provides real-time mirroring of logical volumes between two DS6000s that you
can place up to 300 km from each other. It is a synchronous copy solution where write
operations are completed on both copies (local and remote site) before they are considered
to be complete.
Global Copy (formerly PPRC-XD)
Global Copy copies data non-synchronously and over longer distances than is possible with
Metro Mirror. When operating in Global Copy mode, the source volume sends a periodic,
incremental copy of updated tracks to the target volume, instead of sending a constant
stream of updates. This causes less impact to application writes for source volumes and less
demand for bandwidth resources, while allowing a more flexible use of available bandwidth.
Global Copy does not keep the sequence of write operations. Therefore, the copy is normally
fuzzy, but you can make a consistent copy through synchronization (called a go-to-sync
operation). After the synchronization, you can issue FlashCopy at the secondary site to make
the backup copy with data consistency. After the establishment of the FlashCopy, you can
change the PPRC mode back to non-synchronous mode.
Note: When you change the PPRC mode from synchronous to non-synchronous mode,
you change the PPRC mode from synchronous to suspend mode first, and then you
change PPRC mode from suspend to non-synchronous mode.
If you want to make a consistent copy with FlashCopy, you must purchase a Point-in-Time
Copy function authorization license for the secondary Storage Unit.
Chapter 1. Introduction
5
Global Mirror (Asynchronous PPRC)
Global Mirror provides a long-distance remote copy feature across two sites using
asynchronous technology. This solution is based on the existing Global Copy and FlashCopy.
With Global Mirror, the data that the host writes to the Storage Unit at the local site is
asynchronously shadowed to the Storage Unit at the remote site. A consistent copy of the
data is automatically maintained on the Storage Unit at the remote site.
Global Mirror operations provide the benefit of supporting operations over virtually unlimited
distances between the local and remote sites, restricted only by the capabilities of the
network and the channel extension technology. It can also provide a consistent and
restartable copy of the data at the remote site, created with minimal impact to applications at
the local site.
The ability to maintain an efficient synchronization of the local and remote sites with support
for Failover and Failback modes helps to reduce the time that is required to switch back to the
local site after a planned or unplanned outage.
6
IBM System Storage DS6000 Series: Copy Services in Open Environments
2
Chapter 2.
Copy Services architecture
This chapter is an overview of the structure of the Copy Services communication architecture
in either an open or zSeries environment. The chapter covers the following topics:
򐂰 Introduction to the Copy Services structure.
򐂰 How does the new structure of Copy Services work.
򐂰 System z communication path for Copy Services.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
7
2.1 Introduction to the Copy Services structure
The Copy Services architecture for the IBM ESS 800 used a construct called a Copy Services
Domain to manage Copy Services. The communications architecture for DS6000 and
DS8000 is noticeably different. Instead of a Copy Services Domain, we now use the concept
of Storage Complexes.
You can perform Copy Services operations both within or between Storage Complexes. An
ESS 800 Copy Services Domain (at the correct firmware level) can, for management
purposes, appear to a DS6000 or DS8000 as an independent Storage Complex.
Now within a Storage Complex, you find management consoles, Storage Units, and in the
case of DS8000, Storage Facility Images. First, we define each of these constructs.
2.1.1 What is a management console
A management console is a PC that is used to manage the devices within a Storage Complex.
In the case of the DS6000, it is a PC with a Windows operating system, onto which the user
has installed the DS Storage Manager Console software. This is usually referred to as a
Storage Management Console or SMC. In the case of the DS8000, it is a dedicated PC that
comes with a pre-installed Linux-based operating system and pre-installed management
software. The DS8000 console is called a Hardware Management Console (HMC).
In either case, the user interacts with the management console using either the Web
browser-based DS GUI or the Command-Line Interface (DS CLI). It is possible using the DS
GUI to manage Copy Services operations on multiple Storage Complexes.
2.1.2 What is a Storage Unit
A Storage Unit is the physical storage device (including expansion enclosures) that you see
when you walk into the computer room. If you open a rack door and look at a DS6000 system
enclosure (a 1750-511 or 1750-522) and all of its attached expansion enclosures (1750-EX1
or 1750-EX2), then you are looking at a single DS6000 Storage Unit. If you then look at a
DS8000 sitting on the floor (with any attached expansion racks), then you are looking at a
DS8000 Storage Unit. An example is a 2107-922 or 2107-932 with an attached 2107-9E2.
Another example is a 2107-9A2 with an attached 2107-9AE. In each example, you have a
single DS8000 Storage Unit.
2.1.3 What is a Storage Facility Image (SFI)
The Storage Facility Image construct is different for the DS6000 and DS8000.
DS6000 SFI
For the DS6000, the SFI concept does not really apply. You could say that each DS6000
Storage Unit contains one SFI. However, in general a DS6000 is always referred to as simply
a Storage Unit. When using the DS CLI, there are separate commands to display a DS6000
Storage Unit or a DS6000 SFI. However, both refer to the same device. Using the DS GUI,
you only work with the DS6000 Storage Unit. The GUI does not offer an option to select a
DS6000 SFI. In Example 2-1 on page 9, we connect to a DS6000 Management Console
using the DS CLI. We issue the lssu command to display the DS6000 Storage Unit and the
lssi command to display the DS6000 Storage Facility Image. In each case, we see the same
serial number and the same World Wide Node name (WWNN). The Storage Unit and the SFI
are the same.
8
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 2-1 Difference between a DS6000 Storage Unit and DS6000 SFI
dscli> lssu
Date/Time: 13 November 2005 19:21:17 IBM DSCLI Version: 5.1.0.204
Name
ID
Model WWNN
pw state
=========================================================
13-00247 IBM.1750-1300247 511
500507630EFFFE16 On
dscli> lssi
Date/Time: 13 November 2005 19:21:44 IBM DSCLI Version: 5.1.0.204
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.1750-1300247 IBM.1750-1300247 511 500507630EFFFE16 Online Enabled
DS8000 SFI
For the DS8000, the SFI concept is necessary, because a DS8000 can be ordered as a
model 9A2 or 9B2. In these models, all resources in the DS8000 are divided between two
separate machine images. This means that to all attached hosts, a single DS8000 Storage
Unit effectively contains two DS8000s. It is like having two separate machines in one physical
box.
This means that when using the DS GUI or the DS CLI, you have to be very clear whether
you are working with the SFI or with the Storage Unit. For most operations, you always work
with an SFI. You can tell the difference very easily. The serial number of the Storage Unit
always ends in 0 (zero). The serial number of the first SFI always ends in 1. If you have
ordered a model 2107-9A2 or 2107-9B2, then you also have a second SFI. The serial number
of the second SFI always ends in 2.
DS8000 model 921 and 922 SFI example
In Example 2-2, we connect to a DS8000 Management Console using DS CLI. We issue the
lssu command to display the DS8000 Storage Unit of a 2107-922. We then issue the lssi
command to display the DS8000 Storage Image on that Storage Unit. The Storage Unit serial
number is 7520780. The SFI serial number is 7520781. Note how the Storage Unit and the
SFI have different WWNNs.
Example 2-2 Difference between a DS8000 Storage Unit and DS8000 SFI - model 922
dscli> lssu
Date/Time: 13 November 2005 19:21:01 IBM DSCLI Version: 5.1.0.204
Name
ID
Model WWNN
pw state
=============================================================
2107-7520780 IBM.2107-7520780 922 5005076303FFF9A5 On
dscli> lssi
Date/Time: 13 November 2005 19:21:21 IBM DSCLI Version: 5.1.0.204
Name
ID
Storage Unit
Model WWNN
State ESSNet
=================================================================================
ATS_3_EXP IBM.2107-7520781 IBM.2107-7520780 922 5005076303FFC1A5 Online Enabled
DS8000 model 9A2 SFI example
In Example 2-3 on page 10, we connect to a DS8000 Management Console using DS CLI.
We issue the lssu command to display the DS8000 Storage Unit of a 2107-9A2. We then
issue the lssi command to display the DS8000 Storage Images on that Storage Unit. The
Storage Unit serial number is 7520780. The SFI serial numbers are 7520781 and 7520782.
The Storage Unit and the SFIs all have different WWNNs.
Chapter 2. Copy Services architecture
9
Example 2-3 Difference between a DS8000 Storage Unit and DS8000 SFI - model 9A2
dscli> lssu
Date/Time: 13 November 2005 19:25:25 IBM DSCLI Version: 5.1.0.204
Name
ID
Model WWNN
pw state
================================================================
2107_75ABTV1/V2 IBM.2107-75ABTV0 9A2
5005076303FFFE63 On
dscli> lssi
Date/Time: 13 November 2005 19:25:34 IBM DSCLI Version: 5.1.0.204
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.2107-75ABTV1 IBM.2107-75ABTV0 9A2 5005076303FFC663 Online Enabled
IBM.2107-75ABTV2 IBM.2107-75ABTV0 9A2 5005076303FFCE63 Online Enabled
2.1.4 What is a Storage Complex
A Storage Complex can be one or two DS8000 Storage Units managed by one or two DS
HMCs. A Storage Complex can also be one or two DS6000 Storage Units managed by one or
two DS SMCs. In Figure 2-1, you see a logical view of two Storage Complexes, each with one
DS6000 Storage Unit. They are running Remote Mirror and Copy. Now, because the two DS
SMCs are linked through Ethernet, you can connect using the DS GUI to either DS SMC and
manage Copy Services on both DS6000s.
Remote Mirror Copy Links
DS6000
DS SMC
Storage
Complex 1
DS6000
Ethernet Link
DS SMC
Storage
Complex 2
Figure 2-1 Logical view of a Storage Complex
Currently for the DS8000 and DS6000, we can manage two Storage Units in one Storage
Complex. For ESS 800, we can manage up to eight ESS 800s in a single ESS Copy Services
domain.
Note: Only servers of the same type, such as 2107, can be in the same Storage Complex.
However, Storage Complexes with different server types can be joined together. For
example, a DS8000 Storage Complex and a DS6000 Storage Complex can be connected.
10
IBM System Storage DS6000 Series: Copy Services in Open Environments
2.2 How does the new structure of Copy Services work
With the exception of zSeries commands (see 2.3, “System z communication path for Copy
Services” on page 13), communication to a DS6000 or DS8000 needs an available DS SMC
or DS HMC. See Figure 2-2.
Client
Client
Client
DS CLI or
Web browser
DS CLI or
Web browser
DS CLI or
Web browser
DS SMC
DS HMC
Network Interface
Server
Network Interface
Server
SFI
Network
Interface
Node
Network
Interface
Node
Network
Interface
Node
Microcode
Server 0
Server 1
Network
Interface
Node
Microcode
Controller 0
DS8000
Controller 1
DS6000
Copy
Services
Server
Microcode
Processor
complex1
Processor
complex2
ESS800
Figure 2-2 Network Interface communication structure
DS8000 management structure
Using Figure 2-2, we can see that:
򐂰 The client uses either DS CLI or a Web browser GUI to issue commands to the Network
Interface Server running on the DS HMC.
򐂰 The Network Interface Server software on the HMC communicates with the Client.
򐂰 The Network Interface Server software communicates with the Network Interface Node,
which resides on each server of a Storage Facility Image. From this point, the Network
Interface then talks to the microcode, which operates the DS8000.
1750 DS6000 management structure
Using Figure 2-2, we can see that:
򐂰 The client uses either DS CLI or a Web browser GUI to issue commands to the Network
Interface Server running on the DS SMC.
򐂰 The DS Network Interface Server software runs on the DS SMC, and this is what the client
communicates with.
Chapter 2. Copy Services architecture
11
򐂰 The Network Interface Server software communicates with the Network Interface Node,
which resides on each controller of the DS6000. From this point, the Network Interface
then interacts with the Microcode, which operates the controllers.
2105 ESS 800 management structure
Using Figure 2-2 on page 11, we can see that:
򐂰 The client uses either DS CLI or an ESS Copy Services Web Browser GUI to issue
commands to the ESS 800 Copy Services Server running on an ESS 800 cluster. The
client can also use the DS GUI to issue commands to the ESS 800 Copy Services Server
if a DS6000 SMC or DS8000 HMC is available through which to route them (not shown in
the diagram).
򐂰 The ESS 800 Copy Services Server then interacts with the Microcode, which operates the
ESS 800.
2.2.1 Remote Mirror and Copy between Storage Complexes
It is possible to use Remote Mirror and Copy between Storage Complexes as depicted in
Figure 2-3. In this scenario, we have three Storage Complexes. The complexes are
interconnected at the top of the chart by Storage Area Network (SAN) connections (solid
lines) and at the bottom of the chart by an Ethernet LAN (dotted lines).
SAN B
SAN A
Storage
Complex
2
DS6000
DS6000
(Complex 3)
ESS
800
ESS
800
Storage
Complex
1
DS8000
DS8000
ESS Copy
Services
Domain
SMC1
HMC1
LAN
Client 1
Client 2
Figure 2-3 Remote Mirror and Copy between Storage Complexes
2.2.2 Differences between the DS CLI and the DS GUI
The DS CLI is incapable of managing several domains in a single session. However, if a
single DS CLI client machine has network access to all Storage Complexes, then that client
can issue concurrent DS CLI commands to each complex. Each complex is then managed by
a separate DS CLI session. The DS CLI can be script driven, and, of course, scripts can be
saved. So, by using DS CLI, you can achieve automation.
12
IBM System Storage DS6000 Series: Copy Services in Open Environments
The DS GUI is accessed with a Web browser. The DS GUI cannot save tasks (unlike the
ESS, which can), and you cannot use the DS GUI to initiate a series of saved tasks. Instead,
you must use the DS CLI. If you want to use a single GUI to manage multiple Storage
Complexes, then, in the GUI, you must define all of the Storage Complexes. This definition
allows you to manage FlashCopy or Remote Mirror and Copy on, or between, every Storage
Unit in every defined and reachable Storage Complex.
When you look at the structure in Figure 2-2 on page 11, you can see that you need a
working DS SMC or DS HMC in every Storage Complex to communicate with the DS6000 or
DS8000 for that complex. For an inter-Storage Complex Remote Mirror and Copy, the DS
GUI establishes sessions to the source and targets Storage Complexes to show all paths and
LUNS. If no DS HMC or DS SMC is available at the remote Storage Complex, you cannot
select this Storage Complex as a target.
It is possible to have two SMCs or two HMCs in a Storage Complex for the purpose of
redundancy.
2.3 System z communication path for Copy Services
In a z/OS environment, you can issue Copy Services commands to the DS6000 and DS8000
using inband communication. This is done by sending commands through a FICON® (or
ESCON® in the case of DS8000) host link to a conduit CKD volume. From there, the
command gets passed to the microcode for execution. This is exactly the same as what is
done with an ESS 800. Depending on the operating system, various software packages are
available to issue commands; examples include TSO or ICKDSF.
The ability to send inband commands does not necessarily mean that the DS CLI or DS GUI
does not have a role to play. To be able to issue a command to a DS6000 or DS8000, a
System z™ operating system needs to be able to communicate with the relevant Storage
Unit. We might have a remote Storage Unit that is not connected through FICON (or ESCON
in the case of a DS8000) to an active z/OS. This means we are not able to access a conduit
CKD volume on the remote Storage Unit. Now, provided we have TCP/IP connectivity and an
appropriate management station (such as a Windows host with a Web browser or DS CLI
installed), we can use the DS CLI or the DS GUI to manage the remote Storage Unit. We can
then use inband commands to manage the local Storage Unit. If the remote and local Storage
Units are connected by RMC links, then we can also send some commands inband using the
RMC links, such as commands to establish FlashCopies of the remote RMC target volumes.
Chapter 2. Copy Services architecture
13
14
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 2
Part
2
Interfaces
In this part, we discuss the interfaces available to manage the Copy Services features of the
DS6000. We give an overview of the interfaces, discuss the options available, discuss
configuration considerations, and give some usage examples of the interfaces.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
15
16
IBM System Storage DS6000 Series: Copy Services in Open Environments
3
Chapter 3.
DS Storage Manager (GUI)
This chapter is an introduction to the IBM System Storage DS Storage Manager, which can
be used to configure and to administer the DS storage system. The DS Storage Manager is
an interface that is used to perform logical configurations, service, Copy Services
management, and firmware upgrades.
The topics covered in this chapter are:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
System and hardware requirements
Installation modes
Connecting to the DS6000 SMC
Real-time and Simulated configuration
Figure 3.4 on page 20
Managing the Storage Complex
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
17
3.1 System and hardware requirements
The DS Storage Manager is a Web-based graphical user interface (GUI) that is used to
perform logical configuration and Copy Services management functions. It can be accessed
with a Web browser from any location that has network access to the DS management
console.
The DS Storage Manager software must be installed on a user-provided computer. We refer
to this computer as the DS Storage Management Console (SMC).
Because the DS Storage Manager is required to manage the DS6000, to perform Copy
Services operations, or to allow remote service, the DS SMC computer must always be on.
A management console is the configuration, service, and management portal for the DS6000
series. You must have a computer to use as the management console and this computer
must meet the following minimum set of hardware and operating system compatibility
requirements:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
1.4 GHz Pentium® 4
256 KB cache
512 MB memory
1 GB disk space for the DS Storage Manager software
1 GB work space per managed Integrated RAID Controller (IRC)
IP Network connectivity to each RAID controller card
You might also want to have:
򐂰 IP network connectivity to an external network to enable remote support
򐂰 Serial connectivity to the DS6000
3.1.1 Supported operating systems
The following operating systems support the Management Console:
򐂰
򐂰
򐂰
򐂰
Microsoft® Windows 2000
Microsoft Windows 2000 Server
Microsoft Windows XP Professional
Microsoft Windows 2003 Server
Note: For the most recent information about currently supported operating systems, refer
to the IBM System Storage DS6000 Information Center Web site:
http://publib.boulder.ibm.com/infocenter/ds6000ic/index.jsp
Supported browsers
Following are the currently supported browsers to access the DS Storage Manager:
򐂰 Internet Explorer® 6.X
򐂰 Netscape 6.2
򐂰 Netscape 7.X
3.2 Installation modes
Install the DS Storage Manager using a graphical or silent mode for the Windows operating
systems. Access it using a Web browser from any location that has network access.
18
IBM System Storage DS6000 Series: Copy Services in Open Environments
You can choose to install the IBM System Storage DS Storage Manager on the Windows
operating system using either of the following modes:
򐂰 Graphical mode allows you to use an online wizard that guides you through the installation
process providing prompts and information necessary to complete the installation.
򐂰 Unattended mode (also called silent mode) allows you to customize a response file and
issue a command to complete the installation process.
After you install the DS6000 Storage Manager, the following results occur:
򐂰 Activation of the IBM System Storage DS Storage Manager server and the IBM System
Storage DS Network Interface server. These servers are set to automatic startup, so when
you start your computer, these servers automatically start as Windows services.
򐂰 Activation of the DS Storage Manager application, which includes real-time and simulated
manager components. You can install these components both on the same machine and
integrate them into the user interface. The real-time and simulated manager components
are designed to help you create and manage the physical and logical configurations of
your Storage Complexes and Storage Units. Plus, the Real-time manager application
provides you the opportunity to use the Copy Services features that you purchased.
Note: To see the complete installation process in detail, refer to IBM System Storage
DS6000: Installation, Troubleshooting, and Recovery Guide, GC26-7924.
3.3 Connecting to the DS6000 SMC
To connect to the DS6000 SMC from the browser, enter the Web site address (URL) of the
PC or the DS SMC that you have purchased. The Web site consists of the TCP/IP address as
shown in Figure 3-1, or a fully qualified name that the DNS server can resolve.
Figure 3-1 Connecting to a DS6000 with a Web browser
If, for example, your SMC IP address is 10.0.0.1 and you connect to http://10.0.0.1, you do
not get a successful connection, because the DS Storage Manager is reachable by
connecting to either:
http://10.0.0.1:8451/DS6000
https://10.0.0.1:8452/DS6000
Clearly, you should change the IP address to match the one used by your SMC. If you are
actually working on the SMC PC itself, you can also use 127.0.0.1, because this is the
loopback address that connects to the local PC.
Chapter 3. DS Storage Manager (GUI)
19
Logging in to the DS Storage Manager requires that you provide your user ID and password.
This function is generally administered through your system administrator and by your
company policies. The preconfigured user ID and password are as follows:
򐂰 User ID: admin
򐂰 Password: admin
When you log on to the SMC for the first time using the admin user ID, you must change the
password from admin to something more secure.
3.3.1 Real-time and Simulated configuration
You have the following options to access the DS Storage Manager:
򐂰 Simulated (offline) manager
This application allows you to create or modify logical configurations when disconnected
from the network. After creating an initial configuration, you can save it and then apply it to
a new or unconfigured Storage Unit at a later time. However, you cannot use the
Simulated manager to perform any type of Copy Services configuration. You must perform
Copy Services configuration from the Real-time manager.
򐂰 Real-time (online) manager
This provides real-time management support for logical configuration and Copy Services
functions to a DS6000 Storage Unit.
3.3.2 Advantages of using the DS GUI over the DS CLI
The DS Storage Manager GUI has the following advantages for managing Remote Mirror and
Copy functions:
򐂰 It requires less specialized skills for effective use.
򐂰 It is a real-time graphical interface to the DS6000 subsystem, which can present
information more easily to some users.
򐂰 You do not need to install anything on your PC in order to manage Copy Services - a
simple Web browser pointed to the DS SMC is enough.
3.3.3 Disadvantages of using the DS GUI over the DS CLI
The DS Storage Manager GUI has the following disadvantages for managing Remote Mirror
and Copy functions:
򐂰 You cannot perform all supported functions from this interface.
򐂰 You cannot use this interface for automated actions.
򐂰 You cannot save your actions and reuse them later.
You still need to understand the implications of the actions you are performing (for example,
you need to understand that a FlashCopy overwrites the target LUN or volume).
3.4 Accessing the Information Center
The Software Information Center displays product or application information. The system
provides a graphical user interface for browsing and searching online documentation. The
broad range of topics covered includes accessibility, Copy Services, device storage, host
system attachments, concurrent code loads, input/output configuration programs, and volume
storage.
20
IBM System Storage DS6000 Series: Copy Services in Open Environments
Accessing the Information Center directly
You can connect to the Information Center directly using a Web browser. If your DS SMC IP
address is 10.0.0.1, then you can access the Information Center running on that DS SMC at:
http://10.0.0.1:8455
If you are locally logged onto the DS SMC, then you can also use:
http://127.0.0.1:8455
If you want to access the public version of the DS6000 Information Center, you can find it at:
http://publib.boulder.ibm.com/infocenter/ds6000ic/index.jsp
Accessing the Information Center using the help symbol
Access the help panels by selecting the question mark (?) symbol on the upper-right side of
the panel, as indicated by the arrow in Figure 3-2.
Figure 3-2 Help option location
The help panel opens automatically on the current subject of the configuration panel as
shown in Figure 3-3 on page 22. In this example, the FlashCopy main page was open when
the help icon was clicked, so the FlashCopy help main page was opened.
Chapter 3. DS Storage Manager (GUI)
21
Figure 3-3 FlashCopy help Main page
3.5 Managing the Storage Complex
When you install the DS Storage Manager software on a PC, this automatically creates a
Storage Complex. A procedure then follows to add up to two DS6000s to the Storage
Complex. Each DS6000 is a Storage Unit. Because a DS SMC can manage two DS6000s,
this means that a single Storage Complex can consist of one DS SMC and two DS6000s. In
Figure 3-4, we have connected to the Storage Complex managed by the DS SMC at the
10.0.0.1 IP address. This complex consists of one Storage Unit, as indicated by the arrow.
10.0.0.1
Figure 3-4 A single Storage Complex with a single Storage Unit
A simple scenario can be that we have two sites. At the production site, we have a single
DS6000 managed by its own DS SMC. Then at a remote site, we have a second DS6000
managed by its own DS SMC. There is Ethernet connectivity between the sites. Because
each DS SMC manages only the DS6000 at its own site, this means we have two Storage
Complexes. If we want to use a single GUI to manage Copy Services at either site, then we
22
IBM System Storage DS6000 Series: Copy Services in Open Environments
have to add these Storage Complexes to each other. Once we have done this, we can use
one DS SMC to manage Copy Services on either Storage Unit at either site. We can also use
one DS SMC to set up Copy Services relationships between the two sites.
3.5.1 Procedure to add a Storage Complex
This section describes the procedure for adding a Storage Complex.
Note: To use either DS SMC to manage the Storage Units of the other, you have to do this
operation once in each direction. In other words, you must add Complex1 to Complex2,
and then you add Complex2 to Complex1.
Important: Make sure the user ID you use to log on to the DS6000 DS GUI also exists on
the other DS6000 SMC and that it has the same password. If not, the operation to add the
Storage Complex fails. You must always use this same user ID and password for
multi-complex management.
To add a Storage Complex:
1. Connect to the DS6000 SMC using the DS GUI.
2. Click Real-time manager.
3. Click Manage hardware.
4. Click Storage complexes.
5. Select Add Storage Complex and click Go. Figure 3-5 shows the Add Storage Complex
panel.
Figure 3-5 Add Storage Complex panel
6. Enter the IP address of the other Storage Complex’s DS SMC into the Management
console 1 IP address box. If the other Storage Complex has a second DS SMC, select the
Define a second Management console box and enter the second DS SMC address into
the Management console 2 IP address box. Now, click OK.
7. When the add Storage Complex operation completes, the Storage Complex panel now
shows the DS6000 SMC as an additional Storage Complex. In Figure 3-6 on page 24, we
can see the successful result. We have added a second Storage Complex that manages
two Storage Units.
Chapter 3. DS Storage Manager (GUI)
23
10.0.0.1
10.0.0.100
Figure 3-6 Successful addition of a Storage Complex
8. Having added one DS6000 Storage Complex to another, you are now able to use the
DS6000 DS GUI to create paths and Remote Mirror and Copy pairs between any of the
Storage Units. You can also use the DS6000 DS GUI to manage FlashCopy pairs on any
DS6000. This is assuming that the relevant licenses are present.
Note: You cannot perform the steps to add one DS6000 Storage Complex to another
DS6000 Storage Complex using the DS CLI.
24
IBM System Storage DS6000 Series: Copy Services in Open Environments
Chapter 3. DS Storage Manager (GUI)
25
26
IBM System Storage DS6000 Series: Copy Services in Open Environments
4
Chapter 4.
DS Command-Line Interface
(DS CLI)
This chapter provides an introduction to the DS Command-Line Interface (DS CLI), which you
can use to configure and to administer the DS storage system. We describe how you can use
the DS CLI to manage Copy Services relationships.
In this chapter, we cover:
򐂰
򐂰
򐂰
򐂰
򐂰
System requirements
Command modes
The commands
User assistance
Return codes
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
27
4.1 Introduction and functionality
The IBM System Storage DS Command-Line Interface enables open systems hosts to invoke
and manage FlashCopy and Remote Mirror and Copy functions through batch processes and
scripts.
The Command-Line Interface provides a full-function command set that allows you to check
your Storage Unit configuration and perform specific application functions when necessary.
Before you can use the DS CLI commands, you must ensure the following:
򐂰 You must equip the Storage Management Console (DS SMC) with the DS Storage
Manager graphical user interface (GUI).
򐂰 You must install the Storage Management Console (DS SMC) as a Full Management
Console installation management type.
򐂰 You must configure your Storage Unit (part of the DS Storage Manager post-installation
instructions).
򐂰 You must activate your license activation codes before you can use the DS CLI
commands associated with the Copy Services functions.
The following list highlights a few of the specific types of functions you can perform with the
DS CLI:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
Create user IDs that can be used with the GUI and the DS CLI.
Manage user ID passwords.
Install activation keys for licensed features.
Manage Storage Complexes and Units.
Configure and manage Storage Facility Images.
Create and delete RAID arrays, Ranks, and Extent Pools.
Create and delete logical volumes.
Manage host access to volumes.
Check the current Copy Services configuration that is used by the Storage Unit.
Create, modify, or delete Copy Services configuration settings.
4.2 Supported operating systems for the DS CLI
The DS CLI can be installed on these operating systems:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
AIX 5.1, 5.2, and 5.3
HP-UX 11.0 and 11i v1 and v2
HP-True64 5.1 and 5.1A
Linux (Red Hat 3.0 Advanced Server (AS) and Enterprise Server (ES))
SUSE Linux SLES 8, SLES 9, SUSE 8, and SUSE 9
Novell Netware 6.5
System i system i5/OS 5.3
Sun™ Solaris 7, 8, and 9
Windows 2000, Windows Datacenter, Windows 2003, and Windows XP
Note: The DS CLI cannot be installed on a Windows 64-bit operating system.
28
IBM System Storage DS6000 Series: Copy Services in Open Environments
Important: For the most recent information about currently supported operating systems,
refer to the IBM System Storage DS6000 Information Center Web site:
http://publib.boulder.ibm.com/infocenter/ds6000ic/index.jsp
The DS CLI is supplied and installed using a CD that ships with the machine. The installation
does not require a reboot of the open systems host. The DS CLI requires Java™ 1.4.1 or
higher. Java 1.4.2 for Windows, AIX, and Linux is supplied on the CD. Many hosts might
already have a suitable level of Java installed. The installation program checks for this
requirement during the installation process, and it does not install DS CLI if you do not
have the correct version of Java.
The installation process can be performed using a shell, such as the bash or Korn shell, or
the Windows command prompt, or using a GUI interface. If performed using a shell, the
installation process can be performed silently using a profile file. The installation process also
installs software that allows the DS CLI to be completely uninstalled if it is no longer required.
If you need any assistance to install the DS CLI, you can refer to the IBM System Storage
DS6000: Command-Line Interface User´s Guide, GC26-7922.
4.3 User accounts
The administrator account is set up automatically at the time of installation. It is accessed with
the admin user ID and the default admin password. This password is temporary and you must
change the password, before you can use any of the other functions. There are seven groups
the administrator can assign to a user. The groups and the associated functions allowed by
the assignment are as follows:
򐂰 admin: Allows access to all storage management console server service methods and all
Storage Image resources.
򐂰 op_volume: Allows access to service methods and resources that relate to logical
volumes, Hosts, Host Ports, logical subsystems, and Volume Groups, excluding security
methods.
򐂰 op_storage: Allows access to physical configuration service methods and resources,
including Storage Complex, Storage Image, Rank, Array, and Extent Pool objects.
򐂰 op_copy_services: Allows access to all Copy Services service methods and resources,
excluding security methods.
򐂰 service: Monitors authority and access to all management console server service methods
and resources, such as performing code loads and retrieving problem logs.
򐂰 monitor: Allows access to list and show commands. It provides access to all read-only,
non-security management console server service methods and resources.
򐂰 no access: Does not allow access to any service method or Storage Image resources. By
default, this user group is associated to any user account in the security repository that is
not associated with any other user group.
Note: A user can be assigned to more than one user group.
Important: When a new user is created, the password that was initially set automatically
expires and must be changed when the user first logs on.
Chapter 4. DS Command-Line Interface (DS CLI)
29
4.4 DS CLI profile
You can create default settings for the Command-Line Interface by defining one or more
profiles on the system. For example, you can specify the Storage Management console
(DS SMC) for the session, specify the output format for list commands, specify the number
of rows per page in the command-line output, and specify whether to include a banner with
the command-line output.
If a user enters a value with a command that is different from a value in the profile, the
command overrides the profile.
You have several options for using profile files:
򐂰 You can modify the default profile. The default profile, dscli.profile, is installed in the profile
directory with the software, for example, c:\Program Files\IBM\DSCLI\profile\dscli.profile
for the Windows platform and /opt/ibm/dscli/profile/dscli.profile for UNIX® and Linux
platforms.
򐂰 You can make a personal default profile by making a copy of the system default profile as
<user_home>/dscli/profile/dscli.profile. The home directory, <user_home>, is designated
as follows:
– Windows system: C:\Documents and Settings\<user_name>
– Unix/Linux system: /home/<user_name>
򐂰 You can create a profile for the Storage Unit operations. Save the profile in the user profile
directory. For example:
– c:\Program Files\IBM\DSCLI\profile\operation_name1
– c:\Program Files\IBM\DSCLI\profile\operation_name2
Important: The default profile file created when you install the DS CLI is potentially
replaced every time you install a new version of the DS CLI. It is a better practice to open
the default profile and then save it as a new file. You can then create multiple profiles and
reference the relevant profile file using the -cfg parameter.
These profile files can be specified using the DS CLI command parameter -cfg
<profile_name>. If the -cfg file is not specified, the user’s default profile is used. If a user’s
profile does not exist, the system default profile is used.
Note: A password file, generated using the managepwfile command is located at the
following directory: <user_home>/dscli/security/security.dat.
When you install the Command-Line Interface software, the default profile is installed in the
profile directory with the software. The file name is dscli.profile, for example, c:\Program
Files\IBM\DSCLI\profile\dscli.profile.
The available variables, detailed descriptions, and information about how to handle them can
be found in IBM System Storage DS6000: Command-Line Interface User´s Guide,
GC26-7922.
30
IBM System Storage DS6000 Series: Copy Services in Open Environments
4.5 Command structure
This is a description of the components and structure of a Command-Line Interface
command.
A Command-Line Interface command consists of one to four types of components, arranged
in the following order:
1. Command name: Specifies the task that the Command-Line Interface is to perform.
2. Flags: Modify the command. They provide additional information that directs the
Command-Line Interface to perform the command task in a specific way.
3. Flags parameter: Provides information that is required to implement the command
modification that is specified by a flag.
4. Command parameters: Provide basic information that is necessary to perform the
command task. When a command parameter is required, it is always the last component
of the command, and it is not preceded by a flag.
4.6 Copy Services commands
We can classify the commands available with the DS CLI as follows:
1. ls commands: Give you brief information about the Copy Service state. See Table 4-1.
2. show commands: Give you detailed information about the Copy Service state. See
Table 4-2.
3. mk commands: Used to create relationships. See Table 4-3 on page 32.
4. rm commands: Used to remove relationships. See Table 4-4 on page 32.
5. options commands: Used to modify some options in relationships that were previously
created. See Table 4-5 on page 32.
Table 4-1 List commands
Command
Description
lsflash
Lists FlashCopy relationships
lsremoteflash
Lists Remote FlashCopy relationships (inband)
lspprc
Lists Remote Mirror and Copy volume relationships
lspprcpath
Lists existing Remote Mirror and Copy path definitions
lsavailpprcport
Lists available ports that can be defined as Remote Mirror and Copy
lssession
Displays a list of Global Mirror sessions for an LSS
Table 4-2 List detailed commands
Command
Description
showgmir
Displays detailed properties and performance metrics for Global Mirror
showgmircg
Displays Consistency Group status for a Global Mirror session
showgmiroos
Displays the number of unsynchronized (out of sync) tracks for a Global Mirror session
Chapter 4. DS Command-Line Interface (DS CLI)
31
Table 4-3 Creation commands
Command
Description
mkflash
Initiates a point-in-time copy from a source to a target volume
mkremoteflash
Initiates a remote copy through a Remote Mirror and Copy relationship
mkgmir
Starts Global Mirror for a session
mkpprc
Establishes a Remote Mirror and Copy relationship
mkpprcpath
Establishes or replaces a Remote Mirror and Copy path over a Fibre Channel
mkesconpprcpath
Creates a Remote Mirror and Copy path over an ESCON connection
mksession
Opens a Global Mirror for a session
Table 4-4 Deletion commands
Command
Description
rmflash
Removes a relationship between FlashCopy pairs
rmremoteflash
Removes a relationship between remote FlashCopy pairs
rmgmir
Removes Global Mirror for the specified session
rmpprc
Removes a Remote Mirror and Copy relationship
rmpprcpath
Removes a Remote Mirror and Copy path
rmsession
Closes an existing Global Mirror session
Table 4-5 Options commands
Command
Description
commitflash
Commits data to a target volume to form a consistency between the source and target
resyncflash
Incremental FlashCopy process
reverseflash
Reverses the direction of a FlashCopy pair
revertflash
Overwrites new data with data saved at the last consistency formation
setflashrevertible
Modifies a remote FlashCopy pair that is part of a Global Mirror to revertible
commitremoteflash
Commits data to a target volume to form a consistency (inband)
resyncremoteflash
Incremental FlashCopy process (inband)
revertremoteflash
Overwrites new data with data saved at the last consistency formation (inband)
setremoteflashrevertible
Modifies a remote FlashCopy pair that is part of a Global Mirror to revertible (inband)
unfreezeflash
Resets a FlashCopy Consistency Group
failoverpprc
Changes a secondary device into a primary suspended and keeps the primary in current
failbackpprc
Usually used after a failoverpprc to reverse the direction of the synchronization
freezepprc
Creates a new Remote Mirror and Copy Consistency Group
pausepprc
Pauses an existing Remote Mirror and Copy volume pair relationship
resumepprc
Resumes a Remote Mirror and Copy relationship for a volume pair
32
IBM System Storage DS6000 Series: Copy Services in Open Environments
Command
Description
unfreezepprc
Thaws an existing Remote Mirror and Copy Consistency Group
pausegmir
Pauses a Global Mirror processing for a session
resumegmir
Resumes a Global Mirror processing for a session
chsession
Allows you to modify a Global Mirror session
Important: The Remote Mirror and Copy commands are asynchronous. This means that
the command is issued to the DS CLI server and, if it is accepted successfully, you receive
a successful completion code; however, background activity can still occur. For example, a
Metro Mirror pair takes some time to establish, because the tracks need to be copied
across from the primary to the secondary device. In this example, you should check the
state of the volumes using the lspprc command until the pairs have reached the Duplex
state.
Note: The mkflash and mkpprc commands offer the -wait flag, which delays command
response until copy complete status is achieved. You can choose to use this flag, if you
want to be sure of successful completion.
4.7 Using the DS CLI application
You have to log into the DS CLI application to use the command modes. There are three
command modes for the DS CLI:
򐂰 Single-shot mode
򐂰 Interactive mode
򐂰 Script mode
4.7.1 Single-shot mode
Use the DS CLI single-shot command mode if you want to issue an occasional command, but
do not want to keep a history of the commands that you have issued.
You must supply the login information and the command that you want to process at the
same time. Use the following example to use the single-shot mode:
1. Enter:
dscli -hmc1 <hostname or ip address> -user <adm user> -passwd <pwd> <command>
2. Wait for the command to process and display the end results.
Example 4-1 shows the use of the single-shot command mode.
Example 4-1 Single-shot command mode
C:\Program Files\ibm\dscli>dscli -hmc1 10.10.10.1 -user admin -passwd adminpwd
lsuser
Date/Time: 24 de Maio de 2005 14h38min20s BRT IBM DSCLI Version: X.X.X.X
Name
Group State
=====================
admin
admin locked
admin
admin active
Chapter 4. DS Command-Line Interface (DS CLI)
33
exit status of dscli = 0
Note: When logging in to the DS CLI, you can use the hostname or the IP address of the
DS SMC.
4.7.2 Script command mode
Use the DS CLI script command mode if you want to use a sequence of DS CLI commands.
Administrators can use this mode to create automated processes, for example, establishing
Remote Mirror and Copy relationships for volume pairs.
You can issue the DS CLI script from the command prompt at the same time that you provide
your login information:
1. Enter:
dscli -hmc1 <hostname ip address> -user <user name> -passwd <pwd> -script <full
path of script file>
2. Wait for the script to process and provide a report regarding the success or failure of the
process.
Example 4-2 shows the use of the script command mode.
Example 4-2 Script command mode
C:\Program Files\ibm\dscli>dscli -hmc1 10.10.10.1 -user admin -passwd adminpwd
-script c:\test.cli
Date/Time: 24 de Maio de 2005 14h40min22s BRT IBM DSCLI Version: X.X.X.X DS: IBM
IBM.1750-1367890
ID
WWPN
State Type
topo
portgrp
===============================================================
I0000 500507630E01FC00 Online Fibre Channel-LW SCSI-FCP 0
I0001 500507630E03FC00 Online Fibre Channel-LW 0
I0002 500507630E05FC00 Online Fibre Channel-LW SCSI-FCP 0
I0003 500507630E07FC00 Online Fibre Channel-LW 0
I0100 500507630E81FC00 Online Fibre Channel-LW SCSI-FCP 0
I0101 500507630E83FC00 Online Fibre Channel-LW 0
I0102 500507630E85FC00 Online Fibre Channel-LW SCSI-FCP 0
I0103 500507630E87FC00 Online Fibre Channel-LW 0
Date/Time: 24 de Maio de 2005 14h40min36s BRT IBM DSCLI Version: X.X.X.X
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.1750-1312345 IBM.1750-1312345 511 500507630EFFFC6F Online Enabled
exit status of dscli = 0
Important: The DS CLI script can contain only DS CLI commands. Use of shell commands
results in process failure. You can add comments to the scripts prefixed by the number or
hash sign (#).
When typing the command, you can use the hostname or the IP address of the Storage
Management Console (DS SMC).
4.7.3 Interactive mode
Use the DS CLI interactive command mode when you have multiple transactions to process
that cannot be incorporated into a script. The interactive command mode provides a history
function that makes repeating or checking prior command usage easy to do.
34
IBM System Storage DS6000 Series: Copy Services in Open Environments
1. Log on to the DS CLI application at the directory where it is installed.
2. Provide the information that is requested by the information prompts. The information
prompts might not appear if you have provided this information in your profile file. The
command prompt switches to a dscli command prompt.
3. Begin using the DS CLI commands and parameters. You are not required to begin each
command with dscli, because this prefix is provided by the dscli command prompt.
Example 4-3 shows the use of the interactive command mode.
Example 4-3 Interactive command mode
C:\Program Files\ibm\dscli>dscli
Enter your username: admin
Enter your password:
Date/Time: 24 de Maio de 2005 14h42min17s BRT IBM DSCLI Version: X.X.X.X DS:
IBM.1750-1312345
.
dscli> lsarraysite
Date/Time: 24 de Maio de 2005 15h8min57s BRT IBM DSCLI Version: X.X.X.X DS: IBM.
1750-1312345
arsite DA Pair dkcap (Decimal GB) State
Array
================================================
S1
0
146.0 Assigned A0
S2
0
146.0 Assigned A0
S3
0
146.0 Assigned A1
S4
0
146.0 Assigned A1
.
dscli> lssi
Date/Time: 24 de Maio de 2005 14h42min59s BRT IBM DSCLI Version: X.X.X.X
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.1750-1312345 IBM.1750-1312345 511 500507630EFFFC6F Online Enabled
dscli> quit
exit status of dscli = 0
Note: When typing the command, you can use the hostname or the IP address of the
Storage Management Console (DS SMC).
4.8 Return codes
When you exit the DS CLI, the exit status code is provided. This is effectively a return code. If
the DS CLI commands are issued as separate commands (rather than using script mode),
then a return code is presented for every command. If a DS CLI command fails (for instance,
due to a syntax error or the use of an incorrect password), then a failure reason and a return
code are presented. You can use standard techniques to collect and analyze return codes.
The return codes used by the DS CLI are shown in Table 4-6 on page 36.
Chapter 4. DS Command-Line Interface (DS CLI)
35
Table 4-6 Return code table
Return code
Category
Description
0
Success
The command was successful.
2
Syntax error
There is a syntax error in the command.
3
Connection error
There was a connection problem to the server.
4
Server error
The DS CLI server had an error.
5
Authentication error
Password or user ID details are incorrect.
6
Application error
The DS CLI application had an error.
4.9 User assistance
The DS CLI is designed to include several forms of user assistance. The main form of user
assistance is through the help command. Examples of usage include:
򐂰 help lists all available DS CLI commands.
򐂰 help -s lists all DS CLI commands with a brief description of each command.
򐂰 help -l lists all DS CLI commands with syntax information.
If the user is interested in more details about a specific DS CLI command, they can use -l
(long) or -s (short) against a specific command. In Example 4-4, the -s parameter is used to
get a short description of the mkflash command’s purpose.
Example 4-4 Use of the help -s command
dscli> help -s mkflash
mkflash The mkflash command initiates a point-in-time copy from source volumes to target
volumes.
In Example 4-5, the -l parameter is used to get a list of all parameters that can be used with
the mkflash command.
Example 4-5 Use of the help -l command
dscli> help -l mkflash
mkflash [ { -help|-h|-? } ] [-dev storage_image_ID] [-tgtpprc] [-tgtoffline] [-t
gtinhibit] [-freeze] [-record] [-persist] [-nocp] [-wait] [-seqnum Flash_Sequenc
e_Num] SourceVolumeID:TargetVolumeID ... | -
Man pages
A man page is available for every DS CLI command. Man pages are most commonly seen in
UNIX-based operating systems to give information about command capabilities. This
information can be displayed by issuing the relevant command followed by -h, -help, or
-?, for example:
dscli> mkflash -help
or
dscli> help mkflash
36
IBM System Storage DS6000 Series: Copy Services in Open Environments
4.10 Usage examples
It is not the intent of this section to list every DS CLI command and its syntax. If you need to
see a list of all the available commands, or require assistance using the DS CLI commands,
you should read the IBM System Storage DS6000: Command-Line Interface User´s Guide,
GC26-7922, or you can use the online help.
Example 4-6 shows some of the common commands used on a DS6000.
Example 4-6 Examples of DS CLI commands
# The following command establishes flashcopy pairs
dscli> mkflash -dev IBM.1750-1300861 0100-0102:0300-0302
Date/Time: April 25, 2005 5:59:56 PM IST IBM DSCLI Version: 0.0.0.0 DS: IBM.1750-1300861
CMUC00137I mkflash: FlashCopy pair 0100:0300 successfully created.
CMUC00137I mkflash: FlashCopy pair 0101:0301 successfully created.
CMUC00137I mkflash: FlashCopy pair 0102:0302 successfully created.
# The following command establishes Remote Mirror and copy paths
dscli> mkpprcpath -dev IBM.1750-1300861 -remotedev IBM.1750-1300861 -remotewwnn
5005076303FFC03D -srclss 00 -tgtlss 02 I0030:I0031 I0100:I0101
Date/Time: 18:33:22 IST April 25, 2005 IBM DSCLI Version: 0.0.0.0 DS: IBM.1750-1300861
CMUC00149I mkpprcpath: Remote Mirror and Copy path 00:02 successfully established.
# The following command establishes Remote Mirror and copy pairs
dscli> mkpprc -dev IBM.1750-1300861 -remotedev IBM.1750-1300866 -type gcp 0006:0206
Date/Time: 18:43:31 IST April 25, 2005 IBM DSCLI Version: 0.0.0.0 DS: IBM.1750-1300861
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 0006:0206 successfully
created.
Chapter 4. DS Command-Line Interface (DS CLI)
37
38
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 3
Part
3
FlashCopy
This part of the book describes the IBM System Storage FlashCopy for DS6000 when used in
an open systems environment. We discuss the features of FlashCopy and describe the
options for its setup. We also show you which management interfaces you can use, as well
as important aspects to consider when establishing FlashCopy relationships.
This part ends with examples that show how FlashCopy is used in daily business to support
the various environments where immediate copies of production data are a requirement.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
39
40
IBM System Storage DS6000 Series: Copy Services in Open Environments
5
Chapter 5.
FlashCopy overview
FlashCopy creates a copy of a volume at a specific point-in-time, which we also refer to as a
Point-in-Time copy, instantaneous copy, or time-zero copy (t0 copy).
This chapter explains the basic characteristics of FlashCopy when used in an open systems
environment with the DS6000. This chapter discusses the following topics:
򐂰 FlashCopy operational areas
򐂰 FlashCopy basic concepts
򐂰 FlashCopy in combination with other Copy Services
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
41
5.1 FlashCopy operational environments
It takes only a few seconds to establish the FlashCopy relationships for tens to hundreds or
more volume pairs. The copy is then immediately available for both read and write access. In
a 24x7 environment, the quickness of the FlashCopy operation allows us to use FlashCopy in
very large environments and to take multiple FlashCopies of the same volume for use with
different applications. Some of the different uses of FlashCopy are shown in Figure 5-1:
Production
Backup
System
Production
System
System
Operation
Reverse
Integration
System
Business
Integration
FlashCopy
Data
Backup
System
Production
data
Test
System
Development
other
systems
Application
Help desk
or
System
Operation
System
Operation
Data Mining
System
Analysis
Team
Figure 5-1 FlashCopy uses
FlashCopy is suitable for the following operational environments:
򐂰 Production backup system
A FlashCopy of the production data allows data recovery from an older level of data. This
might be necessary due to a user error or a logical application error. Let us assume that a
user accidentally deletes a customer record. The production backup system could work
with a FlashCopy of the data. The necessary part of the customer data can be exported
and then be imported into the production environment. Thus, the production continues,
and while a specific problem is being fixed, the majority of the users can work with the
application without recognizing any problems.
The FlashCopy of the data could also be used by system operations to reestablish
production in case of any server errors.
򐂰 Data backup system
A FlashCopy of the production data allows the client to create backups with the shortest
possible application outage. The main reason for data backup is to provide protection in
case of source data loss due to disaster, hardware failure, software failure, or user errors.
򐂰 Data mining system
A FlashCopy of the data can be used for data analysis, thus avoiding performance impacts
for the production system due to long running data mining tasks.
42
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Test system
Test environments created by FlashCopy can be used by the development team to test
new application functions with real production data, thus speeding up the test setup
process.
򐂰 Integration system
New application releases (for example, SAP® releases) are likely to be tested prior to
putting them onto a production server. By using FlashCopy, a copy of the production data
can be established and used for integration tests.
With the capability to reverse a FlashCopy, a previously created FlashCopy can be used
within seconds to bring production back to the level of data that it had at the time when the
FlashCopy was taken.
5.2 Terminology
When discussing Metro Mirror, Global Copy, and Global Mirror, you see that the following
terms are frequently used and interchangeably:
򐂰 The terms local, production, application, primary, or source denote the site where the
production applications run while in normal operation. These applications create, modify,
and read the data, that is, the application data. The meaning is extended to the disk
subsystem that holds the data as well as to its components: volumes and LSS.
򐂰 The terms remote, recovery, backup, secondary, or target denote the site to where the
data is replicated, the copy of the application data. The meaning is extended to the disk
subsystem that holds the data as well as to its components: volumes and LSS.
When discussing FlashCopy, we use the term source to refer to the original data that is
created by the application. And we use the term target to refer to the point-in-time backup
copy.
Also, the terms LUN and volume are used interchangeably in our discussions.
5.3 Basic concepts
By doing a FlashCopy, a relationship is established between a source and a target. Both are
considered to form a FlashCopy pair.
As a result of the FlashCopy, either all physical blocks from the source volume are copied (full
copy) or, when using the nocopy option, only those parts which change in the source data
since the FlashCopy has been established. Currently, the target volume needs to be the
same size or bigger than the source volume whenever FlashCopy is used to flash a whole
volume.
Typically, large applications such as databases have their data spread across several
volumes and their volumes should all be FlashCopied at exactly the same point-in-time.
FlashCopy offers Consistency Groups, which allow multiple volumes to be FlashCopied at
exactly the same instance.
The following are basic characteristics of the FlashCopy operation:
򐂰 Establish FlashCopy relationship
When the FlashCopy starts, the relationship between source and target is established
within seconds by creating a pointer table, including a bitmap for the target.
Chapter 5. FlashCopy overview
43
While the FlashCopy relationship is being created, the DS6000 holds off I/O activity to a
volume for a time period by putting the source volume in a SCSI queue full state. During
this state, any new I/Os issued receive a queue full error and are automatically reissued by
the host bus adapter. In this way, no user disruption or intervention is required. I/O activity
resumes when the FlashCopy is established.
If all bits for the bitmap of the target are set to their initial values, this means that no data
block has been copied so far. The data in the target is not modified during setup of the
bitmaps. At this first step, the bitmap and the data look as illustrated in Figure 5-2.
FlashCopy established at time t0 (time-zero)
time
t0
bitmap
1
source volume
1
1
1
1
1
target volume
data
data
t0
t0
t0
t0
t0
t0
Figure 5-2 FlashCopy at time t0
Once the relationship has been established, it is possible to perform read and write I/Os
on both the source and the target. Assuming that the target is used for reads only while
production is ongoing, things look as illustrated in Figure 5-3.
Writing to the source volume and
reading from the source and the target volume
tx
t0
read
ty
write x
tz
write y
write z
time
read 2
read 1
time-zero data not yet
available in target volume:
read it from source volume.
1
0
source volume
1
bitmap
0
1
target volume
data
data
t0
tx
t0
tz
t0
t0
t0
t0
before physical write to the source volume:
copy time-zero data from the source volume to the target volume
Figure 5-3 Reads from source and target volumes and writes to source volume
44
1
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Reading from the source
The data is read immediately (see Figure 5-3 on page 44).
򐂰 Writing to the source
Whenever data is written to the source volume while the FlashCopy relationship exists, the
storage subsystem makes sure that the time-zero-data is copied to the target volume prior
to overwriting it in the source volume.
To identify if the data of the physical block on the source volume needs to be copied to the
target volume, the bitmap is analyzed. If it identifies that the time-zero data is not available
on the target volume, then the data is copied from source to target. If it states that the
time-zero data has already been copied to the target volume, then no further action is
done (see Figure 5-3 on page 44).
It is possible to use the target volume immediately for reading data and also for writing
data.
򐂰 Reading from the target
Whenever a read-request goes to the target while the FlashCopy relationship exists, the
bitmap is used to identify if the data has to be retrieved from the source or from the target.
If the bitmap states that the time-zero data has not yet been copied to the target, then the
physical read is directed to the source. If the time-zero data has already been copied to
the target, then the read is performed immediately against the target (see Figure 5-3 on
page 44).
򐂰 Writing to the target
Whenever data is written to the target volume while the FlashCopy relationship exists, the
storage subsystem makes sure that the bitmap is updated. This way the time-zero data
from the source volume never overwrites updates done directly to the target volume (see
Figure 5-4).
Writing to the target volume
t0
ta
tb
write a
write b
time
bitmap
0
0
source volume
1
0
tx
t0
ty
t0
1
target volume
data
data
t0
1
t0
ta
t0
tb
Figure 5-4 Writes to target volume
򐂰 Terminating the FlashCopy relationship
The FlashCopy relationship is automatically ended when all tracks have been copied from
the source volume to the target volume. The relationship can also be explicitly withdrawn
by issuing the corresponding commands. If the persistent FlashCopy option was specified
then the FlashCopy relationship must be withdrawn explicitly.
Chapter 5. FlashCopy overview
45
5.3.1 Full volume copy
When the copy option is invoked and the establish process completes, a background process
is started that copies all data from the source to the target. Once this process finishes and if
there were no updates on the target, the picture we get is similar to the one in Figure 5-5. If
not explicitly defined as persistent, the FlashCopy relationship ends as soon as all data is
copied.
Background copy
tx
t0
time
ty
bitmap
0
0
source volume
0
0
0
0
target volume
data
data
t0
tx
t0
ty
t0
t0
t0
t0
t0
t0
t0
t0
Background copy will copy all time-zero data from source volume to target volume
Figure 5-5 Target volume after full volume FlashCopy relationship finishes
If there were writes to the target, then the picture we get is similar to the one in Figure 5-6.
Background copy
tx
t0
ta
ty
0
time
bitmap
tb
0
source volume
0
0
0
target volume
data
data
t0
tx
t0
ty
t0
0
t0
ta
t0
t0
tb
t0
t0
Background copy will copy all time-zero data from source volume to target volume.
Figure 5-6 FlashCopy after updates to the target volume
5.3.2 Nocopy option
If FlashCopy is established using the nocopy option, then the result is as shown in Figure 5-3
on page 44 and Figure 5-4 on page 45. The relationship lasts until it is explicitly withdrawn or
until all data in the source volume has been modified. Blocks for which no write occurred on
46
IBM System Storage DS6000 Series: Copy Services in Open Environments
the source or on the target stay as they were at the time when the FlashCopy was
established.
If the persistent FlashCopy option was specified, the FlashCopy relationship must be
withdrawn explicitly.
5.4 FlashCopy in combination with other Copy Services
Volume-based FlashCopy can be used in various combinations with other Copy Services,
while the most suitable combinations depend on the characteristics of the environment and
the requirements.
Note: The sample scenarios discussed in the present section only apply to full-volume
FlashCopy. Data set FlashCopy is not supported in the open systems environment.
5.4.1 FlashCopy and Metro Mirror
FlashCopy
source
target
Metro
Mirror
primary
Metro
Mirror
primary
secondary
source
target
FlashCopy
Figure 5-7 FlashCopy and Metro Mirror
As illustrated in Figure 5-7, at the primary Metro Mirror site, the following combinations are
supported:
򐂰 A FlashCopy source volume can become a Metro Mirror primary volume and the reverse
is also true. The order of creation is optional.
򐂰 A FlashCopy target volume can become a Metro Mirror primary volume and the reverse is
also true. If you wish to use a FlashCopy target volume as a Metro Mirror primary, be
aware of the following considerations:
– The recommended order is to first establish the Metro Mirror, and then create a
FlashCopy to that Metro Mirror primary using the -tgtpprc parameter.
The Metro Mirror secondary is not in a fully consistent state until the Metro Mirror
enters the full duplex state.
Chapter 5. FlashCopy overview
47
– If you create the FlashCopy first and then do a Metro Mirror of the FlashCopy target,
you must monitor the progress of the FlashCopy background copy. In this case, the
following consideration applies:
The Metro Mirror secondary is not in a fully consistent state until the FlashCopy
background copy process is complete.
Use the copy option to ensure that the entire FlashCopy source volume data is copied
to the Metro Mirror secondary.
On the secondary site of the Metro Mirror, a FlashCopy source volume can be the Metro
Mirror secondary volume and the reverse is also true. There are no restrictions on which
relationship should be defined first.
5.4.2 FlashCopy and Global Copy
FlashCopy
source
target
Global
Copy
primary
Global
Copy
primary
secondary
source
target
FlashCopy
Figure 5-8 FlashCopy and Global Copy
As illustrated in Figure 5-8, the following combinations are possible at the primary Global
Copy site:
򐂰 A FlashCopy source volume can become a Global Copy primary volume and the reverse
is also true. The order of creation is optional.
򐂰 A FlashCopy target volume can become a Global Copy primary volume and the reverse is
also true. If you want to use a FlashCopy target volume as a Global Copy primary, be
aware of the following considerations:
– The recommended order is to first establish the Global Copy, and then create a
FlashCopy to that Global Copy primary using the -tgtpprc parameter.
The Global Copy secondary is not in a fully consistent state until the Global Copy
enters the full duplex state.
Run the mkpprc -type mmir command to force the Global Copy to enter the full duplex
state.
– If you create the FlashCopy first, and then do a Global Copy of the FlashCopy target,
you must monitor the progress of the FlashCopy background copy.
48
IBM System Storage DS6000 Series: Copy Services in Open Environments
The Global Copy secondary is not in a fully consistent state until the FlashCopy
background copy process is complete.
Use the copy option to ensure that the entire FlashCopy source volume data is copied
to the Global Copy secondary.
On the secondary site of the Global Copy, a FlashCopy source volume can be based on the
secondary volume for the Global Copy.
5.4.3 FlashCopy and Global Mirror
FlashCopy in combination with Global Mirror supports only one type of relationship at the
primary site; see Figure 5-9.
A FlashCopy source volume can become a Global Mirror primary volume and the reverse is
also true. The relationships can be established in any sequence. A FlashCopy target volume
cannot become a Global Mirror primary volume.
FlashCopy
source
target
primary
Global
Mirror
secondary
Global
Mirror
Figure 5-9 FlashCopy and Global Mirror
On the Global Mirror secondary site, the Global Mirror target volume cannot be used as
FlashCopy source or FlashCopy target unless the Global Mirror pair is first suspended.
Chapter 5. FlashCopy overview
49
50
IBM System Storage DS6000 Series: Copy Services in Open Environments
6
Chapter 6.
FlashCopy options
This chapter discusses the options of FlashCopy when working with the IBM System Storage
DS6000 series in an open systems environment. This chapter explains the following options:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
Multiple Relationship FlashCopy
Consistency Group FlashCopy
FlashCopy on existing Metro Mirror or Global Copy source
Incremental FlashCopy
Remote FlashCopy
Persistent FlashCopy
Reverse Restore and Fast Reverse Restore
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
51
6.1 Multiple Relationship FlashCopy
It is possible to establish up to 12 FlashCopy relationships using the same source. In other
words, a source volume can have up to 12 target volumes. However, a target volume can still
only have one source. Furthermore, cascading FlashCopy is not allowed (that is, a volume
cannot be both a source and a target volume).
Following is a summary of the considerations that apply:
򐂰 A FlashCopy source volume can have up to 12 FlashCopy target volumes.
Note: Only one of those targets can be defined as incremental FlashCopy.
򐂰 For each source volume, only one FlashCopy relationship can be reversed (the one having
the -record attribute).
򐂰 A FlashCopy target volume can have only one FlashCopy source volume.
򐂰 A FlashCopy target volume cannot be a FlashCopy source volume at the same time.
Figure 6-1 illustrates what is possible and what is not with multiple relationship FlashCopy.
FlashCopy
source
target
...
maximum is 12
...
a source can have up to
12 targets
FlashCopy
source
FlashCopy
target
source
not allowed
a target can have only
one source
source and
target
target
not allowed
a volume can be only a source or
target at any given time
Figure 6-1 Multiple Relationship FlashCopy possibilities
Note: At any point-in-time, a volume or LUN can be only a source or a target.
6.2 Consistency Group FlashCopy
Applications might have their data spread over multiple volumes. In this case, if FlashCopy
needs to be used for multiple volumes, they all have to be at a consistent level. Consistency
groups can be used to help create a consistent Point-in-Time copy across multiple volumes,
52
IBM System Storage DS6000 Series: Copy Services in Open Environments
and even across multiple DS6000 storage systems, thus managing the consistency of
dependent writes.
With the DS CLI, you can establish a Consistency Group by using the freeze option and
identifying all FlashCopy pairs and target volumes belonging to a group.
With the Freeze FlashCopy Consistency Group option, the DS6000 holds off I/O activity to a
volume for a time period by putting the source volume in a queue full state. Therefore, a time
slot can be created during which dependent write updates do not occur, and FlashCopy uses
that time slot to obtain a consistent point-in-time copy of the related volumes. I/O activity
resumes when all FlashCopies are established.
Dependent writes occur if the start of one write operation is dependent upon the completion
of a previous write; in this case, the writes are dependent. Application examples for
dependent writes are databases with their associated logging files. For instance, the
database logging file is updated after a new entry has been successfully written to a table
space. The chronological order of dependent writes to the FlashCopy source volumes is the
basis to provide consistent data at the FlashCopy target volumes. For a more detailed
understanding of dependent writes and how the DS6000 enables the creation of Consistency
Groups, thus ensuring data integrity on the target volumes, you can refer to the discussions in
12.4.1, “Data consistency and dependent writes” on page 119.
6.3 FlashCopy target as a Metro Mirror or Global Copy primary
With this option, the FlashCopy target volume can also be a primary volume for a Metro
Mirror or Global Copy relationship. A user might wish to use this capability to create both a
remote copy and a local copy of a production volume.
Figure 6-2 illustrates this capability. In this figure, the FlashCopy target and the Metro Mirror
(or Global Copy) primary are the same volume. They are displayed as two separate volumes
for ease of understanding.
Local site
Remote site
target
source
FlashCopy
primary
Global
Copy
Metro
Mirror
secondary
Global
Copy
Metro
Mirror
Figure 6-2 FlashCopy target is Metro Mirror (or Global Copy) primary
It is possible to either create the FlashCopy relationship first or the Metro Mirror (or Global
Copy) relationship first. However, in general it is better to create the FlashCopy relationship
first to avoid the initial sending of unnecessary data across to the Metro Mirror (or Global
Copy) secondary.
Chapter 6. FlashCopy options
53
6.4 Incremental FlashCopy to refresh target volume
Incremental FlashCopy provides the capability to refresh a FlashCopy relationship, thus
refreshing the target volume.
Important: A refresh of the target volume always overwrites any writes previously written
to the target volume.
Restriction: If a FlashCopy source has multiple targets, an incremental FlashCopy
relationship can be established with one and only one target.
To perform an incremental FlashCopy, you must first establish the FlashCopy relationship with
the Start Change Recording and Persistent FlashCopy options enabled.
target volume
source volume
no updates in source
updates took place
in source volume
updates took place
in source volume
no updates in source
nothing to be
done as data
was already
copied from
source to target
and is identical
on both sides
no updates in target
no updates in target volume
current source
data will be
copied to target
updates took place
in target volume
updates took place
in target volume
Figure 6-3 Updates to the target volume caused by a refresh target FlashCopy
With Incremental FlashCopy, the initial FlashCopy, either copy or nocopy, relationship
between a source and target volume is subject to the following (see Figure 6-3):
򐂰 FlashCopy with nocopy option
If the original FlashCopy was established with the nocopy option, then the bitmap for the
target volume is reset, and of course, the updates on the target volumes are overwritten.
However, using the incremental option automatically converts this relationship to a copy
relationship, and the background copy begins.
򐂰 FlashCopy with copy option
If the original FlashCopy was established with the copy option (full volume copy), then the
updates that took place on the source volume since the last FlashCopy are copied to the
target volume. Also, the updates done on the target volume are overwritten with the
contents of the source volume.
When initializing a FlashCopy with Start Change Recording activated, a second and third
bitmap are used to identify writes done to the source or the target volume (see Figure 6-4 on
page 55). All three bitmaps are necessary for incremental FlashCopy:
54
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Target bitmap: This bitmap keeps track of tracks not yet copied from source to target.
򐂰 Source Change Recording bitmap: This bitmap keeps track of changes to the source.
򐂰 Target Change Recording bitmap: This bitmap keeps track of changes to the target.
These bitmaps allow subsequent FlashCopies to transmit only those blocks of data for which
updates occurred. Every write operation to the source or target volume is reflected in these
bitmaps by setting the corresponding bit to 0.
Reads and writes with Start Change Recording option set
tx
t0
read
ty
write y
tz
ta
read 1
write a
bitmap
0
0
1
0
0
1
write b
time-zero data
not yet available
in target :
read it from
source.
write z
write x
read 2 t b
0
0
1
source
0
0
tx
t0
time
write c
bitmap
target
data
data
t0
0
tc
tz
t0
ta
t0
t0
t
tb
t0
t
before physical write to the source:
copy time-zero data from the source to the target
Figure 6-4 FlashCopy with Start Change Recording set
When the Refresh takes place, the bitmap used for change recording is used to analyze
which blocks need to be copied from the source volume to the target volume; see Figure 6-5.
Refresh FlashCopy target volume
tx
t0
bitmap
0
0
ty
1
0
tz
0
ta
1
tb
0
1
0
0
t0
tz
data
t0
t0
t0
tx
t0
tz
t0
1
1
1
1
1
needs to be copied as a
write occured on the target
update to the source
needs to be copied
update to the source
needs to be copied
needs to be copied as a
write occured on the target
1
1
1
1
1
1
source
t0'
t0'
t0'
t0
bitmap
1
target
data
data
t0'
0
target
data
tx
time
bitmap
0
source
t0
t c t 0'
t0'
t0'
t 0'
t 0'
Figure 6-5 Refresh of the FlashCopy target volume
Chapter 6. FlashCopy options
55
After the refresh, which takes place only on the bitmap level, the new FlashCopy based on
time-0’ is active. The copy of the time-0’ data to the target is done in the background.
Tip: You can do the incremental copy at any time. You do not have to wait for the previous
background copy to complete.
6.5 Remote FlashCopy
With the DS CLI, you can use commands to manage a FlashCopy relationship at a remote
site. The commands can be issued from the local site, and then they are transmitted over the
Metro Mirror or Global Copy links. This eliminates the need for a network connection to the
remote site solely for the management of FlashCopy.
The FlashCopy source volume at the remote site must be the secondary volume of the Metro
Mirror or Global Copy pair.
Local site
Remote site
primary
Global
Copy
secondary
Metro
Mirror
Metro
Mirror
Global
Copy
source
target
FlashCopy
Figure 6-6 Remote FlashCopy
Figure 6-6 illustrates this capability. In this figure, the Metro Mirror (or Global Copy) secondary
and the FlashCopy source are the same volume; they are displayed as two separate volumes
for ease of understanding.
6.6 Persistent FlashCopy
With this option, the FlashCopy relationship continues until explicitly removed; this means
until the user terminates the relationship using one of the interface methods. If this option is
not selected, the FlashCopy relationship exists until all data has been copied from the source
volume to the target.
6.7 Reverse restore
With this option, the FlashCopy relationship can be reversed by copying over modified tracks
from the target volume to the source volume (see Figure 6-7 on page 57). The background
copy process must complete before you can reverse the order of the FlashCopy relationship
56
IBM System Storage DS6000 Series: Copy Services in Open Environments
to its original source and target relationship. Change recording is a prerequisite for reverse
restore.
target volume
will become
source volume
source volume
will become
target volume
no updates in source
updates took place
in source volume
updates took place
in source volume
no updates in source
nothing to be done
as data was already
copied from source
to target and is
identical on both
sides
data of previous
target (now source)
will be copied to
previous source
(now target)
no updates in target
no updates in target volume
updates took place
in target volume
updates took place
in target volume
Figure 6-7 Reverse restore
The source and target bitmaps (illustrated in Figure 6-4 on page 55) are exchanged and then
handled as described with the Incremental FlashCopy option.
6.8 Fast reverse restore
This option is used with Global Mirror. If you specify this option, you can reverse the
FlashCopy relationship without waiting for the completion of the background copy of the
previous FlashCopy.
6.9 Options and interfaces
Now that we have discussed the options available with FlashCopy, in this section we see how
the DS Command-Line Interface (DS CLI) and the DS Storage Manager (DS-SM) interfaces
support them; see Figure 6-8 on page 58.
Note that there are some options that cannot be invoked from the DS-SM. They are indicated
with a cross in Figure 6-8 on page 58.
Chapter 6. FlashCopy options
57
Interface
Function
DS front ends
DS SM
DS CLI
Multiple relationship FlashCopy
Consistency Group FlashCopy
Target on existing Metro Mirror or
Global Copy primary
Incremental FlashCopy
Remote FlashCopy
Persistent Flashcopy
Reverse restore,
fast reverse restore
Figure 6-8 FlashCopy options and interfaces
58
IBM System Storage DS6000 Series: Copy Services in Open Environments
7
Chapter 7.
FlashCopy ordering and
activation
This chapter explains how to order and activate FlashCopy for the IBM System Storage
DS6000.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
59
7.1 Ordering FlashCopy
FlashCopy is an optional feature of the IBM System Storage DS6000 series. This feature is
also referred to as the Point-In-Time Copy (PTC) licensed function.
Note: For a detailed explanation of the features involved and things you must consider
when ordering FlashCopy, we recommend you refer to the IBM System Storage DS6000
Series (IBM 1750-522) announcement letter.
You can read IBM announcement letters at:
http://www.ibm.com/products
All DS6000 series licensed functions are enabled, authorized, managed, activated, and
enforced based upon the physical capacity contained in a 1750 system.
Enablement refers to the purchase of a feature code to technically activate the associated
licensed function. Each licensed function is enabled for a 1750 system through acquiring the
corresponding feature codes.
Particularly for FlashCopy, the 1705 feature that must be ordered is the licensed function
Point-in-Time copy (PTC), feature #52xx (refer to Table 7-1).
Table 7-1 FlashCopy features
Feature
Description
5210
PTC - 1 TB
5211
PTC - 5 TB
5212
PTC - 10 TB
5213
PTC - 25 TB
5214
PTC - 50 TB
Each licensed function feature code purchased for a 1750 machine enables that function for
the entire 1750 system.
Authorization refers to the purchase of a feature code to establish the extent of IBM
authorization, which means license size in terms of physical capacity, for use of an
associated licensed function. This is also referred to as the authorization level.
The extent of IBM authorization for the use of a licensed function on a 1750 system is
established by acquiring the 52xx feature code on the 1750 base enclosure. The same 52xx
feature codes acquired to enable a licensed function also establish the extent of IBM
authorization.
The 52xx feature code on a 1750 base enclosure establishes the authorization level for the
entire 1750 system. For example, a 1750 system with a Model 522 base enclosure and two
Model EX2 expansion enclosures requires the acquisition of 52xx feature codes only on the
Model 522. Therefore, for licensed functions authorized on the basis of physical capacity, as
in the case of PTC, the authorization level established with the acquisition of the 52xx feature
code on the Model 522 must also cover the physical capacity of the attached Model EX2
expansion enclosures.
60
IBM System Storage DS6000 Series: Copy Services in Open Environments
7.2 Activating FlashCopy
IBM System Storage DS6000 licensed functions are managed using the Disk Storage
Feature Activation (DSFA) application. Also, the DS6000 licensed functions are activated by
the installation of feature activation codes into the 1750 system.
The feature activation codes are made available by IBM and are obtained using the DSFA
(Disk Storage Feature Activation) application at:
http://www.ibm.com/storage/dsfa
7.2.1 Management
Use the Disk Storage Feature Activation (DSFA) application to:
򐂰
򐂰
򐂰
򐂰
Assign an order confirmation code.
Select the license scope.
Assign the license value.
Deactivate a licensed function.
The previous activities are referred to as the “management” activities that you can perform
with the DSFA application and in relation to the DS6000 licensed functions.
Order confirmation code
With the purchase of the 52xx feature code, IBM provides an order confirmation code. To
activate the function in your machine, you must assign the order confirmation code to your
1750 system (using the 1750 base enclosure serial number). This activity is performed using
the DSFA application.
You will not be able to retrieve a feature activation code until you have assigned the order
confirmation code to the 1750 serial number for which it was acquired.
License scope
Licensed functions are activated and enforced within a defined license scope. License scope
refers to the type of storage, and therefore the type of servers, that the function can be used
with:
򐂰 Fixed Block (FB): The function can be used only with data from Fibre Channel-attached
servers.
򐂰 Count Key Data (CKD): The function can be used only with data from FICON-attached
servers.
򐂰 Both FB and CKD (ALL): The function can be used with data from all attached servers.
In particular, the license scope of FlashCopy is multiple: it can either be FB, CKD, or ALL. For
this reason when ordering FlashCopy, you must plan ahead to determine what your
point-in-time copy requirements will be. Example 7-1 on page 62 illustrates a sample case.
Chapter 7. FlashCopy ordering and activation
61
Example 7-1 FlashCopy example
A DS6000 has a total physical capacity of 15 TB and that capacity will be configured as:
10 TB open systems (FB)
5 TB zSeries (CKD)
Then, here's the required licenses:
Operating environment ==> 15 TB (equal to total machine capacity)
Parallel access volumes ==> 5 TB (equal to CKD-configured capacity)
Remote mirror for z/OS ==> 5TB (equal to CKD-configured capacity)
FlashCopy must either be:
10 TB if you want to use FlashCopy only with open systems (FB) data (equal to
FB-configured capacity)
5 TB if you want to use FlashCopy only with zSeries (CKD) data (equal to
CKD-configured capacity)
15 TB if you want to use FlashCopy with both open systems (FB) and zSeries (CKD) data
(equal to total machine capacity)
If a licensed function, which is the case with FlashCopy, has multiple license scope options,
you will be required to select a license scope during the initial retrieval of the feature
activation code.
Note: The license scope is not selected during the purchase of the 52xx feature code; the
feature codes only establish the extent of IBM authorization (in terms of physical capacity)
regardless of the storage type.
You can also change the license scope using the DSFA application after a licensed function
has been activated. A new feature activation code will be generated and when you install it in
the machine, the function will be activated and enforced using the newly selected license
scope. Only an increase in the license scope (changing FB or CKD to ALL) is a non-disruptive
activity. A lateral change (changing FB to CKD or changing CKD to FB) or a reduction of the
license scope (changing ALL to FB or CKD) is a disruptive activity and requires a machine
IML.
License value
Licensed functions are activated and enforced based upon an assigned license value.
License value refers to extent of IBM authorization and is referenced as either:
򐂰 Terabytes (TBs) of physical capacity for variable-sized licensed functions
򐂰 ON or OFF for system-level licensed functions
Feature activation codes will always be generated using the total license value for each
variable-sized licensed function and ON for each system-level licensed function, unless you
assign a license value of zero (0.0 TB) or OFF, respectively.
You can change the license value assignment using the DSFA application. You generally
only perform this activity when you want to deactivate a licensed function.
Deactivation
Assigning a license value of zero (0.0 TB) or OFF provides the ability to deactivate (disable) a
licensed function. A new feature activation code will be generated and when you install it in
the machine, the function will be deactivated during the next IML of the machine.
62
IBM System Storage DS6000 Series: Copy Services in Open Environments
You can also reactivate a deactivated licensed function by changing the license value to a
non-zero value. A new feature activation code will be generated and when you install it into
the machine, the function will be activated. Reactivation is a non-disruptive activity.
When you deactivate a licensed function using the DSFA application, the 5xxx feature codes
assigned to your machine (using the order confirmation code) remain assigned to the
machine. This means that you do not need to repurchase your 5xxx feature codes if you want
to reactivate the licensed function at a future date.
7.2.2 Activation
Activation refers to the client retrieval and installation of the feature activation code in the
1750 system.
The feature activation code is made available by IBM and obtained using the DSFA
application at:
http://www.ibm.com/storage/dsfa
Note: To access DSFA, you will be required to enter information about your 1750 machine.
You can find this information on the Storage Unit General Properties page in the DS6000
series Storage Manager application.
The following information is needed for the download of the keyfile and should be prepared
during planning:
򐂰 Model
The model of the DS6000 can be taken from the order.
򐂰 Serial number of the DS6000
The serial number of a DS6000 can be taken from the front of the base frame (lower right
corner). If several servers have been delivered, this is the only way to obtain the serial
number of a DS6000 located in a specific place in the computer center.
򐂰 Machine signature
The machine signature can only be obtained with the DS SM or the DS CLI after
installation of the DS6000. In a Web browser, type the URL of the DS SMC:
https://10.10.10.10:8452/DS6000
Select Real-time manager → Monitor system → Properties. Select General, then the
Storage Complex and the Storage Unit that you are interested in from the list. This shows
the properties of the selected DS6000, including the machine signature.
Order confirmation code
The order confirmation code is printed in the DS6000 series order confirmation code
document, which is sent to the client’s contact person either with the DS6000 or before
delivery.
Chapter 7. FlashCopy ordering and activation
63
64
IBM System Storage DS6000 Series: Copy Services in Open Environments
8
Chapter 8.
FlashCopy interfaces
You can set up FlashCopy in an open systems environment using different interfaces. This
chapter explains and gives examples of the interfaces that you can use for FlashCopy
management when FlashCopy is used with the IBM System Storage DS6000 in an open
systems environment.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
65
8.1 FlashCopy management interfaces: Overview
There are various interfaces available for the configuration and management of FlashCopy
when used in an open systems environment with the DS6000:
򐂰
򐂰
򐂰
򐂰
DS Command Line Interface (DS CLI)
DS Storage Manager (DS SM) Graphical User Interface (GUI)
TotalStorage Productivity Center for Replication (TPC for Replication)
DS Open Application Programming Interface (DS Open API)
The present chapter gives an overview of the DS CLI and the DS SM for FlashCopy
management. DS CLI and DS SM are also covered in Part 2, “Interfaces” on page 15.
TotalStorage Productivity Center (TPC) for Replication provides management of DS6000
series business continuance solutions, including FlashCopy, Metro Mirror, and Global Mirror.
TPC for Replication is covered in Chapter 28, “IBM TotalStorage Productivity Center for
Replication” on page 415. TPC for Replication includes similar functions to Global Mirror
Utility (GMU). GMU users should consider migrating to TPC for Replication.
Also, the DS Open Application Programming Interface (DS Open API), which is a set of
application programming interfaces that are available to integrate into programs, can be used
for FlashCopy management and control. The DS Open API is not covered in the present
book. For information about the DS Open API, refer to the publication IBM System Storage
DS Open Application Programming Interface Reference, GC35-0516.
FlashCopy control with the interfaces
Independently of the interface that is used, the following basic sequence takes place when
managing FlashCopy:
1. FlashCopy is initiated by means of an interface.
The initialization process of a FlashCopy takes only some seconds. At the end of this
process, FlashCopy is established based on the given parameters. This means that all the
necessary meta structures have been established. No data has yet been copied.
2. FlashCopy runs in the background.
This is the moment when FlashCopy copies in the background the necessary data to
create the point-in-time copy. The parameters given at initialization time define how
FlashCopy works in the background. They also define which subsequent activities can be
performed with this FlashCopy.
3. FlashCopy terminates.
FlashCopy either terminates automatically if all tracks have been copied, or needs to be
terminated explicitly by means of an interface if it has been defined as persistent.
8.2 DS CLI and DS SM: Commands and options
This section summarizes the commands and select options you can use when managing
FlashCopy at the local and at the remote site.
The DS CLI can be used to execute both local FlashCopy and remote FlashCopy commands.
The DS SM user interface and the DS Open API do not support remote FlashCopy.
66
IBM System Storage DS6000 Series: Copy Services in Open Environments
8.2.1 Local FlashCopy management
The commands you can use as well as the displayed panels, actions, and options you can
select when working with the DS6000-provided interfaces, DS CLI and DS SM, for local
FlashCopy management are listed in Table 8-1.
Local FlashCopy refers to the situation where the FlashCopy is local as opposed to a remote
FlashCopy. For a discussion of the characteristics of a remote FlashCopy, refer to 6.5
“Remote FlashCopy” on page 56.
Table 8-1 Local FlashCopy using DS CLI and DS SM
Options
Command with
DS CLI
Select option
with DS SM
mkflash
Create
Display a list of FlashCopy relationships.
lsflash
Main panel
Modify a FlashCopy pair that is part of a Global Mirror
relationship to revertible.
setflashrevertible
Restorable,
Record Change
Wizard
Commit data to the target volume.
commitflash
Commit changes
Increment an existing FlashCopy pair.
resyncflash
(prerequisites:
-record and
-persist)
Resynch target
Change the source-target relationship A → B to B → A.
reverseflash
Reverse
relationship
Reestablish contents of target B by contents of source A
as it was during last consistency formation.
revertflash
Consistency
Groups currently
not supported
Reset a FlashCopy Consistency Group.
unfreezeflash
Consistency
Groups currently
not supported
Run new background copy for persistent FlashCopy.
rmflash -cp
Background copy
rmflash
Delete
Create a FlashCopy
Create a local FlashCopy.
Work with an existing FlashCopy
Terminate FlashCopy
Remove local FlashCopy.
Is automatically removed as soon as all
data is copied and FlashCopy pair was
not established using -persist parameter
8.2.2 Remote FlashCopy management
The commands that you can use when working with the DS6000-provided interface DS CLI
for remote FlashCopy management are listed in Table 8-2 on page 68.
Chapter 8. FlashCopy interfaces
67
Table 8-2 Remote FlashCopy using DS CLI commands
Options
Command with DS CLI
Create a FlashCopy
Create a remote FlashCopy.
mkremoteflash
Work with an existing FlashCopy
Display a list of FlashCopy relationships.
lsremoteflash
Modify a FlashCopy pair that is part of a Global Mirror
relationship to revertible.
setremoteflashrevertible
Commit data to the target volume.
commitremoteflash
Increment an existing FlashCopy pair.
resyncremoteflash
(prerequisites: -record and -persist)
Change the source-target relationship A → B to B → A.
reverseremoteflash
Reestablish contents of target B by contents of source A
as it was during last consistency formation.
revertremoteflash
Reset a FlashCopy Consistency Group.
Not available as remote command
Run new background copy for persistent FlashCopy.
rmremoteflash -cp
Terminate FlashCopy
Remove local FlashCopy.
rmremoteflash
Is automatically removed as soon as all
data is copied and FlashCopy pair was
not established using the -persist
parameter
Note: Remote FlashCopy is not supported with the DS SM or DS Open API interfaces.
8.3 Local FlashCopy using the DS CLI
The DS CLI can be downloaded from the IBM Web site and then installed on a workstation. It
communicates with the DS SMC. For detailed information about the DS CLI, refer to the
publication IBM System Storage DS6000: Command-Line Interface User´s Guide,
GC26-7922.
8.3.1 Parameters used with local FlashCopy commands
This section discusses the parameters that can be passed to FlashCopy when using the DS
CLI and what the results are.
68
IBM System Storage DS6000 Series: Copy Services in Open Environments
mkflash
lsflash
Parameters
setflash
revertible
commit
flash
DS CLI Commands
resync
reverse
flash
flash
revert
flash
unfreeze
flash
rmflash
Source
freeze
x
x
Target
tgtpprc
tgtoffline
tgtinhibit
tgtonly
Flashcopy pair
dev
record
persist
nocp
seqnum
source:target
fast
source
cp
source LSS
l
s
activecp
revertible
Command
wait
quiet
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Figure 8-1 Overview of parameters used in DS CLI FlashCopy commands
Figure 8-1 summarizes the parameters and the corresponding DS CLI commands. When
FlashCopy receives these parameters, the following actions result:
򐂰 freeze: Consistency Group FlashCopy.
With the DS CLI, it is possible to establish a Consistency Group by using the -freeze
parameter and identifying all FlashCopy pairs and target volumes belonging to it.
򐂰 tgtpprc: Establish target on existing Metro Mirror source.
When this option is selected, the target volume can be or become a source volume for a
Metro Mirror or Global Copy relationship.
򐂰 tgtoffline: Permit FlashCopy to occur if target volume is not online for host access.
When setting up the relationship, a verification is done to see whether the target volume is
online or offline. This parameter only applies to CKD volumes. Table 8-3 summarizes the
actions that result depending on the target volume status.
Table 8-3 Resulting actions when tgtoffline parameter is used
Status of target
volume: CKD only
Permit FlashCopy to
occur if target
volume is online
Comments
Volume is online.
Not selected
FlashCopy definition is not possible.
Volume is online.
Selected
FlashCopy definition is queued until target
volume goes offline.
Volume is offline.
Selected or not
selected
FlashCopy is started immediately.
Chapter 8. FlashCopy interfaces
69
򐂰 tgtinhibit: Inhibit writes to target volume.
If FlashCopy is active, writes to the target volume are not allowed (inhibited).
򐂰 record: Change recording.
Activating the option change recording during setup of a FlashCopy enables subsequent
refreshes to the target volume. To do so, a second bitmap is created for the source
volume, which keeps track of all writes to the source. This bitmap can later be used to
refresh the target by only copying the updates from the source to the target.
򐂰 persist: Persistent FlashCopy.
The FlashCopy relationship continues to exist until explicitly removed by an interface
method. If this option is not selected, the FlashCopy relationship exists until all data has
been copied from the source volume to the target.
򐂰 nocp: Full volume background copy.
With the parameter nocp, it is possible to indicate whether the data of the source volume
is copied to the target volume in the background or not. If -nocp is not used, a copy of all
data from source to target takes place in the background. With -nocp selected, only
updates to the source volume cause writes to the target volume: this way the time-zero
data can be preserved.
򐂰 seqnum: Sequence number for FlashCopy pairs.
A number that identifies the FlashCopy relationship. Once used with the initial mkflash
command, it can be used within subsequent commands to refer to multiple FlashCopy
relationships.
򐂰 source:target: Identification of source volume and target volume.
򐂰 fast: Reverse FlashCopy before background copy is finished.
Allows you to issue the reverseflash command before the background copy is finished.
򐂰 cp: Restrict command to FlashCopy relationships with background copy.
򐂰 sourceLSS: Reset Consistency Group for source logical subsystems.
򐂰 s: Display of FlashCopy pairs with lsflash command.
The shortened output of the lsflash command returned. Only the FlashCopy pair IDs
display.
򐂰 l: Display additional FlashCopy information.
The standard output of the lsflash command is enhanced. The values for the copy
indicator, out-of-sync tracks, date created, and date synchronized all additionally display.
򐂰 activecp: Selection of FlashCopy pairs with active background copy.
򐂰 revertible: Selection of FlashCopy pairs with revertible attribute.
8.3.2 Local FlashCopy commands: Examples
This section explains the DS CLI commands that you can use to manage FlashCopy and also
shows examples of their use.
Initiate FlashCopy using mkflash
With mkflash, a local FlashCopy can be established. Four coding examples for mkflash are
shown in Example 8-1 on page 71.
70
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 8-1 mkflash command examples
Script
mkflash -dev IBM.1750-7506571
-seqnum 01 0001:0101
Date/Time: July 11, 2005 6:29:51 PM CEST IBM DSCLI Version: 5.0.3.134
CMUC00137I mkflash: FlashCopy pair 0001:0101 successfully created.
mkflash -dev IBM.1750-7506571 -record -seqnum 02 0002:0102
Date/Time: July 11, 2005 6:29:58 PM CEST IBM DSCLI Version: 5.0.3.134
CMUC00137I mkflash: FlashCopy pair 0002:0102 successfully created.
mkflash -dev IBM.1750-7506571 -persist -seqnum 03 0003:0103
Date/Time: July 11, 2005 6:30:02 PM CEST IBM DSCLI Version: 5.0.3.134
CMUC00137I mkflash: FlashCopy pair 0003:0103 successfully created.
mkflash -dev IBM.1750-7506571 -nocp
-seqnum 04 0004:0104
Date/Time: July 11, 2005 6:30:05 PM CEST IBM DSCLI Version: 5.0.3.134
CMUC00137I mkflash: FlashCopy pair 0005:0105 successfully created.
DS: IBM.1750-7506571
DS: IBM.1750-7506571
DS: IBM.1750-7506571
DS: IBM.1750-7506571
Listing of the properties of the FlashCopies
lsflash -dev IBM.1750-7506571 0001-0004
Date/Time: July 11, 2005 6:30:09 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-7506571
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Disabled Disabled
Disabled
Enabled
Enabled
Enabled
0003:0103 00
2
300
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
0004:0104 00
3
300
Disabled
Disabled Enabled
Disabled
Enabled
Enabled
Enabled
0005:0105 00
4
300
Disabled
Disabled Disabled
Disabled
Enabled
Enabled
Disabled
The following explanations apply to the cases presented in Example 8-1:
򐂰 Example 1: 0001 → 0101
The FlashCopy between volume 0001 and volume 0101 is established using the default
parameters. By default, the following properties are enabled: SourceWriteEnabled,
TargetWriteEnabled, and BackgroundCopy (default if not specified differently using the
-nocp parameter). All other properties are disabled. The background copy takes place
immediately and once everything has been copied, the FlashCopy relationship is
automatically removed.
򐂰 Example 2: 0002 → 0102
The FlashCopy between volume 0002 and volume 0102 is established with the following
FlashCopy properties enabled: Recording, Persistent, and BackgroundCopy.
Note: The parameter -persist is automatically added whenever -record is used.
The background copy takes place immediately and the relationship remains as a
persistent relationship. By using other DS CLI commands, it can be reversed and
re-synchronized.
򐂰 Example 3: 0003 → 0103
The FlashCopy between volume 0003 and volume 0103 is established with the following
FlashCopy properties enabled: Persistent and BackgroundCopy. The background copy
takes place immediately. Once the background copy has finished, the FlashCopy
relationship remains because of the persistent flag.
򐂰 Example 4: 0004 → 0104
The FlashCopy between volume 0004 and volume 0104 is established with the -nocp
parameter. This means that no full volume background copy is done, only the data
changed in the source is copied to the target prior to changing it. Over time, this can result
in a situation where all data is copied to the target, and then, the FlashCopy relationship
ends. The FlashCopy relationship also ends after a background copy is initiated using the
Chapter 8. FlashCopy interfaces
71
DS SM. This way the relationship is temporarily persistent even though the property
Persistent is not activated.
Display existing FlashCopy relationships using lsflash
The command lsflash can be used to display FlashCopy relationships and its properties.
Parameters can be used with this command to identify the subset of FlashCopy relationships
to be displayed.
Example 8-2 shows a script with several lsflash commands and the output of the script (this
script is logically based on the example for mkflash).
Example 8-2 lsflash command examples
#--- Example 1
lsflash -dev IBM.1750-13ABC2A 0004
Date/Time: July 8, 2005 3:00:49 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0004:0104 00
4
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Disabled
#--- Example 2
lsflash -dev IBM.1750-13ABC2A 0000-0004
Date/Time: July 8, 2005 3:01:02 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0000:0100 00
0
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
0001:0101 00
1
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
0002:0102 00
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0003:0103 00
3
300
Disabled
Disabled Enabled
Disabled
Disabled
Disabled
Enabled
0004:0104 00
4
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Disabled
#--- Example 3
lsflash -dev IBM.1750-13ABC2A -l 0000-0004
Date/Time: July 8, 2005 3:01:17 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
OutOfSyncTracks DateCreated
DateSynced
==========================================================================================================================================
======================================================================
0000:0100 00
0
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
122051
Fri Jul 08 11:02:22 CEST 2005 Fri Jul 08 11:02:22 CEST 2005
0001:0101 00
1
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
150255
Fri Jul 08 11:02:35 CEST 2005 Fri Jul 08 11:02:35 CEST 2005
0002:0102 00
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
150255
Fri Jul 08 11:02:48 CEST 2005 Fri Jul 08 11:02:48 CEST 2005
0003:0103 00
3
300
Disabled
Disabled Enabled
Disabled
Disabled
Disabled
Enabled
150255
Fri Jul 08 11:03:04 CEST 2005 Fri Jul 08 11:03:04 CEST 2005
0004:0104 00
4
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Disabled
150255
Fri Jul 08 11:03:17 CEST 2005 Fri Jul 08 11:03:17 CEST 2005
#--- Example 4
lsflash -dev IBM.1750-13ABC2A -s 0000-0001
Date/Time: July 8, 2005 3:01:36 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
=========
0000:0100
0001:0101
#--- Example 5
lsflash -dev IBM.1750-13ABC2A -activecp 0000-0004
Date/Time: July 8, 2005 3:01:48 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMMCI9006E No FlashCopy instances named 0000-0004 found that match criteria: dev = IBM.1750-13ABC2A and activecp = true.
#--- Example 6
lsflash -dev IBM.1750-13ABC2A -record 0000-0004
Date/Time: July 8, 2005 3:02:05 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0002:0102 00
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
72
IBM System Storage DS6000 Series: Copy Services in Open Environments
#--- Example 7
lsflash -dev IBM.1750-13ABC2A -persist 0000-0004
Date/Time: July 8, 2005 3:02:17 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0002:0102 00
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0003:0103 00
3
300
Disabled
Disabled Enabled
Disabled
Disabled
Disabled
Enabled
#--- Example 8
lsflash -dev IBM.1750-13ABC2A -revertible 0000-0004
Date/Time: July 8, 2005 3:02:29 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
#--- Example 9
lsflash -dev IBM.1750-13ABC2A -cp 0000-0004
Date/Time: July 8, 2005 3:02:45 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0000:0100 00
0
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
0001:0101 00
1
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
0002:0102 00
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0003:0103 00
3
300
Disabled
Disabled Enabled
Disabled
Disabled
Disabled
Enabled
The following explanations apply to the cases presented in Example 8-2 on page 72:
򐂰 Example 1: List FlashCopy information for a specific volume.
In this example, the lsflash command shows the FlashCopy relationship information for
volume 0004 and shows the status (enabled or disabled) of the FlashCopy properties.
򐂰 Example 2: List existing FlashCopy relationship information within a range of volumes.
In this example, the lsflash command shows the FlashCopy relationship information for
the range of volumes 0000 to 0004 and shows the properties status (enabled or disabled).
򐂰 Example 3: List existing FlashCopy relationships with full information.
Using the -l parameter with the lsflash command displays the default output plus
information about the following properties: OutOfSyncTracks, DateCreated, and
DateSynced.
򐂰 Example 4: List IDs of existing FlashCopy pairs within a volume range.
Using the -s parameter displays only the FlashCopy source and target volume IDs for the
specified range of volumes.
򐂰 Example 5: List FlashCopy relationships with active background copy running.
Using the parameter -activecp displays only those FlashCopy relationships within the
selected range of volumes for which a background copy is actively running. The output
format is the default output. In our example, there were no active background copies.
򐂰 Example 6: List existing FlashCopy relationships with -record enabled.
Using the -record parameter displays only those FlashCopy relationships within the
selected range of volumes that were established with the -record parameter.
򐂰 Example 7: List existing FlashCopy relationships with the Persistent attribute enabled.
When using the parameter -persist, only those FlashCopy relationships within the range
of selected volumes for which the Persistent option is enabled display.
򐂰 Example 8: List existing FlashCopy relationships which are revertible.
When using the parameter -revertible, only those FlashCopy relationships within the
range of selected volumes for which the option Revertible is enabled display. There were
no revertible relationships in our example.
򐂰 Example 9: List existing FlashCopy relationships for which BackgroundCopy is enabled.
Chapter 8. FlashCopy interfaces
73
When using the parameter -cp, only those FlashCopy relationships within the range of
selected volumes for which the BackgroundCopy option is enabled display.
Set an existing FlashCopy to revertible using setflashrevertible
The command setflashrevertible can be used to modify the revertible attribute of a
FlashCopy relationship that is part of a Global Mirror relationship.
The FlashCopy properties Recording and Persistent must be enabled to set a FlashCopy
relationship to revertible using this command. This command needs to be executed prior to
running a commitflash or revertflash.
Example 8-3 illustrates two examples of situations that can occur when using the
setflashrevertible command.
Example 8-3 setflashrevertible command examples
#-----------------------------------------------------------#--- script to set FlashCopy property Revertible to value enabled and display values afterwards
#-----------------------------------------------------------#--- Example 1
setflashrevertible -dev IBM.1750-13ABC2A 0002:0102
Date/Time: July 8, 2005 6:52:49 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00167I setflashrevertible: FlashCopy volume pair 0002:0102 successfully made revertible.
lsflash -dev IBM.1750-13ABC2A 0000-0004
Date/Time: July 8, 2005 7:15:30 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0002:0102 00
0
300
Disabled
Enabled
Enabled
Enabled
Enabled
Disabled
Enabled
#--- Example 2
setflashrevertible -dev IBM.1750-13ABC2A 0003:0103
Date/Time: July 8, 2005 6:53:16 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUN03027E setflashrevertible: 0003:0103: FlashCopy operation failure: action prohibited by current FlashCopy state
The following explanations apply to the cases presented in Example 8-3:
򐂰 Example 1: Set the FlashCopy relationship to revertible.
This command sets the existing FlashCopy for source volume 0002 and target volume
0102 to revertible. Once the property Revertible is enabled, any subsequent commands
result in an error message similar to the one displayed in Example 2.
򐂰 Example 2: Error when trying to set FlashCopy relationship to revertible.
When trying to set a FlashCopy relationship to revertible for which the property Recording
is disabled, an error results. The script ends after this command with return code 2 and
any other commands following the one that causes the error are not executed.
Commit data to target using commitflash
Use the command commitflash to commit data to a target volume to set consistency between
source and target. The command commitflash is intended to be used in asynchronous
remote copy environments such as Global Mirror. Therefore, the usage of the command
commitflash is discussed in greater detail in Part 6, “Global Mirror” on page 241 while this
section discusses the basic usage of the command.
Before the FlashCopy relationship can be committed, it needs to be made revertible. Typically,
this is done automatically by an application such as Global Mirror. However, it can also be set
manually.
74
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 8-4 Commit command examples
#--- Example 1
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 10, 2005 2:45:28 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0005:0105 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
setflashrevertible -dev IBM.1750-13ABC2A
Date/Time: July 10, 2005 2:45:40 PM CEST
CMUC00167I setflashrevertible: FlashCopy
CMUC00167I setflashrevertible: FlashCopy
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 10, 2005 2:46:07 PM CEST
-seqnum 01 0001:0101 0005:0105
IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
volume pair 0001:0101 successfully made revertible.
volume pair 0005:0105 successfully made revertible.
IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Enabled
Enabled
Enabled
Enabled
Disabled
Enabled
0005:0105 00
1
300
Disabled
Enabled
Enabled
Enabled
Enabled
Disabled
Enabled
commitflash -dev IBM.1750-13ABC2A 0001-0005
Date/Time: July 10, 2005 2:46:18 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00170I commitflash: FlashCopy volume pair 0001:0001 successfully committed.
CMUC00170I commitflash: FlashCopy volume pair 0005:0005 successfully committed.
lsflash -dev IBM.1750-13ABC2A -l 0000-0005
Date/Time: July 10, 2005 2:46:47 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Enabled
Disabled
0005:0105 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Enabled
Disabled
#--- Example 2
commitflash -dev IBM.1750-13ABC2A 0003:0103
CMUN03027E commitflash: 0003:0003: FlashCopy operation failure: action prohibited by current FlashCopy state
The following explanations apply to the cases presented in Example 8-4:
򐂰 Example 1: Commit the FlashCopy relationship.
This example shows the properties of the two FlashCopy relationships 0001:0101 and
0005:0105 using the lsflash command prior and after issuing the setflashrevertible
command. After the commitflash command is executed, the properties of the two
FlashCopy relationships are listed again.
򐂰 Example 2: Error when trying to commit a FlashCopy relationship.
When trying to commit a FlashCopy relationship that is not revertible (property Revertible
is disabled), an error is the result. The script ends after this command with return code 2
and any other commands following the one that causes the error are not executed.
Increment FlashCopy using resyncflash
You can increment an existing FlashCopy relationship with the resyncflash command. To
run this command, the FlashCopy relationship must have the options Recording and
Persistent enabled.
Tip: You do not have to wait for the background copy to complete before you do the FlashCopy re-synchronization. You can use the resyncflash command at any time.
To make sure an existing FlashCopy relationship can be incremented multiple times, it is
necessary to repeat the -record and -persist parameters with the resyncflash command.
Example 8-5 on page 76 shows examples where the resyncflash command is used.
Chapter 8. FlashCopy interfaces
75
Example 8-5 resyncflash command examples
#--- Example 1
mkflash -dev IBM.1750-13ABC2A -record -persist -seqnum 01 0001:0101 0005:0105
Date/Time: July 10, 2005 5:04:48 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00137I mkflash: FlashCopy pair 0001:0101 successfully created.
CMUC00137I mkflash: FlashCopy pair 0005:0105 successfully created.
resyncflash -dev IBM.1750-13ABC2A -seqnum 13 0003:0103
Date/Time: July 10, 2005 5:05:18 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00137I mkflash: FlashCopy pair 0003:0103 successfully created.
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 10, 2005 5:05:45 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0003:0103 00
3
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0005:0105 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
resyncflash -dev IBM.1750-13ABC2A -record -persist -seqnum 11 0001:0101 0005:0105
Date/Time: July 10, 2005 5:05:58 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00168I resyncflash: FlashCopy volume pair 0001:0101 successfully resynchronized.
CMUC00168I resyncflash: FlashCopy volume pair 0005:0105 successfully resynchronized.
resyncflash -dev IBM.1750-13ABC2A seqnum 13 0003:0103
Date/Time: July 10, 2005 5:06:25 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00168I resyncflash: FlashCopy volume pair 0003:0103 successfully resynchronized.
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 10, 2005 5:06:39 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
11
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0003:0103 00
13
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
0005:0105 00
11
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
#--- Example 2
mkflash -dev IBM.1750-13ABC2A -seqnum 03 0004:0104
Date/Time: July 10, 2005 5:05:32 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00137I mkflash: FlashCopy pair 0004:0104 successfully created.
lsflash -dev IBM.1750-13ABC2A 0004
Date/Time: July 10, 2005 5:06:39 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0004:0104 00
4
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Enabled
resyncflash -dev IBM.1750-13ABC2A -record -persist -seqnum 14 0004:0104
Date/Time: July 10, 2005 5:06:51 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUN03027E resyncflash: 0004:0104: FlashCopy operation failure: action prohibited by current FlashCopy state
The following explanations apply to the examples shown in Example 8-5:
򐂰 Example 1: Increment a FlashCopy relationship.
In this example, three FlashCopy relationships are created with the -record and -persist
parameters. The resyncflash commands are executed using a different sequence
number, which overwrites the one of the current FlashCopy relationship. The sequence
number only changes if the resyncflash finishes successfully.
The resyncflash for the 0001:0101 and 0005:0105 relationships takes place using the
-record and -persist parameter. Because the two parameters are omitted for the
0003:0103 FlashCopy relationship, the two properties Recording and Persistent change to
disabled for this FlashCopy relationship. As soon as the background copy for the
0003:0103 FlashCopy relationship finishes, then the FlashCopy relationship terminates.
򐂰 Example 2: Error when trying to increment FlashCopy relationship.
When trying to increment a FlashCopy relationship for which the properties Recording and
Persistent are disabled, an error is the result. The script ends after this command with
76
IBM System Storage DS6000 Series: Copy Services in Open Environments
return code 2 and any other commands following the one that caused the error are not
executed.
Reverse source-target relationship using reverseflash
You can use the command reverseflash to change the direction of a FlashCopy relationship.
The former source becomes target and the former target becomes source. The data is copied
from the target to the source.
Example 8-6 illustrates examples of the use of the reverseflash command.
Example 8-6 reverseflash command examples
#--- Example 1
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 11, 2005 5:08:49 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0002:0102 00
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0003:0103 00
3
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0005:0105 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
reverseflash -dev IBM.1750-13ABC2A -record -persist 0001:0101 0005:0105
Date/Time: July 11, 2005 5:09:14 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00169I reverseflash: FlashCopy volume pair 0001:0101 successfully reversed.
CMUC00169I reverseflash: FlashCopy volume pair 0005:0105 successfully reversed.
reverseflash -dev IBM.1750-13ABC2A -record 0002:0102
Date/Time: July 11, 2005 5:09:28 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00169I reverseflash: FlashCopy volume pair 0002:0102 successfully reversed.
reverseflash -dev IBM.1750-13ABC2A 0003:0103
Date/Time: July 11, 2005 5:09:42 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00169I reverseflash: FlashCopy volume pair 0003:0103 successfully reversed.
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 11, 2005 5:09:55 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0101:0001 01
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0102:0002 01
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0105:0005 01
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
#--- Example 2
reverseflash -dev IBM.1750-13ABC2A -record 0002:0102
Date/Time: July 11, 2005 6:16:12 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00169I reverseflash: FlashCopy volume pair 0002:0102 successfully reversed.
lsflash -dev IBM.1750-13ABC2A 0002
Date/Time: July 11, 2005 6:16:27 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0102:0002 01
2
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
#--- Example 3
reverseflash -dev IBM.1750-13ABC2A -record 0102:0002
Date/Time: July 11, 2005 6:28:32 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00169I reverseflash: FlashCopy volume pair 0102:0002 successfully reversed.
lsflash -dev IBM.1750-13ABC2A 0002
Date/Time: July 11, 2005 6:28:46 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0002:0102 00
12
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
Chapter 8. FlashCopy interfaces
77
The following explanations apply to the examples shown in Example 8-6 on page 77:
򐂰 Example 1: Reverse a FlashCopy relationship.
In this example, three FlashCopy relationships are created with the -record and -persist
parameters. The reverseflash commands are executed using a different sequence
number, which overwrites the one of the current FlashCopy relationship.
The reverseflash for the 0001:0101 and 0005:0105 relationships takes place using the
-record and -persist parameter. Because the two parameters are omitted for the
0003:0103 FlashCopy relationship, the two properties Recording and Persistent change to
disabled for this FlashCopy relationship. This terminates the 0003:0103 FlashCopy
relationship as soon it is successfully reversed.
򐂰 Example 2: Reverse a FlashCopy relationship multiple times.
It is possible to reverse a FlashCopy relationship multiple times, thus, recopying the
contents of the original FlashCopy target volume multiple times back to the original source
volume. In this example, the 0002:0102 was reversed once as part of example 1. Then
changes are done to data residing on volume 0002. A subsequent reverseflash for
0002:0102 eliminates the changes done to 0002 and brings back the data from volume
0102 to volume 0002 as it was during the initial FlashCopy.
򐂰 Example 3: Reestablish original FlashCopy direction reversing again.
It is possible to reverse a FlashCopy relationship back again. In example 3, we show this
for the reversed FlashCopy relationship 0102:0002. Reversing it a second time and
referring to it as FlashCopy pair 0102:0002 is similar to establishing a new FlashCopy for
the volume pair 0002:0102. In this case, a sequence number provided with the
reverseflash is used to identify a new FlashCopy relationship.
Reset target to contents of last consistency point using revertflash
You can use the command revertflash to reset the target volume to the contents of the last
consistency point. Like the commitflash command, this command is intended to be used in
asynchronous environments, such as Global Mirror environments. Before this command can
be issued, the relationship must be made revertible, either automatically as with Global
Mirror, or manually using the setrevertible command. See Example 8-7.
Example 8-7 revertflash command example
#--- Example 1
mkflash -dev IBM.1750-13ABC2A -record -persist -seqnum 01 0001:0101
Date/Time: July 11, 2005 9:44:27 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00137I mkflash: FlashCopy pair 0001:0101 successfully created.
mkflash -dev IBM.1750-13ABC2A -nocp -seqnum 04 0001:0104 0001:0105
Date/Time: July 11, 2005 9:44:42 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00137I mkflash: FlashCopy pair 0001:0104 successfully created.
CMUC00137I mkflash: FlashCopy pair 0001:0105 successfully created.
lsflash -dev IBM.1750-13ABC2A 0000-0005
Date/Time: July 11, 2005 9:44:56 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
0001:0101 00
1
300
Disabled
Enabled
Enabled
Disabled
Disabled
Disabled
Enabled
0001:0104 00
4
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Disabled
0001:0105 00
4
300
Disabled
Disabled Disabled
Disabled
Disabled
Disabled
Disabled
setflashrevertible -dev IBM.1750-13ABC2A 0001:0101
Date/Time: July 11, 2005 10:14:33 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00167I setflashrevertible: FlashCopy volume pair 0001:0101 successfully made revertible.
revertflash -dev IBM.1750-13ABC2A 0001
Date/Time: July 11, 2005 10:14:47 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00171I revertflash: FlashCopy volume pair 0001:0001 successfully reverted.
78
IBM System Storage DS6000 Series: Copy Services in Open Environments
In Example 8-7 on page 78, three FlashCopy relationships are created for one source
volume: 0001:0101, 0001:0104 and 0105. The revertflash command is executed for source
0001, and because the FlashCopy relationship 0001:0101 has the Recording and Persistent
property enabled, this command refers to this FlashCopy relationship 0001:0101. Any
updates done to volume 0101 are overwritten.
Run new background copy for persistent FlashCopy using rmflash
Additional background copies for persistent FlashCopy relationships can be created using the
rmflash command in combination with its -cp parameter. See Example 8-8.
Example 8-8 rmflash command to create a new background copy
#--- Example 1
rmflash -dev IBM.1750-13ABC2A -quiet -cp 0001:0101
Date/Time: July 11, 2005 11:21:05 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00143I rmflash: Background copy process for FlashCopy pair 0001:0101 successfully started. The persistent relationship will not be
removed.
In the example presented in Example 8-8, rmflash command to create new background copy
of a persistent FlashCopy relationship, the existing FlashCopy relationship 0001:0101 is used
to create a new background copy.
Remove local FlashCopy using rmflash
You can use the command rmflash to remove a FlashCopy relationship. Unlike other
commands, it does not return an error if the request runs for a FlashCopy relationship that
does not exist.
In scripts, the command rmflash should always be used with the -quiet parameter to avoid
the confirmation prompt. See Example 8-9.
Example 8-9 rmflash command example
#--- Example 1
rmflash -dev IBM.1750-13ABC2A -quiet 0001:0101
Date/Time: July 11, 2005 11:21:39 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A
CMUC00140I rmflash: FlashCopy pair 0001:0101 successfully removed.
In Example 8-9, the existing FlashCopy relationship 0001:0101 is removed.
8.3.3 FlashCopy Consistency Groups
Typically, large applications such as databases have their data spread across several
volumes. In order to create a consistent copy or backup, a FlashCopy of all the volumes
should be done at exactly the same point-in-time. FlashCopy Consistency Groups are
designed to achieve this exact purpose.
Create FlashCopy Consistency Groups using mkflash
Use the mkflash command with the -freeze parameter to create a FlashCopy Consistency
Group. This command causes the DS6000 to briefly prevent I/O to the volumes in the
Consistency Group. During this time, any I/O that comes from the host is returned with a
SCSI queue full error, which the host bus adapter automatically retries. However, if the I/O is
frozen for an extended period of time, there are application errors on the host. For this
reason, the Consistency Group should be unfrozen as quickly as possible.
Example 8-10 on page 80 illustrates the creation of a FlashCopy Consistency Group that
contains two FlashCopy pairs.
Chapter 8. FlashCopy interfaces
79
Example 8-10 Creating FlashCopy Consistency Groups
dscli> mkflash -dev IBM.1750-7506571 -freeze 1500-1501:1502-1503
Date/Time: October 24, 2005 3:37:49 AM PDT IBM DSCLI Version: 5.0.5.6 DS: IBM.1750-7506571
CMUC00137I mkflash: FlashCopy pair 1500:1502 successfully created.
CMUC00137I mkflash: FlashCopy pair 1501:1503 successfully created.
Reset FlashCopy Consistency Group using unfreezeflash
Use the command unfreezeflash to remove a Consistency Group for multiple volumes, for
which FlashCopy relationships were established using the -freeze parameter. This
command removes the queue full condition and allows I/Os to continue on the source
volumes. See Example 8-11.
The unfreezeflash command is issued to the entire logical subsystem (LSS).
Example 8-11 unfreezeflash command example
#--- Example 1
unfreezeflash -dev IBM.1750-7506571/00
Date/Time: July 12, 2005 12:27:06 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-7506571
CMUC00172I unfreezeflash: FlashCopy consistency group for logical subsystem 00: successfully reset.
8.4 Remote FlashCopy using the DS CLI
Remote FlashCopy commands are similar to local FlashCopy commands. The remote
commands can be issued whenever DS6000 mirroring takes place from one DS6000 to
another DS6000. In this situation, the Fibre Channel links between the two DS6000s that are
used for mirroring purposes are also used to transmit the FlashCopy commands to the
remote DS6000. For detailed information about the DS CLI, refer to the publication IBM
System Storage DS6000: Command-Line Interface User´s Guide, GC26-7922.
8.4.1 Remote FlashCopy commands
The syntax of the remote FlashCopy commands is similar to the syntax of the local
FlashCopy commands. The commands themselves have almost similar names except for the
“remote” character string that you can see in the remote FlashCopy commands’ names.
Table 8-4 on page 81 shows the similarity between both sets of commands. Their actions are
also similar.
80
IBM System Storage DS6000 Series: Copy Services in Open Environments
Table 8-4 Command names for local and remote FlashCopy
Local command
Remote command
Comments
mkflash
mkremoteflash
Establish FlashCopy
lsflash
lsremoteflash
List FlashCopy
setflashrevertible
setremoteflashrevertible
Set FlashCopy to revertible
commitflash
commitremoteflash
Commit FlashCopy on target
resyncflash
resyncremoteflash
Increment FlashCopy
reverseflash
reverseremoteflash
Switch source-target
revertflash
revertremoteflash
Reset to last consistency point
unfreezeflash
no equivalent command
Reset Consistency Group
rmflash
rmremoteflash
Remove FlashCopy
8.4.2 Parameters used in remote FlashCopy commands
Figure 8-2 shows the parameters used in remote FlashCopy DS CLI commands.
mkremote
flash
lsremote
flash
Parameters
setremote
flash
revertible
DS CLI Commands
commit
resync
reverse
remote
remote
remote
flash
flash
flash
revert
remote
flash
rmflash
Source
freeze
x
x
Target
tgtpprc
tgtoffline
tgtinhibit
tgtonly
Flashcopy pair
dev
record
persist
nocp
seqnum
source:target
fast
source
cp
source LSS
l
s
activecp
revertible
Command
conduit
quiet
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Figure 8-2 DS CLI remote FlashCopy commands and parameters
Most of the parameters for remote FlashCopy commands are similar to those for local
FlashCopy commands. Two major exceptions between local FlashCopy and remote
FlashCopy commands are:
Chapter 8. FlashCopy interfaces
81
򐂰 Each remote command has the parameter conduit to identify the link to pass the
command to the secondary DS6000.
򐂰 The local FlashCopy command unfreezeflash has no remote equivalent.
Figure 8-2 on page 81 summarizes the parameters and the corresponding DS CLI
commands that can be used when doing remote FlashCopy.
The description of the parameters is similar to the description presented in 8.3.1 “Parameters
used with local FlashCopy commands” on page 68. In regard to the parameter conduit that
only applies to remote FlashCopy, the following explanation applies:
Conduit:
Within a remote mirror environment, the FlashCopy commands are
sent across the mirror paths, thus avoiding the necessity of having
separate network connections to the remote site solely for the
management of the remote FlashCopy. The -conduit parameter
identifies the path to be used for transmitting the commands to the
remote site.
8.5 FlashCopy management using the DS SM
To use the DS SM front-end GUI, a supported Web Browser needs to be installed on the
workstation. The DS SM communicates with the DS SMC.
To start using the DS SM, enter in your Web browser the IP address of the DS SMC as
shown in Example 8-12. The password is set up by the administrator and needs to be
changed during the first use of the DS SM.
Example 8-12 Starting DS SM
http://<ip-address:>:8451/DS6000
or
https://<ip-address>:8452/DS6000
8.5.1 Initiate FlashCopy using Create
Figure 8-3 on page 83 show the FlashCopy Create with DS SM window.
82
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 8-3 FlashCopy Create with DS SM
After you log in to the DS SM, follow this sequence (refer to Figure 8-3):
1. Select Real-time manager.
2. Select Copy services.
3. Select FlashCopy.
4. Use the Storage Complex and Storage Unit pull-down windows to identify the DS6800 for
which you would like to initiate a FlashCopy. Then, use the Resource type and Specify
LSS pull-downs to display the volumes you want to work with on that DS6800.
5. From the Select action pull-down, select Create.
The previous selection gives you several windows, one after the other, to define the attributes
that you use with this FlashCopy:
1. Define relationship type.
Select either A single source with a single target, or if you want to allow a Multiple
Relationship FlashCopy, select A single source with multiple targets.
2. Select source volumes.
Select one or multiple source volumes by checking the box at the left of the volume
identification. For each of the source volumes, you are afterwards presented with a
window to select one or more targets.
3. Select target volumes.
Select the target volumes (a source cannot have more than 12 target volumes).
4. Select common options.
Figure 8-4 on page 84 shows the options that you can choose. The data provided in this
window is used for all defined FlashCopy pairs.
Chapter 8. FlashCopy interfaces
83
Figure 8-4 Common option window for FlashCopy
5. Verification.
In the verification window (not shown), all information regarding the FlashCopy definition
is displayed. If it is not possible to establish the FlashCopy relationship (if, for example, a
volume was requested to be offline, but is not offline), then an informational message is
displayed on this window.
Initial FlashCopy parameter comparison using DS SM and the DS CLI
A FlashCopy can be defined by using the DS SM with a browser (selecting the Create for
FlashCopy) or by using the DS CLI (using the mkflash command).
Using the DS SM, it is possible to define FlashCopies that are generated immediately. It is
currently not possible to store any FlashCopy tasks using the DS SM.
Table 8-5 on page 85 shows the corresponding parameters for the various FlashCopy options
when using the DS CLI command mkflash and the DS SM FlashCopy Create selection.
84
IBM System Storage DS6000 Series: Copy Services in Open Environments
Table 8-5 Comparison of options and parameters used for FlashCopy in DS CLI and DS SM
Options
Parameter
with DS CLI
command
mkflash
Parameter with
DS SM FlashCopy create
Comments
Multiple relationship FlashCopy
list of
source:target
volumes
Multiple target volumes can
be selected during create.
Consistency Groups for
FlashCopy
freeze
N/A
FlashCopy target can be Metro
Mirror or Global Copy primary
tgtpprc
Establish target on existing
Metro Mirror source.
Permit target volume to be online
in z/OS environment
tgtoffline
Permit FlashCopy to occur if
target is online for host
access.
This option exists only for CKD
volumes (z/OS)
Inhibit writes to target volume
tgtinhibit
N/A
Resync target
Identification of FlashCopy pair
dev or
source:target
Selected in windows.
Change Recording
record
Enable change recording.
Persistent FlashCopy
persist
Make relationship persistent.
Full volume background
FlashCopy
nocp
Initiate background copy.
Sequence number
seqnum
Sequence number for this
relationship.
Options for the source volume
Currently not supported with DS
SM front end
Options for the target volume
Options for FlashCopy pair
8.5.2 Display properties of existing FlashCopy
To display the properties of existing FlashCopy relationships, log in to the DS SM user
interface window and follow this sequence (refer to Figure 8-5 on page 86):
1. Select Real-time manager.
2. Select Copy services.
3. Select FlashCopy.
4. Use the Storage Complex and Storage Unit pull-down windows to identify the DS6800 for
which you would like to initiate a FlashCopy. Then, use the Resource type and Specify
LSS pull-downs to display the volumes you want to work with on that DS6800.
This gives you a list of all active FlashCopy relationships (see Figure 8-5 on page 86). By
checking the box to the left of the FlashCopy of interest, the Select Actions for this FlashCopy
relationship are changed based on the attributes of the FlashCopy relationship and the
possible actions to perform on it. Using Select Action → Properties, all attributes of the
selected FlashCopy are displayed.
Chapter 8. FlashCopy interfaces
85
Note: In case you are selecting multiple FlashCopy relationships, Select Action →
Properties is not presented. Select only one FlashCopy relationship to view its properties.
Figure 8-5 FlashCopy display properties
This selection gives you a window with two folders:
򐂰 General
In this folder, all properties of the selected FlashCopy are presented (See Figure 8-6).
Figure 8-6 General folder with FlashCopy properties information
86
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Out-of-synch tracks
The window displaying the out-of-synch tracks can be used to monitor how the FlashCopy
performs in the background. A refresh interval can be set to refresh the display after a
preselected period of time. See Figure 8-7.
Figure 8-7 Display out-of-synch tracks
Properties display: DS CLI versus DS SM
Table 8-6 compares the differences in the way that the FlashCopy information is displayed
when requested either using the DS CLI command lsflash or an existing FlashCopy
relationship was selected for display using the DS SM front end.
Table 8-6 FlashCopy properties as displayed by the DS CLI vs. the DS SM
Properties with DS CLI command lsflash
Properties with DS SM FlashCopy property
Property
Contents
Property
Contents
#
Source LSS in
selection window
From selection list
Source LSS
SrcLSS
Sequence Number to which the FlashCopy belongs. Can also be used for Consistency Groups.
SequenceNum
###
Sequence number
from overview window
###
Shows if a Background Copy is currently running. If the DS CLI returns ActiveCopy=disabled this
could be similar to out-of-synch tracks>0 or copy complete on the DS SM.
ActiveCopy
Enabled or disabled
Status in overview
window
Copy complete or
out-of-synch tracks>0
or background copy
running
Shows if recording was selected for both source and target volume to allow for later usage of resync.
Recording
Enabled or disabled
Change recording
Yes or no
Relationship will
remain
Yes or no
Shows if the FlashCopy is a persistent one.
Persistent
Enabled or disabled
Chapter 8. FlashCopy interfaces
87
Properties with DS CLI command lsflash
Properties with DS SM FlashCopy property
Identifies if the FlashCopy relationship can be changed by copying the contents of the target volume
to the source volume
Revertible
Enabled or disabled
Restorable
Yes or no
Source is
write-inhibited
Yes or no
Identifies if the source can be used to write on it
SourceWriteEnabled
enabled or disabled
Identifies it the target can be used by another server to write on it in parallel to the FlashCopy taking
place
TargetWriteEnabled
Enabled or disabled
Target is
write-inhibited
Yes or no
Background copy
initiated
Yes or no
Identifies if a background copy was already done
BackgroundCopy
Enabled or disabled
using -l parameter with DS CLI
OutOfSyncTracks
A number
Out-of-sync tracks
A number
DateCreated
Date
Created
Date
DateSynced
Date
Last refresh
Date
8.5.3 Reverse existing FlashCopy
To reverse an existing FlashCopy relationship, you first display the list of active FlashCopy
relationships as described in 8.5.2 “Display properties of existing FlashCopy” on page 85.
Then, by checking the box to the left of the FlashCopy of your interest, the available Select
Actions for this FlashCopy relationship is shown. If the existing FlashCopy relationship can
be reversed, it is possible to choose the Select Action and then select Reverse
relationship. Then, the Reverse FlashCopy panel is displayed; see Figure 8-8.
Figure 8-8 Parameters or copy options to use with reverse action
88
IBM System Storage DS6000 Series: Copy Services in Open Environments
The Reverse FlashCopy panel displays those properties of the existing FlashCopy that might
be changed during the reverse process. Changing the values of the parameters and then
clicking OK starts the reverse process for the FlashCopy.
8.5.4 Initiate background copy for a persistent FlashCopy relationship
To initiate a background copy for a persistent FlashCopy relationship, start displaying the list
of active FlashCopy relationships as described in 8.5.2 “Display properties of existing
FlashCopy” on page 85. Then, check the box to the left of the FlashCopy of your interest; the
available Select Actions for this FlashCopy relationship display. Then, choose Select
Action → Initiate background copy (see Figure 8-9).
Figure 8-9 Initiate background copy
The next window that is presented is shown in Figure 8-10. It prompts for the FlashCopy pairs
for which the background copy should run.
Figure 8-10 Prompt window for background copy
Chapter 8. FlashCopy interfaces
89
8.5.5 Re-synchronize target
To re-synchronize a target volume start by displaying the list of active FlashCopy
relationships as shown in Figure 8-11. Then, check the box to the left of the FlashCopy you
want to re-synchronize. After doing so, the available Select Actions for this FlashCopy
relationship are shown. Then, select Resync target.
Figure 8-11 Resync target select option to re-synchronize FlashCopy relationship
The prompt window asks for more details for the resync request (see Figure 8-12).
Figure 8-12 Prompt window to detail resync request for FlashCopy relationship
90
IBM System Storage DS6000 Series: Copy Services in Open Environments
8.5.6 Delete existing FlashCopy relationship
To delete an existing FlashCopy relationship, start by displaying the list of active FlashCopy
relationships that is described in 8.5.2 “Display properties of existing FlashCopy” on page 85.
Then, check the box to the left of the FlashCopy relationship you want to terminate. After
doing so, the available Select Actions for this FlashCopy relationship are shown. Then,
select Delete (see Figure 8-13).
Figure 8-13 Delete select option to delete FlashCopy relationship
The next window (Figure 8-14) is a prompt asking you to confirm the delete request (see
Figure 8-14).
Figure 8-14 Prompt window to confirm delete request for FlashCopy relationship
Chapter 8. FlashCopy interfaces
91
92
IBM System Storage DS6000 Series: Copy Services in Open Environments
9
Chapter 9.
FlashCopy performance
This chapter discusses best practices when configuring FlashCopy for specific environments
or scenarios. The following topics are covered:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
FlashCopy performance overview
FlashCopy establish performance
Background copy performance
FlashCopy impact to applications
FlashCopy options
FlashCopy scenarios
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
93
9.1 FlashCopy performance overview
Many parameters can affect the performance of FlashCopy operations. It is important to
review the data processing requirements and hence select the proper FlashCopy options.
This chapter examines when to use copy versus no copy and where to place the FlashCopy
source and target volumes or LUNs. We also discuss when and how to use incremental
FlashCopy, which you should definitely evaluate for use in most applications.
Note: This chapter is equally valid for System z volumes and Open Systems LUNs. In the
following sections of this chapter, only the term volume or volumes is used, but the text is
equally valid if the terms LUN and LUNs were used, unless otherwise noted.
Terminology
Before proceeding with the discussion about FlashCopy best practices, let us review some of
the basic terminology we use in this chapter.
Server
A DS6000 has two servers: Server 0 and Server 1, one on each
controller card. Each server independently provides major functions
for the disk subsystem: directing host adapters for data transferring to
and from host processors, managing cache resources, and directing
lower device adapters for data transferring to and from physical disks.
You can issue the lsserver command to see the available servers.
Device Adapter (DA) A physical component of the DS6000 that provides communications
between the servers and the storage devices. The lsda command lists
the available device adapters.
Rank
An array site made into an array, which is then made into a rank. For
the DS6000, a rank is a collection of eight disk drive modules (DDMs).
The lsrank command displays detailed information about the ranks.
9.1.1 Distribution of the workload: Source and target volume location
In general, you can achieve the best performance by distributing the load across all of the
resources of the DS6000. In other words, you should carefully plan your usage so that the
load is:
򐂰
򐂰
򐂰
򐂰
Spread evenly across disk subsystems
Within each disk subsystem, spread evenly across servers
Within each server, spread evenly across device adapters
Within each device adapter, spread evenly across ranks
Additionally, it is always best to locate the FlashCopy target volume on the same DS6000
server as the FlashCopy source volume. It is also good practice to locate the FlashCopy
target volume on a different device adapter (DA) than the source volume, but there are some
cases where this decision really does not matter.
Another choice available is whether to place the FlashCopy target volumes on the same ranks
as the FlashCopy source volumes. In general, it is best not to place these two volumes in the
same rank. Refer to Table 9-1 on page 95 for a summary of volume placement
considerations.
94
IBM System Storage DS6000 Series: Copy Services in Open Environments
Tip: If the FlashCopy target volume is on the same rank as the FlashCopy source volume,
then the user runs the risk of a rank failure, causing the loss of both the source and target
volumes.
Table 9-1
FlashCopy source and target volume location
Server
Device adapter
Rank
FlashCopy establish
performance
Same server
Does not matter
Different ranks
Background copy
performance
Same server
Different device adapter
Different ranks
FlashCopy impact to
applications
Same server
Does not matter
Different ranks
Tip: To find the relative location of your volumes, you can use the following procedure:
1. Use the lsfbvol command to find out which Extent Pool contains the relevant volumes.
2. Use the lsrank command to display both the device adapter and the rank for each
Extent Pool.
3. To determine which server contains your volumes, look at the Extent Pool name.
Even-numbered Extent Pools are always from Server 0, while odd-numbered Extent
Pools are always from Server 1.
9.1.2 LSS/LCU versus rank considerations
In the DS6000, it is much more meaningful to discuss volume location in terms of ranks and
not in terms of logical subsystem (LSS) or logical control unit (LCU). On the ESS 800 and
earlier IBM disk subsystems, the physical locations of the volumes were described in terms of
the logical subsystem LSS/LCU. If more than a single rank existed in the LSS/LCU, then each
of the ranks had a range of volumes from that specific LSS/LCU.
The LSSs/LCUs in a DS6000 disk subsystem are logical constructs that are no longer tied to
predetermined ranks. Within the DS6000, the LSS/LCU can be configured to span one or
more ranks but is not limited to specific ranks. There can be individual ranks that contain
volumes from more than a single LSS/LCU, which was not possible before the introduction of
the DS6000.
9.1.3 Rank geometry
Finally, you can achieve a small performance improvement by using identical rank
geometries for both the source and target volumes. In other words, if the source volumes are
located on a rank with a 7+p/RAID-5 configuration, then the target volumes should also be
located on a rank configured as 7+p/RAID-5.
9.1.4 Incremental FlashCopy
This chapter focuses on FlashCopy performance best practices, but there are many other
business requirements that must be weighed when using FlashCopy. The designer should
carefully consider all aspects for the implementation of each specific solution and should
definitely evaluate the use of incremental FlashCopy for all FlashCopy applications.
Chapter 9. FlashCopy performance
95
9.2 FlashCopy establish phase performance
The FlashCopy of a volume has two distinct periods:
򐂰 The initial logical FlashCopy (also called establish)
򐂰 The physical FlashCopy (also called background copy)
The FlashCopy establish phase, or logical FlashCopy, is the period of time when the
microcode is preparing things, such as the bitmaps, necessary to create the FlashCopy
relationship so the microcode can properly process subsequent reads and writes to the
related volumes. During this logical FlashCopy period, no writes are allowed to the volumes.
After the logical relationship has been established, normal I/O activity is allowed to both
source and target volumes according to the options selected.
When there are a large number of volumes, the method used to invoke the FlashCopy
commands can influence the time that it takes to complete the logical FlashCopy for all
FlashCopy pairs. An in-band invocation sources the commands to the microcode faster than
an out-of-band method in most cases. An in-band establish is one that uses the CCW
commands directly, such as in z/OS.
The fastest out-of-band method is the DS CLI. The method to control or invoke FlashCopy
should be selected based upon the total solution requirements and not strictly on logical
FlashCopy performance considerations unless this is a critical performance issue. For open
systems, the DS CLI is the fastest invocation method.
Additionally, there is a modest performance impact to logical FlashCopy performance when
using incremental FlashCopy. In the case of incremental FlashCopy, the DS6000 must create
additional metadata (bitmaps). However, the impact is negligible in most cases.
Finally, the placement of the FlashCopy source and target volumes has an effect on the
establish performance. You can refer to the previous section for a discussion about this topic
as well as to Table 9-1 on page 95 for a summary of the recommendations.
9.3 Background copy phase performance
The background copy phase, or physical FlashCopy, is the actual movement of the data from
the source volume to the target volume. If the FlashCopy relationship was established
requesting the NOCOPY option, then only writing updates to the source volume forces a copy
from the source to the target. This forced copy is also called a collision.
Note: The term collision describes a forced copy from the source to the target, because a
write to the source has occurred. This occurs on the first write to a track. Note that
because the DS6000 writes to non-volatile cache, there is typically no direct response time
delay on host writes. The forced copy only occurs when the write is destaged onto disk.
If the copy option was selected, then upon completion of the logical FlashCopy establish
phase, the source is copied to the target in an expedient manner.
If a large number of volumes have been established, then do not expect to see all pairs
actively copying data as soon as their logical FlashCopy relationship is completed. The
DS6000 microcode has algorithms that limit the number of active pairs copying data. This
algorithm tries to balance active copy pairs across the DS6000 device adapter resources.
Additionally, the algorithm limits the number of active pairs so that bandwidth remains for host
or server I/Os.
96
IBM System Storage DS6000 Series: Copy Services in Open Environments
Tip: The DS6000 gives higher priority to application performance than background copy
performance. This means that the DS6000 throttles the background copy if necessary, so
that applications are not unduly impacted.
The recommended placement of the FlashCopy source and target volumes, regarding the
physical FlashCopy phase, was already discussed in the previous section. Refer to Table 9-1
on page 95 for a summary of the conclusions. For the best background copy performance, the
implementation should always place the source and target volumes in different ranks. There
are additional criteria to consider if the FlashCopy is a full box copy that involves all ranks.
Note: The term full box copy implies that all rank resources are involved in the copy
process. Either all or nearly all ranks have both source and target volumes, or half the
ranks have source volumes and half the ranks have target volumes.
For full box copies, you should still place the source and target volumes in different ranks.
When all ranks are participating in the FlashCopy, it is still possible to accomplish this by
doing a FlashCopy of volumes on rank R0 onto rank R1, and volumes on rank R1 onto rank
R0 (for example). Additionally, if there is heavy application activity in the source rank,
performance is less affected if the background copy target was in some other rank that can
be expected to have lighter application activity.
If background copy performance is of high importance in your environment, you should use
incremental FlashCopy as much as possible. Incremental FlashCopy greatly reduces the
amount of data that needs to be copied, and therefore, greatly reduces the background copy
time.
9.4 FlashCopy impact to applications
The most important consideration when implementing FlashCopy is to achieve an
implementation that has minimal impact to the users’ application performance.
Note: As already mentioned, the recommendations discussed in this chapter only consider
the performance aspects of a FlashCopy implementation. But FlashCopy performance is
only one aspect of an intelligent system design. You must consider all of the business
requirements when designing a total solution. These additional requirements, together with
the performance considerations, guide you when choosing FlashCopy options, such as
copy or no copy and incremental, as well as when making choices on source and target
volume location.
The relative placement of the source and target volumes has a significant impact on
application performance, and we have already discussed this in 9.1.1, “Distribution of the
workload: Source and target volume location” on page 94.
In addition to the relative placement of volumes, the selection of copy or no copy is also an
important consideration in regard to impact to application performance. Typically, the choice
of copy or no copy depends primarily on how the FlashCopy is to be used and for what
interval of time the FlashCopy relationship exists. From a purely performance point of view,
the choice of whether to use copy or no copy depends a great deal on the type of workload.
The general answer is to use no copy, but this is not always the best choice. For most
workloads, including online transaction processing (OLTP) workloads, no copy typically is
the preferred option. However, some workloads that contain a large number of random writes
Chapter 9. FlashCopy performance
97
and are not cache friendly might benefit from using the copy option, which is typically the
preferred option.
Another important performance consideration is whether to use incremental FlashCopy.
Under the correct circumstances, incremental FlashCopy should have no impact to the critical
online write updates and should significantly shorten the period of time needed to complete
the periodic background copy.
Note: The incremental FlashCopy resyncflash command does not have a -nocp option.
Using resyncflash automatically uses the copy option, regardless of whether the original
FlashCopy was copy or no copy.
9.5 FlashCopy options: Considerations
Incremental FlashCopy is only supported at the volume level. Though incremental can be
used with FlashCopy relationships established with either the copy or the no copy option,
there are some differences in how the copy and no copy relationships affect the application
write updates.
The copy approach causes an initial copy of all source volumes, but then only copies
changed tracks when the incremental resync command is invoked. The no copy approach
does not perform an initial copy of the source volumes when the incremental resync
command is invoked, but does copy on all first updates to source tracks since the preceding
FlashCopy. Again, this copy only occurs when that track is destaged, so in many cases, there
is no impact to application performance.
The designer needs to consider the consequences of bursts of background copy, initially and
following each resync (copy option), which might impact the application, versus continuous
impact, when there are collisions that require a source to target copy before the write update
can complete (no copy option).
If running incremental FlashCopy for an extended period of time, the copy option could very
well be the proper choice.
9.6 FlashCopy scenarios
This section describes four scenarios. These scenario discussions assume the primary
concern is to minimize the FlashCopy impact to application performance.
9.6.1 Scenario #1: Backup to disk
In environments where Recovery Time Objective (that is, how quickly you can return to
production after a failure) is of utmost importance, a FlashCopy backup to disk can help to
achieve an extremely fast restore time. As soon as the logical FlashCopy is complete, it is
possible to perform a reverse FlashCopy and restore your production data in seconds,
instead of the several hours it normally takes to retrieve the data from tape.
When backing up to disk, it is important to take the necessary steps to protect your data.
Remember that, until the background copy is complete, you still only have one physical copy
of the data, and that copy is vulnerable. Therefore, it is important to always establish the
FlashCopy with the copy option. Otherwise, in the unlikely event that you have a failure of
your production volumes, you also lose your backup.
98
IBM System Storage DS6000 Series: Copy Services in Open Environments
One method for protecting against failure is to use multiple FlashCopy targets. FlashCopy
supports up to 12 targets per source volume. With this feature, it is possible to keep up to 12
versions of your production data (for example, a FlashCopy backup every two hours for one
day). Another method is to use incremental FlashCopy. Incremental FlashCopy copies only
the data that has changed since the last FlashCopy, so the background copy completes much
faster.
9.6.2 Scenario #2: Backup to tape
If you want to create a copy of the data only to subsequently back up that data to tape, then
FlashCopy with the no copy option is the preferred approach. Still, there are some
implementations where the copy option is employed.
The backup to tape is normally done shortly after the logical FlashCopy relationships have
been established for all the volumes that are going to be backed up to tape.
If you choose the no copy option, that is probably because the data getting backed up to tape
is mostly coming from the FlashCopy source volumes. If this is the case, then the location of
the target volumes is less critical and might be decided by considerations other than
performance.
If you choose the copy option, that is probably because the data getting backed up is coming
from the target volumes, assuming that the backup to tape does not start until the background
copy completes. If the backup starts sooner, the data could be coming from a mixture of
source volumes and target volumes. As the backup continues, more and more of the data
comes from the target volumes as the background copy moves more and more of the data to
the target volumes.
To have the least impact on the application and to have a fast backup to tape, we recommend
that the source volumes are evenly spread across the available disk subsystems and the disk
subsystem resources. Once the backup to tape is complete, make sure to withdraw the
FlashCopy relationship.
Tip: Withdraw the pairs as soon as the backup to tape finishes. This eliminates any
additional copying from the source volume, either due to copy or collisions.
These recommendations are equally valid for a copy or no copy environment.
9.6.3 Scenario #3: FlashCopy during peak application activity
Tip: The recommended solution is to fully explore alternatives that allow no overlapping of
FlashCopy activity with other peak application activity. If such alternatives are not viable for
whatever operative reasons, then consider the topics we discuss in the present section.
As discussed previously, the choice of whether to use copy or no copy depends mostly on
business requirements, but with regard to performance, the choice also depends a great deal
on the type of workload. We discussed this topic in 9.4, “FlashCopy impact to applications” on
page 97.
Chapter 9. FlashCopy performance
99
Still, in general, no copy is the preferred method, but you should also think about the following
considerations when choosing either copy or no copy:
򐂰 Using no copy. The argument here is that the impact caused by the I/O resulting from the
copy option is more significant than that of the no copy option, where less I/O activity
resulting from collisions occurs. Still, because the background copy only occurs when the
writes are destaged from non-volatile cache, there is typically negligible impact. If the
workload is cache friendly, then potentially all of the operations are served from cache,
and there is no impact from collisions at all.
򐂰 Using copy. The goal of using copy is to quickly complete the background copy and,
hence, the overlapping situations between FlashCopy and application processing end
sooner. If copy is used, then all I/Os experience some degradation as they compete for
resources with the background copy activity. However, this impact can be somewhat less
than the impact to the individual writes that a collision causes.
If FlashCopy no copy is active during a period of application high activity, there could be a
high rate of collisions, that is, de-stages being delayed so that the track image can be read
and then written to the FlashCopy target track to preserve the point-in-time copy. The
de-stage delay could cause degradation of the performance for all writes that occur during
the delay de-stage periods. Note that it is only the first write to a track that causes a
collision, and only when that write gets destaged. The reads do not suffer the collision
degradation.
If using the copy option, consider also these tips:
򐂰 Examine the application environment for the highest activity volumes and the most
performance sensitive volumes.
򐂰 Consider arranging the FlashCopy order so that the highest activity and most performance
sensitive volumes are copied early and the least active and least performance sensitive
volumes are copied last.
The intent here is minimize the potential for collisions.
Tip: One approach to achieve a specified FlashCopy order is to partition the volumes into
priority groups. Issue the appropriate FlashCopy commands for all volumes, but use copy
on only the highest priority group and no copy on all other groups. After a specified period
of time or after some observable event, issue FlashCopy commands to the next highest
priority group from no copy to copy. Continue in this manner until all volumes are fully
copied.
If a background copy is the desired end result and FlashCopy is to start just before or during
a high activity period, consider the possibility of starting with no copy and converting to copy
after the high activity period has completed.
You might also want to examine the use of incremental FlashCopy in a high performance
sensitive activity period. Incremental FlashCopy automatically uses the copy option, so if the
no copy option was previously selected, using incremental FlashCopy might impact
performance by causing a full background copy.
If the incremental FlashCopy approach is chosen, it might be best to create a FlashCopy copy
relationship during a quiet time. To minimize the amount of data to be copied when taking the
desired point-in-time copy, schedule an incremental refresh sufficiently in advance of the
point-in-time refresh to complete the copy of the changed data. Finally, take the required
point-in-time copy with the incremental refresh at the required point-in-time.
100
IBM System Storage DS6000 Series: Copy Services in Open Environments
9.6.4 Scenario #4: Ranks reserved for FlashCopy
Another configuration worth considering is where 50% of the ranks (capacity) are all
FlashCopy source volumes, and where the application write I/Os take place, and 50% of the
ranks (capacity) are all FlashCopy target volumes.
This approach does have advantages and disadvantages. The disadvantage is the loss of
50% of the ranks for normal application processing. The advantage is that FlashCopy writes
to target volumes do not compete against applications writing to the target volumes. This
allows the background copy to complete faster, and, thus, reduces the interference with
application I/Os.
Decide whether to use this trade-off based on:
򐂰 Use all ranks for your application:
– Maximize normal application performance.
– FlashCopy performance is reduced.
򐂰 Use only half of the ranks for your applications:
– Maximize FlashCopy performance.
– Normal performance is reduced.
If planning a FlashCopy implementation at the disaster recovery (DR) site, consider two
distinct environments:
򐂰 DR mirroring performance with and without FlashCopy active.
򐂰 Application performance if DR Failoveroccurs.
The solution should provide acceptable performance for both environments.
Chapter 9. FlashCopy performance
101
102
IBM System Storage DS6000 Series: Copy Services in Open Environments
10
Chapter 10.
FlashCopy examples
This chapter presents examples of the use of FlashCopy in the following scenarios:
򐂰 Fast setup of test systems or integration systems
򐂰 Fast creation of volume copies for backup purposes
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
103
10.1 Create test system or integration system
Test systems or integration systems are needed to perform application tests or system
integration tests.
Because it is assumed that with test systems or integration systems, many write operations
occur over the time period involved, we recommend doing a copy in the background
environment.
10.1.1 One time test system
Assume there is an application using one volume, and a test system needs to be created to
allow application tests or integration tests based on the contents of the production data. A
FlashCopy is set up to copy the data once. See Example 10-1.
Example 10-1 Create a one time test system
#--- remove existing FlashCopy relationships for volume 6100
rmflash -dev IBM.1750-7506551 -quiet 6100:6300
#--- establish FlashCopy relationships for source volume 6100
mkflash -dev IBM.1750-7506551 -seqnum 01 6100:6300
#--- list FlashCopy relationships for volume 6100
lsflash -dev IBM.1750-7506551 -l 6100
The application typically needs to be quiesced or briefly suspended before executing the
FlashCopy. Also, some applications cache their data, so this data might need to be flushed to
disk, using application methods, prior to running the FlashCopy, which is not covered in our
example.
10.1.2 Multiple setups of a test system with the same content
Assume that an application test is needed multiple times with the same setup of data. The
production volume is 6100, the test volume is 6300. Volume 6101 is chosen as an
intermediate volume that gets its data one time, copying it from the production volume using
FlashCopy. Then, volume 6101 is used as the base for refresh of the test volume 6300.
Running Part 1 and Part 2 (see Example 10-2) as one job does not work. The job fails,
because volume 6101 cannot be the target for one FlashCopy relationship and the source for
another FlashCopy relationship at the same time. You must wait until the background copy
for 6100 and 6101 finishes successfully before starting Part 2.
Alternatively, you can also issue the rmflash command with the -cp and -wait parameters.
These parameters cause the command to wait until the background copy is complete before
continuing with the next step.
Example 10-2 Create a one time test system
Part 1
#=== Part 1:
#--- remove,
rmflash -dev
rmflash -dev
establish FlashCopy relationship
establish, list FlashCopy relationships
IBM.1750-7506551 -quiet 6100:6101
IBM.1750-7506551 -quiet 6101:6300
mkflash -dev IBM.1750-7506551 -seqnum 01 6100:6101
lsflash -dev IBM.1750-7506551 -l 6100-6400
104
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 2
#=== Part 2: establish FlashCopy 2 relationship
#=== 03:00 pm 6100 → 6300
#--- after the full volume copy of 6100 → 6101 finished
#--- establish relationship from 6101:6300
mkflash -dev IBM.1750-7506551 -seqnum 02 6101:6300
lsflash -dev IBM.1750-7506551 -l 6100-6300
Whenever the test environment needs to be reset to the original data, just run Part 2 of the
scripts or use the DS SM front end to perform a FlashCopy.
10.2 Create backup
Using FlashCopy for backup purposes can be implemented in several manners, which we
discuss in this section.
10.2.1 Create a FlashCopy for backup purposes without volume copy
Volumes that are the result of a FlashCopy can be used by a backup server to back up the
data to tape. Because the backup process merely reads the data, one option is to perform a
FlashCopy without physically copying all data to the target. As soon as the backup of the data
finishes, the FlashCopy relationship can be removed explicitly.
The following are the steps involved in this procedure (refer Example 10-3 and
Example 10-4):
1. Part 1: Establish FlashCopy volume A → volume B with no copy option.
2. Run backup.
3. Part 2: Remove FlashCopy relationship when volume backup completes.
Example 10-3 Create a backup copy
Part 1
#=== Part 1: establish FlashCopy relationship
#--- remove existing FlashCopy relationships for volume 6100
rmflash -dev IBM.1750-7506551 -quiet 6100:6300
#--- establish FlashCopy relationships for source volume 6100
mkflash -dev IBM.1750-7506551 -nocp -seqnum 01 6100:6300
#--- list FlashCopy relationships for volume 6100
lsflash -dev IBM.1750-7506551 -l 6100
After the backup has been taken, remove the FlashCopy relationship if you do not intend to
use it for other purposes. Thus, you can avoid unnecessary writes. See Example 10-4.
Example 10-4 Create a backup copy
Part 2
#=== Part 2: remove FlashCopy relationships
rmflash -dev IBM.1750-7506551 -quiet 6100:6300
Doing backup based on the target volume allows you to use a target multiple times.
Chapter 10. FlashCopy examples
105
In complex application environments such as SAP, FlashCopy is often used as part of the
backup solution. Good examples for backup solutions are the IBM Tivoli® Storage Manager
for Hardware modules, which integrate into the IBM Tivoli Storage Manager backup
infrastructure.
10.2.2 Incremental FlashCopy for backup purposes
To have the safety of a real physical copy without always copying the full volume can be
achieved by using the incremental FlashCopy. An initial full volume FlashCopy is followed by
subsequent incremental FlashCopies, which only copy the updates that took place on the
source volume. See Example 10-5.
Example 10-5 Create an initial FlashCopy ready for subsequent incremental FlashCopies
Part 1
#=== Part 1: establish FlashCopy relationship
#--- remove existing FlashCopy relationships for volume 6100
rmflash -dev IBM.1750-7506551 -quiet 6100:6300
#--- establish FlashCopy relationships
mkflash -dev IBM.1750-7506551 -record -persist -seqnum 01 6100:6300
#--- list FlashCopy relationships for volume 6100
lsflash -dev IBM.1750-7506551 -l 6100
After the initial full volume copy, the following script supports the incremental copy of the
FlashCopy relationship. See Example 10-6.
Example 10-6 Create an Incremental FlashCopy
Part 2
#=== Part 2: resynch FlashCopy relationship
resyncflash -dev IBM.1750-7506551 -record -persist -seqnum 01 6100:6300
lsflash -dev IBM.1750-7506551 -l 6100
10.2.3 Using a target volume to restore its contents back to the source
Logs might need to be applied to the target, and then the target volume is reversed to the
source volume.
For each source volume, one FlashCopy can exist with the -record and -persist attributes
set. Use this volume to refresh the source volume. To reverse the relationship, the data must
have been copied completely to the target before reversing it back to the source. To avoid a
situation where the full volume needs to be copied with each FlashCopy, the Incremental
FlashCopy should be used. Write-enable the target volume, because logs might need to be
applied to the target volume prior to reversing it.
This example consists of the following steps:
򐂰 Part 1: Establish initial FlashCopy (see Example 10-7 on page 107).
򐂰 Part 2: Establish Incremental FlashCopy (see Example 10-8 on page 107).
򐂰 Part 3: Reverse the relationship (see Example 10-9 on page 107).
Applying application or DB logs needs to be carefully considered as well.
106
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 10-7 Run initial FlashCopy to support refresh of source volume
Part 1
#=== Part 1: Establish incremental FlashCopy #--- remove, establish, list FlashCopy
relationships
rmflash -dev IBM.1750-7506551 -quiet 6100:6101
mkflash -dev IBM.1750-7506551 -persist -record -tgtinhibit -seqnum 01 6100:6101
lsflash -dev IBM.1750-7506551 -l 6100
After the initial FlashCopy, the incremental copies can be done (See Example 10-8).
Example 10-8 Create an Incremental FlashCopy
Part 2
#=== Part 2: Resynch FlashCopy relationship
resyncflash -dev IBM.1750-7506551 -record -persist -tgtinhibit -seqnum 01 6100:6101
lsflash -dev IBM.1750-7506551 -l 6100
The reverse of the FlashCopy is done using the reverseflash command (See Example 10-9).
Example 10-9 Reverse the volumes
Part 3
#=== Part 3: Reverse FlashCopy relationship
reverseflash -dev IBM.1750-7506551 -persist -record -tgtinhibit -seqnum 01 6100:6101
lsflash -dev IBM.1750-7506551 -l 6100
Chapter 10. FlashCopy examples
107
108
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 4
Part
4
Metro Mirror
This part of the book describes the IBM System Storage Metro Mirror for DS6000 when used
in an open systems environments. We discuss the characteristics of Metro Mirror and
describe the options for its setup. We also show which management interfaces can be used
as well as the important aspects to consider when establishing a Metro Mirror environment.
This part ends with examples of Metro Mirror management and setup.
Note: Throughout the chapters of this part of the book, in our discussions of Metro Mirror in
open systems environments, you see that the term volume is used interchangeably with
the term LUN. In fact, the term volume is almost always used.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
109
110
IBM System Storage DS6000 Series: Copy Services in Open Environments
11
Chapter 11.
Metro Mirror overview
This chapter explains the basic characteristics of Metro Mirror for DS6000 when used in open
systems environments.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
111
11.1 Metro Mirror overview
Metro Mirror, previously known as synchronous Peer-to-Peer Remote Copy (PPRC),
provides real-time mirroring of logical volumes between two DS6000s that can be located up
to 300 km from each other. It is a synchronous copy solution where write operations are
completed on both copies (local and remote site) before they are considered complete.
It is typically used for applications that cannot suffer any data loss in the event of a failure.
As data is synchronously transferred, the distance between local and remote disk
subsystems determines the effect on application response time. Figure 11-1 illustrates the
sequence of a write update with Metro Mirror.
4
Write acknowledge
Server
write
1
LUN or
volume
Primary
(source)
Write to secondary
2
3
Write complete
acknowledgement
LUN or
volume
Secondary
(target)
Figure 11-1 Metro Mirror
When the application performs a write update operation to a source volume, this is what
happens:
1.
2.
3.
4.
Write to source volume (DS6000 cache and NVS).
Write to target volume (DS6000 cache and NVS).
Signal write complete from the remote target DS6000.
Post I/O complete to host server.
The Fibre Channel connection between the local and the remote disk subsystems can be
direct, through a switch, or through other supported distance solutions, for example, Dense
Wave Division Multiplexor (DWDM).
112
IBM System Storage DS6000 Series: Copy Services in Open Environments
11.2 Metro Mirror volume state
Volumes participating in a Metro Mirror session can be found in either of the following states:
򐂰 Copy pending: Volumes are in the copy pending state after the Metro Mirror relationship is
established, but the source and target volumes are still out-of-sync. In that case, data still
needs to be copied from the source to the target volume of the Metro Mirror pair. This
might be the case immediately after a relationship is initially established, or reestablished
after being suspended. The Metro Mirror target volume is unreachable when the pair is in
copy pending state.
򐂰 Full duplex: The full duplex state is the state of a volume copy pair whose members are in
sync, that is, both source and target volumes contain exactly the same data. The target
volume is unreachable when the pair is in full duplex.
򐂰 Suspended: Volumes are in the suspended state when the source and target storage
subsystems cannot communicate anymore, or when the Metro Mirror pair is suspended
manually. In this state, writes to the source volume are not mirrored onto the target
volume. The target volume becomes out-of-sync. During this time, Metro Mirror keeps a
bitmap record of the changed tracks in the source volume. Later, when the volumes are
re-synchronized, only the tracks that were updated are copied.
򐂰 Target copy pending: Indicates that the source volume is unknown or cannot be queried,
and the target state is copy pending.
򐂰 Target full duplex: Indicates that the source volume is unknown or cannot be queried, and
the target state is full duplex.
򐂰 Target suspended: Indicates that the source volume is unknown or cannot be queried, and
the target state is suspended.
򐂰 Not remote copy pair: Indicates that the relationship is not a Metro Mirror pair.
򐂰 Invalid-state: Indicates that the relationship state is invalid.
11.3 Data consistency
In order to restart applications at the remote site successfully, the remote site volumes must
have consistent data. In normal operation, Metro Mirror keeps data consistency at the remote
site. However, in the case of a rolling disaster type of situation, a specific option is necessary
to keep data consistency at the remote site.
For Metro Mirror, consistency requirements are managed through use of the Consistency
Group option. You can specify this option when you are defining Metro Mirror paths between
pairs of LSSs or when you change the default LSS settings. Volumes or LUNs that are paired
between two LSSs whose paths are defined with the Consistency Group option can be
considered part of a Consistency Group.
Consistency is provided by means of the extended long busy (for z/OS) or queue full (for
open systems) conditions. These are triggered when the DS6000 detects a condition where it
cannot update the Metro Mirror target volume. The volume pair that first detects the error
goes into the queue full condition, so that it does not perform any writes and an SNMP trap
message is issued. These messages can be used as triggers for automation purposes that
provide data consistency.
Data consistency and dependent writes are discussed in detail in 12.4, “Consistency Group
function” on page 118.
Chapter 11. Metro Mirror overview
113
Note: During normal Metro Mirror processing, the data on disk at the remote site is an
exact mirror of that at the local site. During or after an error situation, this depends on the
options specified for the pair and the path. Remember that any data still in buffers or
processor memory is not yet on disk and therefore not mirrored to the remote site. A
disaster then appears as a situation similar to a power failure in the local site.
11.4 Rolling disaster
In disaster situations, it is unlikely that the entire complex fails at the same moment. Failures
tend to be intermittent and gradual, and a disaster can occur over many seconds, even
minutes. Because some data might have been processed and other data lost in this
transition, data integrity on the target volumes is exposed. This situation is called a rolling
disaster. The mirrored data at the recovery site must be managed so that cross-volume or
LSS data consistency is preserved during the intermittent or gradual failure.
Metro Mirror by itself does not offer a means of controlling this scenario. It offers Consistency
Group and Critical attributes, which with appropriate automation solutions can manage data
consistency and integrity at the remote site. The Metro Mirror volume pairs are always
consistent due to the synchronous nature of Metro Mirror; however, cross-volume or LSS
data consistency must have an external management method. IBM offers the GDPS® and
eRCMF service offerings to deliver solutions in this area; visit the IBM Web site and see the
Services & Industry Solutions page for more information.
11.5 Automation and management
Metro Mirror is a hardware mirroring solution. A volume (or LUN) is paired with a volume (or
LUN) in the remote disk subsystem. As the size of the environment grows, so does the
complexity of managing it. You need a means for managing the pairs, ensuring they are in
duplex status, adding volume pairs as required, monitoring for error conditions, and more
importantly, for managing data consistency across LSS and across disk subsystems.
When planning a Metro Mirror environment, consider the following topics:
򐂰
򐂰
򐂰
򐂰
Design of the automation solution
Maintenance
Testing
Support
All these considerations need to be thought about. You do not want to be in a situation where
you are relying on your mirroring implementation for data to be consistent in a disaster
situation, only to find that it has not worked, or perhaps worse, you not aware that your data is
inconsistent.
IBM offers services and solutions for the automation and management of the Metro Mirror
environment. These include GDPS and eRCMF. Visit the IBM Web site and see the Services
& Industry Solutions page for more information.
There are also solutions available for integrating Metro Mirror into a cluster, such as
HACMP/XD for AIX or IBM Total Storage Continuous Availability for Windows (GDS). See
Part 8, “Solutions” on page 413.
114
IBM System Storage DS6000 Series: Copy Services in Open Environments
12
Chapter 12.
Metro Mirror options and
configuration
This chapter discusses the options available when using Metro Mirror for DS6000. It also
discusses the configuration guidelines that you should consider when planning the Metro
Mirror environment.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
115
12.1 Basic Metro Mirror operation
Before discussing the options and configuration guidelines for Metro Mirror, let us review
some basic operations you perform with Metro Mirror.
Establish a Metro Mirror pair
This operation establishes the remote copy relationship between a pair of volumes, the
source (or primary) and the target (or secondary), that normally resides on different disk
subsystems. Initially, the volumes are in the simplex state, and immediately after the pair is
established, they transition to the copy pending state. After the data on the pair has been
synchronized (both volumes have the same data), the state of the pair becomes full duplex.
When establishing a Metro Mirror pair, you can select the following additional options:
򐂰 No copy
This option does not copy data from the source to the target. This option presumes that
the volumes are already synchronized. The data synchronization is your responsibility and
the DS6000 does not check its synchronization.
򐂰 Target read
This option allows host servers to read from the target. For a host server to read from the
target, the pair (source and target) must be in a full duplex state. This parameter applies
to open systems volumes (it does not apply to System z volumes).
In the open systems’ file system environment, even if an application reads data from a file
system, a SCSI write command can be issued to the LUN where the file system resides.
This is because some information, such as the last access timestamp, can be updated by
the file system. In this case, even if you specify this option, the operation might fail.
򐂰 Suspend after data synchronization
This option suspends the volume pairs after the data has been synchronized.
򐂰 Wait option
This option delays the command response until the volume pairs are in one of the final
states: simplex, full duplex, suspended, target full duplex, or target suspended (until the
pair is not in the copy pending state). This parameter cannot be used with -type gcp or
-mode nocp.
򐂰 Reset reserve on target
This option allows the establishment of a Metro Mirror relationship when the target volume
is reserved by another host. This parameter can only be used with open systems volumes.
If this option is not specified and the target volume is reserved, the command fails.
Suspend Metro Mirror pair
This operation stops copying data to the target and the pair transitions to the suspended state.
Because the source DS6000 keeps track of all changed tracks on the source volume, you can
resume the copy operations at a later time.
Resume Metro Mirror pair
This operation resumes a Metro Mirror relationship for a volume pair that was suspended, and
restarts transferring data. Only modified tracks are sent to the target volume, because the
DS6000 keeps track of all changed tracks on the source volume after the volume becomes
suspended. When resuming a Metro Mirror pair, you can use the same options as when
initially establishing a Metro Mirror pair except for the no copy option.
116
IBM System Storage DS6000 Series: Copy Services in Open Environments
Terminate Metro Mirror pair
This operation ends the Metro MIrror relationship between the source and target volumes.
12.2 Clustering
Because the disk attached to the server is mirrored using Metro Mirror, this offers some
improved opportunities for high availability solutions.
There are also solutions available for integrating Metro Mirror into a cluster, such as
HACMP/XD for AIX and IBM Total Storage Continuous Availability for Windows (GDS). For
more information, refer to Part 8, “Solutions” on page 413.
12.3 Failover and Failback
The Metro Mirror Failover and Failback modes are designed to help reduce the time required
to synchronize Metro Mirror volumes after switching between the production and the recovery
sites.
In a typical Metro Mirror environment, processing temporarily switches over to the Metro
Mirror remote site upon an outage at the local site. When the local site is capable of resuming
production, processing switches back from the remote site to the local site.
At the recovery site, the Metro Mirror Failover function combines into a single task the three
steps involved in the switchover (planned or unplanned) to the remote site: Terminate the
original Metro Mirror relationship, and then establish and suspend a new relationship at the
remote site. The state of the original source volume at the normal production site is
preserved. The state of the original target volume at the recovery site becomes a source
suspended. This design takes into account that the original source LSS might no longer be
reachable.
To initiate the switchback to the production site, the Metro Mirror Failback function at the
recovery site checks the preserved state of the original source volume at the production site
to determine how much data to copy back. Then either all tracks or only out-of-sync tracks
are copied, with the original source volume becoming a target full duplex. In more detail, this
is how Metro Mirror Failback operates:
򐂰 If a volume at the production site is in the simplex state, all of the data for that volume is
copied back from the recovery site to the production site.
򐂰 If a volume at the production site is in the full duplex or suspended state and without
changed tracks, only the modified data on the volume at the recovery site is copied back to
the volume at the production site.
򐂰 If a volume at the production site is in a suspended state and has tracks that have been
updated, then both the tracks changed on the production site and the tracks marked at the
recovery site are copied back.
򐂰 Finally, the volume at the production site becomes a write-inhibited target volume. This
action is performed on an individual volume basis.
The switchback is completed with one more sequence of a Metro Mirror Failover followed by a
Metro Mirror Failback operations, both given at the now recovered production site.
Figure 12-1 on page 118 summarizes the whole process.
Chapter 12. Metro Mirror options and configuration
117
Primary site (A)
Normal operation
site (A) production site
site (B) recovery site
When planned/unplanned outage at (A):
At (B): Metro Mirror Failover
restart application processing
With site (A) recovered and operable.
At (B):
1. Metro Mirror Failback
2. stop application processing
With pairs in full duplex.
At (A):
1. Metro Mirror Failover
2. start application processing
3. Metro Mirror Failback
Secondary site (B)
Updates are transferred
Source = A
full duplex
Target = B
full duplex
Establish with PPRC Failover
A full duplex
Source = B
suspended
Establish with PPRC Failback
Target = A
copy pending
Source = B
copy pending
Establish with PPRC Failover
Source = A
suspended
B full duplex
Establish with PPRC Failback
Source = A
full duplex
Target = B
full duplex
Figure 12-1 Metro Mirror Failover and Failback sequence
In this manner, Metro Mirror Failover and Failback are dual operations. It is possible to
implement all site switch operations with two pairs of Failover and Failback tasks (one pair for
each direction).
Note: For a planned switchover from site (A) to site (B), and in order to keep data consistency at (B), the application at site (A) has to be quiesced before the Metro Mirror Failover
operation at (B). Alternatively, you can use the freezepprc and unfreezepprc commands.
The same consideration applies when switching back from site (B) to (A).
The Metro Mirror Failover and Failback modes can be invoked from the DS Storage Manager
GUI or the DS Command-Line Interface (CLI).
We give an example of a Metro Mirror Failover and Failback sequence and how to control it in
14.8, “Failover and Failback functions to switch sites” on page 149.
12.4 Consistency Group function
In order to restart applications at the remote site successfully, the remote site must have
consistent data. In normal operation, Metro Mirror keeps data consistency at the remote site.
However, as mentioned in 11.3, “Data consistency” on page 113, in case of a rolling disaster,
a certain procedure is necessary to keep data consistency even in a synchronous remote
copy environment.
In this section,
118
IBM System Storage DS6000 Series: Copy Services in Open Environments
we explain data consistency and explain how the Metro Mirror Consistency Group function
keeps data consistency at the remote site in case of a rolling disaster.
12.4.1 Data consistency and dependent writes
Many applications, such as databases, process a repository of data that has been generated
over a period of time. Many of these applications require that the repository is in a consistent
state in order to begin or continue processing. In general, consistency implies that the order
of dependent writes is preserved in the data copy. In the Metro Mirror environment, keeping
data consistency means that the order of dependent writes is preserved in all the Metro Mirror
target volumes. For example, the following sequence might occur for a database operation
involving a log volume and a data volume:
1. Write to log volume: Data Record #2 is being updated.
2. Update Data Record #2 on data volume.
3. Write to log volume: Data Record #2 update complete.
If the copy of the data contains any of these combinations, then the data is consistent:
򐂰 Operation 1, 2, and 3
򐂰 Operation 1 and 2
򐂰 Operation 1
If the copy of data contains any of these combinations, then the data is inconsistent (the order
of dependent writes was not preserved):
򐂰
򐂰
򐂰
򐂰
Operation 2 and 3
Operation 1 and 3
Operation 2
Operation 3
When discussing the Consistency Group function, data consistency means this sequence is
always kept in the copied data. And the order of non-dependent writes does not necessarily
need to be preserved. For example, consider the following two sequences:
1.
2.
3.
4.
Deposit paycheck in checking account A.
Withdraw cash from checking account A.
Deposit paycheck in checking account B.
Withdraw cash from checking account B.
In order for the data to be consistent, the deposit of the paycheck must be applied before the
withdrawal of cash for each of the checking accounts. However, it does not matter whether
the deposit to checking account A or checking account B occurred first, as long as the
associated withdrawals are in the correct order. So, for example, the data copy is consistent if
the following sequence occurred at the copy:
1.
2.
3.
4.
Deposit paycheck in checking account B
Deposit paycheck in checking account A
Withdraw cash from checking account B
WIthdraw cash from checking account A
In other words, the order of updates is not the same as it was for the source data, but the
order of dependent writes is still preserved.
12.4.2 Consistency Group function: How it works
In the operation of the Consistency Group function of Metro Mirror, we distinguish two parts.
One is the invocation of the Consistency Group option, and the other one is the freeze/run
Chapter 12. Metro Mirror options and configuration
119
operation. Together, they make it possible for the disk subsystem to hold I/O activity and
subsequently to thaw the held I/O activities.
Consistency Group option
This option causes the disk subsystem to hold I/O activity to a volume for a time period by
putting the source volume into a queue full condition when the DS6000 detects a situation
where it cannot update the Metro Mirror target volume. This operation can be done across
multiple LUNs or volumes, multiple LSSs, and even across multiple disk subsystems. You
can specify this option when you are defining Metro Mirror paths between pairs of LSSs or
when you change the default Consistency Group setting on each LSS (the Consistency
Group option is disabled by default).
In the disk subsystem itself, each command is managed with each logical subsystem (LSS).
This means that there are slight time lags until each volume in the different LSS is subject to
the queue full condition. Some people are concerned that the time lag causes you to lose
data consistency, but that is not true. We explain how to keep data consistency in the
Consistency Group environments in the following section.
In the example in Figure 12-2, three write operations (first, second, and third) are dependent
writes. This means that these operations must be completed sequentially.
.
1st
LSS11
LSS21
LSS12
LSS22
LSS13
LSS23
Wait
Application
on
Servers
2nd
Wait
3rd
Wait
dependent write
operation
Source DS6000
Target DS6000
Figure 12-2 Consistency Group: Example 1
In Figure 12-2, there are two Metro Mirror paths between LSS11 and LSS21. There are
another two Metro Mirror paths for each of the other LSS pairs (LSS12:LSS22 and
LSS13:LSS23). In a disaster, the paths might fail at different times. At the beginning of the
disaster, such as a rolling disaster, one set of paths (such as the paths between LSS11 and
LSS21) might be inoperable while other paths are working. At this time, the volumes in
LSS11 are in an queue full condition, and the volumes in LSS12 and LSS13 are not. The first
120
IBM System Storage DS6000 Series: Copy Services in Open Environments
operation is not completed because of the queue full condition, and the second and third
operations are not completed because the first operation has not been completed. In this
case, the first, second, and third updates are not included in the Metro Mirror target volumes
in LSS21, LSS22, and LSS23. Therefore, the Metro Mirror target volumes at the remote site
keep consistent data.
In the example illustrated in Figure 12-3, the volumes in LSS12 are in an queue full condition
and the other volumes in LSS11 and LSS13 are not. The first write operation is completed,
because the volumes in LSS11 are not in an queue full condition. The second write operation
is not completed because of the queue full condition. The third write operation is also not
completed because the second operation is not completed. In this case, the first update is
included in the Metro Mirror target volumes, and the second and third updates are not
included. Therefore, this case is also consistent.
1st
LSS11
LSS21
LSS12
LSS22
LSS13
LSS23
Completed
Application
on
Servers
2nd
Wait
3rd
Wait
dependent write
operation
Source DS6000
Target DS6000
Figure 12-3 Consistency Group: Example 2
In all cases, if each write operation is dependent, the Consistency Group option can keep the
data consistent in the Metro Mirror target volumes until the Consistency Group time-out
occurs. After the time-out value has been exceeded, all held I/O are released. The
Consistency Group time-out value can be specified at the LSS level.
If each write operation is not dependent, the I/O sequence is not kept in the Metro Mirror
target volumes that are in LSSs with the Consistency Group option specified. In the example
illustrated in Figure 12-4 on page 122, the three write operations are independent. If the
volumes in LSS12 are in a queue full condition and the other volumes in LSS11 and LSS13
are not, the first and third operations are completed and the second operation is not
completed.
Chapter 12. Metro Mirror options and configuration
121
1st
LSS11
LSS21
LSS12
LSS22
LSS13
LSS23
Completed
Application
on
Servers
2nd
Wait
3rd
Wait
Completed
independent
write
operations
Source DS6000
Target DS6000
Figure 12-4 Consistency Group: Example 3
In this case, the Metro Mirror target volumes reflect only the first and third write operation, not
including the second operation.
Typical database management software can recover its databases by using its log files if
dependent write operations are kept. At the same time, you should not need to care whether
independent write operations are kept in sequence.
Freeze and unfreeze operations
The Consistency Group option itself can keep consistent data at the remote site, in case of a
rolling disaster, if all volumes go into the queue full condition within the time interval specified
in the Consistency Group time-out value. However, this is not always true. Therefore, we need
a command that allows us to hold the I/O activity to volumes other than the volumes which the
DS6000 itself detects as having an error condition. We also need a command that allows us
to release the held I/O without having to wait for the Consistency Group time-out in order to
minimize the impact on the applications. These commands are freezepprc and
unfreezepprc, which can be issued at an LSS level, not volume level. These commands are
available using the DS CLI, and we discuss them in this section.
When the DS6000 detects a condition where it cannot update the Metro Mirror target volume,
the Metro Mirror source volume with Consistency Group option becomes suspended and
enters the queue full condition. At the same time, the DS6000 issues an SNMP notification
(Trap 202: source PPRC Devices on LSS Suspended Due to Error).
An automation program, triggered by the SNMP notification, can issue the freezepprc
command to all LSS pairs having volumes related to the application. This command causes
all Metro Mirror source volumes in the LSSs with the Consistency Group option to become
suspended and enter the queue full condition. In addition, this operation removes the Metro
122
IBM System Storage DS6000 Series: Copy Services in Open Environments
Mirror paths. Because all Metro Mirror source volumes become suspended and all related
paths are removed, further updates to the source volumes are not be sent to their targets.
Now you have consistent data at the remote site. It is necessary to have an automation
procedure such as the one just described, in order to have consistent data at the remote site
in case of a rolling disaster.
In case of partial link failures (such as in the case of Figure 12-2 on page 120) and in order to
rapidly resume application I/O processing at the local site, you can use the unfreezepprc
command to resume held I/Os to the affected LSSs. In general, you issue the unfreezepprc
command after successful completion of the freezepprc command to all related LSSs, which
means you have consistent data at the remote site.
Note: By partial links failure, we mean in this section the situation where for a particular
LSS all of its links fail, while at the same time there are other LSSs with their links working.
When any particular LSS has part of its links not working, Metro Mirror can keep sending
data to the target volumes using the other available links in the LSS.
The default two minute timer of the queue full condition gives the automation enough time to
issue a freezepprc command to the necessary LSSs. I/O resumes after the default two
minute interval if a unfreezepprc command is not received first.
It is not possible to issue Consistency Group (freeze/unfreeze) type commands from the DS
Storage Manager GUI, but you can change the time-out values by using the DS GUI.
Important: The queue full condition is presented only for the source volume affected by
the error (in the case of path failures, multiple volumes are often affected). Still the freeze
operation is performed at the LSS level, causing all Metro Mirror volumes in that LSS to go
into suspended state with queue full condition and terminating all associated paths.
Therefore, when planning your implementation, you should consider not intermixing
volumes from different applications in an LSS pair that is part of a Consistency Group.
Otherwise, the not-in-error volumes belonging to other applications are frozen, too.
12.5 Metro Mirror paths and links
Metro Mirror pairs are set up between volumes in LSSs, usually in different disk subsystems,
and these are normally in separate locations. A path (or group of paths) needs to be defined
between the source LSS and the target LSS. These logical paths are defined over physical
links between the disk subsystems.
The physical link includes the host adapter in the source DS6000, the cabling, switches or
directors, any wideband or long distance transport devices (DWDM, channel extenders, or
WAN), and the host adapters in the target disk subsystem. Physical links can carry multiple
Metro Mirror logical paths, as shown in Figure 12-5 on page 124.
Note: For Metro Mirror, the DS6000 supports Fibre Channel links only. To facilitate ease of
testing, the DS6000 does support Metro Mirror source and target on the same DS6000.
Chapter 12. Metro Mirror options and configuration
123
LSS 0
LSS 1
LSS 2
LSS 3
Physical
Fibre Channel link
:
LSS 08
:
LSS 0
LSS 1
LSS 2
LSS 3
:
256 logical paths
per FCP link
LSS nn
LSS 08
:
LSS nn
Figure 12-5 Logical paths
Paths are unidirectional, which means that they are defined to operate in either one direction
or the other. Still, Metro Mirror is bidirectional. Any particular pair of LSSs can have paths
defined among them that have opposite directions; each LSS holds both source and target
volumes from the other particular LSS. Even more, you are allowed to define opposite
direction paths on the same Fibre Channel physical link.
For bandwidth and redundancy, more than one path can be created between the same LSSs.
Metro Mirror balances the workload across the available paths between the source and target
LSSs.
Note: Remember that the LSS is not a physical construct in the DS6000; it is a logical
construct. Volumes in an LSS can come from multiple disk arrays.
Physical links are bidirectional and can be shared by other Metro Mirror pairs as well as other
remote mirror and copy functions, such as Global Copy and Global Mirror.
12.5.1 Fibre Channel links
A DS6000 Fibre Channel port can simultaneously be:
򐂰 Sender for Metro Mirror source
򐂰 Receiver for Metro Mirror target
򐂰 Target for Fibre Channel Protocol (FCP) hosts I/O from open systems and Linux on
System z.
Although one FCP link has sufficient bandwidth for most Metro Mirror environments; for
redundancy reasons, we recommend configuring two Fibre Channel links between each
source and remote disk subsystem.
With the DS6000, you have only eight Fibre Channel ports (FC ports) available, four per
server (controller). And, each sever owns each FC port, unlike the DS8000. Therefore, we
recommend you use two Fibre Channel links for Metro Mirror, one on each controller, for
availability and balanced configuration. See Figure 12-6 on page 125.
124
IBM System Storage DS6000 Series: Copy Services in Open Environments
Metro Mirror Pairs
LSS1A
LSS14
Server0
1A01
Server0
1A00
Fibre Channel links
LSS1B
Fibre Channel links
Server1
1B01
1401
LSS15
Server1
1B00
1400
1500
1501
DS6000#2
DS6000#1
Figure 12-6 DS6000 FC ports are owned by the servers
Dedicating Fibre Channel ports for Metro Mirror use guarantees no interference from host I/O
activity. We recommend this with Metro Mirror, which is time critical and should not be
impacted by host I/O activity. Each Metro Mirror port provides connectivity for all LSSs within
the DS6000 and can carry multiple logical Metro Mirror paths.
Note: If you want the same set of Fibre Channel links to be shared between Metro Mirror
and Global Copy or Global Mirror, you should consider the impact of the aggregate data
transfer. In general, we do not recommend sharing the FCP links used for Metro Mirror,
including the physical network links, with other asynchronous remote copy functions.
Metro Mirror FCP links can be directly connected, or connected by up to two switches.
Important: If you use channel extension technology devices for Metro Mirror links, you
should verify with the product’s vendor what environment (directly connect or connected
with SAN switch) is supported by the vendor and what SAN switch is supported.
12.5.2 Logical Paths
A Metro Mirror logical path is a logical connecting path between the sending LSS and the
receiving LSS. An FCP link can accommodate multiple logical paths.
Figure 12-7 on page 126 shows an example where we have a 1:1 mapping of source to target
LSSs, and where the three logical paths are accommodated in over one physical link:
򐂰
򐂰
򐂰
LSS1 in DS6000-1 to LSS1 in DS6000-2
LSS2 in DS6000-1 to LSS2 in DS6000-2
LSS3 in DS6000-1 to LSS3 in DS6000-2
Alternatively, if the volumes in each of the LSSs of DS6000-1 map to volumes in all three
target LSSs in DS6000-2, there are nine logical paths over the physical link (not fully
illustrated in Figure 12-7 on page 126). Note that we recommend a 1:1 LSS mapping.
Chapter 12. Metro Mirror options and configuration
125
DS6000 1
DS6000 2
3-9 logical paths
LSS 1
LSS 1
1 logical path
1 logical path
switch
LSS 2
1 logical path
LSS 3
1 logical path
Port
1 Link
Port
Metro Mirror
paths
LSS 2
1 logical path
LSS 3
1 logical path
Figure 12-7 Logical paths over a physical link for Metro Mirror
Metro Mirror paths have certain architectural limits, which include:
򐂰 A source LSS can maintain paths up to a maximum of four target LSSs. Each target LSS
can reside in a separate DS6000.
򐂰 You can define up to eight logical paths per LSS-LSS relationship. Each path requires a
separate physical link.
򐂰 An FCP port can host up to 2 048 logical paths. These are the logical and directional paths
that are made from LSS to LSS.
򐂰 An FCP physical link (the physical connection from one port to another port) can host up
to 256 logical paths.
򐂰 An FCP port can accommodate up to 126 different physical links (DS6000 port to DS6000
port through the SAN).
12.6 Bandwidth
Prior to establishing your Metro Mirror solution, you should establish your peak bandwidth
requirement. This helps to ensure that you have enough Metro Mirror links in place to support
that requirement.
To avoid any response time issues, you should establish the peak write rate for your systems
and ensure that you have adequate bandwidth to cope with this load and allow for growth.
Remember that only writes are mirrored across to the target volumes.
Tools to assist with this are operating system dependent, such as iostat. You can refer to
the IBM Redbook IBM System Storage DS6000 Series: Architecture and Implementation,
SG24-6781. Another method, but not so exact, is to monitor the traffic over the FC switches
using FC switch tools and other management tools, and remember that only writes are
mirrored by Metro Mirror. You can also get some feeling about the proportion of read/writes
by issuing datapath query devstats on SDD-attached servers.
12.7 LSS design
Because the DS6000 has made the LSS a topological construct, which is not tied to a
physical array as in the ESS, the design of your LSS layout can be simplified. It is now
126
IBM System Storage DS6000 Series: Copy Services in Open Environments
possible to assign LSSs to applications, for example, without concern regarding
under-allocation or over-allocation of physical disk subsystem resources.
This can also simplify the Metro Mirror environment, because it is possible to reduce the
number of commands that are required for data consistency as well as make Metro Mirror’s
effects more granular. For example, a freeze operation is performed at the LSS level, causing
all Metro Mirror volumes in that LSS to go into suspended state with a queue full condition and
terminating all associated paths. If you assign LSSs to each of your applications, you can
control the impact of the queue full condition caused by the freeze operation at an application
level. On the contrary, if you put volumes used by several different applications into the same
LSS, all those applications sharing the LSS and their Metro Mirror volumes are affected by
the queue full condition. You can issue one freezepprc command to multiple LSSs.
Therefore, you do not need to consolidate the number of LSSs that your applications use.
(See 12.4.2, “Consistency Group function: How it works” on page 119 for more information.)
12.8 Distance
The distance between your source and target DS6000 subsystems has an effect on the
response time overhead of the Metro Mirror implementation. The maximum supported
distance for Metro Mirror is 300 km.
12.9 Symmetrical configuration
When planning your Metro Mirror configuration, you should consider the possibility of a
symmetrical configuration in terms of both physical and logical elements. This has the
following benefits:
򐂰 Simplified management. It is easier to see where volumes are mirrored, and processes
can be easily automated.
򐂰 Reduced administrator overhead. Due to automation, and the simpler nature of the
solution, overhead can be reduced.
򐂰 Simplified capability to add new capacity into the environment. New arrays can be added
in a modular fashion.
򐂰 Easier problem diagnosis. The simple structure of the solution aids in identifying where
any problems might exist.
Figure 12-8 on page 128 shows this idea in a graphical form. DS6000 #1 has Metro Mirror
paths defined to DS6000 #2, which is in a remote location. On DS6000 #1, volumes defined
in LSS 00 are mirrored to volumes in LSS 00 on DS6000 #2 (volume P1 is paired with volume
S1, P2 with S2, P3 with S3, and so on). Volumes in LSS 01 on DS6000 #1 are mirrored to
volumes in LSS 01 on DS6000 #2, and so on. Requirements for additional capacity can be
added in a symmetrical way also by the addition of volumes into existing LSSs, and by the
addition of new LSSs when needed (for example, the addition of two volumes in LSS 03 and
LSS 05, and one volume to LSS 04 bring them to the same number of volumes as the other
LSSs. Additional volumes can then be distributed evenly across all LSSs, or additional LSSs
added).
Chapter 12. Metro Mirror options and configuration
127
Figure 12-8 Symmetrical Metro Mirror configuration
As well as making the maintenance of the Metro Mirror configuration easier, the symmetrical
implementation has the added benefit of helping to balance the workload across the DS6000.
Figure 12-8 shows a logical configuration, this idea applies equally to the physical aspects of
the DS6000. You should attempt to balance workload and apply symmetrical concepts to
other aspects of your DS6000 (for example, the Extent Pools).
12.10 Volumes
You need to consider which volumes should be mirrored to the target site. One option is to
mirror all volumes. This is advantageous for the following reasons:
򐂰 You do not need to consider whether any required data has been missed or not.
򐂰 Users do not need to remember which logical pool of volumes is mirrored and which is
not.
򐂰 Adding volumes to the environment is simplified. You do not need two processes for the
addition of disk (one for mirrored volumes and another for non-mirrored volumes).
򐂰 You can move data around your disk environment easily without worrying about whether
the target volume is a mirrored volume.
You might choose not to mirror all volumes. In this case, you need careful control over what
data is placed on the mirrored volumes (to avoid any capacity issues) and what data you
place on the non-mirrored volumes (to avoid missing any required data). One method of doing
this is to place all mirrored volumes in a particular set of LSSs, in which all volumes are Metro
Mirror-enabled, and to direct all data requiring mirroring to these volumes.
Though mirroring all volumes might be the simpler solution to manage, it could also require
significantly more network bandwidth. Since network bandwidth is a cost to the solution,
minimizing the bandwidth may well be worth the added management complexity.
128
IBM System Storage DS6000 Series: Copy Services in Open Environments
12.11 Hardware requirements
Metro Mirror is an optional licensed function of the IBM System Storage DS6000 series.
All DS6000 series licensed functions are enabled, authorized, managed, activated, and
enforced based upon the physical capacity contained in a 1750 system. Particularly for Metro
Mirror, the DS6000 must have the Remote Mirror and Copy (RMC) licensed function, with the
corresponding feature #531x, based on the physical capacity.
Note: For a detailed explanation of the features involved and things to consider when
ordering Metro Mirror, we recommend you refer to the IBM System Storage DS6000
Series (IBM 1750-522) announcement letter.
You can access IBM announcement letters at:
http://www.ibm.com/products
Interoperability
Metro Mirror pairs can only be established between disk subsystems of the same (or similar)
type and features. For example, a DS6000 can have a Metro Mirror pair with another DS6000,
a DS8000, an ESS 800, or an ESS 750. It cannot have a Metro Mirror pair with an RVA or an
ESS F20. Note that all disk subsystems must have the appropriate Metro Mirror feature
installed. If your DS6000 is mirrored to an ESS disk subsystem, the ESS must have PPRC
Version 2 (which supports Fibre Channel links) with the appropriate licensed internal code
level (LIC).
Refer to the DS6000 Interoperability Matrix for more information:
http://www-1.ibm.com/servers/storage/disk/ds6000/interop.html
Chapter 12. Metro Mirror options and configuration
129
130
IBM System Storage DS6000 Series: Copy Services in Open Environments
13
Chapter 13.
Metro Mirror performance and
scalability
In this chapter, we discuss some of the performance and scalability considerations when
implementing Metro Mirror for DS6000.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
131
13.1 Performance
Because Metro Mirror is a synchronous mirroring technology, it introduces a performance
overhead (for write operations) over a similar environment which has no remote mirroring.
However, it does not consume any CPU activity, unlike operating system mirroring. This is
important to understand this as part of the planning process for Metro Mirror.
Bandwidth analysis and capacity planning for your Metro Mirror links should help to define
how many links you need, and when you need to add more to ensure the best possible
performance.
As part of your implementation project, you might be able to identify and then distribute hot
spots across your configuration, or take other actions to manage and balance the load.
Some basic things you should consider:
򐂰 Is your bandwidth too little so that you might see an increase in the response time of your
applications at moments of high workload?
򐂰 We recommend you do not share the Metro Mirror link I/O ports with host attachment
ports. One result can be unpredictable performance of Metro Mirror and a much more
complicated search in case of performance analysis.
򐂰 Distance is an important topic. Remember light speed is less than 300 000 km/second,
that is less then 300 km/ms on Fibre Channel. The data must go to the other site, and then
an acknowledgement comes back. Add possible latency times of some active components
on the way, and you get approximately a 1 ms overhead per 100 km for write I/Os.
򐂰 Sometimes the problem is not Metro Mirror, but rather hot spots on the disks. Be sure
these problems are resolved before you start with Metro Mirror.
򐂰 Split your workload across the controllers by using more than one Extent Pool for your
Metro Mirror volumes, trying to get an almost symmetrical configuration.
򐂰 Extent pools are associated to a controller, so be sure that the Metro Mirror links you are
using for your LSS use the same controller. You should consider this at the source and the
target site. For example, if you have two connections, one on each controller, and you
need a third connection, it should be placed on the controller where the most active Metro
Mirror volumes are.
13.1.1 Initial synchronization
When doing the initial synchronization of your Metro Mirror pairs, the DS6000 uses a throttling
algorithm to prevent the Metro Mirror process from using too many primary site resources.
This might prolong the synchronization process if your DS6000 is busy at the time. You can
choose to stagger your synchronization tasks or to run them at a time of low utilization to
make this process more efficient.
As an alternative, you can also choose to do the initial synchronization in the Global Copy
mode and then switch to the Metro Mirror mode. This allows you to bring all volumes to
duplex-pending XD state, which has little or no application impact, and then switch to full
duplex state. You can do so using the DS CLI.
132
IBM System Storage DS6000 Series: Copy Services in Open Environments
13.2 Scalability
The DS6000 Metro Mirror environment can be scaled up or down as required. If new volumes
that require mirroring are added to the DS6000, they can be dynamically added. If additional
Metro Mirror paths are required, they also can be dynamically added.
You must remember that in the DS6000 you have only eight Fibre Channel ports available,
four per controller. So we recommend to use two Fibre Channel links exclusively for Metro
Mirror, one on each controller, for availability.
If you need more bandwidth, you can increase the number of Metro Mirror links. It should be
almost symmetrical, and it also depends on the controller used by the Extent Pool of your
Metro Mirror volumes.
We think no more than four exclusive Metro Mirror connections make sense. It is the same as
for your maximum Host attachments at this time. Except if you are using the DS6000 mainly
as a Metro Mirror target, without any or few host activities.
As we have previously mentioned, the logical nature of the LSS has made a Metro Mirror
implementation on the DS6000 easier to plan, implement, and manage. However, if you need
to add more LSSs to your Metro Mirror environment, your management and automation
solutions should be set up to handle this.
The eRCMF and GDPS service offerings are designed to provide this functionality. Visit the
IBM Web site and see the Services & Industry Solutions page for more information. Also see
Part 8, “Solutions” on page 413.
Adding capacity to the same DS6000
If you are adding capacity to an existing DS6000, providing your Metro Mirror link bandwidth
is not close to or over capacity, it is possible that you only have to add volume pairs to your
configuration. If you are adding more LSSs, then you must define Metro Mirror paths before
adding volume pairs.
Adding capacity in new DS6000s
If you are adding new DS6000s to your configuration, you must add physical Metro Mirror
links before defining your Metro Mirror paths and volume pairs. We recommend a minimum of
two Metro Mirror paths per DS6000 pair for redundancy reasons. Your bandwidth analysis
indicates if you require more than two paths.
Chapter 13. Metro Mirror performance and scalability
133
134
IBM System Storage DS6000 Series: Copy Services in Open Environments
14
Chapter 14.
Metro Mirror interfaces and
examples
This chapter describes the interfaces that you can use for Metro Mirror management with the
IBM System Storage DS6000 in an open systems environment. This chapter also presents
examples that illustrate step-by-step how to execute the setup and management tasks of the
Metro Mirror environment.
This chapter presents the following topics and examples:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
DS Command Line Interface and DS Storage Manager overview.
Set up a Metro Mirror environment.
Remove a Metro Mirror environment.
Manage the Metro Mirror environment.
Failover and Failback operations (site switch).
freezepprc and unfreezepprc commands.
Using the DS Storage Manager GUI to manage Metro Mirror.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
135
14.1 Metro Mirror management interfaces overview
There are various interfaces available for the configuration and management of Metro Mirror
for DS6000 when used in an open systems environment:
򐂰
򐂰
򐂰
򐂰
DS Command-Line Interface (DS CLI)
DS Storage Manager (DS SM) Graphical User Interface (GUI)
TotalStorage Productivity Center for Replication (TPC for Replication)
enterprise Remote Copy Facility (eRCMF)
The present chapter gives an overview of the DS CLI and the DS SM for Metro Mirror
management. We also discuss DS CLI and DS SM in Part 2, “Interfaces” on page 15.
TotalStorage Productivity Center for Replication provides management of DS6000 series
business continuance solutions, including FlashCopy, Metro Mirror, and Global Mirror. We
cover TPC for Replication in Chapter 28, “IBM TotalStorage Productivity Center for
Replication” on page 415. TPC for Replication includes similar functions to Global Mirror
Utility (GMU). GMU users should consider migrating to TPC for Replication.
eRCMF is a scalable and flexible solution to protect business data and maintain consistent
data, providing an automation tool that simplifies the disaster recovery operations. For more
information about this solution, see Part 8, “Solutions” on page 413.
You can also use the following interfaces, not covered in the present book:
򐂰 DS Open Application Programming Interface (DS Open API)
򐂰 z/OS interfaces: TSO, ICKDSF, and ANTRQST API
In order to use a z/OS interface to manage open systems LUNs, the DS6000 must have at
least one CKD volume. If you are interested in this capability, refer to the redbook IBM
System Storage DS6000 Series: Copy Services with System z servers, SG24-6782.
For information about the DS Open API, refer to the publication IBM System Storage DS
Open Application Programming Interface Reference, GC35-0516.
DS CLI and DS SM similar functions for Metro Mirror management
Table 14-1 on page 137 compares similar Metro Mirror commands from the DS CLI and the
DS SM, and the corresponding action.
136
IBM System Storage DS6000 Series: Copy Services in Open Environments
Table 14-1 DS CLI and DS SM commands and actions
Task
Command with DS CLI
Select option with DS SM
Metro Mirror and Global Copy path commands
List available I/O ports that can
be used to establish Metro Mirror
paths.
lsavailpprcport
This information is shown
during the process when a
path is established.
List established Metro Mirror
paths.
lspprcpath
Copy Services → Paths
Establish path.
mkpprcpath
Copy Services → Paths →
Create
Delete path.
rmpprcpath
Copy Services → Paths →
Delete
Failback.
failbackpprc
Copy Services → Metro
Mirror → Recover Failback
Failover.
failoverpprc
Copy Services → Metro
Mirror → Recover Failover
List Metro Mirror volume pairs.
lspprc
Copy Services → Metro
Mirror
Establish pair.
mkpprc
Copy Service → Metro
Mirror → Create
Suspend pair.
pausepprc
Copy Services → Metro
Mirror → Suspend
Resume pair.
resumepprc
Copy Services → Metro
Mirror → Resume
Delete pair.
rmpprc
Copy Services → Metro
Mirror → Delete
Freeze Consistency Group.
freezepprc
Not applicable.
Thaw Consistency Group.
unfreezepprc
Not applicable.
Metro Mirror pairs commands
The DS CLI has the advantage that you can make scripts and use them for automation. The
DS Storage Manager GUI is a Web-based graphical user interface and is more intuitive than
the DS CLI.
14.2 Copy Services network components
The network components and connectivity of the DS SMC are illustrated in Figure 14-1 on
page 138.
Chapter 14. Metro Mirror interfaces and examples
137
Customer
Network
DS Storage
Manager
DS CLI
DS API
DS SMC 1
DS CLI
Ethernet
Network1
Controller
0
DS Storage
Manager
DS6000
(DS SMC 2)
DS CLI
Ethernet
Network2
Controller
1
DS Storage
Manager
Figure 14-1 Components and connectivity of the DS Storage Management Console
DS SM and DS CLI (as well as DS Open API) commands are issued through the Ethernet
network to the DS Storage Management Console (DS SMC). When the DS SMC receives the
command request, it communicates with each server in the disk subsystem through the
Ethernet network. Therefore, the DS SMC is a key component to configure and manage the
DS6000 and its functions.
Up to seven DS6000s can be managed by one DS SMC, but it is also possible to have one
DS SMC for each DS6000. Each DS6000 can be administered by one DS SMC only or by a
primary DS SMC and a backup DS SMC. The setup is extremely dependent on the user’s
environment. Still, we recommend that you have a dual DS SMC configuration, such as the
one illustrated in Figure 14-1.
To establish a Metro Mirror pair, the LAN connection between the source and target DS6000s
is not mandatory, unlike the ESS. A similar situation happens with Global Copy. However,
Copy Services DS CLI commands have to be issued to the DS SMC connected to the
DS6000 which has the source volume. When you need to establish a Metro Mirror pair from a
volume at the remote site to a volume at the local site, the DS CLI command has to be issued
to the DS6000 SMC at the remote site. In this case, you need a LAN connection from the
local DS CLI client machine to the DS6000 at the remote site.
14.3 DS Command-Line Interface (DS CLI)
You can use the DS CLI interface to manage all DS6000 Copy Services functions, such as
defining paths, establishing Metro Mirror pairs, and so forth. For a detailed explanation about
the DS CLI, refer to Chapter 4, “DS Command-Line Interface (DS CLI)” on page 27.
When you establish or remove Metro Mirror paths and volume pairs, you must give the DS
CLI commands to the DS SMC that is connected to the source DS6000. Also, when checking
status information at the local and remote sites, you must issue DS CLI list type commands,
such as lspprc, to each DS6000, source and target.
The DS CLI commands are documented in the publication IBM System Storage DS6000:
Command-Line Interface User´s Guide, GC26-7922.
138
IBM System Storage DS6000 Series: Copy Services in Open Environments
14.4 DS Storage Manager GUI
You can use the DS Storage Manager (DS SM) Graphical User Interface (GUI) to set up and
control Metro Mirror for DS6000 functions. It is user friendly; however, you cannot use it for
automation activities, and certain Metro Mirror functions are not supported from this interface.
For a detailed explanation about the DS SM Graphical User Interface, refer to Chapter 3, “DS
Storage Manager (GUI)” on page 17.
Note: The DS SM GUI supports, differently from the DS CLI, a multi-session environment,
but not all functions and options are supported by the GUI.
14.5 Set up Metro Mirror environment using DS CLI
In the following sections, we present an example of how to set up a Metro Mirror environment
using the DS CLI. Figure 14-2 shows the configuration that we are going to implement.
LSS1A
LSS14
LSS15
1A01
LSS1B
(physical links)
Server1
1501
1A00
Fibre Channel connections
Server1
1500
Server0
1401
Server0
1400
1B00
1B01
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 14-2 DS6000 configuration in the Metro Mirror setup example
In our example, we use different LSS and LUN numbers for the Metro Mirror source and
target elements, so that it is easier for you to know which one we are discussing as you follow
this example.
Note: In a real environment as opposed to our example, we recommend that you maintain
a symmetrical configuration in terms of both physical and logical elements to simplify the
management of your Metro Mirror environment.
For a more redundant and balanced configuration, we put two physical paths on separate
servers (controllers) in the DS6000s.
14.5.1 Preparing to work with the DS CLI
As we prepare to work with the DS CLI, we do some initial tasks to simplify our activities
during the configuration process.
Chapter 14. Metro Mirror interfaces and examples
139
Simplifying the DS CLI command syntax
Before setting up the Metro Mirror environment, we recommend that you set up the following
DS CLI environment to simplify the command syntax.
To save you from typing -dev storage_image_ID and -remotedev storage_image_ID in
each command, you can put these values in your CLI profile. Default and target Storage
Image IDs devid and remotedevid are equivalent to the -dev storage_image_ID and
-remotedev storage_image_ID command options; see Example 14-1.
Example 14-1 Part of CLI profile
# SMC ipaddress for DS6000#1
hmc1: 10.10.10.1
# Default and target Storage Image ID
devid: IBM.1750-1300247
remotedevid:IBM.1750-1300819
For further information, see Chapter 4, “DS Command-Line Interface (DS CLI)” on page 27.
Creating password file
With the managepwfile command, you can create the password file on your DS CLI client.
After creating the password file, you do not need to specify a password at login time. When
you create an operational script to manage your remote mirror and copy environment, you
can use this function so that you do not need to write a password in the script file. We
recommend that you use this function from a security point of view. Example 14-2 shows how
to create the password file.
Example 14-2 Creating the password file
dscli> managepwfile -action add -name script1 -pw xyz1234
Date/Time: November 15, 2005 10:45:57 PM JST IBM DSCLI Version: 5.1.0.204
CMUC00206I managepwfile: Record 9.155.86.130/copy successfully added to password file
C:\Documents and Settings\Administrator\dscli\security.dat.
dscli> exit
C:\Program Files\ibm\dscli510204>dscli -cfg ds6kg654 -user script1
Date/Time: November 15, 2005 10:46:13 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300654
IBM.1750-1300819
dscli>
14.5.2 Setup of Metro Mirror configuration
Figure 14-3 on page 141 shows the configuration we set up for this example. The
configuration has four Metro Mirror pairs that reside in two LSSs. Two paths are defined
between each source and target LSS.
140
IBM System Storage DS6000 Series: Copy Services in Open Environments
Source
Metro Mirror paths
LSS14
1400
1401
Target
1A00
Metro Mirror pairs
LSS15
1A01
Fibre Channel connections
(physical links)
1500
LSS1B
Metro Mirror pairs
1501
LSS1A
1B00
1B01
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 14-3 Metro Mirror environment to set up
To configure the Metro Mirror environment, we follow this procedure:
1. Determine the available Fibre Channel links for the definition of the paths.
2. Define the paths that Metro Mirror uses.
3. Create Metro Mirror pairs.
14.5.3 Determine the available Fibre Channel links
First, you have to look at the available Fibre Channel links. With the lsavailpprcport
command, you can do this; see Example 14-3. You see all available port combinations
between the source and the target LSSs. You have to issue this command to the DS SMC
connected to DS6000#1, which is the Metro Mirror source.
Example 14-3 List available Fibre Channel links
dscli> lsavailpprcport -l -remotedev IBM.1750-1300247
Date/Time: November 17, 2005 6:38:21 PM JST IBM DSCLI
IBM.1750-1300819
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0102
I0103
FCP NA
NA
dscli>
dscli> lsavailpprcport -l -remotedev IBM.1750-1300247
Date/Time: November 17, 2005 6:38:33 PM JST IBM DSCLI
IBM.1750-1300819
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0102
I0103
FCP NA
NA
-remotewwnn 500507630EFFFE16 14:1a
Version: 5.1.0.204 DS:
-remotewwnn 500507630EFFFE16 15:1b
Version: 5.1.0.204 DS:
FCP port ID with the lsavailpprcport command has four hexadecimal characters in the
format 0xEEAP, where EE is a port enclosure number (00-3F), A is the adapter number (0-F),
and P is the port number (0-F). The FCP port ID number is prefixed with the character I.
You can use the -fullid parameter to display the DS6000 Storage Image ID in the command
output; see Example 14-4 on page 142.
Chapter 14. Metro Mirror interfaces and examples
141
Example 14-4 List available Fibre Channel links with DS6000 Storage Image ID
dscli> lsavailpprcport -fullid -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16
14:1a
Date/Time: November 17, 2005 6:39:39 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
Local Port
Attached Port
Type
==================================================
IBM.1750-1300819/I0003 IBM.1750-1300247/I0003 FCP
IBM.1750-1300819/I0102 IBM.1750-1300247/I0103 FCP
You need the worldwide node name (WWNN) of your target DS6000 to issue the
lsavailpprcport command. You can get this by using the lssi command; see Example 14-5.
You have to issue this command to the DS SMC connected to DS6000#2, which is the Metro
Mirror target.
Example 14-5 Get WWNN of target DS6000
dscli> lssi
Date/Time: November 17, 2005 6:40:24 PM JST IBM DSCLI Version: 5.1.0.204
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.1750-1300247 IBM.1750-1300247 511 500507630EFFFE16 Online Enabled
14.5.4 Create Metro Mirror paths
Now you can use the mkpprcpath command to create paths between the source and target
LSSs and then verify the result with an lspprcpath command. You have to issue the
mkpprcpath command for each LSS pair; see Example 14-6.
Example 14-6 Create Metro Mirror paths and list them
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 1a i0003:i0003 i0102:i0103
Date/Time: November 17, 2005 6:19:43 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:1a successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 15
-tgtlss 1b i0003:i0003 i0102:i0103
Date/Time: November 17, 2005 6:20:54 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 15:1b successfully established.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 6:21:16 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
14 1A Success FF1A I0003 I0003
500507630EFFFE16
14 1A Success FF1A I0102 I0103
500507630EFFFE16
15 1B Success FF1B I0003 I0003
500507630EFFFE16
15 1B Success FF1B I0102 I0103
500507630EFFFE16
With the lspprcpath command, you can use the -fullid command flag to display the fully
qualified DS6000 Storage Image ID in the command output; see Example 14-7 on page 143.
142
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 14-7 List paths with DS6000 Storage Image ID
dscli> lspprcpath -fullid 14-15
Date/Time: November 17, 2005 6:42:36 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src
Tgt
State
SS
Port
Attached Port
Tgt WWNN
===================================================================================================================
IBM.1750-1300819/14 IBM.1750-1300247/1A Success FF1A IBM.1750-1300819/I0003 IBM.1750-1300247/I0003 500507630EFFFE16
IBM.1750-1300819/14 IBM.1750-1300247/1A Success FF1A IBM.1750-1300819/I0102 IBM.1750-1300247/I0103 500507630EFFFE16
IBM.1750-1300819/15 IBM.1750-1300247/1B Success FF1B IBM.1750-1300819/I0003 IBM.1750-1300247/I0003 500507630EFFFE16
IBM.1750-1300819/15 IBM.1750-1300247/1B Success FF1B IBM.1750-1300819/I0102 IBM.1750-1300247/I0103 500507630EFFFE16
14.5.5 Create Metro Mirror pairs
After creating the paths, you can establish Metro Mirror volume pairs. Do this by using the
mkpprc command and verifying the result with the lspprc command; see Example 14-8.
When creating a Metro Mirror pair, you have to specify -type mmir parameter with the mkpprc
command.
Example 14-8 Create Metro Mirror pairs and verify the result
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 17, 2005 6:23:17 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1400:1A00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1401:1A01 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1500:1B00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1501:1B01 successfully created.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 6:23:28 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
===================================================================================================
1400:1A00 Copy Pending Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Copy Pending Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Copy Pending Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Copy Pending Metro Mirror 15
unknown
Disabled
Invalid
Once the Metro Mirror source and target volumes have been synchronized, the volume state
changes to Full Duplex from Copy Pending; see Example 14-9.
Example 14-9 List status after Metro Mirror initial copy completes
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 6:26:38 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
The states of full duplex and copy pending indicate the Metro Mirror source state. In the case
of the target state, the states are Target Full Duplex and Target Copy Pending; see
Example 14-10. You have to give this command to the DS SMC connected to DS6000#2,
which is the Metro Mirror target.
Example 14-10 lspprc for Metro Mirror target volumes
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 6:23:57 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==========================================================================================================
1400:1A00 Target Copy Pending Metro Mirror 14
unknown
Disabled
Invalid
Chapter 14. Metro Mirror interfaces and examples
143
1401:1A01 Target Copy Pending 1500:1B00 Target Copy Pending 1501:1B01 Target Copy Pending -
Metro Mirror 14
Metro Mirror 15
Metro Mirror 15
unknown
unknown
unknown
Disabled
Disabled
Disabled
Invalid
Invalid
Invalid
<< After the synchronization >>
dscli>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 6:27:00 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
In the copy pending state, you can check the data transfer status of the Metro Mirror initial
copy by using the lspprc -l command. Out Of Sync Tracks shows the remaining tracks to be
sent to the target volume. The size of the logical track for FB volume for the DS6000 is 64 KB.
And, you also can use the lspprc -fullid command to show the DS6000 Storage Image ID
in the command output; see Example 14-11.
Example 14-11 lspprc -l and lspprc -fullid for Metro Mirror pairs
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 17, 2005 6:23:40 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs)
Critical Mode First Pass Stats
==========================================================================================================================================
1400:1A00 Copy Pending Metro Mirror 41472
Disabled Disabled
invalid
14
unknown
Disabled
Invalid
1401:1A01 Copy Pending Metro Mirror 42106
Disabled Disabled
invalid
14
unknown
Disabled
Invalid
1500:1B00 Copy Pending Metro Mirror 33616
Disabled Disabled
invalid
15
unknown
Disabled
Invalid
1501:1B01 Copy Pending Metro Mirror 33856
Disabled Disabled
invalid
15
unknown
Disabled
Invalid
dscli>
dscli> lspprc -fullid 1400-1401 1500-1501
Date/Time: November 17, 2005 6:24:16 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS
Timeout (secs) Critical Mode First Pass
Status
==========================================================================================================================================
IBM.1750-1300819/1400:IBM.1750-1300247/1A00 Copy Pending Metro Mirror IBM.1750-1300819/14 unknown
Disabled
Invalid
IBM.1750-1300819/1401:IBM.1750-1300247/1A01 Copy Pending Metro Mirror IBM.1750-1300819/14 unknown
Disabled
Invalid
IBM.1750-1300819/1500:IBM.1750-1300247/1B00 Full Duplex Metro Mirror IBM.1750-1300819/15 unknown
Disabled
Invalid
IBM.1750-1300819/1501:IBM.1750-1300247/1B01 Full Duplex Metro Mirror IBM.1750-1300819/15 unknown
Disabled
Invalid
14.6 Remove Metro Mirror environment using DS CLI
In this section, we show how to terminate the Metro Mirror environment that was set up in the
previous sections. We go through the following steps:
1. Remove Metro Mirror pairs.
2. Remove logical paths.
14.6.1 Remove Metro Mirror pairs
The rmpprc command removes the volume pair relationships; see Example 14-12. You can
use the -quiet parameter to turn off the confirmation prompt for this command.
Example 14-12 Removing Metro Mirror pairs
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 7:03:18 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
144
IBM System Storage DS6000 Series: Copy Services in Open Environments
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> rmpprc -remotedev IBM.1750-1300247 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 17, 2005 7:06:06 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship
1400-1401:1a00-1a01:? [y/n]:y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully withdrawn.
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship
1500-1501:1b00-1b01:? [y/n]:y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully withdrawn.
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully withdrawn.
You can add the -at tgt parameter to the rmpprc command to remove only the Metro Mirror
target volume; see Example 14-13. The commands are given to the DS SMC connected to
DS6000#2, which is the Metro Mirror target.
Example 14-13 rmpprc with -at tgt
dscli> lspprc 1a00
Date/Time: November 17, 2005 7:10:37 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
dscli>
dscli> rmpprc -remotedev IBM.1750-1300247 -quiet -at tgt 1400:1a00
Date/Time: November 17, 2005 7:11:53 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
dscli>
dscli> lspprc 1a00
Date/Time: November 17, 2005 7:12:15 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lspprc: No Remote Mirror and Copy found.
Example 14-14 shows the Metro Mirror source volume status after the rmpprc -at tgt
command completed and the result of a rmpprc -at src command. In this case, there are still
available paths. Therefore, the source status changed after the rmpprc -at tgt command
completed. If there are no available paths, the state of the Metro Mirror source volumes is
preserved. You have to give the command to the DS SMC connected to DS6000#1, which is
the Metro Mirror source.
Example 14-14 Metro Mirror source volume status after rmpprc with -at tgt and rmpprc with -at src
dscli> lspprc 1400
Date/Time: November 17, 2005 7:09:32 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
<< After rmpprc -at tgt command completes >>
dscli> lspprc 1400
Date/Time: November 17, 2005 7:13:19 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
========================================================================================================
1400:1A00 Suspended Simplex Target Metro Mirror 14
unknown
Disabled
Invalid
dscli>
dscli> rmpprc -remotedev IBM.1750-1300247 -quiet -at src 1400:1a00
Chapter 14. Metro Mirror interfaces and examples
145
Date/Time: November 17, 2005 7:14:38 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
dscli>
dscli> lspprc 1400
Date/Time: November 17, 2005 7:14:46 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprc: No Remote Mirror and Copy found.
14.6.2 Remove paths
The rmpprcpath command removes paths. Before removing the paths, you have to remove all
remote mirror pairs which are using the paths or you have to use the -force parameter with
the rmpprcpath command. See Example 14-15.
Example 14-15 Removing paths
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 7:24:34 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
CMUC00234I lspprc: No Remote Mirror and Copy found.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 7:24:48 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
14 1A Success FF1A I0003 I0003
500507630EFFFE16
14 1A Success FF1A I0102 I0103
500507630EFFFE16
15 1B Success FF1B I0003 I0003
500507630EFFFE16
15 1B Success FF1B I0102 I0103
500507630EFFFE16
dscli>
dscli> rmpprcpath -remotedev IBM.1750-1300247 14:1a 15:1b
Date/Time: November 17, 2005 7:24:56 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
CMUC00152W rmpprcpath: Are you sure you want to remove the Remote Mirror and Copy path
14:1a:? [y/n]:y
CMUC00150I rmpprcpath: Remote Mirror and Copy path 14:1A successfully removed.
CMUC00152W rmpprcpath: Are you sure you want to remove the Remote Mirror and Copy path
15:1b:? [y/n]:y
CMUC00150I rmpprcpath: Remote Mirror and Copy path 15:1B successfully removed.
If you do not remove the Metro Mirror pairs which are using the paths, the rmpprcpath
command fails. See Example 14-16.
Example 14-16 Removing paths that still have Metro Mirror pairs
dscli> lspprc 1400
Date/Time: November 17, 2005 7:33:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
dscli>
dscli> rmpprcpath -quiet -remotedev IBM.1750-1300247 14:1a
Date/Time: November 17, 2005 7:33:54 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUN03070E rmpprcpath: 14:1A: Copy Services operation failure: pairs remain
If you want to remove logical paths while you still have Metro Mirror pairs, you can use the
-force parameter; see Example 14-18 on page 147. After the path has been removed, the
Metro Mirror pair is still in full duplex state until the Metro Mirror source receives I/O from the
servers. When I/O arrives to the Metro Mirror source, the source volume becomes suspended.
146
IBM System Storage DS6000 Series: Copy Services in Open Environments
If you set the Consistency Group option for the LSS in which the Metro Mirror volumes reside,
I/Os from the servers are held with queue full status for the specified timeout value.
Example 14-17 Removing paths while you still have Metro Mirror pairs with force parameter
dscli> lspprc 1400
Date/Time: November 17, 2005 7:33:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
dscli>
dscli> rmpprcpath -quiet -remotedev IBM.1750-1300247 14:1a
Date/Time: November 17, 2005 7:33:54 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUN03070E rmpprcpath: 14:1A: Copy Services operation failure: pairs remain
dscli>
dscli> rmpprcpath -force -quiet -remotedev IBM.1750-1300247 14:1a
Date/Time: November 17, 2005 7:35:19 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00150I rmpprcpath: Remote Mirror and Copy path 14:1A successfully removed.
dscli>
dscli> lspprcpath 14
Date/Time: November 17, 2005 7:35:41 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprcpath: No Remote Mirror and Copy Path found.
dscli>
dscli> lspprc 1400
Date/Time: November 17, 2005 7:36:15 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
14.7 Manage the Metro Mirror environment with the DS CLI
In this section, we show how we can manage the Metro Mirror environment by suspending
and resuming Metro Mirror pairs and changing logical paths.
14.7.1 Suspend and resume Metro Mirror data transfer
The pausepprc command stops Metro Mirror from transferring data to the target volumes.
After this command completes, the Metro Mirror pair becomes suspended. I/Os from the
servers complete at the Metro Mirror source volumes without sending those updates to their
target volumes; see Example 14-18.
Example 14-18 Suspending Metro Mirror data transfer
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:13:01 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> pausepprc -remotedev IBM.1750-1300247
1400-1401:1a00-1a01
Date/Time: November 17, 2005 9:15:54 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully paused.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:16:02 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
Chapter 14. Metro Mirror interfaces and examples
147
=======================================================================================================
1400:1A00 Suspended
Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended
Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
Because the source DS6000 keeps track of all changed data on the source volume, you can
resume Metro Mirror operations at a later time. The resumepprc command resumes a Metro
Mirror relationship for a volume pair and restarts transferring data. You have to specify the
remote mirror and copy type, such as Metro Mirror or Global Copy with the -type parameter;
see Example 14-19.
When resuming the Metro Mirror pairs, the state of the pairs is copy pending, and the way the
copy is done, data consistency at the target volumes is not kept. Therefore, you have to take
specific action to keep data consistency at the recovery site while resuming Metro Mirror
pairs. Taking an initial FlashCopy at the recovery site is one of the ways to keep data
consistent.
Example 14-19 Resuming Metro Mirror pairs
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:16:02 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass
Status
======================================================================================================
1400:1A00 Suspended
Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended
Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> resumepprc -type mmir -remotedev IBM.1750-1300247
1400-1401:1a00-1a01
Date/Time: November 17, 2005 9:17:09 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully resumed.
This message is being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully resumed.
This message is being returned before the copy completes.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:17:19 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
14.7.2 Add and remove paths
You can use the mkpprcpath command to add and reduce the number of paths associating
LSS pairs.
In Example 14-20, for each LSS pair (14:1a and 15:1b), we remove one path (I0102:I0103)
from the existing two paths (I0003:I0003,I0102:I0103).
Example 14-20 Removing paths
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 9:21:54 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
14 1A Success FF1A I0003 I0003
500507630EFFFE16
148
IBM System Storage DS6000 Series: Copy Services in Open Environments
14 1A Success FF1A I0102 I0103
15 1B Success FF1B I0003 I0003
15 1B Success FF1B I0102 I0103
dscli>
500507630EFFFE16
500507630EFFFE16
500507630EFFFE16
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 1a i0003:i0003
Date/Time: November 17, 2005 9:22:35 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:1a successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 15
-tgtlss 1b i0003:i0003
Date/Time: November 17, 2005 9:23:12 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 15:1b successfully established.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 9:23:28 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
14 1A Success FFFF I0003 I0003
500507630EFFFE16
15 1B Success FFFF I0003 I0003
500507630EFFFE16
We add one path (I0102:I0103) to each LSS pair (14:1a and 15:1b) to the existing path
(I0003:I0003); see Example 14-21.
Example 14-21 Adding paths
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 9:23:28 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
14 1A Success FFFF I0003 I0003
500507630EFFFE16
15 1B Success FFFF I0003 I0003
500507630EFFFE16
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 1a i0003:i0003 i0102:i0103
Date/Time: November 17, 2005 9:23:52 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:1a successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 15
-tgtlss 1b i0003:i0003 i0102:i0103
Date/Time: November 17, 2005 9:24:09 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 15:1b successfully established.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 9:24:21 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
14 1A Success FF1A I0003 I0003
500507630EFFFE16
14 1A Success FF1A I0102 I0103
500507630EFFFE16
15 1B Success FF1B I0003 I0003
500507630EFFFE16
15 1B Success FF1B I0102 I0103
500507630EFFFE16
14.8 Failover and Failback functions to switch sites
In this section, we describe the use of the Metro Mirror Failover and Failback functions in
planned and unplanned outage scenarios.
Chapter 14. Metro Mirror interfaces and examples
149
Planned outages
The planned outage procedures rely on two facts:
򐂰 Metro Mirror source and target volumes are in a consistent and current state.
򐂰 Both DS6000s are functional and reachable.
You can swap sites without any full copy operation by combining Metro Mirror initialization
modes.
Unplanned outages
In contrast to the assumptions for planned outages, the situation in a disaster is more difficult:
򐂰 In an unplanned outage situation, only the DS6000 at the recovery site is functioning.
The production site DS6000 might be lost or unreachable.
򐂰 In an unplanned situation, volumes at the production and recovery site might be in
different states.
򐂰 As opposed to a planned situation in which you can stop all I/Os at the production site to
make all volumes across the recovery site reach a consistent status, you cannot do this in
an unplanned situation. If you are not using Consistency Groups, and, for example, the
power fails, you can only assume consistency at the level of a single volume pair, not at
the application level.
14.8.1 Metro Mirror Failover function
In our example, we have used the Metro Mirror environment shown in Figure 14-4. We have
four Metro Mirror source volumes in DS6000#1 (serial number 1300819) at the production
site, and four Metro Mirror target volumes in DS6000#2 (serial number 1300247) at the
recovery site. We call the volumes at the production site, A volumes, which initially are Metro
Mirror source volumes. We call the volumes at the recovery site, B volumes, which initially are
Metro Mirror target volumes. During operations to switch sites, A and B volumes become
alternatively source and target; therefore, we add the A and B terminology for clarification.
Source
(A volumes)
LSS14
1400
1401
LSS15
1500
1501
Metro Mirror paths
Metro Mirror Pairs
FCP links
Metro Mirror Pairs
LSS1A
1A00
1A01
LSS1B
1B00
1B01
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 14-4 Metro Mirror environment for example of switching sites
150
Target
(B volumes)
IBM System Storage DS6000 Series: Copy Services in Open Environments
A planned site switch using the Metro Mirror Failover function involves the following steps;
if the site switch is because of an unplanned outage, then the procedure starts at step 4 on
page 152:
1. When the planned outage window is reached, the applications at the production site (A)
must be quiesced to cease all write I/O activity. This way, there are no more updates to
the source volumes. Depending on the host operating system, it might be necessary to
dismount the source volumes.
2. Next, make sure that all Metro Mirror pairs are in full duplex state. It is better to check on
both sites, on DS6000#1 and on DS6000#2. In order to do this, you have to issue the lspprc
command to each DS SMC; see Example 14-22.
Example 14-22 Checking Metro Mirror volumes state at the production and at the recovery sites
<< DS6000#1 >>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:37:11 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
<< DS6000#2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 9:38:50 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
3. You can give the commands freezepprc and unfreezepprc to ensure no data can
possibly be transferred to the target volumes (B volumes); see Example 14-23. This
operation is optional, because the application at the production site has been quiesced,
therefore, no data is sent to the target volumes.
Note: The freezepprc command is an LSS level command. This means that all remote
mirror and copy pairs, Metro Mirror and Global Copy, in the particular LSS are affected by
this command. This command also removes the paths between the LSS pair.
Example 14-23 freezepprc and unfreezepprc
dscli> freezepprc -remotedev IBM.1750-1300247 14:1a 15:1b
Date/Time: November 17, 2005 9:41:33 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00161W freezepprc: Remote Mirror and Copy consistency group 14:1A successfully created.
CMUC00161W freezepprc: Remote Mirror and Copy consistency group 15:1B successfully created.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:41:45 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
================================================================================================
1400:1A00 Suspended Freeze Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Freeze Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Freeze Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Suspended Freeze Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> unfreezepprc -remotedev IBM.1750-1300247 14:1a 15:1b
Date/Time: November 17, 2005 9:42:04 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Chapter 14. Metro Mirror interfaces and examples
151
CMUC00198I unfreezepprc: Remote Mirror and Copy pair 14:1A successfully thawed.
CMUC00198I unfreezepprc: Remote Mirror and Copy pair 15:1B successfully thawed.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:42:13 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
================================================================================================
1400:1A00 Suspended Freeze Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Freeze Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Freeze Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Suspended Freeze Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> lspprcpath 14-15
Date/Time: November 17, 2005 9:42:39 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprcpath: No Remote Mirror and Copy Path found.
4. Give a failoverpprc command to the DS SMC connected with DS6000#2.
You have to issue the failoverpprc command according to the roles the volumes have
after the command is completed. In our example, you have to specify the B volumes as the
source volumes. After the failoverpprc command is successfully executed, the B
volumes become new source volumes in suspended state; see Example 14-24. The state
of the A volumes is preserved.
Note: In case of an unplanned outage, before you issue the failoverpprc command, you
might consider disconnecting the physical links between the production and the recovery
sites. This ensures that no unexpected data transfer to the recovery site occurs at all.
Example 14-24 failoverpprc command
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 9:42:22 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> failoverpprc -remotedev IBM.1750-1300819
Date/Time: November 17, 2005 9:46:14 PM JST IBM
CMUC00196I failoverpprc: Remote Mirror and Copy
CMUC00196I failoverpprc: Remote Mirror and Copy
CMUC00196I failoverpprc: Remote Mirror and Copy
CMUC00196I failoverpprc: Remote Mirror and Copy
dscli>
-type mmir 1a00-1a01:1400-1401 1b00-1b01:1500-1501
DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
pair 1A00:1400 successfully reversed.
pair 1A01:1401 successfully reversed.
pair 1B00:1500 successfully reversed.
pair 1B01:1501 successfully reversed.
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 9:46:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1A00:1400 Suspended Host Source Metro Mirror 1A
unknown
Disabled
Invalid
1A01:1401 Suspended Host Source Metro Mirror 1A
unknown
Disabled
Invalid
1B00:1500 Suspended Host Source Metro Mirror 1B
unknown
Disabled
Invalid
1B01:1501 Suspended Host Source Metro Mirror 1B
unknown
Disabled
Invalid
Note: The lspprc command shows Target in the State column only for a target volume.
There is no indication for a source volume.
152
IBM System Storage DS6000 Series: Copy Services in Open Environments
5. Create paths in the direction of recovery site to production site (B → A); see
Example 14-25. You have to give the mkpprcpath command to the DS SMC connected to
DS6000#2.
Although it is not strictly necessary to reverse the paths, we recommend you do it so that
you have a well-defined situation at the end of the procedure. Additionally, you need the
paths to transfer the updates back to the production site.
Example 14-25 Create paths from B to A
dscli> lsavailpprcport -l -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F 1a:14
Date/Time: November 17, 2005 9:48:09 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0103
I0102
FCP NA
NA
dscli>
dscli> lsavailpprcport -l -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F 1b:15
Date/Time: November 17, 2005 9:48:19 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0103
I0102
FCP NA
NA
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F -srclss 1a
-tgtlss 14 i0003:i0003 i0103:i0102
Date/Time: November 17, 2005 9:48:47 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00149I mkpprcpath: Remote Mirror and Copy path 1a:14 successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F -srclss 1b
-tgtlss 15 i0003:i0003 i0103:i0102
Date/Time: November 17, 2005 9:49:01 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00149I mkpprcpath: Remote Mirror and Copy path 1b:15 successfully established.
dscli>
dscli> lspprcpath 1a-1b
Date/Time: November 17, 2005 9:49:21 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
1A 14 Success FF14 I0003 I0003
500507630EFE028F
1A 14 Success FF14 I0103 I0102
500507630EFE028F
1B 15 Success FF15 I0003 I0003
500507630EFE028F
1B 15 Success FF15 I0103 I0102
500507630EFE028F
6. Depending on your operating system, it might be necessary to rescan Fibre Channel
devices (to remove device objects for the recovery site volumes and recognize the new
sources) and mount the new source volumes (B volumes).
Start all applications at the recovery site (B).
Now that the applications have started, Metro Mirror starts keeping track of the updated
data on the new source volumes at B.
Summary of the Failover procedure
Briefly, this is the procedure we followed to switch to the recovery site:
1.
2.
3.
4.
5.
6.
Stop applications at the production site.
Verify the Metro Mirror volume pair status.
freezepprc and unfreezepprc (optional).
failoverpprc B to A (to DS6000#2).
mkpprcpath B to A (to DS6000#2).
Start applications at the recovery site.
Chapter 14. Metro Mirror interfaces and examples
153
14.8.2 Metro Mirror Failback function
Once the production site has been restored, you must move your application back. At this
point, we assume that:
򐂰
򐂰
򐂰
򐂰
򐂰
Applications are updating the source volumes (B volumes) at the recovery site.
During operation at the recovery site, data has not been replicated from B to A.
Both DS6000s (source and target) are functional and reachable.
Paths are already established from the recovery site to the production site.
Volumes at the recovery site are in the suspended state (source). Volumes at the
production site are also in the suspended state (source) if you executed the optional step 3
on page 153 in the previous procedure.
A switchback using the Metro Mirror Failback function involves the following steps:
1. Using the lspprcpath command, verify that the paths from the recovery site to the
production site are available. You give this command to the DS SMC connected to
DS6000#2; see Example 14-26. You can check if you have paths between the correct
DS6000s with the -fullid parameter.
Example 14-26 Verify paths between recovery and production sites
dscli> lspprcpath -fullid 1a-1b
Date/Time: November 17, 2005 9:49:43 PM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300247
Src
Tgt
State
SS
Port
Attached Port
Tgt WWNN
===================================================================================================================
IBM.1750-1300247/1A IBM.1750-1300819/14 Success FF14 IBM.1750-1300247/I0003 IBM.1750-1300819/I0003 500507630EFE028F
IBM.1750-1300247/1A IBM.1750-1300819/14 Success FF14 IBM.1750-1300247/I0103 IBM.1750-1300819/I0102 500507630EFE028F
IBM.1750-1300247/1B IBM.1750-1300819/15 Success FF15 IBM.1750-1300247/I0003 IBM.1750-1300819/I0003 500507630EFE028F
IBM.1750-1300247/1B IBM.1750-1300819/15 Success FF15 IBM.1750-1300247/I0103 IBM.1750-1300819/I0102 500507630EFE028F
2. If you did not reverse the paths before (step 5 on page 153 of the previous procedure),
now you have to establish paths from the recovery to the production site before running
the failbackpprc command.
3. Run the failbackpprc command from the recovery site to the production site. You have to
give this command to the DS SMC connected to DS6000#2. The failbackpprc command
copies all the modified tracks from the B volumes to the A volumes. See Example 14-27.
You have to issue the failbackpprc command, according to the roles the volumes have
after the command is completed. In our example, you have to specify the B volumes as
the source volumes and the A volumes as the target volumes. After the failbackpprc
command is successfully executed, the B volumes become source volumes in copy
pending state, and the A volumes become target volumes in target copy pending state.
Note: When issuing the failbackpprc command, if you specify the A volume as the
source and the B volume as the target, and you give the command to the DS SMC
connected to the DS6000#1, then data is copied from A to B.
Example 14-27 failbackpprc command
<< DS6000#2 >>
dscli> lspprcpath 1a-1b
Date/Time: November 17, 2005 9:49:21 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
1A 14 Success FF14 I0003 I0003
500507630EFE028F
1A 14 Success FF14 I0103 I0102
500507630EFE028F
1B 15 Success FF15 I0003 I0003
500507630EFE028F
1B 15 Success FF15 I0103 I0102
500507630EFE028F
dscli> lspprc 1a00-1a01 1b00-1b01
154
IBM System Storage DS6000 Series: Copy Services in Open Environments
Date/Time: November 17, 2005 9:50:31 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1A00:1400 Suspended Host Source Metro Mirror 1A
unknown
Disabled
Invalid
1A01:1401 Suspended Host Source Metro Mirror 1A
unknown
Disabled
Invalid
1B00:1500 Suspended Host Source Metro Mirror 1B
unknown
Disabled
Invalid
1B01:1501 Suspended Host Source Metro Mirror 1B
unknown
Disabled
Invalid
dscli>
dscli> failbackpprc -remotedev IBM.1750-1300819 -type mmir 1a00-1a01:1400-1401
1b00-1b01:1500-1501
Date/Time:
CMUC00197I
CMUC00197I
CMUC00197I
CMUC00197I
November 17, 2005 9:53:50 PM JST IBM
failbackpprc: Remote Mirror and Copy
failbackpprc: Remote Mirror and Copy
failbackpprc: Remote Mirror and Copy
failbackpprc: Remote Mirror and Copy
DSCLI Version:
pair 1A00:1400
pair 1A01:1401
pair 1B00:1500
pair 1B01:1501
5.1.0.204 DS: IBM.1750-1300247
successfully failed back.
successfully failed back.
successfully failed back.
successfully failed back.
failbackpprc initialization characteristics
The failbackpprc initialization mode re-synchronizes the volumes. The following apply:
򐂰 If a volume at the production site is in simplex state, all of the data for that volume is
copied back from the recovery site to the production site.
򐂰 If a volume at the production site is in the full duplex or suspended state and without
changed tracks, only the modified data on the volume at the recovery site is copied back
to the volume at the production site.
򐂰 If a volume at the production site is in a suspended state and has tracks on which data has
been written, then both the tracks changed on the production site and the tracks marked at
the recovery site are copied back.
򐂰 Finally, the volume at the production site becomes a write-inhibited target volume. This
action is performed on an individual volume basis.
Notes on the failbackpprc command
If the server at the production site is still online and accessing the disk, or a crash happens,
so that the SCSI persistent reserve is still set on the source disk (A), the failbackpprc
command fails; see Example 14-28. In this case, the server at the production site locks the
target with a SCSI persistent reserve. This situation can be reset with the varyoffvg
command (in this case on AIX), and the failbackpprc command completes successfully.
There is a -resetreserve parameter for the failbackpprc command. This parameter resets
the reserved status, so the operation can complete. In a situation after a real disaster, you
can use this parameter because the server might go down while the SCSI persistent reserve
was set on the A volume. In a situation after a planned site switch, you must not use this
parameter because the server at the production site still owns the A volume, and might be
using it, while the Failback operation suddenly changes the contents of the volume. This can
corrupt the server file system.
Example 14-28 failbackpprc command fails when A volumes are online
dscli> failbackpprc -dev IBM.1750-13AAVCA -remotedev IBM.1750-13AAGXA -type mmir 1902:1900
Date/Time: June 14, 2005 12:20:01 PM EDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAVCA
CMUN03103E failbackpprc: 1902:1900: Copy Services operation failure: volume long busy
<< After performing varyoffvg A volumes from the AIX servers at the production site >>
dscli> failbackpprc -dev IBM.1750-13AAVCA -remotedev IBM.1750-13AAGXA -type mmir 1902:1900
Date/Time: June 14, 2005 12:15:03 PM EDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAVCA
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1902:1900 successfully failed back.
Chapter 14. Metro Mirror interfaces and examples
155
Let us continue with the switchback procedure:
4. Wait until the Metro Mirror pairs become synchronized. When the pairs are synchronized,
the state of the source is full duplex and the state of the target volumes is target full
duplex; see Example 14-29.
Example 14-29 Confirm synchronization has completed
<< DS6000#2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 9:53:57 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1A00:1400 Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1A01:1401 Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1B00:1500 Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
1B01:1501 Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
<< DS6000#1 >>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:54:42 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1A00:1400 Target Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1A01:1401 Target Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1B00:1500 Target Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
1B01:1501 Target Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
5. Before returning to normal operation, the applications, still updating B volumes at the
recovery site, must be quiesced to cease all write I/O from updating the B volumes.
Depending on the host operating system, it might be necessary to dismount the B
volumes.
6. You should now execute one more failoverpprc command. At this time, you have to
specify the A volumes as the source volumes and the B volumes as the target volumes.
You have to give this command to the DS SMC connected to the DS6000#1.
This operation changes the state of the A volumes from target full duplex to (source)
suspended. The state of the B volumes is preserved; see Example 14-30.
Example 14-30 failoverpprc to convert A volumes to source suspended
<< DS6000#1 >>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:58:30 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1A00:1400 Target Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1A01:1401 Target Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1B00:1500 Target Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
1B01:1501 Target Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
dscli>
dscli> failoverpprc -remotedev IBM.1750-1300247 -type mmir 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time:
CMUC00196I
CMUC00196I
CMUC00196I
CMUC00196I
dscli>
November 17, 2005 9:59:18 PM JST IBM
failoverpprc: Remote Mirror and Copy
failoverpprc: Remote Mirror and Copy
failoverpprc: Remote Mirror and Copy
failoverpprc: Remote Mirror and Copy
DSCLI Version:
pair 1400:1A00
pair 1401:1A01
pair 1500:1B00
pair 1501:1B01
5.1.0.204 DS: IBM.1750-1300819
successfully reversed.
successfully reversed.
successfully reversed.
successfully reversed.
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 9:59:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1400:1A00 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
156
IBM System Storage DS6000 Series: Copy Services in Open Environments
1501:1B01 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
<< DS6000#2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 9:59:36 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1A00:1400 Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1A01:1401 Full Duplex Metro Mirror 1A
unknown
Disabled
Invalid
1B00:1500 Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
1B01:1501 Full Duplex Metro Mirror 1B
unknown
Disabled
Invalid
7. Define paths in the direction of the production site to the recovery site (A → B); see
Example 14-31. You have to create paths if you executed the freezepprc command in the
optional step 3 on page 153 of the production site to recovery site switchover procedure
(freezepprc removed the paths).
Example 14-31 Create Metro Mirror paths from A to B
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 1a i0003:i0003 i0102:i0103
Date/Time: November 17, 2005 10:00:16 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:1a successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 15
-tgtlss 1b i0003:i0003 i0102:i0103
Date/Time: November 17, 2005 10:00:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 15:1b successfully established.
8. Then, issue another failbackpprc command. At this time, you have to specify the A
volumes as the source volumes and the B volumes as the target volumes. You have to
give this command to the DS SMC connected to the DS6000#1.
After you have successfully executed the failbackpprc command, the A volumes become
source volumes in the copy pending state and the B volumes become target volumes in
the target copy pending state; see Example 14-32.
Example 14-32 failbackpprc command to restart A to B Metro Mirror operation
dscli> failbackpprc -remotedev IBM.1750-1300247 -type mmir 1400-1401:1a00-1a01
1500-1501:1b00-1b01
Date/Time:
CMUC00197I
CMUC00197I
CMUC00197I
CMUC00197I
November 17, 2005 10:01:23 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
failbackpprc: Remote Mirror and Copy pair 1400:1A00 successfully failed back.
failbackpprc: Remote Mirror and Copy pair 1401:1A01 successfully failed back.
failbackpprc: Remote Mirror and Copy pair 1500:1B00 successfully failed back.
failbackpprc: Remote Mirror and Copy pair 1501:1B01 successfully failed back.
9. Wait until the Metro Mirror pairs are synchronized. Normally, this operation does not take
much time because no data transfer is necessary. After the Metro Mirror pairs are
synchronized, the state of the source volumes (A) becomes full duplex and the state of the
target volumes (B) becomes target full duplex; see Example 14-33.
Example 14-33 After the Metro Mirror pairs have been synchronized
<< DS6000#1 >>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 17, 2005 10:01:31 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
Chapter 14. Metro Mirror interfaces and examples
157
<< DS6000#2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 17, 2005 10:01:50 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Target Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Target Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
10.Depending on your operating system, it might be necessary to re-scan Fibre Channel
devices and mount the new source volumes (A) at the production site.
Start all applications at the production site and check for consistency.
Now that the applications have started, all the write I/Os to the source volumes (A) are
tracked by Metro Mirror. You should verify the application’s integrity.
11.Eventually, terminate the paths from the recovery LSSs to the production LSSs.
Summary of the Failback procedure
Briefly, this is the procedure we followed to switch back to the production site:
1. Verify path status B to A.
2. Eventually mkpprcpath B to A.
3. failbackpprc B to A (to DS6000#2).
4. Wait for volume pair synchronization.
5. Quiesce applications at the recovery site (B).
6. failoverpprc A to B (to DS6000#1).
7. mkpprcpath A to B.
8. failbackpprc A to B (to DS6000#1).
9. Wait for volume pair synchronization.
10.Start applications at the production site (A).
11.Eventually, rmpprcpath B to A.
14.9 DS Storage Manager GUI examples
This section presents examples of common activities using the Web-based DS Storage
Manager GUI.
14.9.1 Create paths
This example shows you how to create a path, step-by-step, beginning on the Paths
Real-time start menu. Select the correct Storage Complex, the correct Storage Unit, and
select your source LSS from where you want to create a path. You see all available paths and
the status of each path. See Figure 14-5 on page 159. Select Create.
158
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-5 Create path: Paths panel
Now, you select the source LSS from where your path starts and click Next. See Figure 14-6.
Figure 14-6 Create paths: Select source LSS
Now you can select your target Storage Complex, after this, the Storage Unit, and then the
target LSS. Click Next to proceed. See Figure 14-7 on page 160.
Chapter 14. Metro Mirror interfaces and examples
159
Figure 14-7 Create paths: Select target LSS
Select the I/O ports you plan to use for your source LSS. You can use one or more,
depending on the available links and your performance expectations. See Figure 14-8. Click
Next to proceed.
Figure 14-8 Create paths: Select source I/O ports
For each I/O port, you can select your target I/O port separately. Select your target I/O ports;
see Figure 14-9 on page 161.
160
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-9 Create paths: Select target I/O ports
Note: For performance considerations, we recommend that you do not use a single target
port connected to two or more source ports; it is better to have a one to one target port to
source port ratio.
Now, you can create a Consistency Group between the two LSSs. See Figure 14-10.
Information about Consistency Groups and how they work is at 12.4, “Consistency Group
function” on page 118. Click Next.
Figure 14-10 Create paths: Select path options
You can verify your selection and establish the path by clicking Finish. See Figure 14-11 on
page 162.
Chapter 14. Metro Mirror interfaces and examples
161
Figure 14-11 Create paths: Verification
After you finish, you are again in the Paths panel and you see the result of your configuration.
See Figure 14-12.
Figure 14-12 Create path: See result
14.9.2 Add paths
There are situations when you need to add additional paths to Metro Mirror:
򐂰 Getting more redundancy.
򐂰 Getting more bandwidth.
򐂰 Changing the ports you are using for Metro Mirror. With the GUI, you first must increase
the number of paths using the new ports. Only then can you delete the paths over the
ports you no longer want to use. You always have to be observant to make sure that you
have sufficient bandwidth for your applications.
On the Paths panel, select the correct Storage Complex, the correct Storage Unit, and your
source LSS. Now, you see the paths from your source LSS. See Figure 14-13 on page 163.
162
IBM System Storage DS6000 Series: Copy Services in Open Environments
Select Create on the pull-down menu.
Figure 14-13 Add paths: Paths panel
Select your source LSS and click Next. See Figure 14-14.
Figure 14-14 Add paths: Select source LSS
Select the Storage Complex, the Storage Unit, and your target LSS. See Figure 14-15 on
page 164. Click Next to proceed.
Chapter 14. Metro Mirror interfaces and examples
163
Figure 14-15 Add paths: Select target LSS
Select the additional ports for the additional path or paths. See Figure 14-16. Click Next to
proceed.
Figure 14-16 Add paths: Select source I/O ports
Select your target I/O port. See Figure 14-17 on page 165. Click Next to proceed.
164
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-17 Add paths: Select target I/O ports
Note: For performance reasons, we recommend you use different ports from the ones you
are already using for the existing paths.
This panel allows you to create a Consistency Group between the LSSs. See Figure 14-18.
For information about Consistency Groups and how they work, see 12.4, “Consistency Group
function” on page 118. Click Next to proceed.
Figure 14-18 Add paths: Select path options
You can verify your selection and establish the path by clicking Finish. See Figure 14-19 on
page 166.
Chapter 14. Metro Mirror interfaces and examples
165
Figure 14-19 Add paths: Verification
After you finish, you are again in the Paths panel and you see the result of your configuration.
See Figure 14-20.
Note: These paths are in one direction. You might also need to define paths in the
opposite direction. You can use the same Fibre Channel connection.
Figure 14-20 Add paths: Result
14.9.3 Change options
After creating paths, it is possible to change the options of the paths, and also the timeout
values. Select the correct Storage Complex, the correct Storage Unit, your source LSS, and
the path or paths you want to change. Select LSS copy options on the pull-down menu. See
Figure 14-21 on page 167.
166
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-21 LSS copy options: Paths panel
On the LSS Copy Options panel, you can change the timeouts, enable the Critical mode,
and also enable the Consistency Group timeout. See Figure 14-22. For information about
these values, see Chapter 12, “Metro Mirror options and configuration” on page 115.
Figure 14-22 LSS copy options: Change
14.9.4 Delete paths
This example shows how to remove paths step-by-step. Remember that when you delete all
paths, you lose the communication between your Metro Mirror pairs.
Select the Storage Complex you want to delete, the Storage Unit you want to delete, your
source LSS, and the path or paths you want to delete. See Figure 14-23 on page 168. Select
Delete on the pull-down menu.
Chapter 14. Metro Mirror interfaces and examples
167
Figure 14-23 Delete paths: Paths panel
A warning message appears. See Figure 14-24.
Figure 14-24 Delete paths warning
Note: This warning is important, because deleting paths removes bandwidth. After path
deletion, your application might not perform as well, especially at peak times.
When finished, you are again at the Paths panel and you see the result of your command.
See Figure 14-25. If the path is still listed, click Refresh, and check again.
Figure 14-25 Delete paths: Result
168
IBM System Storage DS6000 Series: Copy Services in Open Environments
14.9.5 Create volume pairs
In this example, we see how to create the Metro Mirror volume pairs.
The following are things to remember when you create the volume pairs:
򐂰 All volumes, on the source and also on the target, must exist before starting to create the
Metro Mirror pairs.
򐂰 Source and target volumes should be of the same size. The target volume can be bigger
than the source volume, but if this is the case, you cannot reverse the direction of the pair.
So different sizes of source and target volumes make sense only for migration purposes.
򐂰 Usually before you start creating the Metro Mirror pairs, you have to define the paths
between the source LSS and target LSS, as explained in “Create paths” on page 158.
To start creating volume pairs, you follow these steps from the DS Storage Manager GUI:
1. Select Real-time manager.
2. Select Copy services.
3. Click Metro Mirror.
4. The Metro Mirror panel is displayed; see Figure 14-26. Here you select the source, the
Storage Complex, the Storage Unit, the Resource type, and under the section of the
resource type, possibly your source volumes.
Possible options are:
򐂰
򐂰
򐂰
򐂰
LSS and the LSS number
Host attachment and Host attachment name
Volume group and Volume group name
Show All volumes and All FB volumes or All CKD volumes
Figure 14-26 Create Metro Mirror pairs: Metro Mirror panel
If you select Automated volume pair assignment, the system later automatically selects
volumes of the same size of the target LSS to assign them to the source volumes.
If you select Manual volume pair assignment, you have to assign a target volume to each
source volume, one after the other.
In this case, we select Automated volume pair assignment, see Figure 14-27 on page 170.
Chapter 14. Metro Mirror interfaces and examples
169
Figure 14-27 Create Metro Mirror pair: Volume Pairing Method
You can now select your source volumes and click Next to proceed, see Figure 14-28. If you
have not established paths between the source and target LSSs, you must do this first before
clicking Next.
Figure 14-28 Create Metro Mirror pair: Select source volumes
Now you can select all target volumes from the available volumes of the target. After your
selection, click Next to proceed, see Figure 14-29 on page 171.
170
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-29 Create Metro Mirror pair: Select target volumes (Auto pairing)
For Metro Mirror, you select the Metro Mirror option. You can also specify additional options
as shown in Figure 14-30. For information about these options, see Chapter 12, “Metro Mirror
options and configuration” on page 115.
Figure 14-30 Create Metro Mirror pair: Select copy options
After selecting the Copy Services options, you can see and verify the Metro Mirror volume
pairs; see Figure 14-31 on page 172. Click Finish to execute this configuration.
Chapter 14. Metro Mirror interfaces and examples
171
Figure 14-31 Create Metro Mirror pair: Verification
You see the result of the command in Figure 14-32. In this example, you can see the state of
the volumes as Copy pending. This indicates that the copy of data from the source volumes to
the target volumes is still in progress. Click Refresh to see if the status has changed.
Figure 14-32 Create Metro Mirror pair: Result after establish
After some period of time, depending on the size and number of volumes, all volumes are in
sync. The status Full duplex displays; see Figure 14-33 on page 173.
172
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-33 Create Metro Mirror pair: Result after sync
14.9.6 Suspend volume pairs
To gain access to the target volumes, or for maintenance of the target Storage Unit, you can
suspend the Metro Mirror pairs. The changes on the volumes are tracked, and after you
resume the connection, only the differences are transferred between the volumes.
Select the Storage Complex, the Storage Unit, the Resource type, and possibly under section
of the resource type, your source volumes. See Figure 14-34.
Possible options are:
򐂰
򐂰
򐂰
򐂰
LSS and the LSS number
Host attachment and Host attachment name
Volume group and Volume group name
Show All volumes and All FB volumes or All CKD volumes
Select the Metro Mirror pairs you want to suspend, and select Suspend on the pull-down
menu.
Figure 14-34 Suspend: Metro Mirror panel
Select whether you want to suspend at the source or the target and click OK; see
Figure 14-35 on page 174.
Chapter 14. Metro Mirror interfaces and examples
173
Figure 14-35 Suspend: Suspend at source or target
You now see the result of your action. The pairs you selected are in suspended mode; see
Figure 14-36.
Figure 14-36 Suspend: Result
14.9.7 Resume volume pairs
To return the suspended volumes back to the full duplex state, you need to resume the Metro
Mirror pair. Be sure that a path between these LSSs is available.
Select the Storage Complex, the Storage Unit, the Resource type, and under the section of
the resource type, your source volumes.
Select the Metro Mirror pairs you want to resume, and select Resume on the pull-down menu
to proceed. See Figure 14-37 on page 175.
174
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-37 Resume: Metro Mirror panel
Verify the Metro Mirror pairs you want to resume. Click OK to proceed; see Figure 14-38.
Figure 14-38 Resume: Verify
You now see the results of your action. The pairs you selected are in Full duplex mode; see
Figure 14-39.
Figure 14-39 Resume: Result
Chapter 14. Metro Mirror interfaces and examples
175
14.9.8 Metro Mirror Failover
In the following example, we have two disk subsystems, a DS8000 (machine type 2107) and a
DS6000 (machine type 1750). Each disk subsystem has two volumes. The DS8000 is the
production unit. The DS6000 is the backup unit. See Table 14-2.
Table 14-2 Failover Failback example Storage Units
Machine serial
Location
Normal state
Volume 1
Volume 2
2105-7503461
Production site (A)
Source
1405
1406
1750-1300247
Backup site (B)
Target
1805
1806
Now we use these two disk subsystems to describe a site switch scenario.
Site switch: Production site to recovery site
The Metro Mirror Failover function changes the target volume (B) to a suspended source
volume, while leaving the original source volume (A) in its current state. This command
succeeds even if the paths are down and volume (A) at the production site is unavailable or
nonexistent. You can then access the former target volume (B). The assumption here is that
you are now running production on what was the original target disk subsystem (B). This
means that changes to the volumes on the recovery site disk subsystem later need to be
copied back to the production site disk subsystem (A).
Note: If you try to access this volume on the same server where the previous source was
or still is, some operating systems have some problems with disks with the same serial
number or signature. In most cases, there is a procedure to do this; see Appendix A,
“Open systems specifics” on page 463.
In this example, we assume the production site has failed. We decide to start production
processing using the backup servers at the recovery site. We need to start using the Metro
Mirror target volumes (B). These volumes must become source volumes, because we are
writing them, and these changes must eventually be mirrored back to the production site.
Following is the procedure we follow:
1. Select Real-time manager.
2. Select Copy services.
3. Click Metro Mirror.
4. Select the target Storage Complex, Storage Unit, Storage Image, and Resource Type.
Possible Resource Type options are:
–
–
–
–
LSS from which you select the LSS number.
Host attachment from which you select the host attachment name.
Volume Group from which you select the Volume Group name.
Show All Volumes from which you select All FB volumes or All CKD volumes.
5. Select the volume pairs you want to Fail over.
6. Now, select Recover Failover from the Select Action pull-down menu, as shown in
Figure 14-40 on page 177. Note that in this example, we display the target machine (serial
13-00247) and select the target volumes. We are not working with the source machine.
176
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-40 Failover: Metro Mirror panel
We now verify the volumes with which we are working. When ready, click OK to proceed as
shown in Figure 14-41.
Figure 14-41 Failover verify
The Failoveroperation now takes place. We can see the result in Figure 14-42. Note that
Figure 14-41 shows the status of the B volumes, which are suspended. The A volumes are still
in full duplex. This is because the Metro Mirror Failover function does not care about the state
of the former source volumes (A). We are now able to start the servers at the backup site, and
they can access the B volumes on the backup site DS6000.
Figure 14-42 Result of a failover
Chapter 14. Metro Mirror interfaces and examples
177
14.9.9 Metro Mirror Failback
We are running our production using servers at the recovery site, which access our backup
disk subsystem (B). We are now getting ready to switch processing back to the production site
(A). The first thing to do is mirror the changes back from the backup site (B) to the production
site (A). We do this using the Metro Mirror Failback function.
Site switch: Recovery site to Production site
Metro Mirror Failback synchronizes the volumes at the production site (A) with the volumes at
the recovery site (B). The (B) volumes were updated while production processing was
temporarily run at the recovery site. Following are the steps:
1. Select Real-time manager.
2. Select Copy services.
3. Click Metro Mirror.
4. Select the source Storage Complex, Storage Unit, Storage Image, and Resource Type.
Possible Resource Type options are:
LSS from which you select the LSS number
Host attachment from which you select the host attachment name
Volume Group from which you select the Volume Group name
Show All Volumes from which you select All FB volumes or All CKD volumes
5. Select the volume pairs you want to fail back.
6. Now select Recover Failback from the Select Action pull-down menu, as shown in
Figure 14-43. Note that in this example, we are still working with the Storage Unit at the
backup site (serial 13-00247).
Figure 14-43 Failback: Metro Mirror panel
We now verify the volumes we have selected. We can suspend the pair after synchronization
and reset the LUN reservation. Depending on whether data changes are taking place on the
previous source volume, the failback command operates; see 14.8, “Failover and Failback
functions to switch sites” on page 149. Click OK to proceed; see Figure 14-44 on page 179.
178
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 14-44 Failback: Verify
During the Failback, changes are copied. You can click Refresh to see if the copy is finished.
When the copying is complete, the volumes at the backup site (B) show as full duplex source
volumes, as depicted in Figure 14-45.
Figure 14-45 Failback: Result
Note: If the Failback fails, check whether your prior active server can still access the new
target volume or has not reset the reserve on the volume, and if your Metro Mirror path is
established between these LSSs.
If we display the status of the volumes at the production site (on serial 75-03461), they appear
as full duplex targets as shown in Figure 14-46 on page 180. The source machine is serial
13-00247 at the backup site.
Chapter 14. Metro Mirror interfaces and examples
179
Figure 14-46 Failback: View from the production site Storage Unit
At this point, we shut down the servers at the backup site. Once they are shutdown, we know
that no more changes are written to the volumes at the backup site. Now we repeat the
Failover and Failback process, but we do so from the production site Storage Unit (A).
In Figure 14-47, we select the volumes on the production Storage Unit and perform a Failover.
Figure 14-47 Starting failover: View on production site DS8000
When the Failover is complete, the volumes on the production site show as suspended Metro
Mirror source volumes. Now, perform a Failback using the production site Storage Unit, as
shown in Figure 14-48.
Figure 14-48 Starting Failback on the production site DS8000
180
IBM System Storage DS6000 Series: Copy Services in Open Environments
When the final Failback is complete, the production site Storage Unit is now the source again
and the backup site Storage Unit is now the target again. This means the production site
Storage Unit (in our example, serial 75-03461) now looks like Figure 14-49. This shows serial
75-03461 as the source machine while serial 13-00247 is the target machine. If we instead
take the view from the backup site Storage Unit (in our example, serial 13-00247), it now looks
like it did in Figure 14-40 on page 177, prior to starting this whole exercise.
Figure 14-49 Production site Storage Unit at conclusion of disaster recovery exercise
Note: If the Failback fails, check whether your prior active server can still access the new
target volume, or has not reset the reserve on the volume, and if your Metro Mirror path is
established between these LSSs. You might need to use the Reset Reservation option
depicted in Figure 14-44 on page 179.
Chapter 14. Metro Mirror interfaces and examples
181
182
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 5
Part
5
Global Copy
In this part of the book, we describe the IBM System Storage Global Copy for DS6000. After
presenting an overview of Global Copy, we discuss the options available, the interfaces you
can use, and the configuration considerations. We also provide examples of the use of Global
Copy.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
183
184
IBM System Storage DS6000 Series: Copy Services in Open Environments
15
Chapter 15.
Global Copy overview
In this chapter, we describe the characteristics and operation of Global Copy. Also, we
discuss the considerations for its implementation with the IBM System Storage DS6000.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
185
15.1 Global Copy overview
Global Copy (formerly know as PPRC Extended Distance (PPRC-XD)) is an asynchronous
remote copy function for open systems and System z environments, for longer distances than
are possible with Metro Mirror. It is appropriate for remote data migration, off-site backups,
and transmission of inactive database logs at virtually unlimited distances.
Server
Server
writewrite
11
22 Write
22 Write
acknowledge
acknowledge
11
3 3
44
LUN orLUN or
volume
volume
Write
to secondary
Write
to secondary
(nonsynchronously)
(nonsynchronously)
(source)
PrimaryPrimary
(source)
LUNoror
LUN
volume
volume
Secondary(target)
(target)
Secondary
Figure 15-1 Global Copy
With Global Copy, write operations complete on the source disk subsystem before they are
received by the target disk subsystem. This capability is designed to prevent the local
system’s performance from being affected by wait time from writes on the remote system.
Therefore, the source and target copies can be separated by any distance. Figure 15-1
illustrates how Global Copy operates:
1. The host server makes a write I/O to the source (local) DS6000. The write is staged
through cache and non-volatile storage (NVS).
2. The write returns as completed to the host server’s application.
3. At a later time, that is, in an asynchronous manner, the source DS6000 sends the
necessary data so that the updates are reflected on the target (remote) volumes. The
updates are grouped in batches for efficient transmission.
4. The target DS6000 returns write complete to the source DS6000 when the updates are
secured in the target DS6000 cache and NVS. The source DS6000 then resets its Global
Copy change recording information.
186
IBM System Storage DS6000 Series: Copy Services in Open Environments
Note: The efficient extended distance mirroring technique of Global Copy is achieved with
sophisticated algorithms. For example, if changed data is in the cache, then Global Copy
sends only the changed sectors. There are also sophisticated queuing algorithms to
schedule the processing of updating the tracks for each volume and to set the batches of
updates to be transmitted.
15.2 Volume states and change logic
Figure 15-2 illustrates the basic states and the change logic of a volume that is in either a
Metro Mirror or Global Copy relationship. The following considerations apply to the volume
states when the pair is a Global Copy pair:
򐂰 Simplex: The volume is not in a Global Copy relationship.
򐂰 Suspended: In this state, the writes to the source volume are not mirrored onto the target
volume. The target volume becomes out-of-sync. During this time, Global Copy keeps a
bitmap record of the changed tracks in the source volume. Later, the volume pair can be
restarted, and then only the tracks that were updated are copied.
򐂰 Full duplex: A Global Copy volume never reaches this volume state. In the full duplex
condition, updates on the source volumes are synchronously mirrored to the target
volume. This is possible with Metro Mirror. With Global Copy, even when no tracks are
out-of-sync, the pair still remains in copy pending state.
򐂰 Copy pending: Updates on the source volume are asynchronously mirrored to the target
volume.
Simplex
Terminate
Terminate
Establish Metro
Mirror
Establish Global
Copy
Go to sync
Copy Pending
Full Duplex
Go to sync and
suspend
Resync
Terminate
Resync
Suspend
Suspend
Suspended
Figure 15-2 Global Copy and Metro Mirror state change logic
Chapter 15. Global Copy overview
187
In regard to the state change logic, the following considerations apply:
򐂰 When you initially establish a mirror relationship from a volume in simplex state, you have
the option to request that it becomes a Global Copy pair (establish Global Copy arrow in
Figure 15-2 on page 187), or a Metro Mirror pair (establish Metro Mirror arrow in
Figure 15-2 on page 187).
򐂰 Pairs can change from the copy pending state to the full duplex state when a go-to-SYNC
operation is executed (go-to-SYNC arrow in Figure 15-2 on page 187).
򐂰 You can also request that a pair is suspended as soon as it reaches the full duplex state
(go-to-SYNC and suspend in Figure 15-2 on page 187).
򐂰 Pairs cannot change directly from full duplex state to copy pending state. They must go
through an intermediate suspend state.
򐂰 You can go from suspended state to copy pending state during an incremental copy
(copying out-of-sync tracks only). This process is similar to the traditional transition from
suspended state to full duplex state (Resync arrow in Figure 15-2 on page 187).
15.3 Global Copy positioning
Figure 15-3 lists the main points for considering Global Copy.
Global Copy is a recommended solution for remote data copy, data
migration and offsite backup - over continental distances without
impacting application performance.
It can be used for application recovery implementations if application I/O
activity can be quiesced and non-zero data loss RPO is admissible.
It can be used over continental distances with excellent application
performance
The distances only limited by the network and channel extenders
capabilities
Application write operations do not have synchronous-like overheads
Fuzzy copy of data at the recovery site (sequence of dependent writes may
not be respected at the recovery site)
Recovery data can become a consistent point-in-time copy of the primary
data, if appropiate application checkpoints are set to do global catch-ups
Pairs are synchronized with application group consistency
Synchronizations can be done more frequently, because of short catch-ups
RPO still not zero, but improves substantially
Figure 15-3 Global Copy positioning
As summarized in Figure 15-3, Global Copy is a recommended solution when you want to
perform remote data copy, data migration, off-site backup, and transmission of inactive
database logs without impacting application performance, which is particularly relevant when
implemented over continental distances. Global Copy can also be used for application
recovery solutions based on periodic point-in-time copies of the data; this requires short
quiescings of the application’s I/O activity.
188
IBM System Storage DS6000 Series: Copy Services in Open Environments
Chapter 15. Global Copy overview
189
190
IBM System Storage DS6000 Series: Copy Services in Open Environments
16
Chapter 16.
Global Copy options and
configuration
This chapter discusses the options available when using Global Copy. It also discusses the
configuration guidelines that you should consider when planning for Global Copy with the IBM
System Storage DS6000.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
191
16.1 Global Copy basic options
First, we review the basic options available when working with Global Copy. Many of these
options are common to Metro Mirror, still you must consider that the results can differ
because Metro Mirror and Global Copy have differences in the way they work.
16.1.1 Establish Global Copy pair
This is the operation where you establish a Global Copy relationship between a source
volume and a target volume, that is, you establish a Global Copy pair. During this operation
the volumes transition from the simplex state to the copy pending state.
When you establish a Global Copy pair, the following options are available:
򐂰 No copy
This option does not copy data from the source to the target volume. This option assumes
that the volumes are already synchronized. The data synchronization is your responsibility
and the DS6000 does not check its synchronization.
򐂰 Target read
This option allows host servers to read from the target volume. For a host server to read
the volume, the volume pair must be in full duplex state. This parameter applies to open
systems volumes, it does not apply to System z volumes.
In the open system’s file system environment, even if an application reads data from a file
system, a SCSI write command can be issued to the LUN in which the file system resides,
because some information, such as the last access timestamp can be updated by the file
system. In this case, even if you specify this option, the operation might fail.
򐂰 Suspend after data synchronization
This option suspends the remote copy pairs after the data has been synchronized. You
can use this option with the Go-to-sync operation (the Go-to-sync operation is discussed
later in 16.1.5, “Convert a Global Copy pair to Metro Mirror” on page 193).
򐂰 Wait option
This option delays the command response until the volumes are in one of the final states:
simplex, full duplex, suspended, target full duplex, or target suspended. This parameter
cannot be used with -type gcp or -mode nocp, but it can be used with the Go-to-sync
operation.
򐂰 Reset reserve on target
This option allows the establishment of a Global Copy relationship when the target volume
is reserved by another host. If this option is not specified and the target volume is
reserved, the command fails. This parameter can only be used with open systems
volumes.
16.1.2 Suspend Global Copy pair
This operation stops copying data to the target volume and leaves the pair in suspended state.
Because the source DS6000 keeps record of all changed tracks on the source volume, you
can resume the remote copy operation at a later time.
16.1.3 Resume Global Copy pair
This operation resumes a Global Copy relationship for a volume pair and starts to copy data
back again to the target volume. Only modified tracks are sent to the target volume because
192
IBM System Storage DS6000 Series: Copy Services in Open Environments
the DS6000 kept a record of all changed tracks on the source volume while the volumes were
suspended. When resuming a Global Copy relationship, you can use the same options you
use to initially establish a Global Copy pair, except for the no copy option.
16.1.4 Terminate Global Copy pair
This operation ends the remote copy relationship between the volume pair; the volumes
return to the simplex state.
16.1.5 Convert a Global Copy pair to Metro Mirror
This operation is known as the Go-to-sync operation. There are two common situations when
you convert a pair from Global Copy mode to Metro Mirror mode:
򐂰 Situation 1: You have used Global Copy to complete the bulk transfer of data in the
creation of many copy pairs, and you now want to convert some or all of those pairs to
Metro Mirror mode.
򐂰 Situation 2: You have Global Copy pairs for which you want to make FlashCopy backups
on the recovery site. You convert the pairs temporarily to Metro Mirror mode in order to
obtain point-in-time consistent copies.
You can convert a Global Copy pair to a Metro Mirror pair using the DS CLI or DS GUI.
16.2 Create a consistent point-in-time copy
While the volume pairs are in the copy pending state, the target volumes maintain a fuzzy
copy of the data:
򐂰 Because of the asynchronous data transfer characteristics, at any time there is a certain
amount of updated data that is not reflected at the target volume. This data corresponds to
the sectors that were updated since the last volume bitmap scan was done. These are
out-of-sync sectors.
򐂰 Because of the bitmap scan method, writes are not ensured to be applied onto the target
volume in the same sequence that they are written to the source volume.
When terminating the Global Copy relationship to establish host access to the target volumes,
the first issue might cause loss of transactions. Since a file system’s or database’s
consistency depends on the correct ordering of write sequences, the second issue can cause
inconsistent volumes. Therefore, to use target volumes by the host systems, you need to
make them point-in-time consistent:
򐂰 The application must be quiesced and the volume pairs temporarily suspended. This is
necessary for ensuring consistency not only at the volume level, but also at the application
level.
򐂰 The target volumes have to catch up to their source counterparts. Global Copy catch-up is
the name of the transition that occurs to a Global Copy pair when it goes from its normal
out-of-sync condition until it reaches a full sync condition. At the end of this transition, the
source and target volumes become fully synchronized.
򐂰 You should now perform a FlashCopy of the target volumes onto tertiary volumes, and
then resume the Global Copy pairs. These tertiary volumes are then a consistent
point-in-time copy of the primary volumes.
Chapter 16. Global Copy options and configuration
193
16.2.1 Procedure to take a consistent point-in-time copy
Figure 16-1 summarizes a typical procedure to provide a consistent point-in-time copy of the
data. In Figure 16-1, we call the Global Copy source volume at the production site, the A
volume, the Global Copy target volume at the recovery site, the B volume, and the FlashCopy
target volume at the recovery site, the C volume.
Application running
source
1.Updated data is being sent
in normal Global Copy
A volume
target
B volume
C volume
Quiesce Application
2.Quiesce Application
source
source
target
3.Go-to-sync and suspend
volume pairs
target
Restart Application
source
4.Restart Application
Now we have
consistent
data
target
Application running
source
5.Take FlashCopy B to C.
Return to 1.
Production site
FlashCopy
target
Recovery site
Figure 16-1 Create consistent data using Global Copy
The steps in the procedure are the following:
1. Normal Global Copy operation.
In this phase, Global Copy is running normally. Data written to the source volume (A) is
copied to the target volume (B) independently of the write sequence to the source volume.
Therefore, at this phase, there is no guarantee that the target volume (B) holds a
consistent copy of the source volume (A) data.
2. Quiesce the application at the production site.
When you want to have a consistent copy at the recovery site, the application must be
quiesced to cease all write I/O from updating the source volume (A). Depending on the
host operating system, it might be necessary to dismount the source volume.
Note: In a DB2® environment, you can use a function provided by DB2 to stop the
application write I/O, which is the set write suspend/resume command.
194
IBM System Storage DS6000 Series: Copy Services in Open Environments
3. Go-to-sync and suspend the pairs.
Perform the catch-up by issuing a Go-to-sync operation. The volume pair leaves the copy
pending state and it reaches the full duplex state. Wait for the synchronization of the pair
and suspend the pair after the synchronization has completed. You can use the single
mkpprc -type mmir -suspend command to perform a Go-to-sync and suspend operation.
After this step, the Global Copy target (B) holds a consistent copy of the data.
4. Restart the application at the production site.
Now that you have a consistent copy of the data at the recovery site, while no updates to
the source (A) volumes are sent due to the suspend state, you can restart the application
at the production site.
5. Take a FlashCopy B to C.
In order to resume the Global Copy operation, we have to first perform a FlashCopy from
the Global Copy target volumes (B) to the FlashCopy target volumes (C) to keep consistent
data at the recovery site. Otherwise, further Global Copy data transmissions update the
Global Copy targets (B), modifying the consistent copy that at the moment we have at the
recovery site.
You can use the Remote FlashCopy function provided by the DS6000 to issue the
FlashCopy command to the DS6000 at the recovery site through the Global Copy paths (if
you use Remote FlashCopy, you do not need to set up a LAN connection to the remote
DS6000). After taking the FlashCopy, you have a consistent copy in the FlashCopy target
volume (C). You can then resume the Global Copy normal operation.
16.2.2 Make a FlashCopy at the remote site
In step 5 of the previous procedure, when doing a remote FlashCopy from the production to
the recovery site, you have to take into account that a disaster might occur while the
FlashCopy establishment for multiple volumes is being taken; see Figure 16-2.
FlashCopy
1.Establish FlashCopy
GC target
FC target Vol 1
FlashCopy
2.Establish FlashCopy
GC target
3.Establish FlashCopy
GC target
4.Establish FlashCopy
GC target
DS CLI client
FC target Vol 2
Vol 3
Vol 4
Recovery site DS6000
Figure 16-2 FlashCopy establishment from the production site
In Figure 16-2, four FlashCopy pairs are established at the recovery site. The FlashCopy
targets are Vol 1, Vol 2, Vol 3, and Vol 4. In this case, the DS CLI client issuing the
FlashCopy establish command is in the production site. When the DS CLI client is giving the
Chapter 16. Global Copy options and configuration
195
FlashCopy establish command, a disaster can occur. It might happen, although it is a very
short window, that the DS CLI client can issue the first and second FlashCopy establish
commands before going down. If this scenario occurs, the four volumes at the recovery site
are not at the same step of the point-in-time copy. In other words, Vol 1 and Vol 2 have the
latest copies, but Vol 3 and Vol 4 have previous copies. If you detect this situation,
depending on your Global Copy operational scenario, you might be able to use the Global
Copy target volumes, which can still have consistent point-in-time copies.
As long as you issue the FlashCopy commands from the production site, the above situation
can occur when you use any FlashCopy establish command, such as the mkflash,
resyncflash, mkremoteflash, and resyncremoteflash commands.
In order to avoid or detect this situation, you can implement several procedures, which
include:
򐂰 Have the DS CLI client at the recovery site issue the FlashCopy command to the DS6000
in the recovery site. We also recommend you keep the DS CLI execution log at the
recovery site to determine what processes have been performed during a disaster.
򐂰 If you prefer having the DS CLI client at the production site, keep the DS CLI execution log
at the recovery site to determine what processes have been performed during a disaster.
򐂰 If you use the Incremental FlashCopy function, you can use the FlashCopy sequence
number. When you establish FlashCopy, you can add a certain FlashCopy sequence
number to the set of the FlashCopy pairs. This sequence number can be used as an
identifier for a FlashCopy relationship or group of FlashCopy relations. When a disaster
occurs, you can find whether you have the same set of the FlashCopy or not from this
sequence number. However, for safer implementation, we also recommend to keep the DS
CLI execution log at the recovery site to determine what processes have been performed
during a disaster.
16.3 Hardware requirements
Global Copy is an optional licensed function of the IBM System Storage DS6000 series. The
feature is the Remote Mirror and Copy (RMC) feature, and is the same that is used to acquire
the Metro Mirror and the Global Mirror licensed functions.
You have to purchase the corresponding licensed function for the source and target DS6000
systems.
Note: For a detailed explanation of the features involved and the considerations you must
have when ordering Global Copy, we recommend you refer to the IBM System Storage
DS6000 Series (IBM 1750-522) announcement letter.
You can read IBM announcement letters at:
http://www.ibm.com/products
All DS6000 series licensed functions are enabled, authorized, managed, activated, and
enforced based upon the physical capacity contained in a 1750 system.
Enablement refers to the purchase of a feature code to technically activate the associated
licensed function. Each licensed function is enabled for a 1750 system by acquiring the
corresponding feature codes.
Particularly for Global Copy, the 1750 feature that you must order is the Remote Mirror and
Copy (RMC) licensed function, feature code 53xx; see Table 16-1 on page 197.
196
IBM System Storage DS6000 Series: Copy Services in Open Environments
Table 16-1 DS6000 Remote Mirror and Copy feature codes
Feature Code
Description
5310
RMC - 1 TB Unit
5311
RMC - 5 TB Unit
5312
RMC - 10 TB Unit
5313
RMC - 25 TB Unit
5314
RMC - 50 TB Unit
Each licensed function feature code purchased for a 1750 machine enables that function for
the entire 1750 system.
Authorization refers to the purchase of a feature code to establish the extent of IBM
authorization (license size in terms of physical capacity) for use of an associated licensed
function. This is also referred to as the authorization level.
The extent of IBM authorization for the use of a licensed function on a 1750 system is
established by acquiring 53xx feature code on the 1750 base enclosure. The same 53xx
feature codes acquired to enable a licensed function also establish the extent of IBM's
authorization.
The 53xx feature code on a 1750 base enclosure establishes the authorization level for the
entire 1750 system. For example, a 1750 system with a Model 522 base enclosure and two
Model EX2 expansion enclosures requires the acquisition of 53xx feature codes only on the
Model 522. Therefore, for licensed functions authorized on the basis of physical capacity as
in the case of RMC, the authorization level established with the acquisition of the 53xx feature
code on the Model 522 must also cover the physical capacity of the attached Model EX2
expansion enclosures.
Interoperability
Global Copy pairs can only be established between disk subsystems of the same (or similar)
type and features. For example, a DS6000 can have a Global Copy pair with another DS6000,
a DS8000, an ESS 800, or an ESS 750. It cannot have a Global Copy pair with an RVA or an
ESS F20. Note that all disk subsystems must have the appropriate Global Copy feature
installed. If your DS6000 is being mirrored to an ESS disk subsystem, the ESS must have
PPRC Version 2 (which supports Fibre Channel links) with the appropriate licensed internal
code level (LIC).
Refer to the DS6000 Interoperability Matrix for more information:
http://www-1.ibm.com/servers/storage/disk/ds6000/interop.html
16.4 DS6800 I/O ports
The DS6800 can have a maximum number of eight I/O ports; see Figure 16-3 on page 198.
Each I/O port has an I/O port number.
Chapter 16. Global Copy options and configuration
197
I0000
I0001
I0002
I0003
I0100
I0101
I0102
I0103
Figure 16-3 DS688 I/O port numbers
You can configure the I/O ports for Fibre Channel protocol or FICON protocol. For Global
Copy paths, you need at least one Fibre Channel connection between the two DS6000
subsystems for which you want to set up a Global Copy path. For higher availability, we
recommend you use at least two Fibre Channel connections between the two subsystems on
different controller cards on the DS6800.
The Fibre Channel ports used for Global Copy can be used as dedicated ports, which means
they are only used for the Global Copy paths, or they can be shared between Global Copy
and Fibre Channel data traffic; in which case, you also need Fibre Channel switches for
connectivity.
For supported SAN switches, you can refer to the DS6000 Interoperability Matrix.
16.5 Global Copy connectivity
Global Copy pairs are set up between volumes in LSSs, usually in different disk subsystems,
and these are normally in separate locations. A path (or group of paths) needs to be defined
between the source LSS and the target LSS. These logical paths are defined over physical
links between the disk subsystems.
The physical link includes the host adapter in the source DS6000, the cabling, switches or
directors, any wideband or long distance transport devices (DWDM, channel extenders, or
WAN), and the host adapters in the target disk subsystem. Physical links can carry multiple
Global Copy logical paths, as shown in Figure 16-4 on page 199.
Note: For Global Copy, the DS6000 supports Fibre Channel links only. To facilitate ease of
testing, the DS6000 does support Global Copy source and target on the same DS6000.
198
IBM System Storage DS6000 Series: Copy Services in Open Environments
LSS 0
LSS 1
LSS 2
LSS 3
Physical
Fibre Channel link
:
LSS 08
:
LSS nn
LSS 0
LSS 1
LSS 2
LSS 3
:
256 logical paths
per FCP link
LSS 08
:
LSS nn
Figure 16-4 Logical paths
Paths are unidirectional, that is they are defined to operate in either one direction or the other.
Still, Global Copy is bidirectional. That is, any particular pair of LSSs can have paths defined
between them that have opposite directions; each LSS holds both source and target volumes
from the other particular LSS. Even more, paths in opposite directions are allowed to be
defined on the same Fibre Channel physical link.
For bandwidth and redundancy, more than one path can be created between the same LSSs.
Global Copy balances the workload across the available paths between the source and target
LSSs.
Note: Remember that the LSS is not a physical construct in the DS6000; it is a logical
construct. Volumes in an LSS can come from multiple disk arrays.
Physical links are bidirectional and can be shared by other Global Copy pairs as well as other
remote mirror and copy functions, such as Metro Mirror and Global Mirror.
16.5.1 Fibre Channel links
The discussion presented in 12.5.1, “Fibre Channel links” on page 124, in the Metro Mirror
chapter, is also valid for Global Copy. We recommend its reading.
16.5.2 Logical Paths
The discussion presented in 12.5.2, “Logical Paths” on page 125, in the Metro Mirror chapter,
is also valid for Global Copy. We recommend its reading.
16.6 Bandwidth
To determine the bandwidth requirement for your Global Copy environment, you should
consider both the amount of data you need to transfer to the remote site and the available
window.
Chapter 16. Global Copy options and configuration
199
If using Global Copy for daily off-site backup with the technique mentioned in 16.2, “Create a
consistent point-in-time copy” on page 193, you can estimate the needed window to be the
sum of the application quiesce time, plus the FlashCopy establishment time, plus the
application restart time. However, the FlashCopy background copy operation can have an
influence over the Global Copy operation. In addition, if you take a backup to tape of the
FlashCopy target volumes, this activity also influences the Global Copy activity and the
FlashCopy background copy performance. Therefore, for a safer estimation of your daily
off-site backup window, as a very conservative approach, we recommend you estimate each
time sequentially, such as Global Copy data transmission, application quiesce, FlashCopy
establishment, application restart, FlashCopy background copy operation, and backing up
FlashCopy target.
You can estimate the approximate amount of data to be sent to the target by calculating the
amount of write data to the source. Tools to assist with this are operating system dependent,
such as iostat; see the redbook IBM System Storage DS6000 Series: Architecture and
Implementation, SG24-6781. Another method, but not so exact, is to monitor the traffic over
the Fiber Channel (FC) switches using FC switch tools and other management tools, but
remember that only writes are mirrored by Global Copy. You can get also some feeling about
the proportion of reads and writes by issuing datapath query devstats on SDD-attached
servers.
Finally, you can estimate how much network bandwidth is necessary for your Global Copy
solution by dividing the amount of data you need to transfer by the duration of the backup
window.
The FlashCopy relationship at the remote site can influence the performance of the Global
Copy operation. If you use the FlashCopy with the no copy option at the recovery site, when
the Global Copy target receives an update, the track on the FlashCopy source which is also
the Global Copy target, has to be copied to the FlashCopy target before the data transfer
operation completes. This copy operation to the FlashCopy target can complete by using the
DS6000 cache and NVS without waiting for a physical write to the FlashCopy target.
However, this data movement can influence the Global Copy activity. So, when considering
the network bandwidth, consider that the FlashCopy effect over the Global Copy activity
might in fact decrease the bandwidth utilization during some intervals.
If you are not using the no copy option, but you resume Global Copy before the FlashCopy
background copy finishes at the remote site, this background data movement can also
influence the Global Copy performance.
16.7 LSS design
Because the DS6000 has made the LSS a topological construct, which is not tied to a
physical array as in the ESS, the design of your LSS layout can be simplified. It is now
possible to assign LSSs to applications, for example, without concern regarding
under-allocation or over-allocation of physical disk subsystem resources.
This might also help you when you use the freezepprc command. A freeze operation is
performed at the LSS level, causing all Global Copy volumes in that LSS to go into suspended
state with queue full condition and terminating all associated paths. If you assign LSSs to
each of your applications, you can control the impact of the queue full condition caused by the
freeze operation at an application level. If you put volumes used by several applications into
the same LSS, all those applications sharing the LSS, their Global Copy volumes, are
affected by the queue full condition. You can issue one freezepprc command to multiple
200
IBM System Storage DS6000 Series: Copy Services in Open Environments
LSSs. Therefore, you do not need to consolidate the number of LSSs that your applications
use.
In general, the freezepprc and unfreezepprc commands are used in the Metro Mirror
environment or after converting from Global Copy mode to Metro Mirror mode. However, if
you set the Consistency Group option on an LSS, even when the volumes in the LSS are
Global Copy sources, the freeze operation triggered by the freezepprc command, or a failure
of a target update, causes the queue full condition. Therefore, you should pay attention to the
LSS design if you use the Consistency Group option.
16.8 Distance
The asynchronous characteristics of Global Copy, combined with the great throughput
characteristic of its efficient track mirroring technique, make Global Copy well suited for
remote copy solutions at distances beyond the 300 km available with Metro Mirror. Global
Copy achieves greater distances without having to incur, in the distance, latency penalties
that synchronous write I/O methods exhibit.
The maximum distance for a direct Fibre Channel connection is 10 km. If you want to use
Global Copy over longer distances, the following connectivity technologies can be used to
extend this distance:
򐂰 Fibre Channel switches
򐂰 Channel Extenders over Wide Area Network (WAN) lines
򐂰 Dense Wave Division Multiplexors (DWDM) on dark fibre
Channel extender
Channel extender vendors connect DS6000 systems with a variety of Wide Area Network
(WAN) connections, including Fibre Channel, Ethernet/IP, ATM-OC3, and T1/T3.
When you use channel extender products with Global Copy, the channel extender vendor
determines the maximum distance supported between the source and target DS6000. The
channel extender vendor should be contacted for their distance capability, line quality
requirements, and WAN attachment capabilities.
A complete and current list of Global Copy supported environments, configurations, networks,
and products is available in the DS6000 Interoperability Matrix.
You should also consult the channel extender vendor about hardware and software
prerequisites if you are using their products in a DS6000 Global Copy configuration.
Evaluation, qualification, approval, and support of Global Copy configurations using channel
extender products is the sole responsibility of the channel extender vendor.
Dense Wave Division Multiplexor (DWDM)
Wave Division Multiplexing (WDM) and Dense Wave Division Multiplexing (DWDM) comprise
the basic technology of fibre optical networking. The technique is used to carry many
separate and independent optical channels on a single dark fibre.
A simple way to envision DWDM is to consider that, at the primary end, multiple fibre optic
input channels, such as ESCON, Fibre Channel, FICON, or Gbit Ethernet, are combined by
the DWDM into a single fibre optic cable. Each channel is encoded as light for a different
wavelength. Think of each individual channel as an individual color: the DWDM system is
transmitting a rainbow. At the receiving end, the DWDM fans out the different optical
channels. DWDM, by the very nature of its operation, provides the full bandwidth capability of
the individual channel. Because the wavelength of light is, from a practical perspective,
Chapter 16. Global Copy options and configuration
201
infinitely divisible, DWDM technology is only limited by the sensitivity of its receptors, as the
total aggregate bandwidth possible.
A complete and current list of Global Copy supported environments, configurations, networks,
and products is available in the DS6000 Interoperability Matrix.
The DWDM vendor should be contacted regarding hardware and software prerequisites
when using their products in an DS6000 Global Copy configuration.
16.9 DS6000 configuration at the remote site
Depending on your operational environment when using Global Copy, you might also need
FlashCopy target volumes. If you need FlashCopy target volumes, you might be able to
choose higher capacity DDMs for the DS6000 at the remote site than those at the local site.
However, you should pay attention to the FlashCopy background copy performance
mentioned in the section 16.6, “Bandwidth” on page 199.
If you take backups of the FlashCopy target volumes onto tapes, you must ensure that the
tape resources are capable of handling these dump operations in between the point-in-time
checkpoints.
202
IBM System Storage DS6000 Series: Copy Services in Open Environments
17
Chapter 17.
Global Copy interfaces
The setup of Global Copy can be done using different interfaces. This chapter discusses the
interfaces that you can use for Global Copy management when you use Global Copy with the
IBM System Storage DS6000 in open systems environments.
The information discussed in the present chapter can be complemented with the following:
򐂰 Chapter 3, “DS Storage Manager (GUI)” on page 17
򐂰 Chapter 4, “DS Command-Line Interface (DS CLI)” on page 27
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
203
17.1 Global Copy management interfaces overview
There are various interfaces available for the configuration and management of Global Copy
for DS6000 when used in an open systems environment:
򐂰 DS Command-Line Interface (DS CLI)
򐂰 DS Storage Manager (DS SM) Graphical User Interface (GUI)
The present chapter gives an overview of the DS CLI and the DS SM for Global Copy
management. DS CLI and DS SM are also covered in Part 2, “Interfaces” on page 15.
Also, the following interfaces that are not covered in the present book can be used:
򐂰 DS Open Application Programming Interface (DS Open API)
򐂰 z/OS interfaces: TSO, ICKDSF, and ANTRQST API
In order to use a z/OS interface to manage open systems LUNs, the DS6000 must have at
least one CKD volume. If you are interested in this capability, you can refer to the redbook
IBM System Storage DS6000 Series: Copy Services with System z servers, SG24-6782.
For information about the DS Open API, refer to the publication IBM System Storage DS
Open Application Programming Interface Reference, GC35-0516.
DS CLI and DS SM similar functions for Global Copy management
Table 17-1 on page 205 compares similar Global Copy commands from the DS CLI and the
DS SM, and the corresponding actions.
204
IBM System Storage DS6000 Series: Copy Services in Open Environments
Table 17-1 DS CLI and DS SM commands and actions
Task
Command with
DS CLI
Select option
with DS SM
List available I/O ports that can
be used to establish Global Copy
paths.
lsavailpprcport
This information is shown
during the process when a
path is established.
List established Global Copy
paths.
lspprcpath
Copy Services → Paths
Establish path.
mkpprcpath
Copy Services → Paths →
Create
Delete path.
rmpprcpath
Copy Services → Paths →
Delete
Global Copy paths commands
Metro Mirror and Global Copy pair commands
Failback.
failbackpprc
Copy Services → Global
Copy → Recover Failback
Failover.
failoverpprc
Copy Services → Global
Copy → Recover Failover
List Global Copy volume pairs.
lspprc
Copy Services → Global
Copy
Establish pair.
mkpprc
Copy Service → Global
Copy → Create
Suspend pair.
pausepprc
Copy Services → Global
Copy → Suspend
Resume pair.
resumepprc
Copy Services → Global
Copy → Resume
Delete pair.
rmpprc
Copy Services → Global
Copy → Delete
Freeze Consistency Group.
freezepprc
Not applicable
Thaw Consistency Group.
unfreezepprc
Not applicable
The DS CLI has the advantage that you can make scripts and use them for automation. The
DS Storage Manager GUI is a Web-based graphical user interface and is more intuitive than
the DS CLI.
17.2 Copy Services network components
To implement the Global Copy environment, we need the same network environment as for
Metro Mirror. You can refer to 14.2, “Copy Services network components” on page 137 for a
description.
Chapter 17. Global Copy interfaces
205
17.3 DS Command-Line Interface (DS CLI)
You can use the DS CLI interface to manage all DS6000 Copy Services functions, such as
defining paths, establishing Global Copy pairs, and so forth. For a detailed explanation about
the DS CLI, refer to Chapter 4, “DS Command-Line Interface (DS CLI)” on page 27.
When you establish or remove Global Copy paths and volume pairs, you must give the DS
CLI commands to the DS SMC that is connected to the source DS6000. Also, when checking
status information at the local and remote site, you must issue DS CLI list type commands,
such as the lspprc, to each DS6000, source and target.
The DS CLI commands are documented in the publication IBM System Storage DS6000:
Command-Line Interface User´s Guide, GC26-7922.
In this section, we describe and give examples of the available DS CLI commands that you
can use for Global Copy setup and control.
17.3.1 Global Copy path commands
You can use the following commands to establish and manage Global Copy paths.
lsavailpprcport
The lsavailpprcport command displays the Fibre Channel ports, which can be used to
establish Global Copy paths; see Example 17-1 on page 207.
206
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 17-1 lsavailpprcport command
dscli> lsavailpprcport -l -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA -remotewwnn
500507630EFFFCA0 16:16
Date/Time: June 14, 2005 9:36:35 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0000
I0000
FCP NA
NA
I0000
I0100
FCP NA
NA
I0100
I0000
FCP NA
NA
I0100
I0100
FCP NA
NA
lspprcpath
The lspprcpath command shows you the established paths from an LSS. Example 17-2 lists
the paths from LSS 16.
Example 17-2 lspprcpath command
dscli> lspprcpath -dev IBM.1750-13AAGXA 16
Date/Time: June 14, 2005 9:46:31 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
Src Tgt State SS
Port Attached Port
========================================
16 16 Success FF16 I0000 I0000
16 16 Success FF16 I0100 I0100
mkpprcpath
With the mkpprcpath command, you can establish paths between two LSSs. In Example 17-3,
we establish a path from 1750-13AAGXA / LSS 16 / I/O port I0000 to 1750-13AAVCA / LSS
16 / I/O port I0100.
Example 17-3 mkpprcpath command
dscli> mkpprcpath -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA -remotewwnn
500507630EFFFCA0 -srclss 16 -tgtlss 16 I0000:I0100
Date/Time: June 14, 2005 10:00:48 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00149I mkpprcpath: Remote Mirror and Copy path 16:16 successfully established.
rmpprcpath
The rmpprcpath command removes all paths between two LSSs; see Example 17-4.
Example 17-4 rmpprcpath command
dscli> rmpprcpath -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 16:16
Date/Time: June 12, 2005 5:58:25 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00152W rmpprcpath: Are you sure you want to remove the Remote Mirror and Copy path
16:16:? [y/n]:y
CMUC00150I rmpprcpath: Remote Mirror and Copy path 16:16 successfully removed.
You can suppress the CMUC00152W confirmation message by adding the -quiet parameter in
the command.
Chapter 17. Global Copy interfaces
207
17.3.2 Global Copy pair commands
You can use the following commands to establish and manage Global Copy pairs.
failbackpprc
You can issue the failbackpprc command against any Global Copy source volume that is in a
suspended state. The command copies the required data from the source volume to the target
volume in order to resume mirroring; see Example 17-5. The command is typically used after
a failoverpprc command that has been issued to restart mirroring either in the reverse
direction (recovery site to production site) or original direction (production site to recovery
site).
Example 17-5 failbackpprc command
dscli>failbackpprc -dev IBM.1750-13AAVCA -remotedev IBM.1750-13AAGXA 1600:1600
Date/Time: June 14, 2005 1:40:22 PM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAVCA
CMUC00197I failoverpprc: Remote Mirror and Copy pair 1610:1610 successfully failed back.
failoverpprc
You can use the failoverpprc command to change a target device into a (source) suspended
device while leaving the original source device in its current state; see Example 17-6.
Example 17-6 failoverpprc command
dscli> failoverpprc -dev IBM.1750-13AAVCA -type mmir -remotedev IBM.1750-13AAGXA 1610:1610
Date/Time: June 14, 2005 1:40:22 PM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAVCA
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1610:1610 successfully reversed.
lspprc
The lspprc command shows you which volumes are in a Global Copy relationship; see
Example 17-7.
Example 17-7 lspprc command
dscli> lspprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA -l 1600
Date/Time: June 14, 2005 11:28:35 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date
Suspended
SourceLSS Timeout (secs) Critical Mode First Pass Status
===================================================================================================
1600:1600 Copy Pending Global Copy 0
Disabled Disabled
Disabled
Wed
Dec 31 17:59:59 CST 1969 16
300
Disabled
True
mkpprc
With the mkpprc command, you can create a Global Copy pair. To create a Global Copy pair,
specify parameter -type gcp. See Example 17-8 on page 209.
208
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 17-8 mkpprc command
dscli> mkpprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA -type gcp -mode full 1600:1600
Date/Time: June 14, 2005 11:25:49 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1600:1600 successfully
created.
freezepprc
With the freezpprc command (see Example 17-9), you can stop the write activity on all
Global Copy source and target volumes for a given source and target LSS. The command
creates a new Consistency Group. It places the source logical subsystem (LSS) in the long
busy state so that no I/Os can be directed to it. It sets the source volumes on the source LSS
in a suspended state. Target volumes are left in their current state. The command also
removes paths between the source LSS and target LSS and sets the queue full condition for
the source volume. This causes the host to queue writes to the source volume until the queue
full condition is reset (with the unfreezepprc command). During the queue full condition, the
source volume reports long busy status.
Example 17-9 freezepprc command
dscli> freezepprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 16:16
Date/Time: June 14, 2005 5:45:52 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00161W freezepprc: Remote Mirror and Copy consistency group 16:16 successfully created.
pausepprc
The pausepprc command suspends a Global Copy pair; see Example 17-10.
Example 17-10 pausepprc command
dscli> pausepprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 1600:1600
Date/Time: June 14, 2005 12:36:16 PM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1600:1600 relationship
successfully paused.
resumepprc
With the resumepprc command, you can bring a pair from suspended to copy pending mode
as shown in Example 17-11.
Example 17-11 resumepprc command
dscli> resumepprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA -type gcp 1600:1600
Date/Time: June 14, 2005 12:41:25 PM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1600:1600 relationship
successfully resumed. This message is being returned before the copy completes.
rmpprc
The rmpprc command removes the relationship from a Global Copy pair and sets the volumes
in simplex states; see Example 17-12 on page 210.
Chapter 17. Global Copy interfaces
209
Example 17-12 rmpprc command
dscli> rmpprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 1600:1600
Date/Time: June 14, 2005 12:48:09 PM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair
relationship 1600:1600:? [y/n]:y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1600:1600 relationship successfully
withdrawn.
You can suppress the CMUC00160W confirmation message by adding the -quiet parameter.
unfreezepprc
With the unfreezepprc command, you can thaw an existing Consistency Group that was
created with the freeze command; see Example 17-13. It resumes host application I/O
activity for source volumes on the Consistency Group that was created by an earlier
freezepprc command. The unfreezepprc command is used to reset the queue full condition
for the source volume and release application I/O to the source volumes. All queued writes to
the source volume are written. The status of the volumes does not change; the source
volumes remain in the suspended state as set by the freezepprc command.
Example 17-13 unfreezepprc command
dscli> unfreezepprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 16:16
Date/Time: June 14, 2005 5:46:37 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA
CMUC00198I unfreezepprc: Remote Mirror and Copy pair 16:16 successfully thawed.
Note: Because the freezepprc command deletes the paths between the specified LSS
pair, you have to reestablish them to resume normal Global Copy operation.
17.4 DS Storage Manager
You can use the DS Storage Manager (DS SM) Graphical User Interface (GUI) to set up and
control Global Copy. It is user friendly; however, you cannot use it for automation activities,
and certain Global Copy functions are not supported from this interface.
For a detailed explanation about the DS SM Graphical User Interface, refer to Chapter 3, “DS
Storage Manager (GUI)” on page 17.
Note: The DS SM GUI supports, differently from the DS CLI, a multi-session environment,
but not all functions and options are supported by the GUI.
We show examples using the DS Storage Manager in the following sections.
17.4.1 Path panel
The Path panel is the central point in the DS Storage Manager to establish new paths and
manage existing paths. You can get to the path panel as follows:
򐂰 Select Real-time manager.
򐂰 Select Copy Services.
210
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Click Paths.
Then the Paths panel displays; see Figure 17-1. Here, to display existing paths, select the
Storage Unit and the LSS for which you want to display the paths information.
Figure 17-1 Paths panel
To create a new path, select Create from the Select Actions pull-down menu, and a process
then guides you to establish a path.
Alternatively in the Paths panel, if you select one or more of the existing paths, then the
Select Actions pull-down menu gives you the following additional possible actions:
򐂰 Delete to delete the paths you have selected before.
򐂰 LSS copy options to modify the copy options for an LSS; see Figure 17-2.
Figure 17-2 LSS Copy Options
17.4.2 Metro Mirror panel
The Metro Mirror panel is the central point in the DS Storage Manager to establish new and
manage existing Global Copy pairs. You can get to this panel as follows:
1. Select Real-time manager.
2. Select Copy services.
3. Click Metro Mirror.
Then the Metro Mirror panel displays; see Figure 17-3 on page 212.
Chapter 17. Global Copy interfaces
211
Figure 17-3 Metro Mirror panel
To create a new Global Copy pair, choose Create from the Select Action pull-down menu.
The DS Storage Manager guides you through the process to create a new pair.
Figure 17-4 Metro Mirror panel
If you select an existing Global Copy pair (see Figure 17-4), then the Select Action pull-down
menu shows you additional actions you can perform with the selected volume pair:
򐂰 Delete: You can use this action to delete a Global Copy pair.
򐂰 Suspend: You can use this action to suspend a Global Copy pair.
򐂰 Convert to synchronous: You can use this action to convert a Global Copy pair into a
synchronous Metro Mirror pair.
򐂰 Resume: You can use this action to resume a suspended Global Copy pair.
򐂰 Recover Failover: You can use this action to change a target device into a (source)
suspended device while leaving the original source device in its current state.
򐂰 Recover Failback: You can use this action to switch back to the original site after a
failover.
򐂰 Properties: You can use this action to view the volume pair properties, including the
number of out-of-sync tracks.
212
IBM System Storage DS6000 Series: Copy Services in Open Environments
18
Chapter 18.
Global Copy performance and
scalability
In this chapter, we discuss performance and scalability considerations when using Global
Copy for DS6000.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
213
18.1 Performance
As the distance between the DS6000s increases, Metro Mirror response time is proportionally
affected, and this negatively impacts the application performance. When implementations
over extended distances are necessary, Global Copy becomes an excellent trade-off
solution.
You can estimate Global Copy application impact as that of the application when working with
Metro Mirror suspended volumes. For the DS6000, there is more work to do with the Global
Copy volumes, as compared to the suspended volumes, because with Global Copy, the
changes have to be sent to the remote DS6000. But this is a negligible overhead for the
application compared with the typical synchronous overhead.
There are no processor resources (CPU and memory) consumed by your Global Copy
volume pairs, because this is managed by your DS6000 subsystem.
If you take FlashCopy at the recovery site in your Global Copy implementation, you should
take into account the influence between Global Copy and FlashCopy background copy; refer
to 12.6, “Bandwidth” on page 126.
18.2 Scalability
The DS6000 Global Copy environment can be scaled up or down as required. If new volumes
are added to the DS6000 that require mirroring, they can be dynamically added. If additional
Global Copy paths are required, they also can be dynamically added.
18.2.1 Adding capacity
As we have mentioned, the logical nature of the LSS makes a Global Copy implementation
on the DS6000 easier to plan, implement, and manage. However, if you need to add more
LSSs to your Global Copy environment, your management and automation solutions should
be set up to handle this.
The eRCMF service offerings are designed to provide this functionality. Visit the IBM Web
site and see the Services & Industry Solutions page for more information.
Adding capacity to the same DS6000
If you are adding capacity to an existing DS6000, providing your Global Copy link bandwidth
is not close to or over its limit, you might only need to add volume pairs into your
configuration. If you are adding more LSSs, then you need to define Global Copy paths
before adding volume pairs. Keep in mind that when you add capacity for Global Copy use,
you might have to acquire a new license feature code for Global Copy that corresponds to the
new capacity.
Adding capacity in new DS6000s
If you are adding new DS6000s into your configuration, you need to add physical links prior to
defining your Global Copy paths and volume pairs. We recommend a minimum of two Global
Copy paths per DS6000 pair for redundancy reasons. Your bandwidth analysis indicates
whether you require more than two paths.
214
IBM System Storage DS6000 Series: Copy Services in Open Environments
19
Chapter 19.
Global Copy examples
In this chapter, we provide examples of how to configure and manage Global Copy on the
DS6000 by using the DS CLI and the DS GUI. This includes:
򐂰
򐂰
򐂰
򐂰
򐂰
Set up a Global Copy environment.
Remove a Global Copy environment.
Maintain the Global Copy environment.
Periodical off-site backup procedure.
Using the DS GUI to manage Global Copy.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
215
19.1 Set up Global Copy environment using DS CLI
In the following sections, we present an example of how to set up a Global Copy environment
using the DS CLI. Figure 19-1 shows the configuration that we are going to implement.
LSS1A
LSS14
LSS15
1A01
LSS1B
(physical links)
Server1
1501
1A00
Fibre Channel connections
Server1
1500
Server0
1401
Server0
1400
1B00
1B01
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 19-1 DS6000 configuration in the Global Copy setup example
In our example, we use different LSS and LUN numbers for the Global Copy source and
target elements, so that you can more clearly understand which one we are specifying when
going through the example.
Note: In a real environment and different than our example, we recommend that you
maintain a symmetrical configuration in terms of both physical and logical elements to
simplify the management of your Global Copy environment.
For a more redundant and balanced configuration, we put two physical paths on separate
servers (controllers) in the DS6000s.
19.1.1 Preparing to work with the DS CLI
As we prepare to work with the DS CLI, we do some initial tasks to simplify our activities
during the configuration process; refer to “Simplifying the DS CLI command syntax” on
page 140.
19.1.2 Setup of Global Copy configuration
Figure 19-2 on page 217 shows the configuration that we set up for this example. The
configuration has four Global Copy pairs that reside in two LSSs. Two paths are defined
between each source and target LSS.
216
IBM System Storage DS6000 Series: Copy Services in Open Environments
Source
Global Copy paths
LSS14
1400
1401
Target
1A00
Global Copy pairs
LSS15
1A01
Fibre Channel connections
(physical links)
1500
1501
LSS1A
LSS1B
1B00
Global Copy pairs
1B01
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 19-2 Global Copy environment to set up
To configure the Global Copy environment, we follow this procedure:
1. Determine the available Fibre Channel links for path definition.
2. Define the paths for Global Copy to use.
3. Create Global Copy pairs.
19.1.3 Determine the available Fibre Channel links
This step is similar to the Metro Mirror setup example; refer to 14.5.3, “Determine the
available Fibre Channel links” on page 141.
19.1.4 Create Global Copy paths
This step is similar to the Metro Mirror example; refer to 14.5.4, “Create Metro Mirror paths”
on page 142.
19.1.5 Create Global Copy pairs
After creating the paths, you can establish Global Copy volume pairs. This is done with the
mkpprc command, and you verify the result with the lspprc command; see Example 19-1. If
you want to create a Global Copy pair, you have to specify the-type gcp parameter with the
mkpprc command.
Example 19-1 Create Global Copy pairs and verify the result
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time:
CMUC00153I
CMUC00153I
CMUC00153I
CMUC00153I
dscli>
November
mkpprc:
mkpprc:
mkpprc:
mkpprc:
18, 2005 1:12:52 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Remote Mirror and Copy volume pair relationship 1400:1A00 successfully created.
Remote Mirror and Copy volume pair relationship 1401:1A01 successfully created.
Remote Mirror and Copy volume pair relationship 1500:1B00 successfully created.
Remote Mirror and Copy volume pair relationship 1501:1B01 successfully created.
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:13:08 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
False
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
False
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
False
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
False
Chapter 19. Global Copy examples
217
You can check the status of Global Copy using the lspprc -l command. The Out Of Sync
Tracks column shows the remaining tracks to be sent to the target volume. The size of the
logical track for FB volumes for the DS6000 is 64 KB. And, you also can use the lspprc
-fullid command to show the DS6000 Storage Image ID in the command output; see
Example 19-2.
Example 19-2 lspprc -l and lspprc -fullid for Global Copy pairs
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 18, 2005 1:13:29 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs)
Critical Mode First Pass Status
==========================================================================================================================================
1400:1A00 Copy Pending Global Copy 32206
Disabled Disabled
invalid
14
unknown
Disabled
False
1401:1A01 Copy Pending Global Copy 31546
Disabled Disabled
invalid
14
unknown
Disabled
False
1500:1B00 Copy Pending Global Copy 23206
Disabled Disabled
invalid
15
unknown
Disabled
False
1501:1B01 Copy Pending Global Copy 24766
Disabled Disabled
invalid
15
unknown
Disabled
False
dscli>
dscli> lspprc
-fullid 1400-1401 1500-1501
Date/Time: November 18, 2005 1:13:44 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS
Timeout (secs) Critical Mode First Pass
Status
==========================================================================================================================================
IBM.1750-1300819/1400:IBM.1750-1300247/1A00 Copy Pending Global Copy IBM.1750-1300819/14 unknown
Disabled
False
IBM.1750-1300819/1401:IBM.1750-1300247/1A01 Copy Pending Global Copy IBM.1750-1300819/14 unknown
Disabled
False
IBM.1750-1300819/1500:IBM.1750-1300247/1B00 Copy Pending Global Copy IBM.1750-1300819/15 unknown
Disabled
False
IBM.1750-1300819/1501:IBM.1750-1300247/1B01 Copy Pending Global Copy IBM.1750-1300819/15 unknown
Disabled
False
Unlike the Metro Mirror initial copy (first pass), the state of the Global Copy volumes still
shows Copy Pending after the Out Of Sync Tracks become 0; see Example 19-3.
Example 19-3 List the volume pairs status after Global Copy is established
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 18, 2005 1:17:16 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs)
Critical Mode First Pass Status
==========================================================================================================================================
1400:1A00 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 0
Disabled Disabled
invalid
15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 0
Disabled Disabled
invalid
15
unknown
Disabled
True
Copy Pending is the state of the Global Copy source. The target state is Target Copy
Pending. To see the target state in this example, you have to give the lspprc command to the
DS SMC connected to DS6000#2; see Example 19-4.
Example 19-4 lspprc for Global Copy target volumes
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 18, 2005 1:13:20 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Copy Pending Global Copy 14
unknown
Disabled
Invalid
1401:1A01 Target Copy Pending Global Copy 14
unknown
Disabled
Invalid
1500:1B00 Target Copy Pending Global Copy 15
unknown
Disabled
Invalid
1501:1B01 Target Copy Pending Global Copy 15
unknown
Disabled
Invalid
218
IBM System Storage DS6000 Series: Copy Services in Open Environments
19.2 Remove Global Copy environment using the DS CLI
This step is similar to the Metro Mirror example (see 14.6, “Remove Metro Mirror environment
using DS CLI” on page 144). However, we show examples here for Global Copy, because the
output of the lspprc command are different from that in the Metro Mirror environment.
In general, the following steps should be done to clear the Global Copy environment:
1. Remove Global Copy pairs.
2. Remove logical paths.
19.2.1 Remove Global Copy pairs
The rmpprc command removes a volume pair relationship; see Example 19-5. You can use
the -quiet parameter to turn off the confirmation prompt for this command.
Example 19-5 Removing Global Copy pairs
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:20:51 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> rmpprc -remotedev IBM.1750-1300247 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 1:21:10 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship
1400-1401:1a00-1a01:? [y/n]:y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully withdrawn.
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship
1500-1501:1b00-1b01:? [y/n]:y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully withdrawn.
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully withdrawn.
You can add the -at tgt parameter to the rmpprc command to remove only the available
Global Copy target volumes; see Example 19-6. You have to give this command to the DS
SMC connected to DS6000#2.
Example 19-6 Results of rmpprc with -at tgt
dscli> lspprc 1a00
Date/Time: November 18, 2005 1:24:43 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Copy Pending Global Copy 14
unknown
Disabled
Invalid
dscli>
dscli> rmpprc -at tgt -remotedev IBM.1750-1300247 -quiet 1400:1a00
Date/Time: November 18, 2005 1:25:56 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
dscli>
dscli> lspprc 1a00
Date/Time: November 18, 2005 1:26:02 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lspprc: No Remote Mirror and Copy found.
Chapter 19. Global Copy examples
219
Example 19-7 shows the Global Copy source volume status after the rmpprc -at tgt
command has completed, and it also shows the result of a rmpprc -at src command. In this
case, there were still available paths. Therefore, the source volume state changed after the
rmpprc -at tgt command completed. If there were no available paths, the state of the Global
Copy source volumes is preserved. In Example 19-7, you have to issue this command to the
SMC connected to DS6000#1, which is the Global Copy source.
Example 19-7 Global Copy source volume state after rmpprc with -at tgt and rmpprc with -at src
dscli> lspprc 1400
Date/Time: November 18, 2005 1:20:51 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
<< After rmpprc -at tgt command completes >>
dscli> lspprc 1400
Date/Time: November 18, 2005 1:28:08 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=======================================================================================================
1400:1A00 Suspended Simplex Target Global Copy 14
unknown
Disabled
True
dscli> rmpprc -at src -remotedev IBM.1750-1300247 -quiet 1400:1a00
Date/Time: November 18, 2005 1:31:13 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
dscli>
dscli> lspprc 1400
Date/Time: November 18, 2005 1:31:38 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprc: No Remote Mirror and Copy found.
19.2.2 Remove paths
The rmpprcpath command removes the paths. Before removing the paths, you have to
remove all volume pairs which are using the paths, or you have to use the -force parameter
with the rmpprcpath command; see Example 19-8.
Example 19-8 Remove paths
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:36:53 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprc: No Remote Mirror and Copy found.
dscli>
dscli> rmpprcpath -quiet -remotedev IBM.1750-1300247 14:1a 15:1b
Date/Time: November 18, 2005 1:37:12 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00150I rmpprcpath: Remote Mirror and Copy path 14:1A successfully removed.
CMUC00150I rmpprcpath: Remote Mirror and Copy path 15:1B successfully removed.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 18, 2005 1:37:35 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprcpath: No Remote Mirror and Copy Path found.
If you do not remove the Global Copy pairs which are using the paths, the rmpprcpath
command fails; see Example 19-9 on page 221.
220
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 19-9 Removing paths without removing the Global Copy pairs
dscli> lspprc 1400
Date/Time: November 18, 2005 1:39:54 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
False
dscli>
dscli> rmpprcpath -quiet -remotedev IBM.1750-1300247 14:1a
Date/Time: November 18, 2005 1:40:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUN03070E rmpprcpath: 14:1A: Copy Services operation failure: pairs remain
If you want to remove paths that still have the Global Copy pairs, you can use the -force
parameter; see Example 19-10. After the path has been removed, the Global Copy pair is still
in copy pending state until the Global Copy source receives I/O from the servers. When I/O
goes to the Global Copy source, the source volume becomes suspended. If you set the
Consistency Group option for the LSS in which the volumes reside, I/Os from the servers are
held with queue full status for the specified time-out value.
Example 19-10 Removing paths having Global Copy pair with -force parameter
dscli> lspprc 1400
Date/Time: November 18, 2005 1:39:54 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
False
dscli>
dscli> rmpprcpath -quiet -remotedev IBM.1750-1300247 14:1a
Date/Time: November 18, 2005 1:40:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUN03070E rmpprcpath: 14:1A: Copy Services operation failure: pairs remain
dscli>
dscli> rmpprcpath -force -quiet -remotedev IBM.1750-1300247 14:1a
Date/Time: November 18, 2005 1:42:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00150I rmpprcpath: Remote Mirror and Copy path 14:1A successfully removed.
dscli>
dscli> lspprcpath 14
Date/Time: November 18, 2005 1:42:22 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lspprcpath: No Remote Mirror and Copy Path found.
dscli>
dscli> lspprc 1400
Date/Time: November 18, 2005 1:43:19 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
19.3 Maintain Global Copy environment using the DS CLI
This section shows how we can manage the Global Copy environment by suspending and
resuming Global Copy pairs, changing mode from Global Copy to Metro Mirror, and changing
paths.
19.3.1 Suspend and resume Global Copy data transfer
The pausepprc command stops transferring data to the Global Copy target volume. After this
command completes, the Global Copy pair becomes suspended; see Example 19-11 on
page 222.
Chapter 19. Global Copy examples
221
Example 19-11 Suspend Global Copy data transfer
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:51:39 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> pausepprc -remotedev IBM.1750-1300247
1400-1401:1a00-1a01
Date/Time: November 18, 2005 1:52:02 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully paused.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:52:10 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=======================================================================================================
1400:1A00 Suspended
Host Source Global Copy 14
unknown
Disabled
True
1401:1A01 Suspended
Host Source Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
Because the source DS6000 keeps record of all changed data on the source volume, you can
resume Global Copy data transfer at a later time. The resumepprc command resumes a
Global Copy relationship for a volume pair and restarts transferring data. You have to specify
the copy mode such as Metro Mirror or Global Copy with the -type parameter; see
Example 19-12.
Example 19-12 Resume Global Copy pairs
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:53:36 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
======================================================================================================
1400:1A00 Suspended
Host Source Global Copy 14
unknown
Disabled
True
1401:1A01 Suspended
Host Source Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> resumepprc -type gcp -remotedev IBM.1750-1300247
1400-1401:1a00-1a01
Date/Time: November 18, 2005 1:53:59 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully resumed.
This message is being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully resumed.
This message is being returned before the copy completes.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 1:54:09 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
222
IBM System Storage DS6000 Series: Copy Services in Open Environments
19.3.2 Change copy mode from Global Copy to Metro Mirror
You can change the copy mode from Global Copy to Metro Mirror with the mkpprc command;
see Example 19-13. This operation is called Go-to-sync. Depending on the amount of data to
be sent to the target, it takes time until the pairs become full duplex.
Example 19-13 Changing the copy mode from Global Copy to Metro Mirror
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:29:09 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 6:30:54 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1400:1A00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1401:1A01 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1500:1B00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1501:1B01 successfully created.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:31:02 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
You can use the -suspend parameter with the mkpprc -type mmir command. If you use this
parameter, the state of the pairs becomes suspended when the data synchronization is
completed; see Example 19-14. You can use this option for your off-site backup scenario with
the Global Copy function.
Example 19-14 mkpprc -type mmir with -suspend
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:34:58 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir -suspend 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 6:37:04 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1400:1A00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1401:1A01 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1500:1B00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1501:1B01 successfully created.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:37:12 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1400:1A00 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
Chapter 19. Global Copy examples
223
1501:1B01 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
You can add the -wait parameter with the mkpprc command. With the -wait parameter, the
mkpprc -type mmir -suspend command does not return to the command prompt until the
pairs complete data synchronization and reach the suspended state; see Example 19-15.
Note: If you do not specify the -wait parameter with the mkpprc -type mmir -suspend
command, the mkpprc command does not wait for the data synchronization. If you do not
use the -wait parameter, you have to check the completion of the synchronization with
the lspprc command.
Example 19-15 mkpprc -type mmir -suspend with -wait
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:34:58 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir -suspend -wait 1400-1401:1a00-1a01
1500-1501:1b00-1b01
Date/Time:
CMUC00153I
CMUC00153I
CMUC00153I
CMUC00153I
November 18, 2005 6:38:54
mkpprc: Remote Mirror and
mkpprc: Remote Mirror and
mkpprc: Remote Mirror and
mkpprc: Remote Mirror and
PM JST IBM DSCLI
Copy volume pair
Copy volume pair
Copy volume pair
Copy volume pair
Version: 5.1.0.204 DS:
relationship 1400:1A00
relationship 1401:1A01
relationship 1500:1B00
relationship 1501:1B01
IBM.1750-1300819
successfully created.
successfully created.
successfully created.
successfully created.
<< Waiting for Metro Mirror data synchronization >>
1/4 pair
2/4 pair
3/4 pair
4/4 pair
dscli>
1400:1A00
1401:1A01
1500:1B00
1501:1B01
state:
state:
state:
state:
Suspended
Suspended
Suspended
Suspended
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:39:37 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1400:1A00 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
19.3.3 Change the copy mode from Metro Mirror to Global Copy
You cannot change the copy mode from Metro Mirror to Global Copy directly. In order to do
this, you have to use the pausepprc command to suspend the Metro Mirror pair first, and then
resume the pair in Global Copy mode with the resumepprc -type gcp command; see
Example 19-16 on page 225.
224
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 19-16 Changing the copy mode from Metro Mirror to Global Copy
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:31:02 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Full Duplex Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Full Duplex Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> pausepprc -remotedev IBM.1750-1300247 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 6:33:33 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully paused.
dscli>
dscli> resumepprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 6:34:00 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully resumed.
This message is being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully resumed.
This message is being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully resumed.
This message is being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully resumed.
This message is being returned before the copy completes.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 6:34:58 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
19.3.4 Add and remove paths
This step is similar to the Metro Mirror example; refer to 14.7.2, “Add and remove paths” on
page 148.
19.4 Periodical off-site backup procedure
This section shows how to control the procedure that is discussed in 16.2, “Create a
consistent point-in-time copy” on page 193, to use Global Copy for periodical off-site backup.
Figure 19-3 on page 226 shows a diagram of the DS6000 Global Copy environment for this
example. We use the Remote Incremental FlashCopy function for the FlashCopy at the
recovery site (DS6000#2). We use 1400, 1401, 1500, and 1501 in DS6000#1 for Global
Copy (GC) sources; 1A00, 1A01, 1B00, and 1B01 in DS6000#2 for Global Copy targets and
FlashCopy (FC) sources; and 1C02, 1C03, 1D02, and 1D03 in DS6000#2 for FlashCopy
targets.
Chapter 19. Global Copy examples
225
GC Target
and
FC source
GC Source
Global Copy paths
LSS14
1400
1401
LSS1A
LSS15
LSS1B
1501
1C01
FlashCopy LSS1D
1B00
Global Copy pairs
LSS1C
1C00
1A01
Fibre Channel links
1500
FlashCopy
1A00
Global Copy pairs
FC Target
1D00
1B01
1D01
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 19-3 The DS6000 environment for Global Copy off-site backup
19.4.1 Initial setup for this environment
In order to set up the Global Copy and FlashCopy environment, we follow these steps:
1. Create paths, create pairs, and wait for the completion of the Global Copy initial copy.
2. Create the Incremental FlashCopy relationship at the recovery site and wait for the
completion of the FlashCopy background copy.
Step 1: Create paths and pairs
An example of this step is shown in Example 19-17.
Example 19-17 Step 1 of the initial setup
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 1a i0003:i0003 i0102:i0103
Date/Time: November 18, 2005 7:17:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:1a successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 15
-tgtlss 1b i0003:i0003 i0102:i0103
Date/Time: November 18, 2005 7:17:48 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 15:1b successfully established.
dscli>
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time:
CMUC00153I
CMUC00153I
CMUC00153I
CMUC00153I
dscli>
November 18, 2005 7:18:17
mkpprc: Remote Mirror and
mkpprc: Remote Mirror and
mkpprc: Remote Mirror and
mkpprc: Remote Mirror and
PM JST IBM DSCLI
Copy volume pair
Copy volume pair
Copy volume pair
Copy volume pair
Version: 5.1.0.204 DS:
relationship 1400:1A00
relationship 1401:1A01
relationship 1500:1B00
relationship 1501:1B01
IBM.1750-1300819
successfully created.
successfully created.
successfully created.
successfully created.
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 18, 2005 7:20:08 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date
Suspended SourceLSS Timeout (secs) Critical Mode First Pass Status
======================================================================================================
1400:1A00 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
unknown
Disabled
True
226
IBM System Storage DS6000 Series: Copy Services in Open Environments
1500:1B00
15
1501:1B01
15
Copy Pending Global Copy 0
unknown
Disabled
True
Copy Pending Global Copy 0
unknown
Disabled
True
Disabled Disabled
invalid
-
Disabled Disabled
invalid
-
Step 2: Create FlashCopy relationship
An example of this step is shown in Example 19-18 and Example 19-19.
Example 19-18 Step 2 of the initial setup
dscli> mkremoteflash -record -persist -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247
1a00-1a01:1c00-1c01
Date/Time: November 18, 2005 7:25:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1A00:1C00 successfully created. Use the
lsremoteflash command to determine copy completion.
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1A01:1C01 successfully created. Use the
lsremoteflash command to determine copy completion.
dscli>
dscli> mkremoteflash -record -persist -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247
1b00-1b01:1d00-1d01
Date/Time: November 18, 2005 7:25:43 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1B00:1D00 successfully created. Use the
lsremoteflash command to determine copy completion.
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1B01:1D01 successfully created. Use the
lsremoteflash command to determine copy completion.
Example 19-19 lsremoteflash to check the FlashCopy background copy completion
dscli> lsremoteflash -l -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 18, 2005 7:26:17 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks
==========================================================================================================================================
1A00:1C00 1A
0
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
61036
1A01:1C01 1A
0
Enabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
48823
dscli>
dscli> lsremoteflash -l -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 18, 2005 7:26:39 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks
==========================================================================================================================================
1B00:1D00 1B
0
Enabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
44782
1B01:1D01 1B
0
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
61036
<< After the background copy has been completed >>
dscli> lsremoteflash -l -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 18, 2005 7:32:47 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
rcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks
==========================================================================================================================================
1A00:1C00 1A
0
Disabled Enabled Enabled
Disabled Enabled
Enabled
Enabled
0
Fri Nov 18 21:27:29 JST
2005 Fri Nov 18 21:27:29 JST 2005
1A01:1C01 1A
0
Disabled Enabled Enabled
Disabled Enabled
Enabled
Enabled
0
Fri Nov 18 21:27:29 JST
2005 Fri Nov 18 21:27:29 JST 2005
dscli>
dscli> lsremoteflash -l -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 18, 2005 7:32:54 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
rcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks
==========================================================================================================================================
1B00:1D00 1B
0
Disabled Enabled Enabled
Disabled Enabled
Enabled
Enabled
0
Fri Nov 18 21:27:44 JST
2005 Fri Nov 18 21:27:44 JST 2005
1B01:1D01 1B
0
Disabled Enabled Enabled
Disabled Enabled
Enabled
Enabled
0
Fri Nov 18 21:27:44 JST
2005 Fri Nov 18 21:27:44 JST 2005
Chapter 19. Global Copy examples
227
19.4.2 Periodical backup operation
Depending on your backup requirement and application acceptance, you schedule the
following scenario periodically, for example, daily. We show examples of the DS CLI
commands to control this procedure, based on the scenario illustrated in Figure 19-4.
Application running
source
1.Updated data is being sent
in normal Global Copy
A volume
target
B volume
C volume
Quiesce Application
2.Quiesce Application
source
source
target
3.Go-to-sync and suspend
RMC pairs
target
Restart Application
source
4.Restart Application
Now we have
consistent
data
target
Application running
source
5.Take FlashCopy B to C.
Return to 1.
Production site
FlashCopy
target
Recovery site
Figure 19-4 Global Copy periodical off-site backup
The explanation of the steps is the following:
1.
2.
3.
4.
5.
Normal Global Copy mode operation.
Quiesce the application at the production site.
Go-to-sync and suspend the Global Copy pairs.
Restart the application at the production site.
Take a FlashCopy B to C.
For a detailed description of the preceding steps, you can refer to 16.2.1, “Procedure to take a
consistent point-in-time copy” on page 194. Let us see now how you execute the procedure.
Step 1
In this step, we are operating in the normal Global Copy mode. The lspprc and
lsremoteflash commands show the status in Example 19-20 on page 229.
228
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 19-20 Step 1 of the periodic backup operation
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 8:38:26 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 18, 2005 8:38:43 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
============================================================================================================================
1A00:1C00 1A
0
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
1A01:1C01 1A
0
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 18, 2005 8:38:59 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
============================================================================================================================
1B00:1D00 1B
0
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
1B01:1D01 1B
0
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
Step 2
Depending on your application and platform, you should take the necessary actions to briefly
stop updates to the source volumes, in other words, quiesce the application.
Step 3
The DS CLI command examples for this step are shown in Example 19-21.
Example 19-21 The mkpprc -type mmir -wait -suspend
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir -suspend -wait 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 8:42:03 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1400:1A00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1401:1A01 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1500:1B00 successfully created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1501:1B01 successfully created.
<< It may take time to show below messages depending on the amount of the Out Of Sync Tracks remaining. >>
1/4 pair 1401:1A01 state: Suspended
2/4 pair 1400:1A00 state: Suspended
3/4 pair 1500:1B00 state: Suspended
4/4 pair 1501:1B01 state: Suspended
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 8:42:15 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1400:1A00 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
Chapter 19. Global Copy examples
229
Step 4
Depending on your application and platform, you should take the necessary actions to restart
your application and resume I/O activity to the source volumes.
Step 5
The DS CLI command examples for this step are shown in Example 19-22 and
Example 19-23. We use the resyncremoteflash command to perform the Incremental
FlashCopy from the production site. We add the -seqnum parameter with the mkremoteflash to
add the FlashCopy sequence number to identify this FlashCopy set.
We specify the -type gcp parameter with the resumepprc command to restart the normal
Global Copy mode.
Example 19-22 Take FlashCopy B to C
dscli> resyncremoteflash -record -persist -seqnum 20051118 -conduit IBM.1750-1300819/14 -dev
IBM.1750-1300247 1a00-1a01:1c00-1c01
Date/Time:
CMUC00175I
command to
CMUC00175I
command to
November 18, 2005 8:48:17 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
resyncremoteflash: Remote FlashCopy volume pair 1A00:1C00 successfully resynchronized. Use the lsremoteflash
determine copy completion.
resyncremoteflash: Remote FlashCopy volume pair 1A01:1C01 successfully resynchronized. Use the lsremoteflash
determine copy completion.
dscli>
dscli> resyncremoteflash -record -persist -seqnum 20051118 -conduit IBM.1750-1300819/15 -dev
IBM.1750-1300247 1b00-1b01:1d00-1d01
Date/Time:
CMUC00175I
command to
CMUC00175I
command to
dscli>
November 18, 2005 8:48:55 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
resyncremoteflash: Remote FlashCopy volume pair 1B00:1D00 successfully resynchronized. Use the lsremoteflash
determine copy completion.
resyncremoteflash: Remote FlashCopy volume pair 1B01:1D01 successfully resynchronized. Use the lsremoteflash
determine copy completion.
dscli> lsremoteflash -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 18, 2005 8:49:05 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
============================================================================================================================
1A00:1C00 1A
20051118
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
1A01:1C01 1A
20051118
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 18, 2005 8:49:15 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
============================================================================================================================
1B00:1D00 1B
20051118
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
1B01:1D01 1B
20051118
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
Enabled
Example 19-23 Resume normal Global Copy mode
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 8:51:26 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=====================================================================================================
1400:1A00 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1401:1A01 Suspended Host Source Metro Mirror 14
unknown
Disabled
Invalid
1500:1B00 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
1501:1B01 Suspended Host Source Metro Mirror 15
unknown
Disabled
Invalid
dscli>
dscli> resumepprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 18, 2005 8:51:50 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully resumed. This message is
being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully resumed. This message is
being returned before the copy completes.
230
IBM System Storage DS6000 Series: Copy Services in Open Environments
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully resumed. This message is
being returned before the copy completes.
CMUC00158I resumepprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully resumed. This message is
being returned before the copy completes.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 18, 2005 8:51:59 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
19.5 DS Storage Manager examples
In these examples, we use the DS Storage Manager Graphical User Interface to establish
paths and then establish and manage the Global Copy pairs.
Note: To create and monitor Global Copy pairs, you use the Metro Mirror panels.
19.5.1 Establish paths with the DS Storage Manager
To establish a new path from one LSS to another LSS, you follow this procedure:
1.
2.
3.
4.
Select Real-time manager.
Select Copy services.
Click Paths.
Select your source Storage Complex, Storage Unit, and LSS.
Figure 19-5 shows the Paths panel. Because there are no listed paths, we know that LSS 18
has no paths defined that use LSS 18 as the source LSS. In our example, we want to
configure two paths from the DS6000 serial 1300247 LSS 18 to the DS6000 serial 1300819
LSS 18. From the Select Actions pull-down menu, select Create.
Figure 19-5 Paths panel
We can now select the source LSS from the primary DS6000. When finished, click Next; see
Figure 19-6 on page 232.
Chapter 19. Global Copy examples
231
Figure 19-6 Select source LSS
We can now select the target Storage Complex, Storage Unit, and target LSS. Click Next
when finished. See Figure 19-7. If your intended Storage Unit is not presented as an
alternative, then you need to add the Storage Complex that contains that Storage Unit. See
Chapter 3, “DS Storage Manager (GUI)” on page 17. First, add the Storage Complex and
then restart the process to create paths.
Figure 19-7 Select target LSS
Now, we can select the source I/O ports; see Figure 19-8 on page 233. Because we want to
establish two paths, we select both I/O ports. We then click Next.
232
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 19-8 Select source I/O ports
We now select the target ports. Because there is only one target port available for each
source port, the choices are made for us, as shown in Figure 19-9.
Figure 19-9 Select target I/O ports
In the next panel, we can decide whether or not we want to create a Consistency Group; see
Figure 19-10 on page 234. For our example, we do not need a Consistency Group; therefore,
we do not select this option. We then click Next to get to the Verification panel.
Chapter 19. Global Copy examples
233
Figure 19-10 Select path options
Now, we can verify the information we entered and click Finish to create the logical paths,
see Figure 19-11.
Figure 19-11 Verification panel
Now that the path creation process is finished, the path panel shows us the paths we have
just created; see Figure 19-12 on page 235. Compare this to Figure 19-5 on page 231 where
we had no paths using LSS 18 as the source.
234
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 19-12 Paths
In the next section, we establish the Global Copy pairs.
19.5.2 Establish Global Copy pairs
In this section, we establish the Global Copy pairs. Once the volumes are in the copy pending
state, we synchronize the volumes to change the status to full duplex.
1.
2.
3.
4.
Select Real-time manager.
Select Copy services.
Select Metro Mirror.
Select your source Storage Complex, Storage Unit, Storage Image, and LSS.
From the Select Action pull-down menu, select Create. See Figure 19-13.
Figure 19-13 Metro Mirror panel
The following process guides you through the establishment of the Global Copy pairs. Our
first decision is the Volume Pairing Method. In this example, we select Automated volume
pair assignment, and click Next. See Figure 19-14 on page 236.
Chapter 19. Global Copy examples
235
Figure 19-14 Volume Pairing Method
Now, we select the source volumes as depicted in Figure 19-15. We then click Next.
Figure 19-15 Select source volumes
Now, we select the target volumes as depicted in Figure 19-16 on page 237. For the target
volumes, we need to select the target Storage Complex, Storage Unit, Storage Image (if the
remote machine is a DS8000), and Resource Type. Because in this example, the Resource
type that we chose is LSS, we need to select the LSS number. We then select the target
volumes, which because we chose automated pairing, are automatically paired to source
volumes. We then click Next.
236
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 19-16 Select target volumes
Now, we get the Copy options panel; see Figure 19-17. We select Global Copy and Perform
initial copy. Then, we click Next.
Figure 19-17 Select copy options
We now get the Verification panel; see Figure 19-18 on page 238. Here we can verify our
volume assignment. Click Finish to create the copy pairs.
Chapter 19. Global Copy examples
237
Figure 19-18 Verification
Now the volume pairs are in the copy pending state as shown in the State column of
Figure 19-19.
Figure 19-19 Global Copy volumes in copy pending state
19.5.3 Monitoring the copy status
You might want to see how many tracks of data need to be copied from the source to the
target volume. To do this, you need to select a volume pair and then choose Properties from
the Select Actions pull-down menu. A Properties panel displays. If you select Out-of-synch
tracks, you can see the number of tracks that need to be copied. In Figure 19-20 on
page 239, you can see there are zero tracks to copy.
238
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 19-20 Properties of a Global Copy pair
19.5.4 Convert to Metro Mirror (synchronous)
To have the volume pairs in a full duplex state, we need to synchronize them. In the Metro
Mirror panel, we select the volumes we want to synchronize and then from the Select Actions
pull-down menu, we choose Convert to synchronous. See Figure 19-21.
Figure 19-21 Selecting the action to convert to synchronous
We get a confirmation panel showing all the volumes that we just selected, as shown in
Figure 19-22. Note how the Type column in this figure shows Metro Mirror. This is because
we are converting from asynchronous Global Copy to synchronous Metro Mirror. Click OK to
continue.
Figure 19-22 Convert to synchronous confirmation panel
Chapter 19. Global Copy examples
239
Depending on the write operations on the primary site and the bandwidth of our Fibre
Channel links, after some time, the volumes achieve the full duplex state. Keep clicking
Refresh until the State column changes to full duplex as shown in Figure 19-23.
Figure 19-23 Metro Mirror panel
19.5.5 Suspend a pair
After the volumes are in duplex, we can stop the I/O for a short time on the primary site to
ensure that we have data consistency on the secondary site. After the I/O is stopped, we can
suspend the volumes. In the Metro Mirror panel, we select the volumes we want to suspend,
and then we choose Suspend from the Select Action pull-down. We then get to choose
whether to suspend the volumes at the source or at the target. We typically always suspend
at the source. We then click OK, and the volumes suspend. Figure 19-24 shows the volumes
in suspended state.
Figure 19-24 Metro Mirror panel
The volumes are now suspended. Now, we can restart the I/O on the application site. We
could now make a FlashCopy of the target volumes and resume the pairs afterwards. This is
not shown in this example.
240
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 6
Part
6
Global Mirror
This part of the book describes the IBM System Storage Global Mirror for DS6000 when used
in the open systems environment. We discuss the characteristics of Global Mirror and
describe the options for its setup. We also show which management interfaces you can use
as well as important aspects to consider when establishing a Global Mirror environment. This
part ends with examples of Global Mirror management and setup.
The following are the chapters included in this section:
򐂰
򐂰
򐂰
򐂰
򐂰
Global Mirror overview
Global Mirror options and configuration
Global Mirror interfaces
Performance and scalability
Examples
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
241
242
IBM System Storage DS6000 Series: Copy Services in Open Environments
20
Chapter 20.
Global Mirror overview
In this chapter, we provide an overview of what Global Mirror is. We also discuss the need for
data consistency at a distant site when synchronous data replication such as Metro Mirror is
not possible. This chapter then explains how Global Mirror works in a similar manner to a
distributed application, or a server and client relationship. Finally, in this chapter you see a
step-by-step process to establish a Global Mirror environment.
You can complement the information discussed in this chapter with the following IBM
publication and redbook:
򐂰 IBM System Storage DS6000 Series: Architecture and Implementation, SG24-6781
򐂰 IBM System Storage DS6000: Command-Line Interface User´s Guide, GC26-7922
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
243
20.1 Synchronous and asynchronous data replication
When replicating data over long distances, beyond 300 km, asynchronous data replication is
the valid approach. Because with the asynchronous techniques, the application I/O
processing at the local storage disk subsystem remains independent of the process of data
transmission to the remote storage disk subsystem.
Still with the asynchronous data replication techniques, we need to provide additional means
to ensure data consistency at the remote location. It even requires a solution that guarantees
data consistency not only within a single local-remote pair of storage disk subsystems but
also across multiple local and remote storage disk subsystems.
For a given pair of local and remote storage disk subsystems, a time stamp approach leads to
consistent data at the remote storage disk subsystem. Using this approach and by sorting the
I/Os by their time stamps, the write I/Os can be applied at the remote disk subsystem in the
same sequence as they arrived at the local disk subsystem.
But when the application volumes are spread across multiple storage disk subsystems, this
time stamp concept alone is insufficient to replicate data and provide data consistency at the
remote site. This additionally requires a Consistency Group concept.
In the rest of this section, we discuss how consistent data, that is, dependent writes, are
managed either with a synchronous technique such as Metro Mirror versus as opposed to
different asynchronous techniques such as Global Copy or Global Mirror.
20.1.1 Synchronous data replication and dependent writes
In normal operations, the nature of synchronous data replication preserves data consistency
for dependent writes. Dependent writes and data consistency are explained in detail in 12.4,
“Consistency Group function” on page 118. See Figure 20-1.
Host
Server
1
4
Storage Disk
Subsystem 2
Storage Disk
Subsystem 1
2C00
A
A1
A
Primary
Primary
Primary
Primary
source
local
Synchronous
2
3
Replicate
B1
target
remote
Figure 20-1 Synchronous data replication
In synchronous data replication methods such as Metro Mirror, an application write always
goes through the following four steps in Figure 20-1:
1. Write the data to the source storage disk subsystem cache. Note this does not end the I/O
and does not present an I/O complete to the application.
244
IBM System Storage DS6000 Series: Copy Services in Open Environments
2. Replicate the data from the source storage disk subsystem cache to the target storage
disk subsystem cache.
3. Acknowledge to the source storage disk subsystem that the data successfully arrived at
the target storage disk subsystem.
4. Present the successful I/O completion to the server, which is presented to the application
and concludes this I/O.
Now, the next I/O that depends on the successful completion of the previous I/O can be
issued.
When you have dependent writes across multiple storage disk subsystems, synchronous
data replication alone does not guarantee that you can restart applications at the remote site
without doing some previous data recovery.
Consider a database environment that spreads across multiple storage disk subsystems at
the local or remote site; see Figure 20-2.
Database
Subsystem
source
Switch over to remote site
target
1
Restart DB Subsystem 5
2
Storage Disk
Subsystem 1
A
Primary
A1
A
Primary
Primary
2C00
Log
Primary
Database
Subsystem
4
3
Storage Disk
Subsystem 2
Recover A2
Synchronous
Replicate
3C00
1
B1
Log'
6
Recover A2
Storage Disk
Subsystem 3
2D00
2D00
A
A
A
A2
Primary
Primary
Primary
Primary
Primary
Primary
DB
Storage Disk
Subsystem 5
2D00
2E00
A
A
A3
Primary
Primary
Primary
Primary
Primary
Primary
RECON
Storage Disk
Subsystem 4
3D00
A
B2
Synchronous
Primary
Primary
Primary
DB'
Replicate
Storage Disk
Subsystem 6
Synchronous
Replicate
A
B3
3E00
3
Primary
Primary
Primary
RECON'
Figure 20-2 Synchronous data replication and unnecessary recovery after local site failure
A database update usually involves three dependent write I/Os to ensure data consistency,
even in the event of a failure during this process; see Figure 20-2:
1. Update intent to the logging files; logging can happen to two logging files.
2. Update the data.
3. Indicate update complete to the logging files.
This sequence of I/Os is also called a two-phase commit process.
When you are in a remote copy environment, in an outage situation, the following sequence
can occur; see Figure 20-2:
1. Write update intent to logging volume A1.
Chapter 20. Global Mirror overview
245
2. Update to database on volume A2 eventually fails due to a replication problem from the
local site to the remote site. This might also be the beginning of a rolling disaster.
3. The database subsystem recognizes the failed write I/O and indicates, in a configuration
file on volume A3, that one or more databases on A2 have to be recovered due to an I/O
error. This gets replicated to the remote site, because the paths for this storage disk
subsystem pair are still working and this particular source storage disk subsystem did not
fail yet.
4. The failure eventually progressed and failed the local site completely. The database
subsystem is restarted at the remote site after switching over to the remote site.
5. At startup, the database subsystem discovers in its configuration file that the database on
A2-B2 has to be recovered as indicated before. This is actually unnecessary because the
data was synchronously replicated and both related volumes, A2 and B2, are identical and
at the very same level of data currency, which is the moment previous to the error.
Nevertheless, the database subsystem still recovers all databases that are marked for
recovery as specified in the information found in the configuration files on A3-B3.
6. This recovery is actually unnecessary, because the data in A2-B2 is perfectly synchronized
due to the synchronous replication approach. Nonetheless, the recovery takes place,
because there was no automation window in place to freeze the configuration, which
would have removed the necessary paths between the local and the remote sites,
therefore, ensuring that no further I/Os continue to take place.
This does not happen with automated solutions, such as GDPS (System z environments) and
eRCMF (open systems environments), that make use of the freeze capabilities of the IBM
System Storage DS6000. Had one of those automation software products been there, after
failing over to the remote site, the database subsystem would have just restarted without the
necessity for a lengthy database recovery.
Figure 20-3 illustrates a rolling disaster situation where we have automation software that
makes use of the freeze capability of the DS6000.
Database
Subsystem
1
source
6
3
A
Primary
A1
A
Primary
Primary
Storage Disk
Subsystem 2
Synchronous
Redrive
I/O
2
A
A
A2
A
A
A3
Primary
Primary
Primary
Primary
Primary
Primary
RECON
Replicate
5
Storage Disk
Subsystem 3
2D00
2D00
Storage Disk
Subsystem 5
2D00
2E00
3C00
4
Log
Primary
4
Synchronous
4
Restart
target
2C00
Primary
Primary
Primary
Primary
Primary
Primary
DB
Switch over to remote site
Recover A2
Hold I/O
Storage Disk
Subsystem 1
Database
Subsystem
7
Replicate
4
Synchronous
Replicate
8
8
B1
1
Log'
Storage Disk
Subsystem 4
3D00
A
B2
Primary
Primary
Primary
DB'
Storage Disk
Subsystem 6
A
B3
3E00
Primary
Primary
Primary
RECON'
Figure 20-3 Synchronous data replication, freeze, and restart without recovery required
246
IBM System Storage DS6000 Series: Copy Services in Open Environments
8
The sequence of events in Figure 20-3 on page 246 is the following:
1. Update intent to logging file.
2. Link between A2 and B2 becomes unavailable, or the storage disk subsystem with A2 fails,
and a rolling disaster develops.
3. The first attempt to write to the database volume A2 triggers a queue full condition. This is
also called an automation window.
4. Within this automation window, a freeze operation removes all related paths between both
sites for the selected LSSs. From now on, no data is replicated any longer between the
selected LSSs.
5. After the queue full condition expires, or an un-freeze operation is issued to end the freeze
period, the I/O is redriven, and it might successfully complete when the pair is suspended,
or might fail when the storage disk subsystem 3 is not available any longer.
6. When the I/O fails, then a recovery required is written to the database configuration file on
A3. Note volume A3 is no longer replicated due to the previous freeze. Note also that
volumes A2 and B2 are still identical and at the very same level of data currency, which is
the moment previous to the error.
7. The local site eventually becomes unavailable and a switchover to the recovery site is
carried out.
8. The database subsystem restart process discovers a no recovery required indication,
because all related data is consistent and the recovery required indication issued in step 6
was not transmitted to B3. Then, the database subsystems continue immediately after the
restart phase is finished.
Summary
In summary, the following characteristics are typical of a synchronous data replication
technique:
򐂰 Application write I/O response time is affected; this can be modeled and predicted.
򐂰 Local and remote copies of data are committed to both storage disk subsystems before
the host write I/O is complete.
򐂰 Data consistency is always maintained at the remote site as long as no failures occur. If a
rolling disaster occurs, freeze and run are needed to maintain consistency.
򐂰 Bandwidth between both sites has to scale with the peak write I/O rate.
򐂰 Data at the remote site is always current.
򐂰 No extra means, such as additional journal volumes or tertiary copy, are required.
򐂰 A tier 7 solution is achieved with automation software.
This is different with an asynchronous data replication approach. We still require data
consistency at a distant site in order to just restart at the distant site when the local site
becomes unavailable.
20.1.2 Asynchronous data replication and dependent writes
In normal operations, for asynchronous data replication, data consistency for dependent
writes is preserved depending on the technique used to replicate the data. Dependent writes
and data consistency are explained in detail in 12.4, “Consistency Group function” on
page 118. For example, Global Copy that we explained in Part 5, “Global Copy” on page 183,
and Global Mirror that we are explaining now use different techniques.
Chapter 20. Global Mirror overview
247
An asynchronous remote copy approach is usually required when the distance between the
local site and the remote site is beyond an efficient distance for a synchronous remote copy
solution. Metro Mirror provides an efficient synchronous approach for up to 300 km, when
utilizing Fibre Channel links.
Host
Server
1
Storage Disk
Subsystem 1
2C00
2
A
A1
A
Primary
Primary
Primary
Primary
source
local
Storage Disk
Subsystem 2
3
Asynchronous
4
Replicate
B1
target
remote
Figure 20-4 Asynchronous data replication
In an asynchronous data replication environment, an application write I/O goes through the
following steps; see Figure 20-4:
1. Write application data to the source storage disk subsystem cache.
2. Present successful I/O completion to the host server. The application can then
immediately schedule the next I/O.
3. Replicate the data from the source storage disk subsystem cache to the target storage
disk subsystem cache.
4. Acknowledge to the source storage disk subsystem that data successfully arrived at the
target storage disk subsystem.
From the previous procedure, note how in an asynchronous type technique, the data
transmission and the I/O completion acknowledge are independent processes. This results in
virtually no application I/O impact, or at most, to a minimal degree only. This also is
convenient when you need to replicate over long distances.
Global Copy asynchronous technique
On its own, an asynchronous technique, such as Global Copy, provides no guarantee that the
sequence of arrival of the applications write I/Os to the source volumes is preserved at the
remote site. In other words, the order of dependent writes is not preserved at the remote site.
We illustrate this in Figure 20-5 on page 249.
248
IBM System Storage DS6000 Series: Copy Services in Open Environments
Host
Server
a
1
b
2
Storage Disk
Subsystem 2
A
aPrimary
A
Primary
Primary
b
Primary
2C00
Non synchronous
Replicate
b
3
Storage Disk
Subsystem 1
4
source
target
a in transit
local site
remote site
Figure 20-5 Global Copy sequence of data arrival not preserved at remote site
Global Copy, by itself as an asynchronous data replication method, does not provide data
consistency at the remote site.
In Figure 20-5, the sequence of data arrival at the local site is record b after record a. Due to
the way Global Copy cycles through its out-of-sync bitmap, it can replicate record a to the
remote site after record b. This assumes that, for example, record a is written to a source disk
location that the Global Copy replication cycle just passed before. Global Copy might
approach another disk location that was just updated with record b. Then, record b gets
replicated to the remote site before the replication cycle starts from the beginning and finds
record a.
This situation can get even worse when it involves multiple storage disk subsystems at the
local site. As Figure 20-6 shows, when using Global Copy, dependent writes are not
preserved at the recovery site when a failure occurs. This situation occurs both at a particular
storage disk subsystem pair as well as across multiple storage disk subsystems.
Database
subsystem
local site
source
Storage Disk
Subsystem 1
remote site
target
Storage Disk
Subsystem 2
3
1
A
Primary
A
Primary
Primary
2C00
2C00
3C00
non synchronous
Primary
Log
3
B
1
Log'
Replicate
Storage Disk
Subsystem 3
2D00
2D00
A
A
A
Primary
Primary
Primary
Primary
Primary
Primary
DB
2
non synchronous
Replicate
Storage Disk
Subsystem 4
3D00
A
B
Primary
Primary
Primary
DB'
Figure 20-6 Global Copy asynchronous data replication involving multiple disk subsystems
Chapter 20. Global Mirror overview
249
Notice that the numbers in Figure 20-6 on page 249 indicate the transfer of data as well as
the sequence of its corresponding write I/Os. Record 3 might be replicated to storage disk
subsystem 2 before record 1 arrives to storage disk subsystem 2. Record 2 might never make
it to storage disk subsystem 4 due to the connectivity problem between storage disk
subsystem 3 and storage disk subsystem 4, which in turn might be the beginning of a rolling
disaster. So, when using an asynchronous technique, such as Global Copy, consistent data
cannot be guaranteed at the remote site without any additional measures, functions, and
procedures.
The challenge is to provide data consistency at the distant site at any time and independent of
the combination of storage disk subsystems involved. One solution is to combine Global Copy
with FlashCopy and create a three-copy solution. This requires a series of additional steps
that you must organize and perform:
1. At the local site, temporarily pause the application write I/Os on the source A volumes.
2. Wait for and make sure that the source A volumes and the target B volumes become
synchronized. Note the challenge to manage all volumes.
3. Using FlashCopy, create a point-in-time (PiT) copy of the B volumes at the remote site.
4. The FlashCopy targets, that is, the C volumes, then hold a copy of the A volumes at the
time that the application was paused or stopped. In this manner, data consistency is kept
at the remote site.
5. Restart or resume the application write I/O activity to the A volumes.
Global Mirror asynchronous technique
If we could incorporate the previous five basic steps into the storage disk subsystem
microcode, this would basically be an efficient asynchronous mirroring technique that would
allow the replication of data over long distances, without impacting the application I/O
response time. Its operation would be transparent and autonomic from the user’s point of
view, and most importantly, would provide a consistent copy of the data at the remote site at
all times. This is how Global Mirror works.
To accomplish the necessary activities with minimum impact on the application write I/O,
Global Mirror introduces a smart bitmap approach in the source storage disk subsystem. With
this, Global Mirror can resume the application I/O processing immediately after a very brief
serialization period for all involved source storage disk subsystems. This brief serialization
periodically occurs at the very beginning of a sequence of events similar to the what we
outlined previously. In the following chapters, we explain in detail the characteristics of Global
Mirror, how it operates, and how to manage it.
Summary
In summary, an asynchronous data replication technique should provide the following
characteristics:
򐂰 Data replication to the remote site that is independent from application write I/O
processing at the local site. This has no impact, or at least only minimal impact, on the
application local write I/O response time.
򐂰 Data consistency and dependent writes always maintained at the remote site.
򐂰 Data currency at the remote site lags a little behind the local site. The remote site is
always behind the current local site. In peak write workloads, this difference is going to
increase. Global Mirror does not throttle host write I/Os to eventually manage this
discrepancy.
250
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 The bandwidth requirement between the local and the remote sites does not have to be
configured for peak write workload; link bandwidth utilization is improved over
synchronous solutions.
򐂰 Tertiary copies are required at the remote site to preserve data consistency.
򐂰 Data loss in disaster recovery situations is limited to the data in transit plus the data that
might still be in the queue at the local site that is waiting to be replicated to the recovery
site.
All of the previous attributes are characteristics of Global Mirror.
20.2 Basic concepts of Global Mirror
It is important to understand that Global Mirror works like a distributed application. A
distributed application is usually built on a server to client relationship. The server functions
as a supervisor and instructs the client. The client is able to do some work in an autonomic
fashion but relies on the coordination efforts from the server; see Figure 20-7.
Task1
Task2
Local site
Remote site
Task1
Server
Client
Task3
Network
Task3
Client
Task2
Client
Figure 20-7 Distributed application
The server distributes the work to its clients. The server also coordinates all individual
feedback from the clients and decides further actions based on this feedback.
Looking at Figure 20-7, it is obvious that the communication paths between the server and all
of its clients are key. Without communication paths between these four components, the
functions eventually come to a complete stop. Matters get more complicated when the
communication fails unexpectedly in the middle of an information exchange between the
server and its clients or to some of its clients.
Usually a two-phase commit process helps to provide a consistent state for certain functions
and whether the functions have successfully completed at the client site. Once a function
successfully completes and acknowledges this to the server, the server progresses to the
next functional task. At the same time, the server tries to do as much as possible in parallel to
minimize the impact on throughput due to serialization and checkpoints.
Chapter 20. Global Mirror overview
251
When certain activities are dependent on each other, the server must coordinate these
activities to ensure they occur in the proper sequence.
The server and the client can be replaced by terms, such as Master and Subordinate; see
Figure 20-8. These terms are used later when discussing Global Mirror.
Local site
Remote site
FlashCopy 1
FlashCopy 1
Master
Target
Subord Flash 2
source
Network
FlashCopy2
Subordinate
FlashCopy2
Target
source
Figure 20-8 Global Mirror as a distributed application
Figure 20-8 shows the basic Global Mirror structure. A Master coordinates all efforts within a
Global Mirror environment. Once the Master is started and manages a Global Mirror
environment, the Master issues all related commands over inband communication to its
attached Subordinates at the local site. This can include a subordinate within the Master
itself. This communication between the Master and an internal Subordinate is transparent
and does not require any extra attention from the user. The Subordinates use inband
communication to communicate with their related target storage disk subsystems at the
remote site. The Master also receives all acknowledgements from its Subordinates and their
targets, and coordinates and serializes all the activities in the session.
When the Master and Subordinate are in a single storage disk subsystem, the Subordinate is
internally managed by the Master. With two or more storage disk subsystems at the local site,
which participate in a Global Mirror session, the Subordinate is external and needs separate
attention when you create and manage a Global Mirror session or environment.
The following sections explain how Global Mirror works and how Global Mirror ensures
consistent data at any time at the remote site. First, we go through the process of how to
create a Global Mirror environment. At the same time, this explanation gives us an initial
insight about how Global Mirror works.
20.3 Set up a Global Mirror session
Global Mirror, as a long distance remote copy solution, is based on an efficient combination of
Global Copy and FlashCopy functions. It is the microcode that provides, from the user
perspective, a transparent and autonomic mechanism to intelligently utilize Global Copy in
conjunction with certain FlashCopy operations to attain consistent data at the remote site.
252
IBM System Storage DS6000 Series: Copy Services in Open Environments
In order to understand how Global Mirror works, we first explain how to create and start a
Global Mirror environment, that is, a Global Mirror session. This is a step-by-step approach to
help you to understand the operational aspects of Global Mirror in more detail.
20.3.1 Simple configuration to start
In order for you to understand each step and for us to show the principles, we start with a
simple application environment where a host makes write I/Os to a single application volume
(A); see Figure 20-9.
Host
Write I/O
A
Primary
Primary
Primary
A
A
Primary
Primary
Primary
Primary
Primary
Local site
Remote site
Figure 20-9 Start with a simple application environment
20.3.2 Establish connectivity to the remote site
Now, we add a distant site, which has a storage disk subsystem (B), and we want to
interconnect sites A and B; see Figure 20-10.
Host
Write I/O
A
A
Primary
Primary
Primary
Local site
A
B
Primary
Primary
Primary
Global Copy path
Primary
Primary
Remote site
FCP port
Figure 20-10 Establish Global Copy connectivity between both sites
Note in Figure 20-10 that we establish Global Copy paths over an existing network. This
network can be based on a Fibre Channel Protocol (FCP) transport technology or on an
IP-based network.
Chapter 20. Global Mirror overview
253
Global Copy paths are logical connections that are defined over the physical links that
interconnect both sites. Note that all remote mirror and copy paths (that is, Metro Mirror,
Global Copy, and Global Mirror paths) are similar and are based on FCP. The term Global
Copy path just denotes that the path is intended for Global Copy use.
20.3.3 Create Global Copy relationships
Host
Write I/O
A
A
Primary
Primary
Primary
Global Copy
source
target
copy pending
copy pending
O
O
S
A
B
Primary
Primary
Primary
Primary
Primary
OOS: Out-of-sync bit map
Local site
Remote site
Figure 20-11 Establish Global Copy volume pair
Next, we create a Global Copy relationship between the source volume and the target
volume; see Figure 20-11. This step changes the target volume state from simplex (no
relationship) to target copy pending. This copy pending state applies to both volumes, source
copy pending and target copy pending. At the same time, data starts to be copied from the
source volume to the target volume. After a first complete pass through the entire A volume,
Global Copy constantly scans through the out-of-sync bitmap. This bitmap indicates changed
data as it arrives from the applications to the source disk subsystem. Global Copy replicates
the data from the A volume to the B volume based on this out-of-sync bitmap.
In the following paragraphs, we refer to the source volume as the A volume and to the target
volume as the B volume for simplicity.
Global Copy does not immediately copy the data as it arrives to the A volume. Instead, this is
an asynchronous process. As soon as a track is changed by an application write I/O, it is
reflected in the out-of-sync bitmap with all the other changed tracks. There can be several
concurrent replication processes that work through this bitmap, therefore, maximizing the
utilization of the high bandwidth Fibre Channel links.
This replication process keeps running until the Global Copy volume pair A-B is explicitly or
implicitly suspended or terminated. Note that a Global Mirror session command, for example,
to pause or to terminate a Global Mirror session, does not affect the Global Copy operation
between both volumes.
At this point, data consistency does not yet exist at the remote site.
254
IBM System Storage DS6000 Series: Copy Services in Open Environments
20.3.4 Introduce FlashCopy
FlashCopy is an integral part of the Global Mirror solution, and it is the next step in the course
of establishing a Global Mirror session; see Figure 20-12.
Host
Write I/O
FlashCopy
A
A
Primary
Primary
Primary
A
B
Primary
Primary
Primary
Global Copy
target
source
copy pending
copy pending
Primary
Primary
C
tertiary
O
O
S
SBM: Source Bit Map
TBM: Target Bit Map
Local site
S
B
M
T
B
M
Remote site
Figure 20-12 Introducing FlashCopy in the Global MIrror solution
The focus is now on the remote site. Figure 20-12 shows a FlashCopy relationship with a
Global Copy target volume for the FlashCopy source volume. Volume B is now both a Global
Copy target volume and a FlashCopy source volume at the same time. The corresponding
FlashCopy target volume is in the same disk subsystem.
Note that this FlashCopy relationship has certain attributes that are typical and required when
creating a Global Mirror session. These attributes are:
򐂰 Inhibit target write: Protect the FlashCopy target volume from being modified by anyone
other than Global Mirror-related actions.
򐂰 Enable change recording: Apply changes that have occurred to the source volume
between FlashCopy establish operations, from the source volume to the target volume
only, except for the first time when FlashCopy is initially established.
򐂰 Make relationship persistent: Keep the FlashCopy relationship until it is explicitly or
implicitly terminated. This parameter is automatic due to the nocopy property.
򐂰 Nocopy: Do not initiate background copy from source to target, but keep the set of
FlashCopy bitmaps required for tracking the source and target volumes. These bitmaps
are established the first time that a FlashCopy relationship is created with the nocopy
attribute. Before a track in the source volume B is modified, between Consistency Group
creations, the track is copied to the target volume C to preserve the previous point-in-time
copy. This includes updates to the corresponding bitmaps to reflect the new location of the
track that belongs to the point-in-time copy. Note that each Global Copy write to its target
volume within the window of two adjacent Consistency Groups can cause FlashCopy I/O
operations.
Chapter 20. Global Mirror overview
255
20.3.5 Define the Global Mirror session
Creating a Global Mirror session does not involve any volume within the local or the remote
sites. Our focus is back again on the local site; see Figure 20-13.
Host
Define Global Mirror session
FlashCopy
01
A
B
Primary
Primary
Global Copy
target
copy pending
Primary
Primary
C
tertiary
S
B
M
Local site
T
B
M
Remote site
Figure 20-13 Define Global Mirror session
Defining a Global Mirror session creates a type of token, which is a number between 1 and
255. This number represents the Global Mirror session.
This session number is defined at the LSS level. Each LSS that has volumes, which are part
of the session, needs a corresponding define session command. Currently, only a single
session is allowed per storage disk subsystem. The architecture allows for more than one
session, and there are plans to exploit this architecture in the future.
256
IBM System Storage DS6000 Series: Copy Services in Open Environments
20.3.6 Populate the Global Mirror session with volumes
The next step is the definition of volumes in the Global Mirror session. The focus is still on the
local site; see Figure 20-14. Note that only Global Copy source volumes are meaningful
candidates to become a member of a Global Mirror session.
Host
Add Global Copy source volumes to Global Mirror session
FlashCopy
01
A
Primary
Primary
A
source
A
B
Primary
Primary
Global Copy
target
copy pending
copy pending
Primary
Primary
C
tertiary
O
O
S
S
B
M
Local site
T
B
M
Remote site
Figure 20-14 Add Global Copy source volumes to Global Mirror session
This process adds source volumes to a list of volumes in the Global Mirror session. But at this
stage, it still does still not perform Consistency Group formation.
Note that Global Copy is replicating the application updates that arrive to the Global Copy
source volumes onto the Global Copy target volumes. Initially, the Global Copy source
volumes are placed in a join pending state. Once a Consistency Group is formed, the Global
Copy source volume is then added to the session and placed in an in session state.
Nothing happens to the C volume after its initial establishment in a Global Mirror session.
Chapter 20. Global Mirror overview
257
20.3.7 Start the Global Mirror session
This is the last step, and it starts the Global Mirror session. Upon this step, Global Mirror
starts to form Consistency Groups at the remote site. As Figure 20-15 indicates, the focus
here is on the local site with the start command issued to an LSS in the source storage disk
subsystem. With this start command, you set the Master storage disk subsystem and the
Master LSS. From now on, session-related commands must go through this Master LSS.
Host
Start Global Mirror session
FlashCopy
01
A
A
Primary
Primary
A
B
Primary
Primary
Global Copy
source
target
copy pending
copy pending
Primary
Primary
C
tertiary
C
R
O
O
S
CR: Change recording bit map
Local site
S
B
M
C
R
T
B
M
Remote site
Figure 20-15 Start Global Mirror
This start command triggers events that involve all of the volumes within the session. This
includes very fast bitmap management at the local storage disk subsystem, issuing inband
FlashCopy commands from the local site to the remote site, and verifying that the
corresponding FlashCopy operations successfully finished. This all happens transparently at
the microcode level of the related storage disk subsystems that are part of the session, and in
an autonomic fashion from the user’s perspective.
All C volumes that belong to the Global Mirror session comprise the Consistency Group. Now
let us see more details about the Consistency Group creation at the remote site.
20.4 Consistency Groups
To achieve the goal of creating a set of volumes at a remote site that contains consistent
data, asynchronous data replication alone is not enough. Asynchronous data replication must
be complemented with either some type of journal or a tertiary copy of the target volume. With
Global Mirror, this third copy is automatically created through the use of FlashCopy.
The microcode automatically triggers a sequence of autonomic events to create a set of
consistent data volumes at the remote site. We call this set of consistent data volumes a
Consistency Group. The following sections describe the sequence of events that creates a
Consistency Group.
258
IBM System Storage DS6000 Series: Copy Services in Open Environments
20.4.1 Consistency Group formation
The creation of a Consistency Group requires three steps that are internally processed and
controlled by the microcode. Outside of the Licensed Internal Code (LIC), these steps are
fully transparent and do not require any other external code invocation or user action.
The numbers in Figure 20-16 illustrate the sequence of the events involved in the creation of
a Consistency Group. This illustration provides a high level view that is sufficient to
understand how this process works.
Start
1
Done
Start
Serialize all
Global Copy
source volumes
C
R
A1
source
2
O
O
S
Done
Drain data from local to remote site
Start
3
Done
Perform
FlashCopy
B1
C1
target
tertiary
C
R
A2
source
Local
O
O
S
B2
C2
target
tertiary
Remote
Figure 20-16 Formation of a consistent set of volumes at the remote site
Note that before step 1 and after step 3, Global Copy constantly scans through the
out-of-sync bitmaps and replicates data from A volumes to B volumes as described in 20.3.3,
“Create Global Copy relationships” on page 254.
When the creation of Consistency Group is triggered, the following steps occur in a serial
fashion:
1. Serialize all Global Copy source volumes. This imposes a brief hold on all incoming write
I/Os to all involved Global Copy source volumes. Once all source volumes are serialized,
the pause on the incoming write I/O is released and all further write I/Os are now noted in
the change recording bitmap. They are not replicated until step 3 is done, but application
write I/Os can immediately continue.
2. Drain includes the process to replicate all remaining data that is indicated in the
out-of-sync bitmap and still not replicated. Once all out-of-sync bitmaps are empty (note
that “empty” here does not mean in a literal sense), step 3 is triggered by the microcode
from the local site.
3. Now the B volumes contain all data as an almost point-in-time copy, and are consistent
due to the serialization process in step 1 and the completed replication or drain process in
step 2. Step 3 is now a FlashCopy that is triggered by the local system’s microcode as an
inband FlashCopy command to volume B, as FlashCopy source, and volume C, as
FlashCopy target volume. Note that this FlashCopy is a two-phase process. First, the
FlashCopy command affects all involved FlashCopy pairs in the Global Mirror session.
Then, the Master collects the feedback and all incoming FlashCopy completion messages.
When all FlashCopy operations are successfully completed, then the Master concludes
that a new Consistency Group has been successfully created.
Chapter 20. Global Mirror overview
259
FlashCopy applies here only to changed data since the last FlashCopy operation. This is
because the start change recording property was set at the time when the FlashCopy
relationship was established. The FlashCopy relationship does not end due to the nocopy
property, which is also assigned at FlashCopy establish time; see 20.3.4, “Introduce
FlashCopy” on page 255. Note that the nocopy attribute does not create a physical
background copy for any tracks. Only bitmaps are maintained and updated.
Once step 3 is complete, a consistent set of volumes has been created at the remote site.
This set of volumes, the C volumes, represents the Consistency Group.
At this very moment, the B volumes and the C volumes are also, but only for this brief moment,
equal in their content. Immediately after the FlashCopy process is logically complete, the
local systems’ microcode is notified to continue with the Global Copy process. In order to
replicate the changes to the A volumes that occurred during the step 1 to step 3 window, the
change recording bitmap is mapped against the empty out-of-sync bitmap, and from now on,
all arriving write I/Os end up again in the out-of-sync bitmap. From now on, the conventional
Global Copy process, as outlined in 20.3.3, “Create Global Copy relationships” on page 254,
continues until the next Consistency Group creation process is started.
20.4.2 Consistency Group parameters
In the previous section, we described the steps followed during the creation of a Consistency
Group. These steps can be adjusted by means of the tuning parameters of Global Mirror.
1
1
2
Maximum
Maximum
coordination
drain
time
time
Serialize all
Global Copy
souce volumes
2
3
Drain data from local to remote site
CG interval
time
1
3
Perform
FlashCopy
2
3
Figure 20-17 Consistency Group tuning parameters
Global Mirror provides a set of three externalized parameters that can be used for tuning the
Consistency Group formation process, overriding the default values; see Figure 20-17:
򐂰 Maximum coordination time: Dictates, for the Global Copy source volumes that belong to
this Consistency Group, how long a host write I/O might be impacted due to coordination
and serialization activities. This time is measured in milliseconds (ms). The default is 50
ms.
򐂰 Maximum drain time: Is the maximum time allowed for draining the out-of-sync bitmap
once the process to form a Consistency Group is started and step 1 successfully
completed. The maximum drain time is specified in seconds. The default is 30 seconds.
If the maximum drain time is exceeded, Global Mirror fails to form the Consistency Group
and evaluates the current throughput of the environment. If this indicates that another
drain failure is likely, Global Mirror stays in Global Copy mode while regularly reevaluating
the situation to determine when to start to form the next Consistency Group. If this persists
260
IBM System Storage DS6000 Series: Copy Services in Open Environments
for a significant period of time, then eventually Global Mirror forces the formation of a new
Consistency Group. In this way, Global Mirror ensures that during periods when the
bandwidth is insufficient, production performance is protected and data is transmitted to
the remote site in the most efficient manner possible. When the peak activity has passed,
Consistency Group formation resumes in a timely fashion.
򐂰 Consistency Group interval time: Once a Consistency Group (CG) has been created, the
CG interval time determines how long to wait before starting the formation of the next
Consistency Group. This is specified in seconds, and the default is zero (0) seconds. Zero
seconds means that Consistency Group formation happens constantly. As soon as a
Consistency Group is successfully created, the process to create a new Consistency
Group starts again immediately.
There is no external parameter to limit the time for the FlashCopy operation. This is due to its
very fast bitmap manipulation process.
In the first step, the serialization step, when the Consistency Group spans over more than
one source disk subsystem, Global Mirror not only serializes all related Global Mirror source
volumes but also does the coordination with other storage disk subsystems.
Global Mirror utilizes a distributed approach as well as a two-phase commit technique for
activities between the Master and its Subordinate LSSs. The communication between the
local and the remote site is organized through the Subordinate LSSs. The Subordinates
function here partly as a transient component for the Global Mirror activities, which are all
triggered and coordinated by the Master. This distributed concept always provides a set of
data consistent volumes at the remote site independently of the number of involved storage
disk subsystems at the local or at the remote site.
Chapter 20. Global Mirror overview
261
262
IBM System Storage DS6000 Series: Copy Services in Open Environments
21
Chapter 21.
Global Mirror options and
configuration
In this chapter, we provide a detailed description of Global Mirror options, including how to
create a Global Mirror environment and how to remove it. We discuss how to change Global
Mirror tuning parameters and how to modify an active Global Mirror session. We also discuss
a scenario where a site switch is performed due to the local site failure.
Global Mirror is intended for long distance data replication, and for this, it relies on the
network infrastructure and components used between the remote sites. This chapter does not
cover network-related topics. For information about these topics, refer to the redbook, IBM
TotalStorage Business Continuity Solutions Guide, SG24-6547.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
263
21.1 Terminology used in Global Mirror environments
First, let us review and further define some of the terms and new elements we have
presented so far, and that are commonly used when working in a Global Mirror context.
Dependent writes
If the start of one write operation is dependent upon the completion of a previous write, the
writes are dependent. Application examples for dependent writes are databases with their
associated logging files. For instance, the database logging file is updated after a new entry
has been successfully written to a table space.
Compliance at the remote site, with the chronological order of dependent writes to source
volumes, is the basis to provide consistent data at the remote site through remote copy
operations.
Dependent writes are discussed in detail in the Metro Mirror part of 12.4.1, “Data consistency
and dependent writes” on page 119.
Consistency
The consistency of data is ensured if the order of dependent writes to disks or disk groups is
maintained. With Copy Services solutions, the data consistency at the remote site is
important for the usability of the data. Consistent data, for instance, gives you the ability to
perform a database restart rather than a database recovery that can take hours or even days.
Data consistency across all target volumes spread across multiple storage disk subsystems
is essential for logical data integrity.
Data currency
This term describes the difference of time since the last data was written at the production
site, as opposed to the time that the same data was written to the remote site. It determines
the amount of data you have to recover at the remote site after a disaster. This is also called
recovery point objective (RPO). Only synchronous copy solutions, such as Metro Mirror, have
a currency of zero or RPO = zero. All asynchronous copy solutions have a data currency that
is greater than zero.
With Global Mirror, a data currency of a few seconds can be achieved, while data consistency
is always maintained by the Consistency Group process within Global Mirror.
For asynchronous replication, this means that the data is not replicated at the same time that
the local I/O happens, but the data is replicated with certain time lag. Examples of different
asynchronous replication methods are:
򐂰 Global Copy is an asynchronous method that does not guarantee consistent data at the
remote site.
򐂰 z/OS Global Mirror (formerly XRC) is an asynchronous replication method that guarantees
consistent data at the remote site.
򐂰 Global Mirror is also an asynchronous replication method that provides consistent data at
the remote site.
Session
A Global Mirror session is a collection of volumes that are managed together when creating
consistent copies of data volumes. This set of volumes can reside in one or more LSSs and
one or more storage disk subsystems at the local site. Open systems volumes and z/OS
volumes can both be members of the same session.
264
IBM System Storage DS6000 Series: Copy Services in Open Environments
When you start or resume a session, the creation of Consistency Groups is performed, and
the Master storage disk subsystem controls the session by communicating with the
Subordinate storage disk subsystems.
There is also a session concept at the LSS level. But all LSS sessions are combined and
grouped together within a Global Mirror session.
Master
The Master is a function inside a source storage disk subsystem that communicates with
Subordinates in other storage disk subsystems, and controls the creation of Consistency
Groups while managing the Global Mirror session. The Master is defined when the start
command for a session is issued to any LSS in a source storage disk subsystem. This
determines who becomes the Master storage disk subsystem.
The Master needs communication paths over Fibre Channel links to any one of the LSSs in
each Subordinate disk storage subsystem.
Subordinate
The Subordinate is a function inside a source storage disk subsystem that communicates
with the Master and is controlled by the Master. At least one of the LSSs of each Subordinate
source storage disk subsystems needs Fibre Channel communication paths to the Master.
This enables the communication between the Master and the Subordinate, and is required to
create Consistency Groups of volumes that spread more than one storage disk subsystem.
If all the volumes of a Global Mirror session reside in one source storage disk subsystem,
then no Subordinate is required, because the Master can communicate to all LSSs inside the
source storage disk subsystem.
Consistency Group
This is a group of volumes in one or more target storage disk subsystems whose data must
be kept consistent at the remote site.
Local site
This is the site that contains the production servers. This and other publications use this term
interchangeably with the terms primary, production, or source site.
Remote site
This is the site that contains the backup servers of a disaster recovery solution. This and
other publications use this term interchangeably with the terms secondary, backup, standby,
or target site.
21.2 Create a Global Mirror environment
In the following section, we recommend a certain sequence of steps to establish a Global
Mirror environment. This is independent of the interface used to create a Global Mirror
environment. Note that for illustration purposes, parameters and keywords are used in this
section that apply to the DS CLI commands.
We explain a basic Global Mirror setup, illustrated in Figure 21-1 on page 266.
Chapter 21. Global Mirror options and configuration
265
Subordinate
01
A
Primary
Primary
A
Global Copy
Network
A
B
Primary
Primary
target
source
copy pending
copy pending
Primary
Primary
C
tertiary
paths
Local site
01
A
Primary
Master Primary
A
source
Remote site(s)
A
B
Primary
Primary
Network
target
copy pending
copy pending
Primary
Primary
C
tertiary
Global Copy
Figure 21-1 Global Mirror basic configuration with Master and Subordinate disk subsystems
The order of commands to create a Global Mirror environment is not completely fixed and
allows for some variation. In order to be consistent with other sources and to not confuse the
user with a different sequence of commands, we recommend using a meaningful order. We
suggest the following steps to create a Global Mirror environment:
1. Define the paths between the local site and the remote site. In Figure 21-1, these are the
logical communication paths between corresponding LSSs at the local site and the remote
site, defined over Fibre Channel physical links that are configured over the network. Global
Copy source LSSs are represented by the A volumes and their corresponding Global Copy
target LSSs by the B volumes.
You can also define logical communication paths here between the Master and any
Subordinate storage disk subsystem that will be part of the Global Mirror session. Note
that these paths are defined between source storage disk subsystems at the local site.
With only a single source storage disk subsystem, you do not need to define paths to
connect internal LSSs within the source storage disk subsystem. The communication
between the Master and the Subordinates within a single source storage disk subsystem
is transparent and performed internally.
2. Once the communication paths are defined, start the Global Copy pairs that are to
become part of a Global Mirror session. Global Copy has to be created with the mkpprc
-type gcp command. We recommend that you wait until the first initial copy is complete
before you continue to the next step. This avoids unnecessary FlashCopy background
I/Os in the following step.
3. The next step is to create FlashCopy relationships between the B and C volumes. You can
use the mkremoteflash (abbreviated mkflash) command with the following parameters:
-tgtinhibit, -record, and -nocp. The -persist parameter is automatically set when the
-record parameter is selected. For a discussion of the particular FlashCopy attributes that
are required for a Global Mirror FlashCopy, see 20.3.4, “Introduce FlashCopy” on
page 255.
266
IBM System Storage DS6000 Series: Copy Services in Open Environments
4. With external Subordinates, that is, with more than one involved disk subsystem at the
local site, you need paths between the Master LSS and any potential Subordinate storage
disk subsystem at the local site. If you did not define these paths in the first step, then this
is the time to create these paths before you continue with the next step.
5. Define a token that identifies the Global Mirror session. This is a session ID with a number
between 1 and 255. Define this session number to the Master storage disk subsystem and
also to all potentially involved source LSSs that are going to be part of this Global Mirror
session and contain Global Copy source volumes that belong to the Global Mirror session.
All source LSSs include all LSSs in potential Subordinate storage disk subsystems that
are going to be part of the Global Mirror session. For this step, you can use the mksession
command.
6. The next step populates the session with the Global Copy source volumes. You should put
these Global Copy source volumes into the session after they complete their first pass for
initial copy. For this step, you can use the chsession -action add -volume command.
7. Start the session. This command actually defines the Master LSS. All further session
commands have to go through this LSS. You can specify with the start command the
Global Mirror tuning parameters, such as maximum drain time, maximum coordination
time, and Consistency Group interval time. For this step, you can use the mkgmir
command
Use this recommended sequence independently of the interface you use.
21.3 Modify a Global Mirror session
Once a session is active and running, you may alter the Global Mirror environment to add or
remove volumes. You also can add storage disk subsystems to a Global Mirror session, or
you can change the interval between the formation of Consistency Groups.
21.3.1 Add or remove volumes to the Global Mirror session
Volumes can be added to the session at any time after the session number is defined to the
LSS where the volumes reside. Once the session is started, volumes also can be added to
the session or removed from the session at any time.
Note: You only add Global Copy source volumes to a Global Mirror session.
Volumes can be added to a session in any state, for example, simplex or pending. Volumes
that have not completed their initial copy phase stay in a join pending state until the first initial
copy is complete. If a volume in a session is suspended, it causes Consistency Group
formation to fail.
We recommend that you add only Global Copy source volumes that have completed their
initial copy or first pass, although the microcode itself stops volumes from joining the Global
Mirror session until the first pass is complete. Also, we recommend that you wait until the
initial copy is complete before you create the FlashCopy relationship between the B and the C
volumes.
Note: You cannot add a Metro Mirror source volume to a Global Mirror session. Global
Mirror supports only Global Copy pairs. When Global Mirror detects a volume, which, for
example, is converted from Global Copy to Metro Mirror, the following formation of a
Consistency Group fails.
Chapter 21. Global Mirror options and configuration
267
When you add a rather large number of volumes at one time to an existing Global Mirror
session, then the available resources for Global Copy within the affected ranks might be
utilized by the initial copy pass. To minimize the impact to the production servers when
adding many new volumes, you might consider adding the new volumes to an existing Global
Mirror session in stages.
Suspending a Global Copy pair that belongs to an active Global Mirror session impacts the
formation of Consistency Groups. When you intend to remove Global Copy volumes from an
active Global Mirror session, follow these steps:
򐂰 First, remove the desired volumes from the Global Mirror session.
򐂰 Then, withdraw the FlashCopy relationship between the B and C volumes,
򐂰 Finally, terminate the Global Copy pair to bring volume A and volume B into simplex mode.
Note: When you remove A volumes without pausing Global Mirror, you might see this
reflected as an error condition with the showmigr -metrics command, indicating that the
Consistency Group formation failed. However, this does not mean that you have lost a
consistent copy at the remote site, because Global Mirror does not take the FlashCopy (B
to C) for the failed Consistency Group data. This message indicates that just one
Consistency Group formation has failed, and Global Mirror retries the sequence.
21.3.2 Add or remove storage disk subsystems or LSSs
When you plan to add a new Subordinate storage disk subsystem to an active session, you
have to stop the session first. Then, add the new Subordinate storage disk subsystem and
start the session again. The session start command then contains the new Subordinate
storage disk subsystem. The same procedure applies when you remove a storage disk
subsystem from a Global Mirror session, which can only be a Subordinate. In other words,
you cannot remove the Master storage disk subsystem.
When you add a new LSS to an active session and this LSS belongs to a storage disk
subsystem that already has another LSS which belongs to this Global Mirror session, you can
add the new LSS to the session without stopping and starting the session again. This is true
for either the Master or for a Subordinate storage disk subsystem.
21.3.3 Modify the Global Mirror session parameters
The parameters that are used for tuning a Global Mirror session can be modified by pausing
the session and then resuming the session with the new values. The parameters that you can
modify are:
򐂰 Consistency Group interval time
򐂰 Maximum coordination interval time
򐂰 Maximum Consistency drain time
When you give a start command after a pause command, as opposed to a resume, any value
for the Consistency Group interval time, maximum coordination interval time, or maximum
Consistency Group drain time in the start command is ignored. If these parameters have to
be altered, you have to give a resume command to the paused session with the parameters
specified. If you resume a paused session without specifying these parameters, they are set
to their default values; see 20.4.2, “Consistency Group parameters” on page 260.
268
IBM System Storage DS6000 Series: Copy Services in Open Environments
Important: When setting new values for the tuning parameters, be sure to check for errors
in Consistency Group formation and in draining the out-of-sync bitmaps. A few errors are
insignificant and do not jeopardize the consistency of your Global Mirror. However, if
failures repeatedly occur, such as no more Consistency Groups are formed, the
percentage of successful Consistency Groups is unacceptable, or the frequency of
Consistency Groups is not meeting your requirements or Recovery Point Objective (RPO),
then you need to revise and change the new values that were set for the tuning
parameters.
21.3.4 Global Mirror environment topology changes
The topology of a Global Mirror session, for example, the Master SSID, Subordinate SSIDs,
and Subordinate serial numbers, cannot be altered through a pause and resume command
sequence. When you resume the session after a pause and the Global Mirror topology is not
the same as it was at the pause, the start of the session fails.
If you need to change the topology of your Global Mirror session, you have to stop the session
and start the session again with the new topology structure information in the mkgmir
command.
Topology also refers to the list of storage disk subsystems which are subordinates. You define
control paths between the Master and Subordinate LSSs; just one LSS per subordinate disk
subsystem is sufficient. When you define the control path, you identify the source LSS on the
Master disk subsystem. The target LSS in the path definition command points to a
corresponding subordinate disk subsystem. These LSSs go into the topology specification,
which defines the communication paths between Master and subordinate storage disk
subsystems. To change these values, you must stop the Global Mirror process (rmgmir
command) and start it again with the new topology specifications (mkgmir command).
21.3.5 Remove FlashCopy relationships
When you withdraw a FlashCopy relationship within a Global Mirror session, the Consistency
Group process is affected. If a withdrawal is required, then first remove the Global Copy
source volume from the session.
Once the volumes are removed from the session, you can explicitly terminate the
corresponding FlashCopy relationship that was tied to this source volume.
You might need this ability to terminate FlashCopy relationships when you want to change
the FlashCopy targets within a Global Mirror configuration and choose, for example, another
LSS for the FlashCopy targets. This capability might be needed because you want to replace
the FlashCopy targets due to a skew in the load pattern in the remote storage disk
subsystem. In this situation, you can pause the session before this activity, and then resume
the session again once you have completely removed the FlashCopy relationships.
Note: A pause command (pausemigr) completes a Consistency Group formation in
progress. A stop command (rmgmir) immediately ends the formation of Consistency
Groups.
21.3.6 Remove the Global Mirror environment
To remove a Global Mirror environment and remove all traces of Global Mirror, we
recommend the following sequence of steps:
Chapter 21. Global Mirror options and configuration
269
1.
2.
3.
4.
5.
6.
7.
Terminate the Global Mirror session.
Remove all Global Copy source volumes from the Global Mirror session.
Close the Global Mirror session.
Withdraw all FlashCopy relationships between the B and C volumes.
Terminate all Global Copy pairs.
Remove the paths between the local and remote sites.
Remove the paths between the Master LSS and all related Subordinate LSSs.
21.4 Global Mirror with multiple storage disk subsystems
When you create a Global Mirror environment that spans multiple storage disk subsystems at
the local site, and probably also at the remote site, then you need to define communication
paths between the involved local storage disk subsystems. See Figure 21-2.
Define Global Mirror session
01
Network
A
B
Primary
Primary
Primary
Primary
C
Remote site
Define Global Mirror session
01
A
B
Primary
Primary
Network
Primary
Primary
C
Figure 21-2 Define Global Mirror session to all potentially involved storage disk subsystems
Figure 21-2 shows a symmetrical configuration with a one-to-one mapping. You have to
define the corresponding session, with its number, to all potentially involved LSSs at the local
site. There is still no connection between both local storage disk subsystems. Therefore, we
have to define the corresponding Global Mirror paths; see Figure 21-3 on page 271.
270
IBM System Storage DS6000 Series: Copy Services in Open Environments
Subordinate
01
A
Primary
Primary
A
Global Copy
Network
target
source
copy pending
copy pending
Local site
A
B
Primary
Primary
Primary
Primary
C
tertiary
paths
Remote site
Start Global Mirror session
01
A
Primary
Master Primary
A
source
A
B
Primary
Primary
Network
copy pending
target
copy pending
Primary
Primary
C
tertiary
Global Copy
Figure 21-3 Decide for Master disk subsystem and start Global Mirror session
Through the start command mkgmir, you decide which LSS becomes the Master LSS and
consequently which local storage disk subsystem becomes the Master storage disk
subsystem. This Master acts indeed like a server in a client/server environment.
The required communication between the Master storage disk subsystem and the
subordinate storage disk subsystems is inband over the defined Global Mirror paths. This
communication is highly optimized and minimizes any potential application write I/O impact
during the coordination phase to a few milliseconds. For details, see 20.4, “Consistency
Groups” on page 258.
Note that this communication is performed over Fibre Channel Protocol (FCP) links. At least
one FCP link is required between the Master storage disk subsystem and the Subordinate
storage disk subsystem. Figure 21-4 on page 272 uses dashed lines to show the Global
Mirror paths that are defined over FCP links between the Master storage disk subsystem and
its associated Subordinate storage disk subsystems. These FCP ports are dedicated for
Global Mirror communication between Master and Subordinates.
Chapter 21. Global Mirror options and configuration
271
Subordinate
01
A
Primary
Primary
A
A
B
Primary
Primary
target
source
copy pending
Global Copy links
copy pending
Subordinate
01
A
Primary
Primary
A
A
B
Primary
Primary
C
tertiary
Primary
Primary
target
source
copy pending
Global Copy links
copy pending
Primary
Primary
C
tertiary
GM links
01
A
Master Primary
Primary
A
source
copy pending
Network
A
B
Primary
Primary
target
copy pending
Global Copy links
Primary
Primary
C
tertiary
Figure 21-4 Global Mirror paths over FCP links between source storage disk subsystems
Also, a shared port on the Master storage disk subsystem and dedicated ports at the
Subordinates are shown in Figure 21-4. Not considering availability, from a communications
and traffic viewpoint only, a link is sufficient for the traffic between the Master and its
Subordinates. For redundancy, we suggest configuring two links.
Note that when you configure links over a SAN network, the same FCP ports of the storage
disk subsystem can be used for the Global Mirror session communication, as well as for the
Global Copy communication, and for host connectivity. However, for performance reasons
and to prevent host errors from disrupting your Global Mirror environment, it is often a good
idea to use separate FCP ports.
The sample configuration shown in Figure 21-5 on page 273 shows a mix of dedicated and
shared FCP ports. In this example, an FCP port in the Master storage disk subsystem is used
as the Global Mirror link to the other two Subordinate storage disk subsystems, and is also
used as the Global Copy link to the target disk subsystem. Also, there are ports at the
Subordinate disk subsystem that are used as Global Mirror session links as well as Global
Copy links.
272
IBM System Storage DS6000 Series: Copy Services in Open Environments
Subordinate
01
A
Primary
Primary
A
source
copy pending
Subordinate
A
B
Primary
Primary
target
copy pending
Global Copy links
01
A
Primary
Primary
A
source
copy pending
A
B
Primary
Primary
C
tertiary
Primary
Primary
target
copy pending
Global Copy links
Primary
Primary
C
tertiary
GM links
01
A
Master Primary
Primary
A
source
copy pending
Network
A
B
Primary
Primary
target
copy pending
Global Copy links
Primary
Primary
C
tertiary
Figure 21-5 Dedicated and shared links
If possible, Figure 21-6 shows a better configuration. Again, from a performance and
throughput viewpoint, you do not need two Global Mirror links between the Master and its
Subordinate storage disk subsystems. Still, dedicated ports for Global Mirror communication
control between Master and Subordinate provide the maximum responsiveness.
Subordinate
01
A
Primary
Primary
A
source
copy pending
Subordinate
A
B
Primary
Primary
target
copy pending
Global Copy links
01
A
Primary
Primary
A
source
copy pending
01
A
Master Primary
Primary
A
source
copy pending
A
B
Primary
Primary
C
tertiary
Primary
Primary
target
copy pending
Global Copy links
Primary
Primary
C
tertiary
GM links
Network
A
B
Primary
Primary
target
copy pending
Global Copy links
Primary
Primary
C
tertiary
Figure 21-6 Dedicated Global Mirror links and dedicated Global Copy links
Chapter 21. Global Mirror options and configuration
273
21.5 Recovery scenario after production site failure
This section covers the steps that you need to follow in a Global Mirror environment, when a
production site failure requires you to recover at the remote site.
21.5.1 Normal Global Mirror operation
Figure 21-7 shows a simple configuration with Global Mirror active and running.
Server
FlashCopy
A
Primary
A
Primary
Primary
Primary
A
Global Copy
source
copy pending
A
A
B
Primary
Primary
Primary
Primary
target
copy pending
Local site
A
A
C
Primary
Primary
Primary
Primary
tertiary
Remote site
Figure 21-7 Normal Global Mirror operation
Writes from the server are replicated through Global Copy, and Consistency Groups are
created as tertiary copies. The B volumes are Global Copy target volumes, and they are also
FlashCopy source volumes. The C volumes are the FlashCopy target volumes. The
FlashCopy relationship is a particular relationship; see 20.3.4, “Introduce FlashCopy” on
page 255.
21.5.2 Production site failure
A failure at the local site prevents all I/O to and from the local storage disk subsystems; see
Figure 21-8 on page 275. This might impact the formation of Consistency Groups, because
the entire process is managed and controlled by the Master storage disk subsystem that is
also the source disk subsystem, and it just failed and cannot communicate any longer with its
partners at the remote site.
The goal is to swap to the remote site and restart the applications. First, this requires you to
make the set of consistent volumes at the remote site available for the application before the
application can be restarted at the remote site.
Then, once the local site is back and operational again, we have to return to the local site.
Before returning to the local site, we have to apply the changes that the application made to
the target data while it was running at the remote site to the source volumes. After this, we
quickly swap back to the local site and restart the application.
274
IBM System Storage DS6000 Series: Copy Services in Open Environments
Server
FlashCopy
A
Primary
A
Primary
Primary
Primary
A
A
A
B
Global Copy
Primary
Primary
Primary
Primary
target
source
copy pending
copy pending
A
A
C
Primary
Primary
Primary
Primary
tertiary
Local site
Remote site
Figure 21-8 Production site fails
When the local storage disk subsystem fails, Global Mirror can no longer form Consistency
Groups. Depending on the state of the local storage disk subsystem, you might be able to
terminate the Global Mirror session. Usually this is impossible, because the storage disk
subsystem might not respond any more. Host application I/O might have failed and the
application ended. You typically get messages or SNMP alerts that indicate the problem. With
automation in place, such as eRCMF for example, these alert messages trigger the initial
recovery actions.
If the formation of a Consistency Group was in progress, then, most probably, not all
FlashCopy relationships between the B and C volumes at the remote site have reached the
corresponding point-in-time. Some FlashCopy pairs might have completed the FlashCopy
phase to form a new Consistency Group and committed the changes already. Others might
not have completed yet, and are in the middle of forming their consistent copy, and remain in
a revertible state. And, there is no Master any longer to control and coordinate what might still
be going on. All of this requires a closer look at the volumes at the remote site before we can
continue to work with them. We discuss this more in the following sections. First, however, we
just bring the B volumes into a usable state using the Failover command. See Figure 21-9.
21.5.3 Global Copy Failover from B to A
Server for
GM recovery
Failover B to A
A
Primary
A
Primary
Primary
Primary
A
Global Copy
FlashCopy
A
A
B
Primary
Primary
Primary
Primary
source
source
suspended
copy pending
A
A
C
Primary
Primary
Primary
Primary
tertiary
Local site
Remote site
Figure 21-9 Perform Global Copy Failover from B to A
Chapter 21. Global Mirror options and configuration
275
Because the source storage disk subsystem might no longer be usable, now the recovery
actions and processing occur at the remote site, using a server connected to the remote
storage disk subsystems for storage management console functions. See Figure 21-9 on
page 275.
You can use DS Storage Manager (DS SM), DS Command-Line Interface (DS CLI), or
eRCFM to execute the needed commands to the remote storage disk subsystems.
Doing a Failover (Copy Services Failover function) on the Global Copy target volumes turns
them into source volumes and suspends them immediately. This sets the stage for change
recording when application updates start changing the B volumes. Change recording in turn
allows you to just re-synchronize the changes later from the B to the A volumes, before
returning to the local site to resume the application again at the local site. But at this stage,
the B volumes do not contain consistent data; they are still useless. We just changed their
state from target copy pending to suspended. The state of the A volumes remains unchanged.
The key element when you run a Copy Services Failover is that the B volumes become the
new source volumes. This action just changes the state of the B volumes from target copy
pending to suspended. This action does not require communication with the other storage disk
subsystem at all, even though it is specified in the failoverpprc command. Once all the
Failover commands are successfully executed, we can move on to the next step.
21.5.4 Verify for valid Consistency Group state
Now, you have to investigate whether all FlashCopy relationships are in a consistent state;
see Figure 21-10. This means you have to query all FlashCopy relationships between B and
C, which are part of the Consistency Group, to determine the state of the FlashCopy
relationships. Global Mirror might have been in the middle of forming a Consistency Group
and FlashCopy might not have completed the creation of a complete set of consistent C
volumes.
Server for
GM recovery
Check Consistency Group
FlashCopy
A
Primary
A
Primary
Primary
Primary
A
Global Copy
source
copy pending
A
A
B
Primary
Primary
Primary
Primary
source
suspended
Local site
A
A
C
Primary
Primary
Primary
Primary
tertiary
Remote site
Figure 21-10 Check Consistency Group state
Each FlashCopy pair needs a FlashCopy query to identify its state. If the local site is still
reachable and the source storage disk subsystem is also reachable, you might consider
performing these activities at the production site using remote FlashCopy commands. Most
likely, the source storage disk subsystem does not respond any longer. In this case, you have
to target the query directly to the recovery site storage disk subsystem.
276
IBM System Storage DS6000 Series: Copy Services in Open Environments
When you query a FlashCopy pair, there are two pieces of information that are key to
determining whether the C volume set is consistent or needs intervention: the revertible state
and the sequence number.
The lsflash command reports the Revertible state as either Enable or Disable, which
indicates whether the state of the FlashCopy is revertible or non-revertible. A non-revertible
state means that a FlashCopy process has completed successfully and all changes are
committed. Global Mirror uses the two-phase FlashCopy establishment operation. This
operation allows the storage disk subsystem to prepare for a new FlashCopy relationship
without altering the existing FlashCopy relationship. You can either commit or revert this new
FlashCopy relationship with revertible state by using the revertflash and commitflash
command. During the Consistency Group formation process, Global Mirror puts all
FlashCopy relationships in the revertible state and after they are in the revertible state,
commits all FlashCopy relationships. With this operation, the situation does not occur where
some FlashCopy operations have not started while others have completed.
The Sequence number is an identifier that can be set at FlashCopy establish operations and it
is then associated with the FlashCopy relationship. Subsequent FlashCopy withdrawal
operations can be directed to FlashCopy relationships with specific sequence numbers.
Global Mirror uses the sequence number to identify a particular Consistency Group. The
actual sequence number used by Global Mirror is the platform timer from the Global Mirror
Master storage disk subsystem (in seconds resolution, which is within a maximum of one
second) at the point when the Global Mirror source components have to be coordinated to
form a Consistency Group. This is at a point before the Consistency Group is transferred to
the remote site. If your Master storage disk subsystem platform timer is set to the time of day,
then the FlashCopy sequence for Global Mirror approximates a timestamp for the
Consistency Group.
The best situation is when all the FlashCopy pairs of a Global Mirror session are in the
non-revertible state and all their sequence numbers are equal. No further action is necessary
and the set of C volumes is consistent and the copy is good.
Figure 21-11 shows the Consistency Group creation process. The action required depends
on the state of the Consistency Group creation process when the failure occurred.
Create consistency group by holding
application writes while creating
Transmit updates in Global Copy mode
FlashCopy issued
bitmap containing updates for this
while between consistency groups
with revertible option
consistency group on all volumes Consistency group interval – 0s to 18hrs
design point is 2-3ms
All FlashCopies
Maximum coordination time (eg. 10ms)
revertible
Drain consistency group and send
to remote DS using Global Copy.
Application writes for next
consistency group are recorded in
change recording bitmap
Maximum drain time – eg.1 min
FlashCopy committed
once all revertible
Flashcopies have
successfully completed
Start next consistency group
Action required
Figure 21-11 FlashCopy Consistency Group creation process
Chapter 21. Global Mirror options and configuration
277
Depending on when the failure occurs, there are some combinations of revertible states and
FlashCopy sequence numbers that need different corrective actions. Use Table 21-1 as a
guide. This is a decision table and reads in the following way: When column 2 AND column 3
are true, then take the action in column 4. Column 5 contains additional comments. Do this for
each of the four cases. The cases are described in chronological order, starting from the left.
Table 21-1 Consistency Group and FlashCopy validation decision table
Are all FlashCopy
relationships
revertible?
Are all FlashCopy
sequence numbers
equal?
Action to take
Comments
Case 1
NO.
YES.
No action needed. All
C volumes are
consistent.
Consistency Group
formation ended.
Case 2
SOME. Some
FlashCopy pairs are
revertible and others
are non- revertible.
Revertible FlashCopy
pairs’ sequence
numbers are equal.
And non-revertible
FlashCopy pairs’
sequence numbers
are equal, but do not
match the revertible
FlashCopies’
sequence number.
revert FlashCopy
Some FlashCopy
pairs are running in a
Consistency Group
process and some
have not yet started
their incremental
process.
YES.
YES.
revert to all
Case 3
relations.
FlashCopy relations.
Case 4
SOME. Some
FlashCopy pairs are
revertible and others
are non-revertible.
YES.
commit FlashCopy
relations.
All FlashCopy pairs
are in a new
Consistency Group
process and none
have finished their
incremental process.
Some FlashCopy
pairs are running in a
Consistency Group
process and some
have already finished
their incremental
process.
If you see a situation other than the above four situations, then the Global Mirror mechanism
has been corrupted.
Case 1: FlashCopies still committed
This indicates the situation where all FlashCopy operations have completed their tasks (and
the next FlashCopy operations have not been started). In this situation, we do not need any
action to correct the FlashCopy status, because we have consistent data in all C volumes.
This state is also reached after the FlashCopies are complete, and Global Mirror is waiting to
create the next Consistency Group.
Case 2: FlashCopies partially issued
In this case, there is a group of FlashCopy pairs that are all revertible and another group of
FlashCopy pairs that are all non-revertible. Consistency can be restored if these criteria are
true:
򐂰 The FlashCopy sequence number for all revertible pairs is equal.
򐂰 The FlashCopy sequence number for all non-revertible pairs is equal, too.
278
IBM System Storage DS6000 Series: Copy Services in Open Environments
This indicates that the FlashCopy operations were interrupted. Some FlashCopy operations
for the new Consistency Group were started, but not all of them.
The FlashCopy relationships, which have not started, are in a non-revertible state and all of
them have the same FlashCopy sequence number. Other FlashCopy relationships, which
had already started, are in a revertible state and all of them have the same FlashCopy
sequence number, but the number is different from the sequence number of the
non-revertible FlashCopy relationships.
All this indications suggest that you have to return the revertible FlashCopy relationships to
the previous Consistency Group using the revertflash command and terminate the
FlashCopy relationships.
Case 3: FlashCopies all revertible
In the case where all of the pairs are revertible and all FlashCopy sequence numbers are
equal, this indicates that all FlashCopy operations were running and none completed its task
for the Consistency Group formation. The fact that all relationships are still in a revertible
state denotes that nothing was finished and committed. Also, the identical FlashCopy
sequence numbers denote that all the FlashCopy operations were involved in the very same
Consistency Group set. All of these indications suggest that you have to use the revertflash
command to return to the previous Consistency Group.
The revert action, which is invoked by the revertflash command, restores the Consistency
Group level between the source and target volumes to the prior state before the current
FlashCopy ran, and it resets the revertible state to Disable. The FlashCopy relationship is
preserved.
When the FlashCopy relationship is in a non-revertible state, the revert operation is not
possible. When you issue this command to FlashCopy pairs that are non-revertible, you are
going to see only an error message, and no action is performed.
Case 4: FlashCopies partially committed
If at the failure point, some of the FlashCopy operations had completed their tasks to create a
consistent copy and committed this process, these operations are non-revertible. Other
FlashCopy relationships might not have completed their corresponding part of the new
Consistency Group, so these are still in a revertible state. If all of them, revertible and
non-revertible, have the same FlashCopy sequence number, this means that they all were
involved in the very same Consistency Group. This allows you to commit the revertible
FlashCopy relationships using the commitflash command.
To make the task easier, you can run the commitflash to all FlashCopy pairs. Non-revertible
FlashCopy relationships just return an error message.
Usually, all FlashCopy pairs are non-revertible and all sequence numbers are equal. In this
case, you do not need to take any action. Nevertheless, and depending on the failure
point-in-time, you might have to perform one of the above recovery actions. After this action,
all FlashCopy pairs are non-revertible and all sequence numbers are equal. Now you can
proceed to the next step.
21.5.5 Set consistent data on B volumes
At this point, only the C volumes comprise a set of consistent data volumes. The B volumes by
definition do not provide consistent data, because Global Copy does not provide data
consistency. We want two good copies of the data at the recovery site. The aim is to have a
consistent set of volumes to work with, still keep a good copy to which we can resort if
needed.
Chapter 21. Global Mirror options and configuration
279
The next step then is to create the same consistency on the B volumes that we have on the C
volumes; see Figure 21-12. This can be achieved with the reverseflash command with the
parameter -fast. This operation is called Fast Reverse Restore (FRR). You must add the
-tgtpprc parameter with the reverseflash -fast command, because the B volume is also
the Global Copy source at this step.
Server for
GM recovery
FRR FlashCopy
A
Primary
A
Primary
Primary
Primary
A
source
pending
A
A
B
Primary
Primary
Primary
Primary
GC source
FC source
suspended
Local site
A
A
C
Primary
Primary
Primary
Primary
tertiary
FC target
Remote site
Figure 21-12 Set a consistent set of B volumes using the C volumes as source
Though the Fast Reverse Restore operation starts the background copy from C to B volumes,
in the reverseflash command, you have to specify the B volumes as the FlashCopy sources
and the C volumes as the FlashCopy targets. With the reverseflash command, you must use
the following parameters:
򐂰 -fast
With this parameter, the reverseflash command can be issued before the background
copy completes. This option is intended for use as part of Global Mirror.
򐂰 -tgtpprc
After the Failover B to A operation, which is described in 21.5.3, “Global Copy Failover from
B to A” on page 275, the B volume became a Global Copy source volume in a suspended
state. The -tgtpprc parameter allows the FlashCopy target volume to be a Global Copy
source volume. You have to specify this parameter here, because the B volume becomes a
FlashCopy target in the reverseflash process.
Because you do not specify the -persist parameter, the FlashCopy relationship ends after
the background copy from C to B completes.
The above Fast Reverse Restore (FRR) operation does a background copy of all tracks that
changed on B since the last Consistency Group (CG) formation. This results in the B volume
becoming equal to the image that was present on the C volume. This is the logical view. From
the physical data placement point of view, the C volume does not have meaningful data after
the FlashCopy relationship ends.
You have to wait until all Fast Reverse Restore operations complete successfully and the
FRR background copy completes before you proceed with the next step. Again, when the
background copy completes, the FlashCopy relationship ends. Therefore, you should check if
the FlashCopy relationships remain to determine when all Fast Reverse Restore operations
are completed.
280
IBM System Storage DS6000 Series: Copy Services in Open Environments
21.5.6 Reestablish FlashCopy relationships between B and C
In this step, you establish the former FlashCopy relationship between the B and C volumes, as
it was at the beginning when you set up the Global Mirror environment. This step is in
preparation for returning later to production at the local site. The command at this step is
exactly the same FlashCopy command that you might have used when you initially created
the Global Mirror environment. See 20.3.4, “Introduce FlashCopy” on page 255.
In a disaster situation, you might not want to use the -nocp option for the FlashCopy from B to
C. Using the -nocp option removes the FlashCopy I/O overhead when the application starts.
Now you can restart the applications at the remote site using the B volumes. Note the B
volumes are Global Copy source volumes in the suspended state, which implies that change
recording takes place. Later, this allows you to re-synchronize from B to A before returning to
the local site.
Eventually, here you might also decide to create another copy of this consistent set of
volumes, and create D volumes, or preserve this set of consistent volumes on tape.
When you create the D volumes, this is just a plain FlashCopy command. Note that this is a
full volume FlashCopy, because at a previous step, we reestablished the FlashCopy
relationship between B and C, indicating that it was part of a Global Mirror environment. This
indication carries the start change recording attribute, which implies that you can only use the
same source volume for full volume FlashCopy; only a single incremental relationship is
allowed per source volume.
21.5.7 Restart the application at the remote site
At this stage, you can restart the application at the remote site and work with the consistent
set of B volumes; see Figure 21-13.
Application
Server
a
Restart applications
Global Copy
A
A
A
Primary
Primary
Primary
Primary
A
A
B
Primary
Primary
Primary
Primary
source
suspended
source
A
A
C
Primary
Primary
Primary
Primary
tertiary
pending
Local site
Remote site
Figure 21-13 Restart applications at the remote site
Note the suspended state of the B volumes, which implies change recording and indicates the
changed tracks on the B volumes. When the local site is about to become ready to restart the
applications again and resume the operations, you prepare the remote site for the next step.
Chapter 21. Global Mirror options and configuration
281
21.5.8 Prepare to switch back to the local site
The local site is back and operative again. We assume that the local site did not lose the data
at the time that the swap to the remote site was done. It is possible then to re-synchronize the
changed data from B to A before restarting the application at the local site. This is
accomplished by performing a Failback operation (Copy Services Failback function) from B to
A; see Figure 21-14.
Application
Server
Failback B to A
A
Primary
A
Primary
Primary
Primary
A
target
copy pending
A
A
B
re-synchronize from B to A
Primary
Primary
Primary
Primary
Global Copy
source
copy pending
Local site
A
A
C
Primary
Primary
Primary
Primary
a
tertiary
Remote site
Figure 21-14 Failback operation from B to A in preparation for returning to local site
Note that the Failback operation is issued to the B volumes as the source and the A volumes
as the target. This command changes the A volume from its previous source copy pending
state to target copy pending and starts the re-synchronization of the changes from B to A.
Note: Before doing the Failback operation, ensure that paths are defined from the remote
site LSS to its corresponding LSS at the local site.
Note that with Fibre Channel links, you can define paths in either direction on the very same
FCP link. During the Failback operation, the application continues to run at the remote site to
minimize the application outage.
If the A volume is still online to the server at the local site or it was online when a crash
happened, so that the SCSI persistent reserve is still set on the previous source disk (A
volume), the Global Copy Failback process with the failbackpprc command fails. In this
case, the server at the production site locks the target with a SCSI persistent reserve. After
this SCSI persistent reserve is reset with the varyoffvg command (in this case, on AIX), the
failbackpprc command completes successfully.
There is a -resetreserve parameter for the failbackpprc command. This option resets the
reserved state so that the failback operation can complete. In the Failback operation after a
real disaster, you can use this parameter, because the server might go down while the SCSI
persistent reserve was set on the A volume. In the planned failback operation, you must not
use this parameter, because the server at the local site still owns the A volume and might be
using it and the Failback operation suddenly changes the contents of the volume. This can
corrupt the server file system.
282
IBM System Storage DS6000 Series: Copy Services in Open Environments
21.5.9 Return to the local site
Once the local site is ready, quiesce the application at the remote site. Then, a sequence of
Global Copy Failover - Failback operations from A to B reestablish Global Copy back as it was
originally before the local site outage.
Server for
GM recovery
Application
Server
Failover A to B
A
A
B
A
Primary
A
Primary
Primary
Primary
A
Primary
Primary
Primary
Primary
source
Global Copy
source
copy pending
suspended
A
A
C
Primary
Primary
Primary
Primary
tertiary
Local site
Remote site
Figure 21-15 Global Copy Failover from A to B
Figure 21-15 shows the action at the local site, the Global Copy Failover operation from A to B.
This failoverpprc command changes the state of the A volumes from target copy pending to
source suspended and starts to keep a bitmap record of the changes to the A volumes. You
issue this command to the storage disk subsystem at the local site. The state of the B
volumes does not change.
Once the Failover is completed, a Failback operation from A to B is run; see Figure 21-16.
Server for
Server
GM recovery
Failback A to B
Re-synchronize from A to B
A
A
B
A
Primary
A
Primary
Primary
Primary
A
source
copy pending
Local site
Primary
Primary
Primary
Primary
Global Copy
target
copy pending
A
A
C
Primary
Primary
Primary
Primary
tertiary
Remote site
Figure 21-16 Global Copy Failback from A to B and resync Global Copy volumes
Chapter 21. Global Mirror options and configuration
283
Note: Before doing the Failback operation, ensure that paths are defined from the local site
LSS to its corresponding LSS at the remote site.
Figure 21-16 on page 283 shows the Failback operation at the local site. The failbackpprc
command changes the state of the A volumes from source suspended to source copy pending.
And the state of the B volumes changes from source copy pending to target copy pending.
Also, the replication of updates from A to B begins. This replication ends quickly, because the
application has not started yet at the local site.
Last but not least, if you did not already establish the FlashCopy relationships from B to C
during the Failover - Failback sequence at the remote site, then you have to do it now. This
can be an inband FlashCopy as shown in Figure 21-17.
Server for
GM recovery
Remote FlashCopy B to C
FlashCopy
A
A
B
A
Primary
A
Primary
Primary
Primary
A
Primary
Primary
Primary
Primary
Global Copy
source
copy pending
A
A
C
target
Primary
Primary
Primary
Primary
copy pending
tertiary
FlashCopy target
Local site
Remote site
Figure 21-17 Establish Global Mirror FlashCopy relationship between B and C
The last step is to start the Global Mirror session again, as shown in Figure 21-18. Then, the
application can resume at the local site.
Application
server
I/O
Server for
GM recovery
Start GM session
FlashCopy
A
A
B
A
Primary
A
Primary
Primary
Primary
A
source
copy pending
Primary
Primary
Primary
Primary
Global Copy
A
A
C
target
Primary
Primary
Primary
Primary
copy pending
Tertiary
FlashCopy target
Local site
Remote site
Figure 21-18 Start Global Mirror session and resume application I/O at local site
284
IBM System Storage DS6000 Series: Copy Services in Open Environments
21.5.10 Conclusions of Failover/Failback example
This concludes the sequence of steps you have to go through to swap to the remote site and
then return to the local site after service has been restored at the local site.
In particular, the check on a valid Consistency Group after a production site failure is a
challenge when you consider a large configuration with many volume pairs. Each command
usually addresses a single pair of volumes. When you have a large quantity of volume pairs,
this requires automation, for example, to check all the FlashCopy relationships after a
production site failure. Note that for a planned site swap, you might have not to check for a
valid Consistency Group, because, through using a proper command sequence and verifying
their successful completion, the possibility of an inconsistent group of C volumes is minimal if
at all.
Global Mirror is a 2-site solution, which can bridge any distance between both sites. There
are ready-to-use packages and services available to implement a disaster recovery solution
for 2-site remote copy configurations. eRCMF supports not only Global Copy-based and
Metro Mirror-based configurations, but also Global Mirror configurations.
Chapter 21. Global Mirror options and configuration
285
286
IBM System Storage DS6000 Series: Copy Services in Open Environments
22
Chapter 22.
Global Mirror interfaces
In this chapter, we provide an overview of the interfaces you can use to manage and control
Global Mirror environments. You can complement the information in this chapter with the
following other sources:
򐂰
򐂰
򐂰
򐂰
򐂰
Part 2, “Interfaces” on page 15
Chapter 8, “FlashCopy interfaces” on page 65
Chapter 17, “Global Copy interfaces” on page 203
The examples presented in Chapter 24, “Global Mirror examples” on page 303
The publication IBM System Storage DS6000: Command-Line Interface User´s Guide,
GC26-7922
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
287
22.1 Global Mirror management interfaces overview
Global Mirror combines Global Copy and FlashCopy, which work together in an autonomic
fashion under the DS6000 microcode control. There are commands intended for Global Copy
and FlashCopy, as well as commands that address Global Mirror sessions. The following
interfaces can manage all of them:
򐂰
򐂰
򐂰
򐂰
DS Storage Manager (DS SM) graphical user interface (GUI)
DS Command-Line Interface (DS CLI)
TotalStorage Productivity Center for Replication (TPC for Replication)
enterprise Remote Copy Facility (eRCMF)
This chapter gives an overview of the DS CLI and the DS SM for Global Mirror management.
We also cover DS CLI and DS SM in Part 2, “Interfaces” on page 15.
eRCMF is a scalable and flexible solution to protect business data and maintain consistent
data, providing an automation tool that simplifies the disaster recovery operations. We cover
eRCMF in 22.4, “eRCMF” on page 293, and in more detail in Chapter 29, “IBM TotalStorage
Rapid Data Recovery for UNIX and Windows” on page 447.
TotalStorage Productivity Center for Replication provides management of DS6000 series
business continuance solutions, including FlashCopy, Metro Mirror, and Global Mirror. We
cover TPC for Replication in Chapter 28, “IBM TotalStorage Productivity Center for
Replication” on page 415. TPC for Replication includes similar functions to Global Mirror
Utility (GMU). GMU users should consider migrating to TPC for Replication.
Table 22-1 lists DS CLI commands and equivalent DS SM panel options for Global Mirror
management.
Table 22-1 DS CLI commands and equivalent DS SM panel options
288
Task
DS CLI
DS SM
Start Global Mirror
session.
mkgmir
CS → GM → Create
Stop Global Mirror
session.
rmgmir
CS → GM → Delete
Pause Global Mirror
session.
pausegmir
CS → GM → Pause
Resume Global Mirror
session.
resumegmir
CS → GM → Resume
Show Global Mirror
status.
showgmir
CS → GM →
Properties
Create a Global Mirror
session on an LSS.
mksession
CS → GM → Create
Or
CS → GM → Modify
Remove a Global
Mirror session from an
LSS.
rmsession
CS → GM → Delete
Or
CS → GM → Modify
Change a Global
Mirror session on an
LSS.
chsession
CS → GM → Modify
IBM System Storage DS6000 Series: Copy Services in Open Environments
Task
DS CLI
DS SM
Display a Global Mirror
session on an LSS.
lssession
CS → GM → View
session volumes
Create a complete
Global Mirror
environment.
mkpprcpath
mkpprc
mkflash
mksession
mkgmir
CS → P → Create
CS → MM → Create
CS → FC → Create
CS → GM → Create
Fail over Global Mirror.
rmgmir
failoverpprc
lsflash
revertflash
or
commitflash
reverseflash
mkflash
CS → GM → Delete
CS → MM → Failover
CS → FC →
Properties
CS → FC → Reverse
relationship
CS → FC → Create
Fail back Global
Mirror.
mkpprcpath
failbackpprc
lspprc
mkpprcpath
failoverpprc
failbackpprc
resumegmir
CS → P → Create
CS → MM → Failback
CS → MM →
Properties
CS → MM → Failover
CS → MM → Failback
CS → GM → Resume
Practice Failover
Global Mirror.
pausegmir
showgmir
pausepprc
failoverpprc
lsflash
reverseflash
mkflash
CS → GM → Pause
CS → GM →
Properties
CS → MM → Failover
CS → FC →
Properties
CS → FC → Reverse
relationship
CS → FC → Create
Practice Failback
Global Mirror.
failbackpprc
resumegmir
CS → MM → Failback
CS → GM → Resume
Clean up a complete
Global Mirror
environment.
rmgmir
chsession
rmsession
rmflash
rmpprc
rmpprcpath
CS → GM → Delete
CS → FC → Delete
CS → MM → Delete
CS → P → Delete
CS: Stands for Copy Services menu
GM: Stands for Global Mirror panel
MM: Stands for Metro Mirror panel
P: Stands for Paths panel
FC: Stands for FlashCopy panel
22.2 DS Command-Line Interface
You can use the DS Command-Line Interface to set up and manage Global Mirror functions.
One big usability advantage of the DS CLI is that all platforms use a common syntax. This
can make it easier to learn and make it a common tool for skilled people as well as lesser
Chapter 22. Global Mirror interfaces
289
skilled people. Another big usability advantage is that you can make scripts with the
commands, which saves lots of time when you have to execute several predetermined tasks.
The DS CLI commands are issued through the Ethernet network to the DS Storage
Management Console (DS SMC). The DS CLI server resides on the DS SMC, which works
as a gateway between the clients and the storage disk subsystem. You can install a DS CLI
client to access the DS CLI server on various operating systems. See 14.2, “Copy Services
network components” on page 137 for a detailed description of the DS SMC network
environment.
The DS CLI commands that are used for managing a Global Mirror environment fall in the
following basic groups:
򐂰
򐂰
򐂰
򐂰
For paths definitions and removal
For Global Copy pairs creation, management, and removal
For FlashCopy pairs creation, management, and removal
For Global Mirror session definition, management, and removal
The first three groups in the previous list refer to the same commands you use to manage and
control Global Copy and FlashCopy environments. Still, when working in a Global Mirror
environment specific command parameters must be used for some of the tasks. We cover
how these commands are used and the options that you must select in the discussions and
examples we present in this part of the book.
We have the following DS CLI commands for the Global Mirror session, its definition,
management, modification, and removal:
򐂰 showgmir displays detailed properties and performance figures for Global Mirror.
򐂰 showgmiroos displays the number of out of synchronization (out-of-sync) tracks for the
session.
򐂰 mkgmir starts Global Mirror.
򐂰 rmgmir stops Global Mirror.
򐂰 pausegmir pauses Global Mirror.
򐂰 resumegmir resumes Global Mirror.
򐂰 mksession opens a Global Mirror session on a LSS.
򐂰 chsession allows you to modify a Global Mirror session on a LSS. You can add and
remove a volume for Global Mirror with this command.
򐂰 rmsession removes an existing Global Mirror session on a LSS.
򐂰 lssession displays a Global Mirror session on a LSS.
For most DS CLI commands, you need to know some (or all) of the following information:
򐂰
򐂰
򐂰
򐂰
The serial number and device type of the source and target storage disk subsystem.
The World Wide Node name (WWNN) of the remote storage disk subsystem.
The LSS numbers of the source and target volumes.
The port IDs for source and target. You can specify up to eight port pair IDs.
A full establishment of a Global Mirror environment using DS CLI commands might take a
long time, especially if you need to set up an environment involving many volumes on many
LSSs and in several storage disk subsystems. For the setup and management of large-scale
environments, eRCMF or TPC for Replication is more appropriate, especially if you want to
test disaster recovery scenarios or if you want to run a real disaster recovery plan.
You can see detailed examples of how to set up and manage a Global Mirror environment
using DS CLI commands in Chapter 24, “Global Mirror examples” on page 303.
The DS CLI commands are documented in the publication IBM System Storage DS6000:
Command-Line Interface User´s Guide, GC26-7922.
290
IBM System Storage DS6000 Series: Copy Services in Open Environments
22.3 DS Storage Manager GUI
You can use the DS Storage Manager (DS SM) graphical user interface (GUI) to set up and
manage some Global Mirror functions. You work with the DS SM GUI through a series of
predetermined panels and options you can select to accomplish the desired task. The DS SM
operates from a Web session that communicates to the DS Storage Management Console
(DS SMC). See 14.2, “Copy Services network components” on page 137 for a detailed
description of the DS SMC network environment.
The DS SM interface is user friendly; however, you cannot use it for automation activities, and
certain Copy Services functions are not supported from this interface.
People who have fewer skills or less confidence with the other interfaces can use the DS
Storage Manager interface. It is reasonably simple to use; however, it is usually slower than
the other interfaces, and you cannot save actions for later use.
For Global Mirror setup activities, you start from the following DS Storage Manager panels:
򐂰 For paths definitions, you start from the Paths panel under the Copy Services menu. You
can see this panel in Figure 24-15 on page 348.
򐂰 To create Global Copy pairs, you start from the Metro Mirror panel under the Copy
Services menu. You can see this panel in Figure 24-22 on page 351.
򐂰 To create FlashCopy pairs, you start from the FlashCopy panel under the Copy Services
menu. You can see this panel in Figure 24-29 on page 355.
For Global Mirror session setup and control, you start from the Global Mirror panel under the
Copy Services menu of the DS SM; see Table 22-1 on page 288.
Note: The panels shown in this section correspond to a DS8000 HMC. For the purpose of
the present discussion, they are equally illustrative as if they were from a DS6000 SMC.
The Storage Image field is not present in the DS SM panels of the DS6000 SMC.
Figure 22-1 Global Mirror main panel
If you do not select any check box for a Global Mirror session, the pull-down list only allows
session creation. The pull-down list looks like Figure 22-2 on page 292. If you are working
with a storage disk subsystem that has more than one storage image, then in this panel, you
Chapter 22. Global Mirror interfaces
291
can also use the Storage Complex, Storage Unit, and Storage Image pull-down lists to
access other Storage Images with which the Global Mirror session might be operating.
Figure 22-2 Global Mirror pull-down list - No session selected
If you select any check box for a Global Mirror session, the pull-down list allows session
creation and management. The pull-down list looks as shown in Figure 22-3.
Figure 22-3 Global Mirror pull-down list - Session selected
Consider that a full establishment of a Global Mirror environment requires using each Copy
Services panel in the DS SM, and if you need to set up an environment involving many
292
IBM System Storage DS6000 Series: Copy Services in Open Environments
volumes on many LSSs and in several storage disk subsystems, this can take a very long
time. Therefore, the DS SM is more convenient for specific smaller scale Copy Services
activities, and also it is a very didactic tool for users who are not skilled in other interfaces.
For the setup and management of large-scale environments, eRCMF or TPC for Replication
is more appropriate, especially if you want to test disaster recovery scenarios or if you want to
run a real disaster recovery plan.
You can see detailed examples of how to set up and manage a Global Mirror environment
using the DS Storage Manager GUI in Chapter 24, “Global Mirror examples” on page 303.
22.4 eRCMF
You can use the enterprise Remote Copy Management Facility (eRCMF) to manage Metro
Mirror, Global Copy, and Global Mirror functions, as well as their associated FlashCopy tasks.
You can use eRCMF to manage Copy Services functions in a mixed open systems and z/OS
environment. It is a powerful tool, which can consolidate the equivalent of several DS CLI
commands with just one command; nevertheless, it is not using DS CLI scripts.
eRCMF is convenient for the management of medium and large scale environments. eRCMF
is indispensable if you want to test disaster recovery scenarios or if you want to run a real
disaster recovery plan.
eRCMF allows the automation of the recovery activities. It not only traps SNMP messages,
but it also executes commands in order to shorten downtime.
For a detailed description of eRCMF from its installation through its configuration to its usage,
see Chapter 29, “IBM TotalStorage Rapid Data Recovery for UNIX and Windows” on
page 447.
Chapter 22. Global Mirror interfaces
293
294
IBM System Storage DS6000 Series: Copy Services in Open Environments
23
Chapter 23.
Global Mirror performance and
scalability
This chapter discusses performance considerations for planning and configuring Global
Mirror for DS6000. This chapter also describes the potential impact that the three phases of
Consistency Group formation might have on application write I/Os. Also, this chapter
discusses the distribution of the B volumes and the C volumes across different ranks and how
to provide extra care for very busy volumes.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
295
23.1 Performance aspects for Global Mirror
Global Mirror is basically comprised of Global Copy and FlashCopy and combines both
functions to create a solution that provides consistent data at a distant site. This means that
to have a satisfactory Recovery Point Objective (RPO) without impacting production too
much, you need to analyze performance at the production site, at the recovery site, and
between both sites.
򐂰 At the production site, even if production is a higher priority than replication, the storage
disk subsystem has to handle both the production load and the Global Copy replication
load. If the source storage disk subsystem is overloaded by production, this slows the
Consistency Group formation process, because it needs to wait for the drain of the
out-of-sync tracks.
򐂰 Between both sites, you need to size the bandwidth for production load peaks.
򐂰 At the recovery site, even if production is not running, it is hosting the target Global Copy
volumes and also handling the FlashCopy operations. Therefore, you also need to
analyze the performance of the remote storage disk subsystem to ensure the best
possible RPO.
This section discusses the impact of Global Copy and FlashCopy in the overall performance
of Global Mirror. Global Copy has, at most, only a minimal impact to the response time of an
application write I/O to a Global Copy source volume.
In the Global Mirror environment, FlashCopy is used with the nocopy attribute. This implies
that, for any write I/Os to the FlashCopy source volume, there are additional internally
triggered I/Os in the remote storage disk subsystem. These I/Os preserve the FlashCopy
source volume tracks by making a copy to the FlashCopy target volume before the source
tracks are updated. This happens every time in the interval in between two FlashCopy
establish operations.
Note that FlashCopy with the nocopy option does not start a background copy operation, but
it only maintains a set of FlashCopy bitmaps for the B and C volumes. These bitmaps are
established the first time that a FlashCopy relationship is created with the nocopy option.
Before a source track is modified between the creation of two Consistency Groups, the track
is copied to the target C volume to preserve the previous point-in-time copy. This includes
updates to the corresponding bitmaps to reflect the new location of the track, which belongs
to the point-in-time copy. Each Global Copy write to its target B volume within the window of
two adjacent Consistency Groups causes this type of FlashCopy I/O operation. See
Figure 23-1 on page 297.
An I/O in this context is also a Global Copy I/O when Global Copy replicates a track from a
Global Copy source volume to a Global Copy target volume. This implies an additional read
I/O in the source storage disk subsystem at the local site and a corresponding write I/O to the
target storage disk subsystem at the remote site. In a Global Mirror environment, the Global
Copy target volume is also the FlashCopy source volume. This might have some effect on a
very busy disk subsystem.
296
IBM System Storage DS6000 Series: Copy Services in Open Environments
1
Write
A
A1
Primary
Primary
Read
3
2
Write
Read
target
source
FCP links
copy pending
Host
A
B1
Primary
Primary
copy pending
Local site
4
Write
5
Primary
Primary
C1
tertiary
Remote site
Figure 23-1 Global Copy with write hit at the remote site
Figure 23-1 roughly summarizes what happens between two Consistency Group creation
points when an application writes come in. The application write I/O completes immediately at
the local site (1). Eventually, Global Copy replicates the application I/O and reads the data at
the local site (2). Before the track gets updated on the B volume, it is first preserved on the C
volume because of the FlashCopy nocopy attribute.
Note: What Figure 23-1 and Figure 23-2 show is not necessarily the exact sequence of
internal I/O events, but a logical view approximation. There are internal microcode
optimization and consolidation techniques that make the entire process more efficient.
What is shown in Figure 23-1 is the normal sequence of I/Os within a Global Mirror
configuration. The write (3) to the B1 volume is usually a write hit in the persistent memory
(NVS) of the target disk subsystem. So, this is an instant operation. Eventually at some later
moment, the B1 track is copied from B1 to C1, so that it can be preserved and the update in
NVS can be destaged to the B1 volume.
There is some potential impact on the Global Copy data replication operation depending on
whether persistent memory or non-volatile cache is overcommitted in the target storage disk
subsystem. This might cause the FlashCopy source tracks to first have to be preserved on
target volume C1, before the Global Copy write completes. See Figure 23-2.
1
Write
A
A1
Primary
Primary
Read
5
2
Host
A
B1
Read
Primary
Primary
target
source
copy pending
Write
FCP links
copy pending
Local site
3
Write
4
Primary
Primary
C1
tertiary
Remote site
Figure 23-2 Global Copy with overcommitted NVS at the remote site
Figure 23-2 roughly summarizes what happens when persistent memory or NVS in the
remote storage disk subsystem is overcommitted. A read (3) and a write (4) to preserve the
FlashCopy source track and write it to the C1 volume are required before the write (5) can
complete. Then, when the track is updated on the B1 volume, this completes the write (5)
operation. Nevertheless, what you should usually see is all quick writes to cache and
persistent memory as Figure 23-1 shows.
Chapter 23. Global Mirror performance and scalability
297
All write I/Os to FlashCopy source volumes also trigger bitmap updates. This is the bitmap
that was created when the FlashCopy volume pair was created with the start change
recording attribute. This allows the replication of only the changed recording bitmap to the
corresponding bitmap for the target volume in the course of forming a Consistency Group.
See 20.4, “Consistency Groups” on page 258.
23.2 Performance considerations at coordination time
When looking at the three phases that Global Mirror goes through to create a set of data
consistent volumes at the remote site, the first question that comes to mind is whether the
coordination window imposes an impact on the application write I/O; see Figure 23-3.
Write
I/O
Maximum
coordination
drain
time
time
Serialize all
Global Copy
source volumes
Hold write I/O
1
Maximum
A1
2
Drain data from local to remote site
Global Copy paths
3
Perform
FlashCopy
B1
C1
target
source
Global Mirror
session paths
A2
source
Global Copy paths
tertiary
B2
C2
target
Local site
tertiary
Remote site
Figure 23-3 Coordination time: How it impacts application write I/Os
The coordination time, which you can limit by specifying a number of milliseconds, is the
maximum impact that is allowed to an application’s write I/Os when forming a Consistency
Group. The intention is to keep this time window as small as possible. The default of 50 ms
might be a bit high in a transaction processing environment. A valid number can also be a
very low number in the single-digit range. The required communication between the Master
storage disk subsystem and the Subordinate disk subsystems is inband over the paths
between the Master and the subordinates. This communication is highly optimized and allows
you to minimize the potential application write I/O impact to 3 ms, for example. Note that this
communication is performed over Fibre Channel Protocol (FCP) links. There is at least one
FCP link required between a Master storage disk subsystem and a Subordinate. For
redundancy, we suggest using two FCP links.
The following example illustrates the impact of the coordination time when Consistency Group
formation starts, and whether this impact has the potential to be significant.
Assume a total aggregate number of 5 000 write I/Os over two source disk subsystems with
2 500 write I/Os per second to each disk subsystem. Each write I/O takes 0.5 ms. You
specified 3 ms maximum to coordinate between the Master the Subordinate disk subsystems.
Assume further that a Consistency Group is created every 3 seconds, which is a goal set with
the Consistency Group interval time of zero.
298
IBM System Storage DS6000 Series: Copy Services in Open Environments
The example assumptions are:
򐂰
򐂰
򐂰
򐂰
5000 write I/Os.
0.5 ms response time for each write I/O.
Maximum coordination time is 3 ms.
A Consistency Group is created every 3 seconds.
This is 5 I/Os for every millisecond or 15 I/Os within 3 ms. So, each of these 15 write I/Os
experience a 3 ms delay. This happens every 3 seconds. Then we observe an average
response time delay of approximately:
(15 IOs * 0.003 sec) / 3*5000 IO/sec) = 0.000003 sec or 0.003 ms
The response time increases in average from 0.5 ms to 0.503 ms.
23.3 Consistency Group drain time
Once the Consistency Group is set at the source disk subsystem by setting the
corresponding bitmaps within the coordination time window, all remaining data that is still in
the out-of-sync bitmap is sent (drained) by Global Copy to the target disk subsystem.
This drain period might also be limited to a maximum drain time. The default is 30 seconds
and is considered to be too small in a potentially write-intensive workload. Consider a number
in the range of 300 seconds to 600 seconds.
This replication process usually does not impact the application write I/O. There is a very low
possibility for a track within a Consistency Group to be updated before this track is replicated
to the remote site within this drain time period. When this unlikely event happens, the track is
immediately replicated to the target disk subsystem before the application write I/O modifies
the original track. The involved application write I/O experiences a similar response time delay
as if the I/O had been written to a Metro Mirror source volume. Note that subsequent writes to
this same track do not experience any delay, because the track has already been replicated
to the remote site.
23.4 Remote storage disk subsystem configuration
There are I/O skews and hot spots in the storage disk subsystems. This is true for the local
and remote disk subsystems. In local disk subsystems, you might consider a horizontal
pooling approach and spread each volume type across all ranks. Volume types in this context
are, for example, DB2 database volumes, logging volumes, batch volumes, and temporary
volumes. Your goal can be to have the same amount of each volume type within each rank.
Through a one-to-one mapping from a local to a remote disk subsystem, you achieve the
same configuration at the remote site for the B volumes and the C volumes. Figure 23-4 on
page 300 proposes spreading the B and C volumes across different ranks.
The goal is to put the same numbers of each volume type into each rank. For volume type
here, we refer to B volumes and C volumes within a Global Mirror configuration. In order to
avoid performance bottlenecks, spread busy volumes over multiple ranks. Otherwise, hot
spots can concentrate on single ranks when you put the B and C related volumes on the very
same rank. We recommend spreading B and C volumes as Figure 23-4 on page 300
suggests.
Another approach is to focus on the very busy volumes and keep these volumes on separate
ranks from all of the other volumes.
Chapter 23. Global Mirror performance and scalability
299
Primary
Primary
tertiary
copy pending
copy pending
Rank 1
Rank 1
FCP
Primary
Primary
A
B2
FCP links
Primary
Primary
FCP
C1
target
source
tertiary
copy pending
copy pending
Rank 2
Rank 2
A
Primary
Primary
A3
Primary
Primary
A
B3
C2
target
tertiary
source
Host
C3
target
source
A
Primary
Primary
A2
Primary
Primary
A
B1
A
Primary
Primary
A1
Primary
Primary
copy pending
copy pending
Rank 3
Rank 3
Local site
Remote site
Figure 23-4 Remote disk subsystem with all ranks containing equal numbers of volumes
With mixed Disk Drive Module (DDM) capacities and different speeds at the remote storage
disk subsystem, you might consider spreading B volumes not only over the fast DDMs but
over all ranks. Basically, follow an approach similar to the Figure 23-4 recommendation. You
can keep particularly busy B volumes and C volumes on the faster DDMs.
Figure 23-5 shows a configuration that incorporates the D volumes, which you can create
once in a while for test or other purposes.
A
A1
source
target
tertiary
Rank 1
Rank 1
FCP links
FCP
FCP
source
C1
target
tertiary
Rank 2
Rank 2
A
A3
Primary
Primary
A
B3
C2
target
tertiary
Primary
Primary
source
Primary
Primary
copy pending
copy pending
Rank 3
Rank 3
Local site
Host
Remote
site
A
D1
Primary
Primary
A
D3
Primary
Primary
Primary
Primary
Primary
Primary
Rank 4
Figure 23-5 Remote disk subsystem with D volumes
300
Primary
Primary
A
B2
Primary
Primary
copy pending
copy pending
Host
C3
copy pending
copy pending
A
Primary
Primary
A2
Primary
Primary
A
B1
Primary
Primary
Primary
Primary
IBM System Storage DS6000 Series: Copy Services in Open Environments
D2
D4
For a situation such as the one illustrated in Figure 23-5 on page 300, we suggest a rank with
larger and perhaps slower DDMs as an alternative. The D volumes can be read from another
host and any other I/O to the D volumes does not impact the Global Mirror volumes in the
other ranks.
Note that if you use the nocopy option for the relationship between the B and D volumes, this
reads the data from B when read I/Os to the D volume happen. So for this circumstance, you
can consider using the copy option, thus preventing additional I/O to the ranks with the B
volumes. However in this case, until the background copy between B and D completes, there
might be some impact to the Global Mirror data transfer.
An option here can be to spread all B volumes across all ranks again and also to configure the
same number of volumes in each rank. Still, put the B and C volumes in different ranks. We
further recommend that you configure corresponding B and C volumes in such a way that
these volumes have an affinity to the same server. It is ideal to also have the B volumes
connected to a different Disk Array (DA) pair than the C volumes.
23.5 Balancing the disk subsystem configuration
In this section, we show examples of how to gather information and to analyze how well the
storage disk subsystem is balanced in relation to the I/O load.
Example 23-1 shows showgmiroos commands outputs, which display OutOfSyncTracks for
the storage image scope as well as two LSS scopes. The first showgmiroos command with the
-scope si parameter shows that some Out Of SyncTracks remain within the disk subsystem.
The second showgmiroos command with the -scope lss parameter shows OutOfSyncTracks
with a value of 0 (zero). This means that for LSS 14, there are no tracks remaining at the local
site that have not been transferred yet to the remote site.
This is different with LSS 15 where there is still data that has not yet been transferred to the
remote site. The third showgmiroos command shows that there are 85 921 tracks or roughly
5.58 GBs still waiting to get replicated from the local to the remote site. This situation denotes
a significant write I/O skew between LSS 14 and LSS 15.
Example 23-1 Out Of Sync Tracks shown by the showgmiroos command
dscli> showgmiroos -dev IBM.1750-1300819
Date/Time: November 21, 2005 10:58:11 PM
IBM.1750-1300819
Scope
IBM.1750-1300819
Session
02
OutOfSyncTracks 91484
dscli>
dscli> showgmiroos -dev IBM.1750-1300819
Date/Time: November 21, 2005 10:58:21 PM
IBM.1750-1300819
Scope
IBM.1750-1300819/14
Session
02
OutOfSyncTracks 0
dscli>
dscli> showgmiroos -dev IBM.1750-1300819
Date/Time: November 21, 2005 10:58:30 PM
IBM.1750-1300819
Scope
IBM.1750-1300819/15
Session
02
OutOfSyncTracks 85921
-lss IBM.1750-1300819/14 -scope si 02
JST IBM DSCLI Version: 5.1.0.204 DS:
-lss IBM.1750-1300819/14 -scope lss 02
JST IBM DSCLI Version: 5.1.0.204 DS:
-lss IBM.1750-1300819/15 -scope lss 02
JST IBM DSCLI Version: 5.1.0.204 DS:
Chapter 23. Global Mirror performance and scalability
301
Example 23-2 shows the lspprc command that reports the Out Of Sync Tracks by volume
level. Here, we see that two volumes in LSS 14 have no Out Of Sync Tracks and two volumes
in LSS 15 have some number of Out Of Sync Tracks.
This particular configuration has source and target FlashCopy volumes within the very same
rank. Either you can consider putting one of the two busy volumes in LSS 15 into LSS 14 in
order to distribute the load over both LSSs, or another approach is to configure the
FlashCopy source volume and its corresponding target into different ranks or LSSs. See 23.4,
“Remote storage disk subsystem configuration” on page 299.
Example 23-2 Out Of Sync Tracks shown by the lspprc command
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 21, 2005 10:58:39 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS
=====================================================================================================================
1400:1A00 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
1401:1A01 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
1500:1B00 Copy Pending Global Copy 31982
Disabled Disabled
invalid
15
1501:1B01 Copy Pending Global Copy 51856
Disabled Disabled
invalid
15
When the load distribution is unknown in your configuration, you might consider developing
some rudimentary code based on for example, a script, which regularly issues the
showgmiroos commands as shown in Example 23-1 on page 301 and the lspprc -l
command as shown in Example 23-2. You can then process the output of these commands to
better understand the write load distribution over the Global Copy source volumes.
Note that the numbers in Example 23-1 on page 301 might be just a brief peak period. It is
still feasible to use the conventional approach with I/O performance reports, such as iostat in
the UNIX environment, to investigate the write workload. Or, you can use Tivoli Productivity
Center for Disk to analyze the storage disk subsystem performance.
23.6 Growth within Global Mirror configurations
When you add a rather large number of volumes at once to an existing Global Mirror session,
the available resources for Global Copy within the affected ranks might be overutilized or
even monopolized by the initial copy pass. To avoid too much impact, consider adding many
new volumes in stages to an existing Global Mirror session. If possible and as a rough
general rule, add only a few volumes in a rank during application peak I/O periods. Once the
first initial copy is complete, add the next few volumes. Again, as a general rule, plan for
adding one or two volumes in a rank during peak I/O load for the first initial copy pass. If
possible, then plan a massive add of new Global Copy volumes into an existing session
during off-peak periods.
302
IBM System Storage DS6000 Series: Copy Services in Open Environments
24
Chapter 24.
Global Mirror examples
This chapter provides examples that illustrate how to set up and manage a Global Mirror
environment using the DS CLI and the DS Storage Manager GUI. The examples describe
how to:
򐂰
򐂰
򐂰
򐂰
Set up Global Mirror (paths, volume pairs, and session).
Manage Global Mirror.
Remove the environment.
Manage a site swap from the production site to the recovery site, and back.
You can complement the information discussed in this chapter with the publication IBM
System Storage DS6000: Command-Line Interface User´s Guide, GC26-7922.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
303
24.1 Set up a Global Mirror environment using the DS CLI
In this section, we present an example of how to set up a Global Mirror environment using the
DS CLI. Figure 24-1 shows the configuration that we plan to implement.
In this configuration, different LSS and LUN numbers were used across the A, B, and C
components, so that you can definitely identify every element when it is referenced in the
discussion.
Note: In a real environment, and different than our example, it is better for you to maintain
a symmetrical configuration in terms of both physical and logical elements to simplify the
management of your Global Mirror environment.
For a more redundant and balanced configuration, we put two physical paths on separate
servers (controllers) in the DS6000s.
FlashCopy
Global Copy
A
LSS14
1401
FCP link
Server 0
1400
LSS15
FCP link
Global Copy Paths
Server 1
1501
1A00
LSS1C
1C00
1A01
LSS1B
Server 1
1500
C
FlashCopy
LSS1A
Global Copy Paths
Server 0
GM Master
B
1C01
FlashCopy LSS1D
1B00
1D00
1B01
DS6000#1 (local)
DS6000#2 (remote)
-dev IBM.1750-1300819
-dev IBM.1750-1300247
1D01
Figure 24-1 DS6000 configuration in the Global Mirror example
24.1.1 Preparing to work with the DS CLI
Before starting the tasks to configure the Global Mirror environment, we recommend you first
do an initial DS CLI setup similar to the one explained in “Simplifying the DS CLI command
syntax” on page 140. This initial DS CLI setup allows a simpler syntax of the commands you
use to configure the Global Mirror environment.
24.1.2 Configuration used for the environment
Figure 24-1 shows the configuration we used for this example. The configuration has four A
volumes residing in two LSSs on DS6000#1, four B volumes residing in two LSSs on
DS6000#2, and four C volumes residing in two other LSSs also on DS6000#2. Two paths are
defined for each Global Copy source and target LSS pair (LSS14:LSS1A and LSS15:LSS1B). We
start the Global Mirror Master in LSS14.
304
IBM System Storage DS6000 Series: Copy Services in Open Environments
24.1.3 Setup procedure
The sequence of steps for the creation of a Global Mirror environment is not completely fixed
and allows for some variation. Still, we recommend you follow this procedure:
1. Create Global Copy relationships: A to B volumes.
2. Create FlashCopy relationships: B to C volumes.
3. Start the Global Mirror session.
24.1.4 Create Global Copy relationships: A to B volumes
The first step of the procedure is to create the Global Copy relationships between the A and
the B volumes. For this, you must do the following:
1. Determine the available Fibre Channel Protocol (FCP) links between the local and the
remote disk subsystems.
2. Create the Global Copy paths between the local and the remote LSSs.
3. Create the Global Copy volume pairs.
4. Wait until the first copy of the Global Copy pairs completes.
This procedure is followed in Example 24-1 where you can see the sequence of commands
and the corresponding results.
Note that the tasks you must perform to create the Global Copy relationships in a Global
Mirror environment are similar to what has been presented in Part 5, “Global Copy” on
page 183. You can refer to 19.1.2 “Setup of Global Copy configuration” on page 216.
Example 24-1 Create Global Copy pairs relationships (A to B)
<< Determine the available Fibre links >>
dscli> lsavailpprcport -l -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 14:1a
Date/Time: November 21, 2005 11:50:17 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0102
I0103
FCP NA
NA
dscli>
dscli> lsavailpprcport -l -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 15:1b
Date/Time: November 21, 2005 11:56:56 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0102
I0103
FCP NA
NA
<< Create paths >>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 1a i0003:i0003 i0102:i0103
Date/Time: November 21, 2005 11:59:52 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:1a successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 15
-tgtlss 1b i0003:i0003 i0102:i0103
Date/Time: November 22, 2005 12:00:11 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00149I mkpprcpath: Remote Mirror and Copy path 15:1b successfully established.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 22, 2005 12:00:48 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src Tgt State
SS Port Attached Port Tgt WWNN
=========================================================
Chapter 24. Global Mirror examples
305
14 1A
14 1A
15 1B
15 1B
dscli>
Success
Success
Success
Success
FF1A
FF1A
FF1B
FF1B
I0003
I0102
I0003
I0102
I0003
I0103
I0003
I0103
500507630EFFFE16
500507630EFFFE16
500507630EFFFE16
500507630EFFFE16
<< Create Global Copy pairs >>
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time:
CMUC00153I
CMUC00153I
CMUC00153I
CMUC00153I
November 22, 2005 12:02:01 AM JST IBM
mkpprc: Remote Mirror and Copy volume
mkpprc: Remote Mirror and Copy volume
mkpprc: Remote Mirror and Copy volume
mkpprc: Remote Mirror and Copy volume
DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
pair relationship 1400:1A00 successfully created.
pair relationship 1401:1A01 successfully created.
pair relationship 1500:1B00 successfully created.
pair relationship 1501:1B01 successfully created.
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 22, 2005 12:02:31 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS
Timeout (secs) Critical Mode First Pass Status
=====================================================================================================================
1400:1A00 Copy Pending Global Copy 38376
Disabled Disabled
invalid
14
1401:1A01 Copy Pending Global Copy 38176
Disabled Disabled
invalid
14
1500:1B00 Copy Pending Global Copy 29596
Disabled Disabled
invalid
15
1501:1B01 Copy Pending Global Copy 32956
Disabled Disabled
invalid
15
<< wait to see that the Out Of Sync Tracks shows 0 >>
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 22, 2005 12:05:32 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS
=====================================================================================================================
1400:1A00
1401:1A01
1500:1B00
1501:1B01
Copy
Copy
Copy
Copy
Pending
Pending
Pending
Pending
-
Global
Global
Global
Global
Copy
Copy
Copy
Copy
0
0
0
0
Disabled
Disabled
Disabled
Disabled
Disabled
Disabled
Disabled
Disabled
invalid
invalid
invalid
invalid
-
14
14
15
15
24.1.5 Create FlashCopy relationships: B to C volumes
To create the FlashCopy relationships between the B and C volumes, you use the mkflash or
the mkremoteflash command with the following parameters: -tgtinhibit, -record, and
-nocp. The -persist parameter is automatically selected when the -record parameter is
selected; therefore, you do not specify the -persist parameter explicitly. Following is a brief
explanation for each parameter; see also 20.3.4 “Introduce FlashCopy” on page 255:
򐂰 -tgtinhibit
Prevents host system writes to the target while the FlashCopy relationship exists.
򐂰 -record
Keeps a record of the tracks that were modified on both volumes within a FlashCopy pair.
Select this parameter when you create an initial FlashCopy volume pair that you intend to
use with the resyncflash command.
򐂰 -nocp
Inhibits background copy. Data is copied from the source volume to the target volume only
if a track on the source volume is modified.
򐂰 -persist
Keeps the FlashCopy relationship until explicitly or implicitly terminated.
Depending on your network environment, you can give the FlashCopy command to the local
DS6000#1 for its inband transmission to the remote DS6000#2. In this case, you use the
306
IBM System Storage DS6000 Series: Copy Services in Open Environments
mkremoteflash command. Alternatively, if you have connectivity to the remote DS6000#2,
then you can give the mkflash command directly to the DS6000#2.
In our example, we use the inband functionality of FlashCopy, in which case we have to
specify the LSS having the A volume for the -conduit parameter and the storage image ID at
the remote site for the -dev parameter; see Example 24-2. You have to give this command to
the DS SMC connected to the local DS6000#1.
Because the -nocp parameter is specified and the Global Copy initial copy (first pass)
completed in the previous step, no FlashCopy background copy occurs at this time.
Note: You can create this FlashCopy relationship before the initial copy of Global Copy.
However, to avoid unnecessary FlashCopy background I/Os, we do not recommend it.
Example 24-2 Create FlashCopy relationships: B to C volumes
dscli> mkremoteflash -tgtinhibit -nocp -record -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01:1c00-1c01
Date/Time: November 22, 2005 12:09:08 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1A00:1C00 successfully created. Use the lsremoteflash command
to determine copy completion.
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1A01:1C01 successfully created. Use the lsremoteflash command
to determine copy completion.
dscli>
dscli> mkremoteflash -tgtinhibit -nocp -record -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01:1d00-1d01
Date/Time: November 22, 2005 12:09:20 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1B00:1D00 successfully created. Use the lsremoteflash command
to determine copy completion.
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1B01:1D01 successfully created. Use the lsremoteflash command
to determine copy completion.
dscli>
dscli> lsremoteflash -l -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 22, 2005 12:09:32 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks
==========================================================================================================================================
1A00:1C00 1A
0
Disabled
Enabled
Enabled
Disabled
Enabled
Disabled
Disabled
61036
1A01:1C01 1A
0
Disabled
Enabled
Enabled
Disabled
Enabled
Disabled
Disabled
61036
dscli>
dscli> lsremoteflash -l -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 22, 2005 12:09:48 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks
==========================================================================================================================================
1B00:1D00 1B
0
Disabled
Enabled
Enabled
Disabled
Enabled
Disabled
Disabled
61036
1B01:1D01 1B
0
Disabled
Enabled
Enabled
Disabled
Enabled
Disabled
Disabled
61036
24.1.6 Start Global Mirror
To start the Global Mirror operation, we must perform the following steps:
1. Define the Global Mirror session on the involved LSSs (Master and Subordinates).
2. Add the A volumes to the session.
3. Start the Global Mirror session.
Define the Global Mirror session on the involved LSSs
The mksession command defines the Global Mirror session to the specified LSSs. You do
this to all the LSSs that are going to be involved in the Global Mirror session, Master and
Subordinates. You can verify the results with the lssession command.
In our example, we have two LSSs in the local DS6000 that are going to participate in the
Global Mirror environment: LSS14 and LSS15. Therefore, we give the mksession command
twice, and in each occasion, we use the -lss parameter to specify the selected LSS. You
Chapter 24. Global Mirror examples
307
also specify the Global Mirror session ID with this command. This session ID is used when
you start Global Mirror in a later step. In our example, we specify 02 for the Global Mirror
session ID. Example 24-3 shows the commands mksession and lssession that we used in
our example.
Example 24-3 Define the Global Mirror session on each LSS
dscli> mksession -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 02
Date/Time: November 22, 2005 12:14:01 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00145I mksession: Session 02 opened successfully.
dscli>
dscli> mksession -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 02
Date/Time: November 22, 2005 12:14:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00145I mksession: Session 02 opened successfully.
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 12:14:31 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus SecondaryStatus FirstPassComplete AllowCascading
========================================================================================================
14
02
15
02
-
Add the A volumes to the session
The next step is to add the A volumes to the session that was defined in the previous step. For
this, we use the chsession -action add -volume command, and we verify the results with the
lssession command. Example 24-4 shows the commands that we used in our example.
Example 24-4 Add the A volumes to the session on each LSS
dscli> chsession -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -action add -volume 1400-1401 02
Date/Time: November 22, 2005 12:17:01 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00147I chsession: Session 02 successfully modified.
dscli>
dscli> chsession -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 -action add -volume 1500-1501 02
Date/Time: November 22, 2005 12:17:08 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00147I chsession: Session 02 successfully modified.
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 12:17:16 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
14
02
Normal 1400 Join Pending Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401 Join Pending Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500 Join Pending Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Join Pending Primary Copy Pending Secondary Simplex True
Disable
With the chsession command, you specify the Global Mirror session ID, which is 02 in our
example, and the volumes that are going to be part of the Global Mirror environment. After
adding the A volumes to the session, the lssession command shows the volume’s ID with its
state in the LSS. At this time in the procedure, the volumes’ state is join pending. This means
that the volumes are not active for the current session; we have not yet started the Global
Mirror session.
Note: At this step, we do not have to do anything to add the B and C volumes to the Global
Mirror session. They are automatically recognized by the Global Mirror mechanism through
the Global Copy relationships and the FlashCopy relationships.
308
IBM System Storage DS6000 Series: Copy Services in Open Environments
In addition to the chsession command, you can also add the A volumes with the mksession
command when you define the Global Mirror session on a LSS; see Example 24-5.
Example 24-5 Add the A volumes when you create a Global Mirror session
dscli> mksession -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -volume 1400-1401 02
Date/Time: November 22, 2005 12:19:39 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00145I mksession: Session 02 opened successfully.
dscli>
dscli> mksession -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 -volume 1500-1501 02
Date/Time: November 22, 2005 12:19:50 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00145I mksession: Session 02 opened successfully.
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 12:19:58 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
14
02
Normal 1400 Join Pending Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401 Join Pending Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500 Join Pending Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Join Pending Primary Copy Pending Secondary Simplex True
Disable
Start the Global Mirror session
Now we can start the Global Mirror session, so the process of Consistency Group formation
begins. We use the mkgmir command for this. You can verify the results with the showgmir
command; see Example 24-6.
Example 24-6 Start Global Mirror session 02
dscli> mkgmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 12:22:45 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00162I mkgmir: Global Mirror for session 02 successfully started.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 12:22:57 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 02:31:36 JST
CG Time
11/22/2005 02:31:36 JST
Successful CG Percentage
100
FlashCopy Sequence Number 0x43820478
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
-
In the mkgmir command, the LSS that we specified with the -lss parameter becomes the
Master. In our example, it is LSS14. With this command, we also specify the Global Mirror
session ID of the session that we are starting.
Chapter 24. Global Mirror examples
309
At the time that you start the Global Mirror session, you can change the Global Mirror tuning
parameters of the session with the mkgmir command:
򐂰 -cginterval: Specifies how long to wait between the formation of Consistency Groups.
If this number is not specified or is set to zero, Consistency Groups are formed
continuously.
򐂰 -coordinate: Indicates the maximum time that Global Mirror processing can hold host
I/Os in the source disk subsystem to start forming a Consistency Group.
򐂰 -drain: Specifies the maximum amount of time in seconds that is allowed for the data to
drain to the remote site before failing the current Consistency Group.
For additional discussion about the tuning parameters, refer to 20.4.2 “Consistency Group
parameters” on page 260 and Chapter 23, “Global Mirror performance and scalability” on
page 295.
This showgmir command shows the Global Mirror current status. The Copy State field
indicates Running, which means that Global Mirror is satisfactorily operating. A Fatal state
would have indicated that Global Mirror failed, and the Fatal Reason field would have shown
the reason for the failure.
The showgmir command also shows the current time in the Current Time field, which is the
time when the DS6000 received this command. The time when the last successful
Consistency Group was formed is shown in the CG Time field. You can obtain the current
RPO (Recovery Time Objective) for this Global Mirror session from the difference between
the Current Time and the CG Time.
From the lssession command in Example 24-7, you can see that after starting the Global
Mirror session, the VolumeStatus of the A volumes changes from Join Pending to Active.
Example 24-7 The A volumes’ status after starting the Global Mirror session
dscli> lssession 14-15
Date/Time: November 22, 2005 12:24:19 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status
Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete
AllowCascading
========================================================================================================================
14
02
CG In Progress 1400 Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
CG In Progress 1401 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
CG In Progress 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
CG In Progress 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
If we use the -metrics parameter with the showgmir command, we can obtain metrics for
Global Mirror after we have started the session; see Example 24-8.
Example 24-8 The showgmir command with the -metrics parameter
dscli> showgmir -metrics -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 12:27:04 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Total Failed CG Count
0
Total Successful CG Count
246
Successful CG Percentage
100
Failed CG after Last Success 0
Last Successful CG Form Time 11/22/2005 02:35:43 JST
Coord. Time (seconds)
50
Interval Time (seconds)
0
Max Drain Time (seconds)
30
First Failure Control Unit
First Failure LSS
-
310
IBM System Storage DS6000 Series: Copy Services in Open Environments
First Failure Status
First Failure Reason
First Failure Master State
Last Failure Control Unit
Last Failure LSS
Last Failure Status
Last Failure Reason
Last Failure Master State
Previous Failure Control Unit
Previous Failure LSS
Previous Failure Status
Previous Failure Reason
Previous Failure Master State
No Error
No Error
No Error
-
In Example 24-8 on page 310, the Total Failed CG Count field indicates the total number of
Consistency Groups that did not complete successfully after we have started the Global
Mirror. The Total Successful CG Count indicates the total number of Consistency Groups that
completed successfully. First Failure indicates the first failure after we have started this
session. Last Failure indicates the latest failure, and Previous Failure indicates the failure
before the latest one. Stopping the Global Mirror and starting it again clears all of this failure
information. Pausing and resuming the Global Mirror operation does not reset this
information.
Depending on the Global Mirror parameters that you set and your system environment, the
Consistency Group formation can occasionally fail and the showgmir -metrics might show
the error messages. A typical case is that you see Max Drain Time Exceeded with the
showgmir command when data of the out-of-sync bitmap cannot be drained within the
specified time. However, this failure does not mean that you lose consistent data at the
remote site, because Global Mirror does not take FlashCopy (B to C) for the failed
Consistency Group data. Global Mirror continues to attempt to form additional Consistency
Groups without external intervention. If failures repeatedly continue (no more Consistency
Groups are formed), the percentage of successful Consistency Groups is unacceptable
(many failures occur), or the frequency of Consistency Groups is not meeting your
requirements or Recovery Point Objective (RPO), then the failures are a problem and need to
be addressed.
There is another command related to Global Mirror, which is the showgmiroos command. This
command reports the number of Out Of Sync Tracks that at a given moment Global Mirror
has to transmit to the remote site. The size of the logical track on the DS6000 FB volume is
64 KB. With the -scope parameter, you select either the storage image scope or the LSS
scope for the information to be reported. See Example 24-9.
Example 24-9 The showgmiroos command
dscli> showgmiroos -dev IBM.1750-1300819
Date/Time: November 21, 2005 10:56:29 PM
Scope
IBM.1750-1300819
Session
02
OutOfSyncTracks 14871
dscli>
dscli> showgmiroos -dev IBM.1750-1300819
Date/Time: November 21, 2005 10:56:53 PM
Scope
IBM.1750-1300819/14
Session
02
OutOfSyncTracks 0
dscli>
dscli> showgmiroos -dev IBM.1750-1300819
Date/Time: November 21, 2005 10:57:04 PM
-lss IBM.1750-1300819/14 -scope si 02
JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
-lss IBM.1750-1300819/14 -scope lss 02
JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
-lss IBM.1750-1300819/15 -scope lss 02
JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Chapter 24. Global Mirror examples
311
Scope
IBM.1750-1300819/15
Session
02
OutOfSyncTracks 85055
24.2 Remove a Global Mirror environment with the DS CLI
In this section, we provide an example of how to remove a Global Mirror environment using
the DS Command-Line Interface.
The Global Mirror environment delete process can be structured in five consecutive steps:
1.
2.
3.
4.
5.
6.
End Global Mirror processing.
Remove volumes from the session.
Remove the Global Mirror session.
Terminate the FlashCopy pairs.
Terminate the Global Copy pairs.
Remove the paths.
a. Between the local site to the remote site.
b. Between the Master LSS and the Subordinate LSSs.
24.2.1 End Global Mirror processing
We terminate Global Mirror processing with the rmgmir command. With this command, we
have to specify the Master LSS and the Global Mirror session ID of the session we are going
to close; see Example 24-10. Before you end Global Mirror processing, first display the
session information using the showgmir command.
You can use the -quiet parameter to turn off the confirmation prompt for this command.
Example 24-10 Terminate Global Mirror
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 12:32:54 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 02:41:33 JST
CG Time
11/22/2005 02:41:33 JST
Successful CG Percentage
100
FlashCopy Sequence Number 0x438206CD
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
dscli>
dscli> rmgmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 12:34:34 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00166W rmgmir: Are you sure you want to stop the Global Mirror session 02:? [y/n]:y
CMUC00165I rmgmir: Global Mirror for session 02 successfully stopped.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 12:34:59 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
312
IBM System Storage DS6000 Series: Copy Services in Open Environments
Master Count
Master Session ID
Copy State
Fatal Reason
CG Interval (seconds)
XDC Interval(milliseconds)
CG Drain Time (seconds)
Current Time
CG Time
Successful CG Percentage
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
-
Example 24-10 on page 312 illustrates how to end Global Mirror session 02 processing.
Although this command might interrupt the formation of a Consistency Group, every attempt
is made to preserve the previous consistent copy of the data on the FlashCopy target
volumes. If, due to failures, this command cannot complete without compromising the
consistent copy, the command stops processing and an error code is issued. If this occurs,
reissue the rmgmir command with the -force parameter to force the command to stop the
Global Mirror process.
24.2.2 Remove the A volumes from the Global Mirror session
With the chsession command with the -action remove -volume parameters, you remove the
A volumes from the Global Mirror session for a given LSS; see Example 24-11. First, you give
an lssession command to get Global Mirror session volume information for the required
LSSs.
Example 24-11 Remove the A volumes from the Global Mirror session
dscli> lssession 14-15
Date/Time: November 22, 2005 12:38:52 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCas
===========================================================================================================
14
02
Normal 1400
Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501
Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> chsession -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -action remove -volume 1400-1401 02
Date/Time: November 22, 2005 12:39:25 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00147I chsession: Session 02 successfully modified.
dscli>
dscli> chsession -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 -action remove -volume 1500-1501 02
Date/Time: November 22, 2005 12:39:38 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00147I chsession: Session 02 successfully modified.
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 12:39:45 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus SecondaryStatus FirstPassComplete AllowCascading
========================================================================================================
14
02
15
02
-
Chapter 24. Global Mirror examples
313
24.2.3 Remove the Global Mirror session
With the rmsession command, you undefine the Global Mirror session from an LSS; see
Example 24-12. You do this to all the source LSSs where the Global Mirror session was
defined. At the end, you do an lssession command to verify that the Global Mirror session
has been removed.
Example 24-12 Remove the Global Mirror session from the LSSs
dscli> rmsession -quiet -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 02
Date/Time: November 22, 2005 12:41:26 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00146I rmsession: Session 02 closed successfully.
dscli>
dscli> rmsession -quiet -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 02
Date/Time: November 22, 2005 12:41:33 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00146I rmsession: Session 02 closed successfully.
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 12:41:40 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00234I lssession: No Session found.
Before closing the Global Mirror session on the LSS, we must remove all the A volumes from
the Global Mirror session on that LSS; otherwise, the rmsession command fails. See
Example 24-13.
Example 24-13 rmsession command fails when Global Mirror volumes were not previously removed
dscli> rmsession -quiet -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 02
Date/Time: November 22, 2005 12:43:25 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUN03107E rmsession: Copy Services operation failure: volumes in session
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 12:43:44 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCas
===========================================================================================================
14
02
Normal 1400
Join Pending Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401
Join Pending Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500
Join Pending Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501
Join Pending Primary Copy Pending Secondary Simplex True
Disable
24.2.4 Terminate FlashCopy pairs
Depending on your network environment, you can give the FlashCopy commands to the local
disk subsystem for its inband transmission to the remote disk subsystem. In this case, you
use the rmremoteflash command. Alternatively if you have connectivity to the remote disk
subsystem, then you can give the rmflash command directly to the remote disk subsystem.
In our example, we use the inband functionality of FlashCopy, in which case we have to
specify the LSS having the A volume for the -conduit parameter and the storage image ID at
the remote site for the -dev parameter. See Example 24-14. You have to give this command
to the DS SMC connected to the local DS6000#1.
Before terminating the pairs, we gather information with an lsremoteflash command.
Example 24-14 Remove all FlashCopy relationships between the B and C volumes
dscli> lsremoteflash -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 22, 2005 12:45:49 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
314
IBM System Storage DS6000 Series: Copy Services in Open Environments
========================================================================================================================
1A00:1C00 1A
43820735
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
1A01:1C01 1A
43820735
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 22, 2005 12:46:15 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
========================================================================================================================
1B00:1D00 1B
43820735
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
1B01:1D01 1B
43820735
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
dscli>
dscli> rmremoteflash -quiet -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01:1c00-1c01
Date/Time: November 22, 2005 12:48:35 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00180I rmremoteflash: Removal of the remote FlashCopy volume pair 1A00:1C00 has been initiated successfully. Use the
lsremoteflash command to determine when the
relationship is deleted.
CMUC00180I rmremoteflash: Removal of the remote FlashCopy volume pair 1A01:1C01 has been initiated successfully. Use the
lsremoteflash command to determine when the
relationship is deleted.
dscli>
dscli> rmremoteflash -quiet -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01:1d00-1d01
Date/Time: November 22, 2005 12:49:25 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00180I rmremoteflash: Removal of the remote FlashCopy volume pair 1B00:1D00 has been initiated successfully. Use the
lsremoteflash command to determine when the
relationship is deleted.
CMUC00180I rmremoteflash: Removal of the remote FlashCopy volume pair 1B01:1D01 has been initiated successfully. Use the
lsremoteflash command to determine when the
relationship is deleted.
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 22, 2005 12:49:37 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lsremoteflash: No Remote Flash Copy found.
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 22, 2005 12:49:45 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lsremoteflash: No Remote Flash Copy found.
24.2.5 Terminate Global Copy pairs and delete the paths
For the termination of the Global Copy pairs, you use the rmpprc command, and for the
deletion of the paths, you use the rmpprcpath command. See Example 24-15. Before
terminating the pairs and deleting the paths, you request information by means of the lspprc
and the lspprcpath commands respectively.
Note that the tasks you must perform to terminate the Global Copy relationships in a Global
Mirror environment are similar to what we presented in Part 5, “Global Copy” on page 183.
You can refer to 19.2 “Remove Global Copy environment using the DS CLI” on page 219.
Example 24-15 Remove all Global Copy pairs and remove the paths
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 12:52:30 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
dscli>
dscli> rmpprc -remotedev IBM.1750-1300247 -quiet 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 22, 2005 12:53:09 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully withdrawn.
Chapter 24. Global Mirror examples
315
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully withdrawn.
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully withdrawn.
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully withdrawn.
dscli>
dscli> lspprcpath 14-15
Date/Time: November 22, 2005 12:53:39 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
14 1A Success FF1A I0003 I0003
500507630EFFFE16
14 1A Success FF1A I0102 I0103
500507630EFFFE16
15 1B Success FF1B I0003 I0003
500507630EFFFE16
15 1B Success FF1B I0102 I0103
500507630EFFFE16
dscli>
dscli> rmpprcpath -quiet -remotedev IBM.1750-1300247 14:1a 15:1b
Date/Time: November 22, 2005 12:55:15 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00150I rmpprcpath: Remote Mirror and Copy path 14:1A successfully removed.
CMUC00150I rmpprcpath: Remote Mirror and Copy path 15:1B successfully removed.
24.3 Manage the Global Mirror environment with the DS CLI
In this section, we discuss and give examples of how to perform common Global Mirror
control tasks using the DS CLI. We present the following management activities:
򐂰
򐂰
򐂰
򐂰
Pause and resume the Global Mirror Consistency Group formation.
Change the Global Mirror tuning parameters.
Stop and start Global Mirror.
Add and remove A volumes to the Global Mirror environment.
24.3.1 Pause and resume the Global Mirror Consistency Group formation
The pausegmir command pauses Global Mirror Consistency Group formation. You have to
specify the Global Mirror Master LSS ID and session ID. You can verify the result with the
showgmir command, which shows the Paused state in the Copy State field; see
Example 24-16.
Note: The pausegmir command does not influence the Global Copy data transfer.
When you pause a Global Mirror session with the pausegmir command, it completes the
Consistency Group (CG) formation in progress before it pauses. This is slightly different from
the usage of the rmgmir command; refer to 24.3.3 “Stop and start Global Mirror” on page 320.
Example 24-16 Pause Global Mirror CG formation
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:09:24 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:18:03 JST
CG Time
11/22/2005 03:18:03 JST
Successful CG Percentage
100
316
IBM System Storage DS6000 Series: Copy Services in Open Environments
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
dscli>
dscli> lssession 14-15
0x43820F5B
IBM.1750-1300819
0
-
Date/Time: November 22, 2005 1:10:51 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status
Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCascadin
========================================================================================================================
14
02
CG In Progress 1400 Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
CG In Progress 1401 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
CG In Progress 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
CG In Progress 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> pausegmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 1:11:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00163I pausegmir: Global Mirror for session 02 successfully paused.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:12:11 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Paused
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:20:49 JST
CG Time
11/22/2005 03:19:48 JST
Successful CG Percentage
100
FlashCopy Sequence Number 0x43820FC4
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 1:11:14 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
14
02
Normal 1400 Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 22, 2005 1:11:25 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled Background
========================================================================================================================
1A00:1C00 1A
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
1A01:1C01 1A
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 22, 2005 1:11:33 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled Background
========================================================================================================================
1B00:1D00 1B
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
1B01:1D01 1B
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
<< SequnceNum field does not change >>
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/14 -dev IBM.1750-1300247 1a00-1a01
Date/Time: November 22, 2005 1:11:43 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Chapter 24. Global Mirror examples
317
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled Background
========================================================================================================================
1A00:1C00 1A
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
1A01:1C01 1A
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b00-1b01
Date/Time: November 22, 2005 1:11:51 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled Background
========================================================================================================================
1B00:1D00 1B
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
1B01:1D01 1B
43820FC4
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
The Status shown by the lssession command changes from CG In Progress, which means
that the Consistency Group of the session is in progress, to Normal, which means that the
session is in a normal Global Copy state. In fact, you see this state (Normal) between the
time when a FlashCopy has been taken and the time of the next Global Copy Consistency
Group formation.
Note: The FlashCopy sequence number has not changed after pausing Global Mirror,
because the FlashCopy at the remote site has not been taken; see Example 24-16.
The resumegmir command resumes Global Mirror processing for a specified session; see
Example 24-17. Consistency Group formation is resumed.
Example 24-17 Resuming Global Mirror processing
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:18:40 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Paused
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:27:18 JST
CG Time
11/22/2005 03:19:48 JST
Successful CG Percentage
100
FlashCopy Sequence Number 0x43820FC4
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
dscli>
dscli> resumegmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 1:18:56 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00164I resumegmir: Global Mirror for session 02 successfully resumed.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:19:04 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:27:43 JST
318
IBM System Storage DS6000 Series: Copy Services in Open Environments
CG Time
Successful CG Percentage
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
11/22/2005 03:27:43 JST
100
0x4382119F
IBM.1750-1300819
0
-
24.3.2 Change the Global Mirror tuning parameters
You can change the three Global Mirror tuning parameters by pausing and resuming Global
Mirror. In Example 24-18, we changed the Consistency Group interval time parameter from
zero to 60 seconds.
Example 24-18 Change the Consistency Group Interval (CG interval) time parameter
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:19:04 AM JST IBM DSCLI Version: 5.1.0.204 DS:
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:27:43 JST
CG Time
11/22/2005 03:27:43 JST
Successful CG Percentage
100
FlashCopy Sequence Number 0x4382119F
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
dscli>
dscli>
dscli> pausegmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 1:20:38 AM JST IBM DSCLI Version: 5.1.0.204 DS:
CMUC00163I pausegmir: Global Mirror for session 02 successfully paused.
dscli>
dscli> resumegmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 1:21:27 AM JST IBM DSCLI Version: 5.1.0.204 DS:
CMUC00164I resumegmir: Global Mirror for session 02 successfully resumed.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:21:35 AM JST IBM DSCLI Version: 5.1.0.204 DS:
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
60
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:30:13 JST
CG Time
11/22/2005 03:30:07 JST
Successful CG Percentage
100
FlashCopy Sequence Number 0x4382122F
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
-
IBM.1750-1300819
IBM.1750-1300819
-cginterval 60
IBM.1750-1300819
IBM.1750-1300819
Chapter 24. Global Mirror examples
319
24.3.3 Stop and start Global Mirror
To stop Global Mirror means to stop the Global Mirror Master on a LSS by using the rmgmir
command; see Example 24-19. You stop Global Mirror, for example, when you want to start
using a different topology; see 21.3.4 “Global Mirror environment topology changes” on
page 269.
Example 24-19 Stop and start Global Mirror
dscli> lssession 14-15
Date/Time: November 22, 2005 1:24:59 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
14
02
Normal 1400 Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> rmgmir -quiet -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 1:25:27 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00165I rmgmir: Global Mirror for session 02 successfully stopped.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:25:36 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
Master Session ID
Copy State
Fatal Reason
CG Interval (seconds)
XDC Interval(milliseconds) CG Drain Time (seconds)
Current Time
CG Time
Successful CG Percentage FlashCopy Sequence Number Master ID
Subordinate Count
Master/Subordinate Assoc dscli>
dscli> lssession 14-15
Date/Time: November 22, 2005 1:25:43 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
14
02
Normal 1400 Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> mkgmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 1:28:21 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00162I mkgmir: Global Mirror for session 02 successfully started.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 1:28:30 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 03:37:08 JST
320
IBM System Storage DS6000 Series: Copy Services in Open Environments
CG Time
Successful CG Percentage
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
dscli>
11/22/2005 03:37:08 JST
100
0x438213D4
IBM.1750-1300819
0
-
dscli> lssession 14-15
Date/Time: November 22, 2005 1:28:40 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status
Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete
AllowCascading
====================================================================================================================
14
02
CG In Progress 1400
Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
CG In Progress 1401
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
CG In Progress 1500
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
CG In Progress 1501
Active
Primary Copy Pending Secondary Simplex True
Disable
You do not need to remove the Global Copy and FlashCopy relationships to stop the Global
Mirror Master. After stopping the Global Mirror Master, the Consistency Group formation does
not continue, which means the FlashCopy sequence number at the remote site does not
increment.
Although the operation to stop Global Mirror with the rmgmir command might interrupt the
formation of a Consistency Group, every attempt is made to preserve the previous consistent
copy of the data on the FlashCopy target volumes. If due to failures, the rmgmir command
cannot complete without compromising the consistent copy, the command stops processing
and an error code is issued. If this occurs, reissue the rmgmir command with the -force
parameter to force the command to stop the Global Mirror process.
The mkgmir command restarts the Global Mirror Master. You can specify another LSS on
which you start the Global Mirror Master.
24.3.4 Add and remove A volumes to the Global Mirror environment
First, you create the Global Copy (A to B) and FlashCopy (B to C) relationships for the A
volume that you want to add to the Global Mirror environment. After this, you add the A
volume to the Global Mirror session, using the chsession -action add -volume command.
In Example 24-20, we added A volume 1502 that is associated to B volume 1B02 and C volume
1D02.
Example 24-20 Add an A volume to the Global Mirror environment
<< Preparing an A volume >>
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 1502:1b02
Date/Time: November 22, 2005 1:43:06 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1502:1B02 successfully created.
dscli>
dscli> lspprc -l 1502
Date/Time: November 22, 2005 1:43:16 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs) Cr
==========================================================================================================================================
1502:1B02 Copy Pending Global Copy 43855
Disabled Disabled
invalid
15
unknown
Dis
dscli>
dscli> lspprc -l 1502
Date/Time: November 22, 2005 1:45:10 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs) Crit
==========================================================================================================================================
1502:1B02 Copy Pending Global Copy 0
Disabled Disabled
invalid
15
unknown
Dis
dscli> mkremoteflash -tgtinhibit -nocp -record -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b02:1d02
Date/Time: November 22, 2005 1:47:55 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Chapter 24. Global Mirror examples
321
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1B02:1D02 successfully created. Use the
lsremoteflash command to determine copy completion.
dscli>
dscli> lsremoteflash -conduit IBM.1750-1300819/15 -dev IBM.1750-1300247 1b02
Date/Time: November 22, 2005 1:48:19 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled Background
========================================================================================================================
1B02:1D02 1B
0
Disabled Enabled Enabled
Disabled
Enabled
Disabled
Disabled
<< Add the A volume to Global Mirror >>
dscli> lssession 15
Date/Time: November 22, 2005 1:51:32 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCas
===========================================================================================================
15
02
Normal 1500
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501
Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> chsession -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 -action add -volume 1502 02
Date/Time: November 22, 2005 1:51:46 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00147I chsession: Session 02 successfully modified.
dscli>
dscli> lssession 15
Date/Time: November 22, 2005 1:51:52 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCas
===========================================================================================================
15
02
Normal 1500
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1502
Join Pending Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> lssession 15
Date/Time: November 22, 2005 1:52:14 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCas
===========================================================================================================
15
02
Normal 1500
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1502
Active
Primary Copy Pending Secondary Simplex True
Disable
The A volumes can be in any state, such as simplex (no relationship), copy pending, or
suspended, to be added to a Global Mirror session. Volumes that have not completed their
initial copy phase (also called, first pass) stay in a Join Pending state until the first pass is
complete. You can check the first pass status with the lspprc -l command; see
Example 24-21. The First Pass Status field reports this information, where True means that
the Global Copy first pass has completed.
Example 24-21 Check the first pass completion for the Global Copy initial copy
dscli> lspprc -l -fmt stanza 1502
Date/Time: November 22, 2005 1:54:13 AM JST IBM DSCLI Version: 5.1.0.204 DS:
IBM.1750-1300819
ID
1502:1B02
State
Copy Pending
Reason
Type
Global Copy
Out Of Sync Tracks 0
Tgt Read
Disabled
Src Cascade
Disabled
Tgt Cascade
invalid
Date Suspended
SourceLSS
15
Timeout (secs)
unknown
322
IBM System Storage DS6000 Series: Copy Services in Open Environments
Critical Mode
First Pass Status
Disabled
True
When you want to remove an A volume from the Global Mirror environment, you can use the
chsession -action remove -volume command. You first remove the A volume from the Global
Mirror session and then remove its Global Copy and FlashCopy relationships. See
Example 24-22.
Example 24-22 Remove an A volume from the Global Mirror environment
dscli> lssession 15
Date/Time: November 22, 2005 1:58:13 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
15
02
Normal 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1502 Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> chsession -dev IBM.1750-1300819 -lss IBM.1750-1300819/15 -action remove -volume 1502 02
Date/Time: November 22, 2005 1:58:33 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00147I chsession: Session 02 successfully modified.
dscli>
dscli> lssession 15
Date/Time: November 22, 2005 1:58:38 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus FirstPassComplete AllowCascading
=================================================================================================================
15
02
Normal 1500 Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501 Active
Primary Copy Pending Secondary Simplex True
Disable
Important: Suspending or removing even one Global Copy pair that belongs to an active
Global Mirror session impacts the formation of Consistency Groups. If you suspend or
remove the Global Copy relationship from the A volume without removing the volume from
the Global Mirror session, it causes Consistency Group formation to fail, and periodical
SNMP alerts are issued.
24.4 Recovery scenario after local site failure with the DS CLI
The example presented in this section discusses how to perform the required steps to
recover from a production site failure using DS CLI commands. For a detailed discussion of
the internal Global Mirror considerations, refer to 21.5 “Recovery scenario after production
site failure” on page 274.
Chapter 24. Global Mirror examples
323
Application server and
DS CLI client
Application Backup server and
DS CLI client
LSS14
GM Master
1400
1401
LSS1A
1A00
Global Copy Pair
1C00
1A01
LSS15
1500
FlashCopy LSS1C
LSS1B
1C01
FlashCopy
LSS1D
1B00
Global Copy Pair
1501
1D00
1B01
1D01
A
B
C
GC: Source
Copy Pending
GC: Target
Copy Pending
FC: Source
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-2 Global Mirror example before unplanned production site failure
For this example, we use the configuration that we set up in 24.1 “Set up a Global Mirror
environment using the DS CLI” on page 304. Figure 24-2 shows the configuration during
normal operations.
Summary of the recovery scenario
The typical recovery scenario after the production site failure is:
1.
2.
3.
4.
5.
6.
Stop Global Mirror processing.
Perform Global Copy Failover from B to A.
Verify for valid Consistency Group state.
Create consistent data on B volumes (Reverse FlashCopy from B to C).
Reestablish the FlashCopy relationship from B to C.
Restart the application at the remote site.
The following sections discuss each of these tasks of the recovery scenario.
24.4.1 Stop Global Mirror processing
Depending on the state of the Global Mirror local disk subsystem where the Master was
running, you might be able to stop the Global Mirror session. The rmgmir command stops
Global Mirror processing. You give this command to the DS SMC connected to the local
DS6000 (DS6000#1); see Example 24-23.
Example 24-23 Terminate Global Mirror
dscli> rmgmir -quiet -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 2:12:06 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00165I rmgmir: Global Mirror for session 02 successfully stopped.
324
IBM System Storage DS6000 Series: Copy Services in Open Environments
24.4.2 Perform Global Copy Failover from B to A
Application server and
DS CLI client
Application Backup server and
DS CLI client
failoverpprc B to A
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1501
A
GC: Source
Copy Pending
1C00
1C01
1A01
LSS1B
Global Copy Pair
FlashCopy LSS1C
1B00
FlashCopy
LSS1D
1D00
1B01
1D01
B
GC: Source
Suspended
FC: Source
C
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-3 Site swap scenario after failoverpprc
A Failover (Copy Services Failover function) on the Global Copy target B volumes turns these
volumes into source volumes and also suspends them immediately. You can use the
failoverpprc command to do this.
This Failover operation sets the stage for change recording when application updates start
changing the B volumes. Change recording in turn allows you to re-synchronize just the
changes from the B to the A volumes, later before returning to the local site. But at this stage,
the B volumes do not contain consistent data and are still useless. We just changed their
Global Copy state from target to source and suspended. Figure 24-3 shows the DS6000
environment after the Failover operation.
Example 24-24 shows the command for this operation. You can check the result with the
lspprc command.
Example 24-24 failoverpprc command example
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:15:24 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Copy Pending Global Copy 14
unknown
Disabled
Invalid
1401:1A01 Target Copy Pending Global Copy 14
unknown
Disabled
Invalid
1500:1B00 Target Copy Pending Global Copy 15
unknown
Disabled
Invalid
1501:1B01 Target Copy Pending Global Copy 15
unknown
Disabled
Invalid
dscli>
dscli> failoverpprc -remotedev IBM.1750-1300819 -type gcp 1a00-1a01:1400-1401 1b00-1b01:1500-1501
Date/Time: November 22, 2005 2:16:55 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1A00:1400 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1A01:1401 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1B00:1500 successfully reversed.
Chapter 24. Global Mirror examples
325
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1B01:1501 successfully reversed.
dscli>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:17:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1A00:1400 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1A01:1401 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1B00:1500 Suspended Host Source Global Copy 1B
unknown
Disabled
True
1B01:1501 Suspended Host Source Global Copy 1B
unknown
Disabled
True
24.4.3 Verify for valid Consistency Group state
Now, you have to investigate whether all FlashCopy relationships are in a consistent state.
This means you have to query all FlashCopy relationships between B and C, which are part of
the Consistency Group, to determine the state of the FlashCopy relationship. Global Mirror
might have been in the middle of forming a Consistency Group, and FlashCopy might not
have completed the creation of a complete set of consistent C volumes.
Each FlashCopy pair needs a FlashCopy query to identify its state. You use the lsflash
command to check the SequenceNum and the Revertible field status. Example 24-25 shows
that the Revertible status of all of the FlashCopy pairs is Disabled (that is, non-revertible) and
the SequenceNums of all relationships are equal. Therefore, you do not need to take any
action in this case.
You can find a detailed discussion of this verification process in 21.5.4 “Verify for valid
Consistency Group state” on page 276.
Example 24-25 Verify the FlashCopy state
dscli> lsflash
1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:21:03 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
1A00:1C00 1A
43821E05
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1A01:1C01 1A
43821E05
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B00:1D00 1B
43821E05
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B01:1D01 1B
43821E05
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
With this action, you have verified that all FlashCopy pairs are non-revertible and all
sequence numbers are equal, so now you can proceed to the next step.
24.4.4 Reverse FlashCopy from B to C
At this point, only the C volumes (logical data) comprise a set of consistent data volumes,
although the physical data of the C volumes can be spread over the physical B and C volumes.
The B volumes (logical data) do not provide consistent data volumes, because Global Copy
does not provide data consistency.
We want to have to two good copies of the data at the recovery site. The aim is to have a
consistent set of volumes to work with, still keeping a good copy to which we can resort if
needed. The next step is then to create the same consistency on the B volumes as we have
on the C volumes; see Figure 24-4 on page 327. This can be achieved with the reverseflash
-fast command. This operation is called Fast Reverse Restore (FRR). You have to use the
-tgtpprc parameter with the reverseflash -fast command, because the B volume is also a
Global Copy source at this step.
326
IBM System Storage DS6000 Series: Copy Services in Open Environments
Note: Though the Fast Reverse Restore operation starts a background copy from the C to
the B volumes, in the reverseflash command you have to specify the B volumes as the
FlashCopy sources and the C volumes as the FlashCopy targets.
Application server and
DS CLI client
Application Backup server and
DS CLI client
reverseflash B to C
LSS14
1400
1401
LSS1A Background LSS1C
Copy
1A00
Global Copy Pair
1C01
1A01
Background
LSS1B
LSS1D
Copy
LSS15
1500
1C00
1B00
Global Copy Pair
1501
1D00
1B01
A
1D01
B
GC: Source
Copy Pending
GC: Source
Suspended
FC: Target
C
FC: Source
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-4 Site swap scenario after the reverseflash command was issued
Figure 24-4 shows the remote DS6000 environment after the reverseflash command was
issued. After executing this command and before the C to B background copy is completed,
the C volumes become the FlashCopy source and the B volumes become the FlashCopy
target.
Example 24-26 shows the results of the reverseflash command. The lsflash command
shows volume 1C00 (C volume) as the FlashCopy source.
Example 24-26 The reverseflash B to C
dscli> reverseflash -fast -tgtpprc 1a00-1a01:1c00-1c01 1b00-1b01:1d00-1d01
Date/Time:
CMUC00169I
CMUC00169I
CMUC00169I
CMUC00169I
dscli>
November 22, 2005 9:52:36 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
reverseflash: FlashCopy volume pair 1A00:1C00 successfully reversed.
reverseflash: FlashCopy volume pair 1A01:1C01 successfully reversed.
reverseflash: FlashCopy volume pair 1B00:1D00 successfully reversed.
reverseflash: FlashCopy volume pair 1B01:1D01 successfully reversed.
dscli> lsflash -l 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 9:52:49 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy OutOfSyncTracks DateCreated
================================================================================================================================================================
1C00:1A00 1C
4383317E
300
Enabled
Disabled Disabled
Disabled
Enabled
Enabled
Enabled
5533
Tue Nov 22
1C01:1A01 1C
4383317E
300
Enabled
Disabled Disabled
Disabled
Enabled
Enabled
Enabled
6398
Tue Nov 22
1D00:1B00 1D
4383317E
300
Enabled
Disabled Disabled
Disabled
Enabled
Enabled
Enabled
6331
Tue Nov 22
1D01:1B01 1D
4383317E
300
Enabled
Disabled Disabled
Disabled
Enabled
Enabled
Enabled
9067
Tue Nov 22
dscli>
dscli> lsflash 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 9:52:59 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
1C00:1A00 1C
4383317E
300
Enabled
Disabled Disabled Disabled Enabled
Enabled
Enabled
1C01:1A01 1C
4383317E
300
Enabled
Disabled Disabled Disabled Enabled
Enabled
Enabled
Chapter 24. Global Mirror examples
327
1D00:1B00 1D
1D01:1B01 1D
4383317E
4383317E
300
300
Enabled
Enabled
Disabled
Disabled
Disabled
Disabled
Disabled
Disabled
Enabled
Enabled
Enabled
Enabled
Enabled
Enabled
The above Fast Reverse Restore (FRR) operation does a background copy of all tracks that
changed on the B volumes since the last CG formation. This results in the B volumes
becoming equal to the image that was present on the C volumes. This is the logical view.
From the physical data placement point of view, the C volumes do not have meaningful data
after the FlashCopy relationship ends.
Application server and
DS CLI client
Application Backup server and
DS CLI client
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1C00
1A01
1C01
LSS1B
Global Copy Pair
1B00
1501
LSS1D
1D00
1B01
A
GC: Source
Copy Pending
LSS1C
1D01
B
GC: Source
Suspended
FC: None
C
FC: None
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-5 Site swap scenario after the completion of the background copy
Because you do not specify the -persist parameter, the FlashCopy relationship ends after
the background copy from C to B completes.
You have to wait until all Fast Reverse Restore operations and their background copy
complete successfully before you proceed with the next step. Again, when the background
copy completes, the FlashCopy relationship ends; see Figure 24-5.
Therefore, you should check if the FlashCopy relationships remain to determine when all Fast
Reverse Restore operations have completed; see Example 24-27. This example shows the
result of the lsflash command after the reverseflash background copy completes.
Example 24-27 The lsflash command to confirm the completion of the background copy
dscli> lsflash
1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:26:43 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lsflash: No Flash Copy found.
24.4.5 Reestablish the FlashCopy relationships from B to C
In this step, you create the former FlashCopy relationship between the B and C volumes, as
they were at the beginning when you set up the Global Mirror environment; see Figure 24-6
on page 329. This step is in preparation for returning later to production at the local site.
328
IBM System Storage DS6000 Series: Copy Services in Open Environments
Application server and
DS CLI client
Application Backup server and
DS CLI client
mkflash B to C
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1501
1C00
1C01
1A01
LSS1B
Global Copy Pair
FlashCopy LSS1C
1B00
FlashCopy
LSS1D
1D00
1D01
1B01
A
B
C
GC: Source
Copy Pending
GC: Source
Suspended
FC: Source
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-6 Site swap scenario after mkflash B to C
The mkflash command used in this step is illustrated in Example 24-28 on page 330, and is
exactly the same FlashCopy command you might have used when you initially created the
Global Mirror environment. See 22.3.4, “Introduce FlashCopy” on page 265.
In a disaster situation, you might not want to use the -nocp option for the FlashCopy from B to
C. This removes the FlashCopy I/O overhead when the application starts.
Chapter 24. Global Mirror examples
329
Example 24-28 Re-establish the FlashCopy relationships from B to C
dscli> mkflash -tgtinhibit -nocp -record 1a00-1a01:1c00-1c01 1b00-1b01:1d00-1d01
Date/Time:
CMUC00137I
CMUC00137I
CMUC00137I
CMUC00137I
dscli>
November
mkflash:
mkflash:
mkflash:
mkflash:
dscli> lsflash
22, 2005 2:30:16 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
FlashCopy pair 1A00:1C00 successfully created.
FlashCopy pair 1A01:1C01 successfully created.
FlashCopy pair 1B00:1D00 successfully created.
FlashCopy pair 1B01:1D01 successfully created.
1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:30:26 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
1A00:1C00 1A
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1A01:1C01 1A
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B00:1D00 1B
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B01:1D01 1B
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
24.4.6 Restart the application at the remote site
Depending on your operating system, it might be necessary to rescan Fibre Channel devices
(to remove device objects for the recovery site volumes and recognize the new sources) and
mount the B volumes. And then start all applications at the recovery site.
Now that the application has started at the remote site, all the write I/Os to the new source
volumes, which are the B volumes, are tracked in the bitmaps by the Failover function.
Figure 24-7 shows this environment.
Application server and
DS CLI client
Application Backup server and
DS CLI client
Start Application
I/O
LSS14
1400
1401
LSS1A
1A00
Global Copy Pair
LSS1B
1B00
Global Copy Pair
1501
FlashCopy
LSS1D
1D00
1D01
B
GC: Source
Suspended
FC: Source
C
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-7 Site swap scenario after the application start
330
1C01
1B01
A
GC: Source
Copy Pending
1C00
1A01
LSS15
1500
FlashCopy LSS1C
IBM System Storage DS6000 Series: Copy Services in Open Environments
24.5 Return to the local site
The return to the normal production site typically follows this scenario:
1. Create paths from B to A.
2. Perform Global Copy Failback from B to A.
3. Query for the Global Copy first pass completion.
4. Quiesce the application at the remote site.
5. Query the Out Of Sync Tracks until it shows zero.
6. Create paths from A to B if they do not exist.
7. Perform Global Copy Failover from A to B.
8. Perform Global Copy Failback from A to B.
9. Start Global Mirror.
10.Start the application at the local site.
The following sections discuss each of the preceding steps.
24.5.1 Create paths from B to A
The local site is operational again. If the local site did not lose the data at the time that the
swap to the remote site occurred, then it is possible to re-synchronize the changed data from
B to A in preparation for returning to the local site.
Before doing this Failback process, we need paths to be defined from B to A. For this task, you
use the lsavailpprcport, mkpprcpath, and lspprcpath commands; see Example 24-29. You
give these commands to the DS SMC connected to the remote DS6000#2.
Example 24-29 Create paths from B to A
dscli> lsavailpprcport -l -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F 1a:14
Date/Time: November 22, 2005 2:32:28 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0103
I0102
FCP NA
NA
dscli>
dscli> lsavailpprcport -l -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F 1b:15
Date/Time: November 22, 2005 2:32:41 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Local Port Attached Port Type Switch ID Switch Port
===================================================
I0003
I0003
FCP NA
NA
I0103
I0102
FCP NA
NA
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F -srclss 1a -tgtlss 14 i0003:i0003
i0103:i0102
Date/Time: November 22, 2005 2:33:03 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00149I mkpprcpath: Remote Mirror and Copy path 1a:14 successfully established.
dscli>
dscli> mkpprcpath -remotedev IBM.1750-1300819 -remotewwnn 500507630EFE028F -srclss 1b -tgtlss 15 i0003:i0003
i0103:i0102
Date/Time: November 22, 2005 2:33:14 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00149I mkpprcpath: Remote Mirror and Copy path 1b:15 successfully established.
dscli>
dscli> lspprcpath -fullid 1a-1b
Date/Time: November 22, 2005 2:33:35 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Src
Tgt
State
SS
Port
Attached Port
Tgt WWNN
===================================================================================================================
IBM.1750-1300247/1A IBM.1750-1300819/14 Success FF14 IBM.1750-1300247/I0003 IBM.1750-1300819/I0003 500507630EFE028F
IBM.1750-1300247/1A IBM.1750-1300819/14 Success FF14 IBM.1750-1300247/I0103 IBM.1750-1300819/I0102 500507630EFE028F
IBM.1750-1300247/1B IBM.1750-1300819/15 Success FF15 IBM.1750-1300247/I0003 IBM.1750-1300819/I0003 500507630EFE028F
Chapter 24. Global Mirror examples
331
IBM.1750-1300247/1B IBM.1750-1300819/15 Success FF15 IBM.1750-1300247/I0103 IBM.1750-1300819/I0102 500507630EFE028F
24.5.2 Perform Global Copy Failback from B to A
Application server and
DS CLI client
Application Backup server and
DS CLI client
failbackpprc B to A
LSS14
LSS1A
1400
Global Copy Pair
1401
LSS15
1A00
Global Copy Pair
1C00
1A01
LSS1B
Changed data sent from B to A
1500
FlashCopy LSS1C
1C01
FlashCopy
1B00
LSS1D
1D00
1B01
1501
A
1D01
B
GC: Target
Copy Pending
GC: Source
Copy Pending
FC: Source
C
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-8 Site swap scenario after Global Copy Failback from B to A
After defining the paths from B to A, you use the failbackpprc command to re-synchronize
the changed data from B to A. This process changes the A volume from its previous state
(source) copy pending to target copy pending; see Figure 24-8.
The failbackpprc command is issued to the B volume as the source and the A volume as the
target. You have to add the -type gcp parameter with the failbackpprc command to request
Global Copy mode; see Example 24-30.
Example 24-30 Perform Global Copy Failback from B to A
<< Before the failbackpprc B to A >>
<< B volume status >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:35:31 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1A00:1400 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1A01:1401 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1B00:1500 Suspended Host Source Global Copy 1B
unknown
Disabled
True
1B01:1501 Suspended Host Source Global Copy 1B
unknown
Disabled
True
<< The failbackpprc B to A >>
dscli> failbackpprc -remotedev IBM.1750-1300819
Date/Time: November 22, 2005 2:36:56 AM JST IBM
CMUC00197I failbackpprc: Remote Mirror and Copy
CMUC00197I failbackpprc: Remote Mirror and Copy
CMUC00197I failbackpprc: Remote Mirror and Copy
CMUC00197I failbackpprc: Remote Mirror and Copy
332
-type gcp 1a00-1a01:1400-1401 1b00-1b01:1500-1501
DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
pair 1A00:1400 successfully failed back.
pair 1A01:1401 successfully failed back.
pair 1B00:1500 successfully failed back.
pair 1B01:1501 successfully failed back.
IBM System Storage DS6000 Series: Copy Services in Open Environments
<< B volume state >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:37:29 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1A00:1400 Copy Pending Global Copy 1A
unknown
Disabled
True
1A01:1401 Copy Pending Global Copy 1A
unknown
Disabled
True
1B00:1500 Copy Pending Global Copy 1B
unknown
Disabled
True
1B01:1501 Copy Pending Global Copy 1B
unknown
Disabled
True
<< A volume state >>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 2:38:10 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1A00:1400 Target Copy Pending Global Copy 1A
unknown
Disabled
Invalid
1A01:1401 Target Copy Pending Global Copy 1A
unknown
Disabled
Invalid
1B00:1500 Target Copy Pending Global Copy 1B
unknown
Disabled
Invalid
1B01:1501 Target Copy Pending Global Copy 1B
unknown
Disabled
Invalid
Example 24-30 on page 332 shows the commands given in our example. First, we listed the
status of the B volumes and then performed the Global Copy Failback operation. The
command is given to the DS SMC connected to the remote DS6000#2.
The failbackpprc initialization mode re-synchronizes the volumes in this manner:
򐂰 If a volume at the production site is in simplex state (no relationship), all of the data for
that volume is sent from the recovery site to the production site.
򐂰 If a volume at the production site is in the copy pending or suspended state and without
changed tracks, only the modified data on the volume at the recovery site is sent to the
volume at the production site.
򐂰 If a volume at the production site is in a suspended state and has tracks on which data has
been written, the volume at the recovery site discovers which tracks were modified on any
site and sends both the tracks changed on the production site and the tracks marked at
the recovery site.
The volume at the production site becomes a write-inhibited target volume. This action is
performed on an individual volume basis.
Notes on the failbackpprc command
If the server at the production site is still online and accessing the disk, or a crash happens,
so that the SCSI persistent reserve is still set on the previous source disk, the failbackpprc
command fails. In this case, the server at the production site locks the target with a SCSI
persistent reserve. After this SCSI persistent reserve is reset with the varyoffvg command
(in this case, on AIX), the failbackpprc command completes successfully.
There is a -resetreserve parameter for the failbackpprc command. This option resets the
reserved state so that the Failback operation can complete. In the Failback operation after a
real disaster, you might use this parameter because the server might go down while the SCSI
persistent reserve was set on the A volume. If the server at the production site is operational,
for example, when a Global Mirror Failback operation test is performed, you must not use this
parameter because the server at the local site still owns the A volume and might be using it
and the Failback operation suddenly changes the contents of the volume. This can corrupt
the server file system.
Chapter 24. Global Mirror examples
333
24.5.3 Query for the Global Copy first pass completion
The first pass of Global Copy is the first phase of the re-synchronization process, when all the
data that has changed while the B volumes were suspended is copied to the A volumes. As
long as the first pass copy process continues, the Out Of Sync Tracks does not show zero.
Therefore, depending on your Failback scenario, you can continue to run the application at
the remote site until the Global Copy first pass process completes.
You can query this status with the lspprc command; see Example 24-31. The First Pass
Status field indicates the status of the first pass, where True means that the first pass has
completed.
Example 24-31 Query for the Global Copy first pass completion
dscli> lspprc -l 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:41:16 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs) Critical Mode First Pass
Status
===============================================================================================================================================================
1A00:1400 Copy Pending Global Copy 0
Disabled Disabled
invalid
1A
unknown
Disabled
True
1A01:1401 Copy Pending Global Copy 0
Disabled Disabled
invalid
1A
unknown
Disabled
True
1B00:1500 Copy Pending Global Copy 0
Disabled Disabled
invalid
1B
unknown
Disabled
True
1B01:1501 Copy Pending Global Copy 0
Disabled Disabled
invalid
1B
unknown
Disabled
True
dscli>
24.5.4 Quiesce the application at the remote site
Before returning to normal operation on the local site, the application (still updating B volumes
in the recovery site) must be quiesced to cease all write I/O from updating the B volumes.
Depending on the host operating system, it might be necessary to dismount the B volumes.
24.5.5 Query the Out Of Sync Tracks until it shows zero
After quiescing the application, you wait until the Out Of Sync Tracks for the Global Copy
pairs shows zero to ensure that all the data has been written to the B volumes. You can check
this status with the lspprc -l command; see Example 24-32. You have to issue this
command to the DS SMC connected to the remote DS6000 (DS6000#2).
Example 24-32 Query the Global Copy Out Of Sync Tracks until it shows zero
dscli> lspprc -l 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:43:07 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs) Critical Mode First Pass
Status
================================================================================================================================================================
1A00:1400 Copy Pending Global Copy 0
Disabled Disabled
invalid
1A
unknown
Disabled
True
1A01:1401 Copy Pending Global Copy 0
Disabled Disabled
invalid
1A
unknown
Disabled
True
1B00:1500 Copy Pending Global Copy 0
Disabled Disabled
invalid
1B
unknown
Disabled
True
1B01:1501 Copy Pending Global Copy 0
Disabled Disabled
invalid
1B
unknown
Disabled
True
24.5.6 Create paths from A to B if they do not exist
Most probably there are no paths from A to B at this point. You can check the current paths
status with the lspprcpath command. We recommend that you run this command with the
-fullid command flag so you get fully qualified IDs in the output report. The fully qualified ID
information is helpful when trying to identify whether you have paths between the correct
DS6000s (DS6000#1 to DS6000#2 in this example). You have to run this command on the DS
SMC connected to the local DS6000 (DS6000#1).
334
IBM System Storage DS6000 Series: Copy Services in Open Environments
In our example, the paths were still available; see Example 24-33. If there were no available
paths, then you must define them now using the mkpprcpath command. Example 24-1 on
page 305 shows how to do this task.
Example 24-33 Check available paths from A to B
dscli> lspprcpath -fullid 14-15
Date/Time: November 22, 2005 2:44:28 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Src
Tgt
State SS Port
Attached Port
Tgt WWNN
===================================================================================================================
IBM.1750-1300819/14 IBM.1750-1300247/1A Success FF1A IBM.1750-1300819/I0003 IBM.1750-1300247/I0003 500507630EFFFE16
IBM.1750-1300819/14 IBM.1750-1300247/1A Success FF1A IBM.1750-1300819/I0102 IBM.1750-1300247/I0103 500507630EFFFE16
IBM.1750-1300819/15 IBM.1750-1300247/1B Success FF1B IBM.1750-1300819/I0003 IBM.1750-1300247/I0003 500507630EFFFE16
IBM.1750-1300819/15 IBM.1750-1300247/1B Success FF1B IBM.1750-1300819/I0102 IBM.1750-1300247/I0103 500507630EFFFE16
24.5.7 Perform Global Copy Failover from A to B
In order to return to the original configuration, we have to return the A volumes to their original
Global Copy (source) copy pending volume state. This is a two step procedure. First, the
failoverpprc command converts the state of the A volumes from target copy pending to
(source) suspended. The state of the B volumes is preserved; see Figure 24-9.
Application server and
DS CLI client
Application Backup server and
DS CLI client
failoverpprc A to B
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1C01
FlashCopy
1B00
1501
LSS1D
1D00
1D01
1B01
A
GC: Source
Suspended
1C00
1A01
LSS1B
Global Copy Pair
FlashCopy LSS1C
B
C
GC: Source
Copy Pending
FC: Source
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-9 Site swap scenario after Global Copy Failover from A to B
Example 24-34 shows the result of the failoverpprc command we used in our example, and
the volume state after this command is issued. You have to give this command to the DS SMC
connected to the local DS6000 (DS6000#1).
Example 24-34 Global Copy Failover from A to B
<< DS6000 #1 >>
<< Before the failover >>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 2:47:59 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
Chapter 24. Global Mirror examples
335
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1A00:1400 Target Copy Pending Global Copy 1A
unknown
Disabled
Invalid
1A01:1401 Target Copy Pending Global Copy 1A
unknown
Disabled
Invalid
1B00:1500 Target Copy Pending Global Copy 1B
unknown
Disabled
Invalid
1B01:1501 Target Copy Pending Global Copy 1B
unknown
Disabled
Invalid
<< After the failover >>
dscli> failoverpprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 22, 2005 2:50:09 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1400:1A00 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1401:1A01 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1500:1B00 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1501:1B01 successfully reversed.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 2:50:29 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1400:1A00 Suspended Host Source Global Copy 14
unknown
Disabled
True
1401:1A01 Suspended Host Source Global Copy 14
unknown
Disabled
True
1500:1B00 Suspended Host Source Global Copy 15
unknown
Disabled
True
1501:1B01 Suspended Host Source Global Copy 15
unknown
Disabled
True
<< DS6000 #2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:51:20 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1A00:1400 Copy Pending Global Copy 1A
unknown
Disabled
True
1A01:1401 Copy Pending Global Copy 1A
unknown
Disabled
True
1B00:1500 Copy Pending Global Copy 1B
unknown
Disabled
True
1B01:1501 Copy Pending Global Copy 1B
unknown
Disabled
True
336
IBM System Storage DS6000 Series: Copy Services in Open Environments
24.5.8 Perform Global Copy Failback from A to B
Application server and
DS CLI client
Application Backup server and
DS CLI client
failbackpprc A to B
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1501
A
GC: Source
Copy Pending
1C00
1A01
LSS1B
Global Copy Pair
FlashCopy LSS1C
1C01
FlashCopy
1B00
LSS1D
1D00
1B01
1D01
B
GC: Target
Copy Pending
FC: Source
C
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-10 Site swap scenario after failbackpprc A to B
Next, we return the Global Copy pairs to the original configuration with the failbackpprc
command. Figure 24-10 shows the configuration after this command is executed.
Example 24-35 shows the result of the failbackpprc command used in our example, and the
volume state after this command is issued. You have to give this command to the DS SMC
connected to the local DS6000 (DS6000#1).
Example 24-35 Global Copy Failback from A to B
<< DS6000 #1 >>
dscli> failbackpprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 22, 2005 2:52:29 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1400:1A00 successfully failed back.
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1401:1A01 successfully failed back.
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1500:1B00 successfully failed back.
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1501:1B01 successfully failed back.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 2:52:37 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
<< DS6000 #2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 2:52:41 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=========================================================================================================
1400:1A00 Target Copy Pending Global Copy 14
unknown
Disabled
Invalid
Chapter 24. Global Mirror examples
337
1401:1A01 Target Copy Pending 1500:1B00 Target Copy Pending 1501:1B01 Target Copy Pending -
Global Copy 14
Global Copy 15
Global Copy 15
unknown
unknown
unknown
Disabled
Disabled
Disabled
Invalid
Invalid
Invalid
24.5.9 Start Global Mirror
Application server and
DS CLI client
Application Backup server and
DS CLI client
mkgmir
LSS14
GM Master
Started
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1C00
1A01
LSS1B
Global Copy Pair
FlashCopy LSS1C
1C01
FlashCopy
1B00
1501
LSS1D
1D00
1B01
1D01
A
B
C
GC: Source
Copy Pending
GC: Target
Copy Pending
FC: Source
FC: Target
DS6000#1
-dev IBM.1750-1300819
DS6000#2
-dev IBM.1750-1300247
Figure 24-11 Start Global Mirror
The last step before starting the application at the production site is to start the Global Mirror
session again; see Figure 24-11. If you did not already create the FlashCopy relationships
from B to C volumes, then you have to do it before starting the Global Mirror.
To start the Global Mirror session, use the mkgmir command. Before starting Global Mirror,
you can check the status for the Global Mirror session on each LSS with the lssession
command. After starting Global Mirror, you can use the showgmir command to check its
status.
Example 24-36 shows the commands used in our example, and the corresponding results.
You give this command to the DS SMC connected to the local DS6000 (DS6000#1).
Example 24-36 Start Global Mirror
dscli> lssession 14-15
Date/Time: November 22, 2005 2:54:55 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete AllowCascading
=================================================================================================================
14
02
Normal 1400
Active
Primary Copy Pending Secondary Simplex True
Disable
14
02
Normal 1401
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1500
Active
Primary Copy Pending Secondary Simplex True
Disable
15
02
Normal 1501
Active
Primary Copy Pending Secondary Simplex True
Disable
dscli>
dscli> mkgmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 2:55:31 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00162I mkgmir: Global Mirror for session 02 successfully started.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 2:55:54 AM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
338
IBM System Storage DS6000 Series: Copy Services in Open Environments
Master Count
Master Session ID
Copy State
Fatal Reason
CG Interval (seconds)
XDC Interval(milliseconds)
CG Drain Time (seconds)
Current Time
CG Time
Successful CG Percentage
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
1
0x02
Running
Not Fatal
0
50
30
11/22/2005 05:04:32 JST
11/22/2005 05:04:31 JST
100
0x4382284F
IBM.1750-1300819
0
-
24.5.10 Start the application at the local site
Now we have an environment in which to start the application at the original local site.
Depending on your operating system, it might be necessary to rescan Fibre Channel devices
and mount the new source volumes (A volumes) at the local site. Start all applications and
check for consistency; see Figure 24-12.
Depending on your path design, delete the paths from the recovery to the production LSSs.
Application server and
DS CLI client
Application Backup server and
DS CLI client
Start Application
I/O
LSS14
GM Master
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1A00
1501
1C00
1A01
LSS1B
Global Copy Pair
FlashCopy LSS1C
1C01
FlashCopy
1B00
LSS1D
1D00
1D01
1B01
A
B
C
GC: Source
Copy Pending
GC: Target
Copy Pending
FC: Source
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-12 Start the application at the local site
Chapter 24. Global Mirror examples
339
24.6 Practice disaster recovery readiness
In this section, we discuss how to practice your disaster recovery readiness without stopping
the application at the production site. You can use the same procedure to make a test or
make a regular backup copy at the remote site.
The typical scenario for this activity is the following:
1. Query the Global Mirror environment to have a look at the situation.
2. Pause Global Mirror and check its completion.
3. Pause Global Copy pairs.
4. Perform Global Copy Failover from B to A.
5. Create consistent data on B volumes (Reverse FlashCopy from B to C).
6. Wait for the FlashCopy background copy to complete.
7. Reestablish FlashCopy pairs B to C with original Global Mirror options.
8. Take FlashCopy from B to D.
9. Perform the disaster recovery testing using the D volume.
10.Perform Global Copy Failback from A to B.
11.Resume Global Mirror.
Many steps in the above scenario are the same that we discussed in sections 24.4 “Recovery
scenario after local site failure with the DS CLI” on page 323 and 24.5 “Return to the local
site” on page 331. For steps that are the same, we refer you to the previous sections that
describe them.
24.6.1 Query the Global Mirror environment to look at the situation
There are several commands that you can use to look at the situation.
Query the Global Copy status
The lspprcpath and lspprc commands; see 24.1.4 “Create Global Copy relationships: A to B
volumes” on page 305.
Query the FlashCopy status
The lsremoteflash (or lsflash) command; see 24.1.5 “Create FlashCopy relationships: B to
C volumes” on page 306.
Query the Global Mirror status
The lssession, showgmir, showgmir -metrics, and showgmiroos commands; see 24.1.6
“Start Global Mirror” on page 307.
24.6.2 Pause Global Mirror and check its completion
Example 24-37 shows how to perform this task. For detailed discussion and considerations,
see Figure 24.3.1 on page 316.
You give this command to the DS SMC connected to the local disk subsystem.
Example 24-37 Pause Global Mirror
dscli> pausegmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 7:06:33 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00163I pausegmir: Global Mirror for session 02 successfully paused.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 7:06:43 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
340
IBM System Storage DS6000 Series: Copy Services in Open Environments
ID
Master Count
Master Session ID
Copy State
Fatal Reason
CG Interval (seconds)
XDC Interval(milliseconds)
CG Drain Time (seconds)
Current Time
CG Time
Successful CG Percentage
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
IBM.1750-1300819/14
1
0x02
Paused
Not Fatal
0
50
30
11/22/2005 21:15:13 JST
11/22/2005 21:15:07 JST
100
0x43830BCB
IBM.1750-1300819
0
-
24.6.3 Pause Global Copy pairs
The pausepprc command suspends the Global Copy pairs; see Example 24-38. You give this
command to the DS SMC connected to the local disk subsystem.
Example 24-38 Pause Global Copy pairs
<< DS6000 #1 >>
dscli> pausepprc -remotedev IBM.1750-1300247 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 22, 2005 7:09:35 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1400:1A00 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1401:1A01 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1500:1B00 relationship successfully paused.
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1501:1B01 relationship successfully paused.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 7:09:41 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1400:1A00 Suspended Host Source Global Copy 14
unknown
Disabled
True
1401:1A01 Suspended Host Source Global Copy 14
unknown
Disabled
True
1500:1B00 Suspended Host Source Global Copy 15
unknown
Disabled
True
1501:1B01 Suspended Host Source Global Copy 15
unknown
Disabled
True
<< DS6000 #2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 7:10:42 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
=============================================================================================================
1400:1A00 Target Suspended Update Target Global Copy 14
unknown
Disabled
Invalid
1401:1A01 Target Suspended Update Target Global Copy 14
unknown
Disabled
Invalid
1500:1B00 Target Suspended Update Target Global Copy 15
unknown
Disabled
Invalid
1501:1B01 Target Suspended Update Target Global Copy 15
unknown
Disabled
Invalid
24.6.4 Perform Global Copy Failover from B to A
Example 24-39 on page 342 shows how to perform this task. For detailed discussion and
considerations, see 24.4.2 “Perform Global Copy Failover from B to A” on page 325. You give
this command to the DS SMC connected to the remote disk subsystem.
Chapter 24. Global Mirror examples
341
Example 24-39 Perform Global Copy Failover from B to A
<< DS6000 #2 >>
dscli> failoverpprc -remotedev IBM.1750-1300819 -type gcp 1a00-1a01:1400-1401 1b00-1b01:1500-1501
Date/Time: November 22, 2005 7:12:05 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1A00:1400 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1A01:1401 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1B00:1500 successfully reversed.
CMUC00196I failoverpprc: Remote Mirror and Copy pair 1B01:1501 successfully reversed.
dscli>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 7:12:12 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1A00:1400 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1A01:1401 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1B00:1500 Suspended Host Source Global Copy 1B
unknown
Disabled
True
1B01:1501 Suspended Host Source Global Copy 1B
unknown
Disabled
True
24.6.5 Create consistent data on B volumes
Example 24-40 shows how to perform this task. For detailed discussion and considerations,
see 24.4.4 “Reverse FlashCopy from B to C” on page 326. You give the reverseflash
command to the DS SMC connected to the remote disk subsystem.
Example 24-40 Reverse FlashCopy B to C
dscli> reverseflash -fast -tgtpprc 1a00-1a01:1c00-1c01 1b00-1b01:1d00-1d01
Date/Time: November 22, 2005 7:17:33 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00169I reverseflash: FlashCopy volume pair 1A00:1C00 successfully reversed.
CMUC00169I reverseflash: FlashCopy volume pair 1A01:1C01 successfully reversed.
CMUC00169I reverseflash: FlashCopy volume pair 1B00:1D00 successfully reversed.
CMUC00169I reverseflash: FlashCopy volume pair 1B01:1D01 successfully reversed.
24.6.6 Wait for the FlashCopy background copy to complete
After the FlashCopy background copy completes, the FlashCopy relationship ends. You can
check this with the lsflash command. See Example 24-41.
Example 24-41 Check the FlashCopy background copy completion
dscli> lsflash 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 7:17:40 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lsflash: No Flash Copy found.
24.6.7 Reestablish the FlashCopy relationships
In order to resume the Global Mirror environment quickly, we reestablish the FlashCopy
relationships from B to C with the original options for the Global Mirror environment; see
Example 24-42. For detailed discussion and considerations, see 24.4.5 “Reestablish the
FlashCopy relationships from B to C” on page 328.
You give this command to the DS SMC connected to the remote disk subsystem.
Example 24-42 Re-establish the FlashCopy relationship
dscli> mkflash -tgtinhibit -nocp -record 1a00-1a01:1c00-1c01 1b00-1b01:1d00-1d01
Date/Time: November 22, 2005 7:20:53 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
342
IBM System Storage DS6000 Series: Copy Services in Open Environments
CMUC00137I mkflash: FlashCopy pair
CMUC00137I mkflash: FlashCopy pair
CMUC00137I mkflash: FlashCopy pair
CMUC00137I mkflash: FlashCopy pair
dscli>
dscli> lsflash 1a00-1a01 1b00-1b01
1A00:1C00
1A01:1C01
1B00:1D00
1B01:1D01
successfully
successfully
successfully
successfully
created.
created.
created.
created.
Date/Time: November 22, 2005 7:21:01 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
1A00:1C00 1A
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1A01:1C01 1A
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B00:1D00 1B
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B01:1D01 1B
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
24.6.8 Take FlashCopy from B to D
In the previous step, we created a consistent copy of the data on the B volumes. Now, we
make another copy of the B volumes for the disaster recovery testing. This FlashCopy targets
the D volumes. In our example we have four D volumes, which are 1e00, 1e01, 1f00, and
1f01; see Figure 24-13.
Application server and
DS CLI client
Application Backup server and
DS CLI client
mkflash B to D
FlashCopy
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1501
A
GC: Source
Copy Pending
1C00
1A00
1A01
LSS1B
Global Copy Pair
LSS1C
1B00
1B01
B
GC: Source
Suspended
FC: Source
LSS1E
1E00
1C01
FlashCopy
1E01
LSS1D
1D00
LSS1F
1F00
1D01
1F01
C
D
FC: Target
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-13 After mkflash B to D
Example 24-43 shows the DS CLI log for this operation. Here, we use the -nocp option for the
FlashCopy. You can also use the copy option.
Example 24-43 Take FlashCopy B to D
dscli> lsflash 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 7:23:27 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWriteEnabled TargetWriteEnabled BackgroundCopy
====================================================================================================================================
1A00:1C00 1A
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1A01:1C01 1A
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
1B00:1D00 1B
0
300
Disabled Enabled
Enabled
Disabled Enabled
Disabled
Disabled
Chapter 24. Global Mirror examples
343
1B01:1D01 1B
dscli>
0
300
Disabled
Enabled
Enabled
Disabled
Enabled
Disabled
Disabled
dscli> mkflash -nocp 1a00-1a01:1e00-1e01 1b00-1b01:1f00-1f01
Date/Time:
CMUC00137I
CMUC00137I
CMUC00137I
CMUC00137I
November
mkflash:
mkflash:
mkflash:
mkflash:
22, 2005 7:23:44 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
FlashCopy pair 1A00:1E00 successfully created.
FlashCopy pair 1A01:1E01 successfully created.
FlashCopy pair 1B00:1F00 successfully created.
FlashCopy pair 1B01:1F01 successfully created.
24.6.9 Perform the disaster recovery testing using the D volumes
Depending on your operating system and system environment, it might be necessary to
rescan Fibre Channel devices and mount the D volume at the remote site. After this, you can
perform your disaster recovery testing using the D volumes. You can also use the D volumes
for backup or to make a tape backup.
24.6.10 Perform Global Copy Failback from A to B
In order to return to the normal Global Mirror environment, we have to resume the Global
Copy pairs that we had suspended in a previous step. Because the application at the
production site keeps running and we must not lose these updates, we have to
re-synchronize the A and the B volumes with the A volumes as the source and the B volumes
as the target. You give this command to the DS SMC connected to the local DS6000
(DS6000#1); see Figure 24-14.
Application server and
DS CLI client
Application Backup server and
DS CLI client
failbackpprc A to B
FlashCopy
LSS14
1400
1401
LSS1A
Global Copy Pair
LSS15
1500
1501
LSS1C
1C00
1A00
1A01
LSS1B
Global Copy Pair
LSS1E
1E00
1C01
FlashCopy
1E01
LSS1D
1D00
1B00
1B01
LSS1F
1F00
1D01
1F01
A
B
C
D
GC: Source
Copy Pending
GC: Target
Copy Pending
FC: Source
FC: Target
FC: Target
DS6000#1
DS6000#2
-dev IBM.1750-1300819
-dev IBM.1750-1300247
Figure 24-14 Perform Global Copy Failback from A to B - test scenario
For the Failback operation, we use the failbackpprc command; see Example 24-44.
Example 24-44 Perform Global Copy Failback from A to B - test scenario
<< Before failbackpprc >>
<< DS6000 #1 >>
dscli> lspprc 1400-1401 1500-1501
344
IBM System Storage DS6000 Series: Copy Services in Open Environments
Date/Time: November 22, 2005 7:31:53 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1400:1A00 Suspended Host Source Global Copy 14
unknown
Disabled
True
1401:1A01 Suspended Host Source Global Copy 14
unknown
Disabled
True
1500:1B00 Suspended Host Source Global Copy 15
unknown
Disabled
True
1501:1B01 Suspended Host Source Global Copy 15
unknown
Disabled
True
<< DS6000 #2 >>
dscli> lspprc 1a00-1a01 1b00-1b01
Date/Time: November 22, 2005 7:33:57 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
====================================================================================================
1A00:1400 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1A01:1401 Suspended Host Source Global Copy 1A
unknown
Disabled
True
1B00:1500 Suspended Host Source Global Copy 1B
unknown
Disabled
True
1B01:1501 Suspended Host Source Global Copy 1B
unknown
Disabled
True
<< DS6000 #1 >>
dscli> failbackpprc -remotedev IBM.1750-1300247 -type gcp 1400-1401:1a00-1a01 1500-1501:1b00-1b01
Date/Time: November 22, 2005 7:37:34 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1400:1A00 successfully failed back.
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1401:1A01 successfully failed back.
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1500:1B00 successfully failed back.
CMUC00197I failbackpprc: Remote Mirror and Copy pair 1501:1B01 successfully failed back.
dscli>
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 7:37:43 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
Important: Do not specify the B volume as a source with the failbackpprc command (to
the DS6000#2); otherwise, data on the B volume is copied to the A volume. If the A volume
does not have reserve status, data on the A volume might be overwritten.
24.6.11 Wait for the Global Copy first pass to complete
The Global Copy first pass does not need to complete to resume Global Mirror; however, the
Consistency Group formation does not start until this completion. You can check the status
with the lspprc command; see Example 24-45. The First Pass Status field indicates the
status of the first pass, where True means the first pass is complete. You can also use the
lssession command output field FirstPassComplete to verify the status of the first pass.
Example 24-45 Check the Global Copy first pass completion
dscli> lspprc 1400-1401 1500-1501
Date/Time: November 22, 2005 7:37:43 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1400:1A00 Copy Pending Global Copy 14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 15
unknown
Disabled
True
Chapter 24. Global Mirror examples
345
dscli> lspprc -l 1400-1401 1500-1501
Date/Time: November 22, 2005 7:38:46 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
State
Reason Type
Out Of Sync Tracks Tgt Read Src Cascade Tgt Cascade Date Suspended SourceLSS Timeout (secs) Critical Mode First Pass
Status
================================================================================================================================================================
=====
1400:1A00 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
unknown
Disabled
True
1401:1A01 Copy Pending Global Copy 0
Disabled Disabled
invalid
14
unknown
Disabled
True
1500:1B00 Copy Pending Global Copy 0
Disabled Disabled
invalid
15
unknown
Disabled
True
1501:1B01 Copy Pending Global Copy 0
Disabled Disabled
invalid
15
unknown
Disabled
True
24.6.12 Resume Global Mirror
Now, you can resume Global Mirror with the resumegmir command. You can verify the result
with the showgmir command; see Example 24-46. For detailed discussion and considerations,
see Figure 24.3.1 on page 316. You give this command to the DS SMC connected to the local
DS6000 (DS6000#1).
Example 24-46 Resume Global Mirror
dscli> resumegmir -dev IBM.1750-1300819 -lss IBM.1750-1300819/14 -session 02
Date/Time: November 22, 2005 7:47:29 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
CMUC00164I resumegmir: Global Mirror for session 02 successfully resumed.
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 7:51:20 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 21:59:50 JST
CG Time
11/22/2005 21:59:50 JST
Successful CG Percentage
99
FlashCopy Sequence Number 0x43831646
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
dscli>
dscli> showgmir -dev IBM.1750-1300819 IBM.1750-1300819/14
Date/Time: November 22, 2005 7:51:28 PM JST IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819
ID
IBM.1750-1300819/14
Master Count
1
Master Session ID
0x02
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/22/2005 21:59:58 JST
CG Time
11/22/2005 21:59:58 JST
Successful CG Percentage
99
FlashCopy Sequence Number 0x4383164E
Master ID
IBM.1750-1300819
Subordinate Count
0
Master/Subordinate Assoc
-
346
IBM System Storage DS6000 Series: Copy Services in Open Environments
In our example, the first showgmir output shows Running in the Copy State field, but the
Consistency Group formation is not complete yet. You can see this in the Current Time and
the CG Time fields. The next showgmir command shows that at least one Consistency Group
formation has been completed after the showgmir command.
24.7 DS Storage Manager GUI examples
In this section, we explain how to create and manage a Global Mirror session using the DS
Storage Manager (DS SM) graphical user interface (GUI).
In the DS SM GUI examples we use two DS6000s, serial numbers 1300247 and 1300819 in a
similar configuration to the one used in the previous examples. Still, in this case note that
1300247 is at the local production site and 1300819 is at the remote backup site. Also, on each
machine, we are going to use volumes in LSS 18. You can tell the volumes apart by their
names. The local volumes on serial 1300247 are named av_247_1800 to av_247_1803. The
remote volumes on serial 1300819 are named av_819_1800 to av_819_1803.
Each DS6000 is in a separate Storage Complex, so we have already followed the procedure
to define one Storage Complex to the other one. This way, we can manage the entire
operation using a single DS SMC.
24.8 Set up a Global Mirror environment with the DS GUI
To set up a Global Mirror environment, we follow a similar procedure to the DS CLI. The
configuration we are going to set up was discussed in 24.7 “DS Storage Manager GUI
examples” on page 347.
24.8.1 Define paths
With DS Storage Manager, to create new paths between two LSSs on two different storage
disk subsystems, it is necessary to go through a six step process.
You need to repeat this wizard for each data path between LSSs on the source Storage Unit
on site 1 and the target LSSs on the target Storage Unit on site 2, and for each control path
between the Master Storage Unit LSS and the Subordinate Storage Units’ LSSs.
To launch this wizard, you need to go first to the Paths panel under the Copy Services menu
of the DS Storage Manager GUI. Select from the pull-down lists the Storage Complex, then
the Storage Unit, and finally the LSS that contains the source volumes of the Global Copy
pairs we want to create. In the Select Action pull-down menu, choose Create to proceed
with the first step of the wizard. See Figure 24-15 on page 348.
Chapter 24. Global Mirror examples
347
Figure 24-15 Global Copy paths creation: Launch the creation process
1. Now the creation wizard displays the Select source LSS panel; see Figure 24-16. Here,
you select the LSS that contains the source volumes of the Global Copy pairs you are
going to create from the pull-down list.
Figure 24-16 Global Copy paths creation - Step 1: Select the source LSS
2. Click Next to proceed with the second step of this wizard.
When the creation wizard displays the Select target LSS panel, select the Storage
Complex, then the Storage Unit, and finally the LSS which contains the target volumes
from the pull-down lists; see Figure 24-17 on page 349.
348
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 24-17 Global Copy paths creation - Step 2: Select the target LSS
3. Click Next to proceed with the third step of this wizard.
When the creation wizard displays the Select source I/O ports panel, select using the
check boxes at least one I/O port (two is better) to use for Global Copy replication; see
Figure 24-18. In the location column, four digits indicate the location of the port:
–
–
–
–
The first digit (R) is to locate the frame.
The second digit (E) is for the I/O enclosure.
The third digit (C) is for the adapter.
The fourth digit (P) is for the adapter’s port.
Figure 24-18 Global Copy paths creation - Step 3: Select the source I/O ports
4. Click Next to proceed with the fourth step of this wizard.
When the creation wizard displays the Select target I/O ports panel, for each I/O port
selected during the third step, select from its related pull-down list the target I/O port (see
step 3 of this wizard to get an explanation of the RECP digits). See Figure 24-19 on
page 350.
Chapter 24. Global Mirror examples
349
In this example, we do not have any choices, because there are only two paths in total
between our disk subsystems.
Figure 24-19 Global Copy paths creation - Step 4: Select the target I/O ports
5. Click Next to proceed with the fifth step of this wizard.
Figure 24-20 Global Copy paths creation - Step 5: Select the paths options
When the creation wizard displays the Select path options panel, you can select the option
Define as Consistency Group using the check box; see Figure 24-20. This option is not
mandatory, because the Global Mirror session handles the consistency of the data across
the set of volumes.
6. Click Next to proceed with the sixth and last step of this wizard.
Figure 24-21 on page 351 shows the Verification panel. Here, you check all the
components of your paths configuration. If necessary, click Back to correct any of them or
click Finish to validate the configuration and end the wizard.
350
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 24-21 Global Copy paths creation - Step 6: Verification
24.8.2 Create Global Copy pairs
With DS Storage Manager, to create new Global Copy relationships for one or several
volume pairs, it is necessary to go through a five step process.
Note: If Global Copy pairs are in several LSSs, do not forget to select all of them during
this wizard or you have to run the wizard again on each LSS. If Global Copy pairs are
spread over several Storage Units, run this wizard again on each of them.
To launch this wizard, you need to go first to the Metro Mirror panel under the Copy Services
menu of the DS Storage Manager GUI; see Figure 24-22. In the Select Action pull-down list,
choose Create.
Figure 24-22 Global Copy creation: Launch the creation process
1. When the creation wizard displays the Volume Pairing Method, select the radio button
Manual volume pair assignment; see Figure 24-23 on page 352.
Chapter 24. Global Mirror examples
351
Figure 24-23 Global Copy creation - Step 1: Choose volume pairing method
2. Click Next to proceed with the second step of this wizard.
When the creation wizard displays the Select source volumes panel, select from the
pull-down lists: the Storage Complex, then the Storage Unit, then the resource type, and if
it is necessary, its appropriate parameter to display the list of volumes. See Figure 24-24.
If you have chosen the resource type LSS, then select from the pull-down list the LSS
number that contains the source volumes you are going to use for Global Mirror and then
check the boxes of the selected volumes.
Note: At this step, if paths have not yet been created, you can click Create Paths to
launch the wizard described in 24.8.1 “Define paths” on page 347.
Figure 24-24 Global Copy creation - Step 2: Select the source volumes
3. Click Next to proceed with the third step of this wizard.
352
IBM System Storage DS6000 Series: Copy Services in Open Environments
When the creation wizard displays the Select target volumes panel, notice that only one
source volume is indicated on the top of the panel; see Figure 24-25. This means that you
repeat this third step for each volume that you have selected during the second step.
Select from the pull-down lists: the Storage Complex, then the Storage Unit, then the
resource type, and if it is necessary, its appropriate parameter to display the list of
volumes. Then on the required LSS, check the box for the selected target volume.
Figure 24-25 Global Copy creation - Step 3: Select the target volume 1
4. Click Next to proceed with the selection of the second target volume; see Figure 24-26.
When the creation wizard once again displays the Select target volumes panel, notice that
only the indicated source volume on the top of the panel is different. Once again, select
from the pull-down lists: the Storage Complex, then the Storage Unit, then the resource
type, and if it is necessary, its appropriate parameter to display the list of volumes. Then
on the required LSS, check the box for the selected target volume.
Figure 24-26 Global Copy creation - Step 3: Select the target volume 2
Chapter 24. Global Mirror examples
353
If you had selected more source volumes, you would once again proceed to the next
target volume selection panel. Because it is the second and last volume in your selection,
click Next to proceed with the fourth step of this wizard.
When the creation wizard displays the Select copy options panel, select the radio button
Global Copy to define the type of replication, then check the box for Permit read access
from target, and if it is the first synchronization between source and target volumes,
check the box for Perform initial copy. See Figure 24-27.
Figure 24-27 Global Copy creation - Step 4: Select the copy options
5. Click Next to proceed with the fifth and last step of this wizard.
In the Verification panel, verify all the components of your Global Copy session
configuration and, if necessary, click Back to correct any of them or click Finish to
validate; see Figure 24-28.
Figure 24-28 Global Copy creation - Step 5: Verification
354
IBM System Storage DS6000 Series: Copy Services in Open Environments
24.8.3 Create FlashCopy relationships
With DS Storage Manager, to create new FlashCopy relationships for one or several pairs of
volumes, it is necessary to go through a five step process. We create these FlashCopy
relationships on the remote DS6000.
Note: If FlashCopy pairs are in several LSSs, do not forget to select all of them during this
wizard or you have to run the wizard again on each LSS. If FlashCopy pairs are spread
over several Storage Units, run this wizard again on each of them.
To launch this wizard, you need first to go to the FlashCopy panel under the Copy Services
menu of the DS Storage Manager GUI. In the Select Action pull-down list, choose Create to
proceed with the first step of the wizard. See Figure 24-29.
Figure 24-29 FlashCopy creation: Launch the creation process
1. When the creation wizard displays the Define relationship type, select the radio button A
single source with a single target. This is because you have to activate the record
option on the fourth step of this wizard; see Figure 24-30.
Figure 24-30 FlashCopy creation - Step 1: Select the relationship type
2. Click Next to proceed with the second step of this wizard.
When the creation wizard displays the Select source volumes panel, select from the
pull-down lists: the Storage Complex, then the Storage Unit, then the resource type, and if
it is necessary, its appropriate parameter to display the list of volumes. If you have chosen
Chapter 24. Global Mirror examples
355
the resource type LSS, select from the pull-down list the LSS number that contains the
source volumes you want to use for the Global Mirror environment. Then, check the boxes
of the selected source volumes. See Figure 24-31.
Figure 24-31 FlashCopy creation - Step 2: Select the source volumes
3. Click Next to proceed with the third step of this wizard.
When the creation wizard displays the Select target volumes panel, select the resource
type from the pull-down list, and if it is necessary, its appropriate parameter to display the
list of volumes. If you have chosen the resource type LSS, select the LSS number that
contains the target volumes you want to use from the pull-down list. Then, check the
boxes of the selected target volumes. See Figure 24-32.
Figure 24-32 FlashCopy creation - Step 3: Select the target volumes
4. Click Next to proceed with the fourth step of this wizard.
When the creation wizard displays the Select common options panel, check the box for
Enable change recording. This automatically also checks the box for Make
356
IBM System Storage DS6000 Series: Copy Services in Open Environments
relationship(s) persistent. You can leave the *Sequence number field with its default
value, because Global Mirror takes care of it automatically. See Figure 24-33.
Note: Do not check the box for Initiate background copy. You might need to deselect it.
Figure 24-33 FlashCopy creation - Step 4: Select the common option
5. Click Next to proceed with the fifth and last step of this wizard.
In the Verification panel, verify all the components of the FlashCopy configurations, and if
necessary, click Back to correct any of them or click Finish to validate. See Figure 24-34.
Figure 24-34 FlashCopy creation - Step 5: Verification
Chapter 24. Global Mirror examples
357
24.8.4 Create the Global Mirror session
With DS Storage Manager to create a Global Mirror session, you go through a three step
process. To launch the creation wizard, you need to go first to the Global Mirror panel under
the Copy Services menu of DS Storage Manager GUI. In the Select Action pull-down list,
choose Create to proceed with the first step of the wizard. See Figure 24-35.
Figure 24-35 Global Mirror creation: Launch the creation process
The creation wizard displays the Select volumes panel; see Figure 24-36.
Figure 24-36 Global Mirror creation. Step 1: Select volumes
358
IBM System Storage DS6000 Series: Copy Services in Open Environments
1. When the creation wizard displays the Select volumes panel, click the required Storage
Unit, and then on the required LSS, check the boxes for the source volumes that you want
to be part of the Global Mirror session; see Figure 24-36 on page 358.
Note: At this step, if Global Copy pairs and FlashCopy pairs are not yet created, you can
click Create Metro Mirror to launch the wizard described in 24.8.2 “Create Global Copy
pairs” on page 351, and you can click Create FlashCopy to launch the wizard described in
24.8.3 “Create FlashCopy relationships” on page 355.
2. Click Next to proceed with the second step of this wizard.
Figure 24-37 Global Mirror creation. Step 2: Define properties
When the creation wizard displays the Define properties panel, the session number field
should be filled with the appropriate session number. You can click Get Available
Session Numbers if you have forgotten the one you want to use. See Figure 24-37.
There are also three important fields that can be set at this time using this panel, and
these are the Global Mirror session tuning parameters. For a detailed description of these
options, see 20.4.2 “Consistency Group parameters” on page 260. Following is a brief
discussion of each of them:
– Consistency Group interval time (seconds): Specifies how long to wait between the
formation of Consistency Groups. If this number is not specified or is set to zero,
Consistency Groups are formed continuously.
– Maximum coordination interval (milliseconds): Indicates the maximum time that Global
Mirror can queue I/Os in the source disk subsystem to start forming a Consistency
Group.
– Maximum time writes inhibited to remote site (seconds): Specifies the maximum
amount of time allowed for the consistent set of data to drain to the remote site before
failing the current Consistency Group.
3. Click Next to proceed with the third and last step of this wizard.
Chapter 24. Global Mirror examples
359
In the Verification panel, check all the components of your Global Mirror session
configuration and, if necessary, click Back to correct any of them or click Finish to
validate; see Figure 24-38.
Figure 24-38 Global Mirror creation. Step 3: Verification
To visualize the status of the Global Mirror session, go to the Global Mirror panel under the
Copy Services menu of the DS Storage Manager GUI. Then, select from the pull-down lists:
the Storage Complex; then the Storage Unit, which is the Master; and wait until the panel is
refreshed. If necessary, click Refresh to refresh the panel; see Figure 24-39.
Figure 24-39 Visualize the Global Mirror session status
24.9 Manage the Global Mirror environment with the DS GUI
In this section, we discuss and give examples of how to perform common Global Mirror
control tasks using the DS CLI. We present the following management activities:
򐂰
򐂰
򐂰
򐂰
360
Query a Global Mirror environment.
Pause a Global Mirror session.
Resume a Global Mirror session.
Add or remove volumes from an LSS to the Global Mirror session.
IBM System Storage DS6000 Series: Copy Services in Open Environments
24.9.1 View settings and error information of the Global MIrror session
To see session information, you first to go to the Global Mirror panel under the Copy Services
menu of the DS Storage Manager GUI. Then, check the box for the Global Mirror session for
which you want to display its properties, and in the Select Action pull-down list, choose
Properties to proceed; see Figure 24-40.
Figure 24-40 View Global Mirror session’s properties: Launch the viewing process
The Global Mirror session properties: Real-time panel displays. In the General tab, you can
review the setting for this session; see Figure 24-41.
Figure 24-41 View Global Mirror session settings: General tab
You can then click Failures to see the Failures tab; see Figure 24-42 on page 362.
In the Global Mirror session properties: Real-time panel, in the Failures tab, you can request
selected information using the radio buttons Most recent failure, Previous failure, and First
failure. When you have finished reviewing the information, click OK to finish and go back to
the main Global Mirror panel.
Chapter 24. Global Mirror examples
361
Figure 24-42 View Global Mirror session errors information: Failures tab
24.9.2 View information of volumes in the Global Mirror session
To launch this viewing action, first you need to go to the Global Mirror panel under the Copy
Services menu of the DS Storage Manager GUI. Check the box for the Global Mirror session
for which you want to display its associated volumes information, and in the Select Action
pull-down list, choose View session volumes; see Figure 24-43.
Figure 24-43 View Global Mirror session volumes information: Launch the viewing process
The Global Mirror session volumes: Real-time panel displays and it shows information about
the volumes associated to the Global Mirror session; see Figure 24-44 on page 363. You can
either download or print this information table. Then, click OK to finish and go back to the
main Global Mirror panel.
362
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 24-44 Global Mirror session volumes information
24.9.3 Pause a Global Mirror session
To pause a Global Mirror session, you first go to the Global Mirror panel under the Copy
Services menu of the DS Storage Manager GUI; see Figure 24-43 on page 362. Then, select
the Global Mirror session with which you want to work. Then, in the Select Action pull-down
list, choose Pause to proceed.
When the Global Mirror: Real-time panel displays the warning message (see Figure 24-45),
either click Cancel to return to the main Global Mirror panel without pausing the Global Mirror
session, or OK to pause the Global Mirror session and to return to the main Global Mirror
panel.
Figure 24-45 Pause Global Mirror: Confirm the pause of the Global Mirror session
When the main Global Mirror panel displays, note that the state of the Global Mirror session is
now Paused; see Figure 24-46.
Figure 24-46 Pause Global Mirror: View session status paused
Chapter 24. Global Mirror examples
363
24.9.4 Resume a Global Mirror session
To resume a Global Mirror session, you first go to the Global Mirror panel under the Copy
Services menu of the DS Storage Manager GUI; see Figure 24-43 on page 362. Then, select
the Global Mirror session that you are going to resume. Then in the Select Action pull-down
list, choose Resume to proceed.
When the Global Mirror: Real-time panel displays the warning message, either click Cancel
to return to the main Global Mirror panel without resuming the Global Mirror session, or click
OK to resume the Global Mirror session and to return to the main Global Mirror panel.
When the main Global Mirror panel displays, the state of the Global Mirror session now shows
Running.
24.9.5 Modify a Global Mirror session
When using the DS Storage Manager to modify an existing Global Mirror session, either to
add or remove one or several sets of volumes or to change the Global Mirror settings, it is
necessary to go through a three step process.
To launch the wizard, you first go to the Global Mirror panel under the Copy Services menu of
the DS Storage Manager GUI; see Figure 24-43 on page 362. Then, select the Global Mirror
session you wish to work with. Then in the Select Action pull-down list, choose Modify to
proceed.
When the modification wizard displays the Select volumes panel, click the required Storage
Unit, then the required LSS, and either select or deselect the check box for the source
volumes you wish to either add or remove from the Global Mirror session. The panel looks
similar to Figure 24-36 on page 358.
Note: At this step, if Global Copy pairs and FlashCopy pairs have not yet been created,
you can click Create Metro Mirror to launch the wizard described in 24.8.2 “Create Global
Copy pairs” on page 351, and you can click Create FlashCopy to launch the wizard
described in 24.8.3 “Create FlashCopy relationships” on page 355.
Click Next to proceed with the second and last step of this wizard. When the creation wizard
displays the Define properties panel, the session number field should be filled with the
appropriate session number. This panel looks similar to Figure 24-37 on page 359. In this
panel, there are three important fields you can modify at this time: Consistency Group interval
time (seconds), Maximum coordination interval (milliseconds), and Maximum time writes
inhibited to remote site (seconds). For a detailed discussion of these Global Mirror tuning
parameters, see 20.4.2 “Consistency Group parameters” on page 260.
Click Next to proceed with the third and last step of this wizard. This is the Verification panel.
Here, you verify all the components of your Global Mirror session configuration and, if
necessary, click Back to correct any of them or click Finish to validate.
364
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 7
Part
7
Interoperability
In this part, we discuss the interoperability of the various Copy Services functions of the
DS6000. We also cover the interoperability of the DS6000 with other IBM System Storage
and TotalStorage disk subsystems in Copy Services implementations.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
365
366
IBM System Storage DS6000 Series: Copy Services in Open Environments
25
Chapter 25.
Data migration through double
cascading
In this chapter, we discuss how to ensure a consistent data migration by combining Copy
Services functions in a double cascading configuration.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
367
25.1 Data Migration with double cascading
Combining the Copy Service functions Metro Mirror and Global Mirror into an environment
with double cascading allows you to maintain data consistency while migrating data.
There are two possibilities for accomplishing data consistency during a migration using an
environment that has double cascading. The first possibility is to shut down all applications at
the local site and let the out-of-sync drain completely. The second possibility is to change the
Global Copy relationships to Metro Mirror, then when the out-of-sync is approaching zero, a
freeze is issued to the changed relationships. See Figure 25-1.
DWDM
A
DS6K
Local
PPRC-MM
B
DWDM
PPRC-GC
DS6K
PPRC-GC
<50 Kms
Secondary
C
DS6K
Tertiary
PPRC-GC
D
DS6k
Remote
Figure 25-1 Double cascading example
Figure 25-1 shows double cascading with a Metro Mirror relationship from the local to the
secondary and Global Copy from the secondary to the tertiary site and from the tertiary to the
remote site. There is a cascading relationship at both the secondary and tertiary site where
the volumes are both secondaries and primaries. In this example, if all applications are
stopped at the local site, then the local, secondary, tertiary, and remote reach a point after
some time where they are all equal. Therefore, they are consistent, and data has successfully
been migrated to the remote site volumes.
In the second approach to provide data consistency during migration, again looking at the
example in Figure 25-1, a freeze is first issued to the Metro Mirror relationship from the local
to secondary site. Because Metro Mirror is running between the local to secondary site, there
is already a synchronous relationship; therefore, the secondary volumes are consistent with
the local site volumes at this time. The next step is to change all the Global Copy
relationships to Metro Mirror relationships and then issue a freeze when the out-of-sync has
reached zero. When the out-of-sync is fully drained, then there is a consistent relationship
from the secondary to the remote, and migration to the remote is complete. Data migration
has sent a snapshot of data to the remote site so that there is consistency. You use this
process when production cannot be stopped, because production is still running at the local
site. However, once the freeze is first issued on the Metro Mirror relationship, from that point
forward, the remote is only consistent with the secondary. The local during this process is still
368
IBM System Storage DS6000 Series: Copy Services in Open Environments
being updated and is changing. The remote has a snapshot of the time, since the freeze was
issued.
See Figure 25-2.
DWDM
A PPRC-- GC
DS6K
Local
B
DWDM
PPRC-- GC
DS6K
Secondary
PPRC-- GC
C
PPRC-- MM
DS6K
<50 Kms
Tertiary
D
DS6k
Remote
Figure 25-2 Global Copy changed to Metro Mirror and direction reversed
Once data migration and consistency have been accomplished at the remote site, you can
start production at the remote site, if desired. In this case, all applications must first be
stopped at the local site and the direction of the mirror is reversed so that Metro Mirror is now
running from the remote to the tertiary site, and Global Copy is running from the tertiary site to
the secondary site and then to the primary site. Figure 25-2 shows the new configuration with
the Metro Mirror and Global Copy reversed and production running at the remote site.
In this example, we show that to protect the production at the remote, the remote copy
relationships can be reversed to use the tertiary, secondary, or local volumes as targets.
These copy relationships are created before starting production at the remote site. Because
the relationships from the secondary to the tertiary and from the tertiary to the remote are
both in a cascaded relationship, in order to reverse the directions, Global Copy must be first
removed and then recreated in the reverse directions.
Note: No I/O should be running at any time at any of the sites during the reversal of the
remote copy relations; if there is, the resulting data is corrupted, consequently requiring a
full copy.
Chapter 25. Data migration through double cascading
369
370
IBM System Storage DS6000 Series: Copy Services in Open Environments
26
Chapter 26.
Interoperability between DS6000
and DS8000
In this chapter, we show the interoperability between the Copy Services functions in the
DS6000 and the DS8000. This chapter contains the following sections:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
DS6000 and DS8000 Copy Services interoperability
Preparing the environment
RMC: Establishing paths between DS6000 and DS8000
Managing Metro Mirror or Global Copy pairs
Managing DS6000 to DS8000 Global Mirror
Managing DS6000 and DS8000 FlashCopy
Note: Remote Mirror and Copy (RMC) was formerly called Peer-to-Peer Remote Copy
(PPRC). All references to PPRC are interchangeable with RMC.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
371
26.1 DS6000 and DS8000 Copy Services interoperability
Copy Services operations are supported between the DS6000 and the DS8000. This means
you can have a mixed Copy Services environment that contains both devices.
26.2 Preparing the environment
Before starting Copy Services operations in a mixed DS6000 and DS8000 environment, you
need to ensure this environment is set up correctly.
26.2.1 Minimum microcode levels
To manage Copy Services in a mixed environment, you need to have licensed internal code
bundle 6.0.600.9 or higher on your DS6000. You also need to have your IBM Service
Representative install code bundle 6.0.500.52 or higher on your DS8000.
26.2.2 Hardware and licensing requirements
To establish PPRC pairs between an DS6000 and a DS8000, the DS6000 and the DS8000
must both have Remote Mirror and Copy (RMC) licenses.
The DS6000 and DS8000 must also have Fibre Channel adapters that have connectivity to
each other, either directly or through Fibre Channel switches. ESCON adapters cannot be
used to mirror a DS6000 with a DS8000.
26.2.3 Network connectivity
Network connectivity requirements depend on how you want to manage the environment. If
you plan to use System z interfaces such as TSO or ICKDSF, there are no network
connectivity requirements. The commands are sent inband through FICON or ESCON. If,
however, you plan to use the DS CLI or the DS GUI, your requirements are as follows:
򐂰 For FlashCopy management:
– If you want to use the DS8000 DS GUI to manage FlashCopy on the DS6000, then you
need network connectivity between the DS8000 HMCs and the DS6000 SMCs.
– If you want to use the DS6000 DS GUI to manage FlashCopy on the DS8000, then you
need network connectivity between the DS8000 HMCs and the DS6000 SMCs.
– If you want to use the DS CLI to manage FlashCopy on either the DS6000 or the
DS8000, then the DS8000 HMCs and the DS6000 SMCs do not need to be connected
by a network. However, the server on which the DS CLI is installed must have network
connectivity to both the SMC and the HMC.
– If you have PPRC links between the DS6000 and DS8000, you can also use remote
FlashCopy to establish FlashCopies of the Remote Mirror target volumes.
򐂰 For creating Remote Mirror and Copy (RMC), also known as Peer-to-Peer Remote Copy
(PPRC) paths and pairs:
– In a mixed environment, one system is the source (local) and one is the target. If you
want to be able to manage the PPRC paths and pairs from either machine, then you
need network connectivity between the DS6000 SMC and the DS8000 HMC. If you
use DS CLI, the SMC and HMC must be able to communicate with each other, but the
DS CLI machine must be able to communicate with both the HMC and the SMC.
372
IBM System Storage DS6000 Series: Copy Services in Open Environments
– If you do not plan to use the remote system as a source and you do not intend to use
the DS Storage Manager GUI to manage the pairs and paths, then network
connectivity to the remote machine (assuming the remote machine is at a remote site)
is unnecessary. Because all path and pair establishment is done by connecting to the
source machine (the local machine) when you use the DS CLI. In a disaster recovery
scenario, you must travel to the remote site to perform management tasks for the
remote machine. We do not recommend this setup, because it is less flexible.
26.2.4 Create matching user ids and passwords
When you want to use the DS CLI or DS GUI to perform Copy Services operations, you must
authenticate with a valid user ID and password. When you use the DS6000 DS GUI to
perform an operation that requires it to issue a command to a DS8000, it must authenticate
with the DS8000. The DS user ID and password that you used to log on to the DS6000 DS
GUI is used to authenticate with the DS8000 HMC. The same applies if you use the DS8000
DS GUI to manage a DS6000. This means that the same user ID and password must be
defined in both the SMC and the HMC. This task must be manually performed. If, instead of
the DS GUI, you only use the DS CLI, then this requirement does not exist. However, it is still
of benefit to use a matching user ID and password to manage machines in either Storage
Complex.
Note: A DS8000 Storage Complex is comprised of either one or two HMCs that together
manage one or two DS8000 Storage Units. A DS6000 Storage Complex consists of either
one or two SMCs that together manage one or two DS6000 Storage Units.
26.2.5 Updating the DS CLI profile
If you plan to use the DS CLI to manage your mixed environment, you can create a profile to
simplify connection to the DS CLI commands and script operations. To achieve this, add
extra lines as shown in Example 26-1. In this example, the DS6000 is considered the local
machine, and the DS8000 is considered the remote machine.
Example 26-1 Possible modification to a DS CLI profile
# DS6000
# hmc1 is the DS6000 SMC
hmc1: 10.0.0.100
# devid is the serial number of the DS6000
devid: IBM.1750-1300247
# remotedevid is the serial number of the DS8000
remotedevid: IBM.2107-7503461
# Username is a user created on the ESS Specialist that matches the userid on the DS6000
username:admin
# The password for the admin user id. Placing it here is not very secure.
password:passw0rd
# The password file is created using the managepwfile and is a better way to manage this.
# pwfile:security.dat
Placing the password in the profile is not secure (because it is stored in a plain text file), but it
might prove more convenient. You can create a password file using the managepwfile. This is
a better way to manage this. After creating the password file (which by default is called
security.dat), you can remove the password from the profile and instead specify the pwfile
file.
A simple method to use when you have multiple servers to manage is to create multiple
profile files. Then, when you start DS CLI, you can specify the profile file that you want to use
Chapter 26. Interoperability between DS6000 and DS8000
373
with the -cfg parameter. In a Windows environment, you can have multiple Windows batch
files (BAT files), one for each machine you wish to manage. Save the profile shown in
Example 26-1 on page 373 in the C:\program files\ibm\dscli\profile directory as
1750source.profile. Then, you can create a simple Windows BAT file with three lines as
shown in Example 26-2.
Example 26-2 Windows BAT file to start a specific profile
title DS CLI Local DS6800 1300247 Remote IBM.2107-7503461
cd C:\Program Files\ibm\dscli\profile
dscli -cfg 1750source.profile
We save the BAT file onto the Windows desktop and start it by double-clicking on the icon.
The DS CLI opens and starts the specified profile. By creating a BAT file and profile for each
machine, we can simplify the process of starting the DS CLI. We can also specify the profile
when starting the DS CLI from inside a script, or when running the DS CLI in script mode.
26.2.6 Adding the Storage Complex
If you want to use a single DS GUI to manage both FlashCopy and Remote Mirror and Copy
pairs on your DS6000 and DS8000, you have to add the Storage Complex of one Storage
Unit (such as a DS6000) to the Storage Complex of the other Storage Unit (such as a
DS8000). Note that we have to do this operation once in each direction. In other words, you
have to add the DS6000 to the DS8000 and then add the DS8000 to the DS6000.
Adding the DS6000 Storage Complex to a DS8000 Storage Complex
Important: Make sure the user ID you use to log on to the DS8000 DS GUI also exists on
the DS6000 SMC, and that it has the same password. If not, the operation to add the
Storage Complex fails. You must always use this same user ID for multi-complex
management.
You add a DS6000 Storage Complex to a DS8000 Storage Complex as follows:
1. Connect to the DS8000 HMC using the DS GUI.
2. Click Real-time manager.
3. Click Manage hardware.
4. Click Storage complexes.
5. Select Add.
Figure 26-1 on page 375 shows the Add Storage Complex panel.
374
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 26-1 Add DS6000 complex to DS8000
6. Type the IP address of the DS6000 SMC in the Management console 1 IP field. If you
have a second SMC, select Define a second Management console and type the second
SMC address in the Management console 2 IP field. Click OK.
7. When the add Storage Complex operation is complete, the Storage Complex panel shows
the DS6000 SMC as an additional Storage Complex.
8. After adding the DS6000 Storage Complex to the DS8000 Storage Complex, you are now
able to use the DS8000 DS GUI to create paths and Remote Mirror and Copy pairs, where
the DS6000 is the source device. You can also use the DS8000 DS GUI to manage
FlashCopy pairs on the DS6000.
Note: The steps to add the DS6000 Storage Complex to the DS8000 Storage Complex
cannot be performed using the DS CLI. This is by design. However, if you do not plan to
use the GUI, this is not an issue.
Adding the DS8000 Storage Complex to a DS6000 Storage Complex
Important: Make sure the user ID you use to log on to the DS6000 DS GUI also exists on
the DS8000 HMC, and that it has the same password. If not, the operation to add the
Storage Complex fails. You must always use this same user ID for multi-complex
management.
You add a DS8000 Storage Complex to a DS6000 Storage Complex as follows:
1. Connect to the DS6000 SMC using the DS GUI.
2. Click Real-time manager.
3. Click Manage hardware.
4. Click Storage complexes.
5. Select Add Storage Complex.
Figure 26-2 on page 376 shows the Add Storage Complex panel.
Chapter 26. Interoperability between DS6000 and DS8000
375
Figure 26-2 Add DS8000 complex to DS6000
6. Type the IP address of the DS8000 HMC in the Management console 1 IP address box. If
you have a second DS8000 HMC, select the Define a second Management console box
and type the second HMC address in the Management console 2 IP address box. Now,
select OK.
7. When the Add Storage Complex operation completes, the Storage Complex panel now
shows the DS8000 HMC as an additional Storage Complex. In Figure 26-3, we have a
DS6000 SMC, an ESS 800 Copy Services Server, and a DS8000 HMC all defined on one
DS6000 DS GUI.
10.0.0.1
10.0.0.2
10.0.0.3
Figure 26-3 DS6000 GUI showing multiple complexes
8. Having added the DS8000 Storage Complex to the DS6000 Storage Complex, you are
now able to use the DS6000 DS GUI to create paths and Remote Mirror and Copy pairs,
where the DS8000 is the source device. You can also use the DS6000 DS GUI to manage
FlashCopy pairs on the DS8000.
Note: The steps to add the DS8000 Storage Complex to the DS6000 Storage Complex
cannot be performed using the DS CLI.
User ID management
There are two important considerations when managing multiple Storage Complexes from a
single DS GUI:
򐂰 The user ID you use to add the alternative complex must exist on both complexes. You
must continue to use this user ID for all multi-complex operations. If you log on to either
DS GUI with a user ID that does not exist on the other Complex, the remote Complex does
not appear in the Storage Complex panel.
376
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Even when you have successfully added one complex to another, new user ids created in
one complex do not mirror to the other complex. They have to be manually created and
managed in each complex.
Storage management
You cannot use the DS8000 DS GUI to perform storage configuration on a DS6000.
Likewise, you cannot use the DS6000 DS GUI to perform storage configuration on a DS8000.
You can only perform Copy Services management tasks on the alternative device. If you are
logged on to the DS6000 DS GUI and want to configure some storage on the DS8000, you
need to log off and then log on to the DS8000 DS GUI. The same applies if you are logged on
to the DS8000 DS GUI and want to configure storage on the DS6000. You must log on to the
DS6000 DS GUI to do configure the storage.
26.2.7 Volume size considerations for PPRC
When you create PPRC pairs, it is extremely important that the two volumes, source and
target, are exactly the same size. In PPRC, it is not an issue if the target volume is larger than
the source volume. We simply do not write to the extra space on the target volume. However,
if an attempt is made to reverse the relationship (so that the source becomes the target), this
attempt fails, because now the source is larger than the target. Clearly, the best way to avoid
this failure is to ensure the source and target are exactly the same size.
When you create fixed block volumes on the DS8000, you have three size choices:
ds
The number of bytes allocated is the requested capacity value times
230 .
ess
The number of bytes allocated is the requested capacity value times
109 .
blocks
The number of bytes allocated is the requested capacity value times
512 bytes (because each block is 512 bytes).
When creating volumes that are going to be either PPRC source or PPRC target between
DS6000 and DS8000, we recommend you use -type ds to create volumes, because this
creates binary-sized volumes that do not result in any wasted Extent space. However, if the
planned partner device is an ESS 800, then you should only use -type ess
or -type blocks.
Creating matching fixed block volumes
In our example, we create volumes 1405 to 1407 on the DS8000 in Extent Pool P0 to use as
PPRC target volumes. We can use the following DS CLI command:
mkfbvol -extpool P0 -cap 2 -type ds 1405-1407
Example 26-3 shows the resulting volumes. Each volume shows in the cap (2^30B) column
that it is 2 GB and 4194304 blocks.
Example 26-3 Creating binary sized volumes
dscli> mkfbvol -extpool P0 -cap 2 -type ds 1405-1407
Date/Time: 3 November 2005 19:47:28 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00025I mkfbvol: FB volume 1405 successfully created.
CMUC00025I mkfbvol: FB volume 1406 successfully created.
CMUC00025I mkfbvol: FB volume 1407 successfully created.
dscli> lsfbvol -lss 14
Date/Time: 3 November 2005 19:47:37 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B) cap (10^9B) cap (blocks)
Chapter 26. Interoperability between DS6000 and DS8000
377
====================================================================================================
1400 Online Normal
Normal
2107-900 FB 512 P0
10.0
19531264
1401 Online Normal
Normal
2107-900 FB 512 P0
1.1
2148480
1402 Online Normal
Normal
2107-900 FB 512 P0
1.1
2148480
1403 Online Normal
Normal
2107-900 FB 512 P0
5.5
10742208
1404 Online Normal
Normal
2107-900 FB 512 P0
5.5
10742208
1405 Online Normal
Normal
2107-900 FB 512 P0
2.0
4194304
1406 Online Normal
Normal
2107-900 FB 512 P0
2.0
4194304
1407 Online Normal
Normal
2107-900 FB 512 P0
2.0
4194304
dscli>
Checking the block count at older code levels
Prior to DS8000 code bundle 6.0.500.31 or DS6000 code bundle 6.0.600.9, fixed block
volumes created using decimal sizes (meaning their size is listed in the 10^9 column), could
be up to 32k bytes larger than intended. If you are creating new volumes on a DS6000 or
DS8000 that is at or above this code bundle, you do not need to worry about the block count.
However, if you created volumes on a DS6000 or DS8000 below this code level, and you
foresee the possibility that you might want to use these volumes as targets in a DS8000 to
DS6000 PPRC pair, then you should check the block count to ensure the volume sizes
actually match. To do this, you need to perform an lsfbvol on both the source and the target
devices. Then, take note of the block count of the source and target volumes. If the block
counts do not match, then you must either:
򐂰 Remove the incorrectly sized volume with rmfbvol and create it again (assuming you are
now at the higher bundle).
򐂰 Create a new volume for the purposes of PPRC (again, assuming you are now at the
higher bundle).
򐂰 Use the larger volume only as a PPRC target.
Using a spreadsheet to check expected as opposed to actual block count
You can also use a formula to calculate the correct block size for any decimal volume size.
You can use this in a spreadsheet to determine if your DS6000 or DS8000 has any volumes
that are slightly too large. The formula is = INT((INT(size*10000000000/512)+63)/64)*64. The
size parameter in the formula is the decimal (10^9B) volume size. You can place the output of
lsfbvol into a spreadsheet and calculate the expected block count for each volume using the
contents of the 10^9 (decimal) cap column. Then, compare the column that has the output of
the formula to the block column to ensure the two match. In Table 26-1, the output of the
lsfbvol command has been modified and placed into a spreadsheet. The formula has been
placed into the Expected Block Size column. It uses the values in the (10^9B) column as an
input to determine the correct block size. One volume, 6002, is 64 blocks larger than it should
be. If that volume is to be used for PPRC, it should be deleted and recreated (provided the
data on it can be moved).
Table 26-1 Using a spreadsheet to calculate correct block size
378
ID
Data
type
Extpool cap
2^30B
(10^9B)
(blocks)
Expected
Block Size
1000
FB 512
P2
-
5
9765632
9765632
1001
FB 512
P2
-
5
9765632
9765632
1002
FB 512
P2
-
5
9765632
9765632
1003
FB 512
P0
-
5
9765632
9765632
4400
FB 512
P0
-
10
19531264
19531264
IBM System Storage DS6000 Series: Copy Services in Open Environments
5100
FB 512
P3
-
4.5
8789120
8789120
6000
FB 512
P2
-
9.4
18359424
18359424
6002
FB 512
P2
-
5.5
10742272
10742208
Note: Do not apply this formula to System i volumes or volumes that only have a size in
the 2^30 column, because System i volumes are correctly sized using 520 byte blocks,
while volumes that are binary-sized (their size is listed in the 2^30) also have the correct
number of blocks.
26.2.8 Establishment errors on newly created volumes
When volumes are created on an ESS 800, DS6000, or DS8000, an internal process is used
to format the volumes. If you attempt to use these volumes as PPRC or FlashCopy targets
while this process is occurring, the establishment fails.
In Example 26-4, a fixed block volume ID 1808 was created on the DS6800. We then
immediately attempted to use it as a PPRC target. This attempt fails with the message as
shown. Do not be concerned about the FlashCopy comment; this is normal. Wait for the
volume initialization process to finish and try again. The initialization time varies, depending
on the size of the volume.
Example 26-4 Establishing a PPRC pair where the target is still being formatted
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir 1406:1808
ate/Time: 3 November 2005 23:02:31 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
MUN03117E mkpprc: 1406:1808: Unable to establish FlashCopy or Remote Mirror and Copy pair.
A FlashCopy initialization is in progress.
26.3 RMC: Establishing paths between DS6000 and DS8000
To create a PPRC relationship between a DS8000 and an DS6000, we first need to create
logical paths (over the physical Fibre Channel connections). If we are using the DS GUI, then
provided we added the other machine’s Storage Complex, we can establish paths in either
direction (DS6000 to DS8000 or DS8000 to DS6000).
26.3.1 Decoding port IDs
When viewing port IDs in the DS GUI or the DS CLI, the port IDs can be decoded to show
which physical port on the DS8000 or DS6000 is in use.
For DS8000, port IDs look like I0000 or I0123, which actually breaks out as IEECP. The EE is
the enclosure number (minus 1), C is the card slot number (minus 1), and P is the port
number on the card. So, I0123 is enclosure 2 (1+1), slot 3 (2+1), port 3.
For DS6000, port IDs range from I0000 to I0001 and I0100 to I0103, which actually breaks
out as IEECP. The EE is the controller number (00 or 01), C is the card slot number (always
zero) and P is the port number on the card (0 to 3). So I0103 is controller 1, card slot 0, port 3.
Chapter 26. Interoperability between DS6000 and DS8000
379
26.3.2 Path creation using the DS GUI
Path creation between a DS6000 and a DS8000 is no different than the process used to
create paths between two DS8000s or two DS6000s. See 12.5, “Metro Mirror paths and links”
on page 123.
Paths in the reverse direction
The example above showed how to create a path from the DS8000 to the DS6000. In many
cases, it is likely that you need paths from DS6000 to the DS8000. Regardless, having
established paths in one direction, you can then establish paths in the opposite direction.
Fibre Channel allows for bidirectional mirroring over the same physical path.
Adding or deleting paths
You can add additional paths to an LSS pair by simply creating more paths. To remove a path,
just display the Paths panel, select the relevant source machine and LSS, select the path you
want to delete, and use the delete option from the Select Action pull-down.
26.3.3 Establish logical paths between DS8000 and DS6000 using the DS CLI
You can also use the DS CLI to establish logical paths. One additional step is that you need
to know the World Wide Node name (WWNN) of the target Storage Image. The three
commands that we use to establish a path are: lssi, lsavailpprcport, and mkpprcpath.
Determining the remote device WWNN
First, we need to determine the WWNN of the remote storage device. If you started the DS
CLI by connecting to the DS8000 HMC, then the remote storage device is the DS6000. If you
started the DS CLI by connecting to the DS6000, then the remote Storage Image is the
DS8000.
How to determine the DS6000 WWNN using the DS GUI
The steps are:
1. Click Real-time manager.
2. Click Manage hardware.
3. Click Storage units.
4. Click in the Select column for the DS6000 Storage Unit, and from the Select Action
pull-down, choose Properties.
5. The DS6000 Storage Unit WWNN displays on the subsequent Properties panel.
How to determine DS6000 WWNN using the DS CLI
In Example 26-5, we show how to display the WWNN of a Storage Image using the lssi
command.
Example 26-5 Determine the WWNN of a DS6000
dscli> lssi
Date/Time: 3 November 2005 1:05:50 IBM DSCLI Version: 5.1.0.204
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.1750-1300247 IBM.1750-1300247 511 500507630EFFFE16 Online Enabled
380
IBM System Storage DS6000 Series: Copy Services in Open Environments
How to determine the DS8000 WWNN using the DS GUI
The steps are:
1. Click Real-time manager.
2. Click Manage hardware.
3. Click Storage images (not the Storage Unit).
4. Click in the Select column for the DS8000 Storage Image, and from the Select Action
pull-down, choose Properties.
5. The DS8000 Storage Image WWNN displays on the subsequent Properties panel.
How to determine DS8000 WWNN using the DS CLI
In Example 26-6, we show how to display the WWNN of a DS8000 Storage Image using the
lssi command.
Example 26-6 Determine the WWNN of a DS8000
dscli> lssi IBM.2107-7503461
Date/Time: 1 November 2005 3:08:52 IBM DSCLI Version: 5.1.0.204
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.2107-7503461 IBM.2107-7503460 932 5005076303FFC08F Online Enabled
Listing the available ports using the DS CLI
Having determined the remote devices’ WWNN, we can now display the ports that are
available to establish PPRC paths. In Example 26-7, we are logged on to the DS8000 using
the DS CLI, so the remote device is the DS6000. We display ports available to establish paths
between LSS 14 on the DS8000 and LSS 18 on the DS6000.
Example 26-7 Displaying available ports for PPRC path establishment
dscli> lsavailpprcport -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 14:18
Date/Time: 3 November 2005 20:36:12 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
Local Port Attached Port Type
=============================
I0001
I0103
FCP
I0131
I0003
FCP
Establishing the logical paths using the DS CLI
Having determined which ports are available for each LSS pair between which we want to
copy and mirror, we now establish a one-way path using the mkpprcpath command. We can
then display established paths using the lspprcpath command.
In Example 26-8, we first establish a single path between LSS 14 on the DS8000 and LSS 18
on the DS6000. We then display the paths available for LSS 14 on the DS8000.
Example 26-8 Using the DS CLI to establish PPRC paths
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 18 I0001:I0103
Date/Time: 3 November 2005 20:37:09 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:18 successfully established.
dscli> lspprcpath 14
Date/Time: 3 November 2005 20:37:14 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
Chapter 26. Interoperability between DS6000 and DS8000
381
14 18
Success FF18 I0001 I0103
500507630EFFFE16
In Example 26-9, we add an additional path between LSS 14 on the DS8000 and LSS 18 on
the DS6000. Take careful note, though, that we include the existing path when creating a new
path. Otherwise, the existing path is removed and only the new path is available for use.
Example 26-9 Establishing additional paths using the DS CLI
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14
-tgtlss 18 I0001:I0103 I0131:I10003
Date/Time: 3 November 2005 20:37:49 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:18 successfully established.
dscli> lspprcpath 14
Date/Time: 3 November 2005 20:37:53 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
14 18 Success FF18 I0001 I0103
500507630EFFFE16
14 18 Success FF18 I0131 I0003
500507630EFFFE16
To establish paths where the DS6000 is the source, we should connect to the DS6000 using
the DS CLI and follow the same process, but specify the DS8000 as the remote device.
Important: When you connect using the DS CLI to the DS8000 HMC, the DS8000 is the
local device and the DS6000 is the remote device. If you connect using the DS CLI to the
DS6000, then DS6000 is now the local device and the DS8000 is the remote device.
The rmpprcpath command removes all paths between the source and target LSSs. To just
reduce the path count, use the mkpprcpath command, specifying only the paths you want
to continue using.
26.4 Managing Metro Mirror or Global Copy pairs
Having established paths, we can now establish volume pairs.
26.4.1 Managing Metro Mirror or Global Copy pairs using the DS GUI
Establishing and managing Metro Mirror and Global Copy pairs between a DS6000 and a
DS8000 using the DS GUI are no different from using the DS GUI to establish pairs between
two DS8000s or DS6000s. See 14.4, “DS Storage Manager GUI” on page 139 and 22.3, “DS
Storage Manager GUI” on page 291.
26.4.2 Managing Metro Mirror pairs using the DS CLI
Having established paths, we can now establish volume pairs. In this example, we show how
you can establish Metro Mirror volume pairs between a DS8000 and an DS6000 with the DS
CLI. In this example, we create two Metro Mirror pairs. Volumes 1401 and 1402 from the
DS8000 are the source volumes, and the target volumes on the DS6000 are 1805 and 1806.
In Example 26-10, we create two pairs using the mkpprc command. We then list them using
the lspprc command. We then remove one pair using the rmpprc command.
Example 26-10 Creating Metro Mirror pairs using the DS CLI
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir -mode full 1405:1805 1406:1806
Date/Time: 3 November 2005 20:50:34 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
382
IBM System Storage DS6000 Series: Copy Services in Open Environments
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1405:1805 successfully
created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1406:1806 successfully
created.
dscli> lspprc 1405-1406
Date/Time: 3 November 2005 20:50:41 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First
===========================================================================================
1405:1805 Copy Pending Metro Mirror 14
unknown
Disabled
Invalid
1406:1806 Copy Pending Metro Mirror 14
unknown
Disabled
Invalid
dscli> rmpprc -remotedev IBM.1750-1300247 1405:1805
Date/Time: 3 November 2005 20:51:16 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair
relationship 1405:1805:? [y/n]
y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1405:1805 relationship successfully
withdrawn.
In Example 26-10 on page 382, we connected to the DS8000 HMC using the DS CLI, so the
DS6000 is the remote device. If we want to establish pairs where the DS6000 is the source
device, we need to connect to the DS6000 using the DS CLI. This makes the DS8000 the
remote device.
Important: If you specify the wrong -remotedev, or you are using a profile where a
different remote device is specified than the one you intended to work on, you might get an
error message, CMUN03057, saying you are specifying an invalid subsystem ID. This might
happen because you are specifying the wrong remote device serial number. If you have
multiple possible remote devices, do not place a remotedev entry in your DS CLI profile.
26.4.3 Managing Global Copy pairs using the DS CLI
Having established paths, we can now establish volume pairs. In this example, we show how
you can establish Global Copy volume pairs between a DS8000 and a DS6000 with the DS
CLI. In this example, we create two Global Copy pairs. Volumes 1405 and 1406 from the
DS8000 are the source volumes, and the target volumes on the DS6000 are 1805 and 1806.
In Example 26-11, we create two pairs using the mkpprc command. We then list them using
the lspprc command. We then pause one pair using the pausepprc command.
Example 26-11 Using the DS CLI to manage Global Copy
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 1405:1805 1406:1806
Date/Time: 3 November 2005 20:52:30 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1405:1805 successfully
created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1406:1806 successfully
created.
dscli> lspprc 1405-1406
Date/Time: 3 November 2005 20:52:36 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass
===========================================================================================
1405:1805 Copy Pending Global Copy 14
unknown
Disabled
False
1406:1806 Copy Pending Global Copy 14
unknown
Disabled
False
dscli> pausepprc -remotedev IBM.1750-1300247 1405:1805
Date/Time: 3 November 2005 20:53:24 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1405:1805 relationship
successfully paused.
dscli> lspprc 1405
Chapter 26. Interoperability between DS6000 and DS8000
383
Date/Time: 3 November 2005 20:53:29 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First
===========================================================================================
1405:1805 Suspended Host Source Global Copy 14
unknown
Disabled
True
dscli>
26.5 Managing DS6000 to DS8000 Global Mirror
The establishment of Global Mirror between a DS8000 and an DS6000 is achieved using the
same methods that we explained in Chapter 24, “Global Mirror examples” on page 303.
26.5.1 Managing Global Mirror pairs using the DS CLI
In Example 26-12, we establish a Global Copy pair between volume 1805 on the DS6000 and
volume 1405 on the DS8000. We then create session 30 for LSS 14. We then create a Global
Mirror using session 30 and LSS 14.
Example 26-12 Establishing Global Mirror using the DS CLI
dscli> lspprc 1405
Date/Time: 3 November 2005 20:55:21 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00234I lspprc: No Remote Mirror and Copy found.
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp -tgtread -mode full 1405:1805
Date/Time: 3 November 2005 20:55:53 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1405:1805 successfully created.
dscli> lspprc 1405
Date/Time: 3 November 2005 20:56:28 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1405:1805 Copy Pending Global Copy 14
unknown
Disabled
True
dscli> mkremoteflash -dev IBM.1750-1300247 -conduit IBM.2107-7503461/14 -record -nocp 1805:1806
Date/Time: 2 November 2005 21:49:00 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1805:1806 successfully created. Use the
lsremoteflash command to determine copy completion.
dscli> mksession -lss 14 -volume 1405 30
Date/Time: 3 November 2005 21:01:01 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00145I mksession: Session 30 opened successfully.
dscli> lssession 14
Date/Time: 3 November 2005 21:05:42 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete
===========================================================================================================
14
30
Normal 1405
Join Pending Primary Copy Pending Secondary Simplex True
dscli> mkgmir -lss 14 -session 30
Date/Time: 3 November 2005 21:08:08 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00162I mkgmir: Global Mirror for session 30 successfully started.
dscli> showgmir 14
Date/Time: 3 November 2005 21:08:20 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
ID
IBM.2107-7503461/14
Master Count
1
Master Session ID
0x30
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/03/2005 21:09:32 EST
CG Time
-
384
IBM System Storage DS6000 Series: Copy Services in Open Environments
Successful CG Percentage
FlashCopy Sequence Number
Master ID
Subordinate Count
Master/Subordinate Assoc
0
0x00000000
IBM.2107-7503461
0
-
26.6 Managing DS6000 and DS8000 FlashCopy
Provided a FlashCopy license is present on the device, you can manage FlashCopy using the
DS GUI. The establishment of a FlashCopy pair on a DS6000 using the DS8000 DS GUI is
no different than establishing a DS8000 FlashCopy pair. See 8.5, “FlashCopy management
using the DS SM” on page 82. The establishment of a FlashCopy pair on a DS8000 using the
DS6000 DS GUI is again no different than establishing a DS6000 FlashCopy pair. For the DS
CLI, you connect directly to the relevant DS SMC or DS HMC, so it is also no different.
26.6.1 Creating a remote FlashCopy on a DS6000 using the DS CLI
You can also use the DS CLI to create a remote FlashCopy on a DS6000 where the source
PPRC device is a DS8000 or the reverse is true also. In Example 26-13, we connect using the
DS CLI to a DS8000. We determine that there are established paths from LSS 14 on the
DS8000 to LSS 18 on the DS6000. We create a Metro Mirror pair from volume 1405 on the
DS8000 to volume 1805 on the DS6000. We then create a remote FlashCopy on the DS6000
between DS6000 volumes 1805 and 1806. We then remove the remote FlashCopy.
Example 26-13 Creating a remote FlashCopy where the DS6000 is the remote target
dscli> lspprcpath 14
Date/Time: 3 November 2005 22:39:58 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
14 18 Success FF18 I0001 I0103
500507630EFFFE16
14 18 Success FF18 I0131 I0003
500507630EFFFE16
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir 1405:1805
Date/Time: 3 November 2005 21:41:18 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1405:1805 successfully created.
dscli> mkremoteflash -dev IBM.1750-1300247 -conduit IBM.2107-7503461/14 -persist -nocp 1805:1806
Date/Time: 3 November 2005 22:35:45 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1805:1806 successfully created. Use the
lsremoteflash command to determine copy completion.
dscli> lsremoteflash -dev IBM.1750-1300247 -conduit IBM.2107-7503461/14 1805:1806
Date/Time: 3 November 2005 22:36:00 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled
===========================================================================================================
1805:1806 18
0
Disabled
Disabled Enabled
Disabled Enabled
dscli> rmremoteflash -dev IBM.1750-1300247 -conduit IBM.2107-7503461/14 1805:1806
Date/Time: 3 November 2005 22:36:19 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00179I rmremoteflash: Are you sure you want to remove the remote FlashCopy pair {0}? [y/n]:y
CMUC00180I rmremoteflash: Removal of the remote FlashCopy volume pair 1805:1806 has been initiated
successfully. Use the lsremoteflash command to determine when the relationship is deleted.
You can also reverse the scenario and use the DS6000 as the source machine in a PPRC
pair and then create the remote FlashCopy on the DS8000.
Chapter 26. Interoperability between DS6000 and DS8000
385
Note: When using commands that work with remote FlashCopies, use the -dev parameter
to define the machine on which to perform the FlashCopy. Other commands, such as
mkpprc commands, refer to this same device as the remote device, using -remotedev.
Because a FlashCopy has to be sent to the remote site and then performed locally on that
site, the use of the -dev parameter to refer to the remote machine is correct.
386
IBM System Storage DS6000 Series: Copy Services in Open Environments
27
Chapter 27.
Interoperability between DS6000
and ESS 800
In this chapter, we show the interoperability between the Copy Services functions in the
DS8000 and the IBM Enterprise Storage Server Model 800 (ESS 800). This chapter contains
the following sections:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
DS6000 and ESS 800 Copy Services interoperability
Preparing the environment
RMC: Establishing paths between DS6000 and ESS 800
Managing Metro Mirror or Global Copy pairs
Managing ESS 800 Global Mirror
Managing ESS 800 FlashCopy
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
387
27.1 DS6000 and ESS 800 Copy Services interoperability
Copy Services operations are supported between the DS6000 and the ESS 800 and Model
750. For the rest of this chapter, all references to the ESS 800 also apply to the ESS 750. On
the ESS 800, Remote Mirror and Copy (RMC) is called Peer-to-Peer Remote Copy (PPRC).
All references to PPRC are interchangeable with RMC.
27.2 Preparing the environment
Before starting Copy Services operations in a mixed DS6000 and ESS 800 environment, you
need to ensure this environment is set up correctly.
27.2.1 Minimum microcode levels
To manage the ESS 800 Copy Services from the DS6000, you need to have your IBM
Service Representative install licensed internal code Version 2.4.3.65 or higher on the ESS
800 and DS6000 code bundle 6.0.600.9 or higher on the DS6000.
27.2.2 Hardware and licensing requirements
To establish PPRC pairs between an ESS 800 and a DS6000, the ESS 800 must have a
PPRC license and the DS6000 must have a Remote Mirror Copy (RMC) license.
The ESS 800 must also have Fibre Channel adapters that have connectivity to the DS6000.
ESCON adapters cannot be used to mirror an ESS 800 to a DS6000. You cannot have a
Copy Services relationship between a 2105 E20 or F20 and a DS6000. You could, however,
have a cascaded installation where an ESS F20 was mirrored to an ESS 800 that was then
mirrored to a DS6000. The ESS F20 to ESS 800 relationship has to be managed with ESS
management tools, such as the ESS CLI or the ESS Web Copy Services GUI.
27.2.3 Network connectivity
Network connectivity requirements depend on how you want to manage the environment.
For FlashCopy management:
򐂰 If you want to use the DS6000 DS GUI to manage FlashCopy on the ESS 800, then you
need network connectivity between the DS6000 SMCs and the ESS 800 Copy Services
servers.
򐂰 If you want to be able to use the DS CLI to manage FlashCopy on the ESS 800, then you
need network connectivity between the machine on which you are running the DS CLI and
at least one of the ESS 800 Copy Services servers.
For creating Remote Mirror and Copy (RMC), also known as Peer-to-Peer Remote Copy
(PPRC), paths and pairs:
򐂰 If you want to use the ESS 800 as a PPRC source machine, then you need network
connectivity to the ESS 800 Copy Services servers, either from a machine running the DS
CLI, or from the DS6000 SMC.
򐂰 If, however, the ESS 800 is a remote target for PPRC, and you do not plan to ever use it
as a source machine and you do not intend to use the DS GUI to manage the pairs and
paths, then you do not need to have network connectivity to the ESS 800 Copy Services
servers. Because all path and pair establishment is done by connecting to the source
388
IBM System Storage DS6000 Series: Copy Services in Open Environments
machine (which is the DS6000). We do not recommend this setup, because it is less
flexible.
27.2.4 Create matching user IDs and passwords
When you want to use the DS CLI or DS GUI to perform Copy Services operations, you need
to authenticate with a valid user ID and password. When you use the DS GUI to perform an
operation that requires it to issue a command to an ESS 800, it needs to authenticate with the
ESS 800. To do this, it uses the DS user ID and password with which you logged on to the
GUI. This means that you need to define this user ID and password in the ESS Specialist,
which you need to perform manually. If, instead of the DS GUI, you only use the DS CLI, then
you log on to the ESS 800 Copy Services server directly, so this requirement does not exist.
For simplified management, you might still want to create a matching user ID and password.
Create a user ID on the DS6000
The first step is to log on to the DS6000 SMC using the DS GUI and create a user ID that is in
either the admin, op_storage, or op_copy_services groups. Log off and then log on with that
user ID and change the initial password.
Creating a user ID on the ESS 800
Once you have created a DS user ID, you need to create a matching user ID using the ESS
800 Web Specialist:
1. Start a Web Browser and connect to the IP address of either ESS 800 cluster.
2. Click ESS Specialist.
3. Log on with an ESS Specialist user ID that has admin privileges.
4. Click Users tab.
5. Click Modify Users. Type the user ID name and password you created in the DS CLI or
GUI. It must be given the Administration access level.
6. Click Add to move the user ID to the box on the right.
7. Click Perform Configuration Update and wait for the completion message.
Important: The ESS Web Copy Services user ids are not used by the DS CLI or DS GUI.
You only need to create a matching ESS Specialist user ID.
27.2.5 Updating the DS CLI profile
If you plan to use the DS CLI to manage your ESS 800, you can create a profile to simplify
connection, commands, and scripted operations. Add extra lines as shown in Example 27-1.
Example 27-1 Possible modification to a DS CLI profile
# ESS 800
# SMC1 is the Copy Services server A
SMC1: 10.0.0.100
# devid is the serial number of the ESS 800 - note there are only 5 digits after the 2105
devid: IBM.2105-22399
# remotedevid is the serial number of the DS6000
remotedevid: IBM.1750-1300247
# Username is a user created on the ESS Specialist that matches the userid on the DS6000
username:admin
# The password for the admin user id. Placing it here is not very secure.
password:passw0rd
Chapter 27. Interoperability between DS6000 and ESS 800
389
# The password file is created using the managepwfile and is a better way to manage this.
# pwfile:security.dat
Putting the password in the profile is not secure (because it is stored in plain text), but it can
be more convenient. You can create a password file using the managepwfile command and
this is a better way to manage this situation. After creating the password file (which by default
is called security.dat), you can remove the password from the profile and instead specify the
pwfile file.
The command to create a password file in this example is:
managepwfile -action add -name admin -pw passw0rd
A simple method when you have multiple machines to manage is to create multiple profile
files. Then when starting the DS CLI, you can specify the profile file that you want to use with
the -cfg parameter. In a Windows environment, you can have multiple Windows batch files
(BAT files), one for each machine you want to manage. The profile shown in Example 27-1 on
page 389 can be saved in the C:\program files\ibm\dscli\profile directory as
2105source.profile. Then you create a simple Windows BAT file with three lines, as shown in
Example 27-2.
Example 27-2 Windows BAT file to start a specific profile
title DS CLI Local 2105-22399 Remote IBM.1750-1300247
cd C:\Program Files\ibm\dscli\profile
dscli -cfg 2105source.profile
We save the BAT file onto the Windows desktop and start it by double-clicking the icon. The
DS CLI opens and starts the specified profile. By creating a BAT file and profile for each
machine, we can simplify the process of starting the DS CLI. We can also specify the profile
when starting the DS CLI from inside a script, or when running the DS CLI in script mode.
27.2.6 Adding the Copy Services Domain
If you want to use the DS GUI to manage FlashCopy on an ESS 800, or Remote Mirror and
Copy paths and pairs between an ESS 800 and a DS6000, you have to add the ESS Copy
Services domain to the DS SMC. You need the IP address of the ESS 800 Copy Services
servers (server A, and if desired, server B). You can get these by connecting with a Web
Browser to either cluster of the ESS, and clicking the Copy Services tab. They display on the
first window you see.
You add an ESS Copy Services domain with the DS Storage Manager to the DS6000 as
follows:
1. Click Real-time manager.
2. Click Manage hardware.
3. Click Storage complexes.
4. Select Add 2105 Copy Services Domain.
Figure 27-1 on page 391 shows the Add 2105 Copy Services Domain panel.
390
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 27-1 Add 2105 Copy Services Domain
5. Enter the IP address of the ESS Copy Services server A in the Server 1 IP address box. If
you have a Server B, select the Define a second Copy Services server box and type the
Server B IP address in the Server 2 IP address box. However, if Server B is running on a
2105-F20, you should not define it.
Having added the ESS 800 Copy Services domain to the DS GUI, you are now able to use
the DS GUI to create paths and Remote Mirror and Copy pairs, where the ESS 800 is the
source device. You can also use the DS GUI to manage FlashCopy pairs on the ESS 800.
Note: The steps to add the ESS 800 Copy Services Domain to a DS6000 cannot be
performed using the DS CLI. However, if you do not plan to use the GUI, this is not an
issue.
Restriction: You cannot use the ESS Copy Services Server Web GUI to manage Copy
Services relationships between an ESS 800 and a DS6000. The Volumes tab only shows
ESS 800 volumes, not DS6000 volumes. The Paths tab does not show paths between an
ESS 800 and a DS6000. It only shows paths between ESS 800s. If you are using PPRC
between a DS6000 and an ESS 800, all management of that PPRC relationship must be
done with the DS CLI or through the DS GUI.
Storage management
You cannot use the DS6000 DS GUI to perform storage configuration on an ESS 800.
Likewise, you cannot use the ESS 800 Web Specialist to perform storage configuration on a
DS6000. You can only perform Copy Services management tasks on the alternative device. If
you are logged on to the DS6000 DS GUI and want to configure some storage on the ESS
800, you need to log on to the ESS 800 Web Specialist.
27.2.7 Volume size considerations for RMC (PPRC)
When volumes are created on an ESS 800, they are sized using decimal gigabytes. This
means that when a request is made to a create a 10 GB volume, the ESS 800 allocates a
minimum of 10 000 000 000 bytes. This is a very important consideration when using PPRC
between a DS6000 and an ESS 800. In PPRC, it is not an issue if the target volume is larger
than the source volume. We simply do not write to the extra space on the target volume.
However, if an attempt is made to reverse the relationship (so that the source becomes the
target), this attempt fails, because now the source is larger than the target. Clearly the best
way to avoid this failure is to ensure that the source and target are exactly the same size.
Chapter 27. Interoperability between DS6000 and ESS 800
391
When fixed block volumes are created on the DS6000, you have three size choices:
ds
The number of bytes allocated is the requested capacity value times
230 .
ess
The number of bytes allocated is the requested capacity value times
109 .
blocks
The number of bytes allocated is the requested capacity value times
512 bytes (since each block is 512 bytes).
The correct method is to determine the ESS volume size and then create fixed block volumes
that are sized using the -type ess parameter.
Determining ESS volume size
To view the ESS volume sizes, you can use the ESS Specialist:
1. Start a Web Browser and connect to the IP address of either ESS 800 cluster.
2. Click ESS Specialist.
3. Log on with an ESS Specialist user ID.
4. Click the Storage Allocation tab.
5. Click the Open Systems Storage tab.
6. Click Modify Volumes Assignments.
7. Take note of the value in the size column for the volumes in which you are interested, as
shown in Figure 27-2.
Figure 27-2 Viewing ESS 800 volume size using ESS Specialist GUI
You can also use the ESS CLI to view the volume size, as shown in Example 27-3.
Example 27-3 View ESS 800 volume size using ESS CLI
C:\Program Files\ibm\ESScli>esscli -s 10.0.0.1 -u storwatch -p specialist list volume
Wed Nov 02 01:13:10 EST 2005 IBM ESSCLI 2.4.0
Volume
-----1000
1001
1002
1003
392
Cap
----1.1
1.1
1.1
1.1
Units
----GB
GB
GB
GB
VolType
------FB
FB
FB
FB
LSS
--10
10
10
10
VS
---vs0
vs0
vs0
vs0
Serial
-------00022399
00122399
00222399
00322399
Label
----***
***
***
***
IBM System Storage DS6000 Series: Copy Services in Open Environments
Creating matching fixed block volumes on the DS6000
In Figure 27-2 on page 392 and Example 27-3 on page 392, we listed four volumes which are
all shown as 1.1 GB. In our example, we are going to create volumes 1400 to 1403 on the
DS6000 in Extent Pool P0 to use as PPRC target volumes. We can use the following DS CLI
command:
mkfbvol -extpool P0 -cap 1.1 -type ess 1400-1403
Example 27-4 shows the resulting volumes. Each volume shows in the cap (10^9B) column
as 1.1 GB.
Example 27-4 Volumes
dscli> mkfbvol -extpool P0 -cap 1.1 -type ess -name AV_#h 1800-1803
Date/Time: 3 November 2005 0:56:50 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00025I mkfbvol: FB volume 1800 successfully created.
CMUC00025I mkfbvol: FB volume 1801 successfully created.
CMUC00025I mkfbvol: FB volume 1802 successfully created.
CMUC00025I mkfbvol: FB volume 1803 successfully created.
dscli> lsfbvol -lss 18
Date/Time: 3 November 2005 1:09:12 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Name
ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B) cap (10^9B) cap (blocks)
===========================================================================================================
AV_1800 1800 Online
Normal
Normal
1750-500 FB 512 P0
1.1
2148480
AV_1801 1801 Online
Normal
Normal
1750-500 FB 512 P0
1.1
2148480
AV_1802 1802 Online
Normal
Normal
1750-500 FB 512 P0
1.1
2148480
AV_1803 1803 Online
Normal
Normal
1750-500 FB 512 P0
1.1
2148480
Checking the block count at older code levels
Prior to DS6000 code bundle 6.0.600.9, fixed block volumes created on the DS6000 could be
up to 32k bytes larger than the equivalent-sized volume on the ESS 800. If you are creating
new volumes on a DS6000 that is at or above this code bundle, you do not need to worry
about the block count. However, if you created volumes below this code level, and you
foresee the possibility that you might want to use these DS6000 volumes as targets in a
DS6000 to ESS 800 PPRC pair, then you should check the block count to ensure the volume
sizes definitely match. To do this, you need to first check the block count of the volume on the
ESS 800. To do this, you need to open the ESS Copy Services server:
1. Start a Web Browser and connect to the IP address of either ESS 800 cluster.
2. Click Copy Services.
3. Log on with an ESS Specialist user ID that has admin privileges.
4. Click the Volumes tab.
5. In the source pull-down, choose the LSS where your particular volume is located.
6. When the volumes in that LSS are displayed, left-click a particular volume to highlight it.
7. Now, click on the Information tab on the bottom right corner of the window.
8. In the subsequent window, take note of the sectors count. This is actually the block count.
In Figure 27-3 on page 394, the example is 2148480 sectors (or blocks).
9. Now, take note of the block count from the output of the DS CLI command lsfbvol (for
the proposed source volume on the DS6000). In Example 27-4, the block count is
2148480, which is an exact match.
Chapter 27. Interoperability between DS6000 and ESS 800
393
Figure 27-3 Displaying block count using Copy Services server
10.If the block counts do not match, then you must either:
– Remove the volume with rmfbvol and create it again (assuming you are now at bundle
6.0.600.9 or higher).
– Create a new volume for the purposes of PPRC (again, assuming you are now at
bundle 6.0.600.9 or higher).
– Use this volume only as a PPRC target, since the equivalent ESS 800 source volume
is slightly smaller.
Using a spreadsheet to check expected versus actual block count
You can also use a formula to calculate the correct block size for any DS6000 volume that
has to match in size to an ESS 800 volume. You can use this in a spreadsheet to determine if
your DS6000 has any volumes that are slightly too large.
The formula is = INT((INT(size*10^9/512)+63)/64)*64 where the size parameter in the
formula is the ESS 800 volume size. You can place the output of lsfbvol into a spreadsheet
and calculate the expected block size for each volume using the contents of the 10^9
(decimal) cap column. Then, compare the column that has the output of the formula to the
block column to ensure that the two match. In Table 27-1, the output of the lsfbvol command
has been modified and placed into a spreadsheet. The formula has been placed into the
Expected Block Size column. It uses the values in the (10^9B) column as an input to
determine the correct block size. One volume, 6002, is 64 blocks larger than it should be. If
that volume is to be used for PPRC, it should be deleted and recreated (provided the data on
it can be moved).
Table 27-1 Using a spreadsheet to calculate correct block size
394
ID
Data
type
Extpool cap
2^30B
(10^9B)
(blocks)
Expected
Block Size
1000
FB 512
P2
-
5
9765632
9765632
1001
FB 512
P2
-
5
9765632
9765632
1002
FB 512
P2
-
5
9765632
9765632
1003
FB 512
P0
-
5
9765632
9765632
4400
FB 512
P0
-
10
19531264
19531264
5100
FB 512
P3
-
4.5
8789120
8789120
6000
FB 512
P2
-
9.4
18359424
18359424
6002
FB 512
P2
-
5.5
10742272
10742208
IBM System Storage DS6000 Series: Copy Services in Open Environments
Note: Do not apply this formula to System i volumes or volumes, which only have a size in
the 2^30 column. Because System i volumes are correctly sized using 520 byte blocks,
while volumes that are binary-sized (their size is listed in the 2^30B column) also have the
correct number of blocks.
One option when creating fixed block volumes is to use the parameter -type blocks. The
number of blocks can be calculated using the methods shown in Figure 27-3 on page 394 or
Table 27-1 on page 394. An example command is:
mkfbvol -extpool P0 -type blocks -cap 10742208 1404
27.2.8 Volume address considerations on the ESS 800
On the ESS 800, open systems volume IDs are given in an eight digit format, xxx-sssss,
where xxx is the LUN ID and sssss is the serial number of the ESS 800. Figure 27-2 on
page 392 shows an example of this, where the volumes shown are 000-22399 to 003-22399.
These volumes are open systems, or fixed block volumes. When referring to them in the DS
CLI, you must add 1000 to the volume ID, so volume 000-22399 is volume ID 1000 and
volume 001-22399 is volume ID 1001. This is extremely important, because on the ESS 800,
the following address ranges are actually used:
0000 to 0FFF
1000 to 1FFF
CKD volumes (4 096 possible addresses)
Open systems fixed block LUNs (4 096 possible addresses)
If we intend to use FlashCopy to copy ESS LUN 000-22399 onto 001-22399 using the DS CLI,
we must specify 1000 and 1001. If instead, we specify 0000 and 0001, we actually run the
FlashCopy against CKD volumes. This might result in an unplanned outage on the System z
environment that was using CKD volume 0001.
27.2.9 Establishment errors on newly created volumes
When volumes are created on an ESS 800, DS6000, or DS8000, an internal process is used
to format the volumes. If you attempt to use these volumes as PPRC or FlashCopy targets
while this process is occurring, the establishment fails.
In Example 27-5, a fixed block volume ID 1808 was created on the DS6800. We then
immediately attempted to use it as a PPRC target. This attempt fails with the message as
shown. Do not be concerned about the FlashCopy comment, because this is normal. Wait for
the volume initialization process to finish and try again. The initialization time varies
depending on the size of the volume.
Example 27-5 Establishing a PPRC pair where the target is still being formatted
dscli> mkpprc -remotedev IBM.1750-1300247 -type mmir 1406:1808
ate/Time: 3 November 2005 23:02:31 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22339
MUN03117E mkpprc: 1406:1808: Unable to establish FlashCopy or Remote Mirror and Copy pair.
A FlashCopy initialization is in progress.
27.3 RMC: Establishing paths between DS6000 and ESS 800
To create a PPRC relationship between a DS6000 and an ESS 800, we first need to create
logical paths (over the physical Fibre Channel connections). If we are using the DS GUI, then
provided we added the ESS Copy Services Domain to the DS6000, we can establish paths in
Chapter 27. Interoperability between DS6000 and ESS 800
395
either direction (ESS 800 to DS6000 or DS6000 to ESS 800). If the ESS Copy Services
domain was not added, then paths can only be established where the DS6000 is the source.
27.3.1 Decoding port IDs
When viewing port IDs in the DS GUI or the DS CLI, the port IDs can be decoded to show
which physical port on the DS8000 or ESS 800 is in use. For DS6000, port IDs range from
I0000 to I0001 and I0100 to I0103. These break out as IEECP. The EE is the controller
number (00 or 01), C is the card slot number (always zero), and P is the port number on the
card (0 to 3). So I0103 is controller 1, card slot 0, port 3.
The ESS 800 port IDs do not follow the same rule. To decode them, take the last two digits in
the port ID and then use Figure 27-4. So, port ID I0020 is the adapter in host bay 2, slot 1 and
port ID I00AC is the adapter in host bay 4, slot 4.
00
04
08
Slot 1 Slot 2 Slot 3
0C
Slot 4
20
24
28
Slot 1 Slot 2 Slot 3
Host Bay 1
2C
Slot 4
Host Bay 2
80
84
88
Slot 1 Slot 2 Slot 3
Host Bay 3
8C
Slot 4
A0
A4
A8
Slot 1 Slot 2 Slot 3
AC
Slot 4
Host Bay 4
Figure 27-4 ESS 800 I/O ports decoded
27.3.2 Path creation using the DS GUI
In this example, we show how to establish two PPRC paths with the DS Storage Manager
from DS6000 LSS 18 to ESS LSS 10.
Tip: Open systems LSSs on an ESS 800 are always LSS 10 to LSS 1F. If you select ESS
800 LSS 00 to ESS 800 LSS 0F, you are working with System z LSSs on the ESS 800.
The steps are:
1. Click Real-time manager.
2. Click Copy Services.
3. Click Paths.
4. Select the Storage Complex, Storage Unit, and Storage Image from which you want to
create PPRC paths. For this path, this machine provides the source LSS.
5. Click Create; see Figure 27-5 on page 397.
396
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 27-5 Path panel
6. You are now prompted to select the source LSS of the DS6000 from which you want to
establish the PPRC paths. You then click Next. In this example, we want to establish
PPRC paths from LSS 18. See Figure 27-6.
Figure 27-6 Select source LSS
7. Now, you have to select the target LSS. First, select in the Storage Complex pull-down
menu the Storage Complex for the ESS. Then select from the Storage Unit pull-down
menu the appropriate Storage Unit and the LSS from the Storage Unit. When you are
finished, click Next to continue. In Figure 27-7, the target LSS on the ESS is LSS 10.
Figure 27-7 Select target LSS
8. The next panel shows you which Fibre Channel ports are available to establish Metro
Mirror and Copy paths; see Figure 27-8 on page 398. Select the ports and click Next.
Since we want to establish two paths from the DS6000 to the ESS, we have to select both
Chapter 27. Interoperability between DS6000 and ESS 800
397
I/O ports from the DS6000 that are available for PPRC. You only see physical connections
that actually exist. To get connections listed, the HBAs in the DS6000 and ESS 800 have
to either be zoned together or directly connected.
Figure 27-8 Select source I/O ports
9. In the next panel (see Figure 27-9), you have to select for each source I/O port from the
DS6000 a target I/O port on the ESS. When finished, click Next.
Figure 27-9 Select target I/O ports
10.On the next two panels, you are asked whether you want a Consistency Group built and to
verify the information you entered during the process. After you verify the information you
entered, click Finish to establish the PPRC paths.
398
IBM System Storage DS6000 Series: Copy Services in Open Environments
11.Figure 27-10 shows the two PPRC paths we have established for LSS 18. To manage the
established paths (for example, delete path), you can select the path you want to manage
and then select from the pull-down menu the appropriate action you want to perform.
Figure 27-10 Path panel
Paths in the reverse direction
The previous example showed how to create a path from the DS6000 to the ESS 800. In
many cases, you might need paths from the ESS 800 to the DS6000. Regardless, having
established paths in one direction, you can then establish paths in the opposite direction.
Fibre Channel allows for bidirectional mirroring over the same physical path.
Adding or deleting paths
You can add additional paths to an LSS pair by simply creating more paths. To remove a
path, just display the Paths panel, select the relevant source machine and LSS, select the
path you want to delete, and use the delete option from the Select Action pull-down.
27.3.3 Establish logical paths between DS6000 and ESS 800 using the DS CLI
You can also use the DS CLI to establish logical paths. However, you must know the World
Wide Node Name (WWNN) of the target Storage Image. The three commands you can use to
establish a path are: lssi, lsavailpprcport, and mkpprcpath.
Determining the remote device WWNN
First, we need to determine the WWNN of the remote storage device. If you started the DS
CLI by connecting to the DS6000 SMC, then the remote storage device is the ESS 800. If you
started the DS CLI by connecting to the ESS 800, then the remote Storage Image is the
DS6000.
Determining the DS6000 WWNN using DS GUI
The steps are:
1. Click Real-time manager.
2. Click Manage hardware.
3. Click Storage units.
Chapter 27. Interoperability between DS6000 and ESS 800
399
4. Click in the Select column for the DS6000 Storage Unit, and from the Select Action
pull-down, choose Properties.
5. The DS6000 Storage Unit WWNN displays on the subsequent properties panel.
Determining DS6000 WWNN using the DS CLI
In Example 27-6, we show how to display the WWNN of a Storage Image using the lssi
command.
Example 27-6 Determine the WWNN of a DS6000
dscli> lssi
Date/Time: 3 November 2005 1:05:50 IBM DSCLI Version: 5.1.0.204
Name ID
Storage Unit
Model WWNN
State ESSNet
============================================================================
IBM.1750-1300247 IBM.1750-1300247 511 500507630EFFFE16 Online Enabled
Determining the WWNN of the ESS 800 using the ESS Specialist
The steps are:
1. Point your browser at the IP address of either cluster of the ESS 800.
2. On the opening panel, click ESS Specialist.
3. You receive a log on prompt. Log on to the ESS Specialist using an ESS Specialist user
ID and password.
4. On the welcome panel, you see the WWNN as shown in Figure 27-11. You need to write it
down.
Figure 27-11 Using ESS 800 Specialist GUI to display the ESS 800 WWNN
Determining the WWNN of the ESS 800 using the ESS CLI
If you have the ESS CLI installed on a personal computer, then you can use the list server
command to display the ESS 800 WWNN.
Note: This is not the DS CLI. ESS CLI is a separate software package that you can get
from your IBM Service Representative if you do not already have it.
Example 27-7 shows an example of the command syntax. This technique has the advantage
that you can copy and paste the output.
Example 27-7 Using ESS CLI to display the ESS 800 WWNN
C:\Program Files\ibm\ESScli>esscli -u storwatch -p specialist -s 10.0.0.1 list server
400
IBM System Storage DS6000 Series: Copy Services in Open Environments
Tue Nov 01 03:28:59 EST 2005 IBM ESSCLI 2.4.0
Server
Model Mfg
WWN
CodeEC
Cache
NVS Racks
----------------------------------------------------------------------2105.22399
800
013 5005076300C09517 2.4.3.79 32GB
2GB
1
Listing the available ports using the DS CLI
Having determined the remote device’s WWNN, we can now display the ports that are
available to establish PPRC paths. In Example 27-8, we are logged on to the ESS 800 using
the DS CLI, so the remote device is the DS6000.
Example 27-8 Displaying available ports for PPRC path establishment
dscli> lsavailpprcport -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 10:18
Date/Time: 3 November 2005 1:11:43 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
Local Port Attached Port Type
=============================
I000C
I0103
FCP
I00AC
I0003
FCP
Note: When issuing commands that refer to an ESS 800, the ESS 800 serial number is
only five digits, not seven like the DS6000 or DS8000. So in our examples, the serial
number syntax we use is IBM.2105-22399, not IBM.2105-1322399.
Establishing the logical paths using the DS CLI
Having determined which ports are available for each LSS pair that you want to copy and
mirror between, you now establish a one-way path with the mkpprcpath command. You can
then display established paths using the lspprcpath command.
In Example 27-9, we first establish a single path between LSS 12 on the DS6000 and LSS 15
on the ESS 800. We then display the paths available for LSS 12 on the DS6000.
Example 27-9 Using the DS CLI to establish PPRC paths
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 10
-tgtlss 18 I000C:I0103
Date/Time: 3 November 2005 1:13:54 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00149I mkpprcpath: Remote Mirror and Copy path 10:18 successfully established.
dscli> lspprcpath 10
Date/Time: 3 November 2005 1:14:07 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
10 18 Success FF18 I000C I0103
500507630EFFFE16
In Example 27-10, we add an additional path between LSS 18 on the DS6000 and LSS 10 on
the ESS 800. Note that we include the existing path when creating a new path. Otherwise, the
existing path is removed and only the new path is available for use.
Example 27-10 Establishing additional paths using the DS CLI
dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 10
-tgtlss 18 I000C:I0103 I00AC:I0003
Date/Time: 3 November 2005 1:15:22 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00149I mkpprcpath: Remote Mirror and Copy path 10:18 successfully established.
dscli> lspprcpath 10
Chapter 27. Interoperability between DS6000 and ESS 800
401
Date/Time: 3 November 2005 1:15:56 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
Src Tgt State SS
Port Attached Port Tgt WWNN
=========================================================
10 18 Success FF18 I00AC I0003
500507630EFFFE16
10 18 Success FF18 I000C I0103
500507630EFFFE16
To establish paths where the DS6000 is the source, we should connect to the DS6000 using
the DS CLI and following the same process, but specifying the ESS 800 as the remote
device.
Important: When you connect using DS CLI to the DS6000 SMC, the DS6000 is the local
device, and the ESS 800 is the remote device. If you connect using DS CLI to the ESS
800, then ESS 800 is now the local device, and the DS6000 is the remote device.
Note: The rmpprcpath command removes all paths between the source and target LSSs.
To just reduce the path count, use the mkpprcpath command and specify only the paths
that you want to continue using.
27.4 Managing Metro Mirror or Global Copy pairs
Having established paths, we can now establish volume pairs.
27.4.1 Managing Metro Mirror or Global Copy pairs using the DS GUI
In this example, we show how you can establish Metro Mirror volume pairs between a
DS6000 and an ESS 800 with the DS Storage Manager. In this example, we create two Metro
Mirror pairs. Volumes 1000 and 1001 from the ESS 800 are the source volumes, and the
target volumes are 1800 and 1801 from the DS6000. We follow the same method to set up a
Global Copy pair, except that Global Copy is selected in Figure 27-17 on page 405.
The steps are:
1. Click Real-time manager.
2. Click Copy services.
3. Click Metro Mirror.
4. Select the Storage Complex, the Storage Unit, and the Storage Image where the Metro
Mirror source volumes are.
5. Click Create; see Figure 27-12.
Figure 27-12 Metro Mirror
402
IBM System Storage DS6000 Series: Copy Services in Open Environments
The next panels guide you through the process of creating Metro Mirror volume pairs. In
the last panel, you have the opportunity to review the information you entered before you
establish the Metro Mirror pairs. Also, during the process, you can, at any time, go back to
modify specifications that you have already made, or you can cancel the process.
6. You have the option to choose between Automated volume pair assignment and
Manual volume pair assignment; see Figure 27-13. If you click the Automated volume
pair assignment, the first selected source volume is paired automatically with the first
selected target volume. If you click Manual volume pair assignment, you must select
each specific target volume for each selected source volume. In this example, we assign
the volume pairs manually.
Figure 27-13 Volume Pairing Method
7. The source volumes we want to use are on the ESS 800 in LSS 10. Select both volumes
and click Next; see Figure 27-14. Note, you also have the option from this panel to create
paths.
Figure 27-14 Select source volumes
8. In the next panel, select the Storage Complex and Storage Unit for the LSS with the target
volumes. Then, select the target volume for the first source volume. When you are
finished, click Next as shown in Figure 27-15 on page 404. To expand your choices, you
must select the small blue boxes to the left of the LSSs you want.
Chapter 27. Interoperability between DS6000 and ESS 800
403
Figure 27-15 First Metro Mirror target volume
9. Next, select the target volume for the second source volume as shown in Figure 27-16 on
page 405. To expand your choices, you must select the small blue boxes to the left of the
LSSs you want.
404
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 27-16 Second Metro Mirror target volume
10.In the next panel, you can specify various copy options as shown in Figure 27-17. In this
example, we selected Metro Mirror under Define relationship type and Perform initial
copy. If you want to use Global Copy instead, this is where you would select it.
Figure 27-17 Select copy options
11.The Verify panel opens. Verify the information that you entered, and if everything is
correct, click Finish to establish the Metro Mirror volume pairs. Figure 27-18 on page 406
shows the Metro Mirror pairs for LSS 06 that we established. To manage the established
Chapter 27. Interoperability between DS6000 and ESS 800
405
pairs (for example, suspend pair), you can select the volume pair you want to manage and
then select the appropriate action that you want to perform from the menu.
Figure 27-18 Managing Metro Mirror pairs
27.4.2 Managing Metro Mirror pairs using the DS CLI
Having established paths, we can now establish volume pairs. In this example, we show how
you can establish Metro Mirror volume pairs between a DS6000 and an ESS 800 with the DS
CLI. In this example, we create two Metro Mirror pairs. Volumes 1800 and 1801 from the
DS6000 are the source volumes, and the target volumes on the ESS are 1000 and 1001.
In Example 27-11, we create two pairs using the mkpprc command. We then list them using
the lspprc command. We then remove one pair using the rmpprc command.
Example 27-11 Creating Metro Mirror pairs using the DS CLI
dscli> mkpprc -remotedev IBM.2105-22399 -type mmir 1800:1000 1801:1001
Date/Time: 3 November 2005 2:32:08 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1800:1000 successfully
created.
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1801:1001 successfully
created.
dscli> lspprc 1800-1801
Date/Time: 3 November 2005 2:32:26 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First
Pass Status
===========================================================================================
========
1800:1000 Copy Pending Metro Mirror 18
unknown
Disabled
Invalid
1801:1001 Copy Pending Metro Mirror 18
unknown
Disabled
Invalid
dscli> rmpprc -remotedev IBM.2105-22399 1800:1000
Date/Time: 3 November 2005 2:33:46 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair
relationship 1800:1000:? [y/n]: y
CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1800:1000 relationship successfully
withdrawn.
In Example 27-11, we connected to the DS6000 SMC using the DS CLI, so the ESS 800 is
the remote device. If we want to establish pairs where the ESS 800 is the source device, we
need to connect to the ESS 800 using the DS CLI. This makes the DS6000 the remote
device.
406
IBM System Storage DS6000 Series: Copy Services in Open Environments
Important: If you specify the wrong -remotedev, or if you are using a profile where a
different remote device is specified than the one you intended to work on, you might get an
error message, CMUN03057, saying you are specifying an invalid subsystem ID. This might
occur because you are specifying the wrong remote device serial number. If you have
multiple potential remote devices, do not specify a remotedev in your DS CLI profile.
27.4.3 Managing Global Copy pairs using the DS CLI
Having established paths, we can now establish volume pairs. In this example, we show how
you can establish Global Copy volume pairs between a DS6000 and an ESS 800 with the DS
CLI. In this example, we create two Global Copy pairs. Volumes 1401 and 1402 from the
DS6000 are the target volumes, and the source volumes on the ESS are 1000 and 1001.
In Example 27-11 on page 406, we create two pairs using the mkpprc command. We then list
them using the lspprc command. We then pause one pair using the pausepprc command.
Example 27-12 Using the DS CLI to manage Global Copy
dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 1000:1800
Date/Time: 3 November 2005 3:07:05 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1000:1800 successfully
created.
dscli>
dscli> lspprc 1000
Date/Time: 3 November 2005 3:07:13 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass
===========================================================================================
1000:1800 Copy Pending Global Copy 10
unknown
Disabled
dscli> pausepprc -remotedev IBM.1750-1300247 1000:1800
Date/Time: 3 November 2005 3:08:13 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00157I pausepprc: Remote Mirror and Copy volume pair 1000:1800 relationship
successfully paused.
dscli> lspprc 1000
Date/Time: 3 November 2005 3:08:19 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
ID
State
Reason
Type
SourceLSS Timeout (secs) Critical Mode First
===========================================================================================
1000:1800 Suspended Host Source Global Copy 10
unknown
Disabled
True
27.5 Managing ESS 800 Global Mirror
The establishment of Global Mirror between a DS6000 and an ESS 800 is achieved using the
same methods as explained in Chapter 24, “Global Mirror examples” on page 303.
27.5.1 Managing Global Mirror pairs using the DS CLI
In Example 27-13, we establish a Global Copy pair between volume 1000 on the ESS 800 and
volume 1800 on the DS6000. We then create a FlashCopy on the remote device. We then
create session 20 for LSS 18. We then create a Global Mirror using session 20 and
LSS 18.
Example 27-13 Establishing Global Copy using the DS CLI
dscli> lspprcpath 18
Date/Time: 3 November 2005 3:47:59 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
Src Tgt State SS
Port Attached Port Tgt WWNN
Chapter 27. Interoperability between DS6000 and ESS 800
407
=========================================================
18 10 Success FF16 I0003 I00AC
5005076300C09517
18 10 Success FF16 I0103 I000C
5005076300C09517
dscli> lspprc 1800
Date/Time: 3 November 2005 3:48:07 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00234I lspprc: No Remote Mirror and Copy found.
dscli> mkpprc -remotedev IBM.2105-22399 -type gcp -tgtread -mode full 1800:1000
Date/Time: 3 November 2005 3:48:39 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1800:1000 successfully created.
dscli> lspprc 1800
Date/Time: 3 November 2005 3:49:01 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
State
Reason Type
SourceLSS Timeout (secs) Critical Mode First Pass Status
==================================================================================================
1800:1000 Copy Pending Global Copy 18
unknown
Disabled
True
dscli> mkremoteflash -dev IBM.2105-22399 -nocp -record -conduit IBM.1750-1300247/18 1000:1001
Date/Time: 3 November 2005 3:50:23 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1000:1001 successfully created. Use the
lsremoteflash command to determine copy completion.
dscli> lsremoteflash -dev IBM.2105-22399 -conduit IBM.1750-1300247/18 1000:1001
Date/Time: 3 November 2005 3:50:38 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled
===========================================================================================================
1000:1001 10
0
Disabled
Enabled
Enabled
Disabled Enabled
dscli> mksession -lss 18 -volume 1800 20
Date/Time: 3 November 2005 3:51:48 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00145I mksession: Session 20 opened successfully.
dscli> lssession 18
Date/Time: 3 November 2005 3:52:06 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
LSS ID Session Status Volume VolumeStatus PrimaryStatus
SecondaryStatus
FirstPassComplete
===========================================================================================================
18
20
Normal 1800
Join Pending Primary Copy Pending Secondary Simplex True
dscli> mkgmir -lss 18 -session 20
Date/Time: 3 November 2005 3:52:39 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00162I mkgmir: Global Mirror for session 20 successfully started.
dscli> showgmir 18
Date/Time: 3 November 2005 3:52:52 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
ID
IBM.1750-1300247/18
Master Count
1
Master Session ID
0x20
Copy State
Running
Fatal Reason
Not Fatal
CG Interval (seconds)
0
XDC Interval(milliseconds) 50
CG Drain Time (seconds)
30
Current Time
11/03/2005 05:49:58 EST
CG Time
11/03/2005 05:49:58 EST
Successful CG Percentage
100
FlashCopy Sequence Number 0x43690A56
Master ID
IBM.1750-1300247
Subordinate Count
0
Master/Subordinate Assoc
-
27.6 Managing ESS 800 FlashCopy
If there is a FlashCopy license for the ESS 800, it is possible to manage ESS FlashCopy
using the DS CLI or SM GUI. The establishment of a FlashCopy pair on an ESS 800, using
the DS GUI, is no different than establishing a DS6000 FlashCopy pair. Refer to 8.5,
“FlashCopy management using the DS SM” on page 82. The establishment of a FlashCopy
408
IBM System Storage DS6000 Series: Copy Services in Open Environments
pair on an ESS 800 using the DS CLI is also no different, refer to 8.3, “Local FlashCopy using
the DS CLI” on page 68. In a situation where the ESS 800 is the target device in a PPRC
relationship, it is also possible to create remote FlashCopies on the ESS 800 using the
mkremoteflash command. In each case, creating, changing, and removing the FlashCopy is
done using the same commands.
27.6.1 Creating an ESS 800 FlashCopy using DS GUI
The steps to create an ESS 800 FlashCopy are:
1. Click Real-time manager.
2. Click Copy Services.
3. Click FlashCopy.
4. From the Storage Complex menu, choose the ESS 800 Copy Services Domain.
5. From the Storage Units menu, select the ESS on which you want to perform the
FlashCopy.
6. Make selections from the Resource type and Specify LSS pull-downs.
7. From the Select Action menu, select Create, as shown in Figure 27-19.
Figure 27-19 Creating ESS 800 FlashCopy using DS GUI
8. Choose between A single source with a single target or A single source with multiple
targets, and click Next.
9. Select the source volumes, as shown in Figure 27-20 on page 410. Note that some of the
columns have Not Applicable for 2105 in them. This is normal.
Chapter 27. Interoperability between DS6000 and ESS 800
409
Figure 27-20 Selecting ESS 800 source volumes for FlashCopy
10.You select the target volumes and click Next.
11.You select what options you plan to use and click Next.
12.You get a verification panel, review your selections and click Finish.
27.6.2 Creating an ESS 800 FlashCopy using the DS CLI
You need to start the DS CLI and connect to the ESS 800. Then, issue DS CLI commands as
you would normally. An example is shown in Example 27-14.
Example 27-14 Using the DS CLI to create an ESS 800 FlashCopy
dscli> mkflash -nocp -record -persist 1001:1002
Date/Time: 2 November 2005 20:04:33 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00137I mkflash: FlashCopy pair 1001:1002 successfully created.
dscli> lsflash 1001
Date/Time: 2 November 2005 20:04:37 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
ID
SrcLSS SequenceNum Timeout ActiveCopy Recording Persistent Revertible SourceWrite TargetWrite
===========================================================================================================
1001:1002 10
0
45
Disabled
Enabled
Enabled
Disabled
Enabled
Enabled
dscli> rmflash 1001:1002
Date/Time: 2 November 2005 20:04:54 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00144W rmflash: Are you sure you want to remove the FlashCopy pair 1001:1002:? [y/n]:y
CMUC00140I rmflash: FlashCopy pair 1001:1002 successfully removed.
27.6.3 Creating a remote FlashCopy on an ESS 800 using the DS CLI
It is also possible to use the DS CLI to create a remote FlashCopy on an ESS 800 where the
source PPRC device is a DS6000. In Example 27-15, we connect using the DS CLI to a
DS6000. We create a Metro Mirror pair from volume 1801 on the DS6000 to volume 1002 on
the ESS 800. We then create a remote FlashCopy on the ESS 800 between ESS 800
volumes 1002 and 1003. We have to use a conduit LSS to send the command. We then
remove the remote FlashCopy.
Example 27-15 Creating a remote FlashCopy where the ESS 800 is the remote target
dscli> mkpprc -remotedev IBM.2105-22399 -type mmir 1801:1002
Date/Time: 3 November 2005 3:59:22 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247
CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1801:1002 successfully created.
410
IBM System Storage DS6000 Series: Copy Services in Open Environments
dscli> mkremoteflash -dev IBM.2105-22399 -conduit IBM.1750-1300247/18 -persist -nocp 1002:1003
Date/Time: 3 November 2005 4:00:11 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00173I mkremoteflash: Remote FlashCopy volume pair 1002:1003 successfully created. Use the
lsremoteflash command to determine copy completion.
dscli> lsremoteflash -dev IBM.2105-22399 -conduit IBM.1750-1300247/18 1002:1003
Date/Time: 3 November 2005 4:00:24 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
ID
SrcLSS SequenceNum ActiveCopy Recording Persistent Revertible SourceWriteEnabled
===========================================================================================================
1002:1003 10
0
Disabled
Disabled Enabled
Disabled Enabled
dscli> rmremoteflash -dev IBM.2105-22399 -conduit IBM.1750-1300247/18 1002:1003
Date/Time: 3 November 2005 4:01:36 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399
CMUC00179I rmremoteflash: Are you sure you want to remove the remote FlashCopy pair {0}? [y/n]:y
CMUC00180I rmremoteflash: Removal of the remote FlashCopy volume pair 1002:1003 has been initiated
successfully. Use the lsremoteflash command to determine when the relationship is deleted.
You could also reverse the scenario and use the ESS 800 as the source machine in a PPRC
pair and then create the remote FlashCopy on the DS6000.
Note: Commands that work with remote FlashCopies use the -dev parameter to define the
machine on which to perform the FlashCopy. Other commands, such as mkpprc
commands, refer to this device as the remote device, using -remotedev. However, because
a FlashCopy must be sent to the remote site and then performed locally there, the use of
the -dev parameter to refer to the remote machine is correct.
Chapter 27. Interoperability between DS6000 and ESS 800
411
412
IBM System Storage DS6000 Series: Copy Services in Open Environments
Part 8
Part
8
Solutions
In this part, we discuss solutions offered by IBM to assist you in the management,
automation, and control of your Copy Services implementation on the DS6000. We provide
an overview of the solutions, discuss the options available, and give some examples of using
the solutions.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
413
414
IBM System Storage DS6000 Series: Copy Services in Open Environments
28
Chapter 28.
IBM TotalStorage Productivity
Center for Replication
The IBM TotalStorage Productivity Center for Replication is an automated solution to provide
a management front end to Copy Services. This chapter describes the IBM strategic solution
to manage disaster recovery solutions based on Copy Services functions in DS6000 and
DS8000 storage servers. For details, refer to the IBM Redbook and product manuals listed
below.
In addition to the DS6000 and DS8000, the IBM TotalStorage Productivity Center for
Replication (TPC for Replication) can also help you manage Copy Services for the SAN
Volume Controller, as well as the ESS 800, when using Fibre Channel Protocol (FCP) links
for PPRC paths between ESS 800s or between ESS 800 and DS6000 or DS8000. This also
applies to FlashCopy.
For installation and configuration detail, refer to Replication Management with IBM
TotalStorage Productivity Center, SG24-7259.
The following manuals are mandatory to have at hand when implementing and managing
TPC for Replication configurations:
򐂰 IBM TotalStorage Productivity Center for Replication User’s Guide, SC32-0103
򐂰 IBM TotalStorage Productivity Center for Replication Installation and Configuration Guide,
SC32-0102
򐂰 IBM TotalStorage Productivity Center for Replication Command-Line Interface User’s
Guide, SC32-0104
Note: The IBM TotalStorage Productivity Center for Replication is completely independent
from the TotalStorage Productivity Center for Disk, Data and Fabric (TPC for Disk, Data
and Fabric).
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
415
28.1 IBM TotalStorage Productivity Center
IBM TotalStorage Productivity Center (TPC) is a suite of software products designed to
support clients in monitoring and managing their storage environments.
Design and development emphasis for TPC is on scalability and standards. The approach
based on open standards allows TPC to manage any equipment or solution implementation
that follows the same open standards.
Figure 28-1 shows a split set of products with three components based on former Tivoli
products and a package with two components designed and developed to facilitate storage
replication solutions.
IBM TotalStorage Productivity Center (TPC)
Standard Edtion
TPC for Disk
TPC for Data
Replication
TPC for Fabric
Replication
Replication
Two Site BC
BC Business Continuity
Figure 28-1 TPC software products suite
All three Tivoli products provide a single source and single management tool to cover the
tasks of the storage manager or network manager in their daily business:
򐂰 TPC for Disk is software based on open standard interfaces to query, gather, and collect
all available data necessary for performance management.
򐂰 TPC for Data focuses on data management and addresses aspects related to information
life cycle management.
򐂰 TPC for Fabric is a management tool to monitor and manage a SAN fabric.
TPC for Replication, as a member of the TPC product family, is a stand-alone package and
does not build on anything related to the TPC Standard Edition.
There are two versions of TPC for Replication:
򐂰 TPC for Replication includes support for:
– FlashCopy
– Planned Failover and restart (one direction) for MM and GM
򐂰 TPC for Replication Two Site Business Continuity (BC) includes support for:
– FlashCopy
– Planned and unplanned Failover and Failback for MM and GM
– High availability (two TPC Replication servers)
In the following paragraphs, we cover the basic structures and features of TPC for Replication
and TPC for Replication Two Site BC.
This explanation and discussion are related to Copy Services topics with the DS6000 and
DS8000 only.
416
IBM System Storage DS6000 Series: Copy Services in Open Environments
28.2 Where we came from
Since the advent of Copy Services functions with IBM storage servers, a framework is
required to handle and manage disk storage environments and the various combination of
software, firmware, and hardware used for replication.
To ensure and guarantee data consistency at the remote or backup site, it is even more
important to provide a framework around Copy Services functions, which helps achieve that
data consistency and ease of management.
Other aspects are automation and scalability. Early solution attempts turned out to have
weaknesses that often would surface during actual implementation in real production
environments. Tools offered on an “as-is” basis are not suitable for managing production
environments. Other solutions are perfectly suited for certain host server platforms but cannot
manage other server platforms.
This led to the design and implementation of a framework, which is platform independent and
provides the required production attributes, such as scalability, stability, and security and also
provides as a standard product offering the guarantee to be serviced and maintained. The
concept, furthermore, has the potential for enhancements and can evolve over time and
according to user requirements.
TPC for Replication does build on all previous experience with storage management tools
and framework proposals to organize all aspects of Copy Services for disaster recovery
solutions. TPC for Replication also addresses the need for other solutions, which involve
Copy Services functions available with the DS6000 and DS8000, such as data or volume
migration, data center movements, or other projects, which require copying or moving data
between like devices, such as the DS6000, DS8000, and ESS 800.
28.3 What TPC for Replication provides
TPC for Replication is designed to help administrators manage Copy Services. This applies
not only to the Copy Services provided by DS6000 and DS8000 but also to Copy Services
provided by the ESS 800 and SAN Volume Controller.
The highlights of functions provided by TPC for replication are:
򐂰 Manage Copy Services for the IBM System Storage DS6000 and DS8000.
򐂰 Extend support for ESS 800 to manage Copy Services.
򐂰 Provide disaster recovery support with Failover and Failback capability for the:
– ESS 800
– DS6000
– DS8000
򐂰 Monitor Copy Services performance.
򐂰 The optional high availability feature allows you to continue replication management even
when one TPC for Replication server goes down.
The basic functions of TPC for Replication provide management of:
򐂰 FlashCopy
򐂰 Metro Mirror
򐂰 Global Mirror
Chapter 28. IBM TotalStorage Productivity Center for Replication
417
Note that TPC for Replication does not support Global Copy.
Figure 28-2 is an actual screen capture from the TPC for Replication GUI, showing the
different session types, or Copy Services functions supported.
Figure 28-2 Management of Copy Services Functions supported by TPC for Replication
TPC for Replication is designed to simplify management of Copy Services by:
򐂰 Automating administration and configuration of Copy Services functions with a
wizard-based session and copy set definitions
򐂰 Provide simple operational control of Copy Services tasks, which includes starting,
suspending, and resuming Copy Services Tasks
򐂰 Offer tools to monitor and manage Copy Services sessions
Note that TPC for Replication also manages FlashCopy and Metro Mirror for IBM SAN
Volume Controller.
28.4 Copy Services terminology
Although we have discussed terminology in a previous part of this book, we have included a
brief review of Copy Services terminology here for convenience and to keep it in context with
TPC for Replication.
28.4.1 FlashCopy
FlashCopy is a point-in-time copy of a source volume mapped to a target volume. It offers
various options including the choice between a COPY option for a background copy through
physical I/O operations in the storage server back-end, or a NOCOPY option.
FlashCopy is available on the DS6000 and DS8000, as well as on the ESS 800. FlashCopy is
also available on the DS4000 family and with SAN Volume Controller. Note, however, that the
DS400 and San Volume Controller use different implementations of the FlashCopy function,
which are not compatible with the DS6000, DS8000, and ESS 800 FlashCopy.
418
IBM System Storage DS6000 Series: Copy Services in Open Environments
28.4.2 Metro Mirror
Metro Mirror (MM) is a synchronous data replication mechanism and was previously called
Peer-to-Peer Remote copy (PPRC). I/O completion is signaled when the data is secured on
both the local storage server and the remote storage server. Metro Mirror is popular for two
site disaster recovery solutions.
Metro Mirror is available in any combination between DS6000, DS8000, and ESS 800.
Synchronous data replication is also available on the DS4000 family and with the San
Volume Controller, but again not directly compatible with Metro Mirror on the DS6000,
DS8000, and ESS 800.
28.4.3 Global Copy
Global Copy (GC) is an asynchronous data replication mechanism that used to be called
PPRC for eXtended Distance (PPRC-XD). Asynchronous means in this context that the data
transmission between local and remote storage servers is separated from signaling the I/O
completion. Note that GC does not guarantee that the arriving writes at the local site are
applied to the remote site in the same sequence. Therefore, it does not provide data
consistency. GC is suited for data migration over any distance.
Global Copy is possible in any combination between DS6000, DS8000, and ESS 800.
Restriction: TPC for Replication does not provide an interface to manage a Global Copy
environment.
28.4.4 Global Mirror
Global Mirror (GM) is an asynchronous data replication over any distance. It is a combination
of GC and FlashCopy complemented by unique firmware features. The firmware drives the
copy operations in a fully autonomic fashion to guarantee consistent data at any time at the
remote site. GM meets the requirement to provide consistent data at any time, which can be
spread over more than a single storage server.
GM is also the base for a two-site disaster recovery solution involving any DS6000, DS8000,
and ESS 800. The GM function is also possible between products of the DS4000 family and
the San Volume Controller.
28.4.5 Failover and Failback terminology
There is often a misconception about what Failover and Failback commands do.
Figure 28-3 on page 420 illustrates the Failover and Failback two-step process. The process
is as follows:
1. When the replication process between Primary (P) and Secondary (S) is suspended as a
result of a planned or unplanned outage, you might want to access the S volumes
immediately without any checking on the P volumes.
The Failover command always points to the S volumes and turns the S volumes from a
SECONDARY state into a PRIMARY SUSPEND state, making the volumes immediately
available to the application. This state change for the S volumes at the remote site
happens without any communication or any checking of the P volumes at the local site.
The P volumes at the local site keep the status they had before the Failover command
was issued.
Chapter 28. IBM TotalStorage Productivity Center for Replication
419
Local DS6000/DS8000
Remote DS6000/DS8000
Suspends due to planned
or unplanned event
A
A
P
A
A
S
Replication direction
Primary
Primary
Primary
Primary
Primary
Primary
Primary
Primary
Primary
Secondary
A
A
A
A
Primary
Primary
Primary
Primary
Primary
Primary
Primary
Primary
1.Failover command
P
P
2.Failback command
PRIMARY SUSPENDED
A
A
SECONDARY PENDING
SECONDARY DUPLEX
OR
2.Failback command
PRIMARY PENDING
PRIMARY DUPLEX
Primary
Primary
Primary
Primary
S
A
A
Primary
Primary
Primary
Primary
P
Resynchronization
PRIMARY PENDING
PRIMARY DUPLEX
Resynchronization
SECONDARY PENDING
SECONDARY DUPLEX
A
A
Primary
Primary
Primary
Primary
P
A
A
Primary
Primary
Primary
Primary
S
Figure 28-3 Failover and Failback terminology
2. Once the outage is over and you decide to resume replicating data, you have a choice
whether to resynchronize from the remote site to the local site or the reverse. This
depends on what is required after updating one of the two sites or even updating both
sites after the suspend action.
a. When you decide to resynchronize from the remote site to the local site, the Failback
command is pointing to the remote volumes. You need a PPRC path from the remote
to the local site, before you can perform the corresponding commands. Doing this
resynchronizes both sites with the level of data as it is at the remote site. Only the data
that changed at the remote site, from the moment the replication was suspended and
until the Failover happened, is replicated from remote to local volumes.
Figure 28-3 assumes a Metro Mirror (MM) environment, which puts the corresponding
MM pair in PENDING state during re-synchronization and when everything is
replicated, the MM pair goes into DUPLEX state.
b. If you want to resynchronize both sites with the data level at the local site, you issue
the Failback command toward the volumes at the local site. This replicates the
changed data since the suspend from local to remote site. Also, this resets the tracks
that were potentially changed at the remote site after the Failover.
Figure 28-3 assumes a MM environment, which places the MM pair in PENDING state
during the re-synchronization phase. Once all changed data is replicated, the MM pair
goes into DUPLEX state.
The option to resynchronize from either site is possible due to the availability of change
bitmaps maintained by the DS6000 or DS8000 at both sites.
420
IBM System Storage DS6000 Series: Copy Services in Open Environments
28.5 TPC for Replication terminology
TPC for Replication manages and integrates not only the DS6000 and DS8000 but also the
SAN Volume Controller. In search of a common terminology and to describe the functions for
different disk storage servers in a common way, new terms, different than those normally
used in the context of Copy Services with the ESS 800, DS6000, and DS8000, are introduced
here.
28.5.1 TPC for Replication copy set
A copy set in TPC for Replication is a set of volumes, which has a copy of the same data. In
PPRC terms, this is, for example, a PPRC primary volume and a PPRC secondary volume.
Figure 28-4 shows three Metro Mirror Copy pairs. In TPC for Replication, each pair is
considered as a copy set. A copy set here contains two volumes.
Local DS6000 / DS8000
A
P3
Primary
Primary
Primary
Remote DS6000 / DS8000
PPRC path
S3
Copy Set
Secondary
PPR link
A
P2
Primary
Primary
Primary
PPRC path
S2
Copy Set
Secondary
PPR link
A
P1
Primary
Primary
PPRC path
Primary
A
S1
Primary
Primary
Primary
Copy Set
Secondary
Figure 28-4 TPC for Replication: Metro Mirror copy sets
Figure 28-5 on page 422 represents two copy sets. In this illustration, each copy set contains
three volumes, which indicates a Global Mirror configuration. Note that the third volume is a
kind of journal volume and is the FlashCopy target volume covering for data consistency in a
Global Mirror setup.
Chapter 28. IBM TotalStorage Productivity Center for Replication
421
Global Copy
A
S2
A
P2
FlashCopy
Primary
Primary
Primary
Primary
Primary
Secondary
Primary
Primary
Copy Set
J2
Tertiary
PPRC links
Global Copy
A
P1
A
S1
Primary
Primary
Primary
Primary
Primary
Secondary
FlashCopy
Primary
Primary
Copy Set
J1
Tertiary
Local site
Remote Site
Figure 28-5 TPC for Replication: Global Mirror copy sets
28.5.2 TPC for Replication session
TPC for Replication uses a session concept, which is similar to what Global Mirror for System
z (XRC) uses.
A session is a logical concept, which gathers multiple copy sets, representing a group of
volumes with the requirement to provide consistent data within the scope of all involved
volumes. Commands and processes performing against a session apply these actions to all
copy sets within the session. Again, this is similar to a Global Mirror for System z (XRC)
session.
Figure 28-6 on page 423 shows an example of two storage servers at the local site and two
corresponding storage servers at the remote site. The example further assumes that a Metro
Mirror relationship is established to replicate data between both sites.
422
IBM System Storage DS6000 Series: Copy Services in Open Environments
Local DS6000 / DS8000
Session
1
A
P3
Primary
Primary
Primary
A
P2
Primary
Primary
Prim ary
Remote DS6000 / DS8000
PPRC path
PPRC path
A
P1
A
PB
Primary
Primary
PPRC path
A
PA
Copy Set
A
S1
Primary
Primary
Primary
Copy Set
Secondary
PPRC path
Primary
Primary
Primary
S2
Remote DS6000 / DS8000
Primary
Session
2
Copy Set
Secondary
Local DS6000 / DS8000
Primary
Primary
S3
Secondary
A
SB
Primary
Primary
Primary
Copy Set
Secondary
PPRC path
Primary
Local site
A
SA
Primary
Primary
Primary
Copy Set
Secondary
Remote Site
Figure 28-6 TPC for Replication: Session concept
Metro Mirror primary volume P1 in the second storage server and volumes P2 and P3 in the
first storage server with their corresponding Metro Mirror secondary volumes are grouped
together in a session, session 1. This session can contain copy sets that span across storage
server boundaries.
Metro Mirror primary volumes PA and PB with their counterparts at the remote site belong to
a different session, session 2.
Note that all application dependent copy sets must belong to the same session to guarantee
successful management and to provide consistent data across all involved volumes within a
session. In this context, we recommend to have application dependent volume granularity
within the scope of an LSS. Or, in other words, volumes or copy sets, which require
consistent data and can be subject to a freeze trigger, ought to be grouped in one or more
LSS and that LSS must not contain other copy sets. This is because the scope of the freeze
function is at the LSS level and affects all volumes within that LSS.
Chapter 28. IBM TotalStorage Productivity Center for Replication
423
28.6 TPC for Replication session types
There are two TPC for Replication editions, which include different levels of Copy Services
functions.
28.6.1 TPC for Replication Basic Edition
TPC for Replication Basic Edition includes FlashCopy and also Metro Mirror and Global
Mirror. However, Metro Mirror and Global Mirror are only configured to replicate data in a
unidirectional fashion.
FlashCopy in the Basic edition
FlashCopy includes all of the functions of FlashCopy. This includes FlashCopy with:
– Background copy or no background copy.
– Convert no background copy to background copy.
– Add persistent to a FlashCopy relationship.
– Establish incremental FlashCopy to apply only changed data since a previous
FlashCopy operation.
Metro Mirror
TPC for Replication Basic Edition includes Metro Mirror between a local site (site 1) and a
remote site (site 2) in a unidirectional fashion. It also allows you to switch the replication
direction from site 2 to site 1. This can happen through Failover/Failback operations.
Global Mirror
TPC for Replication Basic Edition includes Global Mirror. Global Mirror implies three volumes
for each Global Mirror copy set. Again, this happens between a local site (site 1) and a distant
site (site 2) in a unidirectional way. It is possible to reverse the replication direction but still
only in a unidirectional fashion.
28.6.2 TPC for Replication Advanced Edition
The TPC for Replication Advanced Edition has, in addition to the Basic Edition, the capability
to replicate data in either direction or in a bidirectional fashion.
Metro Mirror
The TPC for Replication Advanced Edition Metro Mirror can be established from site 1 to site
2 or the reverse or in both directions at the same time (bidirectional).
It also allows Failover/Failback operations for any copy set at any time.
Global Mirror
The TPC for Replication Advanced Edition allows the Global Mirror to operate in either
direction at the same time (bidirectional). It can also swap sites with Failover/Failback
operations.
Global Mirror builds on three volumes per copy set. TPC for Replication also allows you to
manage a configuration, which only replicates data through Global Copy in the opposite
direction than Global Mirror replicated the data before.
424
IBM System Storage DS6000 Series: Copy Services in Open Environments
28.7 TPC for Replication session states
Again, a session contains a group of copy sets (that is RMC (PPRC) volume pairs or
FlashCopy pairs), which belongs to a certain application. You might also consider it as a
collection of volumes, which belongs to a certain application or system with the requirement
for consistency.
This session might be in one of the following states:
Defined
A session is defined and might already contain copy sets or still have no copy sets assigned
yet. However, a defined session is not yet started.
Preparing
The session has started already and is in the process of initializing, which might be, for
example, a first full initial copy for a Metro Mirror. It might also just reinitialize, which, for
example, might be the case for a re-synchronization of a previously suspended Metro Mirror.
Once the initialization is complete, the session state changes to prepared.
Prepared
All volumes within the specific session have completed the initialization process.
Suspending
This is a transition state caused by either a suspend command or any other suspend trigger,
which might be an error in the storage subsystem or loss of connectivity between sites.
Eventually, the process to suspend copy sets ends and copying has stopped, which is
indicated by a session state of suspended.
Suspended
Replicating data from site 1 to site 2 has stopped. Application writes to specific volumes in
site 1 can continue (Freeze and go). An additional recoverable flag indicates whether the data
is consistent and recoverable.
Recovering
A session is about to recover.
TargetAvailable
The recover command has completed and the target volumes are write-enabled and
available for application I/Os. An additional recoverable flag indicates whether the data is
consistent and recoverable.
Important: Do not manage Copy Service volume pairs, which are already managed
through TPC for Replication, through another software product.
28.8 Volumes in a copy set
With TPC for Replication, the role of a volume within Copy Services has been renamed to be
more generic and also to include other storage subsystems, such as San Volume Controller.
A copy set contains the volumes, which are part of a Copy Services relationship. Metro Mirror
knows two volumes per copy set. Global Mirror requires three volumes per copy set.
FlashCopy again consists of two volumes per copy set.
Chapter 28. IBM TotalStorage Productivity Center for Replication
425
Terminology is slightly different from what you might be used to. For example, Metro Mirror
uses a primary or source volume at the sending site and a secondary or target volume at the
receiving end. Such a pair is now called a copy set.
FlashCopy used to mention source and target volume to identify a FlashCopy pair in contrast
to RMC, which designates the volumes in that pair as primary and secondary volumes. A
FlashCopy volume pair is now a copy set.
Global Mirror involves three volumes per copy set. Global Mirror relies on Global Copy and
FlashCopy. Therefore, we have a Global Copy primary volume and a Global Copy secondary
volume. The Global Copy secondary volume has another role as the FlashCopy source
volume. The third volume is then the FlashCopy target volume. All three volumes are part of a
Global Mirror copy set.
28.8.1 Host volume
A host volume is identical to what is called a primary or source volume in Copy Services. The
host designation represents the volume functional role from an application point of view. It is
usually connected to a host or server and receives read, write, and update application I/Os.
When a host volume becomes the target volume of a Copy Services function, it is usually no
longer reachable from the application host. FlashCopy target volumes can be considered an
exception.
28.8.2 Target volume
A target volume is what was also usually designated as a secondary t volume. It receives
data from a host volume or another intermediate volume. It might also be an intermediate
volume, such as in a Global Mirror copy set.
28.8.3 Journal volume
A journal volume is currently the FlashCopy target volume in a Global Mirror copy set. It is
called a journal volume because it functions like a journal and holds the required data to
reconstruct consistent data at the Global Mirror remote site. When a session needs to be
recovered at the remote site, the journal volume is used to restore data to the last consistency
point.
426
IBM System Storage DS6000 Series: Copy Services in Open Environments
28.9 TPC for Replication and scalability
From an architectural point of view, there is no limit to the number of copy sets which can
belong to a single session. The implementation approach of managing copy sets and
sessions provides the scalability that very large installations require. See Figure 28-7.
Local DS6000 / DS8000
A
Intermediate DS6000 / DS8000
A
Primary
Primary
Primary
Primary
Primary
Primary
A
A
A
Primary
Primary
Primary
Primary
Primary
Primary
A
Remote DS6000 / DS8000
A
Session 1
A
A
Primary
Primary
Primary
Primary
Copy Set
Primary
Primary
A
APrimary
Primary
A
Primary
Primary
Primary
Primary
A
A
Primary
Primary
Primary
Primary
A
Session 2
A
A
Primary
Primary
Primary
Primary
A
A
Primary
Primary
Primary
Primary
A
Primary
Primary
Primary
Primary
A
A Primary
Primary
Primary
APrimary
Primary
Primary
Session 3
A
A Primary
Primary
Primary
APrimary
Primary
Primary
A
Session 4
Primary
Primary
A
Primary
Primary
A
Primary
Primary
Figure 28-7 TPC for Replication: Four sessions
There is also no limit to the number of sessions that TPC for Replication can manage.
Figure 28-7 shows four sessions managing different copy sets. The first session, Session 1,
shows a Metro/Global Mirror configuration, which is not supported (at the time of writing this
redbook) by TPC for Replication.
The second session, Session 2, is a Global Mirror session between an intermediate site and a
remote site with two copy sets.
The third session, Session 3, is a Metro Mirror configuration between the local and
intermediate site.
The session at the bottom in Figure 28-7, Session 4, is a Global Mirror session between the
local and remote sites.
28.10 TPC for Replication system and connectivity overview
TPC for Replication is an outboard approach and its software runs on a dedicated server, or
better on two servers for a high availability configuration.
Chapter 28. IBM TotalStorage Productivity Center for Replication
427
Figure 28-8 shows how the TPC for Replication server connects to the storage servers, which
are going to be managed by TPC for Replication servers. It does not show the connectivity of
the TPC for Replication server to the network through which the user connect with a browser
to the server itself.
Primary
Primary
Database
DB2
Websphere
Replication Manager Server
IP Communication
Services
Windows
Linux
AIX
IP network
Series p
Series p
PowerPC
PowerPC
FCP ports
A
Primary
APrimary
A Primary
Primary
Copy Set
Primary
Primary
Ethernet ports
A
A
A
Primary
Primary
Copy Set
Primary
Primary
Primary
Primary
Primary
Primary
Primary
Primary
A
A
Primary
APrimary
PPR FCP links
A
A
Primary
Primary
Primary
Primary
A
A Primary
Primary
Primary
Primary
DS8000
A
A
Primary
Primary
Primary
Primary
Primary
Primary
A
Primary
A Primary
Primary
Primary
A
A
A
Primary
Primary
Primary
Primary
DS6000
Figure 28-8 TPC for Replication system overview
The TPC for Replication server contains, besides the actual functional code, also DB2 UDB
and WebSphere® software.
IP communication services also contain an event listening capability to react on respective
trap messages from the storage servers. This provides a handle to plan for particular events
and provide preestablished scripts, which are going to be triggered by corresponding trap
messages. This includes a capability to distinguish between the different storage servers that
can be managed by the Replication Manager, such as DS6000, DS8000, ESS 800, and San
Volume Controller. This approach has the potential to be enhanced as storage servers
change over time without touching the actual functional code and the database that is
involved.
Figure 28-9 on page 429 shows how the Replication Manager server connects to the DS6000
and DS8000.
428
IBM System Storage DS6000 Series: Copy Services in Open Environments
SMC
Replication Manager Server
IP network
Ethernet ports
server0
System p
server1
PowerPC
PowerPC
server0
server1
DS6000
System p
DS8000
Figure 28-9 Replication Manager server connectivity to DS6000 and DS8000
The actual connectivity between the TPC for Replication server and the storage servers is
based on Ethernet networks and connects to particular Ethernet ports in the System p™ in
the DS8000. This particular Ethernet card is a new card and slides into the first slot out of
these four slots in the p570. Note that Figure 28-9 shows the DS8000 rear view, because
these slots are only reachable from the rear site of the DS8000.
TPC for Replication server connectivity to the DS6000
For the DS6000, the Replication Manager server connects to the network, which connects to
the PowerPC® servers in the DS6000. For the DS6000, this is the same network to which the
Storage Management console connects, because the DS6000 controller or server card
contains only a single Ethernet port.
For the DS6000, there is only a single Ethernet port per cluster or server card. This port is
shared between the Replication Manager server or servers as well as with one or two Storage
Management consoles.
TPC for Replication configuration options
Note that Figure 28-9 outlines the basic connectivity idea. For high availability, you might
consider a second Replication Manager server, which also connects to the same IP network
as the first server. Currently, the Replication Manager server connects only to one Ethernet
port out of the two ports on the Ethernet card that must reside in the first slot of the System p
server in the DS8000.
Note that the internal and potential external HMCs connect to the DS8000 through different
ports than the Replication Manager servers.
The communication between the Replication Manager Server and the DS infrastructure is
direct as shown in Figure 28-9. It is interesting to note that this is different than when the
Replication Manager communicates with the San Volume Controller. Between the Replication
Chapter 28. IBM TotalStorage Productivity Center for Replication
429
Manager Server and the San Volume Controller Nodes is a San Volume Controller
CIMON-based console, which is part of the standard San Volume Controller Master console.
For more details, refer to Replication Management with IBM TotalStorage Productivity
Center, SG24-7259.
28.11 TPC for Replication monitoring and freeze capability
TPC for Replication always uses the Consistency Group attribute when you define PPRC
paths for Metro Mirror between a primary and a secondary storage server. This provides TPC
for Replication with the capability to freeze a Metro Mirror configuration to guarantee
consistent data at the secondary or backup site when an incident happens. See Figure 28-10.
Replication Manager Server
2
1
FREEZE
3
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
Primaries
Session
Secondaries
Figure 28-10 TPC for Replication server freeze
TPC for Replication server listens to incidents from the storage servers and takes action
when notified of a replication error from the storage server.
Figure 28-10 shows a replication error in an LSS that belongs to the session. The TPC server
receives a corresponding SNMP trap message from the storage server. TPC for Replication
server then issues a freeze command to all LSSs which are part of the session. This shows a
suspend of all PPRC pairs or copy sets that belong to this session. During this freeze
process, write I/Os are held until the freeze process ends and the TPC server communicates
to the storage server to continue processing write I/O requests to the primary volumes in the
session.
430
IBM System Storage DS6000 Series: Copy Services in Open Environments
After that, write I/O can continue to the suspended primary volumes. However, both sites are
not in sync any longer but the data on the secondary site is consistent (power drop
consistent). This is a so-called freeze-and-go policy.
28.12 TPC for Replication heartbeat
Because the connectivity between the TPC for Replication server and the storage servers
that the TPC server is managing can fail, the firmware in the storage server waits for a
heartbeat signal from the TPC server. TPC for Replication can enable this heartbeat in the
corresponding LSS for Metro Mirror sessions. See Figure 28-11.
Replication Manager Server
Ethernet ports
1
FCP ports
2 FREEZE
LSS
LSS
LSS
PPRC FCP links
LSS
LSS
Primaries
LSS
Session
Secondaries
Figure 28-11 LSS heartbeat triggers freeze when connectivity to server fails
Figure 28-11 shows failing connectivity between the Replication Manager server and a
primary storage server. Once this heartbeat is set and the storage subsystem cannot
communicate its heartbeat information to the Replication Manager server the storage server
internally triggers a freeze to its involved LSSs and their primary volumes. Because this
heartbeat is a timer-scheduled function, it is based on the Consistency Group time-out value
of each LSS that contains volumes that belong to the session. This session-based heartbeat
timer expires after the lowest time-out value of all the LSSs.
With more than a storage server involved, the Replication Manager server issues freeze
commands to all other LSS pairs in the affected session, once the heartbeat expires and the
RM server could not receive heartbeat information from any one of the involved storage
servers.
The disconnected storage server or LSS resumes I/O after their Consistency Group time-out
has expired. LSSs or storage servers, which might still be connected to the Replication
Manager server receive a corresponding freeze command from the Replication Manager
(RM) server followed by a run command when a freeze-and-go policy has been selected.
Chapter 28. IBM TotalStorage Productivity Center for Replication
431
Note that Extended Long Busy (ELB), automation window, and freeze period are synonyms
for Consistency Group time-out in the above context.
28.13 Supported platforms
TPC for Replication server can currently run under the following operating systems:
򐂰 Windows 2003 Server Edition with SP1
򐂰 Windows 2003 Enterprise Edition SP1
Figure 28-12 Windows 2003 Software level
򐂰 SUSE Linux Enterprise Server 9 SP2
Note: SUSE Linux does not support the Two-Site BC configuration.
򐂰 Red Hat Enterprise Linux RHEL4 AS 2.1
For Replication Two-Site BC: Red Hat Enterprise Linux RHEL4 Update 1 SLES9 SP2.
򐂰 AIX 5.3 ML3
Note: For a TPC for Replication Two-Site BC configuration, which involves two servers, it is
possible to run TPC for Replication under two different operating system platforms.
For the latest software requirements, refer to:
http://www.ibm.com/servers/storage/support/software/tpcrep/installing.html
28.14 Hardware requirements for TPC for Replication servers
For Windows and Linux, we suggest that you use the following minimum hardware
configuration:
򐂰 1.5 GHz Intel® Pentium III processor
򐂰 2 GB RAM memory
򐂰 10 GB free disk space
432
IBM System Storage DS6000 Series: Copy Services in Open Environments
When TPC for Replication runs on AIX, we suggest the following minimum hardware
configuration:
򐂰 Server p, IBM POWER4™ or IBM POWER5™ processor, 1 GHz
򐂰 2 GB RAM memory
򐂰 10 GB free disk space
Disk space is required to hold for data in a DB2 database and WebSphere Express
Application Server code is required in addition to the actual TPC for Replication server code.
You can refer to the following Web site for the latest information about hardware
requirements:
http://www-03.ibm.com/servers/storage/support/software/tpcrep/installing.html
28.15 TPC for Replication GUI
This section describes the graphical user interface (GUI) that is used to communicate with the
the Replication server. You can read additional details in the documents mentioned on the
first page of this chapter; see Chapter 28, “IBM TotalStorage Productivity Center for
Replication” on page 415.
TPC for Replication provides a graphical user interface (GUI) to manage and to monitor any
Copy Services configuration and Copy Services operations. This GUI is Web browser-based
and does not rely on any other product such as TotalStorage Productivity Center or IBM
Director.
URL of GUI (RM server)
Figure 28-13 GUI URL determined during Replication Manager installation
The GUI is simply called through an Internet browser, such as Microsoft Internet Explorer or
Mozilla Firefox, is intuitive, easy to use, and performs very well. The panel structure is not too
deep and allows you to quickly transition to any application through a hyperlink-based menu.
Chapter 28. IBM TotalStorage Productivity Center for Replication
433
Figure 28-14 displays the TPC for Replication GUI and its components used between the
client on the user machine and the server component on the server machine.
Login
Configure sessions
Session States
User machine
Global Mirror settings
Browser / GUI
Advanced tools
LAN
Database
Monitor Server
Replication Manager Server
Hardware Lavers
IP network
Series p
Series p
DS8000
PowerPC
PowerPC
DS6000
Figure 28-14 TPC for Replication GUI
The Web-based client, which runs on the user’s machine, contains the GUI client, a
presentation component. This piece of software on the user’s machine is highly optimized to
minimize data traffic over the LAN to the RM server.
The GUI server component runs on the RM server machine, which contains the interface to
the function code in the connected storage servers and is usually closely installed to the
storage servers, which interface with the RM server.
28.15.1 Connect to the TPC for Replication GUI
You connect to the GUI by specifying the IP address of the TPC for Replication server in your
Web browser.
This presents you with the sign-on panel as shown in Figure 28-15 on page 435.
Once you sign off from the RM server, the same panel displays.
434
IBM System Storage DS6000 Series: Copy Services in Open Environments
URL of GUI (RM server)
user name
password
Figure 28-15 Launch the Replication Manager GUI
You specify a user ID as a text string on the UserID filed and a password in a hidden text
field. You define and set up user ids in the RM server system.
28.15.2 Health Overview panel
After a successful login into the Replication Manager server, the Health Overview panel
displays as shown in Figure 28-16 on page 436.
Chapter 28. IBM TotalStorage Productivity Center for Replication
435
Figure 28-16 Health Overview panel
This health panel displays an overall summary of the Replication Manager system status. It
actually shows information very similar to what is also shown in the small box at the lower left
corner of this panel. This small health overview box at the lower left corner is always present.
The Health Overview panel, however, provides more details. It provides a first overall status
of:
򐂰 Sessions
򐂰 Connected storage subsystems
򐂰 Management servers
Figure 28-16 reports that all sessions are in normal status and working fine. There is no high
availability server environment and one or more storage servers cannot be reached by the
Replication Manager as they have been defined before.
The upper left box in this panel labeled My Work provides a list of applications that you can
use to manage various aspects of a Copy Services environment.
Health Overview
This is the currently displayed panel as Figure 28-16 shows.
Sessions
This hyperlink brings you to the application, which manages all sessions. This is the
application that you use the most.
Storage Subsystems
Start here when you define storage servers to the RM server that are going to be used for
Copy Services.
ESS/DS Paths
This link allows you to manage everything that is related to PPRC path management.
436
IBM System Storage DS6000 Series: Copy Services in Open Environments
Management Servers
This link leads you to the application that manages the RM server configuration.
Advanced Tools
Here you can trigger to collect diagnostic information or set the refresh cycle for the displayed
data.
Console
This link opens a log that contains all of the user’s activities and results.
28.15.3 Sessions panel
This is the panel to all sessions within the RM server. See Figure 28-17.
Figure 28-17 Sessions overview
Each session comprises a number of copy sets, which can be distributed across LSS and
physical storage servers. The session name functions as a token, which is used to apply any
action against the session or against all the volumes, which belong to that session.
Figure 28-18 on page 438 illustrates that you first select a session, and then you choose the
action that you want to perform against that session.
Chapter 28. IBM TotalStorage Productivity Center for Replication
437
(2) Select action
(1) Select session
Figure 28-18 About to create copy sets to session
Figure 28-19 displays a possible next step to perform an action against a session after
selecting the session GM_Session1 as shown in Figure 28-17 on page 437. You can select
any action from the action list shown in Figure 28-19.
Figure 28-19 Actions against a session
This might be an action against the entire session, such as suspending all volumes within the
session. It is also used to modify an existing session and add or remove copy sets.
28.15.4 Storage Subsystems panel
The following panel displays all storage subsystems currently connected to the RM server.
438
IBM System Storage DS6000 Series: Copy Services in Open Environments
Panel heading
Hyperlinks
Interface to functional tasks
RM summary
Figure 28-20 GUI basic layout
By clicking Add Subsystem, you can define another storage subsystem to the RM server.
Figure 28-22 on page 440 shows the panel to add a new server.
Figure 28-21 displays the available action list. From this list, you select, for instance, the
View/Modify Details action and apply it to the previously selected storage server.
Figure 28-21 Select storage subsystem and select View/Modify Details action
Chapter 28. IBM TotalStorage Productivity Center for Replication
439
The selected storage subsystem connectivity details now display as shown in Figure 28-22.
Figure 28-22 Storage subsystem details
You typically use the Storage Subsystems application only to connect a storage server to the
RM server. Figure 28-22 also shows the standard port used by the RM server to
communicate with the storage server. All the other fields are self-explanatory.
28.15.5 Path Management panel
Figure 28-23 displays the entry panel to manage PPRC paths.
Figure 28-23 Path overview panel
Clicking Manage Path triggers the path wizard to help you define a new PPRC path or
remove existing PPRC paths.
440
IBM System Storage DS6000 Series: Copy Services in Open Environments
Clicking on a storage subsystem gives you a list of all the existing paths currently defined for
the selected storage subsystems. Figure 28-24 displays all the defined PPRC paths for the
selected LSSs and the ports used for these PPRC paths.
Figure 28-24 Path overview of a DS8000
You can select any path here. The only available action, in this case, is to then remove the
selected paths.
28.15.6 RM Server Configuration panel
The panel in Figure 28-25 on page 442 displays the status of the Replication Management
server or servers.
Chapter 28. IBM TotalStorage Productivity Center for Replication
441
Figure 28-25 Replication Management Servers
Figure 28-25 shows only one server named Stops, which is an active server.
When it exists, a second row shows the potential second server, which is in the status
Standby. A panel with two servers has a slightly different appearance than what is shown in
Figure 28-25.
You use this panel for basic operations, such as defining a server as Standby or to take over
in the event of a disaster.
In the case of two servers, each server manages its own DB2 database. The communication
between both servers is performed through the LAN to which both servers are connected.
28.15.7 Advanced Tools panel
Figure 28-26 on page 443 displays a panel through which you handle some specific tasks.
442
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 28-26 Advanced tools option
Specific tasks in this context are tasks to create a diagnostic package or to change the
automatic refresh rate of the GUI. A third task is to enable or to disable the heartbeat, which
happens between the Replication Manager server and the connected storage servers. See
also 28.12, “TPC for Replication heartbeat” on page 431.
The diagnostic package contains all RM logs, and its location on the RM server is shown on
the panel.
The browser refresh rate is currently a value between 5 to 3600 seconds. Default value is 30
seconds.
28.15.8 Console log
Figure 28-27 on page 444 displays an example of a console log. This panel shows a list of the
most recent commands, which this user entered through the GUI.
Chapter 28. IBM TotalStorage Productivity Center for Replication
443
Figure 28-27 Console log
In addition to the commands, this log also shows whether the command succeeded. A
message number functions at the same time as a hyperlink to more detailed text about the
result of the specific command execution.
28.16 Command-Line Interface to TPC for Replication
In addition to the GUI, you can also manage TPC for Replication through a Command-Line
Interface (CSMCLI).
As with the DSCLI for DS8000 and DSCLI for DS6000, the CSMCLI command structure is
similar among all three CLI products such as using mk for make, ch for change, and so forth.
CSMCLI is invoked in the same fashion as the DSCLI for the DS products. There are three
modes:
򐂰 Single-shot mode
򐂰 Interactive mode
򐂰 Script mode
Example 28-1 shows a single-shot command.
Example 28-1 Single-shot CLI command
> csmcli lsdevice -devtype ds
To execute more than one command (Interactive mode), you can start the CLI program and
perform the commands at the shell prompt as Example 28-2 on page 445 shows.
444
IBM System Storage DS6000 Series: Copy Services in Open Environments
Example 28-2 Interactive CLI commands
... start csmcli ...
csmcli> lsdevice -devtype ds
csmcli> lsdevice -devtype ess
The third mode is the script mode to run commands out of a file.
Example 28-3 Script mode to execute CLI commands
... start csmcli ...
csmcli -script ~/rm/scripts/devreport
In contrast to DSCLI for the DS storage servers, the CSMCLI does not currently use a
-profile option.
Chapter 28. IBM TotalStorage Productivity Center for Replication
445
446
IBM System Storage DS6000 Series: Copy Services in Open Environments
29
Chapter 29.
IBM TotalStorage Rapid Data
Recovery for UNIX and Windows
IBM TotalStorage Rapid Data Recovery for Unix and Windows is a service offering solution
from IBM Global Services based on enterprise Remote Copy Management Facility (eRCMF).
eRCMF is a stand-alone tool that provides a management layer for ESS800, DS8000, and
DS6000 Copy Services in 2-site or 3-site topologies. It is a common management tool for
both z/OS and open systems environments. The Copy Services functions managed by
eRCMF are Metro Mirror, Global Copy, and Global Mirror. At each site, a FlashCopy of the
local volumes can also be managed.
This solution simplifies the process of repairing inconsistent Remote Copy pairs after an error
with the copy process. eRCMF assists the user in automating the Remote Copy and
FlashCopy processes.
Note: Because of its complex implementation, IBM Global Services (IGS) works with your
IT department to install and configure the solution to match your unique business
environment. The storage servers, servers, and WebSphere software have to be ordered
through the normal process. IGS provides eRCMF software and experts that support the
implementation, education, and skill transfer for this solution.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
447
29.1 Introduction
eRCMF is a scalable and flexible solution that protects business data and maintains
consistent data. It can be used to manage copy relationships for both:
򐂰 Planned outages, such as hardware and software upgrades, or disaster practice
򐂰 Unplanned outages, such as an actual disaster
The solution provides basic commands through the Failover/Failback (FO/FB) Session
Manager Proxy, which utilizes the ESS Open API functions to accomplish either a planned or
unplanned FO/FB sequence. eRCMF commands can be incorporated in any user scripts or
utilities that might further link the client’s overall data center operations. However, it is the
client’s responsibility for creation, maintenance, and any liabilities associated with these
scripts or utilities. Therefore, eRCMF offers a simple way to implement Remote Copy disaster
recovery procedures using the command line. eRCMF applies the commands to a set of
configuration files that describe the disk subsystem configuration (paths and sets of
volumes).
The solution does not depend on the operating system, because no client installation is
required.
29.1.1 Solution highlights
The main solution highlights and benefits are:
򐂰 Provides automation capability
򐂰 Simplifies Disaster Recovery operations
򐂰 Improves protection of critical business data guaranteeing data consistency and ensuring
restart capabilities
򐂰 Avoids human errors
򐂰 Reduces risk of down time
򐂰 Stays competitive, maintaining market readiness by managing risk more efficiently
򐂰 Enables test and practice of Disaster Recovery setups regularly
򐂰 Reduces testing risk
򐂰 Provides a central, scalable, flexible, enterprise class solution that protects your business
򐂰 Simplifies the configuration:
– Simple configuration description
– Tools provided to verify configuration
򐂰 Simplifies the utilization
Simple commands to fix problems
򐂰 No predefined Copy Services tasks necessary
29.2 Overview
In a Business Continuity solution based on disk Remote Mirroring, we want to restart a
database application following an outage without having to restore the database. This
process has to be consistent, repeatable, and fast, measurable in minutes.
448
IBM System Storage DS6000 Series: Copy Services in Open Environments
A database recovery where you have to restore the last set of Image Copy tapes and apply
the log changes to bring the database up to the point of failure is a process that can be
measured in hours or even days. And this is what you want to avoid in a Tier 4 or later
solution. However, in a real disaster (fire, explosion, or earthquake), you can never expect
your complex to fail all at the same moment. Failures are intermittent and gradual, and the
disaster occurs over many seconds or even minutes. This is known as the Rolling Disaster.
The architecture of a viable disk mirroring Disaster Recovery solution must avoid data
corruption caused during a rolling disaster.
In any operating system, the sequence in which updates are made is what maintains the
integrity of the data. If that sequence is changed, data corruption occurs. The correct
sequence must be maintained within a volume, across volumes, and across multiple storage
devices. For instance, in Figure 29-1, we show the relationship between a database and its
log, which demonstrates the requirement for maintaining I/O integrity. Data consistency
across the storage enterprise must be maintained to insure data integrity.
Secondary
Secondary
Primary
Primary
Log
1. Log Update
Volume
3. Mark Log Complete
Database
2. Update Database
Volume
Figure 29-1 Sample update sequence in a database
The order of dependent writes across volumes must be maintained at remote locations.
Failure to do so results in data corruption. The data consistency is lost.
In Figure 29-2 on page 450, we illustrate this concept with an example. When a database
update is about to take place, the change is first logged in the database log files at both the
primary and secondary volumes (step 1). Assume the database datafile is updated at the
primary volume, but the update does not reach the remote volume that contains the mirrored
data file. The primary location is not aware of the write failure to the secondary volume (step
2). The database update is marked complete in the log files at both the primary and remote
locations (step 3). The result is that the secondary site log files say the update was done, but
the updated data is not in the database at the secondary location. There is no way to know
that the data was corrupted.
Chapter 29. IBM TotalStorage Rapid Data Recovery for UNIX and Windows
449
Primary
Primary
Secondary
B
C
L
M
X
Y
B
C
L
M
X
Y
OK
1. Log Update
Database
Application
OK
OK
3. Mark Log
Complete
2. Database Update
Left side disk data
does not match right
side. Is this the
problem?
Yes
Figure 29-2 Sample Rolling Disaster
So, the issue is that the disk subsystem cannot do it all, and we must avoid this Rolling
Disaster system problem.
For this reason, we strongly recommended that, at a minimum, Consistency Groups are put
in place for any mirroring solution. Consistency Groups are an implementation of technology
that assists with the consistency of application data capable of dependent writes. To
guarantee a fully consistent remote copy, multiple volumes require a Consistency Group
functionality. ESS and DS6000/DS8000 Remote Copy microcode already have the
Consistency Group function for both z/OS and open systems. SAN Volume Controller Metro
Mirror’s microcode and DS4000 Remote Mirror’s microcode also have a Consistency Group
function for open systems.
If any volume within a Consistency Group is unable to complete a write to its counterpart in
the Remote Copy relationship, an Extended Long Busy (ELB) for mainframe environments or
a SCSI Queue Full condition for open systems is issued, preventing further writes to any of
the volumes within the Consistency Group. This wait period is the perfect time to issue a
freeze to all volumes involved to maintain consistency. Figure 29-3 on page 451 illustrates
this mechanism. If a write is unable to complete, the storage subsystem does not back out
incomplete transactions on its own. Instead, the application needs to recognize that the
transaction was incomplete and take the appropriate actions. Once the storage subsystem
holds application I/O to the affected primary volumes, the write dependent mechanism of the
application prevents the Remote Copy secondaries from becoming inconsistent.
450
IBM System Storage DS6000 Series: Copy Services in Open Environments
Consistency Group
control function
Database
Application
SNMP trap
B
C
L
M
X
Y
B
C
L
M
X
Y
1. Mirroring failure
2. Storage Server suspends
affected primary and holds
application I/O (SCSI Queue
full condition).
3. Storage Server sends a
notification about the failure.
4. A program, such as eRCMF,
can receive the SNMP trap and
then issue the freeze to all LSSs
in the Consistency Group.
5. Storage Server releases I/O if
told to do so or after Remote
Copy Consistency Group
time-out is over.
Figure 29-3 Dependent write protection of database integrity
freeze is a command available through the DS Command-Line Interface, the DS Storage
Manager (Graphical User Interface), the ESS API, ICKDSF, or TSO that stops the write
activity on all of the active Remote Copy primary and secondary volumes of a given source
and target Logical Subsystem (LSS) pair. This function ensures consistency between the
primary and secondary volumes and can affect any LSS volumes that are in an active
Remote Copy state between the frozen LSS pair: Duplex, duplex pending SYNC, or duplex
pending XD states. It does not affect the suspended and simplex volumes that might be in the
LSS. The freeze operation has three effects:
򐂰 The paths that connect the pair of LSSs being frozen are terminated.
򐂰 The active volumes under the frozen LSS pair are suspended. This state transition to
suspended is then communicated to the host with alert messages. These alert messages
can be used by automation routines to trigger other recovery operations.
򐂰 If the Consistency Group option was enabled at path definition time, then the ELB or SCSI
Queue Full condition is instigated, so that the write activity to the primary LSS is
temporarily queued. During this interval, other automated operations can be triggered,
such as freezing other application-related LSS pairs.
When using freeze through the DS Storage Manager (Graphical User Interface) or ICKDSF,
it takes effect on each LSS individually. This is useful for creating a Point-in-Time Copy of the
data, but because of slight delays between the issuing of each iteration of the freeze
command, it is unsuitable for preventing Rolling Disasters and should be done at periods of
low utilization to ensure that the secondary data can be used.
When Remote Copy is used in conjunction with automation, such as the Geographically
Dispersed Parallel Sysplex™ (GDPS) or enterprise Remote Copy Management Facility
(eRCMF) service offerings from IBM Global Services, a freeze command can be
simultaneously issued to all LSSs within the configuration. This ensures globally consistent
data across all LSSs in the secondary copy of data during a disaster.
Chapter 29. IBM TotalStorage Rapid Data Recovery for UNIX and Windows
451
The goal of the TotalStorage Rapid Data Recovery for UNIX and Windows solution, based on
the combination of ESS or DS6000 or DS8000 Copy Services with enterprise Remote Copy
Management Facility (eRCMF), is to protect your data from being a mirror of a dying scenario.
Using a Remote Copy Consistency Group, this solution freezes your environment at a known
point instead of mirroring literally hundreds of time-offset failures in a short amount of time.
TotalStorage Rapid Data Recovery is based on enterprise Remote Copy Management
Facility (eRCMF), an IBM Global Services offering. The current eRCMF Version 4 is designed
as a 2-site or 3-site Disaster Recovery solution that runs on dedicated machines under
WebSphere on AIX 5L™, which are used as eRCMF servers.
Note: We strongly recommend that the eRCMF servers are installed and run from different
physical sites to ensure their backup capability.
eRCMF is a multi-site Disaster Recovery solution that manages data on volumes for Fixed
Block (open systems hosts) and CKD (z/OS). The solution does not depend on the operating
system, because no client installation is required.
Software Automation for SAP, DB2, Siebel, MS SQL Server,
Exchange, and so forth.
3
Automation for Servers: z/OS, UNIX, Linux, Windows, and
heterogeneous environments.
2
Point-in-Time Copy, Metro Mirror, Global Mirror, TotalStorage
Software Family, DFSMS for z/OS, and TSM.
1
Hardware Infrastructure: ESS, DS family, SAN Switches, VTS,
3584, 3592, and LTO.
+
Business value
4
Autonomic
Services
eRCMF is a Tier 4 and Tier 6 solution, as shown in Figure 29-4.
disk
tape
7 6 5 4
3 2 1
-
Recovery Time
+
Areas covered and tier level
Figure 29-4 eRCMF protection tier
eRCMF is built on the technologies of Remote Copy, Java, WebSphere, and ESS Network
Interface. You can mix these storage servers in this solution, because ESS 800, DS6000, and
DS8000 share the same Copy Services functions. With DS8000 and DS6000, no Copy
Services Server is required since each storage server can handle the Copy Services.
29.3 Architecture
eRCMF is a 2-site or 3-site solution. According to the diagrams in Figure 29-5 on page 453
and Figure 29-6 on page 453, one PCM is placed in each data center. The PCM in the
primary data center runs the Master Process. This process is the interface into eRCMF. It
handles all queries and commands for local processes through either a command-line or
socket interface. It also manages commands and queries from the Backup Process.
A second PCM can be placed in the secondary (or backup) data center, or in a 3-site
strategy, in the tertiary (or remote) data center. This PCM runs the Backup Process, which
maintains the state information of the configuration. This information is transferred to the
452
IBM System Storage DS6000 Series: Copy Services in Open Environments
second PCM in the form of logs from the Master Process. These logs are then used to update
state information in case the Backup Process must take control of the configuration, as well
as for documentation purposes if there is an alternate site failure.
Metro Mirror, up to:
Global Copy, over:
103 km / 300 km
Site 1 - Production
Storage Server (s)
ESCON
Shadow FlashCopy
Host
Volumes
Volumes
/
Site 2 - Backup
Storage Server (s)
FCP
Metro Mirror, Global Copy
Global Mirror
Journal
Volumes
Host FlashCopy Shadow
Volumes
Volumes
Journal
Volumes
TCP/IP / SNMP
AIX – Backup PCM
AIX – Primary PCM
WebSphere - HTTPS
WebSphere - HTTPS
ESS Network Interface
ESS Network Interface
eRCMF
eRCMFconfig.properties
Enterprise.dat
VolumeSet.dat
Hosts.dat
Authentication.file
eRCMF
eRCMFconfig.properties
Enterprise.dat
VolumeSet.dat
Hosts.dat
Authentication.file
Backup process
(Master process)
Management
Interface
Master process
(Backup process)
SNMP
Listener
Figure 29-5 eRCMF architecture: 2-site configuration
Global Copy, over
103 km / 300 km
Metro Mirror, up to
103 km / 300 km
Site 2 - Backup
Site 1 - Production
Storage Server(s)
Shadow
Host
Volumes FlashCopy Volumes
Copy Services
(CSServer on ESS 800 / 750)
AIX – Backup PCM
WebSphere - HTTPS
ESCON
FCP
Metro
Mirror
Storage Server(s)
Host
Shadow
Volumes FlashCopyVolumes
Copy Services
Site 3 - Remote
ESCON
FCP
Global
Copy
Storage Server(s)
Host
Shadow
Volumes FlashCopyVolumes
Copy Services
(CSServer on ESS 800 / 750)
AIX – Primary PCM
WebSphere - HTTPS
TCP/IP
SNMP
ESS Network Interface
ESS Network Interface
eRCMF
eRCMF
eRCMFconfig.properties
eRCMFconfig.properties
Enterprise.dat
VolumeSet.dat
Hosts.dat
Authentication.file
Enterprise.dat
VolumeSet.dat
Hosts.dat
Authentication.file
Backup process
(Master process)
Management
Interface
SNMP
Listener
Master process
(Backup process)
Figure 29-6 eRCMF architecture: 3-site configuration
Chapter 29. IBM TotalStorage Rapid Data Recovery for UNIX and Windows
453
The Master Process continuously monitors and manages the entire environment at two
levels:
򐂰 Enterprise level
򐂰 VolumeSet level
The distinction between these levels is primarily the scope of actions performed at each level.
The Enterprise level represents the entire monitored environment, which consists of two or
three data processing locations containing storage managed by eRCMF. The VolumeSet
level corresponds to a set of storage server volumes, which must be maintained in a
consistent state and managed using the same type of copy operations. Some examples of a
VolumeSet are the volumes used by a set of applications, the volumes allocated to a set of
servers, or the members of a Volume Group.
Management at the Enterprise level affects the entire monitored environment. These actions
typically affect every volume of every VolumeSet that is managed by eRCMF. For example,
the freeze command causes suspension of Remote Copy pairs between the storage
subsystems on multiple sites, as you can see in Figure 29-7.
eRCMF adds a central control function to coordinate the Remote Copy Freeze function and
stops transmission of data to the remote site consistently across single and multiple frames.
When this function is centrally coordinated, data is consistent. Data consistency allows a very
fast, consistent restart time at the secondary site.
yNotification of mirroring
error
yCU holds I/Os to
suspended device until
eRCMF can handle
condition
Host
Host
Host
Host
eRCMF
Control
Unit
Control
Unit
Control
Unit
Control
Unit
Control
Control
Unit
Unit
Control
Unit
Control
Unit
•eRCMF issues freeze command to all Storage
Servers
•suspends all PPRC pairs
•holds I/O until told to continue
•Based on policy, eRCMF will then allow host
operation to resume or continue to hold them
Control
Unit
Control
Unit
Control
Unit
Control
Unit
Control
Control
Unit
Unit
Control
Unit
Control
Unit
During
After, depending on policy:
•Both sites are suspend but in sync
•Sites are not insync, however data
to a site is consistent
Data at remote site is at a “Known State”
Figure 29-7 eRCMF issues freeze command in a Disaster Recovery scenario
There are two methods of maintaining consistent data:
򐂰 Metro Mirror
– Use Synchronous Remote Copy to ensure that data is secure at the remote site before
signaling the host that the write is complete.
– During a disaster, combine the Consistency Group and Freeze functions in the storage
server with software, such as eRCMF, to insure that all Remote Copy pairs of the
Consistency Group are split in a manner that assures data consistency at the remote
site.
454
IBM System Storage DS6000 Series: Copy Services in Open Environments
򐂰 Global Mirror
– The storage server periodically forms Consistency Groups.
– When recovering to the recovery copy, software such as eRCMF examines the states
of the Global Mirror relationships and directs the storage server through the steps
necessary to recover that data to the last committed consistent copy.
Note that eRCMF neither performs an automated recovery nor an automated restart of
systems in the remote site. Instead, eRCMF's freeze performs a controlled site split only. If
a higher level of automation is required, eRCMF provides interfaces such as execution scripts
or a Command-Line Interface, which can be utilized and customized. When this occurs,
mirroring between site 1 and site 2 stops, ensuring the consistency of data. Further response
by the automation is determined by policies that are selected based on individual business
requirements:
򐂰 Freeze and Go quickly executes the freeze to ensure data consistency, but then
immediately allows I/Os to continue. This allows processing to continue in the production
data center while maintaining a consistent copy of the data in either the backup or remote
data center. Meanwhile, the business is able to determine the cause of the event and
whether it is a disaster. If a disaster has not occurred, production has not been impacted
by an unnecessary site switch. If a disaster has occurred, then the data since the time of
the site split might need to be recreated if recovering to a backup or a remote data center.
򐂰 Freeze and Stop quickly executes the freeze to ensure consistency but, rather than
immediately allowing the I/Os to continue, causes the storage server to block all further
I/Os until told otherwise. This ensures that no transactions, other than those in-flight at the
time of the event, need to be recreated following a disaster. If the event turns out not to be
a disaster, however, production is halted until the I/Os are freed.
29.4 Additional information
Further information can be obtained at the IBM Web site:
http://www-1.ibm.com/services/us/index.wss/so/its/a1000110
For further information, you can refer to the following papers:
򐂰 enterprise Remote Copy Management Facility eRCMF V4 Implementation Guide
򐂰 enterprise Remote Copy Management Facility eRCMF V4 User Guide
You can find these at:
http://www-1.ibm.com/support/docview.wss?uid=ssg1S7001074
For more information, contact your IBM Sales Representative.
Chapter 29. IBM TotalStorage Rapid Data Recovery for UNIX and Windows
455
456
IBM System Storage DS6000 Series: Copy Services in Open Environments
30
Chapter 30.
IBM TotalStorage Continuous
Availability for Windows
IBM TotalStorage Continuous Availability for Windows (formerly named IBM TotalStorage
Geographically Dispersed Clusters for Microsoft Cluster Server, or GDS offering) on the
Enterprise Storage Server (ESS), DS6000, DS8000, and SAN Volume Controller is designed
to allow cluster resources (that is, applications and data) to Fail over between two
geographically dispersed sites. By combining the high availability of Microsoft Cluster Server
(MSCS) services for application cluster Failover with Metro Mirror, TotalStorage Continuous
Availability for Windows provides a continuous availability solution to protect critical
cluster-aware applications and data in a Microsoft Cluster environment.
This services offering, available by RPQ through IBM Systems Group, enables high
availability and Disaster Recovery in the Microsoft environment by incorporating the benefits
of Metro Mirror.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
457
30.1 Introduction
IBM TotalStorage Continuous Availability for Windows is the IBM implementation for disk
subsystems within the Microsoft Geographically Dispersed Sites for cluster framework. It
provides clustering Failover support for any MSCS cluster-aware applications, such as DBMS
messaging and file sharing. Any MSCS cluster-aware applications run in a TotalStorage
Continuous Availability for Windows configuration.
IBM TotalStorage Continuous Availability for Windows is a Disaster Recovery solution for
Microsoft Cluster server applications using automated cluster Failover services combined
with IBM storage subsystems using Metro Mirror for remote site mirroring and automated
Failover of data. It can allow for zero or little data loss and little or no down time, and it
provides for complete site server application and data Disaster Recovery across two
geographically dispersed sites.
It monitors the health states of the applications, network, and Metro Mirror storage to
determine automatic actions for Failover to the secondary site.
Solution highlights
The main solution highlights and benefits are:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
Improve continuity of business.
Improve protection of critical business data.
Reduce business risk.
Provide flexibility for the transfer of operations between sites with minimal disruption.
Improve business resiliency.
Stay competitive, maintaining market readiness by managing risk more efficiently.
30.2 Overview
When you are using Microsoft Clustering Services with Windows 2003 Enterprise Edition
Servers, you might not only want to prevent your business from a disaster on a single node
participating in the cluster but might also want to prevent a disaster on the disk subsystems
shared by the nodes. The solution is to have a Geographically Dispersed Solution where both
servers and storage resources are redundant and installed on different sites. In this case, you
need to mirror your shared disks between both sites. Because you cannot use any RAID
controller card in a SAN environment, two choices are possible. Either you ask the Windows
system to do it or you ask the disk subsystem to do it.
The first solution is based on a system mirror. When you use a system mirror, the
synchronization process is supported by the resources of your cluster server node. Because
Microsoft Clustering Services does not support dynamic disks, the only way to perform it is to
add a software layer between your system and the disks. This kind of layer not only adds a
new level of complexity to your system and to your storage management, but it also has
some distance limitations between the two sites.
The second solution is based on a disk subsystem mirror, such as the DS8000 Metro Mirror.
When you use a disk subsystem mirror, the synchronization process is supported by the
resources of your disk subsystem and therefore is not impacting the resources of your cluster
server nodes. At the same time, you can take advantage of the replication technology
installed on the disk subsystems, which allows replication at greater distances. It is a way to
let every participant in this clustering solution do its job. The cluster node is computing while
the disk subsystem is replicating.
458
IBM System Storage DS6000 Series: Copy Services in Open Environments
When you choose the second solution, the challenge is to automate the Failover and Failback
process between remote sites involving not only cluster nodes but also disk subsystems. With
IBM TotalStorage Continuous Availability for Windows, you can perform such tasks
transparently from your cluster management interface.
IBM TotalStorage Continuous Availability for Windows (formerly Geographically Dispersed
Sites for Microsoft Cluster Services) integrates Microsoft Cluster Services and the Metro
Mirror Advanced Copy Service functions of the IBM System Storage DS8000. This product is
implemented on Windows 2003 Enterprise Edition using two sites. Each pair of disks
belonging to the same cluster resource group can only be assigned to a pair of cluster nodes
(one on each site). Therefore, this solution allows either Active/Passive or Active/Active
cluster architectures. Of course, at a clustered resource group level, there are only
Active/Passive solutions, because you cannot write concurrently on two volumes of a Metro
Mirror relationship.
If you choose the Active/Active cluster architecture, it means that every node at both sites can
be actively running applications. There are active resource groups on sites A and B. Your
production is naturally balanced over both nodes sharing these resource groups. When a
resource group on site A has a failure, the data and applications bound to this resource group
automatically Fail over to site B. Depending on your application scripts, it can take a few
seconds or several minutes. This architecture has two advantages. First, you can be sure that
each node hosting a resource group is in a good state, and therefore not discover a wrong
behavior when it is too late (such as during a cluster Failover following a disaster). Second,
you can balance your production’s workload on both nodes and therefore get better
performance. You can encounter some performance issues during a disaster scenario if the
remaining node is not able to handle the full production workload.
If you choose the Active/Passive cluster architecture, when the primary site suffers a failure,
the data and applications automatically Fail over to the backup site, typically in minutes. Once
the primary site is repaired, the resources can be failed back to the original site.
Note: A Failover/Failback process at a cluster level is not similar to a Failover/Failback at a
Metro Mirror level.
For a cluster, a Failover process is when the production is moving from a site A to a site B.
A Failback is when the production is moving back from a site B to a site A.
For Metro Mirror, a Failover is when the replication process is stopped and the production
volume is moved from site A to site B. At that time, both volumes participating in the former
pair are reachable. A Failback is when after a Failover the replication process is
reactivated with the source volume on site B and the target volume on site A.
IBM TotalStorage Continuous Availability for Windows offers high availability for servers and
applications plus Disaster Recovery for files and data. It improves data availability with
continued service during hardware and software failures. It provides disaster-tolerant
capabilities by allowing the cluster servers and associated mirrored storage to be
geographically separated by the Metro Mirror distance for FCP. It can also be used to reduce
or eliminate planned down time, such as during a software or hardware upgrade. GDS is a
Tier 7 High Availability solution. We can see the topology in Figure 30-1 on page 460.
Chapter 30. IBM TotalStorage Continuous Availability for Windows
459
3
Automation for Servers: z/OS, UNIX, Linux, Windows, and
heterogeneous environments.
2
Point-in-Time Copy, Metro Mirror, Global Mirror, TotalStorage
Software Family, DFSMS for z/OS, and TSM.
1
Hardware Infrastructure: ESS, DS family, SAN Switches, VTS,
3584, 3592, and LTO .
+
Business value
Software Automation for SAP, DB2, Siebel, MS SQL Server,
Exchange, and so forth.
Autonomic
Services
4
disk
tape
7
6 5 4 3 2
1
-
Recovery Time
+
Areas covered and tier level
Figure 30-1 TotalStorage Continuous Availability for Windows protection tier
Because the ESS 800, DS6000, and DS8000 share the same Copy Services functions, you
can mix them in this solution, ESS 800 or DS6000 or DS8000. The disk subsystems are
connected to each other using PPRC links, and the host servers are fibre-attached to the disk
subsystems at each site. The cluster-shared disks are mirrored between two disk
subsystems. The solution uses PPRC to coordinate the ownership and movement of disk
resources between sites.
30.3 Architecture
IBM TotalStorage Continuous Availability for Windows is a 2-site architecture with two MSCS
cluster nodes. Each client of the cluster’s resources is connected to the cluster through an
Ethernet LAN or WAN connection. For availability reasons, at least one other Ethernet
connection is linking both nodes for heartbeat purposes. The reason for this double TCP/IP
connection is that if any of this network should fail, each node could use the secondary
network to communicate, and, therefore, avoid an unnecessary failover.
Replication PPRC links must be defined in both directions in order for the replication pairs of
volumes to be able to Fail over and fail back automatically and attempt to always replicate the
writes on the other site.
A quorum volume should be created on each site and a Metro Mirror relationship should be
established between each of them. Then, each volume composing a cluster resource group
should be created on both sites and a Metro Mirror relationship should be established
between each of them. Finally, for the IBM TotalStorage Continuous Availability for Windows
solution to be cluster aware, two pairs of so-called site disks should be created, and a cross
Metro Mirror relationship should be established between each of them. It means that one pair
replicates from site A to site B and the other pair from site B to site A. These Metro Mirror
relationships are never switched in the future.
460
IBM System Storage DS6000 Series: Copy Services in Open Environments
Figure 30-2 Architecture of the solution
30.3.1 Service and ordering
Because of its complex implementation in a clustered environment, IBM TotalStorage
Continuous Availability for Windows is provided as an IBM System Group Services offering.
The offering includes three components:
򐂰 TotalStorage Continuous Availability for Windows software license, which includes C++
code and an Installation package for installation and configuration within an MSCS
environment across two nodes.
򐂰 A 5-day on-site services engagement to help the client set up, configure, conduct skills
transfer on this solution, and validate the installation.
򐂰 An IBM 7x24 Support contract.
As a high-availability mission-critical Disaster Recovery infrastructure, TotalStorage
Continuous Availability for Windows is supported by IBM 7x24 Support. The clients can count
on IBM to provide a single point of support focus and effectively work jointly with Microsoft to
support complex client configurations.
IBM TotalStorage Continuous Availability for Windows has been generally available since
2003 and has been put into production use by clients worldwide. You can order TotalStorage
Continuous Availability for Windows automation code and services through IBM TotalStorage
Chapter 30. IBM TotalStorage Continuous Availability for Windows
461
HUB (RPQ) through your IBM Sales Representative. The storage subsystems and servers
have to be ordered through the normal process.
Additional scalability of extending the current 2-node cluster configuration to 4 nodes and 8
nodes (the maximum number of cluster nodes presently supported by Windows 2003
Enterprise Edition) is planned for a future TotalStorage Continuous Availability for Windows
release.
Additional information can be found on the IBM Web site at:
http://www-03.ibm.com/servers/storage/solutions/business_continuity/continuous_availability
/technical_details.html
For more information, contact your IBM Sales Representative.
462
IBM System Storage DS6000 Series: Copy Services in Open Environments
A
Appendix A.
Open systems specifics
In this chapter, we describe the basic tasks that need to be performed on the individual host
systems when using the DS Copy Services.
We explain how to bring FlashCopy target volumes online to the same host, as well as to a
second host. This chapter covers various UNIX and Windows platforms.
© Copyright IBM Corp. 2004, 2005, 2006. All rights reserved.
463
AIX specifics
In this section, we describe the steps necessary to use volumes created by the DS Copy
Services on AIX hosts.
AIX and FlashCopy
The FlashCopy functionality is to copy the entire contents of a source volume to a target
volume. If the source volume is defined to the AIX Logical Volume Manager (LVM), all of its
data structures and identifiers are copied to the target volume, as well. This includes the
Volume Group Descriptor Area (VGDA), which contains the Physical Volume Identifier (PVID)
and Volume Group Identifier (VGID).
For AIX LVM, it is currently not possible to activate a Volume Group with a physical volume
(hdisk or vpath) that contains a VGID and a PVID that is already used in a Volume Group
existing on the same server. The restriction still applies even if the hdisk PVID is cleared and
reassigned with the two commands listed in Example A-1.
Example: A-1 Clearing PVIDs
#chdev -l <hdisk#> -a pv=clear
#chdev -l <hdisk#> -a pv=yes
Therefore, it is necessary to redefine the Volume Group information on the FlashCopy target
volumes using special procedures or the recreatevg command. This alters the PVIDs and
VGIDs in all the VGDAs of the FlashCopy target volumes, so that there are no conflicts with
existing PVIDs and VGIDs on existing Volume Groups that reside on the source volumes. If
you do not redefine the Volume Group information prior to importing the Volume Group, then
the importvg command fails.
Accessing FlashCopy target volume from another AIX host
The following procedure makes the data of the FlashCopy target volume available to another
AIX host that has no prior definitions of the target volume in its configuration database (ODM).
This host that receives the FlashCopy target volumes can manage the access to these
devices in two ways:
򐂰 LVM or MPIO definitions
򐂰 SDD definitions
If the host is using LVM or MPIO definitions that work with hdisks only, follow these steps:
1. The target volume (hdisk) is new to AIX, and therefore the Configuration Manager should
be run on the specific Fibre Channel adapter:
#cfgmgr -l <fcs#>
2. Find out which of the physical volumes is your FlashCopy target volume:
#lsdev -C |grep 1750
3. Certify that all PVIDs in all hdisks which belong to the new Volume Group were set. Check
this information through the lspv command. If not, run the following command to each one
to avoid the importvg command failing:
#chdev -l <hdisk#> -a pv=yes
4. Import the target Volume Group:
#importvg -y <volume_group_name> <hdisk#>
5. Vary on the Volume Group (the importvg command should vary on the Volume Group):
464
IBM System Storage DS6000 Series: Copy Services in Open Environments
#varyonvg <volume_group_name>
6. Verify consistency of all file systems on the FlashCopy target volumes:
#fsck -y <filesystem_name>
7. Mount all the target file systems:
#mount <filesystem_name>
If the host is using SDD that works with vpath devices, follow these steps:
1. The target volume (hdisk) is new to AIX, and therefore the Configuration Manager should
be run on all Fibre Channel adapters:
#cfgmgr -l <fcs#>
2. Configure the SDD devices:
#smitty datapath_cfgall
3. Find out which of the physical volumes is your FlashCopy target volume:
#lsdev -C |grep 1750
4. Certify that all PVIDs in all hdisks that belong to the new Volume Group were set. Check
this information through the lspv command. If not, run the following command to each one
to avoid the importvg command failing:
#chdev -l <hdisk#> -a pv=yes
5. Import the target Volume Group:
#importvg -y <volume_group_name> <hdisk#>
6. Change the ODM definitions to work with the SDD devices that provide you the load
balance and Failover functions:
#dpovgfix <volume_group_name>
7. Vary on the Volume Group (the importvg command should vary on the Volume Group):
#varyonvg <volume_group_name>
8. Verify consistency of all file systems on the FlashCopy target volumes:
#fsck -y <filesystem_name>
9. Mount all the target file systems:
#mount <filesystem_name>
The data is now available. You can, for example, back up the data residing on the FlashCopy
target volume to a tape device. This procedure can be run once the relationship between the
FlashCopy source and target volume is established, even if data is still being copied in the
background.
The disks containing the target volumes might have been previously defined to an AIX
system, for example, if you periodically create backups using the same set of volumes. In this
case, there are two possible scenarios:
򐂰 If no Volume Group, file system, or logical volume structure changes were made, then use
Procedure One (“Procedure One” on page 466) to access the FlashCopy target volumes
from the target system.
򐂰 If some modifications to the structure of the Volume Group were made, such as changing
the file system size or the modification of logical volumes (LV), then we recommend that
you use Procedure Two and not Procedure One.
Appendix A. Open systems specifics
465
Procedure One
1. Unmount all the source file systems:
#umount <source_filesystem>
2. Unmount all the target file systems:
#umount <target_filesystem>
3. Deactivate the target Volume Group:
#varyoffvg <target_volume_group_name>
4. Establish the FlashCopy relationships.
5. Mount all the source file systems:
#mount <source_filesystem>
6. Activate the target Volume Group:
#varyonvg <target_volume_group_name>
7. Perform a file system consistency check on target file systems:
#fsck -y <target_file_system_name>
8. Mount all the target file systems:
#mount <target_filesystem>
Procedure two
1. Unmount all the target file systems:
#umount <target_filesystem>
2. Deactivate the target Volume Group:
#varyoffvg <target_volume_group_name>
3. Export the target Volume Group:
#exportvg <target_volume_group_name>
4. Establish the FlashCopy relationships.
5. Import the target Volume Group:
#importvg -y <target_volume_group_name> <hdisk#>
6. Perform a file system consistency check on target file systems:
#fsck -y <target_file_system_name>
7. Mount all the target file systems:
#mount <target_filesystem>
Accessing the FlashCopy target volume from the same AIX host
In this section, we describe a method of accessing the FlashCopy target volume on a single
AIX host, while the source volume is still active on the same server. The procedure is
intended to be used as a guide and might not cover all scenarios.
If you are using the same host to work with source and target volumes, you have to use the
recreatevg command.
The recreatevg command overcomes the problem of duplicate LVM data structures and
identifiers caused by a disk duplication process such as FlashCopy. It is used to recreate an
AIX Volume Group (VG) on a set of target volumes that are copied from a set of source
volumes belonging to a specific VG. The command allocates new physical volume identifiers
466
IBM System Storage DS6000 Series: Copy Services in Open Environments
(PVIDs) for the member disks and a new Volume Group identifier (VGID) to the Volume
Group. The command also provides options to rename the logical volumes with a prefix you
specify, and options to rename labels to specify different mount points for file systems.
Accessing FlashCopy target volume using the recreatevg command
In this example, we have a Volume Group containing two physical volumes (hdisks) and want
to FlashCopy the volumes for the purpose of creating a backup.
The source Volume Group is src_flash_vg, containing vpath2 and vpath3.
The target Volume Group will be tgt_flash_vg, containing vpath4 and vpath5.
Perform these tasks to run the FlashCopy and make the target volumes available to AIX:
1. Stop all I/O activities and applications that access the FlashCopy source volumes.
2. Establish the FlashCopy pairs with the No Background Copy option selected. Use the
DS Storage Manager to establish the pairs or run a script with the DS CLI command.
3. Restart applications that access the FlashCopy source volumes.
4. The target volumes, vpath4 and vpath5, now have the same Volume Group data
structures as the source volumes vpath2 and vpath3. Clear the PVIDs from the target
hdisks to allow a new Volume Group to be made:
#chdev -l vpath4 -a pv=clear
#chdev -l vpath5 -a pv=clear
The output of the lspv command shows the result in Figure A-1:
Figure A-1 lspv output before recreating the Volume Group
5. Create the target Volume Group and prefix all file system path names with /backup and
prefix all AIX logical volumes with bkup:
recreatevg -y tgt_flash_vg -L /backup -Y bkup vpath4 vpath5
You must specify the hdisk names of all disk volumes participating in the Volume Group.
The output from lspv shown in Figure A-2 illustrates the new Volume Group definition:
Figure A-2 lspv output after recreating the Volume Group
An extract from /etc/filesystems in Figure A-3 on page 468 shows how recreatevg
generates a new file system stanza. The file system named /prodfs in the source Volume
Group is renamed to /bkp/prodfs in the target Volume Group. Also, the directory
/bkp/prodfs is created. Notice also that the logical volume and JFS log logical volume
have been renamed. The remainder of the stanza is the same as the stanza for /prodfs.
Appendix A. Open systems specifics
467
Figure A-3 Target file system stanza
6. Perform a file system consistency check for all target file systems:
#fsck -y <target_file_system_name>
7. Mount the new file systems belonging to t