Failure Correction Techniques for
Large Disk Array
Garth A. Gibson, Lisa Hellerstein et al.
University of California at Berkeley
1
What is the problem?
• Disk arrays can increase I/O bandwidth and
access parallelism
• The chance of data loss increases with the
increasing number of disk arrays
Figure 1. The mean time to data loss (MTTDL) in
a single-erasure-correcting array.
2
Types of data failure
• Transient or noise-related errors:
Correct by repeating the offending operation
or by applying per sector error-correction facilities
• Media defects:
detect and mask at the factory
• Catastrophic failures -- Head crashes or failures of
the read/write or controller electronics
3
The goal of this paper
• Avoid loss of user data
• Recover the catastrophic disk failures
• Make disk arrays as reliable as an individual
disk
4
Concept 1 -- erasure-correcting codes and
error-correcting codes
• Erasure-correcting codes are designed to recover erased
bits in a message word
– An unreadable bit is called an erasure
– The position of the erased bits are known
– For a catastrophic disk failure, the bits on a failed disk
can be designated as “unreadable”
• Error-correcting codes are designed to correct messages in
which some of the bits may have been flipped, but the
positions of those bits are unknown.
5
Concept 2 -- Redundancy Metrics
• Disk as stack of bits
-ith.bit in each disk forms the ith.Codeword in the redundancy encoding
•
•
•
•
Mean time to data loss (MTTDL): measure of reliability
Check disk overhead: check disks/data disks
Update penalty: number of check disks to be updated
Group size: the information and check disk that must be
accessed during the reconstruction of a failed disk form a group
6
1d - Parity
• Single-erasure-correction scheme
• For G data disks, one check disk with parity of all G disks.
• Overhead: 1/G
• Update penalty: 1
• Group size: G+1
G=4
7
2d - Parity
• Double-erasure-correction scheme
• G2 data disks arranged in 2-dimensional array
• For each row and each column, one check disk
stores parity for that row or column
• Check disk Overhead:
2G/G2 =2/G
• Update penalty: 2
• Group size = G+1
G=4
8
N-dimensional parity (Nd-parity)
• N-erasure-correction scheme
• Check disk overhead: NG(N-1) / GN = N/G
• Update penalty: N
• Group size: G+1
9
Linear Codes
Contain the original information unmodified
within each codeword and compute the check bits
of each codeword as the parity of subsets of the
information bits
Codeword =
1
1
1
1
Parity
10
Parity Check Matrix H = [P | I]
Fig. 4
How to compute the check parity bit? H*X = 0
First row of H = [100101 100]
P
X = [111010 x1 x2 x3]
I
H*X = 1+0+0+0+0+0+x1+0+0 = 0
x1=1
11
Parity Check Matrix for 1d-parity and 2d-parity
Fig. 5
12
Properties of the parity check matrix
• Express in terms of a parameter, t,
whose value is between 0 and c
• H will allow any t erasures to be corrected
• H will allow any t errors to be detected
• The minimum number of bits in which any two codewords
differ, known as the distance of the code, is at least t+1
• Any set of t column selected from will be linearly
independent
13
Implementing Reconstruction
0 1000
0 0110
0 0000
0 0001
0 0000
0 0100
0 1000
1 0011
Fig. 6(a). When 4 disks fail in a 16 information disk 2d-parity array, the
controllers allow us to identify which disks need to be repaired and reconstructed.
14
Implementing Reconstruction cont.
Fig. 6(b) Apply “elementary row operations” (the essence of Gaussian
elimination) to find a matrix M, such that the product MB has the 4*4
identity matrix in its first four rows.
15
Elementary operation
• If we interchange two
equation, the new system is
still equivalent to the old
one.
• If we multiply an equation
with a nonzero number, the
new system is still
equivalent to the old one.
• Replacing one equation with
the sum of two equation, we
obtain an equivalent system
Example:
x+y+z=0
x - 2y + 2z = 4
x + 2y - z = 2
(1)
(2)
(3)
(3) - (1) to replace (3)
x+y+z=0
(1)
x - 2y + 2z = 4 (2)
y - 2z = 0 (4)
(2)-(1) to replace (2)
x+y+z=0
(1)
- 3y +z = 4
(5)
y - 2z = 0
(4)
(5)+(4)*3 to replace (4)
x+y+z=0
(1)
- 3y +z = 4
(5)
- 5z= 10 (6)
result: x=4, y=-2, z=-2
16
Gaussian Elimination
Definition:
Using elementary operation, in
every step the new matrix was
exactly the augmented matrix
associated to the new system.
Once we obtain a triangular
matrix, write the associated
linear system and then solve it.
Example:
x+y+z=0
x - 2y + 2z = 4
x + 2y - z = 2
augmented matrix:
1 1 1 0
1 -2 2 4
1 2 -1 2
(3) - (1) to replace (3)
1 1 1 0
1 -2 2 4
0 1 -2 2
(2)-(1) to replace (2)
(1)
(2)
(3)
The linear equation :
x+y+z=0
- 3y +z = 4
- 5z= 10
1
1
0
1 1 0
-3 1 4
1 -2 2
(5)+(4)*3 to replace (4)
1
0
0
1 1 0
-3 1 4
0 -5 10
17
Implementing Reconstruction cont.
012 34567 89
15
11 10 0 0000000 10000000
01 00 0 0100010 00000100
01 01 1 0100010 01000100
00 00 0 0000111 00010000
10 01 0 1000100 00001000
Fig. 6 (C) The first 4 rows of MA describe the operations that
must be performed to reconstruct our 4 disks.
18
The position for codes with t-erasure-correction
• Be implemented in software
• Run in an I/O processor
• Software learns of failures directly from disk
controllers
19
Conclusion
• Implement the redundancy codes for disk
arrays
• Minimize the number of check disks that
must be updated whenever an information
disk is updated
• Improve the reliability of disk arrays
20
Question
•
•
•
•
What is codeword for redundancy disk?
List three redundancy metrics
What are 1d-parity and 2d-parity schemes?
What mathematical operation to be used for
recovering failed disk?
21