3
TWO NEW DIRECT METHODS FOR SOLVING LINEAR
ALGEBRAIC EQUATIONS BASED ON
ROW-ORTHOGONALIZATION∗
4
QIN SHU† , SILIANG ZHAO‡ , AND YUNXIU YANG§
5
6
7
8
9
10
11
12
13
14
Abstract. Two new direct methods for solving the linear algebraic equation Ax = b are proposed, and they include four steps. The first step is to convert A into a row-orthogonal matrix R using
elementary row operations, i.e., PAx = Rx . The second step is to left-multiply b by the converting
matrix P , which is Pb = b 0 . The third step is to reduce R to diagonal form, i.e., RR T (R T )−1 x = b 0
where D = RR T should be a diagonal matrix. The fourth step is to obtain the solution using
x = R T D −1 b 0 . Compared with the existing direct methods, the clear advantages of our methods
are that 1) the singularity of A can be easily judged through the row-orthogonal matrix R or the
diagonal matrix D, 2) our methods outperform other direct methods regardless of A is singular or
nonsingular, and 3) our methods can quickly calculate an upper bound of the relative error. The
results also hold for the complex domain.
1
2
15
Key words. direct method, row orthogonalization, ill-conditioned equation
16
AMS subject classifications. 65F05, 65F25, 65G99
17
1. Introduction.
18
19
1.1. Origin of the problem. Consider the nth-order linear algebraic equation
as follows:
20
(1.1)
21
22
23
where A ∈ R n×n and x , b ∈ R n×1 . Then to solve x , there are two types of methods
that can be used.
1) Direct methods. This class of methods attempts to solve the problem using
a finite sequence of operations. In the absence of rounding errors, direct
methods would deliver an exact analytic solution. Examples of direct methods
include Gaussian elimination and diagonal pivoting [4].
2) Iterative methods. An iterative method is a mathematical procedure that
uses an initial point to generate a sequence of improving approximate solutions for a class of problems in which the n-th approximation is derived
from the previous approximations. Examples of iterative methods include
the Jacobi method, the Gauss-Seidel method, the SOR (successive over relaxation)method, the Newton-Raphson method and the CG (conjugate gradient)method.
As for the iterative methods, convergence is an unavoidable critical problem that
may lead to the failure to calculate an acceptable solution. The termination criterion is also a key to an iterative methods specific implementation. To transform the
original approximating operator to a better conditioned operator, a class of preconditioned iterative methods has been developed, such as PCG (preconditioned conjugate
gradient)method.
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
Ax = b
∗ Submitted
to the editors 26 April 2019.
of Electrical Engineering and Information, Sichuan University, Chengdu, Sichuan, China.(shuqin@scu.edu.cn).
‡ School of Electrical Engineering and Information, Sichuan University, Chengdu, Sichuan, China.(361799391@qq.com).
§ South-West Institute of Technical Physics, Chengdu, Sichusn, China.(yangyang judy@126.com).
† School
1
This manuscript is for review purposes only.
2
QIN SHU, SILIANG ZHAO, YUNXIU YANG
40
41
42
This paper mainly addresses direct methods. For comparison, we introduced a
recursive method that has some similarities with our methods (see in 1.2). All of
these methods can be applied into the process of algorithms in various fields.
43
44
45
1.2. Summary of the direct methods. According to the classifications of
Abaffy and Spedikatto [1] and Duff and Reid [5], the direct methods for solving linear
systems can be classified as follows.
1) In the first class of methods, the initial system is transformed into a system
with smaller ranks in every step, and they include Gaussian elimination and
diagonal pivoting [4]. In addition, there is another important class of methods
named direct projection methods [2] that have been proved to be equal to
Gaussian elimination [13].
2) The second category includes methods for converting the matrix of the original system into another matrix. Solving the linear algebraic equations with
the other matrix is easy and can be accomplished, for instance, by reducing
the matrix to a triangular or diagonal form.
Moreover, the accuracy of the regular direct methods is very limited, and they
even fail to solve the ill-conditioned algebraic equations that are often encountered in
engineering. In 1971, based on Kahans [7] generalization of a pivot that include 2 × 2
principal submatrices, Bunch [4] provided a diagonal pivoting method by selecting the
pivot as being 1 × 1 or 2 × 2 when A is symmetric. In 1992, Yu et al. [12] proposed a
recursive algorithm combining the Gram-Schmidt technique and the Kaczmarz iterative method; in the work, the authors decomposed the matrix A into LQ, where L, Q
is a lower triangular matrix and row-orthogonal matrix, respectively. Then the Kaczmarz projection method is used to calculate the ultimate solution x , but it requires
a long computing time for the decomposition of LQ, and the numerical error is big.
In 1996, based on the LQ decomposition method, Kim et al. [8] presented a unique
method that could improve the accuracy of the solution by enhancing the condition
of the ill-conditioned matrix.
In this paper, we converted the matrix A into a row-orthogonal matrix, and
then the row-orthogonal matrix was transformed into a diagonal matrix. Therefore
we name our methods the Orthogonal Elimination (OE) method and the Improved
Orthogonal Elimination (IOE) method, respectively.
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
2. Two new methods.
73
74
75
2.1. The principle of new methods. First, we convert the matrix A into the
row-orthogonal matrix R using elementary row operations, such as the Gram-Schmidt
technique. Let P be the converting matrix. Then,
76
(2.1)
77
Second, we perform the same row operations as A for b, which is
78
(2.2)
79
With 2.1 and 2.2, we can obtain Rx = b. Third, suppose that x = R T x 0 . Then,
80
(2.3)
81
Finally, we can obtain the solution x as
82
(2.4)
P Ax = Rx
P b = b0
RRT x0 = b0
x = RT (RRT )−1 b0
This manuscript is for review purposes only.
TWO NEW DIRECT METHODS BASED ON ROW-ORTHOGONALIZATION
3
83
84
85
From the above, compared with the Gauss-Jordan elimination method, no back
substitution is needed in our new methods, so the solution is more stable. The following is the specific implementation of our new methods.
86
87
2.2. Orthogonal Elimination(OE) method. Here is the Orthogonal Elimination(OE) methods specific implementation.
Let A = [ã1 , ã2 , . . . , ãn ]T , b = (b1 , b2 , . . . , bn )T , and x = (x1 , x2 , . . . , xn )T . Then
we can obviously obtain the equations ã1 x = b1 , ã2 x = b2 , . . . , ãn x = bn . Then, we
convert A and b into R = [r̃1 , r̃2 , . . . , r̃n ]T and b 0 = (b10 , b20 , . . . , bn0 )T , respectively, by
performing the same Gram-Schmidt technique. r̃i and b 0 i can be calculated as follows.
When i = 1 , and r̃1 = ã1 , we compute r̃i (i ≥ 2) as
88
89
90
91
92
93
(2.5)
r̃ = ãi −
i−1
X
j =1
94
95
hãi , r̃j i
r̃j
hr̃j , r̃j i
Then, we right-multiply by x on the both sides of equation (2.5) as
(2.6)
r̃i x = ãi x −
i−1
X
j =1
96
hãi , r̃j i
r̃j x
hr̃j , r̃j i
With Ax = b and Rx = b 0 , we obtain
97
98
(2.7a)
ãi x = bi
99
(2.7b)
r̃i x = bi0
100
101
Then, we substitute equations (2.7a) and (2.7b) into equation (2.6), where b10 = b1 .
(2.8)
bi0 = bi −
i−1
X
j =1
102
103
104
105
hãi , r̃j i ˜0
b
hr̃j , r̃j i j
According to the recursions of equations (2.6) and (2.8), we obtain
Rx = b 0
(2.9)
Suppose that x = R T x 0 . Then,
RR T x 0 = b 0
(2.10)
where RR T should be a diagonal matrix in

rd(1,1)
0
 0
r
d(2,2)

RR T =  .
107 (2.11)
..
 ..
.
106
0
0
exact arithmetic.

···
0
···
0 


..
.
0 
···
rd(n,n)
From matrix multiplication theorem, x 0 can be computed as

 0
b1 rd(1,1)
 b02 rd(2,2) 


0
109 (2.12)
x =

..


.
0
bn rd(n,n)
108
This manuscript is for review purposes only.
4
QIN SHU, SILIANG ZHAO, YUNXIU YANG
Finally, we obtain x by substituting x 0 into x = R T x 0 .
Next, we combine equations (2.5), (2.8), (2.11), and (2.12), the new direct method,
the Orthogonal Elimination(OE) algorithm, can be given as follows.
113 Algorithm 2.1 Orthogonal Elimination(OE)
110
111
112
114
115
116
117
118
119
120
121
122
123
set r̃1 = ã1 , b10 = b1
for i = 1 ; i < n; i + + do
for j = 1 ; j < i − 1 ; j + + do
Pi−1 hã ,r̃ i
r̃i = ãi − j =1 hr̃ji ,r̃jj i r̃j
Pi−1 hã ,r̃ i
bi0 = bi − j =1 hr̃ji ,r̃jj i b˜j0
end for
end for

rd(1,1)
0
 0
r
d(2,2)

Compute RR T =  .
..
.
 .
.
···
···
..
.
···
0
0

 0
b1 rd(1,1)
 b02 rd(2,2) 


0
Compute x = 

..


.
0
bn rd(n,n)
Finally, x = R T x 0

0
0
0
rd(n,n)




2.3. Improved Orthogonal Elimination(IOE) method. From the analysis
of equations (2.5) and (2.8), it can be seen that the value of hr̃j , r̃j i may become
increasingly smaller as the recursion continues until the numerical reliability is lost.
In view of this situation, we proposed an improved algorithm. The specific process is
as follows.
129
We multiply hr̃j , r̃j i (j = 1 , 2 , . . . , i − 1 ) on both sides of equations (2.5) and (2.8)
130 in order to improve the accuracy of the calculation when the denominator is getting
131 increasingly closer to zero.
124
125
126
127
128
132
(2.13a)
r̃i
i−1
Y
hr̃j , r̃j i = ãi
j =1
i−1
Y
hr̃j , r̃j i −
i−1
X
i−1
Y
hr̃k , r̃k i hãi , r̃j i r̃j
j =1 k =1 ,k 6=j
j =1
133
134
(2.13b)
bi0
i−1
Y
j =1
hr̃j , r̃j i = bi0
i−1
Y
j =1
hr̃j , r̃j i −
i−1
X
i−1
Y
hr̃k , r̃k i hãi , r̃j i bj0
j =1 k =1 ,k 6=j
Qi−1
Qi−1
Suppose that r̃i0 = r̃i j =1 hr̃j , r̃j i, bi00 = bi0 j =1 hr̃j , r̃j i, R 0 = [r̃10 , r̃20 , . . . , r̃n0 ]T ,
00 T
136 and b 00 = [b100 , b200 , . . . , bn
] . According to the vector orthogonal theorem, R 0 is still
137 a row orthogonal matrix after the above transformation. Therefore,
 0

rd(1,1)
0
···
0
0
 0
rd(2,2)
···
0 


R 0 R 0T =  .
138 (2.14)

.
.
..
..
 ..
0 
0
0
0
· · · rd(n,n)
135
139
By referring to equations (2.9), (2.10) and (2.12), similar formula can be obtained:
This manuscript is for review purposes only.
TWO NEW DIRECT METHODS BASED ON ROW-ORTHOGONALIZATION
5
140
(2.15a)
R0x = D
(2.15b)
R 0 R 0T x 00 = D
(2.15c)
.

0
d1 rd(1,1)
 .



0
d2 rd(2,2)



x 00 = 


..


.
 .

0
dn rd(n,n)
147
(2.15d)
x = R 0T x 00
148
149
150
151
152
The problem that the denominators hr̃j , r̃j i in equations 2.5 and 2.8 tend to zero
is effectively solved by the above procedures. Q
We will show this improvement in 5.
i−1
However, in actual computations, the value of j =1 hr̃j , r̃j i tending to infinity is also
possible as the recursion continues. Given this, we can combine the OE method with
the IOE method. That is, if the Euclidean norm of hr̃j , r̃j i is less than 1, we use the
IOE method(IOE). Conversely, we employ the OE method.
In particular, the value of hr̃j , r̃j i is equal to 0 for some j ∈ [2 , n] when the matrix
A of the system is singular. Therefore, regularization is needed in equation (2.15c),
i.e.

 .
0
b001
rd(1,1)
+ε
 .

 00

0
b2
rd(2,2)
+ε 

00


(2.16)
x =

..


 . .

141
142
143
144

145
146
153
154
155
156
157
b00n
158
159
160
0
rd(n,n)
+ε
where 0 < ε 1 . Until now, the steps of the improved method can be detailed as
follows.
Algorithm 2.2 Orthogonal Elimination(IOE)
169
set r̃1 = ã1 , b10 = b1
for i = 1 ; i < n; i + + do
for j = 1 ; j < i − 1 ; j + + do
if kr̃j k2 < 1 then
set r̃i = hr̃j , r̃j i r̃i = hr̃j , r̃j i ãi − hãi , r̃j i r̃j
set bi0 = hr̃j , r̃j i bi0 = hr̃j , r̃j i bi − hãi , r̃j i b˜j0
else
Pi−1 hã ,r̃ i
set r̃ = ãi − j =1 hr̃ji ,r̃jj i r̃j
Pi−1 hãi ,r̃j i ˜0
set b 0 = bi −
b
170
171
172
end if
end for
end for
161
162
163
164
165
166
167
168
i
j =1 hr̃j ,r̃j i j
This manuscript is for review purposes only.
6
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
QIN SHU, SILIANG ZHAO, YUNXIU YANG
Compute RR T

rd(1,1)
 0

= .
 ..
0
rd(2,2)
..
.
···
···
..
.
0
0
···

 .
0
0
b
r
+ε
 1 . d(1,1)


 0
0
+ε 
b2 rd(2,2)


Compute x 0 = 


..


.
 .

0
b0n rd(n,n)
+ε
0
0
0
rd(n,n)





Finally, x = R T x 0
3. Singularity judgment of matrix A. According to the equations 2.1 and 2.2,
PAx = Rx = Pb = b 0 where P ∈ R n×n is a lower triangular matrix that is obtained
by the elementary row operations. Since the elementary operations do not change the
rank of a matrix [6], rank (PA) = rank (A), i.e., rank (R) = rank (A). According to the
fundamental equalities involving the rank, we know that rank (RR T ) = rank (R), and
thus rank (D) = rank (R) = rank (A), where D = RR T is a diagonal matrix.
Then, the singularity of A can be judged by analyzing the row-orthogonal matrix
R or diagonal matrix D. If A is singular, let the row-vector a~i be linearly dependent
with the first i − 1 linearly independent row-vectors. Without the rounding error,
the row-orthogonal result of a~i would be a zero vector. Therefore, we can judge taht
A is singular when the Euclidean norm of a row of R tends to zero. Correspondingly,
there is a diagonal element that tends to zero in D.
4. Error analysis of OE and IOE. Inspired by the work of Bunch [3], [4] and
Wilkinson [9], [10], [11]. we presented expressions of the error and the relative error.
190 We consider x0 as the exact solution of the system Ax = b, According to equations
191 2.1 and 2.2, its easy to know that Rx0 = b 0 and
−1
192 (4.1)
x = b0
RR T R T
188
189
which implies that x = R T (RR T )−1 b 0 . Finally, we substitute the equation Rx0 = b 0
into equation 4.1, and we obtain the computational expression of x .
−1
195 (4.2)
x = R T RR T
Rx0
193
194
201
Note that the elements of RR T were calculated with finite precision, and thus
RR would no longer be strictly diagonal due to the rounding error. Nevertheless,
we
still treat this variable as a diagonal matrix and define it as diag RR T . If desired,
the same regularization as was mentioned in 2.3 is also taken into account and the
error is given by
h
−1 i
(4.3)
x0 − x = I − R T diag RR T + εI
R x0
202
where 0 < ε 1 . The relative error that is usually a concern in engineering will be
203
(4.4)
196
197
198
199
200
204
205
T
−1
kx0 − x k2
≤ I − R T diag RR T + εI
R
kx0 k2
F
Through the analysis of the above formulas, the closer RR T is to diagonal matrix, the smaller the error that was caused by the rounding error will be. Without
This manuscript is for review purposes only.
TWO NEW DIRECT METHODS BASED ON ROW-ORTHOGONALIZATION
7
−1
the rounding error, the value of the error expression [I − R T diag RR T
R]x0
should be zero. Numerical experiments (see section 6)proved the validity of relative
error expression. As a result, equation (4.4) can be a useful estimation before the
209 calculation.
206
207
208
210
5. Comparisons of operation counts.
5.1. The operation counts of the OE method and the upper bound of
the IOE’s operation counts.
213
• Step 1. The operation counts of the r̃i (i = 2 , 3 , . . . , n) is shown in Table 1
(OE) and Table 2 (IOEs upper bound).
211
212
Table 1
Operation counts of the OE to calculate r̃i (i = 2 , 3 , . . . , n)
The i-th row orthogonal vector
Number of multiplications
Number of additions
2
3
...
i
...
n
3n + 0 × 2n + 1
3n + 1 × 2n + 2
...
3n + (i − 2 ) × 2n + i − 1
...
3n + (n − 2 ) × 2n + n − 2
2n − 2 + 1 × (n − 1 ) + 1 × n
2n − 1 + 2 × (n − 1 ) + 2 × n
...
2n − 2 + (i − 1 ) × (n − 1 ) + (i − 1 ) × n
...
2n − 2 + (n − 1 ) × (n − 1 ) + (n − 1 ) × n
214
Table 2
Operation counts(upper bound) of the IOE to calculate r̃i (i = 2 , 3 , . . . , n)
The i-th row orthogonal vector
Number of multiplications
Number of additions
2
3
...
i
...
n
4n
3 × (2n + 1 )
...
i × (2n + 1 )
...
n × (2n + 1 )
2 × (n − 1 ) + n
3 × (n − 1 ) + 2n
...
i × (n − 1 ) + (i − 1 ) × n
...
n × (n − 1 ) + (n − 1 ) × n
223
224
225
226
According to Table 1 and Table 2, the operation counts when we calculate
r̃i are
3
2
as follows. For the OE method, the number
of
multiplications
is
n
−
n
2
−2n
+
1 /2 ,
and the number of additions is n 3 + n 2 2 − 7n/2 + 2 . For the IOE methods upper
bound, the number of multiplications
is n 3 + 3n 2 2 − 3n/2 − 3 , and the number
3
2
of additions is n − n 2 − 3n/2 + 1 .
• Step 2. When we calculate bi0 , the number of multiplications and additions is
i − 1 , respectively. Then, when i ranges from 2 to n, the number of multiplications
and additions both are n(n − 1 )/2 .
• Step 3. To acquire equation (2.7b), equation (2.8) and x = R T x 0 , the numbers
of calculation that are required are as follows. The number of multiplications is
2n 2 + n, and the number of additions is 2n(n − 1 ).
In summary, the total numbers of this papers algorithm are shown in Table 3.
227
228
229
230
5.2. Operation counts of the LQ decomposition with kaczmarz iteration [12]. An arbitrary nonsingular coefficient matrix A can be decomposed into the
following form, A = LQ. Here, Q = (ã10 , ã20 , . . . , ãn0 )T is a row-orthogonal matrix and
ãi0 can be calculated using the Gram-Schmidt technique as follows.
215
216
217
218
219
220
221
222
This manuscript is for review purposes only.
8
QIN SHU, SILIANG ZHAO, YUNXIU YANG
Table 3
Operation counts of this papers algorithm
Method
Number of multiplications
Orthogonal Elimination
3n
1
n + 2n −
+
2
2
3
2
n + 4n − n − 3
Improvement of Orthogonal Elimination(upper bound)
3
2
Pi−1 hãi ,ãj0 i 0
j =1 hã 0 ,ã 0 i ãj where i ∈ [1 , n]. In addition, L is
j
j
232 matrix that can be calculated as below,

1
0
0
···
0
 hã2 ,ã10 i
 0 0
1
0
···
0
 hã1 ,ã1 i
 ã ,ã 0
0
hã
,
ã
i
3
h 3 1i
2
1
···
0
233
L=
 hã10 ,ã10 i hã20 , ã20 i
 .
..
..
..
..
 .
.
.
.
.
 .
 hãn ,ã 0 i
0
hãn ,ã20 i
hãn ,ã30 i
hãn ,ãn−1
i
1
·
·
·
0
0
0
0
0
0
0
0
hã1 ,ã1 i
hã2 ,ã2 i
hã3 ,ã3 i
hãn−1 ,ãn−1
i
231
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
ã 0 = ãi −
Number of additions
n 3 + 3n 2 − 6n + 2
n 3 + 2n 2 − 4n + 1
a lower-triangular

0

0



0

.. 

.

1
Then, the solution can be obtained by two steps.
• Step 1. We solve the equation Ly = b to obtain vector ỹ = (y1 , y2 , . . . , yn )T ,
Pi−1 hãi ,ã 0 i
where yi = bi − j =1 ã 0 ,ãj0 ỹj0 (i = 1 , 2 , 3 , . . . , n).
h j ji
• Step 2. Then, we solve the equation Qx = y to find the final solution. Since
matrix Q is row-orthogonal, this equation can be solved using Kaczmarz’s method
ã 0
as follows: x0 is arbitrary, x̃i−1 = ã 0 ,ãi 0 (yi − hãi , x̃i−1 i) and we set the number of
h i ii
iteration steps to n.
According to the above two steps, the operation counts of ai0 are the same as that
shown in Table 1, i.e., the numberof multiplications is n 3 − n 2 2 − 2n + 1 /2 and
the number of additions is n 3 + n 2 2 − 7n/2 + 2 . Then, the operation counts
to ob
3
2
tain yi and x̃i are the followding: the number of multiplications
is
(n
+n
)
2
+2n
+1
and the number of additions is (3n 2 + 5n − 4 ) 2 .
In other words, the total operation counts of the LQ decomposition with the
Kaczmarz
iteration algorithm [12] are as follows: the number of multiplications is
3n 3 2 + 3 /2 and the number of additions is n 3 + 2n 2 − n.
5.3. Operation counts of the Gauss-Jordan elimination with the partial
pivoting. The Gauss-Jordan elimination with the partial pivoting method is mainly
divided into three steps, including finding the pivot, the elimination on process and
the substitution process.
• Step 1. Partial pivoting requires one to bring the largest element in the first
column of the reduced matrix into the leading diagonal position. Therefore, the
number of comparisons is n(n − 1 )/2 .
• Step 2. The operation counts of the elimination process are shown in Table 4.
257
From table 4, we see that the number
of multiplications is (2n 3 + 3n 2 − 5n) 6 ,
259 and the number of additions is (n 3 − n) 3 .
260
• Step 3. The operation counts of back substitution process are as follows: the
261 number of multiplications is n(n + 1 )/2 and the number of additions is n(n − 1 )/2 .
258
This manuscript is for review purposes only.
TWO NEW DIRECT METHODS BASED ON ROW-ORTHOGONALIZATION
9
Table 4
Operation counts of the elimination process of the Gauss-Jordan elimination method
k-th row orthogonal vector
Number of multiplications
Number of additions
1
2
...
k
...
n-1
(n + 1 )(n − 1 )
n(n − 2 )
...
(n − k + 2 )(n − k )
...
3 ×1
n(n − 1 )
(n − 1 )(n − 2 )
...
(n − k + 1 )(n − k )
...
2 ×1
Finally, the total operation counts for the Gauss-Jordan
elimination method are
as follows: the number of multiplications is n 3 3 + n 2 − n/3 and the number of
264 additions is n 3 3 + n 2 2 − 5n/6 .
262
263
265
266
267
268
5.4. Operation counts of Bunchs diagonal pivoting method [4]. Based on
P(j )
Bunch’s idea [4], let
denote the summation over those indices i , where 1 6 i 6 n,
such that if A(i) exists, then it yields a j × j pivot, where j = 1 or 2 . Let p be the
number of 1 × 1 pivots that is used. Then, the operation counts for solving Ax = b
are shown in Table 5.
Table 5
Operation counts of the diagonal pivoting method
Operations
multiplications
additions
Exact
Upper bound
P(2 )
3n 2
5n
n3
+
−
+p+3
i
6
2
3
3
P(2 )
13n
n
+ n2 −
+p+3
i
6
6
9n 2
n
n3
+
−
6
4
6
5n 2
5n
1
n3
+
−
+
6
4
3
4
269
270
271
272
Through the comparison, the operation counts of all the abovementioned methods
are on the order of n 3 , but the operation counts of our methods are slightly larger than
those of the other methods, since the LQ decomposition with Kaczmarz iteration [12]
is the most complex. For clarity, we summarized the comparison below in Table 6.
Table 6
Comparison of the methods for solving Ax = b
Method
Orthogonal Elimination
Improvement of Orthogonal Elimination(upper bound)
LQ decomposition with the Kaczmarz iteration
Elimination(partial pivot)
Diagonal pivoting(upper bound)
Number of multiplications
3n
1
n + 2n −
+
2
2
3
2
n + 4n − n − 3
3
3
3n
+
2
2
3
n
n
+ n2 −
3
3
3
2
n
9n
n
+
−
6
4
6
3
2
Number of additions
n 3 + 3n 2 − 6n + 2
n 3 + 2n 2 − 4n + 1
n 3 + 2n 2 − n
n3
n2
5n
+
−
3
2
6
3
2
n
5n
5n
1
+
−
+
6
4
3
4
273
274
275
6. Experimental results. To further demonstrate the validity of our methods
in some computational situations, especially in the case of ill-conditioned equations
This manuscript is for review purposes only.
10
276
277
278
279
280
281
282
QIN SHU, SILIANG ZHAO, YUNXIU YANG
which are often encountered in scientific calculation, we introduced the following three
examples.
Problem 1. Solve the ill-conditioned linear equations Ax = b, where




23.0
5.0 7.0
6.0 5.0
32.0
7.0 10.0 8.0 7.0 



A=
6.0 8.0 10.0 9.0 , b = 33.0
31.0
5.0 7.0
9.0 10.0
Obviously, the theoretical solution is x = (1 .0 , 1 .0 , 1 .0 , 1 .0 )T , and the conditional number cond (A) = kAk2 A−1 2 is approximately 2984 when we only take the
integer part. Then, the comparison results(with 4 significant digits) of the OE and
IOE with the Gauss-Jordan elimination(partial pivot) method are listed in Table 7.
Table 7
Comparison of the results of this paper with the elimination(partial pivot) method(4 significant
digits)
283
284
285
286
287
288
289
290
Solution
OE
IOE
Elimination(partial pivot)
Exact solution
x1
x2
x3
x4
1 .011
0 .9948
0 .9958
0 .9998
1 .011
0 .9948
0 .9958
0 .9998
1 .039
0 .9764
0 .9898
1 .006
1 .000
1 .000
1 .000
1 .000
From Table 7, we can conclude that in the case of 4 significant digits, the results
of our methods are the same, and the relative errors of our methods and the GaussJordan elimination method(partial pivot) are 0 .6454 % and 2 .355 %, respectively.
Apparently, the accuracy of our methods is better than that of the Gauss-Jordan
elimination(partial pivot).
Problem 2. Let A be a Hilbert matrix as follows:


1
1/2
···
1/n
 1/2
1/3
···
1/(n + 1) 
1


A= .
, i.e., A(i , j ) =
.
..
.
.
.
.
i +j −1
 .

.
.
.
1/n 1/(n + 1) · · · 1/(2n − 1)
i , j ∈ [1 , n]
297
We select the exact solution x = (1 , 1 , . . . , 1 )T , and b = Ax . Then, we take the
order n of the equation from 5 to 50 with the adjacent interval 5, and the comparison results of the relative error of our my methods with the three other methods are presented in Table 8. Note that since the accuracy of the LQ decomposition [12] is affected by the initial iteration point xini , we ran it 10 times and took
the average value. In the table, the upper error bound denotes the value of equation
−1
−1
I − R T diag RR T
R
or I − R T diag RR T + εI
R .
298
299
300
301
302
Table 8 shows that the regular Gauss-Jordan elimination(partial pivot) method
and diagonal pivoting method can not find any acceptable numerical solution when
n > 10 . By applying the OE and IOE methods, more accurate numerical solution
can be obtained for any n < 50 than those provided by the LQ decomposition [12],
and the IOE is more stable than the OE as the order increases. In addition, the upper
291
292
293
294
295
296
F
This manuscript is for review purposes only.
F
TWO NEW DIRECT METHODS BASED ON ROW-ORTHOGONALIZATION
11
Table 8
Comparison of the relative error of this paper with three other methods
303
304
305
Unknowns
OE(%)
Upper error bound(%)
IOE(%)
Upper error bound(%)
LQ [12](%)
Elimination(partial pivot)(%)
Diagonal pivoting(%)
5
10
15
20
25
30
35
40
45
50
2.16 × 10−07
1.74 × 10−01
2.53 × 10+00
9.55 × 10+00
7.54 × 10+00
4.40 × 10+01
6.72 × 10+01
8.42 × 10+01
2.65 × 10+01
1.99 × 10+02
1.09 × 10−05
3.46 × 10+02
8.46 × 10+03
1.34 × 10+03
1.75 × 10+03
2.33 × 10+03
2.80 × 10+03
3.27 × 10+03
3.78 × 10+03
4.31 × 10+03
2.46 × 10−01
2.13 × 10+00
4.65 × 10+00
7.28 × 10+00
9.84 × 10+00
1.23 × 10+01
1.45 × 10+01
1.66 × 10+01
1.86 × 10+01
2.05 × 10+01
9.98 × 10+01
2.24 × 10+02
3.16 × 10+02
3.87 × 10+02
4.47 × 10+02
5.00 × 10+02
5.48 × 10+02
5.92 × 10+02
6.32 × 10+02
6.71 × 10+02
2.15 × 10+01
2.88 × 10+01
3.46 × 10+01
3.52 × 10+01
3.69 × 10+01
3.73 × 10+01
4.02 × 10+01
3.95 × 10+01
4.12 × 10+01
4.46 × 10+01
3.49 × 10−11
4.21 × 10−04
1.72 × 10+03
5.93 × 10+03
1.29 × 10+05
3.26 × 10+04
8.10 × 10+04
2.35 × 10+05
1.11 × 10+05
6.68 × 10+04
2.92 × 10−10
1.46 × 10−02
4.63 × 10+02
2.82 × 10+03
1.42 × 10+04
1.20 × 10+04
2.65 × 10+07
8.82 × 10+03
9.96 × 10+03
1.90 × 10+04
bound of the error analysis expression that is obtained in chapter 4 is proved to be
valid.
Problem 3. Consider the ill-conditioned linear equations Ax = b, where
A = (aij )n×n , b = [b1 , b2 , b3 , . . . , bn ]T
306
(
1
aij =
1 + p2
bi =
307
n
X
i 6= j
i=j
aik
i, j = 1, 2, 3, . . . , n
i = 1, 2, 3, . . . , n
k =1
Let p = 0 .5 × 10 −10 and randomly generate the real solution between [0 .5 , 1 .5 ].
Then, take a value for the order n of the equation from 100 to 500 with the adjacent
310 interval 100 . Table 9 shows the comparison results of the relative error of our method
311 with the three other methods. We also ran the LQ decomposition [12] 10 times and
312 took the average.
308
309
Table 9
Comparison of the relative error of this paper with three other methods
Unknowns
OE(%)
IOE(%)
Upper error bound(%)
LQ [12](%)
Elimination(partial pivot)(%)
Diagonal pivoting(%)
100
200
300
400
500
N aN
N aN
N aN
N aN
N aN
4.71 × 10+01
4.97 × 10+01
5.10 × 10+01
4.83 × 10+01
4.90 × 10+01
9.95 × 10+02
1.41 × 10+03
1.73 × 10+03
2.00 × 10+03
2.23 × 10+03
1.19 × 10+01
9.40 × 10+01
8.67 × 10+01
7.55 × 10+01
7.52 × 10+01
N aN
N aN
N aN
N aN
N aN
N aN
N aN
N aN
N aN
N aN
Obviously, the IOE method has the best performance, while the other three methods failed in their calculations. Furthermore, this result provides
Qi−1 an example of a
situation in which the OE method can not work anymore as j =1 hr˜j , r˜j i increases.
316 Finally, the Upper error bound is proved to be valid for estimating error range once
317 again.
313
314
315
318
319
320
321
322
323
7. Conclusions. Two new direct methods based on row orthogonalization have
been developed to solve the linear algebraic equation Ax = b. Compared with the
regular direct method of the Gauss-Jordan elimination(partial pivot) method, diagonal pivoting [4], and the recursive LQ decomposition method presented in [12], our
methods for solving the highly ill-conditioned linear algebraic equations outperform
the other three methods whether the coefficient matrix A is singular or nonsingular.
This manuscript is for review purposes only.
12
QIN SHU, SILIANG ZHAO, YUNXIU YANG
A useful upper bound of the relative error is presented, which is an important estimation in engineering calculations. Another meaningful result is that the singularity
of matrix A can be easily judged using the row-orthogonal matrix R or the diagonal
327 matrix D. In addition, the row-orthogonal matrix R can be used to find the inverse
328 of nonsingular coefficient matrix A or the pseudo-inverse of the singular coefficient
329 matrix A.
324
325
326
330
REFERENCES
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
[1] J. Abaffy and E. Spedicato, ABS projection algorithms: mathematical techniques for linear
and nonlinear equations, Prentice-Hall, Inc., 1989.
[2] M. Benzi and C. D. Meyer, A direct projection method for sparse linear systems, SIAM
Journal on Scientific Computing, 16 (1995), pp. 1159–1176.
[3] J. R. Bunch, Analysis of the diagonal pivoting method, SIAM Journal on Numerical Analysis,
8 (1971), pp. 656–680.
[4] J. R. Bunch and B. N. Parlett, Direct methods for solving symmetric indefinite systems of
linear equations, SIAM Journal on Numerical Analysis, 8 (1971), pp. 639–655.
[5] I. S. Duff and J. K. Reid, A comparison of some methods for the solution of sparse overdetermined systems of linear equations, IMA Journal of Applied Mathematics, 17 (1976),
pp. 267–280.
[6] R. A. Horn and C. R. Johnson, Matrix analysis, Cambridge university press, 2012.
[7] W. Kahan, Numerical linear algebra, Canadian Mathematical Bulletin, 9 (1966), pp. 757–801.
[8] H. J. Kim, K. Choi, H. Lee, H. Jung, and S. Hahn, A new algorithm for solving ill conditioned
linear systems, IEEE Transactions on Magnetics, 32 (1996), pp. 1373–1376.
[9] J. H. Wilkinson, Error analysis of direct methods of matrix inversion, Journal of the ACM
(JACM), 8 (1961), pp. 281–330.
[10] J. H. Wilkinson, The algebraic eigenvalue problem, vol. 662, Oxford Clarendon, 1965.
[11] J. H. Wilkinson, Rounding errors in algebraic processes, Courier Corporation, 1994.
[12] X. Yu, N. Loh, and W. Miller, New recursive algorithm for solving linear algebraic equations,
Electronics Letters, 28 (1992), pp. 2069–2071.
[13] A. I. Zhdanov, A direct sequential method for solving systems of linear algebraic equations,
Dokl. Akad. Nauk, 356 (1997), pp. 442–444.
This manuscript is for review purposes only.