Journal of Inequalities in Pure and
Applied Mathematics
http://jipam.vu.edu.au/
Volume 7, Issue 1, Article 34, 2006
MATRIX EQUALITIES AND INEQUALITIES INVOLVING KHATRI-RAO AND
TRACY-SINGH SUMS
ZEYAD AL ZHOUR AND ADEM KILICMAN
D EPARTMENT OF M ATHEMATICS AND I NSTITUTE FOR M ATHEMATICAL R ESEARCH
U NIVERSITY P UTRA M ALAYSIA (UPM), M ALAYSIA
43400, S ERDANG , S ELANGOR , M ALAYSIA
zeyad1968@yahoo.com
akilic@fsas.upm.edu.my
Received 05 August, 2004; accepted 02 September, 2005
Communicated by R.P. Agarwal
A BSTRACT. The Khatri-Rao and Tracy-Singh products for partitioned matrices are viewed as
generalized Hadamard and generalized Kronecker products, respectively. We define the KhatriRao and Tracy-Singh sums for partitioned matrices as generalized Hadamard and generalized
Kronecker sums and derive some results including matrix equalities and inequalities involving
the two sums. Based on the connection between the Khatri-Rao and Tracy-Singh products (sums)
and use mainly Liu’s, Mond and Pečarić’s methods to establish new inequalities involving the
Khatri-Rao product (sum). The results lead to inequalities involving Hadamard and Kronecker
products (sums), as a special case.
Key words and phrases: Kronecker product (sum), Hadamard product (sum), Khatri-Rao product (sum), Tracy-Singh product (sum), Positive (semi)definite matrix, Unitarily invariant norm, Spectral norm, P-norm, MoorePenrose inverse.
2000 Mathematics Subject Classification. 15A45; 15A69.
1. I NTRODUCTION
The Hadamard and Kronecker products are studied and applied widely in matrix theory,
statistics, econometrics and many other subjects. Partitioned matrices are often encountered in
statistical applications.
For partitioned matrices, The Khatri-Rao product viewed as a generalized Hadamard product,
is discussed and used in [7, 6, 14] and the Tracy-Singh product, as a generalized Kronecker
product, is discussed and applied in [7, 5, 12]. Most results provided are equalities associated
with the products. Rao, Kleffe and Liu in [13, 8] presented several matrix inequalities involving
the Khatri-Rao product, which seem to be most existing results. In [7], Liu established the
connection between Khatri-Rao and Tracy-Singh products based on two selection matrices Z1
and Z2 . This connection play an important role to give inequalities involving the two products
ISSN (electronic): 1443-5756
c 2006 Victoria University. All rights reserved.
148-04
2
Z EYAD A L Z HOUR
AND
A DEM K ILICMAN
with statistical applications. In [10], Mond and Pečarić presented matrix versions, with matrix
weights. In [2, (2004)], Hiai and Zhan proved the following inequalities:
kABk
kA + Bk
≤
,
kAk · kBk
kAk + kBk
kA ◦ Bk
kA + Bk
≤
kAk · kBk
kAk + kBk
(*)
for any invariant norm with kdiag(1, 0, . . . , 0)k ≥ 1 and A, B are nonzero positive definite
matrices.
In the present paper, we make a further study of the Khatri-Rao and Tracy-Singh products.
We define the Khatri-Rao and Tracy-Singh sums for partitioned matrices and use mainly Liu’s,
Mond and Pečarić’s methods to obtain new inequalities involving these products (sums).We collect several known inequalities which are derived as a special cases of some results obtained. We
generalize the inequalities in Eq (*) involving the Hadamard product (sum) and the Kronecker
product (sum).
2. BASIC D EFINITIONS AND R ESULTS
2.1. Basic Definitions on Matrix Products. We introduce the definitions of five known matrix
products for non-partitioned and partitioned matrices. These matrix products are defined as
follows:
Definition 2.1. Consider matrices A = (aij ) and C = (cij ) of order m × n and B = (bkl ) of
order p × q. The Kronecker and Hadamard products are defined as follows:
(1) Kronecker product:
A ⊗ B = (aij B)ij ,
(2.1)
where aij B is the ijth submatrix of order p × q and A ⊗ B of order mp × nq.
(2) Hadamard product:
A ◦ C = (aij cij )ij ,
(2.2)
where aij cij is the ijth scalar element and A ◦ C is of order m × n.
Definition 2.2. Consider matrices A = (aij ) and B = (bkl ) of order m × m and n × n respectively. The Kronecker sum is defined as follows:
(2.3)
A ⊕ B = A ⊗ In + Im ⊗ B,
where In and Im are identity matrices of order n × n and m × m respectively, and A ⊕ B of
order mn × mn.
Definition 2.3. Consider matrices A and C of order m × n, and B of order p × q. Let A = (Aij )
be partitioned with Aij of order mi × nj as the ijth submatrix,C = (Cij ) be partitioned with
Cij of order mi × nj as the ijth submatrix, and B = (Bkl ) be partitioned with Bkl of order
P
P
P
P
pk × ql as the klth submatrix, where,m = ri=1 mi , n = sj=1 nj , p = tk=1 pk , q = hl=1 ql
are partitions of positive integers m, n, p, and q. The Tracy-Singh and Khatri-Rao products are
defined as follows:
(1) Tracy-Singh product:
(2.4)
AΠB = (Aij ΠB)ij = (Aij ⊗ Bkl )kl ij ,
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
3
where Aij is the ijth submatrix of order mi ×nj , Bkl is the klth submatrix of order pk ×ql ,
Aij ΠB is the ijth submatrix of order mi p × nj q, Aij ⊗ Bkl is the klth submatrix of order
mi pk × nj ql and AΠB of order mp × nq.
Note that
(i) For a non partitioned matrix A, their AΠB is A ⊗ B, i.e., for A = (aij ), where aij
is scalar, we have,
AΠB = (aij ΠB)ij
= (aij ⊗ Bkl )kl ij
= (aij Bkl )kl ij = (aij B)ij = A ⊗ B.
(ii) For column wise partitioned A and B, their AΠB is A ⊗ B.
(2) Khatri-Rao product:
A ∗ B = (Aij ⊗ Bij )ij ,
(2.5)
where Aij is the ijth submatrix of order mi × nj , Bij is the ijth submatrix of order
pi × qj , Aij ⊗ Bij is the ijth submatrix
of order mi pi × nj qj and A ∗ B of order M × N
Ps
Pr
M = i=1 mi pi , N = j=1 nj qj .
Note that
(i) For a non partitioned matrix A, their A ∗ B is A ⊗ B, i.e., for A = (aij ),where aij
is scalar, we have,
A ∗ B = (aij ⊗ Bij )ij = (aij B)ij = A ⊗ B.
(ii) For non partitioned matrices A and B, their A ∗ B is A ◦ B, i.e., for A = (aij ) and
B = (bij ), where aij and bij are scalars, we have,
A ∗ B = (aij ⊗ bij )ij = (aij bij )ij = A ◦ B.
2.2. Basic Connections and Results on Matrix Products. We introduce the connection between the Katri-Rao and Tracy-Singh products and the connection between the Kronecker and
Hadamard products, as a special case, which are important in creating inequalities involving
these products. We write A ≥ B in the Löwner ordering sense that A − B ≥ 0 is positive
semi-definite, for symmetric matrices A and B of the same order and A+ and A∗ indicate the
Moore-Penrose inverse and the conjugate of the matrix A, respectively.
Lemma 2.1. Let A = (aij ) and B = (bij ) be two scalar matrices of order m × n. Then (see
[15])
(2.6)
A ◦ B = K10 (A ⊗ B)K2
where K1 and K2 are two selection matrices of order n2 × n and m2 × m, respectively, such
that K10 K1 = Im and K20 K2 = In .
In particular, for m = n, we have K1 = K2 = K and
(2.7)
A ◦ B = K 0 (A ⊗ B)K
Lemma 2.2. Let A and B be compatibly partitioned. Then (see [8, p. 177-178] and [7, p. 272])
(2.8)
A ∗ B = Z10 (AΠB) Z2 ,
where Z1 and Z2 are two selection matrices of zeros and ones such that Z10 Z1 = I1 and Z20 Z2 =
I2 , where I1 and I2 are identity matrices.
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
4
Z EYAD A L Z HOUR
AND
A DEM K ILICMAN
In particular, when A and B are square compatibly partitioned matrices, then we have Z1 =
Z2 = Z such that Z 0 Z = I and
A ∗ B = Z 0 (AΠB) Z.
(2.9)
Note that, for non-partitioned matrices A, B, Z1 and Z2 , Lemma 2.2 leads to Lemma 2.1, as a
special case.
Lemma 2.3. Let A, B, C, D and F be compatibly partitioned matrices. Then
(2.10)
(AΠB)(CΠD) = (AC)Π(BD)
(2.11)
(AΠB)+ = A+ ΠB +
(A + C)Π(B + D) = AΠB + AΠD + CΠB + CΠD
(2.12)
(2.13)
(2.14)
(2.15)
(AΠB)∗ = A∗ ΠB ∗
AΠB 6= BΠA
A ∗ B 6= B ∗ A
(2.16)
B∗F =F ∗B
∗
∗
F = (fij )
where
(A ∗ B) = A ∗ B
(2.17)
in general
in general
fij
and
is a scalar
∗
(A + C) ∗ (B + D) = A ∗ B + A ∗ D + C ∗ B + C ∗ D
(2.18)
(A ∗ B)Π(C ∗ D) = (AΠC) ∗ (BΠD)
(2.19)
Proof. Straightforward.
Lemma 2.4. Let A and B be compatibly partitioned matrices. Then
(AΠB)r = Ar ΠB r ,
(2.20)
for any positive integer r.
Proof. The proof is by induction on r and using Eq. (2.10).
Theorem 2.5. Let A ≥ 0 and B ≥ 0 be compatibly partitioned matrices. Then
(AΠB)α = Aα ΠB α
(2.21)
for any positive real α.
Proof. By using Eq (2.20), we have AΠB = (A1/n ΠB 1/n )n , for any positive integer n. So
it follows that (AΠB)1/n = A1/n ΠB 1/n . Now (AΠB)m/n = Am/n ΠB m/n , for any positive
integers n, m. The Eq (2.21) now follows by a continuity argument.
Corollary 2.6. Let A and B be compatibly partitioned matrices. Then
|AΠB| = |A| Π |B| ,
(2.22)
where
|A| = (A∗ A)1/2
Proof. Applying Eq (2.10) and Eq (2.21), we get the result.
Theorem 2.7. Let A = (Aij ) and B = (Bkl ) be partitioned matrices of order m × m, and n × n
P
P
respectively, where m = ri=1 mi , n = tk=1 nk .Then
(2.23)
(a)
tr(AΠB) = tr(A) · tr(B)
(2.24)
(b)
kAΠBkp = kAkp kBkp ,
where
kAkp = [tr |A|p ]
Proof. (a) Straightforward.
(b) Applying Eq (2.22) and Eq (2.23), we get the result.
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
1/p
, for all 1 ≤ p < ∞.
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
5
Theorem 2.8. Let A, B and I be compatibly partitioned matrices. Then
(AΠI)(IΠB) = (IΠB)(AΠI) = AΠB.
(2.25)
If f (A) is an analytic function on a region containing the eigenvalues of A, then
(2.26)
f (IΠA) = IΠf (A) and
f (AΠI) = f (A)ΠI
Proof. The proof of Equation (2.25) is straightforward on applying Eq (2.10).
Equation (2.26) can be proved as follows:
P
k
Since f (A)is an analytic function, then f (A) = ∞
k=0 αk A . Applying Eq (2.10) we get:
∞
∞
∞
X
X
X
k
k
f (IΠA) =
αk (IΠA) =
αk (IΠA ) = IΠ
αk Ak = IΠf (A).
k=0
k=0
k=0
Corollary 2.9. Let A, B and I be compatibly partitioned matrices. Then
(2.27)
eAΠI = eA ΠI
and eIΠA = IΠeA .
Lemma 2.10. Let H ≥ 0 be a n × n matrix with nonzero eigenvalues λ1 ≥ · · · ≥ λk (k ≤ n)
and X be a m × m matrix such that X = H 0 X, where H 0 = HH + . Then (see [6, Section 2.3])
(2.28)
(X 0 HX)+ ≤ X + H + X
0+
≤
(λ1 + λk )2 0
(X HX)+ .
(4λ1 λk )
Theorem 2.11. Let A ≥ 0 and B ≥ 0 be compatibly partitioned matrices such that A0 = AA+
and B 0 = BB + . Then (see [8, Section 3])
(2.29)
(A ∗ B 0 + A0 ∗ B)(A ∗ B)+ (A ∗ B 0 + A0 ∗ B) ≤ A ∗ B + + A+ ∗ B + 2A0 ∗ B 0
Theorem 2.12. Let A > 0 and B > 0 be n×n compatibly partitioned matrices with eigenvalues
contained in the interval between m and M (M ≥ m). Let I be a compatible identity matrix.
Then (see [8, Section 3]).
m2 + M 2
m2 + M 2
(2.30)
A ∗ B −1 + A−1 ∗ B ≤
I and A ∗ A−1 ≤
I
mM
2mM
3. M AIN R ESULTS
3.1. On the Tracy-Singh Sum.
Definition 3.1. Consider matrices A and B of order m×m and n×n respectively. Let A = (Aij )
be partitioned with Aij of order mi × mi as the ijth submatrix, and let B = (Bij ) be partitioned
P
P
with Bij of order nk × nk as the ijth submatrix m = ri=1 mi , n = tk=1 nk .
The Tracy-Singh sum is defined as follows:
(3.1)
A∇B = AΠIn + Im ΠB,
where In = In1 +n2 +···+nt = blockdiag(In1 , In2 , . . . , Int ) is an n × n identity matrix, Im =
Im1 +m2 +···+mr = blockdiag(Im1 , Im2 , . . . , Imr ) is an m × m identity matrix, Ink is an nk × nk
identity matrix (k = 1, . . . , t), Imi is an mi × mi identity matrix (i = 1, . . . , r) and A∇B is of
order mn × mn.
Note that for non-partitioned matrices A and B, their A∇B is A ⊕ B.
Theorem 3.1. Let A ≥ 0, B ≥ 0, C ≥ 0 and D ≥ 0 be compatibly partitioned matrices. Then
(3.2)
(A∇B)(C∇D) ≥ AC∇BD.
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
6
Z EYAD A L Z HOUR
AND
A DEM K ILICMAN
Proof. Applying Eq (3.1) and Eq (2.10), we have
(A∇B)(C∇D) = (AΠI + IΠB)(CΠI + IΠD)
= (AΠI)(CΠI) + (AΠI)(IΠD) + (IΠB)(CΠI) + (IΠB)(IΠD)
= ACΠI + AΠD + CΠB + IΠBD
= AC∇BD + AΠD + CΠB ≥ AC∇BD.
In special cases of Eq (3.2), if C = A∗ , D = B ∗ , we have
(A∇B)(A∇B)∗ ≥ AA∗ ∇BB ∗
(3.3)
and if C = A, D = B, we have
(A∇B)2 ≥ A2 ∇B 2 .
(3.4)
More generally, it is easy by induction on w we can show that ifA ≥ 0 and B ≥ 0 are compatibly
partitioned matrices. Then
w−1 X
w
w
w
w
(3.5)
(A∇B) = A ∇B +
(Aw−k ΠB k );
k
k=1
(A∇B)w ≥ Aw ∇B w
(3.6)
for any positive integer w.
Theorem
Pt B be partitioned matrices of order m × m and n × n, respectively,
Pr 3.2. Let A and
m = i=1 mi , n = k=1 nk . Then
(3.7)
tr(A∇B) = n · tr(A) + m · tr(B),
(3.8)
kA∇Bkp ≤
√
p
n kAkp +
√
p
m kBkp ,
where kAkp = [tr |A|p ]1/p , 1 ≤ p < ∞, and
eA∇B = eA ΠeB .
(3.9)
Proof. For the first part, on applying Eq (2.23), we obtain
tr(A∇B) = tr [(AΠIn ) + (Im ΠB)]
= tr(AΠIn ) + tr(Im ΠB)
= tr(A) tr(In ) + tr(Im ) tr(B)
= n · tr(A) + m · tr(B).
To prove (3.8), we apply Eq (2.24), to get
kA∇Bkp = k(AΠIn ) + (Im ΠB)kp
≤ kAΠIn kp + kIm ΠBkp
= kAkp kIn kp + kIm kp kBkp
√
√
= p n kAkp + p m kBkp .
For the last part, applying Eq (2.25), Eq (2.27) and Eq (2.10), we have
eA∇B = e(AΠIn )+(Im ΠB)
= e(AΠIn ) e(Im ΠB)
= (eA ΠIn )(Im ΠeB ) = eA ΠeB .
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
7
Theorem 3.3. Let P
A and Bbe non
partitioned matrices of order m × m and n × n
Psingular
r
t
respectively, (m = i=1 mi , n = k=1 nk ).Then
(A∇B)−1 = (A−1 ∇B −1 )−1 (A−1 ΠB −1 )
(3.10)
(i)
(3.11)
(ii)
(A∇B)−1 = (A−1 ΠIn )(A−1 ∇B −1 )−1 (Im ΠB −1 )
(3.12)
(iii)
(A∇B)−1 = (Im ΠB −1 )(A−1 ∇B −1 )−1 (A−1 ΠIn )
Proof. (i) Applying Eq (2.10), we have
(A∇B)−1 = [Im ΠB + AΠIn ]−1
= [(Im ΠB)(Im ΠIn ) + (Im ΠB)(AΠB −1 )]−1
= [(Im ΠB)(Im ΠIn + AΠB −1 )]−1
= [(Im ΠIn + AΠB −1 )]−1 [Im ΠB]−1
= [(AΠIn )(A−1 ΠIn ) + (AΠIn )(Im ΠB −1 )]−1 [Im ΠB −1 ]
= [(AΠIn ){A−1 ΠIn + Im ΠB −1 }]−1 [Im ΠB −1 ]
= [(AΠIn )(A−1 ∇B −1 )]−1 [Im ΠB −1 ]
= (A−1 ∇B −1 )−1 (A−1 ΠIn )(Im ΠB −1 )
= (A−1 ∇B −1 )−1 (A−1 ΠB −1 ).
Similarly, we obtain (ii) and (iii).
Theorem 3.4. Let A ≥ 0 and I be compatibly partitioned matrices such that A+ ΠI = IΠA+ .
Then
A∇A+ ≥ 2AA+ ΠI.
(3.13)
Proof. We know that A∇I = AΠI + IΠI > AΠI. Denote H = M ΠI ≥ 0. By virtue of
H + H + ≥ 2HH + and Eq (2.10), we have
AΠI + (AΠI)+ ≥ 2(AΠI)(AΠI)+ = 2AA+ ΠI
Since, A+ ΠI = IΠA+ , we get the result.
3.2. On the Khatri-Rao Sum.
Definition 3.2. Let A, B, In and Im be partitioned as in Definition 3.1. Then the Khatri-Rao
sum is defined as follows:
(3.14)
A∞B = A ∗ In + Im ∗ B
Note that, for non-partitioned matrices A and B, their A∞B is A ⊕ B, and for non-partitioned
matrices A, B, In and Im , their A∞B is A • B (Hadamard sum, see Definition 4.1, Eq(4.1),
Section 4).
Theorem 3.5. Let A and B be compatibly partitioned matrices. Then
A∞B = Z 0 (A∇B)Z,
(3.15)
where Z is a selection matrix as in Lemma 2.2.
Proof. Applying Eq (2.9), we have A ∗ I = Z 0 (AΠI)Z, I ∗ B = Z 0 (IΠB)Z and
A∞B = A ∗ I + I ∗ B = Z 0 (AΠI) Z + Z 0 (IΠB) Z = Z 0 (A∇B)Z.
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
8
Z EYAD A L Z HOUR
AND
A DEM K ILICMAN
Corollary 3.6. Let A ≥ 0 and I be compatibly partitioned matrices such that A+ ΠI = IΠA+ .
Then
A∞A+ ≥ 2AA+ ∗ I
(3.16)
Proof. Applying Eq(3.13) and Eq (3.15), we get the result.
Corollary 3.7. Let A > 0 be compatibly partitioned with eigenvalues contained in the interval
between m and M (M ≥ m). Let I be a compatible identity matrix such that A−1 ∞I =
I∞A−1 . Then
m2 + M 2
(3.17)
A∞A−1 ≤
I.
mM
Proof. Applying Eq (2.30) and taking B = I, we get the result.
Corollary 3.8. Let A ≥ 0 and I be compatibly partitioned, where A0 = AA+ such that A0 ∗I =
I ∗ A0 . Then
(A∞A0 )(A ∗ I)+ (A∞A0 ) ≤ A ∗ I + A+ ∗ I + 2A0 ∗ I
(3.18)
and if A+ ∗ I = I ∗ A+ , we have
(A∞A0 )(A ∗ I)+ (A∞A0 ) ≤ A∞A+ + 2A0 ∗ I.
(3.19)
Proof. Applying Eq (2.29) and taking B = I, we get the results.
Mond and Pečarić (see [10]) proved the following result:
If Xj (j = 1, 2, . . . , k) are positive definite Hermitian matrices of order n×n with eigenvalues
P
in the interval [m, M ] and Uj (j = 1, 2, . . ., k) are r × n matrices such that kj=1 Uj Uj∗ = I.
Then
(a) For p < 0 or p > 1, we have
!p
k
k
X
X
p ∗
(3.20)
Uj Xj Uj ≤ λ
Uj Xj Uj∗
j=1
j=1
where,
γp − γ
λ=
(p − 1)(γ − 1)
(3.21)
p(γ − γ p )
(1 − p)(γ p − 1)
−p
,
γ=
M
.
m
While, for 0 < p < 1, we have the reverse inequality in Eq (3.20).
(b) For p < 0 or p > 1, we have
!
!p
k
k
X
X
(3.22)
Uj Xjp Uj∗ −
Uj Xj Uj∗
≤ αI,
j=1
j=1
where,
(3.23)
α = mp −
M p − mp
p(M − m)
p
p−1
M p − mp
+
(M − m)
"
M p − mp
p(M − m)
1
p−1
#
−m .
While, for 0 < p < 1, we have the reverse inequality in Eq (3.22).
We have an application to the Khatri-Rao product and Khatri-Rao sum.
Theorem 3.9. Let A and B be positive definite Hermitian compatibly partitioned matrices and
let m and M be, respectively, the smallest and the largest eigenvalues of AΠB. Then
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
9
(a) For p a nonzero integer, we have
Ap ∗ B p ≤ λ(A ∗ B)p
(3.24)
where, λ is given by Eq (3.21).
While, for 0 < p < 1, we have the reverse inequality in Eq (3.24).
(b) For p a nonzero integer, we have
(Ap ∗ B p ) − (A ∗ B)p ≤ αI,
(3.25)
where α is given by Eq (3.23).
While, for 0 < p < 1, we have the reverse inequality in Eq (3.25).
Proof. In Eq (3.20) and Eq (3.22), take k = 1 and instead of U ∗ , use Z, the selection matrix
which satisfy the following property:
A ∗ B = Z 0 (AΠB)Z,
Z 0 Z = I.
Making use of the fact in Eq (2.21) that for any real n (positive or negative), we have
(AΠB)n = An ΠB n ,
then, with Z 0 , AΠB, Z substituted for U , X, U ∗ , we have from Eq (3.20)
Ap ∗ B p = Z 0 (Ap ∗ B p )Z
= Z 0 (A ∗ B)p Z
p
≤ λ {Z 0 (AΠB)Z} = λ(A ∗ B)p ,
where, λ is given by Eq (3.21)
Similarly, from Eq (3.22), we obtain for
(Ap ∗ B p ) − (A ∗ B)p ≤ αI
where, α is given by Eq (3.23).
Special cases include from Eq (3.24):
(2.1) For p = 2, we have
(3.26)
A2 ∗ B 2 ≤
(M + m)2
{A ∗ B}2
4M m
(2.2) For p = −1, we have
(M + m)2
{A ∗ B}−1
4M m
Similarly, special cases include from Eq (3.25):
(2.1) For p = 2, we have
1
(3.28)
(A2 ∗ B 2 ) − (A ∗ B)2 ≤ (M − m)2 I
4
(2.2) For p = −1, we have
√
√
M− m
−1
−1
−1
{I} ,
(3.29)
(A ∗ B ) − (A ∗ B) ≤
Mm
where results in Eq (3.26), Eq (3.27), and Eq (3.28) are given in [7].
(3.27)
A−1 ∗ B −1 ≤
Theorem 3.10. Let A and B be positive definite Hermitian compatibly partitioned matrices.
Let m1 and M1 be, respectively, the smallest and the largest eigenvalues of AΠI and m2 and
M2 , respectively, the smallest and the largest eigenvalues of IΠB. Then
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
10
Z EYAD A L Z HOUR AND A DEM K ILICMAN
(a) For p a nonzero integer, we have
Ap ∞B p ≤ max {λ1 , λ2 } (A∞B)p
(3.30)
where,
(γ1p − γ1 )
λ1 =
[(p − 1)(γ1 − 1)]
(3.31)
(γ2p − γ2 )
λ2 =
[(p − 1)(γ2 − 1)]
(3.32)
p(γ1 − γ1p )
[(1 − p)(γ1p − 1)]
−p
p(γ2 − γ2p )
[(1 − p)(γ2p − 1)]
,
γ1 =
−p
,
γ2 =
M1
,
m1
M2
.
m2
While, for 0 < p < 1, we have the reverse inequality in Eq (3.30).
(b) For p a nonzero integer, we have
(Ap ∞B p ) − (A∞B)p ≤ max {α1 , α2 } I
(3.33)
where,
(3.34)
(3.35)
α1 = mp1 −
α2 = mp2 −
M1p − mp1
p(M1 − m1 )
p
p−1
M2p − mp2
p(M2 − m2 )
p
p−1
M p − mp1
+ 1
M1 − m 1
(
M p − mp2
+ 2
M2 − m 2
(
M1p − mp1
p(M1 − m1 )
1
p−1
M2p − mp2
p(M2 − m2 )
1
p−1
)
− m1
)
− m2
While, for 0 < p < 1, we have the reverse inequality in Eq (3.33).
Proof. Applying Eq (3.24), we have
Ap ∗ I = Ap ∗ I p ≤ λ1 (A ∗ I)p
I ∗ B p = I p ∗ B p ≤ λ2 (I ∗ B)p
Now,
Ap ∞B p = Ap ∗ I + I ∗ B p
≤ λ1 (A ∗ I)p + λ2 (I ∗ B)p
≤ max {λ1 , λ2 } [A ∗ I + I ∗ B]p = max {λ1 , λ2 } (A∞B)p
where, λ1 and λ2 are given in Eq (3.31) and Eq (3.32).
Similarly, from Eq (3.25), we obtain for
(Ap ∞B p ) − (A∞B)p ≤ max {α1 , α2 } I
where, α1 and α2 are given in Eq (3.34) and Eq (3.35).
Special cases include from Eq (3.30):
(2.1) For p = 2, we have
(M1 + m1 )2 (M2 + m2 )2
2
2
(3.36)
A ∞B ≤ max
,
{A∞B}2 .
4M1 m1
4M2 m2
(2.2) For p = −1, we have
(3.37)
−1
A ∞B
−1
≤ max
(M1 + m1 )2 (M2 + m2 )2
,
4M1 m1
4M2 m2
{A∞B}−1 .
Similarly, special cases include from Eq (3.33):
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
11
(2.1) For p = 2, we have
(3.38)
2
2
2
(A ∞B ) − (A∞B) ≤ max
(2.2) For p = −1, we have
(3.39)
−1
−1
−1
(A ∞B ) − (A∞B)
1
2 1
2
(M1 − m1 ) , (M2 − m2 ) I.
4
4
√
≤ max
√
√
√ M1 − m 1 M 2 − m 2
,
I.
4M1 m1
4M2 m2
Theorem 3.11. Let A and B be positive definite Hermitian compatibly partitioned matrices.
Let m and M be, respectively, the smallest and the largest eigenvalues of A∇B. Then
(a) For p a nonzero integer, we have
Ap ∞B p ≤ λ(A∞B)p ,
(3.40)
where λ is given by Eq (3.21).
While, for 0 < p < 1, we have the reverse inequality in Eq (3.40).
(b) For p a nonzero integer, we have
(Ap ∞B p ) − (A∞B)p ≤ αI
(3.41)
where, α is given by Eq (3.23).
While, for 0 < p < 1, we have the reverse inequality in Eq (3.41).
Proof. In Eq (3.20) and Eq (3.22), take k = 1 and instead of U ∗ , use Z, the selection matrix
which satisfy the following property:
A∞B = Z 0 (A∇B)Z,
Z 0Z = I
Then, with Z 0 , A∇B, Z substituted for U , X, U ∗ , we have from Eq (3.20)
Ap ∞B p = Z 0 (Ap ∇B p )Z
= Z 0 (Ap ΠI + IΠB p )Z
≤ Z 0 {A∇B}p Z
p
≤ λ {Z 0 (A∇B)Z} = λ(A∞B)p
where, λ is given by Eq (3.21).
Similarly, from Eq (3.22), we obtain Eq (3.41)
Special cases include from Eq (3.40):
(2.1) For p = 2, we have
(3.42)
A2 ∞B 2 ≤
(M + m)2
{A∞B}2
4M m
(2.2) For p = −1, we have
(M + m)2
{A∞B}−1
4M m
Similarly, special cases include from Eq (3.41):
(2.1) For p = 2, we have
1
(3.44)
(A2 ∞B 2 ) − (A∞B)2 ≤ (M − m)2 I
4
(2.2) For p = −1, we have
√
√
M− m
−1
−1
−1
{I}
(3.45)
(A ∞B ) − (A∞B) ≤
Mm
(3.43)
A−1 ∞B −1 ≤
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
12
Z EYAD A L Z HOUR AND A DEM K ILICMAN
4. S PECIAL R ESULTS ON H ADAMARD
AND
K RONECKER S UMS
The results obtained in Section 3 are quite general. Now, we consider some inequalities in
a special case which involves non-partitioned matrices A, B and I with the Hadamard product
(sum) replacing the Khatri-Rao product (sum) and the Kronecker product (sum) replacing the
Tracy-Singh product (sum). As these inequalities can be viewed as a corollary (some of) the
proofs are straightforward and alternative to those for the existing inequalities.
Definition 4.1. Let A and B be square matrices of order n × n.The Hadamard sum is defined
as follows:
(4.1)
A • B = A ◦ In + In ◦ B = A ◦ In + B ◦ In = (A + B) ◦ In .
Corollary 4.1. Let A > 0. Then
A • A−1 ≥ 2I.
(4.2)
Corollary 4.2. Let A > 0 be a matrix of order n × n with eigenvalues contained in the interval
between m and M (M ≥ m). Then
(m2 + M 2 )
{I} .
mM
Corollary 4.3. Let A and B be n × n positive definite Hermitian matrices and let m and M be,
respectively, the smallest and the largest eigenvalues of A ⊗ B. Then
(a) For p a nonzero integer, we have
A • A−1 ≤
(4.3)
Ap ◦ B p ≤ λ(A ◦ B)p
(4.4)
where, λ is given by Eq (3.21).
While, for 0 < p < 1, we have the reverse inequality in Eq (4.4).
(b) For p is a nonzero integer, we have
(Ap ◦ B p ) − (A ◦ B)p ≤ αI
(4.5)
where, α is given by Eq (3.23).
While, for 0 < p < 1, we have the reverse inequality in Eq (4.5).
Special cases include from Eq (4.4):
(2.1) For p = 2, we have
(4.6)
A2 ◦ B 2 ≤
(M + m)2
{A ◦ B}2
4M m
(2.2) For p = −1, we have
(M + m)2
{A ◦ B}−1 .
4M m
Similarly, special cases include from Eq (4.5):
(2.1) For p = 2, we have
1
(4.8)
(A2 ◦ B 2 ) − (A ◦ B)2 ≤ (M − m)2 I
4
(2.2) For p = −1, we have
√
√
M− m
−1
−1
−1
{I} ,
(4.9)
(A ◦ B ) − (A ◦ B) ≤
Mm
where results in Eq (4.6), Eq (4.7), and Eq (4.8) are given in [11].
(4.7)
A−1 ◦ B −1 ≤
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
13
We note that the eigenvalues of A ⊗ B are the n2 products of the eigenvalues of A by the
eigenvalues of B.Thus if the eigenvalues of A and B are, respectively, ordered by:
(4.10)
δ1 ≥ δ2 ≥ · · · ≥ δn > 0,
η1 ≥ η2 ≥ · · · ≥ ηn > 0,
then in all the previous results in this section M = δ1 η1 and m = δn ηn . Thus Eq (4.6) to Eq
(4.9) become:
(4.11)
A2 ◦ B 2 ≤
(4.12)
A−1 ◦ B −1 ≤
(δ1 η1 + δn ηn )2
{A ◦ B}2
4δ1 η1 δn ηn
(δ1 η1 + δn ηn )2
{A ◦ B}−1
4δ1 η1 δn ηn
1
(A2 ◦ B 2 ) − (A ◦ B)2 ≤ (δ1 η1 − δn ηn )2 {I}
4
(4.13)
√
(4.14)
A
−1
◦B
−1
−1
− (A ◦ B)
≤
√
δ1 η1 − δn ηn
{I} .
δ1 η1 δn ηn
Corollary 4.4. Let A and B be a n × n positive definite Hermitian matrices. Let m1 and M1
be, respectively, the smallest and the largest eigenvalues of A ⊗ I and m2 and M2 , respectively,
the smallest and the largest eigenvalues of I ⊗ B. Then
(a) For p a nonzero integer, we have
(4.15)
Ap • B p ≤ max {λ1 , λ2 } (A • B)p ,
where λ1 and λ2 are given by Eq (3.31) and Eq (3.32).
While, for 0 < p < 1, we have the reverse inequality in Eq (4.15).
(b) For p a nonzero integer, we have
(4.16)
(Ap • B p ) − (A • B)p ≤ max {α1 , α2 } I,
where α1 and α2 are given by Eq (3.34) and Eq (3.35).
While, for 0 < p < 1, we have the reverse inequality in Eq (4.16).
Note that, the eigenvalues of A ⊗ I equal the eigenvalues of A and the eigenvalues of I ⊗ B
equal the eigenvalues of B.
Corollary 4.5. Let A and B be n × n positive definite Hermitian matrices. Let m and M be,
respectively, the smallest and the largest eigenvalues of A ⊕ B. Then
(a) For p a nonzero integer, we have
Ap • B p ≤ λ(A • B)p ,
(4.17)
where, λ is given by Eq (3.21).
While, for 0 < p < 1, we have the reverse inequality in Eq (4.17).
(b) For p a nonzero integer, we have
(4.18)
(Ap • B p ) − (A • B)p ≤ αI,
where, α is given by Eq (3.23).
While, for 0 < p < 1, we have the reverse inequality in Eq (4.18).
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
14
Z EYAD A L Z HOUR AND A DEM K ILICMAN
Special cases include from Eq (4.17):
(2.1) For p = 2, we have
A2 • B 2 ≤
(4.19)
(M + m)2
{A • B}2
4M m
(2.2) For p = −1, we have
(M + m)2
{A • B}−1
4M m
Similarly, special cases include from Eq (4.18):
(2.1) For p = 2, we have
A−1 • B −1 ≤
(4.20)
1
(A2 • B 2 ) − (A • B)2 ≤ (M − m)2 I
4
(4.21)
(2.2) For p = −1, we have
√
(A
(4.22)
−1
−1
−1
• B ) − (A • B)
≤
√
M− m
{I} .
Mm
We note that the eigenvalues of A⊕B are the n2 sums of the eigenvalues of A by the eigenvalues
of B. Thus if the eigenvalues of A and B are, respectively, ordered by:
δ1 ≥ δ2 ≥ · · · ≥ δn > 0,
η1 ≥ η2 ≥ · · · ≥ ηn > 0,
then in all previous results of this section M = δ1 + η1 and m = δn + ηn . Thus Eq(4.19) to Eq
(4.22) become:
(4.23)
A2 • B 2 ≤
(4.24)
A−1 • B −1 ≤
(δ1 + η1 + δn + ηn )2
{A • B}2 ,
4(δ1 + η1 )(δn + ηn )
(δ1 + η1 + δn + ηn )2
{A • B}−1 ,
4(δ1 + η1 )(δn + ηn )
1
(A2 • B 2 ) − (A • B)2 ≤ ((δ1 + η1 ) − (δn + ηn ))2 I,
4
(4.25)
√
(A
(4.26)
−1
−1
−1
• B ) − (A • B)
≤
√
δ1 + η1 − δn + ηn
I.
(δ1 + η1 )(δn + ηn )
Corollary 4.6. Let A ≥ 0 and B ≥ 0 be compatibly matrices. Then
(4.27)
(i)
(4.28)
(ii)
(A ⊕ B)(A ⊕ B)∗ ≥ AA∗ ⊕ BB ∗
(A ⊕ B)w ≥ Aw ⊕ B w ,
for any positive integer w.
Corollary 4.7. Let A and B be matrices of order m × m and n × n respectively. Then
(4.29)
(a)
(4.30)
(b)
tr(A ⊕ B) = n · tr(A) + m · tr(B)
√
√
kA ⊕ Bkp ≤ p n kAkp + p m kBkp ,
where
(4.31)
kAkp = [tr |A|p ]1/p , 1 ≤ p < ∞.
(c) eA⊕B = eA ⊗ eB
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
15
Corollary 4.8. Let A and B be non singular matrices of order m × m and n × n, respectively.
Then
(4.32)
(i) (A ⊕ B)−1 = (A−1 ⊕ B −1 )−1 (A−1 ⊗ B −1 )
(4.33) (ii) (A ⊕ B)−1 = (A−1 ⊗ In )(A−1 ⊕ B −1 )−1 (Im ⊗ B −1 )
(4.34) (iii)
(A ⊕ B)−1 = (Im ⊗ B −1 )(A−1 ⊕ B −1 )−1 (A−1 ⊗ In )
In [1], Ando proved the following inequality;
(4.35)
1
1
A ◦ B ≤ (Ap ◦ I) p (B q ◦ I) q ,
where A and B are positive definite matrices and p, q ≥ 1with 1/p + 1/q = 1.
If k·k is a unitarily invariant norm and k·k∞ is the spectral norm, Horn and Johnson in [3]
proved the following three conditions are equivalent:
(4.36)
(i) kAk∞ ≤ kAk
(ii) kABk ≤ kAk · kBk
(iii) kA ◦ Bk ≤ kAk · kBk
for all matrices A and B.
In [2], Hiai and Zhan proved the following inequalities:
(4.37)
kABk
kA + Bk
≤
kAk · kBk
kAk + kBk
and
kA ◦ Bk
kA + Bk
≤
kAk · kBk
kAk + kBk
for any invariant norm with kdiag(1, 0, . . ., 0)k ≥ 1 and A, B are nonzero positive definite
matrices.
We have an application to generalize the inequalities in Eq (4.37) involving the Hadamard
product (sum) and the Kronecker product (sum).
Theorem 4.9. Let k·k be a unitarily invariant norm with kdiag(1, 0, . . . , 0)k ≥ 1 and A and B
be nonzero positive definite matrices. Then
(4.38)
kA ◦ Bk
kA • Bk
≤
.
kAk · kBk
kAk + kBk
Proof. Let k·k∞ be the spectral norm and applying Eq (4.35) to A/ kAk∞ ≤ I, B/ kBk∞ ≤ I
and using the Young inequality for scalars, we get
p p1 q 1q
A
B
A
B
◦
≤
◦I
◦I
kAk∞
kBk∞
kAk∞
kBk∞
p
q
1
A
1
B
≤
◦I +
◦I
p kAk∞
q kBk∞
1
A
1
B
≤
◦I +
◦I
p kAk∞
q kBk∞
1
A
1
B
=
+
◦I
p kAk∞
q kBk∞
We choose
kAk∞
1
=
p
[kAk∞ + kBk∞ ]
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
and
kBk∞
1
=
.
q
[kAk∞ + kBk∞ ]
http://jipam.vu.edu.au/
16
Z EYAD A L Z HOUR AND A DEM K ILICMAN
Since kAk∞ ≤ kAk and kBk∞ ≤ kBk thanks tokdiag(1, 0, . . . , 0)k ≥ 1, we obtain
kAk∞ · kBk∞
(4.39)
A◦B ≤
(A + B) ◦ I
kAk∞ + kBk∞
kAk · kBk
≤
(A • B)
kAk + kBk
Hence,
kA ◦ Bk ≤
kAk · kBk
kA • Bk
kAk + kBk
or
kA ◦ Bk
kA • Bk
≤
kAk · kBk
kAk + kBk
Corollary 4.10. Let k·k be a unitarily invariant norm with kdiag(1, 0, . . . , 0)k ≥ 1 and A and
B be nonzero positive definite matrices. Then
(4.40)
kA ⊗ Bk
kA ⊕ Bk
≤
.
kAk · kBk
kAk + kBk
Proof. Applying Eq (2.7) and Eq (4.39), we have
K 0 (A ⊗ B)K ≤
kAk · kBk 0
K (A ⊕ B)K
kAk + kBk
and
kAk · kBk
kK 0 (A ⊕ B)Kk .
kAk + kBk
Provided that k·k is unitarily invariant norm, we get the result.
kK 0 (A ⊗ B)Kk ≤
R EFERENCES
[1] T. ANDO, Concavity of certain maps on positive definite matrices, and applications to Hadamard
products, Lin. Alg. and its Appl., 26 (1979), 203–241
[2] F. HIAI AND X. ZHAN, Submultiplicativity vs subadditivity for unitarily invariant norms, Lin. Alg.
and its Appl., 377 (2004), 155–164.
[3] R.A. HORN AND C.R. JOHNSON, Hadamard and conventional submultiplicativity for unitarily
invariant norms on matrices, Lin. Multilinear Alg., 20 (1987), 91–106.
[4] C.G. KHATRI AND C.R. RAO, Solutions to some functional equations and their applications to
characterization of probability distributions, Sankhya, 30 (1968), 51–69.
[5] R.H. KONING, H. NEUDECKER AND T. WANSBEEK, Block Kronecker products and the vecb
operator, Lin. Alg. and its Appl., 149 (1991), 165–184.
[6] S. LIU, Contributions to matrix calculus and applications in econometrics, Tinbergen Institute Research Series, 106, Thesis publishers, Amsterdam, The North land, (1995).
[7] S. LIU, Matrix results on the Khatri-Rao and Tracy-Singh products, Lin. Alg. and its Appl., 289
(1999), 267–277.
[8] S. LIU, Several inequalities involving Khatri-Rao products of positive semi definite matrices, Lin.
Alg. and its Appl., 354 (2002), 175–186.
[9] S. LIU, Inequalities involving Hadamard products of positive semi definite matrices, 243 (2002),
458–463.
[10] B. MOND AND J.E. PEČARIĆ, Matrix inequalities for convex functions, J. of Math. Anal. and
Appl., 209 (1997), 147–153.
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/
M ATRIX E QUALITIES AND I NEQUALITIES I NVOLVING K HATRI -R AO AND T RACY-S INGH S UMS
17
[11] B. MOND AND J.E. PEČARIĆ, Inequalities for the Hadamard product of matrices, SIAM J. Matrix
Anal. and Appl., 19 (1998), 66–70.
[12] D.S. TRACY AND R.P. SINGH, A new matrix product and its applications in matrix differentiation,
Statist. Neerlandica, 26 (1972), 143–157.
[13] C.R. RAO AND J. KLEFFE, Estimation of Variance Components and Applications, North-Holland,
Amsterdam, The Netherlands, (1988).
[14] C.R. RAO AND M.B. RAO, Matrix Algebra and its Applications to Statistics and Econometrics,
World Scientific, Singapore, (1998).
[15] G. VISICK, A quantitative version of the observation that the Hadamard product is a principle
submatrix of the Kronecker product, Lin. Alg. and its Appl., 304 (2000), 45–68.
[16] SONG-GUI WANG AND WAI-CHEUNG IP, A matrix version of the Wielandt inequality and its
applications to statistics, Lin. Alg. and its Appl., 296 (1999), 171–181.
J. Inequal. Pure and Appl. Math., 7(1) Art. 34, 2006
http://jipam.vu.edu.au/