18.S096: Homework Problem Set 3
Topics in Mathematics of Data Science (Fall 2015)
Afonso S. Bandeira
Due on November 3, 2015
Problem 3.1 Given n i.i.d. non-negative random variables x1 , . . . , xn , show that
h
i 1
log n
n
.
E max xi . E xlog
1
i
Problem 3.2 Let y1 , . . . , yn ∈ Rd be i.i.d. random variables such that Eyk = 0 and
Eyk ykT = Id×d .
Show that, if
then
E
1
log d log n
Eky1 k2 log n
. 1,
n
r
!
n
1
log d 1X
T
2 log n 2 log n
yk yk − Id×d .
Eky1 k
n
n
k=1
Note that k · k denotes spectral norm, and . means smaller up to constants.
Problem 3.3 Given a centered1 random symmetric matrix X ∈ Rd×d , we define
p
σ = kEX 2 k,
and
q
σ∗ = max
v: kvk=1
E (v T Xv)2 ,
1. Show that σ ≥ σ∗ .
2. If X has independent entries (except for the fact that Xij = Xji ) such that Xij ∼ N 0, b2ij ,
show that
1
Meaning that EX = 0.
1
2
• σ = max
i
n
X
b2ij
j=1
• σ∗ ≤ 2 max |bij |
ij
Note that k · k denotes spectral norm, and in expressions with E and ah power, the
power binds first.
2
2
2 i
2
T
T
For example, by EX , we mean E X and, by E v Xv , we mean E v Xv
Problem 3.4 (Norm concentration of projection) Let y1 , . . . , yd be i.i.d standard Gaussian rand
k
dom variables
and
Y = (y1 , . . . , yd ). Let g : R → R be the projection into the first k coordinates
and Z = g kYY k = kY1 k (y1 , . . . , yk ) and L = kZk2 . It is clear that EL = kd . Prove that L is very
concentrated around its mean
• If β < 1,
k
k
≤ exp
Pr L ≤ β
(1 − β + log β) .
d
2
• If β > 1,
k
k
Pr L ≥ β
≤ exp
(1 − β + log β) .
d
2
1. Prove
#
d
X
k
(1 − 2tkβ)−(d−k)/2
Pr
yi2 ≤ β
yi2 ≤
d
(1 − 2t(kβ − d))k/2
i=1
i=1
" k
X
for any t > 0 such that 1 − 2tkβ > 0 and 1 − 2t(kβ − d) > 0.
2
(Hint: Prove, and use, that E(esX ) =
√ 1
,
1−2s
for −∞ < s < 1/2, where X is standard normal.)
2. Find a suitable t and conclude the proof of the inequality for β < 1.
3. Use the same idea as above to prove the β > 1 case.
2
MIT OpenCourseWare
http://ocw.mit.edu
18.S096 Topics in Mathematics of Data Science
Fall 2015
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.