LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
1nÙ £8ëê
O
Tianxiao Pang
Zhejiang University
September 28, 2016
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
4
£8 ä
¦ O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
4
£8 ä
5
Box-CoxC†
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
4
£8 ä
5
Box-CoxC†
6
2•
¦
O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
4
£8 ä
5
Box-CoxC†
6
2•
7
õ- ‚5
¦
O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
4
£8 ä
5
Box-CoxC†
6
2•
7
õ- ‚5
8
* O
¦
O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
4
£8 ä
5
Box-CoxC†
6
2•
7
õ- ‚5
8
* O
9
̤© O
¦
O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
SN
1
•
¦ O
2
•
¦ O 5Ÿ
3
å•
¦ O
4
£8 ä
5
Box-CoxC†
6
2•
7
õ- ‚5
8
* O
9
̤© O
10
SteinØ
¦
O
O
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
O£8ëê •Ä •{´•
¦{(Least Squares
Method). Ùcn!?ØXÛA^•
¦{¦£8ëê •
¦ O, ¿ïÄù« O Ä 5Ÿ. 31oÚ1Ê!, ·‚
?Ø£8 . Ä b
·^5±9 ù b Ø·^ž, é
êâAT‰ C†(Box-CoxC†). 318!·‚ò?Ø2•
¦ O. 31Ô!·‚ò?ؘ«AÏ gCþ, ¿?اX
Û‰•
¦ O‘5ˆ³. 1l!Ú1Ê!·‚ò?Øü«#
O•{: * OṲ́© O. • ˜!{ü0 SteinØ
O.
lùpm©, ·‚b½gCþØ´‘ÅCþ, Ï•§
Š
Œ± <•››. Óž·‚ ½: ˜‘ gCþ^
i1L«,
˜‘ ÏCþ´‘ÅCþ, •^
i1L«(kž•L«
*
ÿŠ).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
•
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
¦ O
^yL«ÏCþ, x1 , · · · , xp L«(ŒU)éykK•
§‚ƒm÷vXe ‚5'Xª:
p‡gCþ. b
y = β0 + β1 x1 + · · · + βp xp + e,
(3.1.1)
Ù¥e´‘ÅØ , β0 , · · · , βp L«–
™•ëê. ¡β0 •£8
~ê, ¡β1 , · · · , βp •£8Xê. kžr§‚Ú¡•£8Xê. b
½®²k
(xi1 , · · · , xip , yi ), i = 1, · · · , n,
K§‚÷v
yi = β0 + β1 xi1 + · · · + βp xip + ei , i = 1, · · · , n.
b Ø ‘ei , i = 1, · · · , n÷vGauss-Markovb
Tianxiao Pang
1nÙ
£8ëê
.
O
(3.1.2)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
e^Ý L«, KŒ •
1 x11
y1
y2 1 x21
Y = . = .
..
.. ..
.
yn
½
···
···
..
.
1 xn1 · · ·
O Stein
e1
β0
x1p
β1 e2
x2p
.. .. + .. ,
. . .
xnp
en
βp
•
Y = Xβ + e.
(3.1.3)
Ù¥Y ´n × 1 ‘Å*ÿ•þ(ÏCþ•þ), X´n × (p + 1)
®• OÝ (b p + 1 ≤ n), β´(p + 1) × 1 ™•ëê•þ, e
´n × 1 ‘ÅØ •þ.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
òGauss-Markovb
O Stein
• ¤Ý /ª:
E(e) = 0, Cov(e) = σ 2 In .
ò(3.1.3)Ú(3.1.4)Ü 3˜å, =Œ
(3.1.4)
•Ä
‚5£8 .:
Y = Xβ + e, E(e) = 0, Cov(e) = σ 2 In .
Tianxiao Pang
1nÙ
£8ëê
O
(3.1.5)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
·‚^•
¦•{Ïéβ
O, ù‡ OÏd ¡••
¦ O(LSE: Least Squares Estimator). ù‡•{´Ï阇β
O, ¦
•þe = Y − Xβ •Ýƒ²•kY − Xβk2 ˆ
• .
P
0
Q(β) = kY − Xβk2 = (Y − Xβ) (Y − Xβ)
0
0
0
0
= Y Y − 2Y Xβ + β X Xβ.
éβ¦
, -Ù u", Œ •§|
0
0
X Xβ = X Y .
(3.1.6)
¡ù‡•§|• 5•§|(½ K•§|). ù‡•§|k•˜
0
) ¿‡^‡´X X •´p + 1, ù duX •´p + 1(=X
´ ÷• ). Ï•X ´Œ±<•›› , ¤±·‚ob½X´
÷• . u´·‚
(3.1.6) •˜)
0
0
β̂ = (X X)−1 X Y .
Tianxiao Pang
1nÙ
£8ëê
(3.1.7)
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
±þ ?Ø•U`²β̂´Q(β) ˜‡7:,
eyβ̂(¢´Q(β) • Š:.
é?¿
™7Ò´• Š:.
β, k
kY − Xβk2 = kY − X β̂ + X(β̂ − β)k2
0
0
= kY − X β̂k2 + (β̂ − β) X X(β̂ − β)
0
0
+2(β̂ − β) X (Y − X β̂).
(3.1.8)
0
Ï•β̂÷v 5•§|(3.1.6), ¤±X (Y − X β̂) = 0. ùy²
é?¿ β, k
0
0
kY − Xβk2 = kY − X β̂k2 + (β̂ − β) X X(β̂ − β). (3.1.9)
0
0
0
duX X´šK½Ý , ¤±(β̂ − β) X X(β̂ − β) ≥ 0. u´
kY − Xβk2 ≥ kY − X β̂k2 .
y.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
Pβ̂ = (β̂0 , β̂1 , · · · , β̂p ) , ·‚Œ±
O Stein
(² )£8•§:
Ŷ = X β̂ ½ ŷ = β̂0 + β̂1 x1 + · · · + β̂p xp .
ù‡•§´Ø´£ã y†x1 , · · · , xp
ÚO©Û, 3–± ?n.
Tianxiao Pang
1nÙ
(3.1.10)
ý¢'X, „IŠ?˜Ú
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.1.1 ˜ ‚5£8. b gCþ•k˜‡, P•x.
•(xi , yi ), i = 1, · · · , n. u´k‚5£8 .
yi = β0 + β1 xi + ei , i = 1, · · · , n.
ùž
5•§|•
n
n
X
x
i=1
n
X
i
n
X
i=1
n
X
n
X
yi
xi
β0
i=1
,
n
β1 = X
xy
x2
i i
i
i=1
i=1
OÝ X´ ÷•ž, =xi , i = 1, · · · , nØ ƒ ž,
(xi − x̄)2 6= 0, u´β0 Úβ1
LSE•
i=1
β̂0 = ȳP− β̂1 x̄,
n
(x − x̄)(yi − ȳ)
Sxy
Pn i
β̂1 = i=1
=:
2
Sxx
i=1 (xi − x̄)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
3£8©Û¥, ·‚kžr ©êâ?1¥%zÚIOz. n
1X
xij , j = 1, · · · , p.
x̄j =
n
i=1
ò(3.1.2)U •
yi = α + β1 (xi1 − x̄1 ) + · · · + βp (xip − x̄p ) + ei ,
i = 1, · · · , n. (3.1.11)
ùp, α = β0 + β1 x̄1 + · · · + βp x̄p . ¡(3.1.11)•¥%z .. P
x11 − x̄1 x12 − x̄2 · · · x1p − x̄p
x21 − x̄1 x22 − x̄2 · · · x2p − x̄p
(3.1.12)
Xc =
.
..
..
..
..
.
.
.
.
xn1 − x̄1 xn2 − x̄2 · · ·
xnp − x̄p
K(3.1.11)ŒU •
α
Y = α1n + Xc β + e = 1n Xc
+ e.
β
Tianxiao Pang
1nÙ
£8ëê
O
(3.1.13)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
ùp, β = (β1 , · · · , βp ) . 5¿
0
1n Xc = 0
(3.1.14)
Ïd 5•§|Œ •
0 n
0
α
1n Y
=
,
0
0
β
0 Xc Xc
Xc Y
(3.1.15)
d/ ¤
0
nα = 1n Y ,
0
0
Xc Xc β = Xc Y
(3.1.16)
u´£8ëê LSE•
α̂ = ȳ,
0
0
β̂ = (Xc Xc )−1 Xc Y .
Tianxiao Pang
1nÙ
£8ëê
(3.1.17)
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ïd, éu¥%z‚5£8 .(3.1.11), £8~ê LSEo´Ï
0
0
Cþ
þŠ, £8Xêβ LSE β̂ = (Xc Xc )−1 Xc Y du
0
l‚5£8 .Y = Xc β + e¥OŽβ LSE. 3¢SA^¥, O
0
Ž(Xc Xc )−1 o´'OŽ(X 0 X)−1 ‡•B˜:, …·‚o´AO
'%£8Xê, ¤±¥%z´kÐ? .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.1.2 ˜ ‚5£8(Y). ò~3.1.1¥ ˜ ‚5£8 .?1
¥%z,
yi = α + β1 (xi − x̄) + ei , i = 1, · · · , n.
dúª(3.1.17) LSE
α̂ = ȳ,P
Pn
n
(x − x̄)(yi − ȳ)
(xi − x̄)yi
i=1
Pn i
= i=1
.
β̂1 = Pn
2
2
i=1 (xi − x̄)
i=1 (xi − x̄)
Tianxiao Pang
1nÙ
£8ëê
O
(3.1.18)
(3.1.19)
LSE LSE
,
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
, ·‚„Œ±égCþ‰IOz?n. P
s2j =
n
X
(xij − x̄j )2 , j = 1, · · · , p,
i=1
zij =
xij − x̄j
, i = 1, · · · , n; j = 1, · · · , p.
sj
-Z = (zij )n×p , §äk5Ÿ:
0
1n Z = 0,
0
R = Z Z = (rij ),
Ù¥
Pn
rij =
k=1 (xki − x̄i )(xkj − x̄j )
•gCþxi †xj
.
si sj
, i, j = 1, · · · , p
(3.1.20)
ƒ'Xê. ¤±R´gCþ ƒ'XêÝ
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
égCþ?1IOz Ð?: (1)Œ±^5©Û£8gCþƒm
ƒ''X; (2)IOzžØ þj K•, Bué£8Xê OŠ
ÚO©Û.
²LIOz
‚5£8 .•
xip − x̄p
xi1 − x̄1
β1 + · · · +
βp + ei ,
yi = α +
s1
sp
½
(3.1.21)
¤Ý /ª
α
Y = α1n + Zβ + e = 1n Z
+ e.
β
™•ëê LSE•
0
0
0
α̂ = ȳ, β̂ = (β̂1 , · · · , β̂p ) = (Z Z)−1 Z Y .
£8•§•
ŷ = α̂ +
xp − x̄p
x1 − x̄1
β̂1 + · · · +
β̂p .
s1
sp
Tianxiao Pang
1nÙ
£8ëê
O
(3.1.22)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.1.3 ˜‡Á Nì‚ ðøA9þ, ¦Ù ±ð§, eL¥,
gCþxL«N챌˜íü žm ²þ§Ý(◦C), yL«ü
žmSžÑ
ðþ(L), *ÿ 25 ‡žmü . ã3.1.1´ù
êâ Ñ:ã, éù|êâ, A^¥%z‚5£8 .(3.1.18),
·‚
ȳ = 9.424, x̄ = 52.6,
£8ëê LSE•
α̂ = ȳ = 9.424, β̂1 = −0.0798.
¤±£8•§•
ŷ = 9.424 − 0.0798(x − 52.6),
½ ¤
ŷ = 13.623 − 0.0798x.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
L3.1.1:
SÒ
1
2
3
4
5
6
7
8
9
10
11
12
13
O Stein
ðêâ
x(◦C)
35.3
29.7
30.8
58.8
61.4
71.3
74.4
76.7
70.7
57.5
46.4
28.9
28.1
SÒ
14
15
16
17
18
19
20
21
22
23
24
25
y(L)
9.57
10.94
9.58
10.09
8.11
6.83
8.88
7.68
8.47
8.86
10.36
11.08
Tianxiao Pang
1nÙ
£8ëê
y(L)
10.98
11.13
12.51
8.40
9.27
8.73
6.36
8.5
7.82
9.14
8.24
12.19
11.88
x(◦C)
39.1
46.8
48.5
59.3
70
70
74.5
72.1
58.1
44.6
33.4
28.6
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
ã3.1.1 Ñ:ã
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
R§S:
yx=read.table(”ex p33 data.txt”)
y=yx[, 1]
x=yx[, 2]
mydata=data.frame(y,x)
plot(y∼x)
lm.sol=lm(y∼x,data=mydata)
abline(lm.sol)
summary(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
†
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
£8•§:
ŷ = 13.623 − 0.0798x.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
•
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
¦ O
•
O Stein
5Ÿ
¦ O(LSE)äk˜ ûÐ 5Ÿ:
½n (3.2.1)
0
0
éu‚5£8 .(3.1.5), LSE β̂ = (X X)−1 X Y äke 5Ÿ:
(a) E(β̂) = β;
0
(b) Cov(β̂) = σ 2 (X X)−1 .
y²: (a) ´•E(Y ) = Xβ, ¤±
0
0
E(β̂) = (X X)−1 X · E(Y ) = β.
(b) Ï•Cov(Y ) = Cov(e) = σ 2 In , ¤±
0
0
Cov(β̂) = Cov((X X)−1 X Y )
0
0
0
0
0
= (X X)−1 X Cov(Y )X(X X)−1
0
0
= (X X)−1 X σ 2 In X(X X)−1 = σ 2 (X X)−1 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
c´p + 1‘ ~ê•þ, éu‚5¼êc β(ù´˜‡™•ëê),
0
0
·‚¡c β̂•c β LSE.
íØ (3.2.1)
0
0
(a) E(c β̂) = c β;
0
0
0
(b) Cov(c β̂) = σ 2 c (X X)−1 c.
0
0
0
=é?¿ ‚5¼êc β, c β̂•c β Ã
O, • •
0
0
0
0
0
0
σ 2 c (X X)−1 c. Ï•c β̂ = c (X X)−1 X Y •y1 , · · · , yn ‚5
0
0
¼ê, ¤±c β̂•c β ˜‡‚5Ã
O(‚5 O• ´*
0
ÿy1 , · · · , yn ‚5¼ê). ·‚„Œ± EÑc β Ù§‚5Ã
0
O. ù ¤ c β ‚5Ã
Oa.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n (3.2.2, Gauss-Markov)
0
éu‚5£8 .(3.1.5), 3c β ¤k‚5Ã
O¥, •
0
¦ Oc β̂´•˜ • • ‚5Ã
O(BLUE: best linear
unbiased estimator).
y²:
þβ,
0
0
a Y •c β
˜‡‚5Ã
0
O. u´é˜ƒp + 1‘ •
0
0
c β = E(a Y ) = a Xβ,
Ïd
0
0
aX=c.
0
(3.2.1)
0
Ï•Var(a Y ) = σ 2 a a = σ 2 kak2 , ·‚ékak2 ‰©):
0
0
kak2 = ka − X(X X)−1 c + X(X X)−1 ck2
0
0
= ka − X(X X)−1 ck2 + kX(X X)−1 ck2
0
0
0
0
+2c (X X)−1 X (a − X(X X)−1 c). (3.2.2)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
d(3.2.1)Œí•
0
0
0
0
0
0
0
0
2c (X X)−1 X (a−X(X X)−1 c) = 2c (X X)−1 (X a−c ) = 0.
díØ3.2.1(b)Œí•
0
σ 2 kX(X X)−1 ck2
0
0
0
0
0
0
= σ 2 c (X X)−1 X X(X X)−1 c
= σ 2 c (X X)−1 c
0
= Var(c β̂).
u´, ·‚y²
0
0
éc β
?˜‡‚5Ã
0
Oa Y k
0
0
Var(a Y ) = Var(c β̂) + σ 2 ka − X(X X)−1 ck2
0
≥ Var(c β̂),
(3.2.3)
0
0
Ò¤á …= ka − X(X X)−1 ck = 0, =a = X(X X)−1 c,
0
0
dža Y = c β̂. ½n y.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
3‚5£8 .(3.1.5)¥„k˜‡ëêσ 2 (Ø • ), §‡N
.Ø
Œ , 3£8©Û¥åX-‡ Š^. ·‚y35
2
Oσ .
e = Y − Xβ´Ø •þ, ØŒ*ÿ. ^β̂“Oβ, ¡
ê = Y − X β̂ = Y − Ŷ
0
•í (residual)•þ. Pxi • OÝ X
(3.2.4)
1i1, K
0
êi = yi − xi β̂, i = 1, · · · , n
(3.2.5)
•1ig*ÿ í . g,/, ·‚Œ±òêwŠe ˜‡ O. ·
‚ò^
0
RSS = ê ê =
n
X
ê2i
(3.2.6)
i=1
5 Eσ 2
Ã
Oþ.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
RSS=Residual Sum of Squares, L«í ²•Ú. §‡N ¢Sê
â†nØ .(3.1.5)
l§Ý½ö`[ܧÝ. RSS
L«
ê↠.[Ü
Ð.
½n (3.2.3)
0
0
0
0
(a) RSS = Y (In − X(X X)−1 X )Y =: Y (In − H)Y ;
(b) σ̂ 2 = RSS/(n − p − 1)´σ 2 Ã
Oþ.
0
0
5: ¡H = X(X X)−1 X •lf(hat)Ý , §´˜‡é¡˜
Ý .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
y²: (a)
0
0
RSS = ê ê = (Y − X β̂) (Y − X β̂)
0
0
0
0
0
= [(In − X(X X)−1 X )Y ] [(In − X(X X)−1 X )Y ]
0
0
0
= Y (In − X(X X)−1 X )Y .
(b) dE(Y ) = Xβ, Cov(Y ) = σ 2 In ±9½n2.2.1•
0
0
0
E(RSS) = E[Y (In − X(X X)−1 X )Y ]
0
0
0
0
= β X (In − X(X X)−1 X )Xβ
0
0
+σ 2 tr(In − X(X X)−1 X )
0
0
= σ 2 [n − tr(X(X X)−1 X )].
Šâ,
5Ÿtr(AB) = tr(BA)•
0
0
0
0
tr(X(X X)−1 X ) = tr((X X)−1 X X) = tr(Ip+1 ) = p + 1.
u´E(RSS) = σ 2 (n − p − 1).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
XJb Ø •þeÑl
©Ù, =e ∼ N (0, σ 2 In ), @o·‚
α
β̂Úσ̂ 2 •õ -‡5Ÿ.
½n (3.2.4)
éu‚5£8 .(3.1.5), eØ •þe ∼ N (0, σ 2 In ), K
0
(a) β̂ ∼ N (β, σ 2 (X X)−1 );
(b) RSS/σ 2 ∼ χ2 (n − p − 1);
(c) β̂†RSSƒpÕá.
0
0
y²: (a) 5¿ Y ∼ N (Xβ, σ 2 In )±9β̂ = (X X)−1 X Y ´Y
‚5C†, @od½n2.3.4BŒí (a).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
(b) Šâ½Â
0
0
0
RSS = Y (In − H)Y = Y (In − H)Y = Y N Y ,
ùpN = In − H, §´˜‡é¡˜ Ý . 5¿ N X = 0, ¤
±
0
0
RSS = (Xβ + e) N (Xβ + e) = e N e.
(3.2.7)
5¿ e ∼ N (0, σ 2 In ), ¤±Šâ½n2.4.3·‚•Iy²N •
•n − p − 1. |^˜ Ý
• u§ ,ù˜5Ÿ, ·‚Œ
0
0
rk(N ) = tr(In − X(X X)−1 X )
0
0
= n − tr(X(X X)−1 X )
0
0
= n − tr((X X)−1 X X)
= n − p − 1.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
0
O Stein
0
(c) Ï•β̂ = β + (X X)−1 X e,
RSS = e N e, 5¿
0
0
(X X)−1 X N = 0,
0
0
¤±d½n2.4.5Œ•(X X)−1 X e†RSSƒpÕá, =β̂†RSSƒ
pÕá.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
β 1˜‡©þ•~ê‘β0 ž, c = (0, · · · , 0, 1, 0, · · · , 0) , Ù
0
0
¥1 3c 1i + 1‡ ˜, Kc β = βi . 2Pβ̂ = (β̂0 , · · · , β̂p ) , u
0
´c β̂ = β̂i . 2^(A)ii L«Ý A 1(i, i) ƒ, @o·Xeí
Ø:
íØ (3.2.2)
éu‚5£8 .(3.1.5), ee ∼ N (0, σ 2 In ), K
0
(a) β̂i ∼ N (βi , σ 2 [(X X)−1 ]i+1,i+1 );
(b) 3βi , i = 1, · · · , p ˜ƒ‚5Ã
O¥, β̂i , i = 1, · · · , p´•
˜• • ö.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
ò½n3.2.1Ú½n3.2.4A^u¥%z .(3.1.13), Kk
íØ (3.2.3)
0
éu¥%z .(3.1.13), 5¿ùp β = (β1 , · · · , βp ) , k
0
0
(a) E(α̂) = α, E(β̂) = β, ùpα̂ = ȳ, β̂ = (Xc Xc )−1 Xc Y ;
(b)
1
α̂
0
2 n
Cov
=σ
;
0
0 (Xc Xc )−1
β̂
(c) e?˜Úb e ∼ N (0, σ 2 In ), K
α̂ ∼ N (α,
σ2
0
), β̂ ∼ N (β, σ 2 (Xc Xc )−1 ),
n
…α̂†β̂ƒpÕá.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½Â
R2 =
Ù¥
ESS =
n
X
ESS
,
TSS
(3.2.8)
0
(ȳ − ŷi )2 = (Ŷ − ȳ1n ) (Ŷ − ȳ1n )
i=1
¡•£8²•Ú(½)º²•Ú: Explained Sum of Squares),
TSS =
n
X
0
(yi − ȳ)2 = (Y − ȳ1n ) (Y − ȳ1n )
i=1
¡•o
²•Ú(½¡•o²•Ú: √
Total Sum of Squares).
¡R2 • ½Xê½ÿ½Xê, ¡R = R2 •Eƒ'Xê.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
d
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5•§|Œ±y²
n
X
(yi − ȳ)2
i=1
n
X
=
(yi − ŷi + ŷi − ȳ)2
i=1
n
n
n
X
X
X
=
(yi − ŷi )2 +
(ŷi − ȳ)2 + 2
(yi − ŷi )(ŷi − ȳ)
=
i=1
i=1
n
X
n
X
i=1
(yi − ŷi )2 +
i=1
(ŷi − ȳ)2 .
i=1
=
TSS = ESS + RSS.
ÏdR2 Ýþ £8gCþx1 , · · · , xp éÏCþy [Ü§Ý Ð
€. 0 ≤ R2 ≤ 1, § Š Œ, L²y†ÃgCþk Œ ƒ•'
X.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
e¦^¥%z .(3.1.13), @oESSŒÏLe úªOŽ:
0
0
0
0
0
ESS = β̂ Xc Y = Y Xc (Xc Xc )−1 Xc Y ,
0
ùpβ̂ = (β̂1 , · · · , β̂p ) . ¯¢þ, dŶ = α̂1n + Xc β̂9úª
(3.1.17)Υ
Ŷ − ȳ1n = Ŷ − α̂1n = Xc β̂.
¤±
0
0
ESS = (Ŷ − ȳ1n ) (Ŷ − ȳ1n ) = (Xc β̂) Xc β̂
0
0
0
0
0
0
= β̂ Xc · Xc (Xc Xc )−1 Xc Y
= β̂ Xc Y .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
éu˜ ‚5£8 .
yi = β0 + β1 xi + ei , i = 1, · · · , n.
Œ±y²R2 = r2 , Ù¥r• .¥gCþ†ÏCþ
ê.
¯¢þ, e·‚r .¥%z:
yi = α + β1 (xi − x̄) + ei , i = 1, · · · , n,
@oΥ
Pn
(xi − x̄)yi
β̂1 = Pi=1
,
n
2
i=1 (xi − x̄)
Tianxiao Pang
1nÙ
£8ëê
O
ƒ'X
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
l
ESS = β̂1
n
X
hP
(xi − x̄)yi =
i=1
i2
i2
hP
=
n
i=1 (xi − x̄)yi
Pn
2
i=1 (xi − x̄)
n
i=1 (xi − x̄)(yi − ȳ)
Pn
2
i=1 (xi − x̄)
.
¤±
i2
n
(x
−
x̄)(y
−
ȳ)
i
i
i=1
ESS
Pn
R2 =
= Pn
= r2 .
2
2
TSS
(x
−
x̄)
(y
−
ȳ)
i
i
i=1
i=1
hP
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.2.1 Šâ² •, 3<
pƒ
^‡e, ÙÉØ Â
Øy†N-x1 !c#x2 k'. yÂ8 13¶If ÿþêâ, „e
L. Áïáy'ux1 , x2 ‚5£8•§.
L3.2.1: ÉØêâ
SÒ
1
2
3
4
5
6
7
xi1
152
183
171
165
158
161
149
yi
120
141
124
126
117
125
123
SÒ
8
9
10
11
12
13
xi1
158
170
153
164
190
185
Tianxiao Pang
1nÙ
£8ëê
xi2
50
20
20
30
30
50
60
xi2
50
40
55
40
40
20
O
yi
125
132
123
132
155
147
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
|^¥%z .
yi = α + β1 (xi1 − x̄1 ) + β2 (xi2 − x̄2 ) + ei , i = 1, · · · , 13.
²OŽŒ
x̄1 =
13
13
13
1 X
1 X
1 X
xi1 = 166.8, x̄2 =
xi2 = 38.85, ȳ =
yi = 130,
13 i=1
13 i=1
13 i=1
−14.08
16.92
4.92
−1.08
−8.08
−5.08
Xc = −17.08
−8.08
3.92
−13.08
−2.08
23.92
18.92
Tianxiao Pang
11.15
−18.85
−18.85
−8.85
−8.85
11.15
21.15 ,
11.15
1.15
16.15
1.15
1.15
−18.85
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
O Stein
0
5•§|Xc Xc β = Xc Y •
2078.92β1 − 1533.85β2 = 1607,
−1533.85β1 + 2307.69β2 = −715.
)
β̂1 = 1.068, β̂2 = 0.4,
α̂ = ȳ = 130. ¤±£8•§•
ŷ = α̂ + β̂1 (x1 − x̄1 ) + β̂2 (x2 − x̄2 )
= 130 + 1.068 × (x1 − 166.8) + 0.4 × (x2 − 38.85)
= −62.963 + 1.068x1 + 0.4x2 .
d
, „ŒŽ
0
0
ESS = β̂ Xc Y = 1430.276, TSS = 1512,
¤±R2 = 1430.276/1512 = 0.9459.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
R§S:
yx=read.table(”ex p39 data.txt”)
x1=yx[, 1]
x2=yx[, 2]
y=yx[, 3]
mydata=data.frame(y,x1,x2)
lm.sol=lm(y∼x1+x2,data=mydata)
summary(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
dR$1(J : ŷ = −62.963 + 1.068x1 + 0.4x2 , R2 = 0.9461.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
å•
åLSE
ä BCC† GLSE õ- ‚5 * O PC
¦
O Stein
O
éβØ\?Û å^‡ œ/e, ·‚?Ø § LSE±9§ Ä
5Ÿ. 3˜ AÏ|Ü, ~Xb u ¯K, ·‚I‡¦‘k
˜½ å^‡ LSE.
b
Aβ = b
(3.3.1)
´˜‡ƒN‚5•§|(=•§|k)), Ù¥A´k × (p + 1)®•
Ý , ••k, b´k × 1®••þ. ·‚^Lagrange¦f{¦‚5£
8 .(3.1.5) 3÷v‚5 å(3.3.1)ž LSE. =3(3.3.1)ù‡^
‡e¦β¦Q(β) = kY − Xβk2 ˆ • .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
E9ϼê
0
F (β, λ) = kY − Xβk2 + 2λ (Aβ − b)
0
0
= (Y − Xβ) (Y − Xβ) + 2λ (Aβ − b),
0
Ù¥λ = (λ1 , · · · , λk ) •Lagrange¦f•þ. é¼êF (β, λ)'
uβ¦ ¿-§ u0,
0
0
0
− X Y + X Xβ + A λ = 0.
(3.3.2)
y3, ·‚I‡)d(3.3.1)Ú(3.3.2)|¤ éá•§|.
·‚^β̂c Úλ̂c L«ù‡•§| ). d(3.3.2)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
O Stein
0
0
0
β̂c = (X X)−1 X Y − (X X)−1 A λ̂c
0
0
= β̂ − (X X)−1 A λ̂c .
(3.3.3)
“\(3.3.1)
0
0
b = Aβ̂c = Aβ̂ − A(X X)−1 A λ̂c ,
Œ
d/ ¤
0
0
A(X X)−1 A λ̂c = Aβ̂ − b.
0
(3.3.4)
0
duA ••k, ¤±A(X X)−1 A ´k × kŒ_Ý , Ïd(3.3.4)
k•˜)
0
0
λ̂c = (A(X X)−1 A )−1 (Aβ̂ − b),
2ò§“\(3.3.3)
0
0
0
0
β̂c = β̂ − (X X)−1 A (A(X X)−1 A )−1 (Aβ̂ − b).
Tianxiao Pang
1nÙ
£8ëê
O
(3.3.5)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
eyβ̂c (¢´‚5 åAβ = beβ LSE. ù•Iy²
(a) Aβ̂c = b;
(b) 阃÷vAβ = b β, ÑkkY − Xβk2 ≥ kY − X β̂c k2 .
d(3.3.5)Œí (a). •y(b), òkY − Xβk2 ©)
kY − Xβk2
= kY − X β̂k2 + kX(β̂ − β)k2
= kY − X β̂k2 + kX(β̂ − β̂c + β̂c − β)k2
= kY − X β̂k2 + kX(β̂ − β̂c )k2 + kX(β̂c − β)k2(, 3.3.6)
ùp í ^
vAβ = b β,
0
0
0
eã'X: (Y − X β̂) X = 0±9阃÷
0
0
(β̂ − β̂c ) X X(β̂c − β) = λ̂c A(β̂c − β) = λ̂c (Aβ̂c − Aβ) = 0.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
(3.3.6)L²: 阃÷vAβ = b
O Stein
β, ok
kY − Xβk2 ≥ kY − X β̂k2 + kX(β̂ − β̂c )k2 ,
(3.3.7)
Ò¤á …= X(β̂c − β) = 0, =β = β̂c (Ï•X
d3(3.3.7)¥^β̂c “Oβ, Ò¤á. =
÷•). Ï
kY − X β̂c k2 = kY − X β̂k2 + kX(β̂ − β̂c )k2 .
(3.3.8)
y3, (Ü(3.3.7)Ú(3.3.8)BŒí (b).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
·‚rβ̂c ¡•β
å•
¦
O Stein
O, u´ke
½n:
½n (3.3.1)
éu‚5£8 .(3.1.5), ÷v(3.3.1)
0
0
0
åLSE•
0
β̂c = β̂ − (X X)−1 A (A(X X)−1 A )−1 (Aβ̂ − b),
0
0
Ù¥β̂ = (X X)−1 X Y ´Ã å^‡e LSE.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.3.1 3U©ÿþ¥, éU˜¥ n‡( : ¤ n /
ABC n‡S θ1 , θ2 , θ3 ?1ÿþ,
•y1 , y2 , y3 , du•3ÿ
þØ , ¤±I‡éθ1 , θ2 , θ3 ?1 O, ·‚|^‚5 .L«k
' þ
y1 = θ1 + e1 ,
y2 = θ2 + e2 ,
y = θ3 + e3 ,
3
θ1 + θ2 + θ3 = π.
¤Ý /ª
Y = Xβ + e,
Aβ = b,
0
0
ùpY = (y1 , y2 , y3 ) , β = (θ1 , θ2 , θ3 ) , X = I3 , A = (1, 1, 1),
0
0
b = π. 5¿ β̂ = (X X)−1 X Y = Y , A^½n3.3.1, ²OŽ
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
3
1
y1
X
1
yi − π) 1 ,
β̂c = y2 − (
3
i=1
1
y3
=
3
1 X
θ̂i = yi − (
yi − π), i = 1, 2, 3
3
i=1
•θi
åLSE.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
£8 ä
éu‚5£8 .(3.1.5):
Y = Xβ + e, E(e) = 0, Cov(e) = σ 2 In ,
·‚3?Ø™•ëê •
Xe •Ä
b :
¦ O±9 Oþ ©Ùž, Š
(a) ‚5b : ÏCþ†gCþäk‚5ƒ''X;
(b) • à5b : Var(ei ) = σ 2 , i = 1, · · · , n;
(c) ؃'5b : Cov(ei , ej ) = 0, i 6= j.
(d)
5b : ei ∼ N (0, σ 2 ), i = 1, · · · , n.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
XJù b ؤá, @o·‚±c?Ø •
¦ O±9§
ÚO5ŸÒkŒU´Ø¤á . Ïd, 3¢S¯K¥, ·‚k
˜1êâ , I‡• ·‚ êâ´Ä÷v½öÄ ÷vù
b . ù´ . ä SN.
Ï•ù b ц‘ÅØ ek', í •þêŒw¤´e ˜
‡ O, Ïd·‚Œ±ÏLí 5©Û4‡Ä b ´Ä¤á.
Ï•ù‡ Ï, ùÜ©SN• ¡•í ©Û.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ø I‡é .?1 ä, ·‚„I‡éêâ
•¹É~: äÚrK•: ä.
?1 ä, §
3£8©Û¥, ¤¢É~:(q l+:)´•éQ½ . lé
Œ êâ:. ‰§e˜‡O( ½Â´ƒ (J , –8„vk
Ú˜ ½Â. 8c, éÉ~: •61 w{´: (1)É~:´•
@ †ýŒõêêâ:²wØ N êâ:. ùžÉ~:Œn)
•¤b½©Ù¥ 4à:;(2)É~:Ò´@ À/:, =´•†
ýŒõêêâ:Ø´5gÓ˜©Ù ‡Oêâ:.
É~: ·\òéëê
~:¿ò§íØ.
OE¤K•, Ïd·‚I‡uÿÑÉ
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
êâ8¥ rK•:´•éÚOíä(ëê O!b u
) ŒK• êâ:, 3d‘§¥, A•é£8ëêβ •
Ok ŒK• êâ:.
)
¦
0
Ok˜½
éz˜|êâ:(xi , yi ), ·‚F"§é£8ëê
K•, qF"ù«K•ØU Œ. ù , ·‚
² £8•
§Òk˜½ -½5. ÄK, XJ‡O˜ü|êâé OkÉ~
Œ K•, ·‚GØù ê⃠, ÒU
† 5 ÉéŒ
² £8•§.
Ïd3£8©Û¥, ·‚I‡• z|êâéëê O K•Œ
. ùÜ©SN ¡•K•©Û.
uÿÑrK•: , ·‚I‡Ø ù êâ´Ä ~, eØ ~
KíØƒ, ÄK•ÄÂ8•õ êâ½æ^˜ -è O•{±
/DºrK•:é O K•.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
É~:ÚrK•:´ü‡ØÓ Vg. l ¡ ©ÛŒ±wÑ,
§‚ƒmQk˜½ éX•k˜½ «O.
rK•:ŒUÓž´É~:•ŒUØ´, ‡ƒ, É~:ŒUÓž
q´rK•:•ŒUØ´.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ïd, £8 䕹êâ
ä† .
äùüÜ©SN. Ù
¥êâ
ä•)É~: äÚrK•: ä, .
䕹
‚5 ä!• à5 ä!؃'5 äÚ
5 ä4 Ü©S
N.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
k5?Øí ©Û:
éu‚5£8 .:
Y = Xβ + e, E(e) = 0, Cov(e) = σ 2 In ,
(3.4.1)
·‚^
0
êi = yi − xi β̂, i = 1, · · · , n
(3.4.2)
L«1ig*ÿ í , ¿r§w¤´ei ˜‡ O. e
(3.4.1) (, @oêi Aäkei ˜ 5G.
.
0
PŶ = X β̂, ¡Ŷ •[ÜŠ•þ, ¡ŷi = xi β̂•1i‡[ÜŠ. ´•
0
0
Ŷ = X(X X)−1 X Y = HY ,
0
Ù¥H = X(X X)−1 X
0
(3.4.3)
¡•lfÝ , §´é¡˜
Tianxiao Pang
1nÙ
£8ëê
O
Ý .
LSE LSE
í
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
êŒ L«•
ê = Y − Ŷ = (I − H)Y = (I − H)e.
I − H•´˜‡é¡˜ Ý .
½n (3.4.1)
e(3.4.1)¤á, K
(a) E(ê) = 0, Cov(ê) = σ 2 (I − H);
(b) Cov(Ŷ , ê) = 0.
(c) e?˜Úb e ∼ N (0, σ 2 I), K
ê ∼ N (0, σ 2 (I − H)).
y²: N´, Ñ.
Tianxiao Pang
1nÙ
£8ëê
O
(3.4.4)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ï•Var(êi ) = σ 2 (1 − hii )(hii L«Ý H 1i‡é ‚ ƒ), š
à5, ùkNuêi ¢SA^. Ïd·‚•Ĥ¢ Æ)zí
êi
, i = 1, · · · , n,
ri = √
σ̂ 1 − hii
(3.4.5)
0
ùpσ̂ 2 = RSS/(n − p − 1) = ê ê/(n − p − 1).
=¦3e ∼ N (0, σ 2 I) ^‡e, ri ©ÙE,' E,, ŒC
q/@•ri ƒpÕá…ÑlN (0, 1)(„•FW(1987)). d½n
3.4.1Œ•{ri , i ≥ 1}†{ŷi , i ≥ 1}Õá({êi , i ≥ 1}†{ŷi , i ≥ 1}•
Õá).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
í ã´±,«í (Æ)zí ri ½ÊÏí êi )•p‹I, ±?
ÛÙ§ þ•î‹I Ñ:ã. c®•Ñí Š•Ø ei
O
AT†ei ƒ Ø , Šâí ã5G´Ä†Ak 5Ÿƒ˜—,
ÒŒ±é .b
Ün5Jø˜ k^ &E.
e¡·‚±[ÜŠŷi •î‹I, Æ)zí ri •p‹I í ã
•~?Øí ã äNA^. Š ˜J ´, Ï~œ¹e, ±ÊÏ
í êi •p‹IÚ±Æ)zí ri •p‹I í ã/GŒ—ƒ
Ó, ±,‡gCþxj •î‹I½ö±SÒi•î‹IÚ±[ÜŠŷi
•î‹I í ã/G•Œ—ƒÓ.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
‚5 ä : e‚5b ¤á, @oei Ø•¹5ggCþ ?Û&
E, Ïdí ãØA¥y?Ûk5K /G, ÄKknd~¦‚
5b ؤá.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
• à5 ä : e• à5, @oí ãþ :´/þ!0ÑÙ
, ÄK, í ãÏ~¬¥y//i.0½/ /i.0½üö
o kƒ /G.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
؃'5 ä : eÕá5¤á, @oí ãþ :Ø¥y5K
5, ÄK, Ñ:ãò¥y”8ì5”½/ì
†50 /G.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5 ä: e
5¤á, @oÆ)zí ri ŒCqw¤´ƒ
pÕá…ÑlN (0, 1). ¤±, 3±ri •p‹Iŷi •î‹I í ã
þ, ²¡þ :(ŷi , ri ), i = 1, · · · , n Œ—Aá3°Ý•4 Y²
‘|ri | ≤ 2«•S(ù‡ªÇA395%†m), …Ø¥y?Ûª³.
·‚•Œ±^Æ)zí
QQ ã5‰
5 ä. ˜|Nþ
•n êâ'u,‡©ÙF (x) QQãÒ´±êâ i/n© ê•
p‹I, ±F (x) i/n© ê•î‹I Ñ:ã.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5 ä: e
5¤á, @oÆ)zí ri ŒCqw¤´ƒ
pÕá…ÑlN (0, 1). ¤±, 3±ri •p‹Iŷi •î‹I í ã
þ, ²¡þ :(ŷi , ri ), i = 1, · · · , n Œ—Aá3°Ý•4 Y²
‘|ri | ≤ 2«•S(ù‡ªÇA395%†m), …Ø¥y?Ûª³.
·‚•Œ±^Æ)zí
QQ ã5‰
5 ä. ˜|Nþ
•n êâ'u,‡©ÙF (x) QQãÒ´±êâ i/n© ê•
p‹I, ±F (x) i/n© ê•î‹I Ñ:ã.XJêâ´5
gF (x) ˜‡{ü‘Å
, Kù Ñ:AŒ—3˜^†‚þ,
ÏdÏL äQQã´ÄCq3˜^†‚þŒ
¤‰ êâ8
´ÄÑl¤•½©Ù ˜ ÚO•â.
d , ·‚„Œ±Shapiro-Wilk•{5‰
Tianxiao Pang
1nÙ
5u .
£8ëê
O
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
−2
−1
0
r
1
2
LSE LSE
0.0
0.2
0.4
0.6
0.8
1.0
y^
©Û: Ù5G†
5b Ä ˜—, ÏdŒ@•
e ∼ N (0, σ 2 I)´Ün .
Tianxiao Pang
1nÙ
£8ëê
O
5b
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
r
−1.5
−1.0
−0.5
0.0
0.5
1.0
LSE LSE
0.0
0.2
0.4
0.6
0.8
1.0
y^
©Û: í ã¥y/i., Ïd@•• à5b ØÜn.
Tianxiao Pang
1nÙ
£8ëê
O
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
−6
−4
−2
r
0
2
4
6
LSE LSE
0.0
0.2
0.4
0.6
0.8
1.0
y^
©Û: í ã¥y /i., Ïd@•• à5b
Tianxiao Pang
1nÙ
£8ëê
O
ØÜn.
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
−0.15
−0.10
−0.05
r
0.00
0.05
0.10
LSE LSE
0.0
0.2
0.4
0.6
0.8
1.0
y^
©Û: ã/¥yk5K /G, Ïd@•‚5b ØÜn, í
A¹kgCþ &E.
Tianxiao Pang
1nÙ
£8ëê
O
¥
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
−3
−2
−1
r
0
1
2
3
LSE LSE
0.0
0.2
0.4
0.6
0.8
1.0
y^
©Û: í ã¥y8ì5, Ïd@•؃'5ØÜn.
Tianxiao Pang
1nÙ
£8ëê
O
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
r
−1.0
−0.5
0.0
0.5
1.0
LSE LSE
0.0
0.2
0.4
0.6
0.8
1.0
y^
©Û: í ã¥yì
†5, Ïd@•؃'5ØÜn.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
lí ã äÑ5ŒU /;¾0, •Ò´, b ^‡Ø¤á,
·‚I‡é¯K/éwe†0:
XJkwG¦·‚~¦‚5b ؤá, @o·‚Œ±•Ä3£
8gCþ¥O\, gCþ
g‘, Xx21 , x22 ½
‘x1 x2 ,
äNO\= gCþ = ‘, IÀ¢S J ½;
XJkwG¦·‚~¦• à5b ؤá, @oŒ±•ÄéÏ
CþŠ·
C†¦#CþäkCqƒ
• (• -½5C
†), ½öæ^2•
¦ O;
XJkwG¦·‚~¦Øƒ'5b ؤá, @oŒ±•ÄéÏ
CþŠ/ ©0, ¦#CþäkCqÕá5;
XJkwG¦·‚~¦
5b ؤá, @·‚Œ±•ÄéÏ
CþŠ
5C†–Box-CoxC†. Š ˜J ´, Box-CoxC†´
˜«nÜ£n•Y, e˜!2•[0 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
• -½5C† :
‘ÅCþY þŠ•µ, • •σ 2 , b • •þŠ ¼ê, =
b σ 2 = g(µ), Ù¥g´˜‡®• ¼ê. ~X, eY ∼ B(n, p),
KE(Y ) = np, Var(Y ) = np(1 − p) = µ(1 − µ/n). yI {‰˜
‡C†Z = f (Y ), ¦ Var(Z)•~ê. ùI‡éѼêf Lˆ
ª. (ùp, ·‚^Œ i1Y ÚZL«‘ÅCþ,
i1yÚzL
«š‘Å Cþ)
Pz = f (y), ¿-§3y = µ?TaylorÐm,
Cqª
0
f (y) = f (µ) + f (µ)(y − µ).
òyU¤‘ÅCþ, k
0
Z = f (Y ) = f (µ) + f (µ)(Y − µ),
0
0
Ù• c = Var(Z) = [f (µ)]2 Var(Y ) = [f (µ)]2 g(µ).
p
c/g(µ), K
Z p
f (y) =
c/g(y)dy.
Tianxiao Pang
1nÙ
£8ëê
O
0
f (µ) =
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
e¡´A‡A~:
(1) g(µ)†µ¤ 'ž, Pg(µ) = aµ, K
r
Z r
c
c
0
dy = 2
y+c ,
ay
a
√
Ñ ~êØO, ŒŠ²•ŠC†: Z = Y .
(2)
g(µ)†µ2 ¤ 'ž, Pg(µ) = aµ2 , y > 0, K
r
Z r
c
c
0
dy =
ln y + c .
ay 2
a
Ñ
~êØO, ŒŠéêC†: Z = ln Y .
(3)
g(µ)†µ4 ¤ 'ž, Pg(µ) = aµ4 , K
r
Z r
c
c1
0
dy = −
+c .
ay 4
ay
Ñ
~êØO, ŒŠ êC†: Z = 1/Y .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
3A^þ, Äklí ãoÑ/• ˜eσ 2 †µŒU•3
'X(= O¼êg(·)), , lúª
Z p
f (y) =
c/g(y)dy
A«
¦ÑéA C†. éA«C†L êâ©OŠ•
¦?n, Š
# í ã, w=˜«C† í ãÕ šà5
î. l¥
ÀѕР• -½5C†.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.4.1 •ïÄ^>p¸z ž ^>þy†z o^>þx
X, yÂ8 , 53rêâ.
L3.4.1: ^>þêâ
i
1
2
3
4
5
6
7
8
9
10
11
12
13
14
x
679
292
1012
493
582
1156
997
2189
1097
2078
1818
1700
747
2030
y
0.790
0.440
0.560
0.790
2.700
3.640
4.730
9.500
5.340
6.850
5.840
5.210
3.250
4.430
ŷi
1.669
0.244
2.896
0.984
1.312
3.426
2.840
7.230
3.209
6.822
5.864
5.430
1.920
6.645
êi
-0.879
0.196
-2.336
-0.194
1.388
0.214
1.890
2.270
2.131
0.028
-0.024
-0.220
1.330
-2.215
Tianxiao Pang
1nÙ
√
z= y
0.889
0.663
0.748
0.889
1.643
1.908
2.175
3.082
2.311
2.617
2.417
2.283
1.803
2.105
£8ëê
ẑi
1.229
0.860
1.547
1.052
1.137
1.684
1.532
2.668
1.628
2.562
2.315
2.202
1.294
2.517
O
ẽi
-0.340
-0.197
-0.798
-0.163
0.506
0.224
0.643
0.414
0.683
0.055
0.102
0.080
0.509
-0.412
'
LSE LSE
5Ÿ
åLSE
i
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
ä BCC† GLSE õ- ‚5 * O PC
x
1643
414
354
1276
745
435
540
874
1543
1029
710
1434
837
1748
1381
1428
1255
1777
370
2316
y
3.160
0.500
0.170
1.880
0.770
1.390
0.560
1.560
5.280
0.640
4.000
0.310
4.200
4.880
3.480
7.580
2.630
4.990
0.590
8.190
ŷi
5.220
0.693
0.472
3.868
1.912
0.771
1.157
2.388
4.851
2.958
1.784
4.450
2.251
5.606
4.255
4.428
3.791
5.713
0.531
7.698
êi
-2.060
-0.193
-0.302
-1.988
-1.142
0.619
-0.597
-0.828
0.429
-2.318
2.216
-4.140
1.949
-0.726
-0.775
3.152
-1.161
-0.723
0.059
0.492
Tianxiao Pang
1nÙ
O Stein
√
z= y
1.778
0.707
0.412
1.371
0.877
1.179
0.748
1.249
2.298
0.800
2.000
0.557
2.049
2.209
1.865
2.753
1.622
2.234
0.768
2.862
£8ëê
ẑi
2.148
0.977
0.920
1.798
1.292
0.997
1.097
1.415
2.052
1.563
1.259
1.949
1.380
2.248
1.898
1.943
1.778
2.275
0.935
2.789
O
ẽi
-0.370
-0.270
-0.507
-0.427
-0.415
0.182
-0.348
-0.166
0.245
-0.763
0.741
-1.392
0.670
0.039
-0.033
0.810
-0.156
-0.042
-0.167
0.073
LSE LSE
5Ÿ
i
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
åLSE
x
1130
463
770
724
808
790
783
406
1242
658
1746
468
1114
413
1787
3560
1495
2221
1526
ä BCC† GLSE õ- ‚5 * O PC
y
4.790
0.510
1.740
4.100
3.940
0.960
3.290
0.440
3.240
2.140
5.710
0.640
1.900
0.510
8.330
14.940
5.110
3.850
3.930
ŷi
3.330
0.874
2.004
1.835
2.144
2.078
2.052
0.664
3.743
1.592
5.599
0.892
3.271
0.690
5.750
12.280
4.675
7.348
4.789
Tianxiao Pang
êi
1.460
-0.364
-0.264
2.265
1.796
-1.118
1.238
-0.224
-0.503
0.548
0.111
-0.252
-1.371
-0.180
2.580
2.660
0.435
-3.498
-0.859
1nÙ
O Stein
√
z= y
2.189
0.714
1.319
2.025
1.985
0.980
1.814
0.663
1.800
1.463
2.390
0.800
1.378
0.714
2.886
3.865
2.261
1.962
1.982
£8ëê
ẑi
1.659
1.023
1.316
1.272
1.352
1.335
1.328
0.969
1.766
1.209
2.246
1.028
1.644
0.976
2.285
3.974
2.007
2.699
2.036
O
ẽi
0.530
-0.309
0.003
0.753
0.633
-0.355
0.486
-0.306
0.034
0.254
0.144
-0.228
-0.265
-0.262
0.601
-0.109
0.254
-0.736
-0.054
LSE LSE
5Ÿ
A^•
åLSE
ä BCC† GLSE õ- ‚5 * O PC
¦{,
O Stein
£8•§
ŷ = −0.83130 + 0.00368x.
R§S:
yx=read.table(”ex p47 data.txt”)
x=yx[, 1]
y=yx[, 2]
mydata=data.frame(y,x)
lm.sol=lm(y∼x,data=mydata)
summary(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
‰í ©Û:
y.fit=predict(lm.sol)
e.hat=y-y.fit /*½öe.hat=residuals(lm.sol)*/
e.std=rstandard(lm.sol)
plot(e.hat∼y.fit)
plot(e.std∼y.fit)
plot(e.hat∼x)
ÊÏí
㕌±ÏLXe•ª
:
plot(lm.sol,which=1)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
lí ãŒwÑ, ù´˜‡/i.í ã, ´• à5Ø ÎÜ
√
˜‡wG. •ÄéÏCþyŠC†, }Áz = y, £8•§
ẑ = 0.5822 + 0.000953x.
R§S:
z=sqrt(y)
mydata2=data.frame(z,y,x)
lm.sol2=lm(z∼x,data=mydata2)
summary(lm.sol2)
z.fit=predict(lm.sol2)
e.hat=z-z.fit /*½öe.hat=residuals(lm.sol2)*/
plot(e.hat∼z.fit) /*½ö^plot(lm.sol2,which=1)*/
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
# í ãØ¥y?Û²w5K5, ùL²·‚¤^ C†´Ü
· . •
£8•§•
ŷ = ẑ 2 = (0.5822+0.000953x)2 = 0.339+0.0011x+0.00000091x2 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
e5·‚‰êâ
O Stein
ä: É~: äÚrK•: ä.
É~: ä : duÆ)zí ri ŒCqw¤´ƒpÕá…Ñ
lN (0, 1), @o|ri | > 2´‡ Vǯ‡, u) VÇ •0.05. Ï
0
d, ek,‡|ri | > 2, ·‚Òknd~¦éA
:(xi , yi )´
É~:.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
rK•: ä : kÚ?˜ PÒ, ^Y(i) , X(i) Úe(i) ©OL«
lY , XÚe¥GØ1i1¤
•þ½Ý . GØ1i|êâ ,
•e n − 1|êâ ‚5£8 .•
Y(i) = X(i) β + e(i) , E(e(i) ) = 0, Cov(e(i) ) = σ 2 In−1 . (3.4.6)
rlù‡ .¦
β
LSEP•β̂(i) , K
0
0
β̂(i) = (X(i) X(i) )−1 X(i) Y(i) .
(3.4.7)
•þβ̂ − β̂(i) ‡N 1i|êâé£8Xê O K•Œ , §
´˜‡•þ, ØBuA^©Û, A•ħ ,«êþz¼ê. Cook
ål´Ù¥A^•2• ˜«.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Äk·‚5¦β̂ − β̂(i)
O Stein
°(Lˆª. •d, Ik0 ˜‡ð ª:
Ún
A•n × nŒ_Ý , uÚvþ•n × 1•þ, @ok
0
0
−1
(A − uv )
=A
Tianxiao Pang
−1
A−1 uv A−1
+
.
1 − v 0 A−1 u
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
Pxi • OÝ
n, Υ
O Stein
0
1i1. @oX = (x1 , · · · , xn ) . |^þãÚ
X
0
0
0
(X(i) X(i) )−1 = (X X − xi xi )−1
0
0
= (X X)−1 +
0
0
0
(X X)−1 xi xi (X X)−1
(3.4.8)
,
1 − hii
0
Ù¥hii = xi (X X)−1 xi •lfÝ H 1i‡é ‚ ƒ,
•m\:. e,hii Š Œ, K ¡•pm\:. qÏ•
0
X(i) Y(i) =
X
j6=i
xj yj =
n
X
0
xj yj − xi yi = X Y − xi yi ,
j=1
¤±
Tianxiao Pang
1nÙ
£8ëê
O
¡
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
O Stein
0
β̂(i) = (X(i) X(i) )−1 X(i) Y(i)
0
0
0
h 0
(X X)−1 xi xi (X X)−1 i 0
= (X X)−1 +
(X Y − xi yi )
1 − hii
0
0
(X X)−1 xi xi β̂
0
−1
= β̂ − (X X) xi yi +
1 − hii
0
−1
(X X) xi hii yi
−
1 − hii
0
0
0
(X X)−1 xi yi (X X)−1 xi xi β̂
= β̂ −
+
1 − hii
1 − hii
0
−1
(X X) xi êi
= β̂ −
.
(3.4.9)
1 − hii
¤±
0
(X X)−1 xi êi
β̂ − β̂(i) =
.
1 − hii
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
ål:
CookÚ?e
0
(β̂(i) − β̂) M (β̂(i) − β̂)
Di (M , c) =
,
c
Ù¥M ´‰½
Di (M , c) =
½Ý , c´‰½
~ê. N´wÑ
ê2i
0
0
0
· xi (X X)−1 M (X X)−1 xi .
2
c(1 − hii )
0
M = X X, c = (p + 1)σ̂ 2 , K
Di =
hii
1
ê2i
·
· r2 .
· hii =
2
2
(p + 1)σ̂ (1 − hii )
p + 1 1 − hii i
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n (3.4.2)
Cookål
0
Di =
=
=
0
(β̂(i) − β̂) X X(β̂(i) − β̂)
(p + 1)σ̂ 2
ê2i
· hii
(p + 1)σ̂ 2 (1 − hii )2
1
hii
·
· r2 , i = 1, · · · , n,
p + 1 1 − hii i
Ù¥hii •lfÝ H
1i‡é
(3.4.10)
, ri ´Æ)zí .
ù‡½nL², 3OŽCookål žÿ, ·‚•I‡l
êâ
‚5£8 .ŽÑÆ)zí ri ÚlfÝ
é
hii Ҍ
± , Ø7é?Û˜‡Ø
êâ ‚5£8 .(3.4.6) ?1
OŽ.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
hii
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
¹Â: hii Ýþ 1i|êâxi
O Stein
¥%x̄ = n1
Pn
i=1 xi
ål.
•Ä .
Y = Xβ + e, E(e) = 0, Cov(e) = σ 2 In ,
b
gCþ®¥%z, K
1 x11 − x̄1
..
..
X = .
.
···
..
.
1 xn1 − x̄1 · · ·
0
x1p − x̄p
1 (x1 − x̄)
.
..
..
,
=: ..
.
.
0
xnp − x̄p
1 (xn − x̄)
0
0
ùpx̄ = (x̄1 , · · · , x̄p ) , xi = (xi1 , · · · , xip ) . ²{üOŽŒ
0
n 0
XX=
,
0 L
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
Ù¥L =
ä BCC† GLSE õ- ‚5 * O PC
0
Pn
i=1 (xi − x̄)(xi − x̄) , •p
hii =
O Stein
• . ?˜ÚŒ•
1
0
+ (xi − x̄) L−1 (xi − x̄).
n
e´˜ ¥%z‚5£8 ., K
1 x1 − x̄
1 (xi − x̄)2
0
n 0
..
.
.
X = .
, hii = +
.
. , X X = 0 S
n
Sxx
xx
1 xn − x̄
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
hii
3úª(3.4.10)¥, 1/(p + 1)†iÃ', Pi = 1−h
´hii üNO¼
ii
1 Pn
ê, Ï•hii Ýþ 1i|êâxi ¥%x̄ = n i=1 xi ål. Ï
dPi •x 1i|êâålÙ§êâ
C. Cookål Pi Úri2
Œ ¤û½. ½n3.4.2wŠ·‚, pm\:ŒU´rK•:, •Œ
UØ´; É~:ŒU´rK•:, •ŒUØ´; XJ˜|êâQ
´pm\:q´É~:, @o§Ò´rK•:.
‡‰Cookål˜‡^± ½rK•:
Tianxiao Pang
1nÙ
.Š´é(J .
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
A^˜&ý¥Œ±éCookålí ¥ M , c À ‰ƒ˜½
nØ|±. 3Ø
b e,
0
β̂ ∼ N (β, σ 2 (X X)−1 ),
|^íØ2.4.1•
0
0
(β̂ − β) X X(β̂ − β)
∼ χ2 (p + 1),
σ2
,˜•¡
(n − p − 1)σ̂ 2
∼ χ2 (n − p − 1),
σ2
…
öƒpÕá. ¤±
0
0
(β̂ − β) X X(β̂ − β)
∼ F (p + 1, n − p − 1).
(p + 1)σ̂ 2
Tianxiao Pang
1nÙ
£8ëê
O
(3.4.11)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
8Ü
0
0
n
o
(β − β̂) X X(β − β̂)
S= β:
≤
F
(p
+
1,
n
−
p
−
1)
α
(p + 1)σ̂ 2
¡•β ˜&Y²•1 − α ˜&ý¥. (3.4.10)†(3.4.11)¥ Ú
Oþéƒq, cö¿ØÑlF ©Ù. , /Ïu ö·‚Œ±
éDi Š Œ ‰Ñ±e VÇ)º.
0
~X, eDi = F0.50 (p + 1, n − p − 1), KL²1i|êâ(xi , yi )
GØ , β
Oβ̂(i) £Ä
β ˜&Y²•0.5 ˜&ý¥>.
þ; eDj = F0.80 (p + 1, n − p − 1), Kβ̂(j) £Ä
β ˜&Y²
•0.2 ˜&ý¥>.þ. Œ•1i|êâé O K•'1j|ê
â5 Œ.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
éu~3.4.1, ·‚‰Xe êâ ä©Û:
R§S:
yx=read.table(”ex p47 data.txt”)
x=yx[,1]
y=yx[,2]
z=sqrt(y)
mydata2=data.frame(z,x)
lm.sol2=lm(z∼x,data=mydata2)
summary(lm.sol2)
z.fit=predict(lm.sol2)
r.std=rstandard(lm.sol2)
plot(r.std∼z.fit)
text(z.fit,r.std,type=”1:53”)
r.std
influence.measures(lm.sol2)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
©Û: 126Ò
:•É~:. ek”
Š?1?˜Ú(@.
Tianxiao Pang
O Stein
cüŒ” :, K ÑÆ)zí
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
o(: 126Ò
:•É~:, ·‚I‡u êâ5 ´ÄkL
”, eÃL”KíØd
:. d , 126Ò
:•rK•:,
IÚå5¿.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.4.2 œåÿÁêâ. L3.4.2´ ˜Æ[ÿÁ 21‡ Ö P
¹, Ù¥x´ Ö c#(ü : ), y´,«œå•I, ÏLù
êâ, ·‚‡ïáœå‘c#Cz 'X.
L3.4.2 œåÿÁêâ
SÒ
1
2
3
4
5
6
7
8
9
10
11
y
95
71
83
91
102
87
93
100
104
94
113
SÒ
12
13
14
15
16
17
18
19
20
21
x
9
10
11
11
10
12
42
17
11
10
y
96
83
84
102
100
105
57
121
86
100
Tianxiao Pang
1nÙ
£8ëê
O
x
15
26
10
9
15
20
18
11
8
20
7
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
R §S:
x=c(15,26,10,9,15,20,18,11,8,20,7,9,10,11,11,10,12,42,17,11,10)
y=c(95,71,83,91,102,87,93,100,104,94,113,96,83,84,102,100,
105,57,121,86,100)
lm.sol=lm(y∼x)
summary(lm.sol)
influence.measures(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
plot(lm.sol,which=2) /*QQã*/
y.fit=predict(lm.sol)
e=y-y.fit
r.std=rstandard(lm.sol)
plot(r.std∼y.fit,xlab=expression(hat(y)),ylab=“r”)
text(y.fit,r.std,type=“1:25”)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
o(: 119Ò
:•É~:, 118,19Ò
:´rK•:.
dQQãÚí ã, ·‚ É‚5b !• à5b !؃'
5b Ú
5b .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Box-CoxC†
éu*ÿêâ, e²L£8 ä
•, §‚Ø÷v‚5b !
• à5b !؃'5b Ú
5b ¥ ˜‡½eZ‡,
@o·‚Ò‡éêâæ ”£ ”„–. ¢‚y², êâC†´?
nk¯Kêâ ˜«Ð•{.
!0 Box-CoxC†, § ̇A:´Ú\˜‡ëê, ÏLêâ
OTëê, l (½Aæ
êâC†/ª. ¢‚y²,
Box-CoxC†éNõ¢SêâÑ´1ƒk
.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Box-CoxC†´éÏCþ
(
y (λ) =
O Stein
XeC†:
y λ −1
λ ,
ln y,
λ 6= 0,
λ = 0,
(3.5.1)
ùpλ´˜‡–½ C†ëê.
Box-CoxC†´˜xC†, §•) Nõ~„ C†, ÃXéêC
†(λ = 0), êC†(λ = −1)Ú²•ŠC†(λ = 1/2)
.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
éÏCþ n‡*ÿŠy1 , · · · , yn A^þãC†,
þ
0
(λ)
Y (λ) = (y1 , · · · , yn(λ) ) .
C†
•
·‚‡(½C†ëêλ¦ Y (λ) ÷v
Y (λ) = Xβ + e, e ∼ N (0, σ 2 I).
=‡¦C†
X,Ø ÷v
(3.5.2)
•þY (λ) †£8gCþƒmäk‚5ƒ''
©Ù!• à5!ƒpÕá.
Ïd, Box-CoxC†´ÏLéëêλ ÀJ, ˆ é 5êâ ”n
Ü£n”, ¦Ù÷v˜‡
‚5£8 . ¤kb ^‡.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
·‚^4Œq,•{5(½λ. é ½ λ, βÚσ 2 , Y (λ) q,¼
ê•
o
n
1
1
0
√
exp − 2 (Y (λ) − Xβ) (Y (λ) − Xβ) ,
2σ
( 2πσ)n
¤±Y
q,¼ê•
o
n
1
1
0
L(β, σ 2 ) = √
exp − 2 (Y (λ) − Xβ) (Y (λ) − Xβ) |J|,
2σ
( 2πσ)n
ùpJ•C† Jacobi1 ª
J=
n
(λ)
Y
dy
i
i=1
dyi
=
n
Y
yiλ−1 .
i=1
ln L(β, σ 2 )'uβÚσ 2 ¦ ¿-Ù u", Œ βÚσ 2
(
0
0
β̂(λ) = (X X)−1 X Y (λ) ,
0
∆
σ̂ 2 (λ) = n1 Y (λ) (I − H)Y (λ) = n1 RSS(λ, Y (λ) ).
Tianxiao Pang
1nÙ
£8ëê
O
MLE•
(3.5.3)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
éA q,¼ê•ŒŠ•
Lmax (λ) = L(β̂(λ), σ̂ 2 (λ))
= (2πe)−n/2 · |J| ·
RSS(λ, Y (λ) ) −n/2
n
.(3.5.4)
ù´λ ¼ê,·‚ÏL¦§ •ŒŠ5(½λ. ù duÏL
¦ln Lmax (λ) •ŒŠ5(½λ. Äk, k
h
i
n
ln Lmax (λ) = − · ln RSS(λ, Y (λ) ) + ln |J| + C
2
0
n h Y (λ)
Y (λ) i
= − ln
(I
−
H)
+C
2
|J|1/n
|J|1/n
i
n h
∆
= − ln RSS(λ, Z (λ) ) + C,
(3.5.5)
2
(λ)
(λ) 0
Ù¥Z (λ) = (z1 , · · · , zn ) = Y (λ) /|J|1/n ,
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
RSS(λ, Z (λ) ) = Z (λ) (I − H)Z (λ) .
(3.5.6)
(3.5.5)ªéBox-CoxC†3OŽÅþ ¢y‘5錕B, Ï••
¦ln Lmax (λ) •ŒŠ, •I¦RSS(λ, Z (λ) ) • Š. •,é
Jé ¦RSS(λ, Z (λ) )ˆ • Š λ )ÛLˆª, é˜X
‰½ λŠ, ÏL¦•
¦ O £8§S, N´OŽÑéA
(λ)
RSS(λ, Z ). , xÑRSS(λ, Z (λ) )'uλ ã, lãþŒ±C
q éѦRSS(λ, Z (λ) ) ˆ • Š λ̂.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Box-CoxC† äNÚ½:
(λ)
1. 鉽 λŠ,OŽzi , i = 1 · · · , n.
2. U(3.5.6)ªOŽí ²•ÚRSS(λ, Z (λ) ).
3. é˜X ‰½ λŠ, -EþãÚ½,
ƒA í ²•
ÚRSS(λ, Z (λ) ) ˜GŠ. ±λ•î¶, RSS(λ, Z (λ) ) •p¶, xÑ
ƒA -‚. ^†*•{, éѦRSS(λ, Z (λ) )ˆ • Š λ̂.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.5.1 3~3.4.1¥, ·‚éÏCþyŠ ²•ŠC†, ùƒ u¦
^λ = 0.5 Box-CoxC†. ·‚y35y¢ù
C†´Ü· .
eL‰Ñ 12‡ØÓ λŠéA í ²•ÚRSS(λ, Z (λ) ), {ü
'
Œuy λ = 0.5ží ²•ÚRSS(λ, Z (λ) )ˆ • . Ï
dCq/@•0.5Ò´C†ëêλ •`ÀJ.
L3.5.1
λ
RSS
λ
RSS
-2
34101.04
0.375
100.26
-1
986.04
0.5
96.95
-0.5
291.59
0.625
97.29
Tianxiao Pang
0
134.10
0.75
101.69
1nÙ
£8ëê
0.125
119.20
1
127.87
O
0.25
107.21
2
1275.56
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
R§S:
x<c(679,292,1012,493,582,1156,997,2189,1097,2078,1818,1700,747,
2030,1643,414,354,1276,745,435,540,874,1543,1029,710,1434,837,1748,
1381,1428,1255,1777,370,2316,1130,463,770,724,808,790,783,406,1242,
658,1746,468,1114,413,1787,3560,1495,2221,1526)
y<-c(0.79,0.44,0.56,0.79,2.70,3.64,4.73,9.50,5.34,6.85,5.84,5.21,
3.25,4.43,3.16,0.50,0.17,1.88,0.77,1.39,0.56,1.56,5.28,0.64,4.00,0.31,
4.20,4.88,3.48,7.58,2.63,4.99,0.59,8.19,4.79,0.51,1.74,4.10,3.94,0.96,
3.29,0.44,3.24,2.14,5.71,0.64,1.90,0.51,8.33,14.94,5.11,3.85,3.93)
lm.sol<-lm(y∼x)
library(MASS)
boxcox(lm.sol,plotit=T,lambda=seq(-2,2,by=0.05))
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
2•
åLSE
ä BCC† GLSE õ- ‚5 * O PC
¦
O Stein
O
3c¡ ?Ø¥, ·‚o´b ‚5£8 . Ø ´• à5
…؃' , =Cov(e) = σ 2 In . ´3Nõ¢S¯K¥, êâ
Ø÷vù‡b (ŒÏLí ã ä). ùž·‚Ib Ø •þ
• Ý •Cov(e) = σ 2 Σ, ùpΣ´˜‡ ½Ý . ù‡Σ
•¹™•ëê, ·‚ùpb Σ ´
®• .
·‚‡?Ø ‚5£8 .•
Y = Xβ + e, E(e) = 0, Cov(e) = σ 2 Σ.
̇8 ´ Oβ.
Tianxiao Pang
1nÙ
£8ëê
O
(3.6.1)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ï•Σ´é¡ ½Ý , ¤±•3n × n
P¦
0
Σ = P ΛP ,
ùpΛ = diag(λ1 , · · · , λn ), λi > 0, i = 1, · · · , n, ´Σ A Š.
P
1
1
1
0
Σ 2 = P diag(λ12 , · · · , λn2 )P ,
1
1
1
1
KŒ•(Σ 2 )2 = Σ, ¡Σ 2 ´Σ ²•Š . Σ− 2 L«Σ 2
.
1
C†. ^Σ− 2 †¦(3.6.1). P
·‚r‚5£8 .(3.6.1)?1
1
1
1
Z = Σ− 2 Y , U = Σ− 2 X, ε = Σ− 2 e.
Tianxiao Pang
_Ý
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
1
O Stein
1
Ï•Cov(ε) = Σ− 2 σ 2 ΣΣ− 2 = σ 2 In , u´·‚ Xe
8 .
‚5£
Z = U β + ε, E(ε) = 0, Cov(ε) = σ 2 In .
ù´·‚®?ØL
.. 3ù# .¥, Œ β
0
0
0
0
(3.6.2)
LSE•
β ∗ = (U U )−1 U Z = (X Σ−1 X)−1 X Σ−1 Y .
·‚¡β ∗ •β
ÚO5Ÿ.
2•
(3.6.3)
¦ O(GLSE). ù‡ OäkûÐ
½n (3.6.1)
(a) E(β ∗ ) = β;
0
(b) Cov(β ∗ ) = σ 2 (X Σ−1 X)−1 ;
0
0
(c) é?¿ p + 1‘ •þc, c β ∗ •c β
O.
Tianxiao Pang
1nÙ
•˜• • ‚5Ã
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
y²: (a)
0
0
0
0
E(β ∗ ) = (X Σ−1 X)−1 X Σ−1 E(Y ) = (X Σ−1 X)−1 X Σ−1 Xβ = β.
(b) |^½n2.1.3,
0
0
Cov(β ∗ ) = Cov[(X Σ−1 X)−1 X Σ−1 Y ]
0
0
0
0
0
= (X Σ−1 X)−1 X Σ−1 Cov(Y )((X Σ−1 X)−1 X Σ−1 )
0
0
0
0
0
= σ 2 (X Σ−1 X)−1 X Σ−1 Σ((X Σ−1 X)−1 X Σ−1 )
0
= σ 2 (X Σ−1 X)−1 .
0
(c)
0
b Y ´c β
?˜‚5Ã
0
0
0
O. éu .(3.6.2),
0
0
1
0
1
0
1
c β ∗ = c (U U )−1 U Z, b Y = b Σ 2 Σ− 2 Y = b Σ 2 Z,
0
0
0
1
0
0
=c β ∗ •c β LSE, b Y = b Σ 2 Z•c β
é .(3.6.2)A^Gauss-Markov½n•
0
0
1
‚5Ã
0
Var(c β ∗ ) ≤ Var(b Σ 2 Z) = Var(b Y ),
0
1
0
Tianxiao Pang
1nÙ
0
Ò¤á …= c β ∗ = b Σ 2 Z = b Y .
£8ëê
O
O. ¤±
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n3.6.1(c)Ò´˜„œ/e Gauss-Markov½n, §L²3˜„
‚5£8 .(3.6.1)¥, GLSE β ∗ ´•` (eΣ = In , KGLSEò
z LSE β̂). éu .(3.6.1), N´y²β̂E´Ã
O, ™7
0 ∗
0
´•` ‚5Ã
O, Ï•Var(c β ) ≤ Var(c β̂). ùÒ´`,
éu˜„‚5£8 .(3.6.1), GLSEo´`uLSE.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
.(3.6.1)•{ü ~f´ÏCþ ØÓ*ÿäkØ •
/, =
Cov(e) = diag(σ12 , · · · , σn2 ),
ùp
X
0
œ
0
σi2 , i = 1, · · · , nØ ƒ . Px1 , · · · , xn ©O´ OÝ
n‡1•þ. N´íÑ,
β∗ =
0
n
n
X
x x −1 X x y i=1
i i
i i
σi2
σi2
i=1
0
ü‡Úª©O´xi xi Úxi yi \ Ú(
¡β ∗ •\ •
¦ O(WLSE).
.
(3.6.4)
-Ñ•1/σi2 ), Ïd•
σi2
´™• , ùž·‚I‡ {¦ §‚
Oσ̂i2 , ,
3(3.6.4)¥^σ̂i2 “Oσi2 . ù« O•{¡•üÚ O(two-stage
estimate).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.6.1 b ·‚^˜«°—¤ì3ü‡¢ ¿éÓ˜‡þµ©O
?1 n1 gÚn2 gÿþ, Pù ÿþŠ©O•y11 , · · · , y1n1 Ú
y21 , · · · , y2n2 . r§‚ ¤‚5£8 ./ª
y1i = µ + e1i , i = 1, · · · , n1 ,
y2i = µ + e2i , i = 1, · · · , n2 .
duü‡¢ ¿ •*^‡9¤ì
• Ø .
°ÝØÓ,
§‚ ÿþØ
Var(e1i ) = σ12 , Var(e2i ) = σ22 , σ12 6= σ22 .
0
Pe = (e11 , · · · , e1n1 , e21 , · · · , e2n2 ) , K
2
σ1 In1
Cov(e) =
0
0
σ22 In2
= σ22
θIn1
0
0
In2
£8ëê
O
ùpθ = σ12 /σ22 . b θ®•, KΣ®•.
Tianxiao Pang
1nÙ
∆
= σ22 Σ,
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
5¿ ùp
OÝ X = (1, · · · , 1) , u´µ
µ∗ =
n
n
i=1
i=1
GLSE•
1
2
X
−1 1 X
n1
+ n2
y1i +
y2i .
θ
θ
P
n
1
1 X
y1i ,
ȳ1 =
n1
i=1
1
n1
ω1 =
= 2,
Var(ȳ1 )
σ1
n
2
1 X
ȳ2 =
y2i ,
n2
i=1
ω2 =
1
n2
= 2.
Var(ȳ2 )
σ2
Kµ∗ ŒU •
µ∗ =
ω1
ω2
ȳ1 +
ȳ2 .
ω1 + ω2
ω1 + ω2
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
ω1
=µ∗ ´ü‡¢ ¿*ÿŠþŠ \ ²þ, §‚
ω1 +ω2 Ú
ω2
ω1 +ω2 †ˆ¢ ¿ÿþ Ø • Úÿþgêk'. Ø • Œ
, ÿþgê
, éA
Ò .
µ∗ •¹™•ëêσ12 Úσ22 , ØUGâSA^. ·‚Œ± {
Eσ12 Úσ22
O. ¯¢þ, ùü‡¢ ¿ *ÿêâ©O ¤‚
5£8 .
Yi = µ1ni + ei , i = 1, 2,
0
0
ùpYi = (y1i , · · · , y1ni ) , ei = (ei1 , · · · , eini ) . Ï•Cov(ei ) =
σi2 Ini , ¤±ei , i = 1, 2÷vGauss-Markovb . ¤±σi2 LSE•
σ̂i2 =
1
kYi − ȳi 1ni k2 i = 1, 2.
ni − 1
^σ̂i2 , i = 1, 2“Oµ∗ ¥
σi2 , i = 1, 2, =Œ
Tianxiao Pang
1nÙ
£8ëê
µ
O
üÚ O.
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
õ- ‚5
£8Xê LSEkNõ`û 5Ÿ, Ù¥•-‡ ´GaussMarkov½n, §L²3‚5Ã
Oa¥, LSE´•˜ äk•
•
O. ´ù˜`:, ¦ LSE3‚5ÚO .
OnØ
Ú¢SA^¥Ókýé-‡ / .
´·‚±c?Ø LSEI‡b
OÝ X´ ÷• , =‡
¦Ý X
•þƒm´‚5Ã' . , , 3¢S¯K¥, du
²~‡?n¹k õgCþ Œ.£8¯K, …²LCþƒm
Ø´ á
´ƒpéX , ù Ñ—¦ OÝ X
•þ
ƒmØŒU
‚5Ã'. éõœ¹e, OÝ X
•þƒ
m•3õ- ‚5/E ‚5(multi-collinearity)'X.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½Â
e•3Ø •0
p + 1‡~êc0 , c1 , · · · , cp ¦
c0 + c1 xi1 + · · · + cp xip = 0, i = 1, · · · , n,
K¡gCþx1 , · · · , xp ƒm•3X
X.
õ- ‚5/E ‚5'
3¢S¯K¥,
õ- ‚5/E ‚5'X¿Øõ„, ˜„
Ñy ´˜½§Ýþ
‚5.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½Â (õ- ‚5/E ‚5'X)
e•3Ø •0
p + 1‡~êc0 , c1 , · · · , cp ¦
c0 + c1 xi1 + · · · + cp xip ≈ 0, i = 1, · · · , n,
K¡gCþx1 , · · · , xp ƒm•3Xõ- ‚5/E ‚5'X.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
é²Lêâï ž, õ- ‚5 œ/éõ, õ- ‚5œ/¬
‰õ ‚5£8©Û‘5ŸoK•!XÛ ägCþƒm õ‚5±9XÛŽÑõ- ‚5 K• ¯Kò´ !‡?Ø
̇SN.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
·‚kÚ\˜‡Vg: þ•Ø (MSE: Mean Squared Errors), §
´^5µd˜‡ O`
IO .
½Â
θ•˜ •þ. θ̂•θ
˜‡ O. ½Âθ̂
þ•Ø •
0
MSE(θ̂) = Ekθ̂ − θk2 = E[(θ̂ − θ) (θ̂ − θ)].
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n (3.7.1)
MSE(θ̂) = tr[Cov(θ̂)] + kEθ̂ − θk2 .
y²: ØJwÑ
0
MSE(θ̂) = E[(θ̂ − θ) (θ̂ − θ)]
0
= E[(θ̂ − Eθ̂) + (Eθ̂ − θ)] [(θ̂ − Eθ̂) + (Eθ̂ − θ)]
0
0
= E(θ̂ − Eθ̂) (θ̂ − Eθ̂) + E(Eθ̂ − θ) (Eθ̂ − θ)
∆
= ∆ 1 + ∆2 .
|^, 5Ÿ,
0
∆1 = E{tr[(θ̂ − Eθ̂) (θ̂ − Eθ̂)]}
0
= E{tr[(θ̂ − Eθ̂)(θ̂ − Eθ̂) ]}
0
= tr[E(θ̂ − Eθ̂)(θ̂ − Eθ̂) ] = tr[Cov(θ̂)].
0
∆2 = E[(Eθ̂ − θ) (Eθ̂ − θ)] = kEθ̂ − θk2 ´w, . y..
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
ePθ̂ = (θ̂1 , · · · , θ̂p+1 ) , K
∆1 =
p+1
X
Var(θ̂i ),
i=1
§´θ̂ˆ©þ • ƒÚ.
∆2 =
p+1
X
(Eθ̂i − θi )2 ,
i=1
§´θ̂ˆ©þ
-²•ƒÚ. ¤±, ˜‡ O þ•Ø d§
• Ú
¤û½. ˜‡Ð
OAk
• Ú
.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n
3‚5£8 .(3.1.5)¥,
éβ LSE β̂, k
P
1
(a) MSE(β̂) = σ 2 p+1
;
i=1 λi
P
1
(b) Ekβ̂k2 = kβk2 + σ 2 p+1
i=1 λi ,
0
Ù¥λ1 , · · · , λp+1 > 0•X X A Š.
y²: (a)Ï•LSE β̂´Ã
O, ¤±∆2 = 0,
0
MSE(β̂) = ∆1 = tr[Cov(β̂)] = σ 2 tr[(X X)−1 ].
0
Ï•X X´é¡ ½Ý , ¤±•3
P¦
0
0
X X = P diag(λ1 , · · · , λp+1 )P ,
0
ùpλ1 , · · · , λp+1 > 0•X X
0
(X X)−1 = P diag(
Tianxiao Pang
A Š. ¤±
1
1
0
,··· ,
)P .
λ1
λp+1
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
|^, 5ŸêþŒ
p+1
1
1 X 1
0
tr(X X)−1 = tr diag( , · · · ,
) =
.
λ1
λp+1
λi
i=1
¤±MSE(β̂) = σ
Pp+1 1
2
i=1 λi .
(b) Ï•
0
MSE(β̂) = E[(β̂ − β) (β̂ − β)]
0
0
0
= E(β̂ β̂ − 2β β̂ + β β)
0
= Ekβ̂k2 − β β,
u´
Ekβ̂k2 = kβk2 + MSE(β̂) = kβk2 + σ 2
p+1
X
1
i=1
Tianxiao Pang
1nÙ
£8ëê
O
λi
.
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
(Ø(a)wŠ·‚, XJX X– k˜‡A Šš~ , =š~
Cu", @oMSE(β̂)Ò¬éŒ. lþ•Ø
IO5w, •
¦ Oβ̂Ø´˜‡Ð
O. ùÚGauss-Markov½n¿Øgñ, Ï
•Gauss-Markov½n== y •
¦ O3‚5Ã
Oa
0
¥ • • 5. 3X X– k˜‡A Šš~ ž, ù‡•
• Š
%éŒ, Ï
— éŒ þ•Ø .
0
(Ø(b)wŠ·‚, XJX X– k˜‡A Šš~ , @o•
¦ Oβ̂ •ݲþ`5‡'ý
β •Ý•éõ. ùÒ
— β̂ , ©þ ýéŠLŒ.
0
oƒ, X X– k˜‡A Šš~ ž, •
˜‡Ð
O .
Tianxiao Pang
1nÙ
£8ëê
¦ OØ2´
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
·‚5©Û”– k˜‡A Šš~ ”3 OÝ X½ö£8g
Cþþ¿›XŸo.
0
PX = (1, x1 , · · · , xp ), =xi •X 1i + 1 . λ•X X
0
A Š, φ•ÙéA A •þ, Ù•Ý•1, =φ φ = 1.
˜‡
eλ ≈ 0, K
0
0
0
kXφk2 = φ X Xφ = λφ φ = λ ≈ 0.
0
u´Xφ ≈ 0. Pφ = (c0 , c1 , · · · , cp ) , K
c0 + c1 x1 + · · · + cp xp ≈ 0.
= OÝ X
(3.7.1)
•þƒm(=gCþƒm)kõ- ‚5'X.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
‡ƒ, e OÝ X
•þƒmkõ- ‚5'X, =(3.7.1)¤
0
0
á, džX XE´ ½Ý
|X X| ≈ 0. ddŒ•
p+1
Y
0
λi = |X X| ≈ 0,
i=1
¤±– k˜‡A Šš~ ,
Cu".
•Ò´`– k˜‡A Šš~ †X
•þƒmkõ- ‚
5'X´ d . ùž¡ OÝ X´¾ Ý .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
õ- ‚5
ä BCC† GLSE õ- ‚5 * O PC
O Stein
ä:
(1) • )äÏf(VIF: variance inflation factor)
PRj2 •gCþxj éÙ{p − 1‡gCþ
VIFj =
ä{
½Xê, ½Â
1
, j = 1, · · · , p.
1 − Rj2
duRj2 Ýþ gCþxj éÙ{p − 1‡gCþƒm ‚5ƒ'§
Ý, x1 , · · · , xp ƒm õ- ‚5 î-, Rj2
Cu1, VIFj •Ò
Œ. Ïd^VIF 5Ýþõ- ‚5 §Ý´Ün .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ýþ OK:
k,‡
VIFj ≥ 10
ž, @•gCþƒm•3î- õ- ‚5; ½ö
p
VIF =
1X
VIFj 1
p
i=1
ž, @•gCþƒm•3î- õ- ‚5.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
(2)A І^‡ê(CI: Condition Index) ä{
• žØþj K•, ·‚b gCþ†ÏCþ *ÿŠþ®I
Oz. džŒ@•‚5£8 .vk å‘, OÝ X´n × p
0
Ý , X X´p‡gCþ
ƒ'XêÝ .
0
A Š ä{: XJX Xkm‡A ŠCq•", @oXÒkm
‡õ- ‚5'X, ¿…ùm‡õ- ‚5'X Xê•þÒ´
ùm‡ Cu" A ФéA IO
zA •þ.
0
^‡ê ä{: b X X p‡A Š©O•λ1 , · · · , λp , Ù¥•
ŒA Š•λmax , • A Š•λmin , ¡
κj =
•A Šλj
λmax
, j = 1, · · · , p
λj
^‡ê.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
P
κ = max κj =
j
λmax
.
λmin
0
§Œ±^5ÝþÝ X X A ŠCq•" §Ý, ÏdŒ±^
5 äõ- ‚5´Ä•3±9õ- ‚5 î-§Ý.
^‡ê
½OK:
e0 < κ < 100, K@•Ø•3õ- ‚5;
e100 ≤ κ ≤ 1000, K@••3 r õ- ‚5;
eκ > 1000, K@••3î- õ- ‚5.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.7.1. •ʇk8‡£8gCþ ‚5£8¯K,
„eL.
©êâ
L3.7.1
SÒ
1
2
3
4
5
6
7
8
9
10
11
12
y
10.006
9.737
15.087
8.422
8.625
16.289
5.958
9.313
12.960
5.541
8.756
10.937
x1
8
8
8
0
0
0
2
2
2
0
0
0
x2
1
1
1
0
0
0
7
7
7
0
0
0
Tianxiao Pang
x3
1
1
1
9
9
9
0
0
0
0
0
0
x4
1
0
0
1
1
1
1
1
1
10
10
10
1nÙ
x5
0.541
0.130
2.116
-2.397
-0.046
0.365
1.996
0.228
1.380
-0.798
0.257
0.440
£8ëê
O
x6
-0.099
0.070
0.115
0.252
0.017
1.504
-0.865
-0.055
0.502
-0.399
0.101
0.432
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
R§S:
yx=read.table(”ex p62 data.txt”)
y=yx[, 1]
x1=yx[, 2]
x2=yx[, 3]
x3=yx[, 4]
x4=yx[, 5]
x5=yx[, 6]
x6=yx[, 7]
mydata=data.frame(y,x1,x2,x3,x4,x5,x6)
lm.sol=lm(y∼ x1+x2+x3+x4+x5+x6,data=mydata)
summary(lm.sol)
library(DAAG)
vif(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
VIFÑÑ(J
x1
182.0500
x2
161.3600
x3
266.2600
x4
297.7100
x5
1.9200
x6
1.4553
Ï•co‡VIPÑØ u10, ¤±@•gCþƒm•3î- õ
- ‚5.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
A
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
І^‡ê ä{:
R§S:
X=cbind(x1,x2,x3,x4,x5,x6)
X
rho=cor(X)
rho
eigen(rho)
kappa(rho,exact=TRUE) /*%@´exact=FALSE, ùžk Œ
OŽØ */
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
A
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Š!A •þ†^‡ê:
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
λmin = 0.001106051, éA A
O Stein
•þ•
0
(0.44768, 0.42114, 0.541689, 0.57337, 0.00605, 0.00217) .
¤±IOzgCþx∗i , i = 1, · · · , 6ƒm•3Xe õ- ‚5'
X
0.44768x∗1 + 0.42114x∗2 + 0.541689x∗3 + 0.57337x∗4 + 0.00605x∗5
+0.00217x∗6 ≈ 0.
ù`²éu ©gCþxi , i = 1, · · · , 6, •3~êc0 ¦
c0 + 0.44768x1 + 0.42114x2 + 0.541689x3 + 0.57337x4
+0.00605x5 + 0.00217x6 ≈ 0.
½ •
c0 + 0.44768x1 + 0.42114x2 + 0.541689x3 + 0.57337x4 ≈ 0.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
^‡êκ = 2195.908 > 1000, •`²gCþƒm•3î- õ‚5'X.
žØõ- ‚5 •{:
OŒ
Nþ, žØ½…)gCþ
ã+Ã 5, Ïék
O.
‚5ƒ'5.
O, ·‚ò0 *
Tianxiao Pang
1nÙ
£8ëê
OṲ́©
O
LSE LSE
5Ÿ
*
O
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
gCþƒmäkõ- ‚5ž, • ŽÑLSE²wC€ ¯K,
Hoerlu1962cJÑ ˜«U? •
¦ O•{, =*
O(Ridge Estimate). ƒ , HoerlÚKennard31970céT OŠ
?˜Ú •[?Ø.
* O gŽ: gCþƒm•3õ- ‚5ž, OÝ X´
0
0
¾
, =|X X| ≈ 0, l (X X)−1 CÛÉ. •;•ù˜y–,
0
‰X X \þ˜‡ ~êé Ý kI(k > 0),KÝ
0
(X X + kI)−1
0
CÛÉ ŒU5‡'(X X)−1
CÛÉ
ŒU5
õ. Ïd
^
0
0
β̂(k) = (X X + kI)−1 X Y
Š•™•ëêβ
OAT'•
Tianxiao Pang
(3.8.1)
¦ O‡-½˜ .
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½Â
0
0
鉽 0 < k < ∞, ¡β̂(k) = (X X + kI)−1 X Y •£8X
êβ * O. d* O¤ïá £8•§¡•*£8•§.
¡k•*ëê. éuβ̂(k) ©þβ̂j (k), r3²¡† ‹IX
¥β̂j (k)‘kCz¤LyÑ5 -‚¡•*,(ridge trace).
5: kØÓ, ·‚
ØÓ
O. Ïd* Oβ̂(k)´˜‡ Oa.
k = 0ž, β̂(k)Ò´Ï~ LSE. ˜„œ¹e, ·‚Jå* O,
´Ø•)LSE .
3?1* Oƒc, • žØþj K•, ·‚ob gCþ†
ÏCþþ®IOz, Ïdùp
OÝ X´n × pÝ .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
* O 5Ÿ :
5Ÿ1: β̂(k)´β
β.
k
O, =é?¿ 0 < k < ∞, E(β̂(k)) 6=
k 5´* O†•
¦ O ˜‡-‡ ØÓƒ?. ˜‡
O þ•Ø d• ƒÚÚ
²•Ú|¤. •3õ- ‚
5ž, •
¦ O•, ±
Ü©•", § • Ü©%
éŒ, •ª —§ þ•Ø éŒ. ·‚Ú?* O 8 ´ã
+à 5, † • Ü© ŒÌÝ~ , •ªü$Ùþ•Ø .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
5Ÿ2: β̂(k)´•
¦ Oβ̂
O Stein
˜‡‚5C†.
y²: •I5¿
0
0
0
0
0
0
β̂(k) = (X X + kI)−1 X Y
0
0
= (X X + kI)−1 X X · (X X)−1 X Y
= (X X + kI)−1 X X β̂.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5Ÿ3: é?¿ k > 0, ekβ̂k =
6 0, K·‚okkβ̂(k)k < kβ̂k.
=* O´r•
¦ Oβ̂• :Š·Ý Ø
,*
O´˜‡Ø k
O.
y²: •Äõ ‚5£8 .Y = Xβ + e, 0
Z = XP , α = P β,
Ù¥P •
Ý ÷v
0
0
P (X X)P = Λ = diag(λ1 , · · · , λp ),
0
λ1 , λ2 , · · · , λp > 0•X X
•
A Š. ùž, õ ‚5£8 .Œ
Y = Zα + e, E(e) = 0, Cov(e) = σ 2 In .
(3.8.2)
·‚¡(3.8.2)•‚5£8 . ;K/ª, ¡α•;K£8Xê.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
0
0
O Stein
0
5¿ Z Z = P X XP = Λ, ¤±
0
0
0
α̂ = (Z Z)−1 Z Y = Λ−1 Z Y .
0
0
0
0
0
β̂ = (X X)−1 X Y = P Λ−1 P X Y = P Λ−1 Z Y = P α̂.
§‚ƒA * O©O•
0
0
0
α̂(k) = (Z Z + kI)−1 Z Y = (Λ + kI)−1 Z Y ,
0
0
β̂(k) = (X X + kI)−1 X Y
0
0
0
0
= P P (X X + kI)−1 P P X Y
= P α̂(k).
Ïd
kβ̂(k)k = kα̂(k)k = k(Λ + kI)−1 Λα̂k < kα̂k = kβ̂k.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5: ´•þ•Ø 3 OÚëê
C†e ±ØC, ¤±;
K£8Xê •
¦ O(½* O)Ú £8Xê •
¦
O(½* O)kƒÓ þ•Ø :
MSE(α̂) = MSE(β̂), MSE(α̂(k)) = MSE(β̂(k)).
Tianxiao Pang
1nÙ
£8ëê
O
(3.8.3)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n (3.8.1, * O•35½n)
•3k > 0¦
MSE(β̂(k)) < MSE(β̂).
=•3k > 0, ¦ 3þ•Ø ¿Âe, *
O`u•
¦ O.
y²: d(3.8.3), •Iy²•3•3k > 0¦
MSE(α̂(k)) < MSE(α̂).
(3.8.4)
Pf (k) = MSE(α̂(k)), k ≥ 0. 5¿f (0) = MSE(α̂). e·‚Uy
0
²f (k)3[0, ∞)þ´ëY¼ê…f (0) < 0, K7•3˜‡
k > 0 ¦ (3.8.4)¤á.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5?Øf (k). 5¿
0
E(α̂(k)) = (Λ + kI)−1 Z E(Y )
0
= (Λ + kI)−1 Z Zα
= (Λ + kI)−1 Λα,
…
0
Cov(α̂(k)) = σ 2 (Λ + kI)−1 Z Z(Λ + kI)−1
= σ 2 (Λ + kI)−1 Λ(Λ + kI)−1 .
¤±
h
i
f (k) = MSE(α̂(k)) = tr Cov(α̂(k)) + kE(α̂(k)) − αk2
= σ
2
p
X
i=1
p
X
αi2
λi
2
+
k
(λi + k)2
(λi + k)2
i=1
∆
= f1 (k) + f2 (k).
Tianxiao Pang
(3.8.5)
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
w,f (k)´[0, ∞)þ ëY¼ê. q
0
f1 (k) = −2σ 2
p
X
i=1
p
X 1
λi
0
, f1 (0) = −2σ 2
< 0 (3.8.6)
3
(λi + k)
λ2i
i=1
±9
0
f2 (k) = 2k
p
X
i=1
0
0
λi αi2
0
, f2 (0) = 0,
3
(λi + k)
0
¤±f (0) = f1 (0) + f2 (0) = −2σ 2
Tianxiao Pang
Pp
1
i=1 λ2i < 0. y..
1nÙ
£8ëê
O
(3.8.7)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
* O•35½n3nØþy² •3,‡* O`u•
¦
O, ‡éÑù‡*ëêk´ØN´ . N´wÑ, nØþ, ù
‡k´e •§ ):
0
f (k) = −2σ
2
p
X
i=1
=
p
X λi α 2
λi
i
+
2k
(λi + k)3
(λi + k)3
i=1
p
X
λi (kαi2 − σ 2 )
2
= 0.
(λi + k)3
i=1
(3.8.8)
ù‡)•6u™•ëêαi , i = 1, · · · , pÚσ 2 , ¤±ØŒUl)•
§
ݼ *ëêk. ÚOÆ[‚lÙ§å»JÑ ÀJ*ë
êk •{.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
*ëêÀJ•{ :
(1) Hoerl-Kennardúª
HoerlÚKennardJÑ
ÀJk
úª´
k̂ =
σ̂ 2
.
max α̂i2
(3.8.9)
i
¼ ù‡úª Ž{Xe: d(3.8.8)•, ekαi2 − σ 2 < 0éi = 1,
0
· · · , pѤá, Kf (k) < 0. u´
k∗ =
σ2
,
max αi2
i
0
0 < k < k ∗ ž, f (k)o´ u", Ï f (k)3(0, k ∗ )þ´üN4
~¼ê, 2df (k)3[0, ∞) ëY5 f (k ∗ ) < f (0). , 2^α̂i
Úσ̂ 2 “Oαi Úσ 2 , B (3.8.9).
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
(2) *,{
òβ̂1 (k), · · · , β̂p (k) *,x3˜Üãþ, Šâ*, Czª³À
Jk. ±e´A^ÀJk OK:
ˆ£8Xê * OŒ—' -½;
^•
¦ OžÎÒØÜn £8Xê, Ù* O ÎÒ
òC Ün;
£8XêvkØÜn
ÎÒ;
í ²•ÚØ‡þ,
õ.
˜„œ¹e, ·‚ÀJ•
½.
kŠU¦ ˆ^*,Ñm©ªu-
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.8.1 {I²LóŠöF"ÏLISo Šx1 ,•;þx2 ,ož¤
þx3 ýÿ?•o y, ±þCþ ü þ•›·{J. •dÂ8
1949-1959 11c êâ, „eL.
c°
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
x1
149.3
161.2
171.5
175.5
180.8
190.7
202.1
212.4
226.1
231.9
239.0
Tianxiao Pang
x2
4.2
4.1
3.1
3.1
1.1
2.2
2.1
5.6
5.0
5.1
0.7
x3
108.1
114.8
123.2
126.9
132.1
137.7
146.0
154.1
162.3
164.3
167.6
1nÙ
y
15.9
16.4
19.0
19.1
18.8
20.4
22.7
26.5
28.1
27.6
26.3
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
R§S:
library(MASS)
yx=read.table(”ex p68 data.txt”)
x1=yx[, 1]
x2=yx[, 2]
x3=yx[, 3]
y=yx[, 4]
mean(x1);mean(x2);mean(x3);mean(y)
sd(x1);sd(x2);sd(x3);sd(y)
mydata=data.frame(y, x1, x2, x3)
lm.sol=lm(y∼x1+x2+x3)
summary(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
¼
5Ÿ
åLSE
ˆCþ
ä BCC† GLSE õ- ‚5 * O PC
þŠ!
O Stein
IO êâ:
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
¦^•
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
¦•{ (J:
£8•§: ŷ = −10.128 − 0.051x1 + 0.587x2 + 0.287x3 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
£8©Û (JL²: x1 £8Xê
O´Kê, ùØÎÜÙ
²L¿Â, ù´Ï•{I´˜‡ á ?•I, ISo Šx1
O\ž, ?•o y•AO\. ¤±, £8Xê ÎÒ†¢SØÎ.
Ù Ï´n‡gCþƒm•3Xõ- ‚5, ùŒ{ü/lx1 ,
x2 , x3
ƒ'XêÝ wÑ:
Ïdx1 †x3 •3pÝ ƒ'5.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Šõ- ‚5 ä, Œ±²(uygCþƒm•3Xõ- ‚5:
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
^* O•{Ïé*£8•§:
kéêâ?1IOz, R§S:
yxs=scale(yx)
x1=yxs[, 1]
x2=yxs[, 2]
x3=yxs[, 3]
y=yxs[, 4]
mydata2=data.frame(x1,x2,x3,y)
mydata2
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
rr.sol=lm.ridge(y∼0+x1+x2+x3,data=mydata2,lambda=
c(seq(0,0.01,by=0.001),seq(0.02,0.1,by=0.01),seq(0.2,1,by=0.1)))
rr.sol /∗w«* O∗/
plot(rr.sol) /∗Š*,ã∗/
matplot(rr.sol$lambda,t(rr.sol$coef),type=”l”,col=c(”red”,”blue”,
”black”),main=”ridge trace”,xlab=expression(lambda),
ylab=expression(hat(beta)(lambda))) /∗Š*,ã∗/
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
*
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Oêâ:
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Šâ*,ãÀJ*ëêk = 0.4, IOzCþ *£8•§•
v̂ = 0.416u1 + 0.213u2 + 0.531u3 .
• , I‡=†¤
ŷ − 21.891
4.544
©Cþ *£8•§:
x1 − 194.591
x2 − 3.300
+ 0.213 ×
30.000
1.649
x3 − 139.736
+0.531 ×
,
20.634
= 0.416 ×
=
ŷ = −8.655 + 0.063x1 + 0.587x2 + 0.117x3 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
¤kCþ þjј—ž, Œ±† é
©êâ?1* O:
rr.sol=lm.ridge(y∼x1+x2+x3,data=mydata,lambda=
c(seq(0,0.01,by=0.001),seq(0.02,0.1,by=0.01),seq(0.2,1,by=0.1)))
rr.sol
plot(rr.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
Tianxiao Pang
1nÙ
O Stein
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
ÀJ*ëêk = 0.4,
O Stein
*£8•§:
ŷ = −8.621 + 0.063x1 + 0.588x2 + 0.117x3 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
2Â* O :
PK = diag(k1 , · · · , kp ), ki ≥ 0, i = 1, · · · , p. ¡
0
0
β̂(k) = (X X + K)−1 X Y
•β
2Â* O.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
̤© O
̤©(Principle Component) O´dW.F.Massyu1965cJÑ
,˜«k
O, 8 ´• ŽÑ OÝ X•¾ Ý ž•
¦ O -½5òC é ù˜"€. ̤© O Ä g
Ž´: Äk/Ïu
C†ò£8gCþC•éA ̤©(Ì
¤© *ÿ•þ´p؃' , l žØ õ- ‚5¯K), ,
l¤k ̤©¥À ˜Ü©-‡ ̤©(å ü‘ Š
^)¿±§‚Š•# £8gCþïá# £8 ., ^•
¦
•{ O# .¥ £8Xê¿
£8•§. Äu
£8
•§2ò§‚=†• ©Cþ £8•§.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
•
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
žØþj K•, b gCþ†ÏCþþ®IOz.
•Ä£8 .:
Y = Xβ + e, E(e) = 0, Cov(e) = σ 2 In ,
0
Ù¥X´n × p OÝ . Pλ1 ≥ · · · ≥ λp > 0•X X
φ1 , · · · , φp •éA IO
zA •þ. K
(3.9.1)
A Š,
Φ = (φ1 , · · · , φp )
•p × p
Ý …
0
0
∆
Φ X XΦ = diag(λ1 , · · · , λp ) = Λ.
0
2PZ = XΦ, α = Φ β, K .(3.9.1)ŒU •
Y = Zα + e, E(e) = 0, Cov(e) = σ 2 In .
Tianxiao Pang
1nÙ
£8ëê
O
(3.9.2)
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
3þã ‚5£8;K .(3.9.2)¥, #
OÝ
∆
Z = (z1 , · · · , zp ) = (Xφ1 , · · · , Xφp ),
ePX = (x1 , · · · , xp ), K#gCþ *ÿ•þ†
*ÿ•þ 'X•
©gCþ
zj = φ1j x1 + · · · + φpj xp , j = 1, · · · , p.
ù´é ©gCþ *ÿ•þ ˜‡‚5Cþ, C† Xê•þ
•A Šλj ¤éA IO
zA •þ.
ÚOþ, ¡*ÿ•þzj , j = 1, · · · , péA #gCþzj , j = 1, · · · ,
p•p‡Ì¤©. z‡Ì¤©Ñ´ ©gCþ ‚5|Ü:
zj = φ1j x1 + · · · + φpj xp , j = 1, · · · , p.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
̤© 5Ÿ : ?¿ü‡ ̤©
*ÿ•þÑ´p؃' ,
Pn
…1j‡Ì¤©
²•Ú i=1 (zij − z̄j )2 = λj .
0
0
0
y²: Ï•Z Z = Φ X XΦ = Λ = diag(λ1 , · · · , λp ), ¤±
0
zj zk = 0, ∀j 6= k
0
…zj zj = λj , j = 1, · · · , p. qÏ•X´I5z OÝ , ¤±
p
n
n
i=1
i=1 k=1
p
n
k=1
i=1
X
1X
1 XX
1X
φkj xik =
φkj
xik = 0.
zij =
z̄j =
n
n
n
Ïdk
n
X
i=1
2
(zij − z̄j ) =
n
X
0
2
zij
= zj zj = λj , j = 1, · · · , p.
i=1
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
λj Ýþ 1j‡Ì¤©zj
ŠCÄŒ . Ï•λ1 ≥ · · · ≥ λp > 0,
¤±·‚¡z1 •1˜Ì¤©, z2 •1 ̤©, · · · . ùp‡Ì¤©
*ÿ•þ´p؃' , ¤±# gCþz1 , · · · , zp (½ OÝ
Z)Ø•3õ- ‚5¯K.
d5Ÿ1Œ•, z1 éÏCþ )ºUå•r, z2 gƒ, · · · , zp •f.
0
e OÝ X´¾ Ý , @ok˜ X X A Šé , Ø”
b
λr+1 , · · · , λp ≈ 0.
ùž, ¡ p − r‡Ì¤©
ŠCÄÒé …þ3"NC Š.
¤±ùp − r‡Ì¤©éÏCþ K•ÒŒ± ÑK, Œò§‚l
£8 .¥GØ. ^•
¦{é•e r‡Ì¤©(=r‡# g
Cþ)Š£8=Œ. • 2C£
©Cþ £8•§.
5: ̤©£8 ̇8 :
Žþ), , ïá£8•§.
Tianxiao Pang
(žØõ- ‚5), ü‘(~ O
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
éΛ, α, Z, ΦŠ©¬:
.
.
Λ1 0
α1
Λ=
, α=
, Z = (Z1 ..Z2 ), Φ = (Φ1 ..Φ2 ),
0 Λ2
α2
Ù¥Λ1 •r × rÝ , α1 •r × 1•þ, Z1 •n × rÝ , Φ1 •p × r
Ý . GØZ2 α2 , .(3.9.2)C•
Y ≈ Z1 α1 + e, E(e) = 0, Cov(e) = σ 2 In .
Z1 Ø´¾ Ý , ¤±Œ† A^•
¦ O α1
0
0
α̂1 = (Z1 Z1 )−1 Z1 Y .
Tianxiao Pang
1nÙ
£8ëê
O
(3.9.3)
LSE
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
c¡·‚l .¥GØ
¡ p − r‡Ì¤©, ùƒ u·‚
^α̂2 = 0
Oα2 . |^'Xβ = Φα, β ̤© O•
α̂1
β̃ = Φ
α̂2
α̂1
= (Φ1 , Φ2 )
0
0
= Φ1 Λ−1
1 Z1 Y
0
0
= Φ1 Λ−1
1 Φ1 X Y .
ƒA ̤©£8•§•Ŷ = X β̃.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
̤© O L§ :
Step1: ‰
C†Z = XΦ, ¼ # gCþ, ¡•̤©.
Step2: ‰£8gCþÀJ, GØéA A Š'
©.
@ ̤
Step3: ò•{ ̤©éy‰•
¦£8, 2ˆ£
O,
'u ©Cþ ̤©£8•§.
5 ëê
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
̤© O 5Ÿ :
0
5Ÿ1. β̃ = Φ1 Φ1 β̂, =̤© O´•
†.
y²: Šâe
¦ O ˜‡‚5C
'X:
.
.
0
0
Z = (Z1 ..Z2 ) = (XΦ1 ..XΦ2 ), Φ1 Φ1 = Ir , Φ1 Φ2 = 0
9
0
0
0
0
X X = ΦΛΦ = Φ1 Λ1 Φ1 + Φ2 Λ2 Φ2
0
¿5¿ X (I − H) = 0(H•lfÝ ), Œ•
0
0
0
0
−1
β̃ = Φ1 Λ−1
1 Φ1 X Y = Φ1 Λ1 Φ1 X X β̂
0
0
0
0
0
0
−1
= Φ1 Λ−1
1 Φ1 Φ1 Λ1 Φ1 β̂ + Φ1 Λ1 Φ1 Φ2 Λ2 Φ2 β̂
= Φ1 Λ−1
1 Φ1 Φ1 Λ1 Φ1 β̂
0
= Φ1 Φ1 β̂.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
5Ÿ2. E(β̃) = Φ1 Φ1 β. =•‡r < p, ̤© OÒ´k
y²: •I5¿ E(β̂) = β=Œ.
5Ÿ3. kβ̃k ≤ kβ̂k, =̤©
O´Ø
O.
y²: -I˜ = diag(Ir , 0), KdΦ ½Â•
0
˜ 0.
Φ1 Φ1 = ΦIΦ
l k
0
0
Tianxiao Pang
1nÙ
0
˜ β̂k = kIΦ
˜ β̂k ≤ kΦ β̂k = kβ̂k.
kβ̃k = kΦIΦ
£8ëê
O
O.
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
½n (3.9.1)
©gCþ•3v î- õ- ‚5ž, · ÀJ 3 Ì
¤©‡êŒ¦Ì¤© O'•
¦ Ok
þ•Ø , =
MSE(β̃) < MSE(β̂).
0
y²: b X X
JwÑ
p − r‡A
Šλr+1 , · · · , λp é Cu". Ø
α̂1
MSE(β̃) = MSE
0
i
h
2
α̂1
α̂1
+ E
−α
= tr Cov
0
0
2
= σ 2 tr(Λ−1
1 ) + kα2 k .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
Ï•
−1
2
MSE(β̂) = σ 2 tr(Λ−1 ) = σ 2 tr(Λ−1
1 ) + σ tr(Λ2 ),
¤±
MSE(β̃) = MSE(β̂) + (kα2 k2 − σ 2 tr(Λ−1
2 )).
u´
MSE(β̃) < MSE(β̂)
…=
2
kα2 k2 < σ 2 tr(Λ−1
2 )=σ
p
X
1
.
λi
(3.9.4)
i=r+1
õ- ‚5v î- žÿ, λr+1 , · · · , λp Œ±¿©
ÏdþªmàŒ±v Œ¦ Ø ª(3.9.4)¤á.
Tianxiao Pang
1nÙ
£8ëê
O
Cu".
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
0
Ï•α2 = Φ2 β, (3.9.4)Œ •
β 0
σ
Φ2 Φ2
0
β σ
< tr(Λ−1
2 ),
(3.9.5)
ùÒ´`, βÚσ÷v(3.9.5)ž, ̤© Oâ'•
¦ O
k
þ•Ø . (3.9.5)L« ëê˜m¥(Àβ/σ•ëê)˜
‡¥%3 : ý¥. u´l(3.9.5)Œ Xe(Ø:
0
(a) é ½ ëêβÚσ 2 , X X
p − r‡A Š'
¤© O'•
¦ Ok
þ•Ø .
0
(b) 鉽 X X, = ½ Λ2 , éƒé
'•
¦ Ok
þ•Ø .
Tianxiao Pang
1nÙ
£8ëê
ž, Ì
β/σ, ̤© O
O
LSE LSE
5Ÿ
åLSE
̤©‡êr
ä BCC† GLSE õ- ‚5 * O PC
À
O Stein
:
(1)Ñ A Š Cu" @ ̤©.
(2)ÀJr¦ cr‡A ŠƒÚ3p‡A ŠoÚ¥¤Ó '
~(¡•\O zÇ)ˆ ýk‰½ Š. žX, ÀJ•
r¦
Pr
λi
Pi=1
> 0.85.
p
i=1 λi
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
~3.9.1 (Y~3.8.1){I²Lêâ©Û¯K.
k?1~1 •
¦ O†õ- ‚5 ä:
yx=read.table(”ex p68 data.txt”)
x1=yx[, 1]
x2=yx[, 2]
x3=yx[, 3]
y=yx[, 4]
economy=data.frame(x1,x2,x3,y)
economy
lm.sol=lm(y∼x1+x2+x3,data=economy)
summary(lm.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
X=cbind(x1,x2,x3)
rho=cor(X)
rho
library(DAAG)
vif(lm.sol)
eigen(rho)
kappa(rho,exact=TRUE)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
•žØõ- ‚5 K•, ‰Ì¤©£8:
economy.pr=princomp(∼x1+x2+x3,data=economy,cor=TRUE)
summary(economy.pr,loadings=TRUE)
1n‡A Šλ3 = 0.05187378392 = 0.00269 ≈ 0.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
éA n‡IO
O Stein
zA •þ•
0
φ1 = (−0.706, 0, −0.707) ,
0
φ2 = (0, −0.999, 0) ,
0
φ3 = (0.707, 0, −0.707) .
n‡Ì¤©©O•
z1 = −0.706x1 − 0.707x3 ,
z2 = −0.999x2 ,
z3 = 0.707x1 − 0.707x3 .
Ï•1˜‡A Š \O zÇ•0.666385 ≤ 0.85, cü‡A
Š \O zÇ0.9991030 > 0.85, ¤±·‚í 1n‡Ì¤©,
• 3cü‡Ì¤©.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
OŽÌ¤© ©(=#Cþ *ÿŠ•þ):
pre=predict(economy.pr)
pre
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
?1̤© O:
z1=pre[, 1];z2=pre[, 2]
yxs=scale(yx)
y=yxs[, 4]
mydata=data.frame(y,z1,z2)
pc.sol=lm(y∼0+z1+z2,data=mydata)
summary(pc.sol)
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
̤©£8•§:
û = −0.65787z1 − 0.1824z2
= −0.65787 × (−0.706x∗1 − 0.707x∗3 ) − 0.1824 × (−0.999x∗2 )
= 0.46446x∗1 + 0.18222x∗2 + 0.46511x∗3 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
5¿±þ•IOzCþ £8•§. =z• ©Cþ £8•§,
x1 − 194.591
x2 − 3.300
ŷ − 21.891
= 0.46446 ×
+ 0.18222 ×
4.544
30.000
1.649
x3 − 139.736
+0.46511 ×
,
20.634
=
ŷ = −7.768 + 0.070x1 + 0.502x2 + 0.102x3 .
ùp y, x1 , x2 , x3 L« ©Cþ.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
eL‰Ñ •
¦ O!*
•{
•
¦ O
* O(k = 0.04)
̤© O(r = 2)
O Stein
OṲ́© O ' :
~ê‘
-10.128
-8.655
-7.768
x1
-0.051
0.063
0.070
x2
0.587
0.587
0.502
x3
0.287
0.117
0.102
o 5`, * OṲ́© O' ƒC. ‹•
¦ Oƒ',
* OṲ́© Oўؽ…) õ- ‚5¤‘5 K•,
¤±x1 £8Xê ÎÒ•u) Cz.
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
SteinØ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
O Stein
O
* OṲ́© Oùü«k
O´é•
¦ Oβ̂• :
Š·
Ø . ¦‚éëê Oβ̂ ˆ©þŠ ´šþ!Ø .
!òÄuStein Ø gŽ?Ø•
¦ Oβ̂ þ!Ø
O, T O´Stein31955cJÑ5 .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
½Â (Stein
ä BCC† GLSE õ- ‚5 * O PC
O Stein
O)
3‚5£8 .(3.1.5)¥, Ù£8Xêβ
0
0
(X X)−1 X Y . ·‚¡
•
¦ O•β̂ =
β̂s (c) = cβ̂
•Stein O, Ù¥0 ≤ c ≤ 1 ¡•Ø Xê.
w,, β̂s (c)´éβ̂ z˜©þŠ ˜
˜«þ!Ø
O.
Tianxiao Pang
1nÙ
Ø , ¤±Stein O´
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
SteinØ
O 5Ÿ :
5Ÿ1.
c 6= 1ž, β̂s (c)´β̂
5Ÿ2. •30 < c < 1, ¦
y²: β̂s (c) þ•Ø
k !Ø
O Stein
O.
MSE(β̂s (c)) < MSE(β̂).
•
MSE(β̂s (c)) = tr[Cov(β̂s (c))] + kE(β̂s (c)) − βk2
0
= c2 σ 2 tr[(X X)−1 ] + (c − 1)2 kβk2
p
X
2 2
2
2
= c σ
λ−1
i + (c − 1) kβk
i=1
∆
= g(c).
ùp, • žØþj K•, ·‚b gCþÚÏCþþ®IO
z, Ïd OÝ X•n × pÝ .
Tianxiao Pang
1nÙ
£8ëê
O
LSE LSE
5Ÿ
åLSE
ä BCC† GLSE õ- ‚5 * O PC
ég(c)'uc¦ ¿-Ù
c∗ =
σ
O Stein
u"Œ) c •`Š•
kβk2
< 1.
−1
2
i=1 λi + kβk
(3.10.1)
Pp
2
dug(c)'uc
ê u
2σ 2
p
X
2
λ−1
i + 2kβk > 0,
i=1
Ïdg(c) = MSE(β̂s (c))3c∗ ?ˆ • , ¿… c∗ ≤ c < 1ž,
kMSE(β̂s (c)) < MSE(β̂).
Tianxiao Pang
1nÙ
£8ëê
O
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )