GB – 304 – The PRESS statistic (pg. 282)
PRESS is the Predicted REsidual Sum of Squares
-- An Out-of-sample validation statistic where you hold
out just one point at a time
n
2
ˆ
PRESS   ( yi  y(i ) )
i 1
-- Hold out the ith point
-- Find the regression eqtn using the remaining n –1 points
-- Predict the held out value to get yˆ ( i )
-- Compare to the value yi
-- Do this for all points, square the differences and sum
them up to get the PRESS statistic
-- Calculate this statistic for several candidate models; the
one with the smallest PRESS is considered the best.
Alternatively, an easier formula to use is…
 eˆi 

PRESS   
i 1  1  hii 
n
2