STAT444 29 Jan
araising
January 2024
Cubic Sphere
f (x) = β0 + β1 x + β2 x2 + β3 x3 +
Pk
j=1
βj+3 (x − tj )3 +
Natural Cubic Spline (NCS)
A cubic spline is called natural cubic spline with knots {t1 , ..., tk } if f (x) is
linear when x ∈
/ [t1 , tk ].
That is
(
f0 (x) = a0 + b0 x x < t1
f (x) =
fk (x) = ak + bk x x > tk
# parameters:[]
1. cubic spline: k + 4
2. # constraints: 2+2 = 4 (we require the quadratic and cubic term to be 0)
3. # parameters: k + 4 - 4 = k
Find the expression for NCS:
step 1: f(x) is a cubic spline
Pk
f (x) = β0 + β1 x + β2 x2 + β3 x3 + j=1 βj+3 (x − tj )3 +
step 2: Linear constraints
f (x) is linear when x < t1 =⇒ β2 = β3 = 0 f (x) is linear when x > tk =⇒
Pk
f (x) = β0 + β1 x + j=1 βj+3 (x3 − t3j − 3x2 tj + 3xt2j )
Pk
The cubic term is ( j=1 βj+3 )x3
Pk
The quadratic term is (− j=1 βj+3 tj )x2
Pk
Pk
=⇒
j=1 βj+3 = 0and
j=1 βj+3 tj = 0
Let dj (x) =
(x−tj )3 + −(x−tk )3 +
tk −tj
1
Then the NCS can be expressed as
f (x) =
k
X
βj Nj (x)
j=1
with basis function
N1 (x) = 1
N2 (x) = x
Nj (x) = dj−1 (x) − d1 (x), j = 3, ..., k
An alternative basis is
N1 (x) = 1, N2 (x) = x and
Nj (x) = dj−2 (x) − dk−1 (x), j = 3, ..., k
The fixed-knot splines, such as cubic spline and the NCS are called regression
splines.
Use the NCS as a concrete example
Recall yi = f (xi ) + ϵi
Pk
We can approximate f (x) by j=1 βj Nj (x)
yi ≈
k
X
βj Nj (x) + ϵj
j=1
Design
 matrix

x1
N1
N2
...
Nk
...
Nk (x1 ) More generally, for a p diX =  ... N1 (x1 ) N2 (x1 )
xN N1 (xn )
...
...Nk (xn )
mensional input vector, x, we can consider the following approximation to f (x)
e
e
k
X
βj kj (x)
f (x) ≈
e
e
j=1
where {hj }k1 is a series of basis functions. Then the design matrix X = (hj (xi ))
i is row index and j is column index
Examples 1) hj (x) = xj , j = 1, ..., p 2) hj (x) = log(xj ), i.e., transformae
tion 3) When p = e1, hj (x) = xj , i.e., polynomial
terms 4) When p = 1,
hj (x) = Nj (x), the NCS basis 5) When p = 1, hj (x) = cos(2πjx)
Back to NCS, how to choose k and t1 , ..., tk ?
2