Predicting Solar Generation from Weather Forecasts
Advisor: Professor Arye Nehorai
Chenlin Wu, Yuhan Lou
Department of Electrical and Systems Engineering
Principal Component Analysis (PCA)
Kernel Trick for SVR
Background

Smart grid: increasing the contribution of renewable in grid
energy

Solar generation: intermittent and non-dispatchable
The kernel trick is a way of mapping observations from a
general set S (Input space) into an inner product space V
(high dimensional feature space)
Φ: ℝ𝑛 → ℝ𝑚
Goals
Creating automatic prediction models
𝑓(𝑥)

Predicting future solar power intensity given weather forecasts
=
𝑚≫𝑛
𝑖
𝛼𝑖 − 𝛼𝑖 ∗ 𝑘(𝑥𝑖 , 𝑥)
Experiments
+𝑏
where 𝑘 𝑥𝑖 , 𝑥 = ϕ 𝑥𝑖 , ϕ 𝑥 .

NREL National Solar Radiation Database 1991-2010

Hourly weather and solar intensity data for 20 years
Gaussian Processes (GP)

Station: ST LOUIS LAMBERT INT’L ARPT, MO
GP regression model:
Input: (combination of 9 weather metrics)
𝑦𝑖 = 𝑓 𝑥𝑖 + 𝜀𝑖 , where noise 𝜀𝑖 ~𝑁(0, 𝜎 2 𝐼)

Date, Time , Opaque Sky Cover, Dry-bulb Temperature, Dew-point
Temperature, Relative Humidity, Station Pressure,Wind Speed, Liquid
Precipitation Depth
Output :

Amount of solar radiation (Wh/m2) received in a collimated beam on a
surface normal to the sun
Methods


In our research, regression is used to learn a mapping from
some input space of n-dimensional vectors to an output space of
real-valued targets
We apply different regression methods including:

Linear least squares regression

Support vector regression (SVR) using multiple kernel functions

Gaussian processes
Linear Model
𝑦 = 𝑓 X = 𝑋𝑇 𝑎 + 𝑒
where 𝑦 ∈ ℝ𝑛 : measurement (solar intensity)
X ∈ ℝ𝑛×𝑝+1 : each row is a p-dimensional input
𝐴 ∈ ℝ𝑝+1 : unknown coefficient
𝑒 ∈ ℝ𝑛 : random noise
Loss function(Square error): 𝑦 − 𝑦 2 = 𝑦 − 𝑋 𝑇 𝑎 2
Support Vector Regression (SVR)
Given training data {(𝒙𝟏 , 𝑦1 ), (𝒙𝟐 , 𝑦2 )…(𝒙𝒏 , 𝑦𝑛 )
Linear SVR Model：
𝑓 𝒙 = 𝒘, 𝒙 + 𝑏 =
minimize
1
2
𝒘 2 +𝐶
Applying PCA to remove redundant information
The graph shows the MSE with different input
dimensions. The feature set with 8 dimensions performs
the best with the lowest test error.
And as long as we keep more than 5 principle
components, the errors are lower than linear regression
𝑖
Data Source

Such as: Temperature & Time of the day
𝛼𝑖 − 𝛼𝑖 ∗ ϕ 𝑥𝑖
𝜔=


Some weather metrics correlate strongly
𝑤𝑇𝒙
+𝑏
∗
(ξ
+ξ
𝑖 𝑖 𝑖 )
𝑦𝑖 − 𝑓(𝑥𝑖 ) ≤ 𝜀 + ξ𝑖
∗
𝑓(𝑥
)
−
𝑦
≤
𝜀
+
ξ
subject to
𝑖
𝑖
𝑖
ξ𝑖 , ξ𝑖 ∗ ≥ 0
Loss function: (epsilon intensive)
0
𝑖𝑓 ξ ≤ 𝜀
ξ𝜀≔
ξ − 𝜀 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.

Predictions are made by proposed methods

20% of data is used to train & 10% of the data is used to test
Linear regression

Assume a zero mean GP prior distribution
MSE is used to evaluate the
result of regression. Followings
are the prediction errors of the 3
different methods:

over inference functions 𝑓 ∙ . In particular,
Linear Regression
𝑓 𝑥 1 , . . . , 𝑓 𝑥 𝑛 ~𝑁 0, 𝐾 , 𝐾𝑝,𝑞 = 𝐶𝑜𝑣(𝑓 𝑥 𝑝 , 𝑓 𝑥 𝑞 ) = 𝐾(𝑥 𝑝 , 𝑥 𝑞 )
215.7884
To make predictions 𝑦 ∗ at test points 𝑋 ∗ , where 𝑦 ∗ = 𝑓 𝑋 ∗ + ε
𝑓 ∗ : 𝑓𝑓∗
𝐾 𝑋, 𝑋
~ 𝑁 0,
𝐾 𝑋∗, 𝑋
∗
It follows that p 𝑦 𝐷, 𝑋
𝑋, 𝑋 ∗
𝐾
𝐾 𝑋∗, 𝑋∗
∗
,
𝜀
𝜀∗
~ 𝑁 0,
𝜎2𝐼

130.1537
0
𝜎2𝐼
0
−1

SVM regression
= 𝑁(𝜇, Σ)
where 𝜇 = 𝐾 𝑋, 𝑋 ∗ [𝐾(𝑋, 𝑋) + 𝜎 2 𝐼]−1 𝑦
Σ = 𝐾 𝑋 ∗ , 𝑋 ∗ − 𝐾 𝑋, 𝑋 ∗ 𝐾 𝑋, 𝑋 + 𝜎 2 𝐼
SVR
122.9167

𝐾 𝑋∗, 𝑋 .
SPGP
Followings are 24-hour prediction
Sparse Pseudo-input GP (SPGP)
GPs are prohibitive for large data sets due to the inversion
of the covariance matrix.
Consider a model parameterized by a pseudo data set 𝐷 of
size 𝑚 ≪ 𝑛, where n is the number of real data points.
Reduce training cost from 𝑂 𝑛3 to 𝑂 𝑚2 𝑛 , and prediction
cost from 𝑂 𝑛2 to 𝑂 𝑚2
Pseudo data set 𝐷: 𝑿 = 𝒙𝑖 𝑖=1…𝑚 , 𝒇 = 𝑓𝑖 𝑖=1…𝑚
SPGP regression
Prior on Pseudo targets: 𝑝 𝒇 𝑿 = 𝑁(0, 𝐾𝑀 )
Likelihood:
𝑇
−1
𝑝 𝑦 𝒙, 𝑿, 𝒇 = 𝑁 𝐾𝒙 𝐾𝑀 𝒇,
𝑇
𝐾𝒙𝒙 − 𝐾𝒙 𝐾𝑀
−1
LR
SVR
GP
𝐾𝒙 + 𝜎 2
Posterior distribution over 𝒇 :
𝑝 𝒇 𝐷, 𝑿 = 𝑁 𝐾𝑀 𝑄𝑀 −1 𝐾𝑀𝑁 (𝜦 + 𝜎 2 𝑰)−1 𝒚, 𝐾𝑀 𝑄𝑀 −1 𝐾𝑀
where 𝑄𝑀 = 𝐾𝑀 + 𝐾𝑀𝑁 (𝜦 + 𝜎 2 𝑰)−1 𝐾𝑁𝑀
Given new input 𝑥 ∗ , the predictive distribution:
𝑝 𝑦 ∗ 𝐷, 𝑋 ∗ = 𝑑 𝒇𝑝 𝑦 ∗ 𝒙∗ , 𝑿, 𝒇 𝑝 𝒇 𝐷, 𝑿 = 𝑁 𝜇∗ , Σ ∗
−1
𝑇
∗
where 𝜇 = 𝐾∗ 𝑄𝑀 𝐾𝑀𝑁 (𝜦 + 𝜎 2 𝑰)−1 𝒚
Σ ∗ = 𝐾∗∗ − 𝐾∗ 𝑇 𝐾𝑀 −1 − 𝑄𝑀 −1 𝐾∗ + 𝜎 2
Predicting Error
191.5258
93.2988
90.2835
Conclusions

Using machine learning to automatically model the function of
predicting solar generation from weather forecast lead to a
acceptable result

Gaussian processes achieved lowest error among all the methods