Time Series Prediction
Using
Support Vector Machine: A Survey
By
Ma Yongning
Outline

Introduction to time series prediction

Classic techniques

Introduction to SVM

Variants of SVM

Hybrid techniques

Application

Conclusion
Time Series Prediction
The objective of time series prediction is to find a
function f(x) such that , the predicted value of the time
series at a future point in time is unbiased and
consistent.
Goal of time series prediction is to estimate some
future value based on current and past data samples
ANN Approach –
Autoregression Filter



Linear, stationary process
The current term of the series can be estimated by a
linear weighted sum of previous terms in the series
A number of techniques exist for computing AR
coefficients. The main two categories are least squares
and Burg method
ANN Approach – Kalman Filter


Also known as linear quadratic estimation
(LQE), is an algorithm which uses a series
of measurements observed over time,
containing noise (random variations) and
other inaccuracies, and produces
estimates of unknown variables
Assume linear, stationary process, and
model is known
Introduction to SVM

A technology used to find the optimal separating
hyperplane
f(x)= wx + b or f(x)= wg(x) + b



Linear SVM developed by Vapnik and Chervonenkis in
1960s
Non-Linear SVM developed in 1990s by applying the
kernel trick
Unbiased if b = 0
Computation of Separating
Hyperplane

Assume f(x)= wx + b (Linear Case)

Task: Maximize the margin M = 2/||w||

Which means to minimize ||w||

Also, we allow some error ξ
Some Math

Our problem now is

Subject to

And use convex optimization to solve for f(x)
The Kernel Trick



In the linear case we have K(x, y) = <x, y>. If the data
points are not linearly separable, we construct
Such that
Which means K(x, y) can be expressed in dot product of
terms in inner product space V (e.g. Hilbert Space)
whose dimension is dim(V), therefore separable in V.
An Simple Example


Common kernel functions are polynomial or Gaussian
To see how they transform input vectors into higher
dimension, consider quadratic kernel

Which corresponds to feature

The dimension of the feature is O(n^2):
Non-Linear Decision Boundary
Least-Square SVM (Among
other Variants)


Express the error term as in linear regression:
Then problem can be solved by letting partial derivatives
of Lagrangian to be zero, which eliminates w and b, and
leads to a linear system of a and b, that can be directly
solved. LS-SVM sacrifices a bit accuracy for efficiency
Hybrid Approaches

Fuzzy SVM (blurred decision boundaries)

SOM and SVM (SOM used to cluster data first)



GA and SVM (used to optimize the free variables in
SVM; a broad type will be adaptive SVM -ASVM, where
free variables are subject to change in the training
process)
Specific to time series: use ARIMA to predict linear
region
Smooth data first by SOM, Kalman filter. But in the case
of SOM, it worsened the performance
Application of SVM for Time
Series in Different Industries

Financial Data Time Series Prediction



Use of SVR to predict five specific
financial time series sources including
the S&P 500 and several foreign bond
indices.
The SVR significantly outperformed the
BP NN because of it’s ability to
appropriately fit the data.
A hybrid system of using SOM
combined with SVR yielded not only
better prediction performance but also
superior convergence speed.
Application of SVM for Time
Series in Different Industries

General Business Appications





Electricity Price Forecasting
Credit Rating Analysis
Customer “Churning”- Auto Insurance
Market Prediction
Financial Failure of Dotcoms – Financial
Analysis using 24 financial ratios
Production Value Prediction of the
Taiwanese Machinery Industry
Application of SVM for Time
Series in Different Industries

Environmental Parameter Estimation
Using SVR

Used to predict parameters such as
rainfall estimation and detection,
weather forecasting, short term air
quality(environmental pollution such as
nitrogen oxides) etc…
Financial Data Time Series
Prediction

Electric Utility Load Forecasting
Applications Using SVR.

Forecasting of electric power
consumption demands by consumers,
which is a non-linear prediction
problem.

(Factors influencing usage for example
are “holiday time periods”,
weather(temperature and humidity),
electricity pricing etc…
Financial Data Time Series
Prediction

Machine Reliability Forecasting
Applications



The prediction of machine reliability is a non-linear
problem.
Example is to predict the “period reliability ration”
for the automotive industry based on time series
data containing vehicle damage incidents and the
number of damages repaired.
Another example is the use of SVR to predict
engine failure in both the repair and design process
of mechanical engines. Data used as input was the
engine age at the time of unscheduled
maintenance action per maintenance period and
output was the predicted engine age of the next
unscheduled maintenance action.
Financial Data Time Series
Prediction

Control System and Signal Processing
Applications.







Mobile Position Tracking
Internet Flow Control
Adaptive Inverse Disturbance
Cancelling
Narrowband Interference Suppression
Antenna Beamforming
Elevator Traffic Flow Prediction
Dynamically Tuned Gyroscope Drift
Modeling
Financial Data Time Series
Prediction

Miscellaneous Applications

Biological Neuron Application(Australian
Crayfish)

Kalman Filtering Method of Switching Dynamics
Associated with Unsupervised Segmentation

Natural Gas Load Forecasting(using weather
related factors such as temperature, day of week
and holidays etc…)

Transportation Travel Time Estimation

Use of Particle Swarm Optimization in conjuncture
with SVR.
Conclusion



Support Vector Machines/Support Vector
Regression(SVR) are powerful learning mechanisms that
have been developed and matured over the last 15
years.
Useful for predicting and forecasting time series for a
myriad of applications.
SVR research continues to be a viable approach in the
prediction of time series data in non-linear systems.