International Journal of Mechanical Engineering and Technology (IJMET)
Volume 10, Issue 01, January 2019, pp. 1026–1032, Article ID: IJMET_10_01_106
Available online at http://www.iaeme.com/ijmet/issues.asp?JType=IJMET&VType=10&IType=1
ISSN Print: 0976-6340 and ISSN Online: 0976-6359
© IAEME Publication
Scopus Indexed
SMOG PREDICTION MODEL USING TIME
SERIES WITH LONG-SHORT TERM MEMORY
S Geetha
Assistant Professor (Sr. Grade), Department of Computer Applications,
Mepco Schlenk Engineering College, India
L Prasika
Assistant Professor, Department of Computer Applications,
Mepco Schlenk Engineering College, India
ABSTRACT
Smog triggered due to air pollutants and fog. Deep Learning techniques are
applied to predict the smog severity. This paper presents deep learning based
predictive model for various air pollutants (NO2, NOx, CO, SO2, O3, PM2.5, PM10) for
metropolitan area Air pollution dataset. Central Pollution Control Board (CPCB) is
monitoring air, water, waste, etc through nationwide programs. Through National Air
Quality Monitoring program, the primary and secondary air pollutants are captured
and available in online. In this paper, two traditional predictive models along with
deep learning technique Long-Short Term Memory (LSTM) are used for predicting the
air pollutants. Before training the model, the missing values and noise in dataset were
imputed using mean value. Then, the models are built with LR, ARIMA, and LSTM.
Finally, the models performance is measured using Mean Absolute Error and Root
Mean Square Error (RMSE). LSTM performed better than LR and ARIMA.
Key words: Smog; Linear Regression; Autoregressive Integrate Moving Average;
Long Short Term Memory; Predictive Model; NO2; NOx; CO; SO2; O3; PM2.5;
PM10;.
Cite this Article: S Geetha, L Prasika, Smog Prediction Model using Time Series
with Long-Short Term Memory, International Journal of Mechanical Engineering
and Technology 10(1), 2019, pp. 1026–1032.
http://www.iaeme.com/IJMET/issues.asp?JType=IJMET&VType=10&IType=1
1. INTRODUCTION
Currently, Globe is getting alarmed by smog. Air pollution with fog leading to Smog. Due to
Ground level ozone and fine particles are released in the air leads to smog. Air is getting
polluted aggressively in recent years due to vehicle emissions, industries, fuel and stubble
burning. The primary air pollutants (CO, NOx, PM10, PM2.5, and SO2), and secondary air
pollutants (NO2 and O3) assorted with the meteorological factors (Temperature, Humidity,
Wind) forming the smog. With substantial evidences, Particulate Matter and O3 with NO2 are
http://www.iaeme.com/IJMET/index.asp
1026
editor@iaeme.com
Smog Prediction Model using Time Series with Long-Short Term Memory
identified as major risk factors for smog. Smog is creating the top most consequences in an
environment and health. It may lead to Smog disasters like London‟s great smog of 1952 [1].
As per World Health Organization records, smog hanging over cities are causing
approximately seven million mortalities each year from stroke, lung cancer, cardiovascular
disease, and acute respiratory infections [2]. Smog causes poor visibility which is triggering
the percentage of road accidents. Smog has also affected the investment in industries [3].
Hence, the Prediction of smog becomes essential to enrich the quality of human life and to
save living beings. Location to location and time to time, all pollutants (NO2, CO, NOx, SO2,
O3, PM2.5, PM10) are spreading and changing. There are many traditional prediction models
such as Logistic Regression, Time series analysis, Decision Trees, and latest ANN, Machine
Learning Algorithms are available for predicting air pollutions. However, Deep learning is the
latest technology applied in various domains to bring automation and accuracy in work. A
surfeit of studies has been done using Deep learning to create prediction model especially
with time series data. This paper derived to bring traditional techniques Linear Regression and
Autoregressive Integrated Moving Average (ARIMA) along with deep learning technique
LSTM to deal with time series dataset and to create prediction model for smog.
2. RELATED WORKS
IBM is using real-time air quality data of Beijing with machine learning techniques to monitor
and decrease the airborne pollutant. Air pollution trends are detected fir Los Angeles station
using time-series analysis [4]. Extreme learning machine (ELM) is used for efficiently predict
the concentration of air pollutants [5]. Principal component regression technique used to
predict air pollution with respect to daily Air Quality Index [6]. A Time-Series prediction
done using ANN and ARIMA models [7]. MLP is applied for predicting the pollution levels
using pollutants SO2, PM10, CO and meteorological variables temperature, wind direction,
pressure, day and night temperature, relative humidity and wind speed, and obtained higher
error for SO2 than other pollutants [8]. In LSTM, the output of hidden layers is controlled
using memory cells and gates [9]. Thus, we developed the model using LR, ARIMA, and
LSTM and compared the performance in this paper.
3. METHODS AND MATERIALS
3.1. Linear Regression (LR)
Linear Regression is used to show the relationship between independent variable and
dependent variables. The LR model is defined as
, where i=1, 2, …, n
(1)
Where Y is the dependent variable, β0, β1, , βk are (k+1) parameters, X1, , Xk are kindependent variables, ε are the independent, identically and Gaussian-distributed errors, and
n is the number of observations. The parameters are estimated by the ordinary least square
method.
3.2. Autoregressive Integrated Moving Average (ARIMA)
The ARIMA method consists of three key steps as tentative identification, parameter
estimation and diagnostic checking [10]. The ARIMA consists of the components autoregressive (AR), integrated component, and moving average (MA). Effects pf previous
observations are represented by Auto regressive component. Trends and seasonality is
represented by Integrated component. Moving Average represents remaining effects of
previous random shocks or error. The general mathematical form of ARIMA model is
represented in Eqs. (2–6) [11].
http://www.iaeme.com/IJMET/index.asp
1027
editor@iaeme.com
S Geetha, L Prasika
(2)
When
Similarly,
when: t = t – p,
then
(3)
After Substituting,
(4)
(5)
Finally,
(6)
3.3. Long-Short Term Memory Networks (LSTM)
LSTM Networks is one kind of Recurrent Neural Network (RNN). To overcome the problem
of failing to store previous data in memory cell, LSTM over RNN is emerged. In LSTM
Network, memory cells are recurrently connected, with three vital gate units: Input Gate,
Output Gate, and Forget Gate. The input gate controls the maximum flow of values into the
cell, the forget gate controls how long the values can be in the cell and the output gate
controls the cell output by activation. The Fig. 1 shows the traditional architecture of LSTM.
LSTM is using activation function to determine whether the memory cell is containing the
value or forgotten [12], with the help of the following formula.
[
]
(7)
Input gate determines which values used for updating and tanh is used to compute ̃ . The
formulas are
[
]
(8)
̃
[
]
(9)
To update the old cell value, the following formula is used.
̃
(10)
Finally, to get the output of the model, the following formulas are used.
[
]
(11)
(12)
http://www.iaeme.com/IJMET/index.asp
1028
editor@iaeme.com
Smog Prediction Model using Time Series with Long-Short Term Memory
Figure 1 LSTM Architecture & Proposed Methodology with LSTM
Here, x is the input, t is the time, h is the hidden layer output, c is the cell output state, ft, it,
and ot are the outputs of the forget gate, input gate and output gate. Sigmoid activation
function is used in three gates.
Because of the advantages of LSTM, we use LSTM as the basic part of our model to
predict the smog and the performance through the abstract features generated with the help of
LSTM layers. In this study, the computations are done using Python 3.6.
3.4. Proposed Methodology
The flow diagram of the proposed methodology, is presented in Fig 1. Initially, raw data is
pre-processed as it contains missing values and noise. Pre-processed data is split into training
set and testing set. In training phase, the training data set is used to train the LSTM model.
The best fit LSTM model is identified with the help of Root Mean Square Error in testing
phase.World Health Organization provides Air Quality Standards as shown in Table 1.
Table 1 WHO‟s Air Quality Standards
Time CO NOx O3 SO2 NO2 PM2.5 PM10
10 min - 500
1 hr
30
200
8 hrs
- 100 Day
20
25
50
Annual 30
40
10
20
3
Units: μg/m
http://www.iaeme.com/IJMET/index.asp
1029
editor@iaeme.com
S Geetha, L Prasika
3.5. Evaluation Measures
Performance and accuracy of model is measured using Root Mean Square Error (RMSE),
Mean Absolute Error (MAE) are used to evaluate the results. Equations for RMSE and MAE
are as follows.
R
√ ∑
̃
where
is the actual value,
observations.
∑
|
(13)
̃ is the predicted value and n is the total number of
̃|
(14)
where n is the total number of predictions, y is the predicted value.
The air pollutants data is used to predict the smog through LR, ARIMA, and LSTM.
3.6. Dataset Preparation
The time series plots of metropolitan area Air quality data from the CPCB shows that the air
quality is starting to declining at a consistent rate. Pollutant concentrations of SO2, NOx, O3,
CO, PM2.5, PM10 and NO2 below the limits specified by the Central Pollution Control Board
(CPCB). The daily average air pollutant data is collected from Jan 2015 – May 2018. Dataset
contains some missing values. The missing values are imputed with the help of mean.
Meteorological data also collected which contains lot of missing values. So, those data are not
used for prediction model. Time series data is plotted and it showed that PM 2.5, PM10, NOx,
O3, PM2.5 are stationary in all cases and NO2, and SO2, data are nonstationary in all cases.
4. EXPERIMENTS & RESULTS
4.1. Linear Regression Model
The forecasting of air pollutants SO2, NOx, O3, CO, PM2.5, PM10 and NO2 with actual and
predicted comparison using Linear Regression.
4.2. Autoregressive Integrated Moving Average (ARIMA) Model
The forecasting of air pollutants SO2, NOx, O3, CO, PM2.5, PM10 and NO2 with actual and
predicted comparison using ARIMA. The ARIMA Model parameters (p, d, q) are important
to predict. Here, p is auto-regressive terms and q is moving average terms d is the differences.
ARIMA model is created for O3, CO, PM2.5, PM10 with the order (p, d, q as 0,1,1), SO2, NOx,
with the order (p, d, q as 1,1,1) and NO2 with the order (p, d, q as 1,1,2). The given orders
helped to create fitted ARIMA model for all the air pollutants.
4.3. Long-Short Term Memory Networks (LSTM) Model
The forecasting of air pollutants SO2, NOx, O3, CO, PM2.5, PM10 and NO2 with actual and
predicted comparison using LSTM. LSTM model created with fitted hyper parameters (look
back, epochs, neurons). No. of neurons lies between 10 - 120, No. of epochs lies between 50 –
1000 and look back lies between 1 – 3. Almost, the look back 3 has given better result to most
of the air pollutants. Batch size varies from 10 – 100. The model created using „linear‟
activation function and „RMSprop‟ optimizer. The Fig. 2 shows only the NO2 predictions as
sample.
http://www.iaeme.com/IJMET/index.asp
1030
editor@iaeme.com
Smog Prediction Model using Time Series with Long-Short Term Memory
Figure.2 Linear Regression, ARIMA, LSTM Prediction for NO2
The Linear Regression, ARIMA and LSTM model are generated for all the air pollutants.
The models are evaluated with Mean Absolute Error and Root Mean Square Error.
5. EVALUATION
The models are created using three approaches and evaluated with RMSE and MAE. The
LSTM-RMSE and LSTM-MAE values for PM10 is 0.049 and 0.039, for PM2.5 is 0.068 and
0.005, for CO is 0.096 and 0.009, for NO2 is 0.149 and 0.116, for NOx is 0.049 and 0.039, for
SO2 is 0.011 and 0.009, for O3 is 0.098 and 0.071, which is very lower than the RMSE and
MAE value of Linear Regression and ARIMA.
Among, three models, LSTM is predicting the values more accurate than other two
models.
http://www.iaeme.com/IJMET/index.asp
1031
editor@iaeme.com
S Geetha, L Prasika
Table 2:MAE & RMSE values for LR, ARIMA and LSTM on all Air Pollutants
PM10
MAE RMSE
LR
42.64 85.10
ARIMA 78.44 98.42
LSTM 0.039 0.049
Model
PM2.5
MAE RMSE
41.23 62.37
45.37 61.89
0.005 0.068
CO
MAE RMSE
0.66 0.88
0.57 0.78
0.009 0.096
NO2
MAE RMSE
16.28 24.33
42.019 54.758
0.116 0.149
NOx
SO2
MAE RMSE MAE RMSE
92.30 127.65 1.79 3.64
39.61 66.585 5.5
6.87
0.039 0.049 0.009 0.011
O3
MAE RMSE
8.57 13.15
8.67 12.11
0.071 0.098
All the developed models have been trained and validated with the daily average data of
air pollutants. The RMSE and MAE computations between observed and predicted SO2, NOx,
O3, CO, PM2.5, PM10 and NO2 are given in Table 2. The values of RMSE is as close to 0.0 for
all models. Overall, the performance of the LSTM model is found satisfactory. Thus, the
developed model can be used for predicting smog over urban areas.
6. CONCLUSIONS
The proposed model achieves a significant progress on the current state-of-the-art for time
series regression using deep neural networks. This LSTM model shown effective predictions
compared with the conventional LR and ARIMA methods. However, the current result is on
univariate SO2, NOx, O3, CO, PM2.5, PM10 and NO2 data using the LSTM model. Hence,
further research will be required on the LSTM for predicting with multivariate data.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
D.L. Davis, “A look back at the London smog of 1952 and the half century since”,
Environ. Health Perspect, 110 (12) A734–A735, 2002.
Sundeep Mishra, “Is smog innocuous? Air pollution and cardiovascular disease”, Indian
Heart Journal, 2017.
Jingbo Luo, “How does smog affect firms‟ investment behavior? A natural experiment
based on a sudden surge in the PM2.5 index”, China Journal of Accounting Research,
2017.
P.H. Merz, L.J. Painter, P.R. Ryason, “Aerometric data analysis time series analysis and
forecast and an atmospheric smog diagram”, Atmos. Environ., 6 (5) (1972) 319–342,
1967.
Jiangshe Zhang, Weifu Ding, “Prediction of Air Pollutants Concentration Based on an
Extreme Learning Machine: The Case of Hong Kong”, Int. J. Environ. Res. Public
Health, 14, 114.
Anikender Kumar, Pramila Goyal, “Forecasting of air quality in Delhi using principal
component regression technique”, Atmospheric Pollution Research,Volume 2, Issue 4,
Pages 436-444, October 2011.
Khashei, M., Bijari, M, “A novel hybridization of artificial neural networks and ARIMA
models for time series forecasting”, Applied Soft Computing 11(2), 2664–2675,2011.
Kurt, B. Gulbagci, F. Karaca, O. Alagha, “An online air pollution forecasting system
using neural networks”, Environment International, 34, pp. 592-598, 2008.
Sepp Hochreiter and Jurgen Schmidhuber, “Long short-term memory”, Neural
computation, 9(8):1735–1780,1997.
Hanke, J.E., Wichern, D.W, “Business forecasting”, Pearson United States of America,
9thEd., 2008.
Naveen V, Anu N, “Time Series Analysis to Forecast Air Quality Indices in
Thiruvananthapuram District, Kerala, India”, J Eng Res Appl, 7(6):66–84, 2017.
Xike Zhang, et al., “A Novel Hybrid Data-Driven Model for Daily Land Surface
Temperature Forecasting Using Long Short-Term Memory Neural Network Based on
Ensemble Empirical Mode Decomposition”, International Journal of Environmental
Research and Public Health, 15, 1032,2018.
http://www.iaeme.com/IJMET/index.asp
1032
editor@iaeme.com