NACLIM Deliverable D32.27
1st Batch-Updates of data sets containing time series of
relevant key ocean quantities
This deliverable consists of collecting and organizing relevant data sets, time series and model outputs from the first
NACLIM period to be used in the model – observation synthesis.
WP No.
3.2
WP title
Impact of Arctic initialization on forecast skill
Work
duration1)
24
Due delivery deadline:
30 October 2014
R= report
Nature of the
deliverable
P= prototype
D= demonstrator
X
O= Other
PU = public
Dissemination
level
X
PP= restricted to other programme participants, including the Commission services
RE= restricted to a group specified by the consortium, including the Commission services
CO= confidential, only for members of the consortium, including the Commission services
1) Work duration = project month
Authors of this report
University of Hamburg (UHAM): Detlef Stammer, Armin Köhl, Reema Agrawal
NACLIM project is financed by the European Commission
through the 7th Framework Programme for Research
Theme 6 Environment
Grant Agreement 308299
Page 1
Index
1. Executive summary ....................................................................................................................................... 3
2. Project objectives .......................................................................................................................................... 3
3. Detailed report on the deliverable .................................................................................................................. 4
4. References .................................................................................................................................................. 15
5. Dissemination and uptake ........................................................................................................................... 15
6. The delivery is delayed:  Yes  No ....................................................................................................... 16
7. Changes made and difficulties encountered, if any ...................................................................................... 16
8. Efforts for this deliverable ............................................................................................................................ 16
9. Sustainability ............................................................................................................................................... 16
10. Dissemination and uptake activities ........................................................................................................... 16
Uptake by the targeted audience ....................................................................................................................... 16
Page 2
1. Executive summary
Work performed
Climate models are tuned using various approaches to bring models closer to real world observations. An
automatic optimization procedure is required to find the best estimate of a set of parameters. Our approach is a
parameter optimization problem in which optimal parameter values are obtained by reducing the model-data
errors (cost function). We use multivariate data assimilation technique for the optimization of parameters of the
THOR coupled assimilation system, which is now established as the CESAM (CEN Earth System Assimilation
Model). A sensitivity analysis of CESAM is carried out. The behaviour of the cost function with respect to
changes in each control parameter is studied, the most sensitive parameters are identified and regional effects
of parameter perturbation are discussed. The results of the sensitivity analysis serve as a guideline for a useful
application of the assimilation procedure and will help interpreting the results. Results are evaluated with
improvements in basic physical properties predicted by model such as surface temperature, precipitation and
net heat flux with respect to observations.
The Simultaneous Perturbation Stochastic Approximation (SPSA) method is used for assimilating annual mean
data (pseudo and ERA-interim reanalysis data). The advantage of using SPSA method is its ease of
implementation, robustness to noise in cost function and cost effectiveness.
Main results achieved
-
Identical twin testing demonstrates that the method reliably finds an estimate of parameters. The errors in
optimized state for example in surface temperature and net heat flux reduced up to 16% and 30%,
respectively.
-
Tuning the model to reanalysis data by adjusting parameters yields a reduction in temperature (all levels)
and net heat flux. The optimized state of CESAM shows an improvement in climate variables such as
temperature at 1000 hPa, sea level pressure, zonal winds at 500 hPa and surface precipitation.
2. Project objectives
With this deliverable, the project has contributed to the achievement of the following objectives (see DOW
Section B.1.1):
No.
1.
2.
3.
4.
5.
6.
Objective
Assessing the predictability and quantifying the uncertainty in forecasts of the North
Atlantic/Arctic Ocean surface state
Assessing the atmospheric predictability related to the North Atlantic/Arctic Ocean
surface state
Monitoring of volume, heat and fresh water transports across key sections in the
North Atlantic
Quantifying the benefit of the different ocean observing system components for the
initialization of decadal climate predictions
Establishing the impact of an Arctic initialization on the forecast skill in the North
Atlantic/European sector
Quantifying the impact of predicted North Atlantic upper ocean state changes on the
Page 3
Yes
X
No
X
X
X
X
X
oceanic ecosystem
Quantifying the impact of predicted North Atlantic upper ocean state changes on
socioeconomic systems in European urban societies
8. Providing recommendations for observational and prediction systems
9. Providing recommendations for predictions of the oceanic ecosystem
10. Disseminating the key results to the climate service community and relevant
endusers/stakeholders
11. Constructing a dataset for sea surface and sea ice surface temperatures in the Arctic
7.
X
X
X
X
X
3. Detailed report on the deliverable
As part of Task 3.2 we performed model optimization though parameter estimation with the THOR coupled
assimilation system, which is now established as the CESAM (CEN Earth System Assimilation Model). During
the THOR Project, the adjoint model was already used to explore the capabilities of the adjoint method for
optimizing models by estimating ocean and atmospheric parameters. Due to the nonlinear dynamics of the
atmospheric component the length of the feasible assimilation window is limited to the period over which the
model can be approximated by a linear model. Although parameter optimization turned out to be feasible for up
to several weeks, another result of the test experiments showed that the parameter optimization with the full
model parameterizations needed to simulate a realistic climate is limited to one day. For climate change
predictions, an important task is to reduce errors in the model and bring the model closer to observations.
Improving the initial conditions results in reducing short term forecast errors, however, their influence on
climate scale predictions is negligible. Since parameters other than the initial conditions influence the climate of
the model on longer time scales than a few days extending the window of assimilation is essential for matching
the climate of the model to the observations by parameter optimization
To cope with these problems we employ here a stochastic optimization algorithm because of its ability to cope
with inherent system noise and highly nonlinear systems. We use stochastic optimization algorithms for
parameter optimization in CESAM. The stochastic optimization algorithm Simultaneous perturbation stochastic
approximation (SPSA) techniques is implemented and after testing used for the tuning the model to the
observed climate by fitting the climatological output from the model to climatological data from ERA-Interim
(Dee et al., 2011).
3.1 Model configuration and control run
For all experiments in this study, the horizontal resolution of CESAM is set to spectral T21 (approximately 5.6◦
grid spacing) with 10 vertical sigma levels and an integration time step of 45 minutes. Monthly averaged
climatological sea ice concentration (sea ice model is switched off), sea surface temperature (SST), glacier
coverage, albedo and soil temperature from ERA40 reanalysis (Uppala et al., 2005) are used as boundary
conditions. Beginning from a cold start, a forward run of CESAM was carried out for 25 years. A restart file was
recorded at the end of 25 years and this restart which corresponds to 1st January is used as initial condition in
Page 4
all the experiments. A control simulation is made from the initial and boundary conditions and a forward run of
CESAM for a period of one year is carried out. Default values of control parameters given in Table 3.1 are
used. This simulation is referred to as the control run and is used as a reference in all the experiments. The
control run is referred to as model’s original state, i.e. the state before parameter optimization.
Table 3.1— List of 15 control parameters used in optimization procedure
3.2 Comparison of model state with ERA-Interim reanalysis data
In order to validate the model before optimization basic climate variables such as temperature at 1000 hPa,
sea level pressure, zonal winds at 500 hPa and surface precipitation are selected and a comparison of the
annual mean control run (1 year period as discussed in section 2.2) with annual mean ERA-Interim reanalysis
(referred to as observations hereafter) is performed. The annual mean observations are constructed from 21
years of data (from 1989 to 2009) and the values are averaged on to the model grid. Figure 3.1 shows the
comparison of annual mean temperature at 1000 hPa (which is near the surface). Overall patterns and range
of temperature distribution in the control run (top panel) agree well with the observations (middle panel).
However from the difference control - observations plot (bottom panel), it can be clearly seen that CESAM in its
original state in general has colder temperature with respect to observations. Exceptions are the regions over
middle east, eastern China, south-west Australia, most parts of central North America and over north west
region over South America. In tropics the difference is around -2 to -4 K, however in polar regions the model is
far too cold (-10 to -15 K), specially over the Arctic Ocean. Addtitional comparisons are performed for e.g.
precipitation, pressure or zonal wind which demonstrated that the model in general produces a realistic climate
Page 5
while it simulates colder surface temperatures, weaker zonal winds over the Southern Ocean and a poorly
resolved ITCZ
Figure 3.1— Annual mean temperature (K) at 1000 hPa from (top) control run, (middle) observations and (bottom)
difference (control-observations). Control run represents the annual mean state resulting from one year run discussed in
section 2.2. Observations represents annual mean of 21 years (1989-2009) from ERA-Interim reanalysis.
Page 6
3.3 Parameter estimation using SPSA technique
The details of SPSA are given by (Spall 1998); however the technique is briefly discussed here. In SPSA the
cost function Y() is minimized using the iterative procedure
k+1 = k − akgk(k)
where gk(k) is a stochastic approximation of ∇Y(k), which denotes the gradient of the cost function with
respect to the parameters  evaluated at previous iterate k. The stochastic gradient gk(k) in SPSA is
calculated by
𝑔𝑘 (𝜃𝑘 ) =
𝑌(𝜃𝑘 + 𝑐𝑘 ∆𝑘 ) − 𝑌(𝜃𝑘 − 𝑐𝑘 ∆𝑘 )
2𝑐𝑘 ∆𝑘
∆𝑘 is a p-dimensional perturbation vector generated by Monte Carlo method where each of the p components
of ∆𝑘 are independently generated from a zero mean probability distribution. Here ∆𝑘 is sampled from a
Bernoulli ± distribution with probability of 0.5 for each ±1 outcome. The gain sequences (ak and ck) are defined
as
𝑎𝑘 =
𝑎
𝑐
, 𝑐𝑘 =
𝛼
(𝐴 + 𝑘 + 1)
(𝑘 + 1)𝛾
The choice of gain sequences (ak and ck) is critical for the performance of SPSA. It is observed that in a high
noise setting it is necessary to pick smaller a and larger c than in a low noise setting. The asymptotically
optimal values of  and  are 1.0 and 1/6, respectively (Fabian 1971;Chin 1997). As a rule of thumb (with the
Bernoulli ±1distribution for the elements of ∆𝑘 ), it is effective to set c at a level approximately equal to standard
deviation of the noise in model cost function Y from equation 1. However from Figure 3.2, it is observed that
model is highly nonlinear and so is the model cost function Y.
3.4 Cost function (Model data misfit)
It is assumed that the difference between data and simulation results (model-data misfit) is only due to
measurement errors and incorrectly prescribed model parameters. The prior information on the control
parameters is quantified by a Gaussian probability density function (PDF) in control space and observational
information by a PDF in the space of observations. The problem of estimation is then solved by minimizing the
cost function Y with respect to the parameters
Y = 0.5[(M(x) − d)TCd −1 (M(x) − d)+(x − x0)TC0−1(x − x0)]
Page 7
Here M(x) denotes the model operated as a mapping of the parameters onto observations used. d is the
observation vector and x0 is model’s background state vector. Cd and C0 represents uncertainities in the
observations and the model respectively (Blessing et al 2013). Due to the chaotic nature of the model, annual
means of model data differences are used. In equation 1, the first term is the model-data misfit and second
term
represents
the
prior
information
(background
term).
Since
the
number
of
observations
(64*32*365*16=11960320) are more than control parameters (15), the influence of background term will be
much smaller than model-data misfit. Considering the negligible contribution of background term, we set the
background term to zero i.e. we are not using prior information on the control variables in this study.
Table 3.2. Variables contributing for cost function computation. The second column list standard deviation (STD) that are
globally averaged values computed from annual averaged data sets spanning (1979-2012)
In this study, the cost function comprises of 16 variables which are listed in Table 3.2 together with their
corresponding globally averaged STD (C0). C0 is computed from ERA-Interim reanalysis (Dee and Uppala et al
2011) using the annual averages of variables over a period of 34 years (1979-2012). For real data
experiments, ERAInterim reanalysis data is used and therefore variances are calculated using ERA-Interim
data.
3.6 Sensitivity Analysis
In order to see the effect of control parameters used in various parameterization procedures on model’s cost
function, 15 control parameters are chosen for SA. These parameters are listed in Table 3.1 along with their
definitions. Cloud and radiation parameters are commonly chosen optimization procedures (Annan et al 2005;
Severijins and Hazeleger 2005) as they are mainly responsible for accurate temperature and precipitation
Page 8
simulations. Recently, (Blessing et al 2013) used diffusion time scale related parameters for the coupled model
in their optimization experiments using 4D var method. In this study, parameters involved in the
parameterization of long wave; short wave radiation; cloud parameters and vertical diffusion time scales have
been used.
The parameters tfrc1 and tfrc2 are Rayleigh friction time scales in level 10 and level 9 respectively. The
parameters tdissz, tdissd and tdisst represents time scale of damping in the atmosphere and are used in
horizontal diffusion parameterizations. The parameters vdiff lamm, vdiff b, vdiff c and vdiff d are used in
calculation of mixing lengths, drag and transfer coefficients. The parameters tpofmt, th2oc, tswr1, tswr2, tswr3
and acllwr are used in cloud parameterizations. A high value of th2oc increases the absorption of radiation in
the atmosphere leading to increase in surface temperature. This results in more evaporation and hence
increases the moisture in air. The small value of parameter tswr1 increases the cloud albedo and leads to
more precipitation. In order to have balance of precipitation and evaporation these two parameters should be
consistent. The parameters tswr2, tswr3 and acllwr are also responsible for cloud back scattering and
absorption coefficient for clouds.
Figure 3.2. Plot of model cost function vs. perturbation applied to model parameters. The cost function is based on annual
mean pseudo data over a period of 1 year model integration. It also shows the range of minimum cost function that can
achieved if parameter perturbations are not precisely zero. 15 parameters that are used in the procedure are defined in
Table 3.1
Page 9
Sensitivity studies give an idea of the range of perturbation for each parameter that corresponds to a certain
change in cost function. The range of true minimum of cost function in case of identical twin experiment can be
identified from sensitivity plot. Even for shorter time scales (less than 7 days) the cost function doesn’t have a
unique minimum (Figure not shown), however, due to the non- linearity of an atmosphere system, for longer
integration time scales, several secondary minima exist.
For the present sensitivity experiments, the parameters are perturbed (one at a time) from -90% to 800% (with
an interval of 10%) of the original parameter value and the model is integrated for a period of 1 year. The
synthetic data used are annual mean values of temperature, precipitation and heat flux. From Figure 1, the
range of cost function values for all parameters perturbed in above mentioned range varies between 0 and
400. Because of the stochastic nature of the cost function, the true minimum, which is zero in case of an
identical twin experiment, cannot be reached; instead the minimum can only be reached by the precise set of
parameter values but any deviation leads to sizable cost function values, this suggests that in identical twin
experiments unless we obtain exact values of original parameters, minimal value of cost function that could be
reached would be between 10 and 20 and further reduction in cost function may not be possible.
3.7 Identical twin Experiments
A default setup is used to generate pseudo data for one year period. The iterative procedure is started using a
perturbed vector of chosen 15 parameters as mentioned in Table 1, known as the control vector. The size of
the perturbation is based upon the sensitivity of cost function to each parameter. As seen in Figure 3.2, the
parameters tfrc1, tfrc2, tdissz, tdissd, tdisst, vdiff b, vdiff c, vdiff d and tpofmt are contributing almost same as
the noise in the cost function and therefore the size perturbation applied to these is smaller (~ 20% of the
original value) as compared to the remaining parameters (~ 200% of the original value). The assimilation is
carried out over 1-year integration time scale. The experiment is successful if we can approximately recover
the default values through the assimilation of the data via reduction in cost function.
The value of a and c in the gain sequence were chosen to be 0.01 and 0.2 respectively. a is used for gain
sequence ak which relates to the inverse of Hessian in the Quasi-Newton algorithm. The value of c is decided
by the variability of the cost function over certain range of perturbations in parameters (Spall 1998). All 15
parameters (normalised to 1) are perturbed simultaneously and using the SPSA method, gradients are
computed and corrections to the parameter are applied.
Page 10
Figure 3.3. Cost function vs. iteration number for selected 15 parameters in the identical twin experiment. Cost function is
computed for a period of 1 year model integration
Figure 3.3 shows the cost function vs. iterations plot. It can be seen that the cost function achieves the
minimum value of 15 in around 120 iterations and then continues to fluctuate about this value further on. As
discussed in section 3.1 this value of cost function lies in the range of an acceptable minimum and it cannot
reduce further. At this stage about 2/3 of the cost corresponds to temperature and 1/3 to the surface fluxes.
Ideally, in identical twin experiments, the reduction in cost function and in Euclidean distance of control vector
to the default values is achieved simultaneously. The parameters are expected to converge to their original
values, however in our case only values for 8 out of 15 parameters were retrieved, while the remaining 7
parameters were not converging to their default values. From the sensitivity experiments curve in Figure 3.2 it
can be seen that the cost function is less sensitive to the parameters, which were not correctly retrieved and
that some parameters were not contributing in the cost function more than the noise level.
3.8 Optimization with ERA-Interim Reanalysis data
The observational data used are from ERA-Interim reanalysis. These datasets are interpolated on model levels
and then averaged to the model horizontal grid. We have used annual means of the temperature (all levels),
large scale precipitation (LSP), convective precipitation (CP), surface sensible heat flux (SSHF), surface latent
heat flux (SLHF), surface solar radiation (SSR) and surface thermal radiation (STR). As expected, with ERAInterim data the cost function has a larger value (order 3 more) as compared to cost function against pseudo
data and therefore the value of a in the algorithm was chosen to be low (~1e-4) for stable results in successive
iterations. The model was integrated for 1 year using default values of parameters and using SPSA, the
gradients with respect to ERA-Interim data were used to apply corrections to parameters iteratively. The cost
function reduces until it reaches a value after which there is no substantial decrease and the cost function
fluctuates around that minimum value. This happens after about 500 iterations and by then the reduction in
Page 11
cost function is around 25%. The main contribution for reduction in cost function is from tfrc1, tpofmt, th2oc,
tswr1 and tswr3 control parameters. Remaining parameters contribution is significantly less as compared to
mentioned parameters.
Figure 3.4. Total cost function (using ERA-Interim renalysis data) vs iteration number for three ensemble members.
The robustness of the experiment results is assessed by an ensemble with three members, each of which start
from a different point in control space. Three ensembles (Ens-1, Ens-2 and Ens-3) using different initial values
of control parameters are carried out with 15 control parameters. In Ens-1, Ens-2 and Ens-3 the value of each
control parameter is increased by 0% (default parameter values), 60% and 100% respectively. Figure 3.4
shows the reduction in cost function with successive model iterations for three ensembles. All three cases
converging to same value of the cost function gives an impression that the technique is robust.
The performance of optimization was judged by means of Root Mean Square Error (RMSE) normalized by the
prior error which is the same as the square root of normalized cost function number of observations (Figure
3.5). The model state resulting after using optimized parameters shows improvement on the original state
when compared to ERA-Interim reanalysis. The total RMSE which was ~ 3.1 in the original run is reduced by ~
16% to 2.6 while for the individual variables the reduction is 5%, 11% and 18% for precipitation, net heat flux
and surface temperature, respectively.
Page 12
Figure 3.5. RMSE of global mean Temperature between ERA-Interim and original and optimized model temperature. Y
axis denotes height of the atmosphere represented as sigma levels with model level 10 corresponding to the bottom and
model level 1 corresponding to the top of the atmosphere
RMSE of global mean temperature profile from original and an optimized state is shown in Figure 3.5. The
errors have reduced at almost all levels barring level 2. The levels of maximum errors experienced significant
reduction. Overall there is ~10-20% of reduction of error in temperature at various levels in the atmosphere.
Page 13
Figure 3.6. Root mean squared differences (RMSD) of ERA-Interim with Original (top panels) and Optimized (bottom
panels) model near surface temperature.
Spatial distribution of errors of surface temperature from original and optimized state is shown in Figure 3.6.
Most of error reduction in surface temperature takes place in the equatorial West Pacific, North Atlantic and
Southern Indian Ocean regions. Net heat flux shows improvement almost in every region with large errors of >
50 Watts/m2 coming down within 10 Watts/m2 in the North Pacific, south-east Africa and also in South Atlantic.
There are some places where optimization has resulted in larger errors, most prominent of these are the
regions around Greenland in the North Atlantic and in the eastern equatorial Indian Ocean.
Page 14
4. References
Annan, J. D., et al. "Parameter estimation in an atmospheric GCM using the ensemble Kalman filter."
Nonlinear processes in geophysics 12.3 (2005): 363-371.
Blessing and co-authors; Testing variational estimation of process parameters and initial conditions of an Earth
System Model, Tellus, 66:22606. DOI: 10.3402/tellusa.v66.22606
Chin, Daniel C. ”Comparative study of stochastic algorithms for system optimization based on gradient
approximations.” Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on 27.2
(1997): 244-249.
Dee, D. P., et al. ”The ERA Interim reanalysis: Configuration and performance of the data assimilation system.”
Quarterly Journal of the Royal Meteorological Society 137.656 (2011): 553-597.
Fabian, Vclav. Stochastic approximation. Department of Statistics and Probability, Michigan State University,
1971.
Spall, James C. ”Implementation of the simultaneous perturbation algorithm for stochastic optimization.”
Aerospace and Electronic Systems, IEEE Transactions on 34.3 (1998): 817- 823.
Severijns C. A., and W. Hazeleger. Optimizing parameters in an atmospheric general circulation model.
Journal of climate 18.17 (2005): 3527-3535.
Uppala, Sakari M., et al. ”The ERA40 reanalysis.” Quarterly Journal of the Royal Meteorological Society
131.612 (2005): 2961-3012.
5. Dissemination and uptake
Plan for future publication
In
preparation
OR
submitted?
Title
Main author
All authors
Title of the
periodical or the
series
Is/Will open access[2]
provided to this
publication?
To be
submitted
Parameter
optimization in an
atmospheric GCM
using
Simultaneous
Perturbation
Stochastic
Approximation
(SPSA)
Variational
assimilation of SSMI
sea ice concentration
in a coupled Arctic
sea-ocean model
Agrawal, R., A. Köhl,
D. Stammer
Tellus
Will be open access
Nikolay V. Koldunov,
Armin Köhl, Nuno
Serra, Detlef
Stammer
J. Geophys. Res
Will be open access
To be
submitted
Page 15
6. The delivery is delayed:
 Yes
 No
7. Changes made and difficulties encountered, if any
Difficulties were encountered during the usage of the adjoint method for the estimation of parameters in chaotic
atmospheric models. For this reason, as explained above a different Method the Simultaneous Perturbation
Stochastic Approximation (SPSA) method was employed, which showed success and enabled the estimation
of parameters.
8. Efforts for this deliverable
How many person-months have been used up for this deliverable?
Partner
Person-months
25
25
UHAM
Period covered
1 Nov.2012-30 October 2014
Total
Total estimated effort for this deliverable (DOW) was 25 person-months. The names of the scientist involved
are mentioned on the cover page of this report.
9. Sustainability
Parameter estimation with chaotic climate models is a challenging task and topic of ongoing research. Despite
the momentary switch from the adjoint to the SPSA method, which was needed to cope with the chaotic
dynamics of the atmospheric component of CESAM, we will continue to explore alternative modifications to the
adjoint method.
10. Dissemination and uptake activities
The complete list of dissemination activities is available on the NACLIM website
http://naclim.zmaw.de/Dissemination.2509.0.html and on the ECAS portal, section “List of Dissemination
Activities”.
Uptake by the targeted audience
According to the DOW, your audience for this deliverable is:
X
The general public (PP)
The project partners, including the Commission services (PU)
A group specified by the consortium, including the Commission services (RE)
This reports is confidential, only for members of the consortium, including the Commission services (CO)
How are you going to ensure the uptake of this deliverable by the targeted audience?
The work will be published in in scientific journals accessible to scientist and the general public.
A summary for the general public will be made available on the website as soon as the work is published.
Page 16