Estimation of satellite observation
impact on numerical weather
forecast using adjoint-based
method
Hyun Mee Kim1, Sung-Min Kim1, Sangwon Joo2, and Eun-Jung Kim2
1Yonsei University and 2Korea Meteorological Administration
March 31, 2014
The 19th International TOVS Study Conference (ITSC-19)
Jeju, Korea
Background
•
Recently, the number of observations used in a data assimilation system
is increasing enormously. Because it is not clear that all of these
observations are always beneficial to the performance of the NWP, it is
important to evaluate the effect of observations on these forecasts.
•
Traditionally, the impact of observations has been assessed with
observation system experiments (OSEs). The OSEs require much
computational resources.
•
The alternative way to evaluate the impact of observations on the
forecast is the adjoint-based method, introduced by Baker and Daley
(2000).
•
The adjoint-based observation impact can simultaneously evaluate the
observation impact for all dataset, with lesser computation compared to
OSEs, by using the adjoints of DA and forecast system.
•
In this study, the impact of observation on the forecast is evaluated by
the adjoint-based method in a global (UM) modeling and analysis
system.
Forecast sensitivity to observation
δR is measure of forecast error reduction [e.g. energy norm]
δ R = (x fa − xt )T C(x fa − xt ) − (x fb − xt )T C(x fb − xt )
∂R ∂x a ∂R
=
∂y ∂y ∂x a
Sensitivity to the analysis
Analysis sensitivity to observation
<
∂R
∂R
, δ x a >=<
, x a − xb >
∂x a
∂x a
∂R
=
<
, K (y - h(xb )) >
∂x a
∂R
=
< KT
, (y - h(xb )) >
∂x a
∂R
=
<
,δ y >
∂y
x a = xb + K(y - h(xb ))
∂R
∂x a
∂x a
= K T = R −1H (B −1 + HT R −1H ) −1
∂y
Forecast impact :
∂R
=
δR
(y − h(xb ))
∂y
Experimental design
•
Period
- 2011. 6. 1. 00 UTC ~ 2011. 8. 31. 00 UTC [Summer months]
2011. 12. 1. 00 UTC ~ 2012. 2. 29. 00 UTC [Winter months]
- 24hour forecast, evaluate on 00, 06, 12, and 18 UTC
•
Model configuration
- KMA UM N512 (vn7.7) and 4D-Var N144 (vn27.2)
- Dry TE norm (projected on sfc to 150 hPa) + simple moisture physics
Global and east Asia domain
Fig. 1. Global and east Asian domain (black solid box).
Forecast error reduction
δ R = (x fa − xt )T C(x fa − xt ) − (x fb − xt )T C(x fb − xt )
a
2011.06.01
2011.08.31
b
2011.12.01
2012.02.29
Fig. 2. Nonlinear forecast error reduction (black line) and its linear estimation in model space (red line)
and in observation space (blue line) for analysis time in (a) summer and (b) winter months.
Observation impact
Total OBSI
Time-averaged observation impact (DJF)
95 % confidence interval (DJF)
Time-averaged observation impact (JJA)
95 % confidence interval (JJA)
AMSUA
TEMP
Aircraft
IASI
SYNOP
GOES
BUOY
GPSRO
ScatWind
AIRS
MSG
MTSAT
HIRS
MFG
MHS
SHIP
Profiler
PILOT
KMA COMS
SSMI/S
Dropsonde
BOGUS
-2.0
-1.2
-0.8
-0.4
-1
0.0
Total Impact [J kg per day]
0.4
TEMP
GPSRO
GOES
MTSAT
MSG
Aircraft
BUOY
MFG
AMSUA
IASI
SHIP
SYNOP
AIRS
HIRS
BOGUS
MHS
ScatWind
KMA COMS
SSMI/S
Profiler
PILOT
Dropsonde
-0.0036
b
-0.0012
-0.0006
Beneficial observation
Number of observation profiles or platforms (DJF)
Number of observation profiles or platforms (JJA)
Mean observation impact (DJF)
Mean observation impact (JJA)
a
-1.6
# (observation platform)
Mean OBSI
0.0000
Mean Impact [J kg-1 per day]
Aircraft
SYNOP
AMSUA
GOES
HIRS
MHS
ScatWind
IASI
BUOY
MSG
SHIP
AIRS
MTSAT
SSMI/S
MFG
KMA COMS
TEMP
GPSRO
PILOT
Dropsonde
Profiler
BOGUS
c
0.0
1.5e+4
3.0e+4 1.0e+5
Number of observation profiles or platforms
[per day]
Fraction of beneficial observation (DJF)
Fraction of beneficial observation (JJA)
BUOY
MFG
MTSAT
GOES
MSG
Aircraft
ScatWind
PILOT
HIRS
TEMP
GPSRO
BOGUS
IASI
SSMI/S
SYNOP
AMSUA
SHIP
AIRS
MHS
Profiler
KMA COMS
Dropsonde
d
0 1
46
48
50
52
Fraction of beneficial observation [%]
Fig. 3. Time-averaged statistics (mean and 95 % confidence interval) stratified by each observation type for
(a) total observation impact, (b) mean observation impact, (c) number of observation profiles or platforms,
and (d) fraction of beneficial observation in summer (JJA) and winter (DJF) months in the global region.
54
Observation impact
Time-averaged observation impact (DJF)
95 % confidence interval (DJF)
Time-averaged observation impact (JJA)
95 % confidence interval (JJA)
Mean observation impact (DJF)
Mean observation impact (JJA)
Mean OBSI
b
SSMI/S
GPSRO
-0.0014
AIRS
SSMI/S
GPSRO
AIRS
IASI
METOP-A HIRS
NOAA19 HIRS
NOAA17 HIRS
METOP-A AMSU-A
NOAA19 AMSU-A
NOAA18 AMSU-A
-1.6
-0.0012
IASI
a
-0.0010
METOP-A HIRS
-1.4
Total OBSI
NOAA19 HIRS
-1.2
NOAA17 HIRS
-1.0
-0.0002
METOP-A AMSU-A
-0.8
NOAA19 AMSU-A
-0.6
NOAA18 AMSU-A
-0.4
0.0000
NOAA15 AMSU-A
-0.2
Mean Impact [J kg-1 per day]
0.0
NOAA15 AMSU-A
Total Impact [J kg-1 per day]
0.2
Fig. 4. Time-averaged statistics (mean and 95 % confidence interval) stratified by sounder-type satellite
observation assimilated for (a) total observation impact, (b) mean observation impact in summer (JJA) and
winter (DJF) months in the global region.
Observation impact
Total Impact [J kg-1 per day]
Time-averaged NOAA19 AMSU-A observation impact (DJF)
95 % confidence interval (DJF)
Time-averaged NOAA19 AMSU-A observation impact (JJA)
95 % confidence interval (JJA)
0.05
0.00
-0.05
-0.10
-0.15
a
-0.20
-0.25
4
5
6
7
9
10
11
12
13
14
NOAA19 AMSU-A Channel number
0.02
0.01
686.50 cm-1 ~ 1410.75 cm-1
1986.75 cm-1
~ 1995.00 cm-1
0.00
-0.01
-0.02
b
-0.03
-0.04
38
51
57
63
109
116
122
128
135
141
148
154
161
167
173
179
180
185
187
193
199
205
207
210
212
214
217
219
222
224
226
230
232
236
239
242
243
246
249
252
254
260
262
265
267
269
275
280
282
294
296
299
306
335
386
2907
2948
2958
2988
3027
3029
3053
3058
3064
3168
3248
3506
3577
3583
3589
5130
5368
5371
5379
5381
5383
5397
5399
5401
5403
5405
Total Impact [J kg-1 per day]
Time-averaged IASI observation impact (DJF)
95 % confidence interval (DJF)
Time-averaged IASI observation impact (JJA)
95 % confidence interval (JJA)
METOP-A IASI channel number
Fig. 5. Time-averaged statistics (mean and 95 % confidence interval) stratified by (a) NOAA 19 AMSU-A
and (b) METOP-A IASI channel assimilated for total observation impact in summer (JJA) and winter (DJF)
months in the global region. The wave length range corresponding to large observation impact is indicated
by the dashed line.
Observation impact in the east Asia region
Mean observation impact (DJF)
Mean observation impact (JJA)
Number of observation platforms (DJF) / 1.e+06
Number of observation platforms (JJA) / 1.e+06
Time-averaged observation impact (DJF)
95 % confidence interval (DJF)
Time-averaged observation impact (JJA)
95 % confidence interval (JJA)
KMA COMS
a
MTSAT
MTSAT
MFG
MFG
KMA COMS
GOES
GOES
MSG
MSG
Total OBSI
-0.020
-0.016
b
Mean OBSI
-0.012
-0.008
-0.004
Total Impact [J kg-1 per day]
0.000
-0.0004
0.0000
# (obs platform)
0.0004
0.0020
Mean Impact [ J kg-1 per day]
& number of observation platforms [per day]
Fig. 6. Time-averaged statistics (mean and 95 % confidence interval) stratified by imager type for (a) total
observation impact, (b) mean observation impact and number of observation platforms at summer (JJA) and
winter (DJF) months in the east Asia region. In Fig. 12b, number of observation platforms is normalized by
1*106. At summer (JJA) months, the COMS observation is not assimilated.
Summary
•
The linear δR underestimates the nonlinear δR (75% in summer and 80% in
winter), similar to Langland and Baker (2004).
•
The magnitude of δR depends on the number of observational data
assimilated. The impact of each observation does not change depending on
season.
•
For the global region, the impact of AMSUA is largest, followed by Temp,
AIRCRAFT, IASI, Synop, GOES, Buoy, GPSRO, ScatWind, etc.
Sensitivity to error covariance parameters
Courtesy of Daescu and Todling (2010)
Define:
δ B = δ sbB δ Ri = δ si R
Sensitivity to background
error covariance weighting
δ R = [y − h(x )]T δ R
a
b
δy
δs
Sensitivity to observation
error covariance weighting
δ R = [h (x ) − y ]T δ R
i a
i
δ yi
δ sio
y Observation
xb Background
xa Analysis
B Background error covariance
R Observation error covariance
h Observation operator
Experimental design
•
Period
- 2012. 7. 1. 00 UTC ~ 2012. 7. 31. 18 UTC [To estimate the error covariance parameter]
2012. 8. 1. 00 UTC ~ 2012. 8. 31. 18 UTC [To verify the additional forecast error reduction]
- 24 hour forecast, evaluate on 00, 06, 12, 18 UTC
•
Model configuration
- KMA UM N512 (vn7.7) and 4D-Var N144 system (vn27.2), before Hybrid-system
- Dry total energy norm (projected on sfc to 150 hPa) + simple moisture physics
•
Observation
- Surface : Synop, Ship, Buoy
- Sonde : TEMP (radiosonde), PILOT, Wind profiler, Dropsonde
- Aircraft : AMDAR, AIREP
- Satwind : GOES (+ MODIS, AVHRR), KMA (COMS), JMA, MSG, Meteosat
- ATOVS : NOAA 15~19, MetOp2 (i.e., MetOp-A)
- Scatwind, SSMIS, AIRS, IASI, GPSRO
Statistics of Forecast sensitivity to error covariance parameter
δ R = R(B + δ B, R + δ R ) − R(B, R )
δR
(Linear) forecast error reduction
δR/δs Sensitivity to error parameters
δs
Error covariance parameter (weighting)
δR δR 
≈ δ Rb + ∑ δ Rio = 
, o  , (δ sb , δ sio )
i
 δ sb δ si 
Time-averaged sensitivity to error covariance parameter
99 % confidence interval (July 2012)
관측 종
July
2012
B matrix
ATOVS
AIRS
IASI
Sat_W
Scat_W
SSMI/S
GPSRO
AIRC_T
AIRC_W
SONDE_T
SONDE_W
SONDE_RH
SFC_T
SFC_W
SFC_P
SFC_RH
건조 에너지 놈의 사용
으로 민감도가 매우 작
다.
-3.6 -2.7
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
Foreacast Sensitivity to the error covariance parameter [J kg-1 per day]
1.0
Error covariance parameter
δ R = R(B + δ B, R + δ R ) − R(B, R )
δR δR 
≈ δ Rb + ∑ δ Rio = 
, o  , (δ sb , δ sio )
i
 δ sb δ si 
δR
(Linear) forecast error reduction
δR/δs Sensitivity to error parameters
δs
Error covariance parameter (weighting)
When every potential outlier (> 3 sigma) is rejected:
August
August
August
August
July 2012
B matrix
ATOVS
IASI
AIRS
SAT_W
0.3000
-0.7238
-1.0467
-0.3746
-0.5235
SCAT_W
GPSRO
SSMI/S
SONDE_T
SONDE_W
0.2183
-0.6425
-2.0419
-0.9712
-0.4131
SONDE_RH
SFC_P
SFC_T
SFC_W
SFC_RH
-0.9745
-0.7905
-0.5636
-0.7638
0.2676
AIRC_T
AIRC_W
Rejection rate
-0.3410
-1.2261
7/124 (5.6 %)
~ Tuning variable not used for DA system
Forecast error reduction
δ R = R(B + δ B, R + δ R ) − R(B, R )
δR δR 
≈ δ Rb + ∑ δ Rio = 
, o  , (δ sb , δ sio )
i
 δ sb δ si 
Verification: July 2012
δR
(Linear) forecast error reduction
δR/δs Sensitivity to error parameters
δs
Error covariance parameter (weighting)
L2-norm
1.1
0.7
TEMP Temperature
TEMP Wind
TEMP Humidity
PILOT Wind
SYNOP Ps
SYNOP Temperature
SYNOP Wind
SYNOP Humidity
BUOY Ps
BUOY Temperature
BUOY Wind
SHIP Ps
SHIP Temperature
SHIP Wind
SHIP Humidity
METAR Ps
METAR Temperature
METAR Wind
METAR Humidity
BOGUS
AIRCRAFT Temperature
AIRCRAFT Wind
IASI Radiance
AIRS Radiance
AMSUA Radiance
AMSUB Radiance
HIRS Radiance
SSMIS Radiance
GOES Wind
KMA Wind
JMA Wind
MSG Wind
ESA Wind
ASCAT Wind
GPSRO
Per day
More Observations assimilated
Verification : Observation residual
y − h(x a (σˆ ))
y − h ( x a ( σ ))
2
Time-averaged ||y-h(xa)||2 in ATOVS_def
/ Time-averaged ||y-h(xa)||2 in CTL
2
Average 10.62% residual reduction
1.0
0.9
0.8
ATOVS [AMSUA, AMSUB, HIRS]
0.6
0.5
0.4
Verification : Forecast error (time-average) in the
observation space
SONDE Zonal Wind
SFC
76.06
68.66
62.12
56.35
51.28
46.8
42.86
39.39
36.33
33.63
31.25
29.13
27.25
25.58
24.07
22.72
21.5
20.39
19.37
18.43
17.55
16.74
15.97
15.24
14.55
13.89
13.26
12.65
12.06
11.49
10.93
10.4
9.87
9.37
8.87
8.39
7.93
7.47
7.03
6.61
6.19
5.79
5.41
5.03
4.67
4.33
3.99
3.67
3.37
3.07
2.79
2.53
2.27
2.03
1.81
1.59
1.39
1.21
1.03
0.87
0.73
0.59
0.47
0.37
0.27
0.19
0.13
0.07
0.03
0.01
0.0
SONDE Meridional Wind
CTL
ATOVS_def
Vertical level [Height, km]
Vertical level [Height, km]
Top (76 km)
2.0e+1 4.0e+1 6.0e+1 8.0e+1 1.0e+2 1.2e+2
-1
RMSE [wind speed, m s ]
76.06
68.66
62.12
56.35
51.28
46.8
42.86
39.39
36.33
33.63
31.25
29.13
27.25
25.58
24.07
22.72
21.5
20.39
19.37
18.43
17.55
16.74
15.97
15.24
14.55
13.89
13.26
12.65
12.06
11.49
10.93
10.4
9.87
9.37
8.87
8.39
7.93
7.47
7.03
6.61
6.19
5.79
5.41
5.03
4.67
4.33
3.99
3.67
3.37
3.07
2.79
2.53
2.27
2.03
1.81
1.59
1.39
1.21
1.03
0.87
0.73
0.59
0.47
0.37
0.27
0.19
0.13
0.07
0.03
0.01
0.0
CTL
ATOVS_def
August
2.0e+1 4.0e+1 6.0e+1 8.0e+1 1.0e+2 1.2e+2
2012
RMSE [wind speed, m s-1]
Summary
• This study calculated the forecast sensitivity to error covariance parameter
(Daescu and Todling 2010) for July 2012, and proposed the optimized error
covariance by using the multiple linear regression. The modified error
covariance was applied to the forecast trajectory during August 2012, and the
forecast errors using the modified error covariance were evaluated in the
observation and model space.
• The multiple linear regression analysis for the sensitivity to error covariance
parameter suggested that the ATOVS observation (i.e., AMSU-A, AMSU-B and
HIRS) error needs to be decreased by 72.38 % when the background error
covariance is increased by 30 %. It is indicated that the more ATOVS
observation must be assimilated in the numerical model, compared to the
current operational system.
• According to the verification for August 2012 period, the more ATOVS
observation assimilated in the model decreased the distance between the
forecast and the observation in the observation space.
Thank you