Development of AMSU-A Pre-processing and Quality
Control Modules at KIAPS Observation Processing System
10p.07
Sihye Lee, Ju-Hye Kim, Jeon-Ho Kang, and Hyoung-Wook Chun
Korea Institute of Atmospheric Prediction Systems (KIAPS), Seoul, South Korea
INTRODUCTION
• Microwave radiometers onboard satellites have been used to measure a wide variety of atmospheric and surface parameters. The Advanced Microwave Sounding Unit-A (AMSU-A) is one of the satellites with the largest impact to reduce forecast errors in data assimilation.
• All data assimilation systems are affected by biases, caused by problems with the data, by approximations in the observation operators used to simulate the data, by limitations of the assimilating model, or by the assimilation methodology itself. A clear symptom of bias in
the assimilation is the presence of systematic features in the analysis increments (Dee, 2005).
• The objective of this study is to introduce the AMSU-A radiance pre-processing and quality control modules including bias correction at the KIAPS observation processing system.
KIAPS AMSU-A Processing System
• Observation Extraction: AMSU-A level-1d radiance data have been extracted
using the ECMWF BUFR decoder.
 Extracted AMSU-A radiance data (TB) monitoring
• Observed TB (0.35° x 0.23°)
• Sanity Check: Physical reality checks on geolocation and observation, blacklisting
of broken channels, and QC flagging for clear-sky radiance assimilation.
In channel 5, monthly mean of observed TB is
high at low latitude for November 2012, but it
• Background Ingest: Atmospheric variables of model background have been
matched to the observation state with space interpolation.
decreases at high latitude. The land variability
November 2012
• First Thinning: Duplicate observations in a defined grid box have been eliminated
using the removal scores.
November 2012
(i.e., standard deviation) is more than ~4.5 K.
• Observation Operator: The RTTOV_10.2 fast RTM have been implemented to
convert the atmospheric variables of model state to the radiance of observation
state, and to calculate the Jacobian matrices of model state.
• Background TB (0.35° x 0.23°)
Monthly mean of background (Unified Model
• Initial Quality Control: The pixels contaminated by cloud, precipitation, and sea
ice have been removed and assimilation channels have been selected, with
considering surface type and topography.
output: e.g., qwqu00.pp_006) is similar to
observed TB but land variation of background
November 2012
November 2012
TB is less than that of observation.
• Bias Correction: Scan and airmass bias correction modules have been developed
in two steps based on 30-day innovation statistics.
• Outlier Removal: The expected standard deviation of first guess (FG) departure
has been estimated from assigned observation errors to eliminate outliers.
•
O-B (innovation)
Both monthly mean and standard deviation of
• Final Thinning: Final thinning have been performed with considering the
assimilation resolutions, and then survived radiance data have been prepared to
pass KIAPS data assimilation system.
innovation are high in land, especially for high
topography such as the Andes mountains and
November 2012
November 2012
desert area.
• Monitoring and Statistics: Bias correction coefficients and observation errors
have been updated by off-line monitoring codes of statistics and QC scores.
Quality Control and Bias Correction
Multicollinearity of Airmass Predictors
 QC flags: Scattering index, Cloud liquid water, Sea ice index [Grody et al., 1999, 2001]
 Spatial distributions of airmass predictors
Threshold for initial QC
• Scatt Index > 40
• CLW > 0.2
• Sea-ice index > 50
November 2012
Scatt _ indx =
−113.2 + (2.41 − 0.0049 TB (CH 1) ) TB (CH 1)
CLW =
µ [ D0 + 0.754 ln(285.0 − TB (CH 1) )
+ 0.454 TB (CH 2) − TB (CH 15)
− 2.265 ln(285.0 − TB (CH 2) )]
November 2012
November 2012
November 2012
For latitudes beyond 50 degrees of the equator,
• Multicollinearity is a statistical phenomenon in which two or more predictor variables are highly correlated.
µ = cos( sat _ zenith)
• In this situation the coefficient of the multiple regression may change erratically in response to small changes,
and it may not give valid results about estimation of parameters.
 Bias correction
• Variance Inflation Factor (VIF)
(1) Step 1: mean innovation at each scan angle to equal to the mean innovation at the center scan angle
Observed TB (SD)
𝑗,𝑠
𝜃=0
bscan : scan bias j : channel s : satellite θ : scan angle
ch04
ch05
ch14
ch13
ch06
240
ch12
ch07
220
ch11
ch08
ch10
ch09
200
0
8
16
24
32
40
48
ch13
ch14
12
November 2012
10
ch04
ch09
6
ch05
ch10
4
ch06
ch08
56
Scan position
Thick850-300
Thick200-50
Thick50-5
Thick10-1
1.04
1.04
10.42
1.24
 Remedies for multicollinearity of airmass predictors
• Selection of different airmass predictors or one predictor
0
• Ridge regression or principle component regression (PCR) with 4 atmospheric thickness predictors
-1
ch07
2
1
1 − R 2j
MetOp-A
ch12
ch11
NOAA-15
NOAA-18
NOAA-19
1
8
VIF j =
AMSU-A:
AMSU-A : ch05
ch05
2
O - B (K)
𝑗,𝑠
𝜃 − 𝑂−𝐵
260
Brightness Temperature (K)
Observed TB
Brightness Temperature (K)
𝑠𝑠𝑠𝑠
𝑏𝑗,𝑠
𝜃 = 𝑂−𝐵
November 2012
 Problems in multiple linear regression
2.85 + 0.20 TB (CH 1) − 0.028 TB (CH 3)
Seaice _ indx =
D0 =8.240 − (2.622 − 1.846 µ ) µ
November 2012
November 2012
-2
0
8
16
24
32
40
48
56
Scan position
0
4
8
12 16 20 24 28 32 36 40 44 48 52 56
Scan position
Similar patterns for observed and background TBs
(2) Step 2: global multiple linear regression of the scan-corrected innovations against 4 predictors (thickness 850-300, 200-50,
 Step of PCR to calculate new airmass bias coefficients
50-5, 10-1 hPa) to correct the airmass bias
Covariance matrix (S) of predictors (X)
𝑎𝑎𝑎
𝑏𝑗,𝑠
= 𝑎𝑗,𝑠 (𝑍850 −𝑍850 )+𝑏𝑗,𝑠 (𝑍200 −𝑍200 )+ 𝑐𝑗,𝑠 (𝑍50 −𝑍50 )+ 𝑑𝑗,𝑠 (𝑍10 −𝑍10 )+ 𝑒𝑗,𝑠
bair : airmass bias
j : channel
s : satellite
multiple linear
regression
a, b, c, d, e : airmass coefficients
Z850 : Thickness850−300 Z200 : Thickness200−50 Z50 : Thickness50−10 Z10 : Thickness10−1
Eigenvalues (D) of S
𝑍−Z
monthly dataset
(November 2012)
Eigenvectors (V) of S
PC: Score matrix (T) = X * V
 Global distributions for quality control and bias correction
PC regression of new data set
FG departures after sanity check:
y o − H (M (x b ))
FG departures after sanity check and
duplicate removal: y o − H (M (x b ))
FG departures after sanity check, duplicate
removal and initial QC: y o − H (M (x b ))
 Experiments to remedy multicollinearity
① Multiple linear regression
with 4 predictors
CH05
2012/11/02/00UTC
Bias corrected FG departures:
y o − b − H (M (x b ))
2012/11/02/00UTC
CH05
y o − b − H (M (x b ))
Bias corrected FG departures after outlier
2012/11/02/00UTC
2012/11/02/00UTC
CH05
2012/11/02/00UTC
removal and final thinning: y o − b − H (M (x b ))
CH08
2012/11/02/00UTC
③ Principal component egression
with 4 PCs
2012/11/02/00UTC
Bias corrected FG departures after
outlier removal:
② Linear regression with 1 predictor
for each channel
2012/11/02/00UTC
CH08
2012/11/02/00UTC
2012/11/02/00UTC
CH08
2012/11/02/00UTC
2012/11/02/00UTC
SUMMARY
• We have developed the AMSU-A data pre-processing and quality control system to provide the well-qualified radiance data for KIAPS data assimilation system.
• It appears to be successful in controlling the scan and airmass bias in the crucial channels which sound tropospheric and stratospheric temperature below 50 km altitude.
• However, multicollinearity is observed when 4 thickness predictors are highly correlated among themselves. We have tried to find a small set of linear combinations of the covariates which are uncorrelated with each other.
As a result, multicollinearity of predictors are resolved with PCR of 4 PCs, the bias correction performance at lower stratospheric channels is not shown improved much, though.
The 19th International TOVS Study Conference, Jeju Lsland, South Korea, 26 March - 1 April 2014