Linear Form of the Radiative Transfer Equation Revisited Bormin Huang Cooperative Institute for Meteorological Satellite Studies, Space Science and Engineering Center University of Wisconsin–Madison The 15 th International TOVS Study Conference (ITSC-15) Maratea, Italy, 4-10 October 2006 Outline • Smith’s linear form (Applied Optics, 1991) vs. its dual form. • The linear form with exact analytical Jacobians vs. the tangent-linear and adjoint models. • The linear form with exact analytical Jacobians vs. the linear form with inexact analytical Jacobians. • Why use the forward model with exact analytical Jacobians for physical retrieval and data assimilation? • Summary • Future work Bill Smith’s Achievement in Physical Retrieval • Bill has been pioneering the physical retrieval of temperature and absorbing constituent profiles from the radiance spectra since the late ’60s. • His most recent linear form of the RTE was published in the landmark paper in Applied Optics (1991). • This monochromatically approximate linear form and its variant have been still used in the physical retrieval [e.g. Ma et al., 2000, Li et al., 2001]. • His work inspired Huang et al. (Applied Optics, 2002) to successfully derive the linear form with exact analytical Jacobians for the widely-used McMillinFleming-Eyre-Woolf type of forward models (e.g. RTTOV, RTIASI). • In this talk I will prove that there exists the dual representation of Smith’s linear form (1991), and show some mathematically interesting outcomes derived from this dual form. Smith’s Linear Form (1991) vs. its Dual Form obs Rv R 0 v = Bv ( ps ) τ v ( ps ) − = B ( ps )τ ( ps ) − 0 v 0 v ∫ ∫ PS 0 ps 0 Bv ( p ) dτ v ( p ) B v0 ( p ) d τ v0 ( p ) δ Rv ≡ Rvobs − Rv0 δ Bv ( p ) ≡ Bv ( p ) − Bv0 ( p ) δ τ v ( p ) ≡ τ v ( p ) − τ v0 ( p ) Smith’s Linear Form Its Dual Form δ Rv = Bv ( ps )δτv ( ps ) + δ Bv ( ps )τv0 ( ps ) δ Rv = δ Bv ( ps )τv ( ps ) + Bv0 ( ps )δτv ( ps ) − ∫ Bv ( p) d [δτv ( p)] − ∫ δ Bv ( p)dτ ( p) ps ps 0 0 0 v − ∫ δ Bv ( p)dτv ( p) − ∫ Bv0 ( p)d [δτv ( p)] ps ps 0 0 δ Rv = Bv ( ps ) δ τ v ( ps ) + δ Bv ( ps ) τ v0 ( ps ) δ Rv = δ Bv ( ps )τv ( ps ) + Bv0 ( ps )δτv ( ps ) − ∫ Bv ( p) d [δ τ v ( p)] − ∫ δ Bv ( p) dτ v0 ( p) ps ps 0 0 δ Bv ( p) = − ∫ δ Bv ( p)dτv ( p) − ∫ Bv0 ( p)d [δτv ( p)] d lnτ v0i ( p) i =1 dUi0 ( p) ∫ dτ v0 ( p ) = τ v0 ( p ) ∑ d ln τ v0i ( p ) d lnτ ( p) dT ( p) dT ( p) = d lnτ ( p) 0 , dU ( p) dUi ( p) 0 vi − ∑ ∫ β ( p)τ ( p)δ T ( p)d lnτ ( p) N ps +∑∫ i=1 0 0 v 0 vi dT ( p) β ( p)τ ( p)δUi ( p) 0 d lnτ v0i ( p) dUi ( p) 0 v 0 q i ( p ′) d p ′ τ v ( ps ) ≈ τ v0 ( ps ) 0 v dUi ( p) d lnτ v0i ( p) dUi0 ( p) = dUi ( p) 0 d ln τ ( p) v 0 i dUi ( p) δ Rv ≈ βv0 ( ps )τ v0 ( ps ) δ Ts δ Rv ≈ βv0 ( ps )τ v0 ( ps )δ Ts 0 p i=1 0 vi 0 i i=1 0 N N 0 v 0 ∂ Bv (T ( p)) δT ( p) ≡ βv0 ( p) δT ( p) ∂ T ( p) 1 U i ( p) ≡ g ps ps 0 δ τ v ( p) ≈ τ v0 ( p) ∑δ Ui ( p) N ps N − ∑ ∫ βv0 ( p)τ v0 ( p) δ T ( p) ps i=1 0 N ps +∑∫ i=1 0 dUi ( p) 0 ln τ d vi ( p) o dUi ( p) d T 0 ( p) 0 β ( p)τ ( p) δ τ d ( ) ln ( p) U p v i 0 i dUi ( p) 0 v 0 v δ Rv ≈ βv0 ( ps )τ v0 ( ps )δTs N δ Rv ≈ βv0 ( ps )τv0 ( ps )δ Ts N − ∑ ∫ β ( p)τ ( p) δT ( p) d lnτ ( p) ps i =1 0 N ps + ∑∫ i =1 0 0 v 0 v − ∑ ∫ βv0 ( p)τv0 ( p)δ T ( p) 0 vi dT ( p) β ( p)τ ( p)δUi ( p) 0 d lnτ v0i ( p) dUi ( p) 0 v ps i =1 0 N ps + ∑∫ 0 v i =1 The effective temperature profile of the i th absorbing gas: 0 dUi ( p) 0 d τ ln vi ( p) o dUi ( p) dT 0 ( p) β ( p)τ ( p) 0 δ Ui ( p) d lnτv0i ( p) dUi ( p) 0 v 0 v The effective temperature profile of the i th absorbing gas: d T ( p) δ Ti ( p ) ≡ δ T ( p ) − δ U i ( p ) d U i0 ( p ) dUi ( p) dT 0 ( p) δ Ti ( p) ≡ δ T ( p) 0 −δ Ui ( p) 0 dUi ( p) dUi ( p) Final Linear Form N δ Rv ≈ β ( ps )τ ( ps ) δ Ts − ∑ ∫ βv0 ( p) δTi ( p)τ v0 ( p) d lnτ v0 ( p) 0 v 0 v i =1 ps 0 i d T ( p) δ Ti ( p ) ≡ δ T ( p ) − δ U i ( p ) d U i0 ( p ) dUi ( p) dT 0 ( p) δ Ti ( p) ≡ δ T ( p) 0 −δ Ui ( p) 0 dUi ( p) dUi ( p) d U i ( p) 0 0 d U ( p ) ⎡ ⎤ i U i ( p) = U i0 ( p) + [T ( p) − Ti ( p)] ⎣ T ( p ) − T ( p ) ⎦ d T 0 ( p ) + U i ( p ) d T ( p) d U i0 ( p ) 0 0 ⎡ = U i ( p) + T ( p ) − Ti ( p ) ⎤⎦ 0 ⎣ d T ( p) A special case: T ( p) = T 0 ( p) ⇒ d Ui0 ( p) U i ( p ) = U ( p) + [T ( p) − Ti ( p)] d T ( p) 0 i The general case: T ( p ) ≠ T 0 ( p ) 1 Ui ( p) = 0 × T ( p) − T ( p) ∫{ p 0 ⇒ } dUi0 ( p′) 0 dT 0 ( p′) ⎡⎣T ( p′) − Ti ( p′)⎤⎦ U ( p) + d p′ 0 dT ( p′) d p′ 0 i The retrieval quality of absorbing gas profiles depends on the quality of temperature first guess! Matlab example The exact analytical Jacobians approach vs. the tangent-linear & adjoint approach 1. Same numerical precision! 2. The exact analytical Jacobian approach is several times faster !! Example: T T0 A ( A+ B ) (1− Td0 ) V p (Td ) = C ( ) e Td Compute δ Vp = dV p dTd δ Td Source: Paul van Delst Why Use Exact Analytical Jacobians in Retrieval and Data Assimilation? • Physical Retrieval (1D-Var): C ost ( X ) = R obs − R ( X ) or C ost ( X ) = R obs − R ( X ) + λ X − X 0 • NWP Data Assimilation (3D/4D-Var): Cost ( X ,...) = Atmospheric Dynamic Term( X ,...) + R obs − R( X ) ~ ideal choice for multispersptral sounders (e.g. HIRS, GOES, MODIS) ~ an underdetermined problem (with respect to a typical forward model) Cost ( X ,...) = Atmospheric Dynamic Term( X ,...) + X retrieval − X ~ ideal choice for hyperspctral sounders (e.g. AIRS, IASI, GIFTS) ~an overdetermined problem (with respect to a typical forward model) • • Atmospheric dynamic term is basically governed by the Navier-Stokes equation, which has no analytical Jacobians. Thus, its tangent-linear/adjoint models are needed for data assimilation. The exact analytical Jacobians for the widely-used McMillin-Fleming-Eyre-Woolf type of radiative transfer/forward models (e.g. RTTOV, RTIASI) are derivable (Huang et al., Applied Optics, 2002). Summary 1. The classical derivation of the linear form of the RTE by Smith et al. (1991) is reviewed. Its dual form is derived. 2. The original linear form appears to be a special case of its dual form when the temperature first guess happens to be the true temperature profile. 3. Linear forms with inexact analytic Jacobians make retrieval results unreliable! 4. The exact analytic Jacobians implementation is an efficient alternative to the tangent-linear/adjoint models for hyperspectral retrieval and data assimilation problems with the widely-used McMillin-Fleming-Eyre-Woolf type of forward models (e.g. RTTOV, RTIASI). Future Work The Remote Sensing GENOME (Geometrical Exploration of Nonlinear Optimization in Measurement Environment) Project: Unveiling the Radiance Hyperspace for Quantifying Geophysical Retrievals The remote sensing GENOME project aims to solve the following long-standing fundamental problems in passive remote sensing Given a sensor specification, its forward model and exact Jacobians, to quantify ¾ the information content of each channel, i.e., the best expected contribution from each channel to the retrieval of temperature and absorbing gases at each pressure level, ¾ the information content of a sensor, i.e., the best expected retrieval accuracy that sets the statistical limit for all retrieval algorithms, ¾ the error of a fast model in terms of the degradation of the best expected retrieval accuracy, as compared to its LBL counterpart, ¾ the impact of sensor noise level on the best expected retrieval accuracy, ¾ the “first guess tolerance” – a safety measure beyond which no retrieval algorithm can reach the best expected retrieval, and ¾ the “retrieval efficiency” – a robustness measure for any retrieval algorithm, as compared to the best expected retrieval. Applications: Lossy compression retrieval impact studies: to conclude the retrieval degradation (due to lossy compression) by the best expected retrieval accuracy that sets the limit for all possible retrieval algorithms. ¾ Optimal channel selection for retrieval with partial channels: to relieve the computational burden in retrieval and data assimilation with the minimum retrieval degradation for hyperspectral sounders (e.g. AIRS, IASI, GIFTS). Future sensor design & trade-off study for risk reduction: to assess the information content (the best expected retrieval accuracy) of a sensor designed with various spectral ranges, ILS resolutions, and noise levels. Different living species have different genomes. So do different sensors!