How to make composites out of multiple observations ?
Thierry Dudok de Wit1, Greg Kopp2 1LPC2E, CNRS/University of Orléans, 2LASP, Boulder
"
"
No composites without data
stitching…
"
How to handle data gaps ?
"
"
The TSI composite
"
"
"
"
"
"
Total Solar Irradiance (TSI) records overlap but disagree in their absolute level. Merging them into one
single composite is a crucial but also very challenging task. This has also become a topic of
considerable debate [Krivova et al., Fröhlich & Lockwood, Willson & Scafetta, etc.].
Data gaps are a problem when computing the wavelet transform. However, they can be easily filled in by
expectation-maximization [Dudok de Wit, A&A 533 (2011)]. We flag them, so that they do not affect the final
outcome.
"
Rationale: all TSI records emanate from observations of the same Sun with instrument measurement noise
added.
Next, to build the composite: 1) Do a weighted average of the wavelet coefficients by using a Bayes scheme to determine the optimal
weights (based on the uncertainties given for each record). 2) Inverting the wavelet transform gives the Bayes composite and its confidence interval.
"
1374
"
"
"
1370
ACRIM 1
ACRIM 2
ACRIM 3
ERBS/ERBE
NIMBUS/ERB
PMO 6
DIARAD
TIM
1368
1366
1364
1362
1362
Some of the TSI observations
made since 1980.
1360
"
"
*Bayes ± 1
*ACRIM
*SARR
*PMOD
ACRIM3
TSI [arb units for offset]
"
1372
TSI [W m−2]
"
1362.5
1358
1980
1985
1990
1995
year
2000
2005
VIRGO
TSI [W/m2]
"
ERBS
Bayes composite
(green) and 3 common
composites
1361.5
1361
TIM
1360.5
2010
"
Jan02
What is the best strategy ?
take a weighted average
Apr02
Jul02
Oct02
Jan03
year
Apr03
Jul03
Example of observed TSI records (color) and their interpolation (grey),
based on the expectation-maximization technique.
for each day, select the least noisy instrument
1360
1975
Oct03
1980
1985
1990
1995
year
2000
2005
2010
"
"
for each day, select your favorite instrument
How does each TSI record compare with our composite ?
take a simple average
"
none of the above
"
• work in wavelet domain rather than in the time domain: compute the wavelet transform to convert
each record into a series of wavelet coefficients
• for each day, merge the wavelet coefficients by using a Bayesian approach = use all the available information - don’t discard any data
• do an inverse wavelet transform to get back in the time domain. End result is the Bayes composite and
its confidence interval (posterior probability distribution)
"
Multiscale analysis is needed
"
"
"
Using a discrete wavelet transform, we convert each record into a series of time-dependent wavelet
coefficients (time-scale decomposition).
0
−0.2
−0.4
ACRIM3
VIRGO
TIM
*SARR
−0.6
−0.8
−1
−1.2
−1.4
2008
1.5
year
2009
1
0.5
wavelet
decomposition
ACRIM3
VIRGO
TIM
*SARR
0
−0.5
"
"
"
"
• The Bayesian approach has been successfully used before for paleoclima@c reconstruc@ons [Tingley et al., J. Climate 23 (2010)]. Here we advocate the same approach for merging TSI records because it is rigorous and naturally incorporates all available informa3on.
"
• By merging of the observa@ons in wavelet space, we overcome problems caused by discon@nui@es in @me & are able to pick out at each scale those records which are most relevant.
"
• The Bayes composite we obtain (s@ll preliminary) indeed shows be4er noise proper3es than other composites.
"
• This approach is now being considered by an ISSI team (lead by Greg Kopp) that will deliver a new TSI composite. It will also be used for merging SSI records in the SOLID project (see talks by Margit Haberreiter & Micha Schöll).
"
"
S3ll in progress
"
• A fully Bayesian scheme requires a lot of computa@on & mathema@cs. We’re s@ll working on that… and this will take @me
"
• This method requires realis3c confidence intervals for the observa3ons (@me-­‐dependent or not), which hardly exist. We’re now developing our own (empirical) scheme for es@ma@ng such confidence intervals.
ACRIM1
ACRIM2
"
2
ACRIM3
1
ERBS
Th residuals = difference
between observations and
the Bayes composite. 0
PMO6
DIARAD
red = observed
grey = interpolated
TIM
*ACRIM
"
*SARR
"
*PMOD
"
1980
1985
1990
1995
year
2000
2005
2010
"
Before 1990 the VIRGO and ACRIM composites don’t describe the most probable value of the TSI but
rather the upper and lower edges of the distribution. The agreement is better for the last solar cycle,
except for the SARR composite.
"
"
The Bayes composite has lower high-frequency noise than each individual record
"
"
1
Spectrum of TSI records − normalised to same total power
10
"
"
"
"
"
"
"
0
10
10
−2
10
−3
10
−4
"
Power spectral densities of
all records (estimated by
windowed Fourier
transform).
−1
"
"
−1
−1.5
0
Conclusions
HF
residuals [W/m2]
arbitrary offset
The approach we advocate
"
"
"
"
"
"
residual error vs Bayesian composite
power spectral density [a.u.]
"
2015
10
ACRIM1
ERBS
HF
ACRIM2
HF
ACRIM1
ACRIM2
ACRIM3
ERBS
PMO6
DIARAD
TIM
Bayes
−2
10
ACRIM3
PMO6
DIARAD
TIM
Bayes
−1
10
frequency [1/day]
0
10
The Bayes composite exhibits a lower noise floor at high frequency (daily-weekly variations).
100
200
coefficient number
300
400