Validated data and removal of bias through
Traceability to SI units
Nigel Fox
Centre for Optical and Analytical Measurement
Dec 03
Resolution adopted by CEOS Plenary 14 (Nov 2000)
 1/ All EO measurement systems should be verified
traceable to SI units for all appropriate measurands.

2/ Pre-launch calibration should be performed using
equipment and techniques that can be demonstrably
traceable to and consistent with the SI system of units,
and traceability should be maintained throughout the
lifetime of the mission.
Traceability – Property of the result of a measurement or the value of a
standard whereby it can be related to stated references, usually through an
unbroken chain of comparisons all having stated uncertainties
Vocabulary for International Metrology (VIM)




SI units – The coherent system of units
adopted and recommended by the
General Conference of Weights and
Measures (CGPM).


Accuracy of measurement – Closeness ofParameter, associated with the
the agreement between the result of a result
 Uncertainty
of measurement
–
of a measurement,
that
measurement and a true value of the
Parameter, associated with the result of
characterises
the dispersion of the
a measurement, that characterises the
measurand.
values
that could
be
dispersion
of thereasonably
values that could
reasonably
be attributed
to the
attributed
to the
measurand.
Precision
– No –metrological
Error
of measurement
Result of a
measurand.
measurement
minus a to
truestate
value that
of theit
definition except
measurand
 Stability – Ability of a measuring
should never be used in the context
instrument to maintain constant its
Precision
– No metrological
definition
of “accuracy”
and, because
of
metrological characteristics with time.
except to state that it should never be
possible confusion its use, should

Reproducibility of results of measurements –
Closeness of the agreement between the
results of measurements of the same
measurand carried
out under changed
Uncertainty
of measurement
–
conditions of measurement.
used in the context of “accuracy” and,
 Traceability – Property of the result of a
normally
be avoided
in metrological
because
of possible
confusion
its use,
measurement or the value of a standard
applications.
should
normally be avoided in
whereby it can be related to stated
metrological applications.
references, usually through an
unbroken chain of comparisons all
Repeatability of results of measurements –
having stated uncertainties.
closeness of the agreement between the
results of successive measurements of
the same measurand carried out under
the same conditions of measurement.
Convention of the Meter – established 1875
CPGM
Conference Generale des Poids et Mesures
66 member & associate states
CIPM
Comite International des Poids et Measures
BIPM
Bureau de International
des Poids et Mesures
CCPR
Consultative Committee
for Photometry and Radiometry
CCT
Consultative Committee
for Thermometry
RMO
Regional Metrology Organisation
CCM
Consultative Committee for
Mass and Related Quantities
CC...
SI Traceability: The Mututal Recognition Arrangement (MRA)
CCPR
SIM
CCPR Key comparisons
• Spectral Irradiance
• Spectral Responsivity
• Luminous intensity
• Luminous Flux
• Spectral transmittance
• Spectral diffuse reflectance
(total hemispherical)
Monitoring and interpreting the Earth’ systems
Incoming Solar Radiation
Drives all the processes of the Earth
System and potentially damaging (UV)
to Biosphere (Human health)
Solar Reflected Radiation
Atmosphere - Aerosol (size & distribn)
- Clouds
- Pollution
(impact on health)
Water - Pollution (originator)
- Algae plumes
Land - Useage / condition
- Type/quantity of vegetation
- Minerals
- Carbon & hydrological cycles
Governments – Treaties, Tax, Planning
Spatial variability requires good
stability and SNR (signal to noise
ratio) from a single sensor - but
long term studies “climate change”
need accuracy and consistency
Engineering specification of SNR
should equate to accuracy
Thermal Emitted Radiation
Atmosphere – Atmospheric chemistry
Water
– Temperature
Land
– Fires, Volcanoes, Pollution,
Need
Need for
for improved
improved Quality
Quality Assurance
Assurance
Anomalies in NOAA/AVHRR data
N-6 N-7
N-9
N-11
N-14
Requirement
- baseline for climate studies
- improve models
- prediction of weather systems
- identify crops from weeds
Total solar Irradiance (Solar
Difficulties
constant) – only normalisation
- bias between
allows asensors
long term record to be
established (Biases are 10 X
- instruments
change
on launchtoand
larger than
necessary
detect
impact
on climate
degrade
in-orbit
(gain andchange)
spectral)
- application of correction for atmosphere
loss
- global warming - Man or Nature?
- lack of cohesion between networks and
- detection of change
Normalised Difference Vegetation “ground truth” validation data (atmosphere
Index (NDVI)
over “stable” desert asan exception)
- monitoring
the treaties
measured
AVHRR
- auditing by
carbon
sinks
- models inadequate
efficiency of carbon
–- Demonstrates
bothsinks
in-flight
“ageing” and initial calibration biases
- QA of operational services (GMES)
- no consistent statements of uncertainty or
LOS 1998 IEEE Trans G & RS
- instrument
synergychange difficult to identify
confidence.
Temporal
even using
“identical” instrumentation without normalisation
To strengthen the evidence
Traceability chain for optical radiation measurement
Electrical Substitution Radiometry a 100 yr old technology
When thermometer
temperature T=To=TE then
Po=PE
Optical power
=Po
Absorbing black
coating
Electrical Heater Power = PE
Copper disk
Thermal shroud
Optical power
=Po
When T =To=TE then
Po=PE
Mechanical cryogenic cooler
“Fridge” (T = 20 K)
Shutter
Cooling improves sensitivity
by 1000 X
Absorbing cavity (~ 0.99999)
Electrical Heater Power = PE
Principle of Cryogenic radiometry
25 yrs of cryogenic radiometry at NPL
Fundamental constants (SI)
Primary standard lamp
Satellite Pre-flight
Lamp performance:
FELs (approx 1 in 3 show
a deviation)
Calibration
Primary standard
cryogenic radiometer
Spectroradiometer
Traceability ??
Laser
Cal interval ~100nm
4.00%
4.5%
2.00%
Cal interval ~0.1 nm
-4.00%
-6.00%
Filter Radiometer-8.00%
-10.00%
Radiance Temperature
-12.00%
-14.00%
Ultra High Temperature
Black Body (3500 K)
Radiance continuum
(Planck)
Primary Scale
4.0%
3.5%
Cal Lab Primary lamp
3.0%
Spectroradiometer
BB radiance
Solar
Lamp
SRIPS Repeatability
illuminated
UVFW modelling
Diffuser
2.5%
2.0%
Vicarious
Cal Lab working std Lamp
1.5%
Spectroradiometer
1.0%
250 0.5% 500
0.0%
250
750
500
1000
1250
1500
User Cal Lamp
Wavelength
750
1000
1750
1250 1500 1750
Wavelength (nm)
User Instrument
LAND
OCEAN
Atmosphere/
Model
2000
2250
2000
2250
Data products
Spectroradiometer
(multi-band filter radiometer
Spectral Radiance/Irradiance
calibrations
Satellite In-flight
Calibration
Spectroradiometer
Radiance uncertainty (%)
Difference
Photodiode
(spectral responsivity0.00%
-2.00%
Laser
SRIPS primary scale uncertainty:
Working
lamp
Overall
at standard
95% confidence
level
ATMOSPHERE
2500
2500
Traceability for in-flight / in-situ / vicarious calibration
Diffuse reflectance (BRDF)
Spectral Radiance
The NPL diffuse
reflectance scale is
derived goniometrically
for the spectral region
300 to 2500 nm
- lamp illuminated spheres
- Filter radiometers (spectroradiometers)
- Irradiance source + diffuser
Via models / atmosphere
correction
Lamp + spectralon or ….
Uncertainty of <0.2 % in
to visible
satellite
for
cal/val
the
and
shown
equivalence with NIST
Sun + spectralon or ….
(radiances)
Sun + Moon
Spectralon 400 nm
Detector
To bio/geophysical
quantities
Sample
(refelectances)
1.05
radiance factor
Spectral Reflectance
-0.95in-situ absolute ratio (using radiometers)
-0.85Ratio to “standard” reflector/diffuser
Beta-NPL (400 nm), Tile 1
Beta-PTB (400 nm), Tile 1
0.75
Beta-NPL (400 nm), Tile 2
Beta-NIST (400 nm), Tile 2
0.65
-90
-60
-30
0
30
60
90
Validated data products
require all processing
steps and data to be QA
– Accredited?
 Pre-flight
User specification
- Instrument build compliance
- Calibration?
 Post-launch
- In-flight checks
- Ground “Truth” comparison
- Inter-sensor cross calibration
 Processed data released
- “validated”
- Uncertainty statement?
-
Rare for all these activities to have been
independently reviewed and/or audited
Global Monitoring Environment and Security (GMES)
Joint initiative of ESA and EU
Aim: to establish “operational services” for Earth Observation data to
meet needs of key stakeholders , public services, private industry,
academia and the citizen with a view to financial self-sufficiency.
Success requires:
- Combination of data from many sources, (satellites, in-situ, aircraft)
- Efficient production of cost effective, reliable, data products / maps
- Data must provide the evidence to allow decisions to be taken with
confidence.
- Innovation in measurement and analysis
Reliability: Implies Quality assurance and statements of confidence
associated with data (not only for end users but also
“operational service” providers
Users generally assume QA
Robust evidence
requires
robust QA
Infrastructure for innovation in measurement, validation and
QA of EO data
• Transfer standards
• Comparisons
Post-launch
airborne
• Measurement & test protocols
• International link
NPL ++
Modelling & Data
processing
• Innovation on techniques
• Independence
NIST ++
Calibration
QA
Traceability
Pre-flight
In-situ
Advice
Public
sector
Audit
Validation
Private
Industry
Academia
Summary
 Primary scales, transfer standards and techniques now allow high
accuracy to be achieved for both pre-flight and vicarious calibration
(particularly for radiance)
 All aspects/steps of producing EO data products needs validation and
traceability (instrument calibration and algorithms/models)
 Consistent presentation and breakdown of uncertainty budgets
 Flexibility to allow innovation
 Regular comparisons to evaluate biases
 Establish well characterised ground targets as “reference standards”
 Develop improved “in-flight” calibration methods e.g TRUTHS
(Fox et al Proc. SPIE 4881, p395 2003)
For Earth Observation to provide the
evidence to support policy
requires the industry and its data to be
as robust as traditional industries