Fiducial Reference Measurements for validation of Surface Temperature from
Satellites (FRM4STS) - Laboratory Calibration of Participants Radiometers and
Blackbodies
Archive of the FRM TIR radiometer calibration and verification data
ESA Contract No. 4000113848_15I-LG
Evangelos Theocharous & Nigel Fox
OCTOBER 2015
Reference
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Issue
1
Revision
1
Date of Issue
30 October 2015
Status
DRAFT
Document Type
D110- ARCH
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
INTENTIONALLY BLANK
Page 2 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Fiducial Reference Measurements for validation of Surface Temperature
from Satellites (FRM4STS): Laboratory Calibration of Participants
Radiometers and Blackbodies
D-110 Archive of the FRM TIR radiometer calibration and verification
data
Evangelos Theocharous & Nigel Fox
Environment Division
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
 Queen’s Printer and Controller of HMSO, 2015
National Physical Laboratory
Hampton Road, Teddington, Middlesex, TW11 0LW
Page 4 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
CONTENTS
DOCUMENT VERSION HISTORY
DOCUMENT APPROVAL
APPLICABLE DOCUMENTS .............................................................................................................9
ACRONYMS AND ABBREVIATIONS ............................................................................................10
1. INTRODUCTION ........................................................................................................................11
2. ORGANIZATION ........................................................................................................................11
2.1 PILOT ..........................................................................................................................................11
2.2 PARTICIPANTS .........................................................................................................................12
3. PARTICIPANTS’ INFORMATION ..........................................................................................12
4. RADIOMETER COMPARISON ...............................................................................................12
4.1 TECHNICAL DESCRIPTION OF THE RADIOMETERS ........................................................12
4.2 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON THE
LABORATORY CALIBRATION OF RADIOMETERS ...........................................................15
4.3 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON THE FIELD
CALIBRATION OF RADIOMETERS .......................................................................................20
5 BLACKBODY COMPARISON..................................................................................................25
5.1 TECHNICAL DESCRIPTION OF THE BLACKBODIES ........................................................25
5.2 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON
BLACKBODIES ..........................................................................................................................27
6. REFERENCES .............................................................................................................................30
APPENDIX A: REPORTING OF MEASUREMENT RESULTS ...............................................31
APPENDIX B: REPORTING OF MEASUREMENT RESULTS................................................34
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
DOCUMENT MANAGEMENT
Issue Revision Date of
Issue/revision
1
1
30-Oct-15
Description of Changes
Creation of document
DOCUMENT APPROVAL
Contractor Approval
Name
Role in Project
Dr Nigel Fox
Technical Leader
Mr David Gibbs
Project Manager
Signature & Date (dd/mm/yyyy)
CUSTOMER APPROVAL
Name
Role in Project
C Donlon
ESA Technical Officer
Signature
Date (dd/mm/yyyy)
Page 8 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
APPLICABLE DOCUMENTS
AD Ref.
EOPSM/2642
Ver.
/Iss.
1
Title
Fiducial Reference Measurements for Thermal Infrared Satellite
Validation (FRM4STS) Statement of Work
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
ACRONYMS AND ABBREVIATIONS
AMBER
CEOS
DMI
IR
ISO
KIT
LST
NIST
NMI
NPL
PTB
RMS
SST
SI
WGCV
WST
Absolute Measurements of Black-body Emitted Radiance
Committee on Earth Observation Satellites
Danish Meteorological Institute
Infra-Red
International Organization for Standardization
Karlsruhe Institute of Meteorology
Land Surface Temperature
National Institute of Standards and Technology (USA)
National Metrology Institute
National Physical Laboratory
Physikalisch Technische Bundesanstalt
Root Mean Square
Sea Surface Temperature
Système Internationale
Working Group for Calibration and Validation
Water Surface Temperature
Page 10 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
1. INTRODUCTION
The measurement of the Earth’s surface temperature is a critical product for meteorology and an
essential parameter/indicator for climate monitoring. Satellites have been monitoring global surface
temperature for some time, and have established sufficient consistency and accuracy between in-flight
sensors to claim that it is of “climate quality”. However, it is essential that such measurements are fully
anchored to SI units and that there is a direct correlation with “true” surface/in-situ based measurements.
The most accurate of these surface based measurements (used for validation) are derived from field
deployed IR radiometers. These are in principle calibrated traceably to SI units, generally through a
reference radiance blackbody. Such instrumentation is of varying design, operated by different teams
in different parts of the globe. It is essential for the integrity of their use, to provide validation data for
satellites both in-flight and to provide the link to future sensors, that any differences in the results
obtained between them are understood. This knowledge will allow any potential biases to be removed
and not transferred to satellite sensors. This knowledge can only be determined through formal
comparison of the instrumentation, both in terms of its primary “lab based” calibration and its use in the
field. The provision of a fully traceable link to SI ensures that the data are robust and can claim its status
as a “climate data record”.
The “IR Cal/Val community” is well versed in the need and value of such comparisons having held
highly successful exercises in Miami and at NPL in 2001 [1, 2] and 2009 [3, 4]. However, six years
will have passed and it is considered timely to repeat/update the process. Plans are in place for the
comparisons to be repeated in 2016. The 2016 comparison will consist of five stages:
i.
ii.
iii.
iv.
v.
Laboratory comparisons of the radiometers and reference radiance blackbodies of the
participants.
Field comparisons of Water Surface Temperature (WST) scheduled to be held at Wraysbury
fresh water reservoir, near NPL.
Field comparisons of Land Surface Temperature (LST) scheduled to be held on the NPL
campus.
Field comparisons of Land Surface Temperature (LST) scheduled to be held at two sites
(Gobabeb Training and Research Centre on the Namib plain and the “Farm Heimat” site in
the Kalahari bush) in Namibia in 2016.
Field comparisons of Ice Surface Temperature (IST) scheduled to be held in the Arctic.
During the 2016 comparison, NPL will act as the pilot laboratory and will provide traceability to SI units
during laboratory and “field” comparisons at and around NPL. NPL will be supported by the PTB
providing traceability to SI units during laboratory measurements. This document describes how the
calibration and verification data of the laboratory FRM TIR radiometers and blackbodies will be
established and archived, following good data stewardship practices, which will provide access to
records by research on request.
2. ORGANIZATION
2.1 PILOT
NPL, the UK national metrology institute (NMI) will serve as pilot for the 2016 laboratory comparisons,
supported by the PTB, the NMI of Germany. NPL, the pilot, will be responsible for inviting participants
and for the analysis of data, following appropriate processing by individual participants. NPL, as pilot,
will be the only organisation to have access and to view all data from all participants. This data will
remain confidential to the participants and NPL at all times, until the publication of the report showing
results of the comparison to participants.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
2.2 PARTICIPANTS
The list of the potential participants, based on current contacts and expectation who will be likely to take
part can be found elsewhere [5]. Dates for the comparison activities are also provided in ref 5. A full
invitation to the international community through CEOS and other relevant bodies will be carried out to
ensure full opportunity and encouragement is provided to all. All participants should be able to
demonstrate independent traceability to SI of the instrumentation that they use, or make clear the route
of traceability via another named laboratory.
By their declared intention to participate in this key comparison, the participants accept the general
instructions and the technical protocols written down in this document and commit themselves to follow
the procedures strictly. Once the protocol and list of participants have been reviewed and agreed, no
change to the protocol may be made without prior agreement of all participants. Where required,
demonstrable traceability to SI will be obtained through participation of PTB and NPL as pilot.
The laboratory comparison of the radiometers and blackbodies of the participants will take place at NPL
during the week beginning 20th June 2016. Full details of this comparison can be found elsewhere [5].
During the laboratory comparison, the participating radiometers will measure the brightness temperature
of the NPL ammonia heat-pipe reference blackbody, while the NPL transfer radiometer (AMBER) [6]
and the PTB broad band infrared radiometer will be used to measure the brightness temperature of the
participating blackbodies at 10.1 µm and in the 8 µm to 14µm wavelength regions, respectively.
3. PARTICIPANTS’ INFORMATION
Each organisation which will be participating in this comparison will have its own directory on the
website which will include the following:
A. Name of the organisation: e.g. the Mediterranean Centre for Environmental Studies
(Fundación Centro de Estudios Ambientales del Mediterraneo, CEAM)
B. Contact person(s) within that organisation responsable for the measurements: e.g. Dr
Raquel Niclos
C. Contact information/Address: e.g. 14 Charles Darwin, 46980 PATERNA, Valencia, Spain.
D. Website of organisation: e.g. www.ceam.es
E. Email of contact person(s): e.g. niclos@ceam.es
F. Summary of the activities of the organisation in this field:
4. RADIOMETER COMPARISON
4.1 TECHNICAL DESCRIPTION OF THE RADIOMETERS
A.
Technical descriptions of the radiometers which will take part in the 2016
comparisons will be included in this Section
E.g.: The radiometer uses a thermopile detector to provide a millivolt output which is
dependent on the temperature difference between the target being viewed and the radiometer
body temperature. A thermistor is used to measure the temperature of the sensor body, which
is used to reference the target temperature. The radiometer operates in the 8 m to 14 m
Page 12 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
spectral band. The radiometer uses a germanium lens to image the target being viewed and has
a field of view of 18º half-angle.
Radiometer type and model
APOGEE Model IRR-PN
FOV
Spectral band (μm)
18° half angle
8-14
Full information on this radiometer can be found at:
http://www.apogeeinstruments.com/infrared_temperature_sensors.htm
B. References of description and work done using this radiometer
E.g.: Currently, the radiometer is being used for field measurements (hand held and mounted
on meteorological towers) of land surface temperature and emissivity, both for research on
surface thermal emission properties and for testing thermal infrared products. References to the
work done using this radiometer:
1. Campbell Scientific and Apogee Instruments, 2007, “IRR-P Precision Infrared Temperature
Sensor”, Instruction manual. 10pp.
2. Bugbee, B., Blonquist, M. and Lewis, N., “Design and Calibration of a New Infra-red
Radiometer (website)”
C. Description of the traceability route for the primary calibration of the radiometer.
E.g. According to the manufacturer of the radiometer (Apogee Instruments Inc.), the radiometer
was calibrated using a blackbody, with k = 1 combined uncertainties shown below:
Combined uncertainty of ±0.2 K in the 263 K to 338 K range and
Combined uncertainty of ±0.5 K in the 233 K to 343 K range.
The custom calibration by the manufacturer resulted with a unique calibration coefficient for
the radiometer (calibration date: February 2016).
Because of the limited detail about the traceability of this primary calibration, the institute
carried out its own uncertainty analysis which was based on laboratory measurements using a
blackbody
Model
P80P
http://landinstruments.net/infrared/products/calibration_sources/p80p.htm during April, 2016.
Calibration measurements were performed in the laboratory following the technical procedures
described in the Draft Protocol. The Landcal Blackbody Source P80P was set at four
temperatures (278 K, 283 K, 293 K and 303 K) in 3 different runs. Enough time was allowed
for the blackbody to reach equilibrium at each temperature. Radiometers were aligned with the
blackbody cavity and placed at a distance so that the field of view of the radiometer under-filled
the blackbody aperture. The values of the component uncertainties reported in Table 1 are
typical for the range of blackbody temperatures being considered.
D. Date of the last primary calibration of the radiometer.
E.g. 18th February 2016
E. Uncertainty budget associated with measurements completed with the radiometer.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Table 1 tabulates the uncertainty contributions associated with measurements completed with the
APOGEE IRR-PN radiometer at k=1:
-
Repeatability: Typical value of the standard deviation of 180 measurements (1 per second, the
radiometer response time is shorter than 1 s) at a fixed blackbody temperature without
radiometer re-alignment. A repeatability value of ±0.05 K was measured in the 263 K to 338 K
range while the value of repeatability increased to ±0.1 K in the 238 K to 343 K temperature
range.
-
Reproducibility: Typical value of the standard deviation of three measurements of the reference
blackbody maintained at constant temperature, including re-alignment between successive
measurements.
-
Primary calibration: Combined uncertainty of the primary calibration of the APOGEE IRRPN radiometer as given in the calibration certificate of the primary calibration.
-
Drift since calibration: 0.01 K estimated since last primary calibration of the APOGEE IRRPN radiometer.
Table 1, is indicative of the component uncertainties associated with the calibration of the APOGEE
IRR-PN radiometer. It should be noted that some of these components may sub-divide further depending
on their origin. The Root Mean Square (RMS) total refers to the usual expression i.e. square root of the
sum of the squares of all the individual uncertainty terms as shown in the example for Type A
uncertainties.
Table 1: Uncertainty budget for the IRR-PN - APOGEE Instruments radiometer
Uncertainty Contribution
Repeatability of
measurement (1)
Reproducibility of
measurement (2)
Type A
Uncertainty in
Value / %
Type B
Uncertainty in
Value /
(appropriate
units)
Uncertainty in
Brightness temperature
K
URepeat
0.009K / 0.003%
URepro
0.06K / 0.019%
Primary calibration(3)
0.15 K
UPrim
Linearity of radiometer
0.04 K
ULin
Drift since calibration
0.01 K
UDrift
Ambient temperature
fluctuations
0.007 K
Uamb
Atmospheric
absorption/emission
0.003 K
Uatm
RMS total
((Urepeat)2+(URepro)2))½
(1) Typical value of the standard deviation of 180 measurements at fixed black body temperature without
re-alignment of radiometer.
Page 14 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
(2) Typical value of the standard deviation of three measurements of the reference blackbody maintained
at constant temperature, including re-alignment between successive measurements.
(3) Combined uncertainty of the primary calibration of the APOGEE IRR-PN radiometer as given in the
calibration certificate.
F. Sea surface temperature measurements
The Type A and Type B uncertainty contributions of the APOGEE IRR-PN radiometer in measuring
the land surface temperature are shown below:
Apogee IRR-PN
Operating
wavelength
µm
Uncertainty
Type A
K
Uncertainty
Type B
K
11.5 µm
0.02 to 0.18
0.21
4.2 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON THE
LABORATORY CALIBRATION OF RADIOMETERS
The results of the measurements of the brightness temperature of the reference blackbody by the
participating radiometers should be reported on the form shown in Appendix A of this document. The
measurements should also be provided in an excel file in the form shown in Table 2 shown below. These
results will be available on the project’s website along with the corresponding measurements of the
brightness temperature of the reference blackbody based on the temperature measurements made using
Platinum Resistance Thermometers. Table 3 shows how the brightness temperature of the reference
blackbody made by the pilot will be reported on the website.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Table 2: Format with which the measurement of the brightness temperature of the reference
blackbody made by a participant should be reported. The table caption should include the date
of the measurement (e.g. 22nd June 2016)
Time
UTC
Radiometer measurement
K
Combined uncertainty
K
13:22:01
13:22:31
13:23:01
13:23:33
13:24:12
13:24:42
13:25:12
13:25:42
13:26:12
13:26:42
13:27:12
13:27:42
13:28:12
13:28:42
13:29:12
13:29:42
13:30:12
13:30:42
13:31:12
13:31:42
13:32:12
13:32:42
13:33:12
13:33:42
13:34:12
13:34:42
13:35:12
13:35:42
13:36:12
13:36:42
13:37:12
13:37:42
13:38:12
13:38:42
13:39:12
13:39:42
13:40:12
13:40:42
20.158
20.159
20.166
20.159
20.151
20.151
20.146
20.146
20.146
20.148
20.145
20.151
20.151
20.147
20.140
20.134
20.140
20.150
20.152
20.149
20.148
20.143
20.144
20.123
20.131
20.136
20.136
20.131
20.129
20.130
20.132
20.133
20.140
20.145
20.145
20.140
20.133
20.132
0.031
0.031
0.031
0.035
0.031
0.033
0.033
0.033
0.034
0.033
0.033
0.033
0.029
0.030
0.034
0.033
0.032
0.032
0.035
0.033
0.031
0.032
0.030
0.030
0.033
0.032
0.035
0.034
0.033
0.031
0.030
0.033
0.033
0.033
0.033
0.033
0.034
0.033
Page 16 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
Table 3: Format with which the measurement of the brightness temperature of the reference
blackbody made by the pilot will be reported. The table caption should include the date of the
measurement (e.g. 22nd June 2016).
Time
UTC
Blackbody brightness
temperature
K
Combined
uncertainty
K
13:22:32
13:23:20
13:24:08
13:24:56
13:25:44
13:26:32
13:27:20
13:28:08
13:28:56
13:29:43
13:30:31
13:31:19
13:32:07
13:32:55
13:33:43
13:34:31
13:35:19
13:36:07
13:36:55
13:37:43
13:38:31
13:39:19
13:40:07
13:40:55
13:41:43
13:42:31
13:43:19
20.138
20.130
20.109
20.093
20.101
20.116
20.117
20.122
20.115
20.097
20.107
20.119
20.117
20.127
20.103
20.112
20.108
20.090
20.093
20.107
20.104
20.095
20.083
20.100
20.104
20.117
20.132
0.035
0.038
0.041
0.043
0.041
0.05
0.038
0.034
0.042
0.036
0.03
0.027
0.031
0.039
0.043
0.037
0.031
0.029
0.029
0.036
0.044
0.036
0.043
0.035
0.036
0.034
0.033
The website will also show graphical plots of the measurements made by each participant as a function
of time. These plots will include the actual brightness temperature of the reference blackbody, as
measured by the test radiometer, as shown in Figure 1. The average value of the measurements made by
the test radiometer (20.109 oC in this example) and the corresponding average value of the actual
blackbody temperature (20.141 oC) will also be given on the website. The difference between the two
average values (0.032 oC in this example) will also be given, to indicate the “error” with which the test
radiometer measures the brightness temperature of the reference blackbody at that particular temperature
setting.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
20.2
BB Temperature
20.18
Radiometer Reading
20.16
Temperature/C
20.14
20.12
20.1
20.08
20.06
20.04
20.02
20
13:19:12
13:26:24
13:33:36
13:40:48
13:48:00
Time (UTC)
Figure 1: Plots of the measurements made by every participant radiometer along with the actual
brightness temperature of the blackbody which was being monitored, will be given on the website for
all laboratory measurements completed in this comparison. Figure 1 shows how the measurements will
be presented graphically.
The website will also show tables which summarise the differences/“errors” of the measurements made
by participating radiometers of the brightness temperature of the reference blackbody at different
temperatures from the actual blackbody cavity temperature. Table 4 shows an example of such a Table
from the 2009 radiometer comparison which was completed at NPL [3]. Finally the website will show
these differences in a graphical presentation; Figure 2 shows the plot of the differences of the mean of
the radiometer readings from the mean of the temperature of the NPL reference blackbody during the
same period (blue circles). The blackbody was maintained at a nominal temperature of 10 oC during the
acquisition of the measurements shown in Figure 2. The red squares in the same Figure show the points
corresponding to when the University of Miami blackbody (also maintained at a nominal temperature
of 10 oC) was being viewed by the same radiometer.
Page 18 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
Table 4: Difference of the mean of the measurement of the brightness temperature of the NPL
reference blackbody completed by the participants during the 2009 comparison at NPL from the actual
blackbody temperature. Similar tables will be created for the 2016 laboratory comparison sctivities.
Set
Temperature
o
C
NPL VTBB
1st measur.
mK
NPL VTBB
2nd measur.
mK
GOTA
30
1147
8-14 μm
20
9
79
10
-1276
-1165
Set
Temperature
o
C
NPL VTBB
1st measur.
mK
NPL VTBB
2nd measur.
mK
Valencia DEPT
30
-108
CE312-1-Unit 1
20
118
79
10.7 μm
10
321
292
5
416
Valencia DEPT
30
-160
5
-2137
GOTA
30
67
11.35 μm
20
189
189
CE312-1-Unit 1
20
221
160
10
254
235
12 μm
10
464
442
5
293
5
701
66
GOTA
30
87
Valencia DEPT
30
10.65 μm
20
199
139
CE312-1-Unit 1
20
82
101
10
204
185
10.8 μm
10
140
145
5
243
5
208
372
GOTA
30
-43
Valencia DEPT
30
9.1 μm
20
169
109
CE312-1-Unit 1
20
83
104
10
54
185
8.8 μm
10
-179
-154
5
-209
30
20
10
5
-57
189
423
529
134
366
5
253
GOTA
8.7 μm
30
20
10
5
-93
169
74
253
129
185
Valencia DEPT
CE312-1-Unit 2
10.7 μm
Canary
8.3 μm
30
20
10
5
47
149
44
133
169
125
Valencia DEPT
CE312-1-Unit 2
12 μm
30
20
10
5
14
233
467
557
243
383
IPL
CE312-2
10.54 μm
30
20
-393
162
691
40
956
-104
-298
190
683
14
Valencia DEPT
CE312-1-Unit 2
10.8 μm
30
20
10
5
104
204
289
328
168
237
-278
136
644
74
860
30
-283
154
633
62
Valencia DEPT
CE312-1-Unit 2
8.8 μm
30
20
10
5
-12
225
514
605
153
392
RAL
SISTER
30
20
10
5
-24
-25
-16
-6
-333
150
716
26
888
-16
-178
240
655
-40
Southampton
ISAR
30
20
10
5
6
126
87
77
28
69
-438
74
881
80
1434
206
-507
228
934
125
OUC
ISAR
30
20
10
5
59
94
145
237
48
64
136
279
KIT
Heidronics
KT - 15
-413
225
773
-223
190
30
-208
-632
152
-492
169
896
-93
1502
238
30
20
10
5
-402
173
983
291
1538
486
-197
296
809
127
IPL
CE312-2
11.29 μm
IPL
CE312-2
10.57 μm
IPL
CE312-2
9.15 μm
IPL
CE312-2
8.69 μm
IPL
CE312-2
8.44 μm
10 (uncorrected)
10 (corrected)
5 (uncorrected)
5 (corrected)
30
20
10 (uncorrected)
10 (corrected)
5 (uncorrected)
5 (corrected)
30
20
10 (uncorrected)
10 (corrected)
5 (uncorrected)
5 (corrected)
30
20
10 (uncorrected)
10 (corrected)
5 (uncorrected)
5 (corrected)
30
20
10 (uncorrected)
10 (corrected)
5 (uncorrected)
5 (corrected)
30
20
10 (uncorrected)
10 (corrected)
5 (uncorrected)
5 (corrected)
NPL - Commercial
-33
-19
-23
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
10 oC (corrected)
NPL
RSMAS
1
0.5
0
-0.5
-1
-1.5
DLR
CEAM R5
OUC
KIT
RAL
ISAR
IPL (8.44)
IPL (9.15)
IPL (11.29)
GOTA (8.3)
GOTA (8.7)
GOTA (9.1)
GOTA (10.65)
GOTA (11.35)
GOTA (8 to 14)
Val 2 (8.8)
Val 2 (12)
Val 1 (10.8)
Val 1 (10.7)
-2.5
CEAM R3
-2
CEAM R1
Difference from reference temperature ( oC)
1.5
Radiometer-participant
Figure 2: Plot of the mean of the differences of the radiometer readings from the temperature of the
NPL reference blackbody (blue circles), maintained at a nominal temperature of 10 oC. The red
squares show the points corresponding to the University of Miami blackbody. Similar plots
corresponding to the 2016 comparison will be shown on the comparison website.
4.3 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON THE
FIELD CALIBRATION OF RADIOMETERS
The results of the measurements of the Water Surface Temperature (WST) at Wraysbury reservoir and
Land Surface Temperature (LST) should be reported by the participants on the forms shown in Appendix
B of this document. The measurements should also be provided in excel files in the form shown in
Table 5, shown below. These results will be available on the project’s website along with the
corresponding measurements of other participants. The website will also include excel summary files
showing the processed measurements.
Page 20 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
Table 5: Format with which the WST and LST measurements made during the field comparison
of the radiometers should be reported by a participants. The table caption should include the
angle of view of the radiometer from the Nadir and the wavelength and bandwidth at which the
radiometer is operating.
Time
UTC
WST
o
C
WST
Uncertainty
o
C
Sky Temperature
o
C
Sky Temp.
Uncertainty
o
C
12/05/2009 18:00
12/05/2009 18:10
12/05/2009 18:20
12/05/2009 18:30
12/05/2009 18:40
12/05/2009 18:50
12/05/2009 19:00
12/05/2009 19:10
12/05/2009 19:20
12/05/2009 19:30
12/05/2009 19:40
12/05/2009 19:50
12/05/2009 20:00
12/05/2009 20:10
12/05/2009 20:20
12/05/2009 20:30
12/05/2009 20:40
12/05/2009 20:50
12/05/2009 21:00
12/05/2009 21:10
12/05/2009 21:20
12/05/2009 21:30
12/05/2009 21:40
12/05/2009 21:50
12/05/2009 22:00
12/05/2009 22:10
12/05/2009 22:20
12/05/2009 22:30
12/05/2009 22:40
12/05/2009 22:50
12/05/2009 23:00
12/05/2009 23:10
12/05/2009 23:20
12/05/2009 23:30
12/05/2009 23:40
12/05/2009 23:50
13/05/2009 00:00
13/05/2009 00:10
13/05/2009 00:20
13/05/2009 00:30
13/05/2009 00:40
29.212
29.156
29.162
29.195
29.218
29.249
29.303
29.294
29.231
29.288
29.313
29.346
29.326
29.324
29.346
29.320
29.251
29.330
29.269
29.279
29.281
29.242
29.264
29.213
29.172
29.107
29.074
29.067
28.978
28.913
28.486
28.532
28.578
28.565
28.747
28.818
28.884
28.942
28.909
28.994
28.974
0.123
0.121
0.103
0.096
0.128
0.113
0.115
0.119
0.112
0.090
0.111
0.107
0.109
0.131
0.144
0.122
0.122
0.117
0.101
0.139
0.115
0.132
0.117
0.129
0.124
0.130
0.117
0.147
0.158
0.127
0.158
0.149
0.177
0.165
0.152
0.146
0.128
0.167
0.159
0.074
0.128
-20.788
-20.844
-20.838
-20.805
-20.782
-20.751
-20.697
-20.706
-20.769
-20.712
-20.687
-20.654
-20.674
-20.676
-20.654
-20.680
-20.749
-20.670
-20.731
-20.721
-20.719
-20.758
-20.736
-20.787
-20.828
-20.893
-20.926
-20.933
-21.022
-21.087
-21.514
-21.468
-21.422
-21.435
-21.253
-21.182
-21.116
-21.058
-21.091
-21.006
-21.026
0.198
0.196
0.178
0.171
0.203
0.188
0.190
0.194
0.187
0.165
0.186
0.182
0.184
0.206
0.219
0.197
0.197
0.192
0.176
0.214
0.190
0.207
0.192
0.204
0.199
0.205
0.192
0.222
0.233
0.202
0.233
0.224
0.252
0.240
0.227
0.221
0.203
0.242
0.234
0.149
0.203
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
The website will also show graphical plots of the measurements made by each participant as a function
of time. These plots will include the actual brightness temperature of the target, as measured by the test
radiometer, as shown in Figure 3.The website will also include measurements made by every participant
together plotted together in the same Figure. Figure 4 shows an example of the measurements of four
continuously measuring radiometers during the 2009 comparison. Figure 5 shows the difference of the
four continuously measuring radiometers from their mean value. Similar plots will be produced for the
measurements made by radiometers during the 2016 comparison.
25 deg.
55 deg.
29.5
Temperature/oC
29
28.5
28
27.5
27
5/12/2009 12:00
5/13/2009 12:00
5/14/2009 12:00
5/15/2009 12:00
Time (UT)
Figure 3: Plots of the WST measurements made by a particular participant radiometer, as shown in
this Figure, will be given on the website for all field measurements completed during the 2016
comparison. Measurements are shown for two angles of view, 25o and 55o to the nadir.
Page 22 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
MAERI
29.4
ISAR
29.2
KIT
RAL
29
SST ( oC)
28.8
28.6
28.4
28.2
28
27.8
27.6
12/05/2009 12/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009
19:12
14:24
09:36
04:48
00:00
19:12
14:24
Time (UT)
Figure 4: Plots of the measurements of the four continuously measuring radiometers made during the
2009 Miami comparison. Similar plots will be plotted for the measurements made during the 2016
comparison.
0.4
MAERI
ISAR
Difference from mean/oC
0.3
0.2
KIT
RAL
0.1
0
-0.1
-0.2
-0.3
-0.4
12/05/2009 12/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009
14:24
19:12
00:00
04:48
09:36
14:24
19:12
Time (UT)
Figure 5: Difference of the four continuously measuring radiometers from their mean value.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
The radiometers do not provide continuous field measurements for a number of reasons. Moreover,
measurements are taken at different times, so linear interpolation will be used to calculate the values the
measurements of each radiometer at 5 minute intervals. The output of radiometers which have to be
operated manually has to be treated differently. Figure 6 shows plots of the measurements made by all
the manually operating radiometers during the 2009 comparison. Similar plots corresponding to
measurements made during the 2016 comparison will also be included in the website. Finally the
difference of all the field measurements made by all the radiometers (both continuously monitoring and
manual radiometers) from the mean will also be presented. Figure 7 shows the difference of the
measurements made by every radiometer taking part from the corresponding measurements made by
ISAR during the 2009 SST radiometer comparison. The ISAR radiometer was chosen because it was
the only radiometer which produced continuous measurements during the entire duration of the 2009
SST comparison.
29.5
8 to 14 um
29.3
8.7 um
29.1
8.3 um
Temperature/ o C
9.1 um
28.9
10.65 um
28.7
11.35 um
RA1
28.5
28.3
28.1
RA2
RA3
DLR data
27.9
27.7
27.5
12/05/200 13/05/200 13/05/200 14/05/200 14/05/200 15/05/200 15/05/200 16/05/200
9 12:00
9 00:00
9 12:00
9 00:00
9 12:00
9 00:00
9 12:00
9 00:00
Time (UT)
Figure 6: Plots of the measurements made by all the manually operating radiometers during the 2009
comparison. Similar plots corresponding to measurements made during the 2016 comparison will be
included in the new website.
Page 24 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
0.80
MAERI
KIT
RAL
CEAM RA1
Difference from ISAR (oC)
0.40
0.00
CEAM RA2
CEAM RA3
GOTA 8to14
GOTA 8.7
GOTA8.3
-0.40
-0.80
GOTA9.1
GOTA10.65
GOTA
DLR
-1.20
-1.60
12/05/2009 12/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009
19:12
14:24
09:36
04:48
00:00
19:12
14:24
Time (UT)
Figure 7: Difference of the measurements made by every radiometer from the corresponding
measurement made by ISAR during the 2009 SST radiometer comparison.
5 BLACKBODY COMPARISON
5.1 TECHNICAL DESCRIPTION OF THE BLACKBODIES
A. Blackbody used for the comparison
E.g. Land Infrared Landcal Blackbody Source P80P.
(http://landinstruments.net/infrared/products/calibration_sources/p80p.htm).
B. Technical description of the radiometer
E.g. the radiometer was made of Aluminium and had a black, high temperature refractory coating. The
body of the radiometer was cylindrical with a 50 mm diameter and 155 mm in length. The blackbody
cavity had a 120o cone at the closed end and a cavity emissivity was estimated to be greater than 0.995.
The temperature of the cavity is measured using an internal Platinum Resistance Thermometer (PRT)
whose output was displayed on a digital display with 0.01 K resolution. The temperature of the
blackbody cavity was also measured using an external PRT-100 which was calibrated traceable to
National Standards (UKAS calibration certificate available).
C. Establishment of traceability route and breakdown of uncertainty:
The following error analysis is based on laboratory measurements (April 6 and 7, 2016).
-
Type A uncertainty:
Repeatability: 0.01% or 0.03 K (corresponding to variations of 0.01 Ω in external PRT-100
measurements).
Reproducibility: 0.01% or 0.03 K (corresponding to variations of 0.01 Ω in external PRT-100
measurements).
Total Type A uncertainty (RSS): 0.015% or 0.04 K
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Type B uncertainty
- Emissivity: Uncertainty due to emissivity <0.005 (value provided by the blackbody manufacturer).
This corresponds to a temperature error of 0.3 K in the 8 m to14 m wavelength range.
- Blackbody thermometer calibration: 0.1 K (External PRT calibration at the ice point). Also
comparison with an identical blackbody from the University of La Laguna, Spain, which has a
UKAS certificate.
- Blackbody isothermal variance: 0.3 K (value provided by manufacturer).
- Primary source: 0.6 K. Primary source calibration is not available for this blackbody. The assigned
value is the same for the identical University of La Laguna blackbody (calibrated with UKAS
certificate) corresponding to the uncertainty (coverage factor k=1) of the standard radiation
thermometer used in the blackbody calibration for temperature range from 10 oC to 30 oC.
Total Type B uncertainty: 0.7 K.
Combined Type A and Type B uncertainty: 0.7 K.
D. Operational methodology during measurement campaign
E.g. the blackbody was set to a fixed temperature (ice point at 0.00 oC) and sufficient time was
allowed for the source to reach the fixed point. Temperatures were read from the internal
thermometer, the external PRT-100 and another available PRT. An ice point apparatus was used to
calibrate the external PRTs.
E. Uncertainties associated with the Landcal P80P blackbody
The table 5 shown below shows the uncertainty budget for the brightness temperature of the test
blackbody. Uncertainty contributions include Type A uncertainty contributions (repeatability and
reproducibility) as well as Type B uncertainty contributions, blackbody emissivity, stability, PRT
calibration, temperature variations in the cavity bottom and cylindrical wall.
Page 26 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
5.2 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON
BLACKBODIES
The brightness temperature of the participating blackbodies should be provided by the participants on
the form shown in Appendix A of this document. The measurements should also be provided in an excel
file, in the form shown in Table 6. These results will be available on the project’s website along with
the corresponding measurements of the brightness temperature of the blackbodies made by the NPL
AMBER reference radiometer and PTB broadband reference radiometers. Table 7 shows how the
brightness temperature of a participant blackbody made by AMBER, the NPL reference radiometer, and
by the PTB broadband radiometer will be reported on the website.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Table 6: Format with which the participants should report the brightness temperature of their
blackbodies, along with the combined uncertainty of each measurement. The table caption will
include the date of the measurement (e.g. 23rd June 2016)
Time from start
Minutes
Blackbody Temperature
K
Combined Uncertainty
K
0.500
1.501
2.501
3.502
4.502
5.503
6.503
7.504
8.504
9.505
10.505
11.506
12.506
13.507
14.507
15.508
11.075
11.101
11.128
11.154
11.181
11.207
11.234
11.261
11.288
11.315
11.342
11.369
11.396
11.423
11.451
11.478
0.034
0.036
0.032
0.036
0.032
0.034
0.035
0.033
0.035
0.038
0.032
0.036
0.034
0.033
0.034
0.036
Table 7: Format with which the measured brightness temperature of the test blackbody
measured by the reference radiometers during measurements of the test blackbodies. The
combined uncertainty of each measurement will also be provided. The table caption will include
the date of the measurement (e.g. 23rd June 2016)
Time from start
minutes
Blackbody Temperature
K
Combined Uncertainty
K
0.500
1.501
2.501
3.502
4.502
5.503
6.503
7.504
8.504
9.505
10.505
11.506
12.506
13.507
14.507
15.508
11.089
11.115
11.141
11.168
11.194
11.221
11.248
11.275
11.301
11.329
11.356
11.384
11.414
11.438
11.465
11.493
0.054
0.053
0.053
0.054
0.052
0.053
0.055
0.058
0.056
0.053
0.052
0.052
0.059
0.054
0.058
0.052
Page 28 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
The same measurements will also be shown in plots as a function of time. Figure 8 shows the values of
brightness temperature measured by AMBER when it was monitoring the cavity of a test blackbody
during the 2009 comparison. The average value of the measurements made by AMBER will also be
given (9.836 oC in this example). The corresponding value of the blackbody brightness temperature
provided by the participant will also be plotted on the same Figure, as a function of time. In the example
shown in Figure 8, the participant stated that the brightness temperature of the test blackbody was
10.000 oC at all times during the period of the test. The difference between the average value of the
blackbody brightness temperature during the monitoring period and the average value measured by the
reference radiometer will also be given (0.164 oC in the case of the data shown in Figure 3) to indicate
the “error” with which the blackbody “reads” its own brightness temperature at that particular
temperature setting.
GOTA BB at 10 o C, 1st run, mean=9.836 oC, Difference=-164 mK
9.846
9.844
o
Temperature ( C)
9.842
9.84
9.838
9.836
9.834
9.832
9.83
0
2
4
6
8
10
12
Time (min)
Figure 8: Plot of the brightness temperature values of the test blackbody, as measured by the AMBER
radiometer, when the test blackbody temperature was set to 10 oC. The average blackbody brightness
temperature measured by the reference radiometer was 164 mK lower than the set value.
The website will also show tables which summarise the differences/“errors” of the measurements made
by the reference radiometer and the brightness temperature values of the test blackbodies reported by
the participants. Table 8 shows an example of such a Table from the 2009 radiometer comparison which
was completed at NPL [4]. Note that there will be two sets of Tables 6, 7 and 8. The first set will
correspond to the measurements made by AMBER, the NPL reference radiometer, while the second set
will correspond to the measurements made with the PTB broadband infrared radiometer. Similarly, there
will be two version of Figure 3; the first version will correspond to the measurements made by AMBER,
the NPL reference radiometer, while the second version will correspond to the measurements made with
the PTB broadband infrared radiometer on the test blackbody under identical conditions.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Table 8: Difference between the brightness temperature of a blackbody provided by the
participant and the temperature of the same blackbody measured by the AMBER radiometer at
different blackbody set temperatures in the 5 oC to 30 oC temperature range.
Set temperature
o
C
Temperature "error"
Temperature "error"
21st April run
mK
22nd April run
mK
RAL
SISTeR BB
30
20
10
14
-8
-15
6
-5
-14
Southampton
ISAR BB
30
20
10
-7
-16
-19
3
-14
-18
GOTA
La Laguna Univ.
Canary Island
30
20
10
-176
-152
-164
-188
-181
-177
DEPT
Valencia University
LAND P80P
30
20
10
-167
-143
-74
-185
-166
-87
Participant
6. REFERENCES
1. Barton, I. J., Minnett, P. J., Maillet K. A., Donlon, C. J., Hook, S. J., Jessup, A. T. and
Nightingale, T. J., 2004,” The Miami 2001 infrared radiometer calibration and intercomparison:
Part II Shipboard results”, Journal of Atmospheric and Oceanic Technology, 21, 268-283.
2. Rice, J. P., Butler, J. I., Johnson, B. C., Minnett, P. J., Maillet K. A., Nightingale, T. J, Hook, S.
J., Abtahi, A., Donlon, and. Barton, I. J., 2004, ”The Miami 2001 infrared radiometer calibration
and intercomparison. Part I: Laboratory characterisation of blackbody targets”, Journal of
Atmospheric and Oceanic Technology, 21, 258-267.
3. Theocharous, E., Usadi, E. and Fox, N. P., “CEOS comparison of IR brightness temperature
measurements in support of satellite validation. Part I: Laboratory and ocean surface
temperature comparison of radiation thermometers”, NPL REPORT OP3, July 2010.
4. Theocharous E. and Fox N. P., “CEOS comparison of IR brightness temperature measurements
in support of satellite validation. Part II: Laboratory comparison of the brightness temperature
comparison of blackbodies”, NPL Report COM OP4, August 2010.
5. Theocharous, E. and Fox, N. P., 2016, “Fiducial Reference Measurements for validation of
Surface Temperature from Satellites (FRM4STS) - Laboratory Calibration of Participants
Radiometers and Blackbodies”, Protocol for the FRM4STS LCE (LCE-IP), ESA Contract No.
4000113848_15I-LG, NPL report OFE-D-90A-V1-Iss-1-Ver-1
6. Theocharous, E., Fox, N. P., Sapritsky, V. I., Mekhontsev, S. N. and Morozova, S. P., 1998,
“Absolute measurements of black-body emitted radiance”, Metrologia, 35, 549-554.
Page 30 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
APPENDIX A:
REPORTING OF MEASUREMENT RESULTS
The attached measurement summary should be completed by each participant for each completed set of
laboratory measurements. A complete set being one, which may include multiple measurements on, or
using the same instrument but does not include any realignment of the instrument. For each realignment
a separate measurement sheet should be completed.
For clarity and consistency the following list describes what should be entered under the appropriate
heading in the tables.
Time
The time of the measurements should be UTC.
Measured
Brightness temperature
Brightness temperature measured or predicted by participant.
Measurement uncertainty
Combined/total uncertainty of the measurement.
Uncertainty
The total uncertainty of the measurement of brightness temperature
separated into Type A and Type B. The values should be given for a
coverage factor of k=1.
Wavelength
This describes the assigned centre wavelength used for the measured
brightness temperature. For the case of Fourier Transform
spectrometers, the wavelength range and wavelength resolution
should be specified.
Bandwidth
This is the spectral bandwidth of the instrument used for the
comparison, defined as the Full Width at Half the Maximum.
Standard Deviation
The standard deviation of the number of measurements made
to obtain the assigned brightness temperature without realignment
Number of Runs
The number of independent measurements made to obtain the
specified standard deviation.
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
Measurement Laboratory Results: Blackbody Comparison
Instrument Type ...…… …………………………. Identification No ………………………….
Date of measurement: …………………………… Ambient temperature …………………….
Time of measurement
(UTC)
Blackbody
Brightness
Temperature
K
BB Brightness
Temperature
Uncertainty
mK
Uncertainty
A % B
Participant: ……………………………………………………………………………………
Signature: …………………………….. Date: ……………………………
Page 32 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
Measurement Laboratory Results: Radiometer Comparison
Instrument Type ...…… …………………………. Identification No ………………………….
Date of measurement: …………………………… Ambient temperature …………………….
Time of
measurement
(UTC)
Measured
Brightness
Temperature
Combined
Measurement
Uncertainty
Wavelength
Bandwidth
Uncertainty
No.
of
K
mK
m
nm
A % B
Runs
Participant: ……………………………………………………………………………………
Signature: …………………………….. Date: ……………………………
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
APPENDIX B:
REPORTING OF MEASUREMENT RESULTS
The attached measurement summary should be completed by each participant for each completed set of
WST field measurements. A complete set being one, which may include multiple measurements on, or
using the same instrument but does not include any realignment of the instrument. For each realignment
a separate measurement sheet should be completed. A separate measurement sheet should also be
completed if a different view angle from nadir, or a different wavelength or bandwidth is used by the same
radiometer.
For clarity and consistency the following list describes what should be entered under the appropriate
heading in the tables.
Time
The time of the measurements should be UTC.
Measured Water
Surface Temperature
Brightness temperature measured or predicted by participant.
Measurement uncertainty
Combined/total uncertainty of the measurement.
Measured Sky Temperature
Brightness sky temperature measured or predicted by participant.
Uncertainty
The total uncertainty of the measurement of brightness temperature
separated into Type A and Type B. The values should be given for a
coverage factor of k=1.
Wavelength
This describes the assigned centre wavelength used for the measured
brightness temperature. For the case of Fourier Transform
spectrometers, the wavelength range and wavelength resolution
should be specified.
Bandwidth
This is the spectral bandwidth of the instrument used for the
comparison, defined as the Full Width at Half the Maximum.
Standard Deviation
The standard deviation of the number of measurements made
to obtain the assigned brightness temperature without realignment
Number of Runs
The number of independent measurements made to obtain the
specified standard deviation.
View angle from Nadir
The angle of view of the radiometer to the surface of the water from
Nadir.
Page 34 of 37
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
WST Measurement Results at Wraysbury Reservoir
Instrument Type ...………… Identification Number ……… Ambient temperature …………
Date of measurement: ……………………
View angle from nadir (degrees)………………
Wavelength (µm) ……………………………
Bandwidth (µm) …………………………….
Time
(UTC)
Measured
WST
K
Combined
WST
Uncertainty
K
Measured
sky
temperature
K
Uncert. in
sky
temperature
K
Uncertainty
No.
of
A % B
Runs
Participant: ……………………………………………………………………………………
Signature: …………………………….. Date: ……………………………
NPL - Commercial
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
LST Measurement Results on the at NPL site
Instrument Type ...………… Identification Number ……… Ambient temperature …………
Date of measurement: ……………………
View angle from nadir (degrees)………………
Wavelength (µm) ……………………………
Bandwidth (µm) …………………………….
Time
(UTC)
Measured
LST
K
Combined
LST
Uncertaint.
K
Measured
sky
temperat.
K
Uncert. in
sky
temper.
K
Uncertainty
A
%
B
No.
of
Runs
Target
emissivity
used
Participant: ……………………………………………………………………………………
Signature: …………………………….. Date: ……………………………
Page 36 of 37
NPL - Commercial
Uncert.
in
emissiv.
OFE-D-110-V1-Iss-1-Ver-1-DRAFT
19
-END OF DOCUMENT-
NPL - Commercial