Studies of Radio Frequency Interference Detection Methods in
Microwave Radiometry
DISSERTATION
Presented in Partial Fulfillment of the Requirements for
the Degree Doctor of Philosophy in the
Graduate School of The Ohio State University
By
Barış Güner, M.S., B.S.
*****
The Ohio State University
2009
Dissertation Committee:
Approved by
Professor Joel T. Johnson, Adviser
Professor Robert Lee
Dr. Inder Jeet Gupta
Adviser
Graduate Program in
Electrical and Computer
Engineering
c Copyright by
°
Barış Güner
2009
ABSTRACT
Recent surveys conducted from orbiting radiometers have shown that corruption of
radiometric data due to Radio Frequency Interference (RFI) may significantly impact
the accuracy of the retrieved environmental data. These findings have sparked an
interest in RFI detection and mitigation methods; several future microwave remote
sensing satellites, like the SMAP (Soil Moisture Active and Passive) mission of NASA,
plan to incorporate RFI mitigation strategies in their design. A digital radiometer
with very high temporal and spectral resolution developed at OSU/ESL may be
used to address this issue; versions of this radiometer working at L-band and Cband are called L-band/C-band Interference Suppressing Radiometer (LISR/CISR),
respectively. Although the high resolution obtained with this radiometer makes its use
in a space based system unlikely due to the stringent datarate requirements required
in such systems (unless data is further integrated after mitigating RFI onboard or
flagging the data), LISR/CISR sensors are very beneficial for learning about time
and frequency domain characteristics of the existing RFI environment, for serving as
“ground-truthing” devices for other radiometers, and for analyzing the effectiveness
of pulse and cross-frequency detection methods against observed RFI sources.
In this thesis, results from several groundborne and airborne radiometric campaigns performed using LISR/CISR systems are given. RFI sources observed in these
experiments and RFI mitigation methods that use the high resolution obtained via
LISR/CISR to remove such sources are described. Effectiveness of RFI mitigation
ii
methods against the diverse RFI sources encountered in campaigns is analyzed. The
ability of LISR/CISR in eliminating very weak RFI sources even on the order of
natural geophysical variations is demonstrated. Comparisons with other radiometers
that participated in these campaigns are given when possible.
A novel method for RFI detection that uses the Shapiro-Wilk test of normality is
also elucidated. Comparisons are made with another method based on the normality
of thermal noise, the kurtosis detection technique, for the pulsed sinusoidal RFI
case. Results prove that the Shapiro-Wilk technique is a viable alternative for RFI
mitigation in actual systems. The thesis is concluded with a theoretical performance
comparison against pulsed sinusoidal RFI using the three most commonly used RFI
detection methods: Pulse detection, cross-frequency detection, and kurtosis detection.
Particular emphasis is given to the cross-frequency detection method. It is shown that
cross-frequency detection method provides good detection performance regardless of
duty cycle for this important type of RFI.
iii
Dedicated to my parents...
iv
ACKNOWLEDGMENTS
I would like to thank my adviser, Professor Joel T. Johnson, for his constant help
and guidance throughout my Ph.D. studies. I also would like to acknowledge Prof.
Robert Lee, Dr. Inder J. Gupta, and graduate faculty representative Prof. Timothy
Rhodus for participating in my Ph.D. defense committee and for reviewing this thesis.
I want to thank all the ESL community as well; especially my friends (in alphabetical
order) Yakup Bayram, Metin Demir, Burkay Dönderici, Feridun Gündeş, Erdinç Ircı,
Mustafa Kuloğlu, Gökhan Mumcu, Praphun Naenna, Noppasin Niamsuwan, Koray
Tap, Salih Yarga, and Mehmet Emre Yavuz.
Lastly, to my mom and dad, no amount of thanks would suffice.
v
VITA
April 30, 1980 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Born - Eskişehir, Turkey
2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.S. Electrical and Electronics Eng.,
Bilkent University, Turkey
2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M.S. Electrical and Electronics Eng.,
Bilkent University, Turkey
2004-present . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Graduate Research Associate,
ElectroScience Laboratory,
Electrical and Computer Eng.,
The Ohio State University
PUBLICATIONS
Journal Publications
1. V. B. Ertürk and B. Güner “Analysis of finite arrays of circumferentially oriented
printed dipoles on electrically large cylinders,” Microwave and Optical Technology
Letters, vol. 42, no. 4, pp. 299 - 304, Aug. 2004.
2. V. B. Ertürk, O. Bakır, R. G. Rojas, and B. Güner “Scan blindness phenomenon
in conformal finite phased arrays of printed dipoles,” IEEE Transactions on Antennas
and Propagation, vol. 54, no. 6, pp. 1699 - 1708, Jun. 2006.
3. J. T. Johnson, A. J. Gasiewski, B. Güner, G. A. Hampson, S. W. Ellingson, R.
Krishnamachari, N. Niamsuwan, E. McIntyre, M. Klein, and V. Y. Leuski “Airborne
radio frequency interference studies at C-band using a digital receiver,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 7, pp. 1974 - 1985, Jul.
2006.
4. B. Güner, J. T. Johnson, and N. Niamsuwan “Time and frequency blanking for
radio frequency interference mitigation in microwave radiometry,” IEEE Transactions
on Geoscience and Remote Sensing, vol. 45, no. 11, pp. 3672 - 3679, Nov. 2007.
vi
Conference Publications
1. V. B. Ertürk and B. Güner, “Finite phased arrays of printed dipoles on large
circular cylinders: a comparison with the planar case,” URSI EMTS International
Conference on Electromagnetics Theory, vol. 2, pp. 972 - 974, May 2004, Pisa, Italy.
2. B. Güner, V. B. Ertürk, and O. Bakır, “A parametric analysis of finite phased
arrays of printed dipoles on large circular cylinders and comparisons with the planar case,” IEEE AP-S International Symposium and USNC/URSI National Radio
Science Meeting, vol. 4, pp. 4116-4119, Jun. 2004, Monterey, CA.
3. J. T. Johnson, A. Gasiewski, B. Güner, M. Valerio, and M. Klein, “High altitude
measurements of C-band radio frequency interference using a digital receiver,” IEEE
International Conference on Geoscience and Remote Sensing, pp. 2301-2304, Jul.Aug. 2006, Denver, CO.
4. N. Niamsuwan, B. Güner, and J. T. Johnson, “Observations of an ARSR system
in Canton, MI with the L-band interference suppressing radiometer,” IEEE International Conference on Geoscience and Remote Sensing, pp. 2285-2288, Jul.-Aug.
2006, Denver, CO.
5. J. T. Johnson and B. Güner, “Cross frequency blanking for RFI mitigation:
A C-band case study,” IEEE International Conference on Geoscience and Remote
Sensing, pp. 2710 - 2713, Jul. 2007, Barcelona, Spain.
6. B. Güner, M. Frankford, and J. T. Johnson, “On the Shapiro-Wilk test for
the detection of pulsed sinusoidal radio frequency interference,” IEEE International
Conference on Geoscience and Remote Sensing, vol. 2, pp. 157 - 160, Jul. 2008,
Boston, MA.
FIELDS OF STUDY
Major Field: Electrical and Computer Engineering
Studies in:
Microwave Remote Sensing
Prof. Joel T. Johnson
Communications and Signal Processing Assis. Prof. Elif Uysal-Biyikoglu
Mathematics
Prof. U. Gerlach
vii
TABLE OF CONTENTS
Page
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ii
Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iv
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v
Vita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xiii
Chapters:
1.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1
1.2
.
.
.
.
.
.
.
1
7
9
10
12
12
14
RFI Observations at L-band:
Canton Campaign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
1.3
2.
2.1
Passive Microwave Remote Sensing: A Brief Review
Introduction to RFI Mitigation . . . . . . . . . . . .
1.2.1 Time-domain detection methods . . . . . . .
1.2.2 Cross-frequency detection . . . . . . . . . . .
1.2.3 Methods based on Gaussianity tests . . . . .
1.2.4 Other RFI detection methods . . . . . . . . .
Outline of Thesis . . . . . . . . . . . . . . . . . . . .
Radiometer Front End and Downconversion
2.1.1 Radiometer front end . . . . . . . .
2.1.2 Downconversion stage . . . . . . . .
2.1.3 Physical properties . . . . . . . . . .
2.1.4 Radiometer state information . . . .
viii
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
.
.
.
.
.
.
.
.
.
.
17
17
18
19
21
2.2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
22
22
23
23
24
26
27
27
30
32
32
38
39
49
56
62
63
63
70
73
73
76
77
L-band Interference Suppressing Radiometer (LISR) Ground-based Observations with JPL’s Passive-Active L/S Band (PALS) Sensor . . . . .
81
2.3
2.4
2.5
2.6
2.7
3.
3.1
3.2
3.3
A Brief Description of LISR . . . . . . . . . . .
2.2.1 LISR schematic . . . . . . . . . . . . . .
2.2.2 Capture mode . . . . . . . . . . . . . .
2.2.3 APB . . . . . . . . . . . . . . . . . . . .
2.2.4 Integrating modes . . . . . . . . . . . .
2.2.5 Computer interface . . . . . . . . . . . .
Hardware Issues . . . . . . . . . . . . . . . . .
2.3.1 Observed gain pattern . . . . . . . . . .
2.3.2 Reference load switch issues . . . . . . .
2.3.3 Noise diode on/off switch issues . . . . .
2.3.4 Calibration . . . . . . . . . . . . . . . .
Summary of LISR Observations on June 17th .
2.4.1 Integration, APB off . . . . . . . . . . .
2.4.2 Integration, APB on . . . . . . . . . . .
2.4.3 Capture mode data . . . . . . . . . . . .
2.4.4 Post-processing . . . . . . . . . . . . . .
Observations Near 14:51 and 16:03 UTC . . . .
2.5.1 Observations near 14:51 UTC . . . . . .
2.5.2 Observations near 16:03 UTC . . . . . .
Laboratory Observations on June 20th and 21st
2.6.1 June 20th data . . . . . . . . . . . . . .
2.6.2 June 21st data . . . . . . . . . . . . . .
Summary and Remarks . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
System Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.1.1 PALS radiometer front end and downconversion stages . . . 82
3.1.2 PALS state timing . . . . . . . . . . . . . . . . . . . . . . . 84
3.1.3 LISR overview . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.1.4 LISR computer and control interface . . . . . . . . . . . . . 88
Dataset Overview and Calibration . . . . . . . . . . . . . . . . . . 90
3.2.1 Observed gain pattern . . . . . . . . . . . . . . . . . . . . . 90
3.2.2 Noise diode delay . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.3 LISR recorded datasets . . . . . . . . . . . . . . . . . . . . 95
3.2.4 LO tuning tests and 1390 MHz interference . . . . . . . . . 97
3.2.5 PALS state classification . . . . . . . . . . . . . . . . . . . . 97
3.2.6 LISR calibration . . . . . . . . . . . . . . . . . . . . . . . . 102
LISR Overnight Observations . . . . . . . . . . . . . . . . . . . . . 103
3.3.1 Horizontally polarized brightnesses during the night of April
24th-25th . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
ix
3.3.2
3.4
3.5
4.
107
109
113
115
119
119
124
125
Airborne C-band RFI Measurements with PSR/CXI and CISR from the
WB-57 aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.1
4.2
4.3
4.4
4.5
5.
Vertically polarized brightnesses during the night of April
24th-25th . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.3 Overnight observations on April 25th-April 26th . . . . . .
3.3.4 Overnight observations on April 27th-April 28th . . . . . .
3.3.5 Overnight observations on May 10th-May 11th . . . . . . .
RFI Detection and Mitigation . . . . . . . . . . . . . . . . . . . . .
3.4.1 Properties of observed RFI . . . . . . . . . . . . . . . . . .
3.4.2 Time domain RFI mitigation . . . . . . . . . . . . . . . . .
Summary and Remarks . . . . . . . . . . . . . . . . . . . . . . . .
Instrumentation . . . . . . . . . . . . . . . . . . .
4.1.1 PSR/CXI . . . . . . . . . . . . . . . . . . .
4.1.2 CISR . . . . . . . . . . . . . . . . . . . . .
4.1.3 Interface between PSR/CXI and CISR . . .
4.1.4 CISR modifications for the WB-57 aircraft .
4.1.5 ADD system . . . . . . . . . . . . . . . . .
4.1.6 Measurement process . . . . . . . . . . . . .
Experiment Conditions . . . . . . . . . . . . . . . .
Comparisons with PSR . . . . . . . . . . . . . . .
4.3.1 PSR scan images . . . . . . . . . . . . . . .
4.3.2 CISR observations near DFW . . . . . . . .
4.3.3 CISR observations in more rural Texas . . .
4.3.4 CISR observations over the Gulf of Mexico .
Comparisons with ADD . . . . . . . . . . . . . . .
Summary and Remarks . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
128
128
130
131
133
136
137
140
144
144
154
155
157
158
161
A Study of the Shapiro-Wilk Test for the detection of Pulsed Sinusoidal
RFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
5.1
5.2
5.3
5.4
Introduction . . . . . . . . . . . . . . . . . .
The Shapiro-Wilk test . . . . . . . . . . . . .
5.2.1 Expressions for the ai coefficients . . .
5.2.2 Implementation in digital hardware . .
5.2.3 Quantization effects . . . . . . . . . .
Simulation Procedure . . . . . . . . . . . . .
5.3.1 Signal model and notations . . . . . .
5.3.2 Cases considered . . . . . . . . . . . .
5.3.3 Computation of the kurtosis statistics
Results . . . . . . . . . . . . . . . . . . . . .
5.4.1 Histograms of W . . . . . . . . . . . .
x
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
166
168
169
170
171
171
173
175
176
177
177
5.4.2
5.4.3
5.5
6.
Performance Study of a Cross-Frequency Detection Algorithm for Pulsed
Sinusoidal RFI in Microwave Radiometry . . . . . . . . . . . . . . . . . . 188
6.1
6.2
6.3
6.4
6.5
7.
Receiver operating characteristic curves . . . . . . . . . . . 179
Comparison of ROC curves for the Shapiro-Wilk and kurtosis
tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Summary and Remarks . . . . . . . . . . . . . . . . . . . . . . . . 185
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2.1 Signal model and notations . . . . . . . . . . . . . . . .
6.2.2 Cross-frequency detection model . . . . . . . . . . . . .
6.2.3 Pulse detection model . . . . . . . . . . . . . . . . . . .
6.2.4 Kurtosis detection model . . . . . . . . . . . . . . . . .
Detection Performance Results . . . . . . . . . . . . . . . . . .
6.3.1 Effects of scalloping loss . . . . . . . . . . . . . . . . . .
6.3.2 A simplified method for the calculation of scalloping loss
System Temperature Estimation Issues . . . . . . . . . . . . . .
Summary and Remarks . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
188
190
190
191
194
196
198
202
205
207
213
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
xi
LIST OF TABLES
Table
2.1
Page
Summary of Integration, APB off observations on June 17th. Sample
numbers refer to the vertical axis of Figures 2.11 and 2.12. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states
are as follows: A=antenna, T=terminated, and a number indicates an
amount of attenuation in dB added between the antenna output and
radiometer receiver input. . . . . . . . . . . . . . . . . . . . . . . . .
40
Summary of Integration, APB on observations on June 17th. Sample
numbers refer to the vertical axis of Figures 2.16 and 2.17. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states
are as follows: A=antenna, T=terminated, and a number indicates an
amount of attenuation in dB added between the antenna output and
radiometer receiver input. . . . . . . . . . . . . . . . . . . . . . . . .
50
Summary of Capture mode observations on June 17th. LO frequency
is 1386.5 MHz unless otherwise indicated. H- and V-pol states are
as follows: A=antenna, T=terminated, and a number indicates an
amount of attenuation in dB added between the antenna output and
radiometer receiver input. . . . . . . . . . . . . . . . . . . . . . . . .
57
3.1
Summary of the data recorded in the experiment . . . . . . . . . . .
96
4.1
ADD subchannel frequencies within the 75-175 MHz IF band . . . . . 136
4.2
Time history of flight on August 25th, 2005 . . . . . . . . . . . . . . 143
4.3
Statistics from PSR four sub-band interference suppression algorithm:
21600 pixels over rural Texas (18:08-18:23 UTC) . . . . . . . . . . . . 150
4.4
Statistics from PSR four sub-band interference suppression algorithm:
21600 pixels near DFW (17:54-18:08 UTC) . . . . . . . . . . . . . . . 150
2.2
2.3
xii
LIST OF FIGURES
Figure
Page
2.1
Truck, boom and horn antenna used in Canton campaign . . . . . . .
20
2.2
Block diagram of LISR . . . . . . . . . . . . . . . . . . . . . . . . . .
22
2.3
Basic operation procedure of APB . . . . . . . . . . . . . . . . . . . .
25
2.4
Normalized LISR raw data for reference load observations versus frequency for H-pol and V-pol. Black horizontal line marks approximate 3
dB point, while red vertical lines mark boundaries of protected spectrum. 28
2.5
Mean LISR raw power in 1399-1428 MHz as a function of delay after
state change. (a) Antenna states (b) Reference load states . . . . . .
31
Mean LISR raw power in 1399-1428 MHz as a function of delay after
state change. (a) Noise diode plus antenna states (b) Noise diode plus
reference load states . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
Mean LISR brightness 1397-1430 MHz for H- and V-pol observations
of a controlled target; H pol observations are shown inside boxes. Horizontal lines mark expected brightnesses when target is viewed through
0, 1, 2, and 3 dB pads, respectively. . . . . . . . . . . . . . . . . . .
35
Mean LISR brightness 1406-1412 MHz for V-pol observations of a terminator (near 14:00 UTC) . . . . . . . . . . . . . . . . . . . . . . . .
36
Calibrated LISR brightnesses versus frequency during sky observations
near 19:25 UTC on June 20th. . . . . . . . . . . . . . . . . . . . . . .
37
2.10 Location of the truck (circled in red) with respect to the ARSR. . . .
38
2.6
2.7
2.8
2.9
xiii
2.11 Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency. Refer to Table 2.1 for information on the vertical axis of
the plot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
2.12 Same as Figure 2.11, but for H-pol.
. . . . . . . . . . . . . . . . . .
42
2.13 Calibrated V-pol LISR data in Integration, APB off mode, versus
RF frequency, at 1.3 msec time resolution. Data acquired near time
14:52:15 UTC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
2.14 Same as Figure 2.13, but for H-pol.
. . . . . . . . . . . . . . . . . .
46
2.15 Calibrated brightness 1399-1428 MHz for horizontally polarized observations including additional attenuation. A 21 dB attenuator was
used at times less than 19:24 UTC, and a 41 dB attenuator was used
at times greater than 19:34 UTC. Between these two times at 71 dB
attenuator was used. . . . . . . . . . . . . . . . . . . . . . . . . . . .
48
2.16 Calibrated V-pol LISR data in Integration, APB on mode, versus RF
frequency. Refer to Table 2.2 for information on the vertical axis of
the plot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52
2.17 Same as Figure 2.16, but for H-pol.
. . . . . . . . . . . . . . . . . .
53
2.18 Calibrated V-pol LISR data in the band 1399-1428 MHz versus time,
for both APB off and APB on modes. Text in the plot indicates the
azimuth angle of the radiometer antenna, with regions between the
vertical lines indicating periods of antenna rotation. . . . . . . . . .
54
2.19 Same as Figure 2.18, but for H-pol.
. . . . . . . . . . . . . . . . . .
55
2.20 Percent of samples blanked by the APB processor . . . . . . . . . . .
56
2.21 Example radar pulse observed near time 14:58:42; amplitude level of
V pol data is shifted by 2500 for clarity purposes . . . . . . . . . . .
58
2.22 Thirty radar pulses obtained during a sweep over azimuth near time
18:39 UTC in horizontal polarization. The vertical axis label indicates
the approximate azimuth angle of each of three sets of captures. . .
60
xiv
2.23 Maximum capture raw-data amplitudes observed (following integration
to 1.28 µsec resolution) in horizontal polarization during sweeps over
azimuth from 18:34 to 18:44 UTC. Reference data is interpolated in
the intervals with no measurements. Refer to the text for information
on the map from time to azimuthal observation angle. . . . . . . . .
61
2.24 Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency, near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from
Table 2.1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
65
2.25 Same as Figure 2.24, but following post-processing described in Section
2.4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
2.26 Calibrated LISR data in channels similar to the UM ADD system,
near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table
2.1); results included before and after post-processing . . . . . . . . .
67
2.27 Calibrated V-pol LISR data in Integration, APB on mode, versus RF
frequency, near times 14:56:43 to 14:58:18 (Samples 363-416 from Table
2.2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
2.28 Calibrated LISR data in channels similar to the UM ADD system, near
times 14:56:43 to 14:58:18 (APB on, Samples 363-416 from Table 2.2);
results included before and after post-processing . . . . . . . . . . .
69
2.29 Calibrated LISR data in channels similar to the UM ADD system, near
times 16:03:41 to 16:04:14 UTC (Samples 1358-1379 from Table 2.1);
results included before and after post-processing . . . . . . . . . . .
71
2.30 Calibrated LISR data in channels similar to the UM ADD system, near
times 16:05:32 to 16:06:06 (APB on, Samples 613-636 from Table 2.2);
results included before and after post-processing . . . . . . . . . . .
72
2.31 Calibrated H-pol LISR data in Integration, APB off mode, versus RF
frequency, on June 20th . . . . . . . . . . . . . . . . . . . . . . . . .
74
2.32 Same as Figure 2.31 but in Integration, APB on mode . . . . . . . .
75
2.33 Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency, on June 21st . . . . . . . . . . . . . . . . . . . . . . . . .
78
2.34 Same as Figure 2.33 but for H-pol in Integration, APB on mode . . .
79
xv
3.1
PALS state diagram . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
3.2
LISR block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . .
86
3.3
LISR measured reference load raw power versus frequency, vertical
polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
91
LISR measured reference load raw power versus frequency, horizontal
polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
92
3.5
Raw power vs. time in the Ant+ND state for the capture data . . . .
94
3.6
Calibrated brightness temperatures vs. IF frequency as PALS LO
tunes in horizontal polarization . . . . . . . . . . . . . . . . . . . . .
98
3.4
3.7
Average raw power levels for a single file for the night of April 27th . 100
3.8
Calibrated H-pol brightness temperature vs. RF frequency, April 25th 104
3.9
Comparison of calibrated brightnesses for PALS and LISR, April 25th 106
3.10 Calibrated V-pol brightness temperature vs. RF frequency, April 25th 108
3.11 Change in the passband properties of reference looks on April 25th . 109
3.12 Comparison of calibrated brightnesses for PALS and LISR, April 26th 110
3.13 Calibrated H-pol brightness temperature vs. RF frequency, April 26th 111
3.14 Calibrated V-pol brightness temperature vs. RF frequency, April 26th 112
3.15 Calibrated brightness temperatures vs. time for ADD, LISR and PALS
on April 28th . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.16 Calibrated H-pol brightness temperature vs. RF frequency, April 28th 115
3.17 Calibrated V-pol brightness temperature vs. RF frequency, April 28th 116
3.18 Calibrated brightness temperatures vs. time for LISR and PALS on
May 11th . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.19 Calibrated H-pol brightness temperature vs. RF frequency, May 11th
xvi
117
3.20 Calibrated V-pol brightness temperature vs. RF frequency, May 11th
118
3.21 Calibrated brightness temperatures vs. time for ADD, LISR and PALS
sensors on April 28th between 4 and 6 am UTC time . . . . . . . . . 120
3.22 Image of pulsed interferers for V-pol, April 28th . . . . . . . . . . . . 122
3.23 Image of pulsed interferers for H-pol, April 28th . . . . . . . . . . . . 123
3.24 H-pol and V-pol total channel calibrated brightnesses before and after
time domain RFI mitigation, April 28th . . . . . . . . . . . . . . . . 125
4.1
WB-57 high-altitude aircraft . . . . . . . . . . . . . . . . . . . . . . . 128
4.2
Simplified schematic of PSR/CXI . . . . . . . . . . . . . . . . . . . . 130
4.3
Simplified schematic of CISR
4.4
Photograph of the CISR enclosure installed on the WB-57 aircraft
4.5
Flight path, including nearby Texas cities (circles)
4.6
Altitude of WB-57 aircraft versus time . . . . . . . . . . . . . . . . . 142
4.7
Locations of C-band RFI sources within the JSC source database.
Flight path is indicated by the thick red line. . . . . . . . . . . . . . . 145
4.8
Distribution of source center frequencies for the JSC database records
plotted in Figure 4.7. Note that records centered between 5.9-6.2 GHz
are described as “not included” in the database. . . . . . . . . . . . . 146
4.9
Calibrated brightnesses from PSR conical scans of the entire flight, in
all four PSR main channels . . . . . . . . . . . . . . . . . . . . . . . 148
. . . . . . . . . . . . . . . . . . . . . . 132
. 134
. . . . . . . . . . 141
4.10 Average of Figure 4.9 over scan angle, versus time . . . . . . . . . . . 149
4.11 Comparison of calibrated brightness temperatures vs. time between
original and mitigated data for CISR channels 8,12,16 and 20 . . . . . 153
4.12 Spectrogram images and time history of brightness temperatures for
original and mitigated data over an urban landscape, CISR channel 16 156
xvii
4.13 Spectrogram images and time history of brightness temperatures for
original and mitigated data over a rural landscape, CISR channel 17 . 157
4.14 Spectrogram images and time history of brightness temperatures for
original and mitigated data over the Gulf of Mexico, CISR channel 8
158
4.15 Time history of calibrated brightnesses for ADD and CISR, for ADD
subchannel 6 and tuned PSR channels 12 to 15 . . . . . . . . . . . . 160
4.16 Scatter plot for calibrated brightnesses of ADD vs. CISR, for ADD
subchannel 6 and tuned PSR channels 12 to 15 . . . . . . . . . . . . 162
4.17 Scatter plot after the filtering on calibrated CISR data, for ADD subchannel 6 and tuned PSR channels 12 to 15 . . . . . . . . . . . . . . 163
4.18 Comparison of brightness temperature spectrograms for CISR (left)
and ADD (right), tuned PSR channels 12 to 15 . . . . . . . . . . . . 164
5.1
Weight coefficients ai before and after quantization using 8-bit resolution172
5.2
Histograms (scaled to correspond to probability density functions) of
W for non-quantized (upper Figure) and quantized data (G = 4, lower
Figure), d ≈ 1%. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
5.3
ROC curves for d ≈ 1% . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.4
ROC curves for d = 50% . . . . . . . . . . . . . . . . . . . . . . . . . 181
5.5
ROC curves for d = 100% . . . . . . . . . . . . . . . . . . . . . . . . 182
5.6
ROC curves vs. test size N , R=2.5, d ≈ 1% . . . . . . . . . . . . . . 183
5.7
ROC curves vs. test size N , R=5, d = 50% . . . . . . . . . . . . . . . 183
5.8
Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d ≈ 1%184
5.9
Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d = 50%185
5.10 Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d =
100% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
6.1
Schematic of the cross-frequency detector . . . . . . . . . . . . . . . . 192
xviii
6.2
Schematic of the pulse detector . . . . . . . . . . . . . . . . . . . . . 195
6.3
Schematic of the kurtosis detector . . . . . . . . . . . . . . . . . . . . 197
6.4
ROC curves for the cross-frequency, pulse, and subsampled kurtosis
algorithms: CW RFI, 8 channels (N = 16 for the pulse detector).
R = 1, R = 2 and R = 3 cases shown for the cross-frequency case,
R = 3 only for the pulse and kurtosis cases. . . . . . . . . . . . . . . 199
6.5
Same as Figure 6.4, but for RFI with 50% duty cycle. . . . . . . . . . 200
6.6
ROC curves for the cross-frequency, pulse, and subsampled kurtosis
algorithms: 0.1% duty cycle, R = 1, 8 channels (N = 16 for the pulse
detector). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
6.7
AUC image for the cross-frequency detector vs. RFI strength and duty
cycle, four channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
6.8
AUC image comparison for the cross-frequency (upper), subsampled
kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and
duty cycle, four channels/N = 8 . . . . . . . . . . . . . . . . . . . . . 203
6.9
AUC image comparison for the cross-frequency (upper), subsampled
kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and
duty cycle, sixteen channels/N = 32 . . . . . . . . . . . . . . . . . . . 204
6.10 Probability of detection vs. RFI strength for a constant Pf a of 1%,
CW RFI. Results when RFI is assumed to be centered in a channel
(“bin centered”) are compared with the general case of random RFI
frequency (“with scalloping loss”) for 8 and 16 frequency channels. . 205
6.11 AUC image comparison for the cross-frequency algorithm with (upper)
and without (lower) scalloping loss vs. RFI strength and duty cycle,
sixteen channels/N = 32 . . . . . . . . . . . . . . . . . . . . . . . . . 206
6.12 Probability of detection vs. RFI strength for a constant Pf a of 1%,
CW RFI. Results obtained with an approximation to the scalloping
loss is compared with the complete solution for 8 and 16 frequency
channels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
6.13 Mean and standard deviation of the system temperature estimate as
channels are discarded in the threshold estimation procedure of the
cross-frequency algorithm, for 4, 8, and 16 frequency channels. . . . 210
xix
6.14 ROC curves obtained using the analytical approximation to threshold
estimation effects compared with results from Monte Carlo simulations.
CW RFI having R = 3, Mdrop = 2, and for 4,8 and 16 channels. . . . 212
6.15 Results of Figure 6.10 are compared with curves including system temperature estimation effects. CW RFI . . . . . . . . . . . . . . . . . . 213
xx
CHAPTER 1
INTRODUCTION
This thesis details studies of the Radio Frequency Interference (RFI) environment
and RFI detection/mitigation techniques for earth observing microwave radiometers.
The aim of this chapter is introducing the reader to some fundamental concepts of
microwave radiometry, providing the motivation behind this research, describing the
relevant studies in the literature, and providing an outline for the rest of the thesis.
The next section explains the basic theory behind passive microwave remote sensing. The benefits of microwave remote sensing are described in this section, and a
history of earth observing passive microwave remote sensing systems in space is given.
In Section 1.2, Radio Frequency Interference is defined. Results from surveys of RFI
at microwave frequencies are given as a motivation for the research on RFI mitigation
techniques. A literature overview of RFI mitigation methods in passive microwave
radiometry is presented. The chapter concludes by giving a basic outline of the rest
of the thesis.
1.1
Passive Microwave Remote Sensing: A Brief Review
Radiometers are passive remote sensing systems that measure the natural radiation emitted by objects [1]. If an object is in thermal equilibrium with its surroundings, it emits all the energy that it absorbs. A perfect absorber that does not reflect
1
any energy is known as a “blackbody”. If an ideal receiver of bandwidth B were to
measure the power radiated by a blackbody, the received power would be equal to:
P = kT B
(1.1)
where k is Boltzmann’s constant and T is the physical temperature of the blackbody
in Kelvin. On the other hand, radiated power by a non-absorbing material (i.e. a
perfect conductor) would be zero. The materials in real life fall in between these
two extremes; hence, the power radiated by them is bounded by 0 and kT B at a
certain temperature T. Therefore, if P is the power radiated by the object and B is
the bandwidth of the receiver again, an equivalent temperature called the “brightness
temperature” (TB ) can be defined as:
TB =
P
kB
(1.2)
If external calibration techniques are not present, the power measured by a receiver is also affected by its antenna’s radiation pattern. In that case, the ratio measured in Equation 1.2 is called the “antenna temperature” rather than the brightness
temperature.
The ratio of the brightness temperature of an object to its physical temperature is
called the “emissivity” (e) which has a value between 0 for a non-absorbing medium
and 1 for a blackbody:
e=
2
TB
T
(1.3)
The emissivity of the scene observed is related to the “reflectivity” of the media.
Assuming the surface observed is flat and homogeneous, the emissivity can be written
in terms of the so-called “specular” reflectivity (Γsp ) as:
e(f, θ, p) = 1 − Γsp (f, θ, pol)
(1.4)
Γsp (f, θ, p) = |R(f, θ, pol)|2
(1.5)
where
In this equation, R(f, θ, pol) is the Fresnel reflection coefficient and depends on the
operation frequency f , incidence angle θ and polarization pol. The Fresnel reflection
coefficient is related to the dielectric constant of the medium, which is in turn affected
by medium’s physical properties like its physical composition and temperature. In
most cases, it is possible to establish a model of the relationship between these properties of a medium and its dielectric constant. For rough surfaces, Equation 1.4 is
no longer exact. If the effect of the roughness is significant, an empirical relationship
that takes the roughness of the surface into account can be developed [2]. Therefore,
environmental parameters that do not directly alter the dielectric constant but modify the emissivity like wind vector over oceans can also be recovered by microwave
remote sensing.
As it may be evident from the previous paragraph, the main use of microwave
remote sensing is in geoscientific fields like hydrology, meteorology, climatology and
agriculture [1]. Radiometers are also widely used in extraterrestrial observations
and have some military applications. Note that radiometric systems that observe
earth from space for geoscientific purposes can further be divided into two main
categories [3]:
3
• Atmospheric sounders that are used to measure vertical profiles of environmental parameters like atmospheric temperature and pressure using resonances of
molecules that make up the atmosphere.
• Surface sensors that operate on frequency windows where the atmospheric absorption is low; these sensors can measure surface variables like soil moisture,
water-ice boundary in polar regions, sea surface temperature and salinity.
Focus of this research is on the effects of radio frequency interference on passive
microwave remote sensing of environmental parameters; therefore, we are primarily
interested in surface sensors (in Chapters 2 and 3, sky observations will also be
described). However, the principles and conclusions of this research is valid for all
other radiometer types as well.
Obviously, microwave remote sensing is not the only way to measure environmental parameters. For example, sea surface temperature profiles can be obtained with
a “brute-force” method of direct measurements. It is also possible to make passive
remote sensing measurements at other frequencies like infrared and visible portions
of the spectrum. Thus, it would be beneficial to discuss the benefits of passive microwave remote sensing at this point. Possible advantages of passive remote sensing
at microwave frequencies over other measurement methods are listed below [1]:
• Sensitivity of brightness temperature to an environmental parameter changes
with frequency. As a result, information obtained from the observed scene is not
the same at microwave and infrared or visible frequencies. Even at microwave
frequencies, dominant physical factor affecting emission can change with a slight
change in frequency.
4
• Microwave remote sensing systems do not require a source of illumination like
Sun in contrast to the observations at the visible spectrum.
• Detrimental effects of clouds and rain are also much less pronounced at microwave frequencies compared to the visible and infrared parts of the spectrum.
• Microwaves can penetrate vegetation or even the ground itself.
• Space observations can provide accurate and up-to-date information from a
great portion of the earth surface. On the other hand, if we consider the sea
surface temperature example, a brute-force approach to the same measurements
would require enormous amount of resources.
The realization of these advantages can be dated back to the ground based measurements in the 1940s by Dicke et al. [4]. However, more than 20 years had to pass
before the launch of the first passive microwave remote sensing radiometer that observed earth from space. This radiometer, launched in 1968, was onboard the Soviet
Satellite Cosmos 243 and it measured atmospheric water vapor, water-ice content, sea
surface temperature and ice temperature [5]. Further account of passive microwave
remote sensing of earth from space upto the early 1980s can be found in Reference [3].
Scanning Multichannel Radiometer (SMMR) is another important turning point
for the microwave remote sensing history. This device was operational in the first
half of the 1980s [6] and it was capable of sea surface temperature and soil moisture
measurements. SMMR system could perform dual (V- and H-) polarization measurements at 6.6, 10.69, 18, 21 and 37 GHz. This instrument was mounted on two
different satellites: Nimbus G satellite which had a spatial resolution of 148x95 km
at 6.6 GHz, and Seasat A whose resolution was 121x79 km at the same frequency.
5
Recent radiometric campaigns include Advanced Scanning Radiometer (AMSR),
Advanced Scanning Radiometer for Earth Observing System (AMSR-E), and WindSat missions. The AMSR-E radiometer was a slightly modified version of the Japanese
AMSR radiometer onboard the Japanese ADEOS-2 satellite [7], and it was carried on
as a joint project between NASA and Japan Aerospace Exploration Agency (JAXA).
The spatial resolution of the AMSR system was around 50 km [8], which doubled the
resolution of the SMMR instrument. Just like SMMR, AMSR and AMSR-E sensors
which were launched in 2001 and 2002, respectively, are multichannel instruments.
The operating frequencies for the AMSR-E channels are 6.925, 10.65, 18.7, 23.8, 36.5
and 89 GHz; AMSR has two additional channels centered at 50.3 and 52.8 GHz in
addition to the channels of AMSR. These instruments were used for the measurement
of water vapor, precipitation, sea surface wind speed, sea surface temperature and
soil moisture. WindSat polarimetric radiometer was developed for the U.S. Navy and
the National Polar-orbiting Operational Environmental Satellite System (NPOESS)
Integrated Program Office (IPO) by the Naval Research Laboratory Remote Sensing
Division and the Naval Center for Space Technology [9]. WindSat was launched in
2003 and it was the first radiometer designed specifically for wind vector retrieval.
However, it is also capable of sea surface temperature, precipitation, cloud water
content and rain rate measurements.
NASA is preparing to launch a satellite remote sensing mission to measure sea
surface salinity at L-band called Aquarius [10] in 2010. European Space Agency
(ESA) has planned a similar mission called Soil Moisture and Ocean Salinity (SMOS)
mission [11] in which observations over land will be used to determine soil moisture
and the sea observations will be used for predicting the sea surface salinity. Another
mission planned is the Soil Moisture Active/Passive (SMAP) mission [12] by the Jet
6
Propulsion Laboratory of NASA. SMAP will include an L-band radar and an L-band
radiometer. A conically scanning reflector antenna will be used in the device, and
the goal is to have a 10 km. spatial resolution by combining the attributes of the
radar and the radiometer. The planned launch date of this mission is between 2010
and 2013.
1.2
Introduction to RFI Mitigation
As described in the previous section, radiometers are built to measure natural
thermal radiation. RFI in this context can be described as the radiation due to
the anthropogenic emissions that radiometers involuntarily receive. RFI is additive,
that is it shows as an increase in the power received by the radiometers, and as a
result it causes a bias in the predictions of the environmental parameter that is being
measured.
Several recent works have documented the detrimental effects of radio frequency
interference (RFI) on Earth observing passive microwave radiometer systems [13–
17]. In [13], analysis of RFI for interferometric radiometers was made and possible
consequences to the L-band Microwave Imaging Radiometer with Aperture Synthesis
(MIRAS) were discussed. Although interferometric radiometers are not the subject of
our research, this work is still useful for providing an insight to the possible sources
of RFI at L-band. In spite of the fact that a portion of the L-band from 1400 to
1427 MHz is reserved for passive microwave remote sensing operations, emissions
from sources that operate at frequencies close to this band may cause corruption
in radiometric observations if they are not properly filtered. Possible sources of
RFI include L-band radars, mobile satellite services and military tactical services.
Harmonics of emitters at VHF and UHF bands like broadcast satellite services, mobile
7
satellite services, meteorological satellite services and jamming might be the other
sources of RFI at the protected portion of L-band. The potential contribution of
RFI to the measurements of space-based radiometers due to L-band radars in terms
of equivalent brightness temperatures was computed in Reference [14]. It was found
that RFI due to a typical radar may contribute an equivalent brightness temperature
that is as high as 10 K at the receiver of a radiometer orbiting the earth at a 675 km.
altitude. It is not possible to attain the required accuracy for most environmental
parameters in the presence of such a high level of RFI, and thus authors suggest the
use of RFI mitigation techniques in future sensors.
In [15], a survey of RFI at C-band over continental United States was made
using the difference between brightness temperatures at X- (10.65 GHz) and C- band
(6.925 GHz) of the AMSR-E radiometer. C-band does not have a protected portion
of the spectrum while the frequencies between 10.68-10.7 GHz are protected at Xband. However, this protected portion is only a small part of the reported 100 MHz
bandwidth at the X-band channel of AMSR-E. Results of the survey show significant
RFI corruption, especially around urban centers. Facilities for cable TV relay and
wireless communication, airport radars and manufacturing operations are mentioned
as possible sources of RFI at C-band. A similar survey over United States at C- and
X- bands was performed in [17], this time using data from the WindSat radiometer.
This study confirmed the severity of the RFI corruption at C-band. The situation
at X-band was relatively better. However, authors managed to show the presence
of RFI sources at X-band using the correlation channels of the WindSat radiometer.
These sources, albeit weaker than their C-band counterparts, may still cause severe
accuracy issues for environmental data retrievals at X-band.
8
Results of these surveys have demonstrated the need for RFI mitigation for future passive microwave remote sensing systems. RFI differs from natural radiation in
intensity, spatial variability, polarization, and spectral and distribution characteristics [15]. Using these facts, methods for the detection and mitigation of RFI [18–27]
have been developed, and several radiometric systems capable of RFI detection and
mitigation with varying performance against different RFI types have been implemented. Techniques used for RFI mitigation in these systems can be divided into
three main groups; time-domain detection methods, cross-frequency detection methods and methods based on the Gaussianity of thermal noise. These techniques are
reviewed briefly in the next few sections. A description of some other methods employed in RFI detection are given in section 1.2.4.
1.2.1
Time-domain detection methods
Time domain detection methods try to detect RFI sources concentrated in time.
An example of such an RFI source is a pulsed radar. Detectors that search for
energy outliers in time-domain are called “pulse” detectors (or sometimes glitch detectors) [20–23]. Detection is accomplished by comparing the power of the received
fields with a threshold, which may be regarded as a conceptually simple technique.
However, the fact that changes in the system temperature should be accounted for
in the determination of the threshold brings some additional complexity in its implementation. Application of similar concepts for RFI detection was first investigated in
radio astronomy; for example a time-domain threshold method was demonstrated in
1996 by Fridman et al. [20] while in [28] a correlator output was used to detect RFI;
if RFI is detected, received signal is blanked in time-domain in this work.
9
First earth remote sensing system that is capable of pulse blanking was developed at OSU/ESL. This detector, called an Asynchronous Pulse Blanker (APB), was
implemented in FPGA and it is capable of pulse blanking in real time. APB was incorporated into the digital radiometers designed at OSU/ESL called L-band/C-band
Interference Suppressing Radiometer (LISR/CISR). A basic description of how APB
works will be given in Chapter 2 and performance of APB against pulsed sources will
be demonstrated using data from a campaign carried out in close vicinity of an air
route surveillance radar (ARSR).
It should be noted that in a space based system, since most satellites are not
geostationary, a time-domain detector will be discriminating RFI based on its spatial variation. Such a glitch detection method is planned for use in the Aquarius
radiometer and the theory of its operation can be found in [29].
1.2.2
Cross-frequency detection
Detectors that search outliers in the frequency domain are called “cross-frequency”
detectors in this work. This type of detector also has its roots in radio astronomy
applications; Fridman et al. describes thresholding in frequency domain in [20]. First
use of cross-frequency detection for RFI mitigation in earth observations is dated to
year 2002 [18]. Cross-frequency detection method employed in this work was tested
using data from an airborne campaign over Southern Great Plains in 1999 obtained
with NOAA (authors are now with University of Colorado) Polarimetric Scanning
Radiometer C-band (PSR/C). This system had 4 analog frequency subchannels with
passbands between 5.8-6.2 GHz, 6.3-6.7 GHz, 6.75-7.10 GHz and 7.15-7.50 GHz. RFI
10
detection was performed in post-processing. A spectral fit was applied to the brightness temperatures of these four channels. If fit parameters were not physically reasonable or brightness temperature of the channels were greater than physical limits,
process was repeated by applying the procedure to channels within spectral fit and
brightness temperature limits. When a reasonable spectral fit was obtained, brightness temperatures of the corrupted channels were replaced with a value based on this
fit. On the other hand, if spectral fit could not be obtained, minimum brightness
temperature among the channels was used as to replace the brightness temperatures
of all the remaining channels. In the case that even the minimum brightness temperature was over the physical limits, RFI was deemed to be “uncorrectable”. Statistics
of PSR data corrected with this method for different RFI environments is provided
in Chapter 4.3.
Although PSR algorithm demonstrated the effectiveness of a cross-frequency detection method, number of frequency channels used in this analog system were limited
which restricted its efficiency. Also, with such large bandwidths it would be difficult to resolve narrowband RFI sources since these sources might only cause a weak
change in brightness when averaged over a larger channel. Increasing the number of
channels is expensive and would mean a significant increase in the size of such an
analog device. On the other hand, it is easy to obtain a high spectral resolution in
a digital system with little additional hardware complexity employing Fast Fourier
Transform (FFT) methods in hardware. LISR/CISR sensor of OSU uses such technology [23]. LISR/CISR is capable of performing a 1024 point FFT operation and
resolving spectrum in the order of a 100 KHz. This high resolution and large number
of channels allowed us to demonstrate the effectiveness of cross-frequency detection
against narrowband RFI as described in this work.
11
1.2.3
Methods based on Gaussianity tests
RFI detection can also be performed by testing the distribution of the fields received by the radiometer, which should be Gaussian for thermal noise; a deviation
from the Gaussian distribution indicates the presence of RFI. Although numerous
methods exist to test Gaussianity, to date only the kurtosis test of Gaussianity has
been used in practical implementations [24–26]. The kurtosis test has been shown to
be an effective tool against a wide variety of RFI types, but a blind spot in detecting
pulsed sinusoidal interference has also been reported [24]. Later studies have shown
that it is possible to remove this blind spot and to improve detection performance
by subsampling in time and frequency [30–32]. Blind spot can also be avoided using
the sixth moment of the data, however increase in resources and high RFI strength
required make this method impractical [33].
In this thesis, Shapiro-Wilk test of Gaussianity is studied as a possible alternative
for RFI detection [34,35]. Results show the possibility of designing a radiometer that
use this method for RFI detection.
1.2.4
Other RFI detection methods
As mentioned before, pulse detection, cross-frequency detection and kurtosis detection methods are the main methods used for RFI detection in current and planned
future radiometric systems and they form the focus of this work. However, several
other RFI detection methods which are used primarily for radio astronomy applications can also be found in literature.
In [18], it is mentioned that polarization diversity may be used in earth observing
radiometers for RFI detection in addition to the cross-frequency detection method
described in the paper. Although this method might be useful for RFI detection,
12
mitigation of RFI is not possible in this case other than throwing out the corrupted
data and it has not been implemented to date in earth observing radiometers. Making
radiometric observations at places or times when corruption due to RFI is low is a
widely used practice. Work done on eliminating potential RFI sources by engaging
stricter filtering and power requirements for emitters close to the protected portions
of the spectrum should also be mentioned here [36]. Although this is not exactly an
RFI mitigation method, it is obvious that if these efforts were to succeed significant
improvement in corruption due to RFI will be obtained.
Interferometric nulling is a RFI mitigation technique that is used in radio astronomy [20]. However, this technique requires an antenna array (such that radiation
pattern is minimized in the direction of the RFI source) and it is not very useful
for earth remote sensing where it is not easy to get a discrimination between the
directions of data and that of the RFI source. A technique conceptually similar to interferometric nulling that is called “RFI masking” was suggested for earth observing
radiometers in [37]. In this method, a global mask of RFI would be produced by the
analysis of data over a long time period and this mask would be used to eliminate
data obtained from locations corrupted by RFI. However, as the authors suggest, such
a method would not account for new RFI locations and would be overly conservative
in the sense that it eliminates data from locations where RFI is not always present
(i.e intermittent).
Estimating the interference waveform and subtracting from the received signal was
suggested as a way to mitigate RFI in radio astronomy [38]. However, this method
requires that the form of the interference to be known and hence it is not very
useful for an earth observing system where RFI sources vary by type and numbers
depending on the observation point. An adaptive filtering scheme is also suggested
13
for radio astronomy but this method requires the direction of the interference signal
to be known and thus again is not suitable for our purposes [39].
1.3
Outline of Thesis
This thesis will basically follow the order in which the research was performed with
experimental results provided first and theoretical results given in the later parts of
the thesis. However, to maintain continuity between L-band campaigns, an L-band
campaign conducted at Jet Propulsion Laboratory will be described in Chapter 3,
although it was performed after the C-band observations narrated in Chapter 4.
Chapter 2 describes observations made by LISR during a groundborne campaign
at Canton, Michigan in 2005. Descriptions of the LISR system and the APB operation procedure are given. Parts of the experiment were performed in close vicinity to
an ARSR, therefore the measured data was very suitable in demonstrating the performance of APB. Other than the aforementioned observations close to the ARSR, some
laboratory observations where artificial RFI sources were injected to the antenna will
also be described. Methods for pulse detection and cross frequency detection in post
processing devised for this campaign will be explained and their success in removing
RFI will be demonstrated.
Chapter 3 also details groundborne observations at L-band using LISR; this time
using data from a campaign at the Jet Propulsion Laboratory in Pasadena, California
in 2006. Observations of the sky were made in this experiment, and the presence of
RFI sources even in the protected portion of the spectrum was demonstrated using
the high frequency resolution of the LISR system. It is again shown that most of
the obvious RFI can be eliminated using the cross-frequency detection and the pulse
14
detection methods applied in post-processing. Intercomparisons with PALS and ADD
sensors that were also present in this campaign are provided.
An airborne C-band campaign performed over Texas and Gulf of Mexico is explained in Chapter 4. This campaign provided an excellent opportunity to show
different RFI environments including urban and rural land observations as well as
water observations over the Gulf of Mexico. Results are shown to illustrate the success achieved against both strong and weak RFI sources. Comparisons with other
sensors that were present (PSR and ADD) are also provided.
A novel method for RFI detection is described in Chapter 5. As explained in
Section 1.2.3, tests of normality can be used for detecting RFI. While numerous
tests of normality are described in the statistical literature, the kurtosis test was
the only such test considered previously in radiometric systems. With this fact in
mind, Shapiro-Wilk test was considered as an alternative RFI mitigation technique.
Performance of the Shapiro-Wilk test against pulsed sinusoidal RFI is demonstrated
in this chapter. Results prove the feasibility of employing this technique for RFI
mitigation.
In Chapter 6, a theoretical performance comparison is made for the three main RFI
detection methods described in this introductory chapter: Pulse detection method,
cross-frequency detection method, and the kurtosis method. RFI is again assumed
to be pulsed sinusoidal which might model pulsed radars as well as CW sources. The
focus of this chapter is on the cross-frequency detector. It is shown that the crossfrequency detector maintains a high a performance regardless of duty cycle for this
type of RFI.
Chapter 7 concludes this study. Contributions provided by this research is listed
and a summary is provided in this chapter.
15
CHAPTER 2
RFI OBSERVATIONS AT L-BAND:
CANTON CAMPAIGN
On June 16th and June 17th, 2005, a ground-based campaign involving L-band
microwave radiometer observations of an ARSR in Canton, MI was conducted by
researchers from the University of Michigan [24], NASA Goddard Space Flight Center [40], and The Ohio State University (OSU). The purpose of the campaign was
to demonstrate the level of interference observed in L-band observations within the
protected band 1400-1427 MHz, as well as to demonstrate the ability of three distinct
radiometer backends at detecting and removing this interference. The campaign was
conducted in close proximity to the radar location (within approximately 200 m of the
radar antenna), ensuring that strong out-of-band interference would be experienced
in the observations. While this configuration certainly emphasizes the effects of the
radar system compared to observations from a space-based platform, the experiment
was designed to demonstrate that the backends developed could successfully remove
interference in both extremely strong and slight interference environments. The latter case was obtained through choice of the observation direction of the radiometer
antenna, as well as the fact that the radar center frequency (reported as 1315 MHz
by the station’s literature) was far out-of-band of the radiometer observations. Other
properties of the radar reported by the station are a peak radiated power of 4 MW,
16
antenna gain of 35 dB, an antenna tilt angle of 2.6 degrees, a PRF of 360 Hz (or 2.78
msec between pulses), and a pulse width of 2 µsec.
The campaign was continued on June 20th and 21st to include observations of
artificial pulsed and continuous RFI sources in the laboratory at the University of
Michigan. The June 21st data set also includes measurements of a well characterized
cold load for calibration purposes.
This chapter documents observations by The Ohio State University backend primarily from the June 17th dataset (the 167th day of the year 2005). The next section
provides a basic description of the overall RF and downconversion systems of the
radiometer; these systems were common to the backend units of the three research
groups. Section 2.2 then provides a brief overview of The Ohio State University
backend, LISR. Section 2.3 describes several hardware issues that were observed in
processing the data, and Section 2.4 reports general properties of the observed brightnesses. Detailed examinations of a few specific datasets and the performance of LISR
RFI removal post-processing algorithms are provided in Section 2.5. Section 2.6 then
provides a brief overview of the data obtained from the laboratory measurements on
June 20th and 21st, and final conclusions of the study are described in Section 2.7.
2.1
2.1.1
Radiometer Front End and Downconversion
Radiometer front end
The L-band radiometer antenna and front end utilized in this experiment were
being developed by the University of Michigan for delivery to Prof. Brian Hornbuckle
at Iowa State University. The campaign provided an opportunity for the performance
of the radiometer to be assessed in the field prior to delivery. Although a detailed
schematic of the radiometer was not made available, a qualitative description was
17
provided. The radiometer utilizes an L-band dual polarized horn antenna of approximate dimensions 40 x 50 cm; this is not particularly high gain but sufficient to
allow angular discrimination. The radiometer front end contains dual receivers so
that simultaneous observations of horizontal and vertical polarization are provided.
Following the antenna connection, each receiver begins with a mechanical switch to
allow measurement of an internal reference load, followed by a coupler to allow injection of a noise diode signal. Both the noise diode powers (on or off) as well as
the reference load switches were under control of the University of Michigan backend
system (called the Advanced Digital Detector or ADD [24]). The ADD backend is
capable of measuring received field moments upto their 4th order which are then used
to calculate kurtosis statistic for RFI detection purposes. Kurtosis statistic will be
explained in detail in Chapters 5 and 6.
Following these internal calibration standards is a bandpass cavity filter from
1403.5-1423.5 MHz, reported as having 3 cavities. Next is the system low noise
amplifier, followed by 2 additional bandpass cavity filters with the same cutoff frequencies, each reported as containing 4 cavities. An additional RF amplifier is then
followed by a final filter containing 6 cavities and a final RF amplifier.
2.1.2
Downconversion stage
The downconversion stage consists of a mixer fed by an LO at either 1386.5 or
1440.5 MHz. This frequency plan results in the RF filter passband occupying IF
frequencies 17-37 MHz. Note all image rejection in this process is provided by the
RF filters. In the case of LO frequency 1386.5 MHz, the image band (lower side band)
is 1351.5-1371.5 MHz, while when using LO frequency 1440.5 MHz the image band
(the upper sideband) is 1457.5-1477.5 MHz. Rejection in both these bands by the RF
18
filters is assumed to be similar. The mixer is followed by additional IF amplification
and a lowpass filter of 50 MHz bandwidth. With regard to the downconverter effect
on the ARSR transmissions at 1315 MHz, it should be expected that the 50 MHz
lowpass filter likely would result in greater attenuation when using LO frequency
1440.5 MHz, as the radar is then at IF frequency 125.5 MHz as opposed to 71.5 MHz
with LO frequency 1386.5 MHz.
The IF outputs for both horizontal and vertical polarizations are then power
divided three ways and supplied to the three backend systems. The resulting outputs
occupy the approximate bandwidth 17-37 MHz and have been amplified to provide
sufficient noise power levels for further processing.
2.1.3
Physical properties
Both the radiometer front end and downconversion stages were operated in a single
temperature controlled enclosure. Unfortunately means for recording the enclosure
temperature were not available at the time of the campaign; it will be assumed in
what follows that the reference load temperatures were maintained at the temperature
set point of 305 K throughout the campaign. Any inaccuracy in this information will
contribute to errors in the calibrated brightnesses presented later; however these
effects are not expected to be large as the temperature control had been previously
tested and found to be reasonably stable.
The front end and downconversion stage enclosure was placed in close proximity
to the radiometer antenna, and the entire structure was mounted on the boom of a
truck from the Microwave Geophysics research group of the University of Michigan.
A picture of the truck, boom and the radiometer front end can be seen in Figure
2.1. The boom included an elevation positioner for the antenna structure, so that
19
Figure 2.1: Truck, boom and horn antenna used in Canton campaign
the radiometer antenna angle could be controlled in both azimuth (through rotation
of the boom) as well as elevation.
IF outputs from the front end/downconversion stage enclosure were routed down
the boom and into the truck roof; the three backend systems were then all located
inside the truck. In addition, the LO source for the front end/downconversion stage,
as well as control lines for the reference load and noise diode switches, were provided
up the boom from inside the truck. The LO source utilized was a bench oscillator
followed by a frequency doubler.
20
2.1.4
Radiometer state information
The experiment plan involved a periodic observation of radiometer states with
a period of 1.46 seconds. The basic pattern involves a 324 msec observation of the
antenna, followed by a 36 msec observation of the antenna plus the noise diodes. This
pattern is repeated (i.e. 324 msec antenna, 36 msec antenna plus noise diode), then
the reference load switches are thrown. The reference load is then observed for 324
msec, followed by 36 msec of the reference load plus the noise diode. This pattern
is repeated again, and then the reference load switch is thrown again and the cycle
repeats beginning with the antenna observations.
A 6 msec delay state was also included following the command to throw the
reference load switch, as well as a 1 msec delay state following the noise diode on or
off commands.
As stated previously, this state machine was controlled by the ADD system, and
information on these states was provided to the OSU and GSFC systems through
three control lines. The three control lines are labeled “N” (high when the noise
diodes are on), “R” (high when the switches are set to observe the reference loads),
and “I” (high when the radiometer is not in a delay state.) These were open drain
outputs of the ADD, pulled up to a voltage of 3.3 volts for the other backends. While
the NASA GSFC system was configured to measure continuously while recording
information on these state lines, LISR was configured to trigger off the “I” line.
Thus, LISR measurements were begun upon observation of a high state in the “I”
line.
21
100 MSPS
V pol
ADC
ADC
50 MHz LPF
(H,V), join to
100 MSPS I/Q
Asynchronous Pulse
Blanker (APB)
1024 point FFT
H pol
Integrate 128
FFT’s: 1.3 msec
256K FIFO
National
Instruments
interface
H and V pol:
50 MHz@
97.7 kHz
resolution
every
10.24 µsec
PC-104
Computer
Figure 2.2: Block diagram of LISR
2.2
2.2.1
A Brief Description of LISR
LISR schematic
Figure 2.2 presents a block diagram of LISR as configured for this experiment.
Further information on LISR can be obtained from [21, 22, 41–50]. LISR samples
both the incoming H and V pol 17-37 MHz IF’s at 100 MSPS, using two 10-bit A/D
converters for this purpose. The LISR A/D converters are actually clocked at 200
MSPS, but in this case one half of these samples are discarded to simplify operations
at the approximate 25 MHz IF center frequency. The remaining LISR operations
described in Figure 2.2 take place in a single large Altera FPGA. The first operation
involves digitally low-pass filtering each of the incoming IF’s, then combining the
two polarizations into a single complex datastream (16 bits in both I and Q) at 100
MSPS, occupying the band -50 to 50 MHz (typically V pol -50 to 0 MHz, H pol 0 to
50 MHz). The digital low pass filter used has a bandwidth wider than that set by
the RF filters, and therefore is not expected to impact the measured data. However
22
its use allows removal of the image spectrum for each of the input polarizations so
that the combination into a single 100 MSPS complex datastream is possible.
Following this output is a set of processors that can be controlled by the operator in
real time through an ethernet interface between the system computer and the FPGA.
In the results presented here, LISR was operated in three distinct modes, labeled
“Integration, APB off”, “Integration, APB on”, and “Capture” in what follows.
2.2.2
Capture mode
In “Capture” mode, the 100 MSPS complex datastream is directly passed to the
control and recording computer without further processing. This extremely high
temporal resolution (time sampled at 10 nsec) results in an extremely high datarate,
so that only low duty cycle observations are possible. However the high temporal
resolution and coherent data acquired allows detailed studies of the properties of the
observed interference. It was found that 6 256K captures could be acquired and
transferred to the control computer during a 324 msec antenna or reference load
observation, this represents 15.73 msec observed out of 324 possible, or a duty cycle
of 4.85%. Due to an accidental operator error in the field, noise diode data were not
acquired in capture mode; capture data are therefore studied as raw data only.
2.2.3
APB
The other LISR modes utilize data that has passed through an asynchronous
pulse blanking stage. The term “asynchronous” is used since no periodic properties
of the RFI source is assumed [22]. The APB is designed to detect and blank radar
pulses in real time, so that their effects can be removed without further processing
of the measured data, while maintaining accurate calibration of the noise-free data.
The basic operation principle of APB is illustrated in Figure 2.3. To detect pulsed
23
interferers, the APB maintains a running estimate of the mean and variance of the
incoming power in the time domain (through use of an exponential averaging operation as described in [22].) Whenever a sample magnitude greater than a threshold
number of standard deviations from the mean is detected, the APB blanks (sets to
zero) a block of samples of length NBLANK beginning from a predetermined period
before the triggering sample (shown as NWAIT in the figure), through and hopefully
including any multi-path components associated with the detected pulses. APB operating parameters are adjustable and can be set by the user. In the experiments
described here, the APB threshold was usually set to approximately 9 standard deviations from the mean power level, and a blanking window of 40 µsec around detected
pulses was used (note 40 µsec corresponds to 6 km of radar range.) This window began 10.24 µsec prior to the detected sample. The APB provides information on the
amount of blanking to further processor stages so that blanking effects on calibration
can be corrected in real time. APB operates on the 100 MSPS complex datastream
directly, so that the two polarizations are not separately blanked; however the effects
on calibration remain accurately computed for each polarization.
2.2.4
Integrating modes
Following the APB is a length-1K complex FFT utilizing a triangular window to
reduce truncation effects. This FFT operation provides an output every 10.24 µsec,
with two sets of 512 bins obtained in a 50 MHz bandwidth; the first set is for horizontal
and the second set for vertical polarization. This results in a spectral resolution of
approximately 97 kHz, much finer than the bandwidth of many expected RFI sources.
FFT outputs are then passed through detection and integration operations, with the
final datastream comprised of 1024 frequency bins obtained every 1.3 msec. These
24
Power (linear units)
NBLANK
NWAIT
Threshold
0
20
40
60
80
100
120
Time (µ sec)
140
160
Figure 2.3: Basic operation procedure of APB
25
180
200
data are represented in 32 bits for each bin. Though not utilized in this experiment,
the integration module is also capable of computing a max-hold operation in RFI
detection applications.
The distinction between “Integration, APB on” and “Integration, APB off” modes
involves whether the APB blanking is turned on or off in the data input to the FFT
operation. Comparing brightnesses obtained in these modes will allow the effectiveness of the APB real-time pulse removal algorithm to be examined.
2.2.5
Computer interface
Results following integration are stored in a 32 bit, 256K FIFO. FIFO data is then
passed through a National Instruments interface to the system control computer.
Timing tests showed that 205 spectra (i.e. FFT outputs) could be acquired and
reliably transferred to the system computer within 324 msec. This represents a duty
cycle of 82.9% in the integrating modes. While this duty cycle could have been
increased by increasing the number of FFT’s integrated in the integration operation,
it was deemed desirable to maintain a time resolution in the final dataset that was finer
than the PRF period of the ARSR system. The final data obtained is approximately
0.8 MB for each antenna or reference load trigger. Noise diode observations obtained
17 spectra, representing 22.3 msec out of the 36 msec available (duty cycle 61.9%.)
The final datarate if operating continuously is approximately 8.3 GB per hour.
One final influence on the system datarate involves the control software. For efficiency in writing measured data to the computer hard drives, data obtained from the
FIFO is first stored in a computer RAM buffer of approximately 36 MB. This buffer
is written to the hard drives when full, resulting in a cessation of LISR operations
26
until the write operation is completed. In the integrating modes, the delay incurred
is approximately 2.55 seconds following continuous operations for 16.45 seconds.
2.3
2.3.1
Hardware Issues
Observed gain pattern
Figure 2.4 plots the mean raw power versus frequency obtained by LISR for reference load observations, normalized so that the maximum value of the frequency
response is approximately 0 dB. Curves for both horizontal (blue and dashed) and
vertical (black and dotted) polarizations are included. These results show a relatively
smooth instrument gain pattern versus frequency, although an increase in gain with
frequency is observed as well as a passband “ripple” of approximately 1 dB. Behaviors
for the vertical and horizontal receivers are similar. Lines in the Figures mark the
approximate gain level of -3 dB, defining a 3dB passband of around 1402 to 1425
MHz, while the gain at the protected spectrum boundaries of 1400 and 1427 MHz is
found to be around 9 to 10 dB below the maximum. The patterns reach the LISR
noise level of around -18 dB for a band around 1397 to 1430 MHz. Outside this region
LISR is incapable of determining instrument passband properties.
While this passband appears acceptable, it seems somewhat wide with regard
to possible RFI effects for a spaceborne instrument. Given the possibility of RFI
sources up to 1398-1399 MHz (though not observed in the data to be reported), the
passband of the instrument utilized may not provide sufficient reduction of these
slightly out-of-band sources.
The nature of the instrument passband brings up issues related to data calibration. The majority of the LISR data to be described begin as raw data of the type in
27
Reference Load Power vs. Frequency
2
H pol
V pol
0
Relative noise amplitude (dB)
−2
−4
−6
−8
−10
−12
−14
−16
−18
1390
1395
1400
1405 1410 1415 1420 1425
Frequency (MHz) with LO=1440.5 MHz
1430
1435
1440
Figure 2.4: Normalized LISR raw data for reference load observations versus frequency for H-pol and V-pol. Black horizontal line marks approximate 3 dB point,
while red vertical lines mark boundaries of protected spectrum.
28
Figure 2.4, which represent 50 MHz of spectrum sampled into 512 bins. The majority of the LISR results to be illustrated perform calibration of each of these 512 bins
separately; larger bandwidth channel brightnesses are then obtained by averaging
the calibrated brightnesses over the desired bandwidth. However, instruments with
less fine spectral resolution instead observe raw powers integrated over larger bandwidths originally. In this case, particular parts of the spectrum have been weighted
according to the instrument passband before calibration is performed. For example,
the instrument passband results in a total power calibration over the entire passband highly emphasizing channels within the 3 dB bandwidth of the radiometer,
compared to those outside the 3dB bandwidth. In addition, the instrument gain pattern emphasizes brightnesses at higher frequencies within the passband compared to
lower frequencies. These differences can make direct comparisons of calibrated data
difficult.
One method for addressing this issue involves first combining the LISR raw data
into a larger channel before performing calibration. For example, comparison of LISR
calibrated data with that obtained by the ADD system (channels approximately 3
MHz wide) can be performed directly if the exact properties of the ADD channels are
utilized first in weighting the LISR spectrum. Several tests performed in this manner using estimated properties of the ADD channels (from [24]) showed only minor
differences between pre- and post-calibration combination of LISR channels. For this
reason, post-calibration combination of LISR channels was deemed acceptable, and is
utilized in the results to be shown. However differences in brightnesses are introduced
by this procedure so that precise inter-comparisons among sensors of brightnesses (i.e.
beyond the level of a few K) are not possible.
29
2.3.2
Reference load switch issues
Examination of LISR data showed issues regarding transition of the antenna/reference
load switch in the state machine pattern of the radiometer. Recall again that the
basic switching pattern is an antenna observation, followed by antenna plus noise
diode, followed by a second antenna observation, and another antenna plus noise
diode, then a change of the antenna/reference load switch position to the reference
load. A 6 msec delay is included in the state machine following the switch command
before observations of the reference load are begun. The pattern is repeated again
for reference load observations.
Figure 2.5 illustrates the mean raw power in LISR channels representing 13991428 MHz as a function of additional delay introduced after the beginning of the
state, relative to that with no additional delay. These data were recorded near time
14:51 UTC on June 17th in “Integration, APB off” mode, and represent means over
approximately 20 seconds of observed data.
Recall that within one 324 msec antenna or reference load state, LISR records
205 1.3 msec spectra, so that the power in these spectra can be investigated as a
function of time from the state beginning. Figure 2.5, plot (a) represents antenna
measurement data, while plot (b) represents reference load measurement data. The
multiple curves in the plots are for the two polarization channels and for the two
possible states: either following a noise diode on/off transition or following a transition of the antenna/reference switch. Results show all of the mean powers have only
slight trends versus the additional delay, with the exception of the vertically polarized channel following transition of the antenna/reference load switch. A decrease in
the power measured with time is observed, for both the antenna and reference load
30
(b) Doy 167: 14:51 UTC Reference
100.05
100
100
Mean Power Relative to No Delay (%)
Mean Power Relative to No Delay (%)
(a) Doy 167: 14:51 UTC Antenna
100.05
99.95
99.9
Ant after Ant: H
Ant after Ref: H
Ant after Ant: V
Ant after Ref: V
99.85
99.8
99.95
99.9
Ref after Ant: H
Ref after Ref: H
Ref after Ant: V
Ref after Ref: V
99.85
99.8
99.75
0
50
100
150
200
Delay after switch command (msec)
99.75
0
50
100
150
200
Delay after switch command (msec)
Figure 2.5: Mean LISR raw power in 1399-1428 MHz as a function of delay after
state change. (a) Antenna states (b) Reference load states
31
states. Although the relative power changes are small, calibration coefficients to be
described later show these trends to be on the order of at least 1 K.
To avoid calibration issues introduced by this effect, all states following transition
of the antenna/reference load switch are omitted in what follows for both polarizations; only antenna or reference load data following observation of a noise diode is
utilized. While this reduces the data available to only 50%, the majority of the
data obtained were recorded in a stable environment over long time periods, so that
sufficient data remains for the goals of this investigation.
2.3.3
Noise diode on/off switch issues
A similar issue was observed with regard to noise diode observations. Due to
the expected rapid response of the noise diode on/off switch, a time delay of only
1 msec was allocated following a noise diode on/off switch command. Figure 2.6
plots the mean LISR raw power in 1399-1428 MHz as a function of delay after the
state beginning, relative to the power obtained with no additional delay. In this case,
no significant trends are observed at long times, but the first 1.3 msec data sample
shows a significantly reduced power relative to those at later time delays. Calibration
coefficients show this change to be on the order of 3-4 K. For this reason, the first 1.3
msec LISR spectrum was removed in all noise diode data in what follows.
2.3.4
Calibration
Calibration of LISR measured data was performed using the reference and reference plus noise diode states to determine the radiometer gain and offset coefficients,
both of which were obtained in H and V polarizations for 512 frequency channels.
Means of the reference and reference plus noise data over a period of plus or minus
32
(a) Doy 167: 14:51 UTC ND
(b) Doy 167: 14:51 UTC Reference
101
Mean Power Relative to No Delay (%)
Mean Power Relative to No Delay (%)
101
100.8
100.6
100.4
ND after Ant: H
100.2
ND after Ant: V
100
99.8
100.8
100.6
100.4
ND after Ref: H
100.2
ND after Ref: V
100
0
5
10
15
20
Delay after switch command (msec)
99.8
0
5
10
15
20
Delay after switch command (msec)
Figure 2.6: Mean LISR raw power in 1399-1428 MHz as a function of delay after
state change. (a) Noise diode plus antenna states (b) Noise diode plus reference load
states
33
5 seconds surrounding the measurement of interest were utilized in determining the
calibration coefficients.
As stated previously, it was assumed that reference load observations represented a
305 K brightness, while the noise diode added an additional 128.4 K. The latter figure
was obtained from calibration studies by the University of Michigan, and did not
include any possible state transition delay effects. Although a detailed examination
was not performed for LISR, a few qualitative tests of calibration accuracy were
performed using the June 21st observations of a liquid nitrogen target. Figure 2.7
illustrates LISR mean brightnesses in the band 1397-1430 MHz during observations
of the liquid nitrogen target, including extra 0, 1, 2, and 3 dB pads. Horizontal lines
in the figure mark the expected brightnesses as the degree of padding is varied, and
results are illustrated in both H (APB on) and V (APB off) polarizations. Results
show the calibration procedure to yield absolute brightnesses accurate to within 3-4
K, with the absolute error increasing for larger brightnesses. Modifications to the
noise diode and reference load assumed brightnesses could be used to address these
errors, but were not pursued further.
An additional indication of the calibration and receiver stability obtained is illustrated in Figure 2.8, which plots the calibrated LISR brightness in the band 1406-1412
MHz for V-pol channel observation of a terminator at times near 14:00 UTC on June
17th. Results are shown for both APB off and APB on modes. At times prior to
14:36 UTC on the 17th, the V-pol radiometer channel was connected to an external
terminator rather than to the radiometer antenna. Results show a relatively stable terminator brightness near 288 K (288.099K APB off, 288.08K APB on) for this
case over several minutes, with a standard deviation among 324 msec states of 0.55K
APB off and 0.63K APB on. Clearly the calibration procedure utilized is producing
34
Total Brightness 1397−1430 MHz
220
210
200
190
Brightness (K)
180
170
160
150
V pol, APB off (18:00)
H pol, APB on (19:00)
140
130
120
110
20
30
40
50
60
70
Minutes past UTC 18:00 or 19:00
80
90
100
Figure 2.7: Mean LISR brightness 1397-1430 MHz for H- and V-pol observations
of a controlled target; H pol observations are shown inside boxes. Horizontal lines
mark expected brightnesses when target is viewed through 0, 1, 2, and 3 dB pads,
respectively.
35
Terminator observations Doy 167
290
APB off
APB on
V pol Brightness 1406−1412 MHz (K)
289.5
289
288.5
288
287.5
287
286.5
286
27
28
29
30
31
Minutes past 14:00 UTC
32
33
34
Figure 2.8: Mean LISR brightness 1406-1412 MHz for V-pol observations of a terminator (near 14:00 UTC)
a stable means for measuring brightness temperatures, and the APB calibration correction is functioning properly. Note the APB reports 1.8% of data being blanked in
the dataset shown; this blanking is due to triggering of the APB detection algorithm
by large noise peaks, and is consistent with expectations for Gaussian noise given the
APB parameters used.
Figure 2.9 provides another test of calibration involving observations of the sky
near 19:25 UTC on June 20th. During this time period, the radiometer antenna elevation angle was 62 degrees, providing near zenith measurements. Although some
influence of the RFI sources could be possible, the LISR spectra shown appear RFI
36
Calibrated Sky Brightness 170, 19:25−19:26
80
H Pol
V pol
70
Brightness (K)
60
50
40
30
20
10
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 2.9: Calibrated LISR brightnesses versus frequency during sky observations
near 19:25 UTC on June 20th.
free, with the exception of narrowband sources near 1400 and 1403 MHz. Neglecting these sources, the sky brightnesses obtained appear reasonable compared to the
reported values of 44.7 K from the ADD system (believed to be in H-pol). Mean
brightnesses from 1405-1430 MHz from LISR are found to be 45.0 K for H-pol (blue
in the Figure) and 50.0 K for V-pol (red). Again, there are several possible sources
of small differences between brightnesses of LISR and the ADD system, but overall
the results validate that a reasonable calibration of LISR results is being obtained.
37
Figure 2.10: Location of the truck (circled in red) with respect to the ARSR.
2.4
Summary of LISR Observations on June 17th
On June 17th, LISR obtained datasets between times 13:25 to 16:28 UTC as well
as times 18:34 to 19:53 UTC. A satellite map of the experiment site can be seen in
Figure 2.10. Location of the truck is circled in red. In its default position, antenna
is facing north at an azimuth of angle of ≈ 160◦ with respect to the ARSR. The first
period of data contains primarily observations with the radiometer antenna held at a
fixed azimuthal observation angle, while the second dataset contains primarily observations during periods when the antenna azimuth angle is varied. Data in all three
LISR modes was recorded during the experiment; the three modes are considered
separately in the next sections.
38
2.4.1
Integration, APB off
Table 2.1 summarizes LISR observations in Integration, APB off mode. These
data are originally acquired as 1024 point spectra sampled at 1.3 msec; however, to
reduce the data volume, the majority of the results to be shown in this mode will be
integrated over the 205 spectra in an antenna observation (approximately 269 msec).
Figures 2.11 and 2.12 plot calibrated brightnesses in vertical and horizontal polarizations, respectively, obtained in this mode for all data in Table 2.1. The horizontal
axis of these figures is RF frequency, mapped from the LISR IF frequencies of 0 to 50
MHz given the known LO frequency in a given observation. Although these plots do
not provide detailed quantitative information on the observed brightnesses, the basic
qualitative variations with observation of the sky, the external environment, and a
terminator are obvious in these figures.
RFI source near 1413-1415 MHz
A strong RFI source in the range 1413-1415 MHz is also observed in Figures 2.11
and 2.12; the strength of this source is found to be highly correlated to the azimuth
position of the antenna. Later analysis will show this source to originate from the
ARSR system. The source is seen to have some apparent slight variations in its
frequency throughout the experiment, and to be stronger in vertical than in horizontal
polarization. While later results will be reported to indicate some possibility of
saturation of the radiometer receiver when directly observing the radar (azimuth
angle approximately 160 degrees), the majority of the data does not show evidence
of saturation. However, measurements with a spectrum analyzer showed very high
emitted powers (near -5 dBm received in a 1 MHz bandwidth at 1315 MHz), so that
39
Start
Time
13:25:32
14:13:31
14:21:25
14:27:09
14:46:34
14:51:43
15:00:58
15:07:09
15:14:28
15:20:38
15:24:40
15:27:42
15:35:44
15:39:55
15:42:59
15:45:51
15:47:37
15:51:57
15:54:29
16:01:52
16:03:41
16:07:23
16:09:14
16:13:19
16:17:10
16:20:38
16:25:52
16:26:59
19:16:14
19:30:53
19:39:42
19:53:25
Stop
Time
13:27:07
14:14:29
14:22:20
14:30:42
14:48:29
14:56:15
15:05:43
15:12:51
15:16:25
15:23:33
15:26:35
15:28:37
15:36:46
15:41:46
15:44:31
15:46:26
15:48:12
15:53:10
15:55:04
16:02:26
16:04:14
16:08:17
16:09:29
16:15:34
16:17:25
16:21:13
16:26:08
16:27:53
19:18:45
19:31:29
19:42:33
19:55:36
Start
Sample
1
54
87
120
245
312
469
621
811
869
969
1037
1072
1120
1166
1220
1244
1268
1313
1336
1358
1380
1414
1425
1474
1485
1508
1519
1553
1643
1666
1767
Stop
Sample
53
86
119
244
311
468
620
810
868
968
1036
1071
1119
1165
1219
1243
1267
1312
1335
1357
1379
1413
1424
1473
1484
1507
1518
1552
1642
1665
1766
1846
LO
(MHz)
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
H-pol
state
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
21
71
41
41
V-pol
state
T
T
T
T
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
El
(deg)
1
1
50
50
50
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
x
44
44
89
89
89
1
1
1
1
Az
(deg)
0
0
0
0
0
0
x
x
100
x
135
135
135
x
x
x
160
160
160
0
0
0
0
0
0
0
0
0
x
x
x
170
Table 2.1: Summary of Integration, APB off observations on June 17th. Sample
numbers refer to the vertical axis of Figures 2.11 and 2.12. LO frequency is 1386.5
MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna,
T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input.
40
V pol
300
200
250
400
Sample
600
200
800
150
1000
1200
100
1400
50
1600
1800
0
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 2.11: Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency. Refer to Table 2.1 for information on the vertical axis of the plot.
41
H pol
300
200
250
400
Sample
600
200
800
150
1000
1200
100
1400
50
1600
1800
0
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 2.12: Same as Figure 2.11, but for H-pol.
42
significant power at this frequency may still reach the receiver even after passing
through the front end filters.
If the ARSR transmissions at 1315 MHz were insufficiently attenuated by the radiometer front end filters, the resulting IF frequencies would be 71.5 MHz or 125.5
MHz for LO frequencies 1386.5 MHz and 1440.5 MHz, respectively. These transmissions would again be attenuated by the IF 50 MHz lowpass filter (more severely with
LO 1440.5 MHz), and when sampled would eventually be mapped to RF frequency
1415 MHz. The 1-2 MHz offset observed has not been explained, but could result due
to either inaccuracy of the specified LO frequency of the radiometer or inaccuracy in
the reported ARSR frequency. Data will be shown in Section 2.6 that suggest that
the LO frequency utilized on June 20th and 21st was highly stable. In addition, the
maximum brightness observed with LO frequency 1440.5 MHz is only slightly (on
the order of 10-20 K) less than that observed with LO frequency 1386.5 MHz. If the
proposed explanation is accurate, this result must indicate that the IF lowpass filter
utilized has only a slight change in attenuation between frequencies of 71.5 MHz and
125.5 MHz, which is quite surprising. For these reasons, the exact means by which
the out-of-band ARSR transmission is observed can not be considered completely
resolved.
Narrowband sources near 1400, 1403, and 1420 MHz
In addition to the ARSR emissions, narrowband RFI is observed near frequencies
1400 and 1403 MHz throughout the experiment, in both polarizations. These sources
also are correlated (through less than the 1414 MHz source) to the radiometer antenna location. Later evidence will be shown that these sources are more continuous
in nature. The 1400 MHz source occupies a bandwidth < 500 kHz, and is stronger in
43
horizontal than vertical polarization. The reported brightness in horizontal polarization exceeds 1800 K in some measurements. The 1403 MHz has similar properties and
bandwidths, and achieves a maximum brightness of approximately 420 K. However
the 1403 MHz source is observed less frequently than the source at 1400 MHz.
Though not evident in the Figures illustrated, some evidence of narrowband interference is observed near 1420.5 MHz, particularly during periods when the radiometer
antenna observed the sky at high elevation angles. This source appears to produce
brightnesses up to 30 K larger than the background in a bandwidth less than 1 MHz.
It is noted that both the 1400 and 1403 MHz sources are near or outside the 3
dB band edges of the radiometer front end filters. This implies that their influence
is reduced in data that is integrated over the front end filter response before being
calibrated. In addition, the small bandwidth of these sources greatly reduces their
contribution when integrated over frequency. Nevertheless, these small sources retain large amplitudes and could contribute measurable RFI if the radiometer filter
response provides insufficient attenuation. The origin of these sources is not known at
present, although similar emissions have been observed in ground based experiments
at both Ohio State and NASA GSFC. One possible producer of these emissions is
radiation from various electronics and test equipment within the vicinity of the measurements.
Sample high time resolution data
Figures 2.13 and 2.14 are images of V- and H-pol data at 1.3 msec time resolution,
near time 14:52:15 UTC. At this time, the radiometer antenna was directed at azimuth
0 degrees, so that received ARSR emissions were relatively weak. Although the images
of Figures 2.13 and 2.14 are noisier than those obtained from longer time integrations,
44
Figure 2.13: Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency, at 1.3 msec time resolution. Data acquired near time 14:52:15 UTC.
the high time resolution allows the pulsed nature of the source near 1414 MHz to
be clearly observed. Fifty LISR spectra are plotted (65 msec of data); note that
the pulse power observed varies significantly within successive pulse observations.
This variation can be attributed to inclusion of a triangular window prior to FFT
computation within LISR; the pulse amplitude is then influenced by where it falls
within a 1024 point LISR FFT “frame”. Overall these plots illustrate the pulsed
nature of the 1414 MHz source.
45
Figure 2.14: Same as Figure 2.13, but for H-pol.
46
Effect of adding attenuation to H-pol
Data from 19:16 to 19:55 UTC were taken with additional attenuation inserted
between the horizontally polarized antenna port and the radiometer receiver input.
As indicated in Table 2.1, the radiometer antenna was being rotated in azimuth
throughout these measurements. Calibrated brightnesses in horizontal polarization
in the band 1399-1428 MHz are plotted in Figure 2.15 versus time. Results at times
less than 19:24 UTC still show significant variations, indicating the continued reception of radar pulses even with 21 dB extra attenuation. Results with 71 dB (19:2419:34) and 41 dB attenuation however do not show such variations. Overall these
results demonstrate that the ARSR emissions have an extremely significant effect on
brightnesses integrated over the total instrument bandwidth. Further demonstration
of this fact will be shown in the next section.
Summary of Integration, APB off data
The results of Figure 2.11 and 2.12 show the ability of the LISR system in detecting
RFI within a wideband radiometer channel. Although no real-time mitigation of the
observed sources was performed in APB off mode, post-processing can be performed
to remove obvious RFI sources. It is important to note that any narrowband RFI
sources with amplitudes significant enough to influence the brightness of a large (i.e.
10-20 MHz) channel must appear dramatically larger than the noise background when
resolved in frequency. Algorithms for detecting and removing RFI sources in APB
off mode data will be described in Section 2.4.4, and applied to specific datasets in
Section 2.5.
47
H pol antenna port padded, 167 19:00
Brightness Temp 1399−1428 MHz (K)
305
300
295
290
285
280
15
20
25
30
35
40
45
Minutes past 19:00 UTC
50
55
60
Figure 2.15: Calibrated brightness 1399-1428 MHz for horizontally polarized observations including additional attenuation. A 21 dB attenuator was used at times less
than 19:24 UTC, and a 41 dB attenuator was used at times greater than 19:34 UTC.
Between these two times at 71 dB attenuator was used.
48
2.4.2
Integration, APB on
Table 2.2 summarizes LISR observations in Integration, APB on mode. Again,
the majority of the results to be shown in this mode will be integrated over the 205
spectra in an antenna observation (approximately 269 msec). Two different threshold
settings were used for the APB processor. The symbol β will be used to represent the
threshold number of standard deviations above the mean power at which a detection
is declared by the APB. A value of β 2 = 90 was used for the observations before
16:17:45 UTC time and a β 2 = 40 value was used for observations later in the day.
However, the β 2 = 40 cases were found to contain excessive blanking (on the order of
45% of the data blanked, as could have been predicted from statistical analysis), and
so are omitted in the following discussions. Information on the percent of samples
blanked is available from the APB processor, and will be discussed in what follows
for the time periods before 16:17 UTC.
Figures 2.16 and 2.17 plot calibrated brightnesses in vertical and horizontal polarizations, respectively, obtained in this mode for data in Table 2.2 with β 2 = 90.
Again the basic qualitative variations with observations of the sky, environment, and
terminator are obvious in these Figures. An additional variation occurs in the vertically polarized data near sample 500; in this case, the connector for the V-pol IF
signal became loose, so that the V-pol data plotted in this period (samples 479 to
524) is meaningless. An obvious feature of these data compared to those in Figures
2.11 and 2.12 for most of the same antenna configurations is the dramatic reduction
of the 1414 MHz source. This is consistent with expected performance of the APB
algorithm against pulsed sources, and indicates that the APB approach is highly effective against the ARSR observed for real-time pulsed RFI mitigation. The sources
49
Start
Time
13:27:56
14:15:30
14:23:07
14:31:50
14:49:08
14:56:43
15:17:34
15:29:13
15:37:12
15:53:41
15:55:27
16:05:32
16:17:45
16:21:50
16:24:53
16:28:15
18:48:57
18:50:57
18:53:56
19:11:27
19:15:41
19:25:38
19:26:15
19:35:53
19:52:01
Stop
Time
13:30:10
14:19:03
14:25:38
14:33:45
14:49:23
14:58:19
15:19:28
15:30:28
15:38:26
15:53:56
15:56:21
16:06:07
16:18:58
16:22:05
16:25:39
16:28:50
18:49:13
18:53:25
18:57:26
19:14:00
19:15:57
19:25:53
19:30:04
19:39:03
19:52:54
Start
Sample
1
76
198
285
352
363
417
479
525
568
579
613
637
682
693
715
738
750
804
926
1015
1027
1038
1174
1287
Stop
Sample
75
197
284
351
362
416
478
524
567
578
612
636
681
692
714
737
749
803
925
1014
1026
1037
1173
1286
1320
LO
(MHz)
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
1440.5
H-pol
state
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
21
21
71
71
41
41
V-pol
state
T
T
T
T
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
El
(deg)
1
50
50
50
50
1
1
1
1
1
1
1
44
89
89
89
0
0
0
0
0
0
0
0
0
Az
(deg)
0
0
0
0
0
0
100
135
135
160
160
0
0
0
0
0
x
x
x
x
x
x
x
x
170
Table 2.2: Summary of Integration, APB on observations on June 17th. Sample
numbers refer to the vertical axis of Figures 2.16 and 2.17. LO frequency is 1386.5
MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna,
T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input.
50
near 1400 and 1403 MHz remain visible and are not strongly affected by the APB
processor, indicating that these sources are of a more continuous nature.
Comparison of APB on and APB off brightnesses
To provide a more quantitative examination of the APB algorithm, Figures 2.18
and 2.19 plot V-pol and H-pol calibrated brightnesses in the band 1399-1428 MHz,
respectively, versus time, for both APB on and APB off modes. Vertical lines in
the plot mark the boundaries during which the radiometer antenna azimuth angle is
varied. Data when the connector of the V pol became loose (around 15:30 UTC time)
is omitted in Figure 2.18. Results show the APB on data always to be lower than that
of the APB off data, with the difference increasing as 160 degrees azimuth (looking
directly at the ARSR) is approached. At 160 degrees azimuth, differences on the
order of 20K are observed, while the difference reduces to around 1-2 K at azimuth
angle 0 degrees. The APB on data also shows much smaller variations among samples
at a fixed azimuthal angle compared to the APB off results (with the exception of the
period during which the vertically polarized IF signal was lost), and APB on data
never shows the extremely high brightnesses obtained in some of the APB off cases
(up to 350 K). These behaviors are similar between the two polarizations. Again
these results, combined with the calibration studies of Section 2.3.4 show that the
APB algorithm is extremely effective in this RFI environment, even with apparently
“weak” pulses when integrated over the radiometer bandwidth.
Figure 2.20 is a plot of the percent of samples blanked by the APB processor
during the period 13:25 to 16:06 UTC. Results for both the antenna and reference
load observations are included. A base level of around 1.6% of samples blanked is
observed for the reference load observations throughout most of this period, consistent
51
V pol, blanker on
Sample
300
100
250
200
200
300
150
400
100
500
50
600
0
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 2.16: Calibrated V-pol LISR data in Integration, APB on mode, versus RF
frequency. Refer to Table 2.2 for information on the vertical axis of the plot.
52
H pol, blanker on
Sample
300
100
250
200
200
300
150
400
100
500
50
600
0
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 2.17: Same as Figure 2.16, but for H-pol.
53
Doy 167: V pol mean brightness 1399−1428 MHz
350
APB off
APB on
V pol Brightness (K)
300
0°
70°
100°
135°
160°
0°
250
200
150
100
50
60
70
80
90
100
Minutes past 14:00 UTC
110
120
Figure 2.18: Calibrated V-pol LISR data in the band 1399-1428 MHz versus time,
for both APB off and APB on modes. Text in the plot indicates the azimuth angle
of the radiometer antenna, with regions between the vertical lines indicating periods
of antenna rotation.
54
Doy 167: H pol mean brightness 1399−1428 MHz
350
APB off
APB on
H pol Brightness (K)
300
0°
70°
100°
135°
160°
0°
250
200
150
100
50
60
70
80
90
100
Minutes past 14:00 UTC
110
120
Figure 2.19: Same as Figure 2.18, but for H-pol.
with expected statistical properties of the incoming noise signal and the chosen APB
parameters. The percent of antenna samples blanked is consistently higher than that
of the reference load, with the exception of the period of time during which the
antenna observed the sky at elevation angle 50 degrees. Following these observations,
the percent of antenna samples blanked is found to increase as the radar observation
direction is approached, up to around 4.5% in some cases. Given the radar’s reported
duty cycle of 1.44%, combined with the basic noise power blanking of 1.6%, the
obtained value near 3% for many of the antenna observations appears reasonable.
Higher values indicate that a single radar pulse is resulting in multiple APB 40 µsec
blanking windows, which is a possibility for the APB processor. Overall, the behavior
of these statistics is consistent with expectations for the APB system.
55
APB Statistics 167
5
Antenna
4.5
Reference load
Percent of samples blanked
4
3.5
3
2.5
2
1.5
1
0.5
0
0
50
100
Minutes past 13:00 UTC
150
200
Figure 2.20: Percent of samples blanked by the APB processor
One variation that has not been explained involves the increased blanking levels
observed for times near 13:25 UTC (as the experiment was beginning). This might
be related to an error in the tabulated value of the azimuth angle.
2.4.3
Capture mode data
In capture mode, 6 256 K captures sampled at 10 nsec (a total 15.7 msec) are
acquired on each antenna observation. This high time resolution allows detailed
temporal properties of the RFI sources to be observed. Note that each capture is 2.6
msec long, near the PRF rate of the radar system, so that each capture contains at
56
Start
Time
13:30:49
14:19:42
14:26:09
14:58:42
15:20:07
15:30:50
15:38:44
15:56:49
15:59:34
16:06:36
16:09:50
18:34:17
18:39:04
19:44:12
19:45:03
19:46:07
19:50:20
Stop
Time
13:32:54
14:20:12
14:26:33
14:59:03
15:20:19
15:31:34
15:39:00
15:57:15
15:59:40
16:06:53
16:10:04
18:38:12
18:43:30
19:44:32
19:45:35
19:47:09
19:51:23
LO
(MHz)
1440.5
1440.5
1440.5
1440.5
H-pol
state
A
A
A
A
A
A
A
A
A
A
A
A
A
41
41
41
41
V-pol
state
T
T
T
A
A
A
A
A
A
A
A
A
A
A
A
A
A
El
(deg)
1
50
50
1
1
1
1
1
1
1
44?
0
0
0
0
0
0
Az
(deg)
0
0
0
0
100
135
135
160
0
0
0
x
x
170
170
170
170
Comment
Radar@18:35:53
Radar@18:39:55
Table 2.3: Summary of Capture mode observations on June 17th. LO frequency
is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows:
A=antenna, T=terminated, and a number indicates an amount of attenuation in
dB added between the antenna output and radiometer receiver input.
most one radar pulse, and in some cases, no radar pulses. Table 2.3 summarizes the
capture data measured on June 17th.
A variety of means exist for examining the capture data, including a detailed
pulse detection study of the type described in [41]. Here only a few basic pulse
properties are illustrated. Capture mode data is presented as the raw LISR outputs,
no calibration is applied.
Figure 2.21 plots an example pulse observed near time 14:58:42 in both horizontal
and vertical polarizations. During this time, the radiometer antenna is directed at
azimuth angle 0 degrees, so that radar pulses observed are weak compared to those
57
observed with the antenna directed toward the radar. The horizontal axis in the
plot is time in microseconds; only a portion of the 2.6 msec capture is plotted. The
pulse observed is significantly higher in amplitude than the surrounding noise power,
and slightly stronger in vertical than in horizontal polarization. The pulse width is
observed to be near 2 µ sec, as reported in the ARSR literature, although other cases
show longer pulses due to possible multi-path effects.
Pulse near doy 167 14:58:42 UTC
5000
H pol
V pol
4500
4000
Pulse amplitude
3500
3000
2500
2000
1500
1000
500
0
0
2
4
6
8
10
12
Time (µ sec)
14
16
18
20
Figure 2.21: Example radar pulse observed near time 14:58:42; amplitude level of V
pol data is shifted by 2500 for clarity purposes
Several radar pulses are illustrated Figure 2.22, which plots a set of captures on
the vertical axis along with time in µsec on the horizontal axis. The raw LISR data
here is plotted in decibels, and the captures utilized were obtained during a scan of
58
the radiometer antenna in azimuth near time 18:39 UTC. The vertical axis label of
this plot explains that captures 1 through 12 were obtained near azimuth angle -90
degrees, while captures 13 through 19 were obtained near azimuth 160 degrees, and
captures 20 through 30 were obtained near azimuth 0 degrees. These three cases
contain moderate, strong, and weak radar pulses, respectively. The figure arranges
the pulses in time by attempting to place the maximum pulse amplitude observed at
time zero; this results in some shifting of the pulses between captures, but still allows
comparison of these cases.
Results show the expected near 2 µsec pulse width in the moderate and weak
RFI cases, but a significant increase in the pulse width to 3-4 µsec during direct
observation of the radar. Further analysis of these data shows clear evidence of
receiver saturation during times in which the radiometer antenna was directed toward
the ARSR location. An strong correlation of the pulse amplitudes observed with the
radiometer antenna azimuth angle is also clearly present. Spectral analysis of these
pulses shows their spectrum to be centered near 1414 MHz.
To allow analysis of a large number of capture files, a reduction process was
utilized in which capture power was integrated to a time resolution 1.28 µsec and
stored. Figure 2.23 is a plot of the maximum raw data value observed following this
power integration for a set of captures taken from times 18:34 to 18:44 UTC. During
this period, the antenna azimuthal angle was increased in a continuous sweeping
process from directly North at 18:34 through a full rotation then to 60 degrees West
of North at 18:37:30. The antenna was directed toward the radar at times near 18:36.
The antenna azimuth remained fixed toward the 60 degrees west of North from times
18:37:30 to 18:38:50. The rotation was reversed at 18:38:50, reaching the radar at
times near 18:40 and paused at 18:42 again at 60 degrees West of North.
59
H pol Time domain pulses near 18:39 UTC: Amplitude in dB
(1:12)−90 Az, (13:19) 160 Az, (20:30) 0 Az
80
75
5
70
10
65
15
60
55
20
50
25
45
30
−5
40
−4
−3
−2
−1
0
1
Time (µ sec)
2
3
4
5
Figure 2.22: Thirty radar pulses obtained during a sweep over azimuth near time 18:39
UTC in horizontal polarization. The vertical axis label indicates the approximate
azimuth angle of each of three sets of captures.
60
Maximum capture values obtained from the horizontally polarized antenna observations show an obvious correlation with the azimuth sweeping process described,
along with an obvious absence of these effects in the reference load captures. Note
the occasional small capture maxima obtained occur when a 2.6 msec capture fails
to contain a radar pulse. The broad maxima versus time in the antenna data again
indicate saturation effects in the receiver.
Doy 167 H pol
100
Antenna
Reference
Capture Raw Date Maximum (dB)
95
90
85
80
75
70
65
60
34
35
36
37
38
39
40
Minutes past UTC 18:00
41
42
43
44
Figure 2.23: Maximum capture raw-data amplitudes observed (following integration
to 1.28 µsec resolution) in horizontal polarization during sweeps over azimuth from
18:34 to 18:44 UTC. Reference data is interpolated in the intervals with no measurements. Refer to the text for information on the map from time to azimuthal
observation angle.
Polarimetric processing of the capture data is potentially possible, given the coherent recorded simultaneous horizontally and vertically polarized data. A simple
61
observation of polarization correlations among the capture data showed the expected
noise-like behavior when pulses were not present, along with dramatic increases during pulse observations.
2.4.4
Post-processing
Although the APB algorithm has been shown to be effective against the primary
interference source in this dataset (the ARSR), other more continuous sources were
also observed that are not affected by the APB. However the high spectral resolution
of LISR still allows cross-frequency mitigation algorithms to be developed for removal
of these sources in post-processing. Such algorithms along with additional pulsed
source removal algorithms can be applied to both APB off and APB on data in an
attempt to discern the degree of RFI remaining within data acquired in these modes.
Two primary algorithms were developed for this purpose. The first algorithm is
primarily for the removal of pulsed-type sources, while the second is for CW interferers. The former algorithm utilizes the raw LISR data, while the latter utilized
calibrated data, both of which are obtained as 205 1024 point spectra (each representing 1.3 msec) for each antenna observation.
The pulsed source algorithm examines the maximum and mean values for the 205
raw LISR data points in each frequency bin. The max to mean ratio is computed
and used as the detection statistic; each antenna observation thus produces 512 ratio
values for a given antenna observation and polarization. The 512 ratio values are
sorted, and the mean and standard deviation of the lower 85% of these values are
computed. Bins for which the ratio value is greater than 5 standard deviations from
its mean are then deemed corrupted throughout the antenna observation. Calibrated
brightnesses corresponding to the corrupted bins as well as the brightnesses within 4
62
bins of a corrupted bin are all replaced with the mean brightness of the points deemed
RFI-free. Due to the emphasis on time variations of the max over mean ratio, this
algorithm is suited for detecting and removing pulsed sources.
The cross-frequency algorithm is applied following the pulsed source algorithm.
In this algorithm, the 512 point LISR brightnesses at 269 msec time resolution are
integrated over frequency to a 3 MHz spectral resolution. An interpolated version of
this lower frequency resolution curve is then generated at the 512 point spectral resolution. The detection statistic utilized is obtained by subtracting the low frequency
resolution curve from the original LISR brightnesses. Again the mean and standard
deviation of the low 85% values of this statistic are computed, and RFI detection is
declared when the LISR brightness exceeds the low frequency resolution curve by 3
standard deviations. Bins deemed as containing RFI have their brightnesses as well
as those of the surrounding 4 bins replaced by the mean brightness of the RFI-free
channels. Due to the emphasis of this algorithm on detecting variations in frequency,
this is a narrowband source detection algorithm.
Codes for both these algorithms were developed and applied to the LISR observed
data for this campaign. Results from this process will be illustrated in the following
detailed analysis of two specific datasets.
2.5
2.5.1
Observations Near 14:51 and 16:03 UTC
Observations near 14:51 UTC
At times between 14:50 and 15:00 UTC, the radiometer antenna was directed to
the North (away from the radar) at elevation angle 1 degree. The LO frequency was
set to 1440.5 MHz. This configuration represents a fairly weak RFI environment, due
63
to the small antenna gain toward the radar. Data was acquired in both APB off and
APB on modes during this period of time.
Figure 2.24 is an image of vertically polarized brightnesses in APB off mode
from times 14:51 to 14:56. Though this is a somewhat weak RFI environment, ARSR
transmissions (1414 MHz) are obvious in the figure, as well as narrowband interference
near 1400 MHz. Horizontally polarized images are similar and are not shown.
Figure 2.25 illustrates the same data as in Figure 2.24, but following application
of the post-processing RFI removal techniques described in the previous section. The
processing algorithms have clearly removed both the 1414 and 1400 MHz sources,
replacing them by the mean over all brightnesses deemed acceptable. While this
process produces some distortion of the image, the overall effect is to remove RFI
while retaining a reasonable mean brightness over the radiometer passband.
Figure 2.26 compares mean channel brightnesses in both horizontal and vertical
polarizations for these images before and after the RFI removal processing. The
channels considered correspond roughly to those of the UM ADD system, although a
simple rectangular weighting over a 3 MHz bandwidth is utilized here rather than a
particular filter shape. Results show the algorithm to produce a dramatic reduction
in channel brightnesses near the 1414 MHz source (around 10 K in the fifth channel),
while showing only small corrections in other channels. Note the narrow bandwidth
of the 1400 MHz source results in this source having almost no effect even on the first
modeled channel.
Figure 2.27 is an image of calibrated V-pol brightnesses in APB on mode for the
period 14:56:43 to 14:58:18. Results show the APB algorithm again to be effective in
removing ARSR emissions; the 1400 MHz source however is not affected, suggesting
again that this source has a more continuous nature.
64
Vertical APB off 167−145142
240
20
220
40
200
60
Sample
180
80
160
100
140
120
120
140
100
1400
1405
1410
1415
Frequency (MHz)
1420
1425
80
Figure 2.24: Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency, near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table 2.1)
65
Vertical RFI removed APB off 167−145142
240
20
220
40
200
Sample
60
180
80
160
100
140
120
120
140
100
1400
1405
1410
1415
Frequency (MHz)
1420
1425
80
Figure 2.25: Same as Figure 2.24, but following post-processing described in Section
2.4.4
66
Horizontal, APB off 167−145142
Mean brightness (K)
165
Original
Automatically Corrected
160
155
150
145
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
1420.92
1423.95
1420.92
1423.95
Vertical, APB off 167−145142
Mean brightness (K)
190
185
180
175
170
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
Figure 2.26: Calibrated LISR data in channels similar to the UM ADD system, near
times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table 2.1); results included
before and after post-processing
67
Vertical APB on 167−145642
240
5
10
220
15
200
20
Sample
180
25
30
160
35
140
40
120
45
100
50
1400
1405
1410
1415
Frequency (MHz)
1420
1425
80
Figure 2.27: Calibrated V-pol LISR data in Integration, APB on mode, versus RF
frequency, near times 14:56:43 to 14:58:18 (Samples 363-416 from Table 2.2)
68
Horizontal, APB on 167−145642
Mean brightness (K)
154
Original
Automatically Corrected
152
150
148
146
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
1420.92
1423.95
1420.92
1423.95
Vertical, APB on 167−145642
Mean brightness (K)
178
176
174
172
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
Figure 2.28: Calibrated LISR data in channels similar to the UM ADD system, near
times 14:56:43 to 14:58:18 (APB on, Samples 363-416 from Table 2.2); results included
before and after post-processing
69
Figure 2.28 compares mean channel brightnesses in APB on mode before and after
RFI removal post-processing. In this case, brightnesses remain within 0.5K before
and after post-processing, indicating that the APB algorithm has removed detectable
ARSR emissions. The small changes observed are likely due to false detections in postprocessing. A comparison of brightnesses in Figures 2.26 and 2.28 shows variation of
up to 3-4 K between corrected APB off and APB on data. The source of these changes
is not known, but because the measurements are not simultaneous, it is possible that
variations in the environmental brightness at this level could occur over this time
span, as well as possible errors introduced by temperature variations in the receiver.
2.5.2
Observations near 16:03 UTC
A similar dataset is available from times 16:03 to 16:07 UTC; at this time the
antenna configuration was identical to that at times near 14:50. Again both APB
off and on data are available, and images are similar to those shown in the previous
section.
Figure 2.29 is an plot of mean brightnesses before and after post-processing for the
APB off mode, while Figure 2.30 is the corresponding plot with the APB on. In this
dataset, the ARSR emissions are weaker than those at 14:51, perhaps due to a slight
change in the radiometer azimuth angle. Results show the post-processing algorithm
again to be effective at removing ARSR emissions near channel 5, although in this case
the maximum correction is only approximately 2 K. Results in APB on mode show
a much smaller effect of post-processing, again indicating that the APB is removing
ARSR pulses even when their net effect on channel brightnesses is weak. Comparison
of corrected brightnesses in Figures 2.29 and 2.30 shows them to be within 1-2 K, a
reasonable difference given the time delay between the two observations.
70
Horizontal, APB off 167−160339
Mean brightness (K)
156
Original
Automatically Corrected
154
152
150
148
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
1420.92
1423.95
1420.92
1423.95
Vertical, APB off 167−160339
Mean brightness (K)
180
179
178
177
176
175
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
Figure 2.29: Calibrated LISR data in channels similar to the UM ADD system, near
times 16:03:41 to 16:04:14 UTC (Samples 1358-1379 from Table 2.1); results included
before and after post-processing
71
Horizontal, APB on 167−160532
Mean brightness (K)
156
154
Original
Automatically Corrected
152
150
148
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
1420.92
1423.95
1420.92
1423.95
Vertical, APB on 167−160532
Mean brightness (K)
182
180
178
176
174
1402.95
1405.98
1408.96
1411.94
1414.96
1417.94
UMich Radiometer Channel Center Freq (MHz)
Figure 2.30: Calibrated LISR data in channels similar to the UM ADD system, near
times 16:05:32 to 16:06:06 (APB on, Samples 613-636 from Table 2.2); results included
before and after post-processing
72
Overall these results quantitatively demonstrate LISR capabilities at detecting
and removing RFI through the use of high temporal and spectral resolution information. The APB is also quantitatively demonstrated to remove pulsed sources so that
further post-processing yields no pulsed source detection, even in apparently weak
RFI environments.
2.6
2.6.1
Laboratory Observations on June 20th and 21st
June 20th data
Observations on June 20th involved removal of the radiometer antenna from the
truck boom and placing it on the ground in an upward viewing geometry (near elevation angle 62 degrees). Artificial RFI was introduced into the experiment by placing
a small log-periodic dipole antenna on the horn antenna aperture; this antenna was
used to radiate power produced by a bench source. Source emissions were centered
at either 1398 MHz or 1412 MHz, and were varied to either CW or pulsed type transmissions. Pulses were set to use a PRF of either 360 or 720 Hz, along with pulse
widths of either 2 or 4 µsec. Source output power was varied over a range of 35 dB,
from -20 dBm to -55 dBm.
Figures 2.31 and 2.32 are images are H-pol brightnesses from June 20th, in APB
off and APB on modes, respectively. A detailed log of the map from sample number
in the figures to source properties is not listed here, but the presence of RFI of varying
intensity at 1398 and 1412 MHz is obvious in Figure 2.31. In addition, narrowband
emissions at 1400 and 1403 MHz are again observed, though these were likely not
produced by the RFI generator, but rather are sourced by emissions from other
electronic equipment. The APB on image in Figure 2.32 shows a dramatic reduction
in RFI; the sources observed near 1412 MHz occur during the period of CW operation
73
Doy 170 H pol APB off
200
100
180
200
160
300
140
400
Sample
120
500
100
600
700
80
800
60
900
40
1000
20
1100
1400
1405
1410
1415
1420
Frequency (MHz)
1425
1430
0
Figure 2.31: Calibrated H-pol LISR data in Integration, APB off mode, versus RF
frequency, on June 20th
74
Doy 170 APB on H pol
200
180
100
160
140
200
Sample
120
300
100
80
400
60
40
500
20
600
1400
1405
1410
1415
1420
Frequency (MHz)
1425
1430
Figure 2.32: Same as Figure 2.31 but in Integration, APB on mode
75
0
of the RFI generator only (samples 70 to 125), while pulsed 1412 MHz emissions of
varying power levels were included during samples 126 to 259 and 470 to 620. Periods
when the 1398 MHz source is observed correspond to periods when the RFI source
power was set to lower values: due to this low power level as well as the fact that the
1398 MHz source is reduced by the radiometer front end filters, these pulses were of
insufficient amplitude to be detected by the APB algorithm, and therefore remain in
the data. However, data between the two periods of 1398 MHz interference in Figure
2.32 (i.e. samples around 320-420) contain 1398 MHz interference at a power level
sufficient to trigger the APB detector. Again the APB algorithm does not affect the
1400 and 1403 MHz emissions.
While examination of the APB on mode brightness integrated over the radiometer
bandwidth as RFI source properties are varied is of interest, such comparisons are
hindered by the fact that the RFI source generator apparently also produced a lowlevel, broadband noise output that varied with the generator power level. Analysis
of capture datasets from June 20th and 21st were performed to provide evidence of
this fact. Therefore no further analysis of this dataset is reported here.
2.6.2
June 21st data
Data on June 21st was acquired inside the laboratory, and involved observations
of a well characterized liquid nitrogen target, with additional RFI introduced through
a coupler into the radiometer datastream. These measurements were conducted only
in a single polarization, with the other receiver polarization input being terminated.
Again RFI source properties were varied throughout these observations in a manner
similar to that used on June 20th. In addition, pads of 0, 1, 2, or 3 dB were introduced
onto the calibration target input in order to produce well-characterized changes in
76
the calibration target brightness. Mean brightnesses observed by LISR in this process
were reported in Section 2.3.4.
Figure 2.33 is an image of V-pol APB off brightnesses during the time period
18:15 to 19:26 UTC. Variations in the mean brightness are observed throughout this
plot as pads are added or removed from the calibration target. RFI emissions at
1398 and 1412 MHz are also obvious, although the period from sample 452 to 518
is specified as including 1412 MHz source emissions at power level -83.5 dBm. Note
apparent sources at frequencies less than 1398 MHz or larger than 1427 MHz are due
to calibration problems for these far out-of-band frequencies when the LO is set to
1440.5 MHz.
Figure 2.34 is an image of H-pol APB on brightnesses during the time period
19:33 to 20:30 UTC. Note the APB mode was varied only at time 19:30 so that APB
off and on mode data are not available for observations with identical source settings
and polarizations. Results show all 1412 MHz emissions to be removed, while 1398
MHz emissions remain only in cases where the source power level was set to -63 dBm
so that pulse powers when filtered by the front end filters of the radiometer were
insufficient to trigger the APB detector.
2.7
Summary and Remarks
Results of Canton campaign show the capabilities of the LISR system at detecting
and removing pulsed interference in real-time while maintaining calibration of pulsefree noise, as well as the RFI detection and removal that can be accomplished in
post-processing based on LISR’s high temporal and spectral resolution. Analysis of
the capture data also allows detailed properties of ARSR emissions to be examined. It
was shown that the APB algorithm is sufficiently conservative so that data is removed
77
Doy 171, V pol APB off
100
200
Sample
300
400
500
600
700
800
1390 1395 1400 1405 1410 1415 1420 1425 1430 1435
Frequency (MHz)
Figure 2.33: Calibrated V-pol LISR data in Integration, APB off mode, versus RF
frequency, on June 21st
78
Doy 171, H pol APB on
350
50
300
100
150
250
Sample
200
200
250
300
150
350
100
400
450
50
500
550
1400
1405
1410
1415
1420
Frequency (MHz)
1425
1430
0
Figure 2.34: Same as Figure 2.33 but for H-pol in Integration, APB on mode
79
only when obviously corrupted, and hence is suitable for implementation in a space
based radiometer. Algorithms utilized here in post-processing for RFI removal were
the starting point for the algorithms used in campaigns that will be described in the
next few sections, as well as the simplified algorithm used in theoretical comparisons.
80
CHAPTER 3
L-BAND INTERFERENCE SUPPRESSING
RADIOMETER (LISR) GROUND-BASED
OBSERVATIONS WITH JPL’S PASSIVE-ACTIVE L/S
BAND (PALS) SENSOR
Between April 24th and May 12th, 2006, a ground-based measurement campaign
was conducted at the Jet Propulsion Laboratory (JPL) (Pasadena, California) to
demonstrate advanced radio frequency interference (RFI) mitigation methods for Lband microwave radiometry. Backend RFI mitigating systems included in the campaign were the L-Band Interference Suppressing Radiometer of The Ohio State University , ADD of the University of Michigan, and the Double-Detector system developed by NASA Goddard Space Flight Center as was the case in Canton campaign.
These three backend systems simultaneously observed IF signals provided from the
L-band radiometer of the Passive/Active L-S band sensor (PALS) of JPL [51,52]. The
goals of the campaign included identification and mitigation (if possible) of RFI observed by PALS as well as intercomparison of the results from the multiple backends
in order to assess the RFI mitigation performance of each.
This chapter describes results obtained by the LISR backend along with comparisons of brightness temperatures with other sensors when available. The next section
describes the basic components of the PALS radiometer front-end, modifications to
the LISR backend made for this campaign and the modes of observation used for
81
LISR. An overview of the dataset and its basic properties, along with a description
of the method used for state identification and calibration, is provided in Section 3.2.
Section 3.3 then summarizes LISR observations on specific dates of the campaign,
while Section 3.4 describes properties of the RFI observed and RFI mitigation techniques developed for these sources. A final analysis and overall conclusions for this
campaign follow in Section 3.5.
3.1
3.1.1
System Configuration
PALS radiometer front end and downconversion stages
The Passive/Active L/S-band Sensor ( [51,52]) of JPL has been deployed in several
remote sensing field campaigns in the past, and has provided important information
regarding the use of combined radar/radiometer instruments for observing several
geophysical quantities while emphasizing soil moisture and sea surface salinity measurements in particular. Only the radiometer portion of PALS, which operates at
L-band, was used in the campaign described here.
The PALS configuration for the campaign involved zenith looking sky observations
using a dual-polarized L-band patch array antenna located on the roof of building
168 at JPL, similar to the configuration described in [51]. Given the expected slow
variation with time of the observed sky brightness, any RFI varying rapidly in time
should be readily discernible.
The relevant portion of the PALS radiometer front end has dual channels for
the two polarizations; a hybrid coupler is also included to allow measurement of the
third Stokes parameter, but these data were not recorded by any of the backend
systems, except that of PALS. For each channel, a portion of the front end following
the antenna is located on the roof; this portion includes a Dicke switch to allow
82
observation of a temperature controlled reference load, a coupler to allow injection
of a known noise brightness, a bandpass filter centered at 1415 MHz with a 3 dB
bandwidth of 60 MHz, and a low noise amplifier with a gain of 36 dB. The number of
poles of the bandpass filter was not provided, but it assumed to be a modest number
in order to minimize the impact on system noise figure. The 1415 ± 30 MHz output
of this front end is then fed down cables approximately 100 ft long from the roof into
a laboratory in building 168.
In the laboratory, the RF signals are again bandpass filtered using a 1413 ± 37.5
MHz filter (number of poles again not provided), passed through an amplifier with 31
dB gain, and then mixed with an LO at 1210 MHz to obtain an IF signal occupying
an approximate bandwidth of 205±30 MHz. Note that 67 dB of gain neglecting cable
and other component losses is applied to the nominal band 1415 ± 30 MHz prior to
the mixer; a review of the mixer specifications [53] shows 1 dB compression when the
total input power reaches +7 dBm.
IF signals are then passed through a 300 MHz lowpass filter followed by a final
band-defining filter of 200 ± 10 MHz. After an additional 40 dB of gain and a second
300 MHz low pass filter, a portion of the IF signal (now nominally 200 ± 10 MHz) is
coupled out for use by the backend systems. The remainder of the IF signal passes into
the PALS backend system, which performs measurements of the first three brightness
Stokes’ parameters. Note the use of the 1210 MHz LO frequency results in the IF
band representing the RF frequencies 1400 − 1420 MHz at the 3 dB point, while the
protected portion of L-band ranges from 1400 to 1427 MHz.
The double-detector backend system of NASA GSFC directly observed the PALS
IF provided, but both ADD and LISR used an additional downconversion (LO 173
MHz) and gain stage to center the IF signal at 27±10 MHz. Component details
83
on the second stage downconverter are not available. The final input to the LISR
backend is two 27 ± 10 MHz IF cables containing the horizontally (H) and vertically
(V) polarized radiometer channels as in the Canton campaign.
For calibration purposes, JPL reported that the effective additional brightness
provided by the noise diode was approximately 500 K. Measured front end temperature information was not provided, but JPL personnel stated that the reference
load physical temperature could be reasonably estimated to be 33 C during overnight
hours. These estimates were used in obtaining the LISR calibrations to be reported.
Note JPL local time during the campaign was 7 hours behind UTC; sunset occurred
around 1:30 UTC.
3.1.2
PALS state timing
The PALS front-end state was controlled by PALS during the campaign; the basic
timing cycle consecutively observed the antenna+noise diode (Ant+ND), antenna,
and reference loads for 4.2 msec each. Each 4.2 msec interval was further divided
into 12 350 µsec periods, within which the first 300 µsec were used for radiometer
measurements while the following 50 µsec were not used. These 50 µsec are used by
the radar when PALS is performing active/passive measurements; during these times
the radiometer observes the reference load in all states.
Figure 3.1 provides plots of the radiometer state control lines versus time in order
to illustrate the basic timing cycle. The “R-line” plot illustrates the state of the
reference load Dicke switch, with high values indicating that the reference load is
observed. The “N-line” plot corresponds to the noise diode on/off state, with a high
value indicating that the noise diode is on. Finally a high value on the “I-line”
represents that radiometer measurements should be performed; the low values of the
84
R−Line
PALS State Diagram
1
0
2000
4000
I−Line
8000
10000
12000
REF
ANT
ANT+ND
1
0
N−Line
6000
2000
4000
2000
4000
6000
8000
10000
12000
6000
8000
Time (µ secs)
10000
12000
1
0
Figure 3.1: PALS state diagram
85
Figure 3.2: LISR block diagram
“I-line” mark the 50 µsec excluded intervals during each 350 µstate interval. Note
that JPL also recommends that data from the first 350 µsec interval within a 4.2
msec state be discarded.
3.1.3
LISR overview
Figure 3.2 is a block diagram of LISR as used in this campaign. The basic configuration is very similar to that described in Chapter 2 for the Canton campaign, and
will not be repeated here. However, several modifications were required as it will be
readily explained.
86
As in Canton campaign, LISR was operated in three distinct modes: “Integration,
APB off”, “Integration, APB on”, and “Capture” modes. LISR capture mode in
this campaign was designed to acquire 28K samples (286.72 µsec) during a 300 µsec
antenna or reference observation. Due to the large amount of data recorded and the
need to transfer recorded data to the control computer, a duty cycle of approximately
5% was achieved for capture mode observations (i.e. one 286.72 µsec observation per
18 350 µsec intervals.) However, the acquired data remains useful due to its high
temporal resolution and coherency, which allows detailed studies of the temporal
properties of the observed interference.
With the exception of Section 3.2.2, results from LISR capture mode observations
are not presented in detail in this work. However, conclusions obtained from the
capture mode data are consistent with those to be described in what follows.
For the integrating modes in this campaign (i.e “Integration, APB on” and “Integration, APB off” modes), the basic FPGA output data unit was an integration
of 3 FFT outputs; this data unit consists of the power in 1024 frequency channels
integrated over a time period of 30.72 µsec and reported in 16 bits. These data units
are obtained every 40.96 µsec from the FPGA, because an additional 10.24 µsec is
needed to write the data unit out of the FPGA. Seven of these data units therefore
can be recorded within the 300 µsec radiometer integration period.
This report focuses on the “Integration, APB off” data, and all following data in
an integrating mode should be assumed to be with “Integration, APB off”. Although
the “Integration, APB on” mode has been shown to be highly effective against pulsed
interference in Chapter 2, an analysis of the “Integration, APB on” PALS campaign
data shows little advantage over “Integration, APB off,” due to the configuration of
87
blanker parameters which were not optimized for the relatively rapid switching of
radiometer states utilized.
3.1.4
LISR computer and control interface
The LISR FPGA is free running and not synchronized to any external time control.
FPGA output data units are passed to a “capture board” containing a smaller FPGA
as well as a 16 bit, 256 K FIFO that serves as a buffer to a data-recording and control
computer. In past campaigns including Canton campaign, external time information
such as “trigger” signals that tell LISR when to begin a measurement were typically
passed into the control computer, which then through a program running on the
control computer could reset the buffer FIFO’s to begin recording data following a
trigger signal.
This process makes clear that LISR was designed primarily for radiometer systems
operating with integration periods on the order of milliseconds. In such systems, an
increased level of software system control is acceptable, because time delays introduced by software components remain negligible compared to the overall radiometer
time scales. In this campaign however, timing intervals as short as 50 µsec required
modification of several LISR computer and control aspects in order to achieve reasonable measurements.
To avoid delays introduced by the software triggering described above, LISR was
modified to input the trigger line directly into the capture board, allowing the trigger
signal to reset the FIFO’s directly. Software on the control computer was revised to
communicate with the capture board so that FIFO resets are allowed only when the
computer is ready. This modification allowed successful hardware triggering of LISR
in the campaign.
88
FIFO output data is passed to the control computer through a National Instruments high speed digital I/O card. An additional system timing issue was introduced
by total latencies and transfer times on the order of 1.75 msec associated with this
card; such delays are encountered following every 350 µsec period observed due to
the absence of synchronization control of the LISR main FPGA.
Modifications to LISR were implemented to reduce the impact of these latencies.
FPGA data units were modified from the previous width of 32 bits to 16 bits; this
change allowed only a single FIFO to be read instead of two FIFO’s (there are 2
FIFO’s actually on the capture board) read in the Canton campaign. The LISR
FPGA was modified appropriately, including a saturation of the data unit output if
16 bits were found insufficient.
While the data recorded during a 300 µsec radiometer observation consists of
7 1024 point spectra each integrated for 30.72 µsec, the free running nature of the
LISR FPGA also requires that the first of these spectra be discarded, since the precise
time at which the first integration began relative to the trigger signal is not known.
Therefore the final LISR data unit consisted of 6 30.72 µsec spectra, or 184.32 µsec
observed out of 300 µsec available. Given the required 1.75 msec to transfer this data,
the final duty cycle of LISR in the integrating modes is around 10%, and the final
data output for a single trigger is recorded approximately every 2.1 msec. Following
the recording of 8743 of these data units in approximately 18.6 seconds, the LISR
computer paused to write data to its internal hard drive. To reduce data storage
requirements on this hard drive, the average of the 6 obtained spectra in each trigger
was recorded; the final dataset consists of 512 point spectra in each of H and V pols
representing the bands 1383 to 1433 MHz at 184.32 µsec time resolution. During
the experiment, LISR alternated between recording files successively in “Integration,
89
APB off” and “Integration, APB on” modes, with every twentieth file in “Capture”
mode. Files were approximately 17 MB in all modes; the final data rate was around
2.6 GB per hour.
3.2
3.2.1
Dataset Overview and Calibration
Observed gain pattern
To illustrate basic spectral properties of the PALS IF signal, Figures 3.3 and 3.4
plot the mean raw power (in dB) versus frequency obtained by LISR for reference load
observations in vertical and horizontal polarizations, respectively. Data is normalized
so that the maximum value of the frequency response is 0 dB. Averages are taken over
an entire file (around 18.6 seconds real time, 0.54 seconds reference load integration
time) for two example files acquired within 13 minutes of each other during the
April 24th observations. Lines in the Figures mark the approximate gain level of
-3 dB, and confirm a 3 dB passband of about 1400 (one of the protected spectrum
boundaries) to 1420 MHz. The gain at the protected spectrum boundary 1427 MHz
is found to be around 13-16 dB below the maximum. The figures show relatively
similar but non-identical behaviors for vertical and horizontal polarizations, due to
the varying frequency responses of these separate receiver channels. Passband plots
for the two files show slight differences, indicating the potential for change in the
passband characteristics with time. Such changes are observed in calibrated data as
well, and will be described further in Section 3.3.2.
While this passband appears acceptable, it seems somewhat wide with regard
to possible RFI effects for a spaceborne instrument. Given the possibility of RFI
sources up to 1398-1399 MHz, the passband of the instrument utilized may not provide
sufficient reduction of these slightly out-of-band sources. It is recommended in future
90
Normalized Average Raw Power in Reference Observations for Vertical Polarization
5
data_114_222946_000002_00
data_114_224200_000002_00
Normalized Power in dB
0
−5
−10
−15
−20
1395
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 3.3: LISR measured reference load raw power versus frequency, vertical polarization
91
Normalized Average Raw Power in Reference Observations for Horizontal Polarization
5
data_114_222946_000002_00
data_114_224200_000002_00
Normalized Power in dB
0
−5
−10
−15
−20
1395
1400
1405
1410
1415
Frequency (MHz)
1420
1425
Figure 3.4: LISR measured reference load raw power versus frequency, horizontal
polarization
92
deployment of the PALS system at a minimum that an LO frequency of 1213 MHz
be used, in order to place the observed passband more within the boundaries of the
protected spectrum. A narrower second RF passband filter or filters could also be
advantageous, although such filters would require a large number of sections.
3.2.2
Noise diode delay
One radiometer state issue that appeared in analysis of LISR capture mode data
involves apparent transients in the noise diode turn on response. Figure 3.5 plots the
horizontally polarized raw power vs. time obtained from LISR capture mode observations averaged over all Ant+ND state observations from a single file obtained on
April 27th. Plots for vertical polarization (not shown) show nearly identical behaviors. A settling time of about 100 µsec following the trigger signal (time zero in LISR
capture mode observations) is observed from the plots. There is another transient
state beginning around 260 µsec, but the 286.72 µsec capture duration does not allow analysis of the turn off response for the full 300 µsec interval. This behavior was
observed throughout the experiment for Ant+ND states, and remains relatively consistent for individual Ant+ND triggers. Tests performed following the JPL campaign
showed that capture mode delays inherent in the LISR system were less than 1 µsec
following a trigger signal. Data acquired by GSFC at 2 µsec time resolution (and
using the 200 MHz IF of PALS directly) also showed a similar transient response. It
appears that this delay is related to a transient response within PALS, although JPL
personnel have indicated that the source of this transient is unknown at that time.
As a result of this behavior, the noise diode effective temperature reported for
PALS, while still allowing accurate PALS calibration, is likely to be inaccurate for
systems not integrating over precisely the time period used by PALS, including LISR
93
6
x 10
Averaged Raw Power over Time for File data_117_234605_000105_02
1.6
1.5
1.4
Raw Power
1.3
1.2
1.1
1
0.9
0.8
0.7
50
100
150
Time (µ sec.)
200
250
Figure 3.5: Raw power vs. time in the Ant+ND state for the capture data
94
due to the use of only 184.32 out of 300 µsec available. Such inaccuracies will create
biases in final calibrated brightness temperatures. Therefore the ability to quantitatively compare brightnesses among the multiple systems within the campaign is
limited. Qualitative comparisons remain possible however, and can still provide a
good indication of RFI detection and mitigation effectiveness.
3.2.3
LISR recorded datasets
The campaign began on Monday, April 24th (local time), with overnight PALS
observations performed (local time, beginning day) on the 24th, 25th, 26th, and 27th.
The campaign was then halted due to other commitments by campaign personnel,
and resumed for overnight observations on May 8th, 9th, 10th, and 11th. Table 3.1
summarizes datasets recorded by LISR during the campaign. While several small
datasets were often recorded during the day, and are included here for completeness,
the larger datasets were primarily acquired during overnight observations.
The best overnight datasets acquired by LISR occurred on the 24th, 25th, and
27th, with a smaller overnight dataset obtained during the early morning of May
11th. At other times, LISR encountered either long-term saturation or missing data
problems. The latter were later found to be associated with a loose wire on one of
the ADC card power supply inputs; unfortunately correction of this problem was
exacerbated by the fact that OSU personnel were unable to be present at JPL during
the second campaign week. The source of the long-term saturation problem is not
known; while this could be caused by the presence of RFI, by changes in overall system
gain, or intermittent LISR changes, it is difficult to reach any definitive conclusion.
95
Day (UTC)
April 24th
April 24th
April 24th
April 24th
April 24th-April 25th
April 25th
April 25th
April 25th
April 25th-April 26th
April 26th
April 26th
April 27th
April 27th
April 27th
April 27th
April 27th
April 27th-April 28th
May 10th
May 11th
May 11th
May 11th
May 11th
May 11th
May 11th
May 11th
Start time
22:07:01
22:28:59
22:41:13
22:07:01
22:59:21
19:06:04
19:23:44
19:25:19
22:58:27
22:07:01
22:23:40
17:13:31
17:19:41
18:10:31
18:40:24
21:23:55
22:39:49
22:24:13
03:46:50
05:01:09
05:18:22
06:01:48
10:39:40
11:58:50
15:18:27
End Time
22:08:13
22:30:31
22:42:23
22:08:13
14:45:15
19:07:18
19:24:53
19:26:51
14:56:52
22:18:59
22:35:50
17:13:50
17:22:35
18:18:10
18:41:14
21:50:01
15:21:57
22:24:36
03:47:13
05:01:32
05:58:49
06:28:45
10:41:55
13:23:19
15:25:05
Description
Overnight Run
LO tests
LO tests
Overnight Run
LO tests
LO tests
LO tests
Overnight Run
Part of overnight
Part of overnight
Part of overnight
Part of overnight
Part of overnight
Part of overnight
Part of overnight
Part of overnight
data
data
data
data
data
data
data
data
Table 3.1: Summary of the data recorded in the experiment
96
3.2.4
LO tuning tests and 1390 MHz interference
The files marked “LO tests” in Table 3.1 were measured during periods in which
either the PALS LO of 1210 MHz or the second stage downconverter LO of 173
MHz were varied. Figure 3.6 illustrates calibrated brightnesses (calibration procedure described in the next sections) in horizontal polarization for the 18:10:31 UTC
observation on April 27th. The horizontal axis is the IF frequency, while the vertical axis is UTC time in decimal hours. During this measurement, the PALS LO
was tuned from its original value of 1210 MHz, so that the IF frequencies of any
external RFI sources are changed. Note a strong source of brightness around IF
frequency 17 MHz in the Figure whose frequency is changed as the LO is changed;
this source is likely to be RFI at a frequency of 1400 MHz, as will be more apparent
in the next Section. Movement of the PALS passband through varying portions of
the IF band-defining filter is also apparent as the LO is tuned. However, a source of
large brightness around 7 MHz IF frequency is also observed that is not affected by
changes in the PALS LO. This source appears to be an IF source, either at 180 MHz
in the PALS IF band or at 7 MHz following the second downconversion. Analysis
of a dataset in which the second stage LO was tuned seems to indicate that this
source is more likely to be at 180 MHz. This source in future plots of brightness
versus RF frequency will be seen at a frequency of 1390 MHz. Because this source
is outside the passband of the radiometer, it produces only a small contribution to
overall brightness when total power within the radiometer passband is calibrated.
3.2.5
PALS state classification
PALS provided state line outputs to the three backend observing systems for triggering and state classification purposes. However the state lines provided were found
97
Brightness vs. Frequency as LO Frequency Changes
250
18.19
240
18.2
230
Decimal UTC Time
18.21
220
18.22
210
18.23
200
18.24
190
18.25
180
18.26
170
18.27
160
18.28
150
18.29
140
5
10
15
20
25
30
IF Frequency (MHz)
35
40
Figure 3.6: Calibrated brightness temperatures vs. IF frequency as PALS LO tunes
in horizontal polarization
98
to be relatively noisy, resulting in errors in real-time state classification. A method
for classifying PALS states in post-processing was developed to eliminate these problems for LISR. While such a process could be developed based on comparisons with
timing information from the other backend systems, the post-processing method described below has been found to be reasonably accurate while requiring no additional
information other than the LISR data itself.
Figure 3.7 plots LISR raw powers integrated over frequency for “integration, APB
off” mode using triggers obtained from a single file on the night of April 27th. The
horizontal axis is the trigger number (i.e. time), while the vertical axis is the raw
power averaged out throughout the 1024 point spectrum. It can be seen that power
levels cluster into three main groups; the group with the highest power level contains
primarily Ant+ND observations, the middle group contains primarily reference load
observations, and the lowest power level group contains antenna observations.
A simple thresholding procedure was used as a first stage detector of Ant+ND
observations. For example in Figure 3.7, an integrated raw power value of 1050 can
be set as a threshold, and triggers with raw power levels above this deemed to be in
the Ant+ND state. A specific threshold level was found to be applicable throughout
an overnight observation, so that thresholds were determined empirically by looking
at a small number of such power plots for each day where data was collected.
A drawback of this method is the fact that an antenna observation can be confused
with the other states if RFI is present. Bias caused by improper triggering due to noise
on the trigger line may also affect the results. To overcome these problems, a twostage algorithm was developed that utilizes the fact that the Ant+ND observations
are separated by a relatively constant time interval, as can be seen in Figure 3.1. The
algorithm is summarized as follows:
99
File: data_118_043503_000868_00
2000
1800
1600
Raw Power
1400
1200
1000
800
600
400
200
0
1000
2000
3000
4000 5000
Trigger Number
6000
7000
8000
Figure 3.7: Average raw power levels for a single file for the night of April 27th
100
• Saturated or all-zero (missing data) spectra are discarded in all cases. A file
is discarded entirely if more than 2/3 of the data in the file are saturated or
missing.
• A threshold is applied on the frequency averaged raw power for each remaining
trigger to determine the cases likely to be Ant+ND.
• Time differences between states classified as Ant+ND are examined; states are
retained as Ant+ND if time differences between Ant+ND states are less than
4.2 msec or between 8.4-16.8 msec (note that average time delay between each
observation is around 2.1 msec so a longer time delay is highly unlikely).
• Time center of Ant+ND states are estimated by taking the average of the times
of the triggers closer than 4.2 msec. If a second trigger is not present within
4.2 msec of the first one, this trigger is not used in further classifications since
there is too much uncertainty in determining the time center of the state.
• If a trigger is closer than 1.8 msec to an Ant+ND state center time, it is classified as an Ant+ND observation, else if the time difference is between 2.3 and
6.1 msec after the center of And+ND states it is classified as an Antenna observation, and if it is between 2.3 to 5.3 msec before the center of an Ant+ND
state it is classified as a Reference observation. These time differences are based
on the PALS state timing shown in Figure 3.1, with the triggers which are close
to state transitions (and hence more ambiguous) discarded.
• If the resulting number of Ant+ND, Antenna, and Reference triggers found
in a file are each above 1000, calibration is proceeded. Otherwise this file is
discarded.
101
This algorithm was found to provide satisfactory state classifications; less than
1% of the triggers in a given file were left as uncertain (and therefore discarded.)
3.2.6
LISR calibration
Gain and offset parameters for LISR calibration were determined for each frequency sub-channel using reference load, antenna, and Ant+ND data integrated in
time over an entire file (around 18.6 seconds real time, 0.54 seconds antenna observation time.) Calibrated brightness spectra were then computed for each 184.32 µsec
data period. To reduce data management requirements over longer observation periods and also to reduce brightness standard deviations, a second stage integration was
performed over 64 triggers (around 11.8 msec antenna observation time, 409 msec
real time) and stored as a “level one” data. A further integration over an entire file
(18.6 seconds real time, 0.54 seconds antenna observation time) was performed in
plots over time periods on the order of hours. The calibrated brightness temperature
obtained through this procedure is an estimate of the antenna temperature, which
should include a roughly 10 K sky brightness at L-band but also contributions from
any surrounding structures observed in the antenna sidelobes as well as contributions
from losses in the antenna or any components prior to the location of the internal
calibration loads. Brightnesses reported for PALS have apparently removed many of
these contributions through an additional calibration step not utilized for LISR.
Note that, as mentioned in Canton campaign, other backends in this campaign
(including PALS) with less spectral resolution are calibrated using the raw powers
integrated over larger bandwidths initially, which emphasizes the contributions of
stronger portions of the passband while in LISR, the average of the data over larger
bandwidths is taken after the calibration when making a comparison with other
102
sensors. Results from the Canton campaign have shown the resulting error not to
be substantial, however very precise comparisons with the other sensors (beyond the
level of a few K) are affected. These differences are expected to be smaller than those
associated with the noise diode transient issue discussed in Section 3.2.2.
3.3
3.3.1
LISR Overnight Observations
Horizontally polarized brightnesses during the night of
April 24th-25th
The campaign began on April 24th; during the daytime hours, LISR was tested
and found to be functioning without problem. PALS, LISR, and the double detector
system recorded datasets during the overnight period.
Figure 3.8 is a “spectrogram” of calibrated brightnesses in horizontal polarization
for the overnight period starting at midnight (UTC) April 25th. The horizontal axis is
RF frequency while the vertical axis is the UTC time in hours. Individual brightness
spectra illustrated in the figure are integrated over an entire LISR file (around 18.6
seconds real time, around 0.54 seconds of antenna observing time).
The spectrogram of Figure 3.8 illustrates the high spectral resolution of LISR, and
is very useful for identifying RFI or other emissions that are localized in frequency.
The long integration time of Figure 3.8 as well as the large amount of data plotted
reduces the figure’s utility in identifying time-localized interference. Note brightnesses
over the full spectral range of LISR’s observations (1383-1433 MHz) are shown, even
though the instrument passband is nominally 1400-1420 MHz.
Several emissions localized in frequency are observed in Figure 3.8, including
apparent hydrogen-line emissions at 1420.4 MHz, the IF RFI source mentioned in
Section 3.2.3 at 1390 MHz, and narrowband sources near 1397.5 and 1400 MHz.
103
th
April 25 , H pol, APB off
0
250
240
2
230
4
220
UTC (hours)
210
6
200
8
190
180
10
170
12
160
150
14
140
1390
1400
1410
Frequency (MHz)
1420
1430
Figure 3.8: Calibrated H-pol brightness temperature vs. RF frequency, April 25th
104
Classification of the 1420.4 MHz source as natural hydrogen line emissions appears
reasonable given the increased level of these emissions as the galactic plane comes into
view during later parts of the image. The contributions of specific frequency localized
sources to the entire channel brightness can be estimated by comparing brightnesses
integrated over the receiver passband including or excluding these sources. Using this
procedure, the contribution of the hydrogen line to the total channel brightness is a
maximum of 0.2-0.3K while that of the 1400 MHz source is around 0.1 K.
Several instances of apparent time-localized interference are also apparent in the
image, particularly at times from 3 to 9 AM UTC. However, during these periods, a
substantial portion of the data was saturated and was removed. Further discussions
of time domain properties of LISR observed brightnesses and comparisons with PALS
data are provided in Section 3.4.
To illustrate general trends of brightness versus time, Figure 3.9 plots LISR total
channel brightnesses versus time, including both horizontal and vertical polarizations.
LISR channel brightnesses are averaged both in time (to the level of one LISR file)
and in frequency over the channel 1402.5-1417 MHz. PALS brightnesses for the same
period (each point corresponding to 4 seconds of observing time) are also included on
the plot; due to the previously discussed differences in calibration between different
sensors, brightness temperatures are shifted for illustration purposes, as indicated in
the Figure legends. PALS brightnesses are linearly interpolated across the boundaries
of periods of time for which LISR data is missing (see for example around 9 AM
UTC.) The comparison shows overall similar trends in general brightness trends with
time, including relatively higher brightness temperatures near midnight UTC (which
corresponds to 5 pm local time when the sun was still up) and at the end of the
experiment as the galactic center enters the antenna pattern. PALS brightnesses
105
April 25th, Horizontal, APB off
Brightness (K)
196
LISR+7K
PALS+175 K
194
192
190
188
186
0
5
10
15
UTC (hours)
th
April 25 , Vertical, APB off
Brightness (K)
190
LISR−2K
PALS+172 K
188
186
184
182
0
5
10
15
UTC (hours)
Figure 3.9: Comparison of calibrated brightnesses for PALS and LISR, April 25th
show much larger apparent RFI contributions than those of LISR however, likely
due to LISR’s averaging over a longer real time period than that used for PALS, so
that time localized interference is reduced, as well as LISR’s lower overall duty cycle.
LISR nevertheless captures many of the high brightness features observed by PALS.
It is interesting to note that LISR also obtains some brightnesses substantially less
than those of the mean brightness, as well as an abrupt change in the vertical channel
brightness around time 3 AM UTC. The source of these effects is discussed in the
next section.
106
3.3.2
Vertically polarized brightnesses during the night of
April 24th-25th
Figure 3.10 is a spectrogram of the vertically polarized brightness analogous to
Figure 3.8 for horizontal polarization. While many properties of Figure 3.10 are
similar to those of Figure 3.8, a clear difference is observed in an abrupt change in
observed brightness spectral properties beginning at 2:52 AM UTC. This change in
spectral properties occurs at the same time as the abrupt change in vertically polarized total channel brightnesses in Figure 3.9. It is interesting to note in Figure 3.9
that the dramatic change in brightness spectral properties observed in the spectrogram of Figure 3.10 results in only small changes (less than 2 K) in total channel
brightness.
An additional investigation as to the source of these spectral variations was performed by examining spectral properties of the reference load before and after 3 AM
UTC. Figure 3.11 plots reference load raw power versus frequency for horizontal and
vertical polarizations averaged over Figure 3.10 before and after 3 AM UTC. Figure
3.11 shows that the change in vertically polarized brightnesses follows an associated
change in reference load raw power versus frequency, with an oscillating pattern versus frequency occurring in the latter case after 2:52 AM. This change is not observed
in horizontal polarization. The source of this change in observed power spectral
properties is not known; possible explanations include an additional VSWR effect
introduced into the system somehow, a phenomenon associated with receiver saturation in vertical (but not horizontal) polarization, issues in either the first or second
stage downconverters, or problems with LISR associated with one of the ADC cards.
Similar changes in brightness spectral properties are observed (but not as clearly) in
other overnight datasets. A possible receiver saturation effect due to contributions
107
th
April 25 , V pol, APB off
0
250
240
2
230
4
220
UTC (hours)
210
6
200
8
190
180
10
170
12
160
150
14
140
1390
1400
1410
Frequency (MHz)
1420
1430
Figure 3.10: Calibrated V-pol brightness temperature vs. RF frequency, April 25th
108
Change in the Passband Properties for Vertical Polarization (Reference Looks)
800
Before 3 am
After 3 am UTC
Raw Power
600
400
200
0
0
5
10
15
20
25
30
IF Frequency (MHz)
35
40
45
50
Change in the Passband Properties for Horizontal Polarization (Reference Looks)
1000
Before 3 am
After 3 am UTC
Raw Power
800
600
400
200
0
0
5
10
15
20
25
30
IF Frequency (MHz)
35
40
45
50
Figure 3.11: Change in the passband properties of reference looks on April 25th
from either in- or out-of-band sources does not seem impossible due to the high gain
and limited filtering prior to the PALS mixer discussed in Section 3.1.1, but again
conclusions in this regard are not conclusive.
Further discussion of general RFI properties as well as mitigation algorithms for
the observed RFI is provided in Section 3.4.
3.3.3
Overnight observations on April 25th-April 26th
The campaign continued on April 25th, and all three backend systems obtained
data during the overnight observations. Figure 3.12 compares LISR total channel
brightnesses versus time with those obtained from PALS; behaviors similar to those
in Figure 3.9 are observed. Note LISR saturation occurred frequently following 6 AM
109
April 26th, Horizontal, APB off
Brightness (K)
196
LISR+12K
PALS+177 K
194
192
190
188
186
0
5
10
15
UTC (hours)
th
April 26 , Vertical, APB off
Brightness (K)
190
LISR−4K
PALS+172 K
188
186
184
182
0
5
10
15
UTC (hours)
Figure 3.12: Comparison of calibrated brightnesses for PALS and LISR, April 26th
UTC. Calibrated brightness spectrograms are illustrated in Figures 3.13 and 3.14
for horizontal and vertical polarizations, respectively, at times prior to the onset of
frequent saturation at 6 AM UTC. Brightness general spectral properties are similar
to those observed in Figures 3.8 and 3.10, including the abrupt change in vertically
polarized brightness spectral properties around time 2:48 UTC. Narrowband sources
remain similar to those seen on the night of April 24th-25th, although the presence of
a narrowband source at 1395 MHz in vertical polarization is more apparent in Figure
3.14. Apparent time localized RFI in Figure 3.13 shows some evidence of spectral
localization simultaneously near the low (around 1395-1400 MHz) and high (around
1418-1423 MHz) portions of the band; properties of time localized RFI are discussed
further in Section 3.4.
110
th
April 26 , H pol, APB off
250
0.5
240
1
230
1.5
220
UTC (hours)
2
210
2.5
200
3
190
3.5
180
4
170
4.5
5
160
5.5
150
1390
1400
1410
Frequency (MHz)
1420
1430
140
Figure 3.13: Calibrated H-pol brightness temperature vs. RF frequency, April 26th
111
th
April 26 , V pol, APB off
250
0.5
240
1
230
1.5
220
UTC (hours)
2
210
2.5
200
3
190
3.5
180
4
170
4.5
5
160
5.5
150
1390
1400
1410
Frequency (MHz)
1420
1430
140
Figure 3.14: Calibrated V-pol brightness temperature vs. RF frequency, April 26th
112
3.3.4
Overnight observations on April 27th-April 28th
LISR data on the third campaign day encountered frequent saturation and is not
discussed further here. On the fourth campaign day, all three backends acquired data
during the overnight observations. Data from the ADD was made available and is
included in the following discussion.
Figure 3.15 plots total channel LISR brightness temperatures versus time, as well
as the calibrated brightness temperatures of ADD and PALS for both horizontal
and vertical polarizations. Brightnesses are shifted by the constants indicated in
the Figure legend in order to make the curves distinguishable. General brightness
properties are similar to those observed on earlier campaign days, and results from
the three systems show similar general behaviors. No major saturation or missing
data periods were encountered by LISR during the overnight measurement. Abrupt
changes in the vertically polarized LISR total channel brightness are also absent in
Figure 3.15. However, a closer examination of the horizontally polarized data did
show variations in spectral properties of the calibrated brightness over short (order of
10-20 seconds) intervals of time that are not obvious in Figure 3.15. These variations
occurred approximately every 10-11 minutes for times prior to approximately 4:15 AM
UTC; again the source of these variations is not known, and they are only weakly
correlated to changes in the total channel averaged brightness shown in Figure 3.15.
Brightness spectrograms for horizontal and vertical polarization are provided in
Figures 3.16 and 3.17, respectively. Narrowband sources are again present at 1397 and
1400 MHz for H-pol, and 1395 and 1400 MHz for V-pol, as well as the IF interference
at 1390 MHz for both polarizations (but stronger in H-pol.) The hydrogen line is again
visible with an increased brightness near noon UTC (5 am local time). Variations
113
April 28th, Horizontal, APB off
198
ADD
LISR+5K
PALS+164 K
Brightness (K)
194
190
186
182
178
174
0
2
4
6
8
UTC (hours)
10
12
14
th
April 28 , Vertical, APB off
202
ADD
LISR−3.5K
PALS+168 K
Brightness (K)
198
194
190
186
182
178
0
2
4
6
8
UTC (hours)
10
12
14
Figure 3.15: Calibrated brightness temperatures vs. time for ADD, LISR and PALS
on April 28th
114
th
April 28 , H pol, APB off
250
2
240
4
230
220
UTC (hours)
6
210
200
8
190
180
10
170
160
12
150
14
1385
1390
1395
1400
1405
1410
1415
Frequency (MHz)
1420
1425
1430
140
Figure 3.16: Calibrated H-pol brightness temperature vs. RF frequency, April 28th
in brightness spectral properties in horizontal polarization are not obvious in Figure
3.16 due to the low time resolution shown.
3.3.5
Overnight observations on May 10th-May 11th
After April 28th, the campaign was halted and restarted on May 8th. As discussed
in Section 3.2.3, LISR encountered missing data issues during the overnight datasets
of May 8th, 9th, and 11th; these measurements are not discussed further. Although
data was acquired during the overnight of May 10th-11th, frequent saturation was
again encountered; the largest portions of the final dataset available consist of observations from 5:18-5:50 and 12-13:15 UTC on May 11th. Figure 3.18 compares LISR
total channel brightnesses versus time with those obtained from PALS for these time
periods. Properties of LISR observed brightnesses are qualitatively similar to those
measured by PALS during these periods, although the different duty cycles of the
115
th
April 28 , V pol, APB off
250
2
240
4
230
220
UTC (hours)
6
210
200
8
190
180
10
170
160
12
150
14
1385
1390
1395
1400
1405
1410
1415
Frequency (MHz)
1420
1425
140
1430
Figure 3.17: Calibrated V-pol brightness temperature vs. RF frequency, April 28th
th
th
May 11 , Horizontal, APB off
May 11 , Vertical, APB off
196
202
LISR
PALS+175 K
Brightness (K)
194
192
198
190
196
188
194
186
192
184
190
182
5.2
5.4
5.6
5.8
LISR
PALS+179 K
200
188
5.2
6
5.4
5.6
5.8
6
202
LISR
PALS+175 K
Brightness (K)
195
LISR
PALS+179 K
200
198
196
190
194
192
185
190
12
12.5
13
UTC (hours)
13.5
188
12
12.5
13
UTC (hours)
13.5
Figure 3.18: Calibrated brightness temperatures vs. time for LISR and PALS on
May 11th
116
th
May 11 , H pol, APB off
240
UTC (hours)
5.4
220
5.5
200
5.6
180
5.7
160
5.8
140
1390
1400
1410
1420
1430
240
UTC (hours)
12.2
220
12.4
12.6
200
12.8
180
13
160
13.2
140
1390
1400
1410
Frequency (MHz)
1420
1430
Figure 3.19: Calibrated H-pol brightness temperature vs. RF frequency, May 11th
two instruments result in high brightness features that are similar but not completely
correlated.
Spectrograms for H- and V-polarized data are shown in Figures 3.19 and 3.20,
respectively, for the times periods 5:18-5:50 and 12-13:15 UTC. Since the data plotted
for this day is shorter than the other days, only a 64 sample integration (≈ 11.5 msec
antenna observation time) was used. Overall brightness properties versus frequency
remain similar to those of previous days, although in this case, the 1395 MHz V-pol
and 1397 MHz H-pol sources are not as apparent. A new narrowband interferer at
1398 MHz also is present for V-pol.
117
May 11th, V pol, APB off
240
UTC (hours)
5.4
220
5.5
200
5.6
180
5.7
160
5.8
140
1390
1400
1410
1420
1430
240
UTC (hours)
12.2
220
12.4
12.6
200
12.8
180
13
160
13.2
140
1390
1400
1410
Frequency (MHz)
1420
1430
Figure 3.20: Calibrated V-pol brightness temperature vs. RF frequency, May 11th
118
3.4
3.4.1
RFI Detection and Mitigation
Properties of observed RFI
The previous Section showed the presence of several persistent narrowband RFI
sources (for example at 1395, 1397, and 1400 MHz) throughout the campaign, as
well as an IF source that appears as 1390 MHz and apparent natural hydrogen line
emissions at 1420.4 MHz. LISR’s high spectral resolution allows these sources to
be discarded easily, simply by computing total channel brightnesses over regions of
the spectrum that do not contain these sources. The discussions of the previous
Section showed that the contributions of these sources to total channel brightnesses
was typically on the order of a few tenths of a Kelvin. These contributions would
also be reduced for instruments (such as PALS) that integrate over frequency before
calibration due to the fact that these sources encounter reduced gain in the instrument
passband. The emitters of the RFI at 1395 to 1400 MHz are unknown, but, as
described in Chapter 2, they were encountered in Canton campaign as well and some
of these emissions may be associated with radiation from monitors, computers, or
other common electronic devices.
Larger time-localized changes in brightness were also observed throughout the
campaign. An example of these time-localized changes is highlighted in Figure 3.21,
which plots PALS, ADD, and LISR total channel brightnesses versus time from the
April 27th-28th overnight observation (as in Figure 3.15) for the shorter time period
4:00 to 6:00 UTC. Periodic interference is observed by all three sensors, with the RFI
level and other properties closely correlated among sensors.
An examination of LISR data at the highest time resolution of 184.32 µsec was
performed during the periods of observed RFI. Several examples of “pulsed” time
119
April 28th, Horizontal, APB off
198
ADD
LISR+5K
PALS+164 K
Brightness (K)
194
190
186
182
178
174
0
10
20
30
40
50
60
70
Minutes past 4:00 UTC
80
90
100
110
120
th
April 28 , Vertical, APB off
202
ADD
LISR−3.5K
PALS+168 K
Brightness (K)
198
194
190
186
182
178
0
10
20
30
40
50
60
70
Minutes past 4:00 UTC
80
90
100
110
120
Figure 3.21: Calibrated brightness temperatures vs. time for ADD, LISR and PALS
sensors on April 28th between 4 and 6 am UTC time
120
RFI were observed, with the total channel brightness during a pulsed 184.32 µsec
observation period substantially exceeding that of surrounding periods. RFI spectral
properties during a 184.32 µsec observation period are somewhat difficult to discern
due to the relatively short integration time. In many cases, there are no obvious
spectral signatures observed for a specific pulse, but rather simply an increase in
overall power level throughout the band, suggesting either a very wide band interference source, corruption of the antenna measurement by the reference load or noise
diode internal calibration loads, or possible saturation of the receiver due to in or
out of band interference. If pulse-like interference without an obvious spectral signature is removed, the remaining pulsed signatures can be examined for the spectral
information they contain.
An example of such an examination is provided in Figures 3.22 and 3.23, which
plot V- and H-pol spectra, respectively, acquired at 184.32 µsec time resolution for
pulse-like events during the time period 1:54-4:27 UTC on April 28th. In these
Figures, V-pol “pulse” spectra having their highest mean brightness located between
1402.5-1405.3 MHz have been plotted, while the H-pol image retains pulses with
the highest mean brightness in the band 1399.5-1402.4 MHz. These criteria were
found to retain a large set of pulses having apparent spectral signatures. The color
scale utilized is logarithmic in order to capture high brightness events. Results show
evidence of broadband interference in the band 1400-1410 MHz for V-pol, and in the
band 1400-1404 MHz in H-pol. The latter case also indicates a correlation between
high brightnesses in the frequency range 1400-1404 MHz and those in the range
1412-1420 MHz. The sources shown are clearly of a “wideband” nature given than
bandwidths substantially larger than 1 MHz are occupied.
121
th
Image of Detected Pulses in V pol, April 28
600
50
500
100
Trigger Number
400
150
300
200
250
300
200
350
1400
1405
1410
Frequency
1415
1420
Figure 3.22: Image of pulsed interferers for V-pol, April 28th
122
Image of Detected Pulses in H pol, April 28
th
600
50
500
100
Trigger Number
400
150
200
300
250
300
200
350
400
1400
1405
1410
Frequency
1415
1420
Figure 3.23: Image of pulsed interferers for H-pol, April 28th
123
3.4.2
Time domain RFI mitigation
Although it is difficult to conclusively identify the RFI sources observed by PALS,
given the broadband spectral properties discussed in the last section, it is nevertheless
clear that the observed RFI occurs on short time scales, likely shorter than the 184.32
µsec integration period of LISR. In these situations, a simple time-domain blanking
strategy can be highly effective at mitigating RFI.
A simple algorithm was developed based on blanking data at 184.32 µsec time resolution before the 64 point integration of the calibration process described in Section
3.2.6. The algorithm executes as follows:
1. Compute the total channel brightness for all antenna triggers in a file; sort these
brightnesses and find the mean and standard deviation of the lower 90%.
2. Declare antenna observations with total channel brightnesses more than 5 standard deviations above the mean to be pulses, and set their contributions to zero
in the 64 point integration.
3. Integrate the data over 64 antenna observations; scale result to account for
presence of blanked pulses
A similar second-stage blanking process was applied at 11.5 msec time resolution
when integrating brightnesses over an entire file, as shown in the majority of the plots
illustrated in this report.
Figure 3.24 displays sample LISR total channel brightnesses before and after this
mitigation procedure for horizontally and vertically polarized data during the time
period 08:30 to 12:00 UTC on April 28th. Although a few examples are observed
where apparent low-level interference remains after the mitigation, results in general
124
April 28
th
180
LISR
Mitigated
B
H−pol T (K)
179
178
177
176
175
30
50
70
90
110 130 150 170
Minutes after 8:00 UTC
190
210
230
50
70
90
110 130 150 170
Minutes after 8:00 UTC
190
210
230
193
191
B
V−pol T (K)
192
190
189
188
187
30
Figure 3.24: H-pol and V-pol total channel calibrated brightnesses before and after
time domain RFI mitigation, April 28th
show this simple mitigation algorithm to be effective in removing pulsed RFI from
the dataset, due to the pulsed nature of the RFI encountered.
3.5
Summary and Remarks
This chapter has described LISR observations with JPL’s PALS in a ground-based
measurement campaign conducted April 24th-May 11th, 2006. A description of modifications made to LISR for this campaign compared to previous deployments and its
125
observation modes during the campaign were provided. A review of LISR measurements showed that LISR results were reasonably consistent with those obtained from
PALS in terms of observed RFI properties.
LISR’s high spectral resolution indicated the presence of several narrowband interference sources throughout the campaign, primarily in the region 1395-1400 MHz,
although time-varying contributions from hydrogen line emissions were also observed.
The dominant RFI sources were of a pulsed type; examination of these pulses in the
time domain showed them to be generally of a wideband nature, with some evidence
of frequency localization in the region 1400-1410 MHz as well as 1412-1420 MHz in
some cases. The source of these emissions is not known but simple pulse and crossfrequency detection strategies were found to be highly effective against the observed
RFI.
Other questions regarding the accuracy of the results include the observed significant variations in brightness spectral properties, as well as system calibration given
the noise diode transient issue in the PALS system. Even given these unresolved problems, the LISR dataset represents a highly useful tool for examining RFI properties,
particularly in the spectral domain, for this campaign.
126
CHAPTER 4
AIRBORNE C-BAND RFI MEASUREMENTS WITH
PSR/CXI AND CISR FROM THE WB-57 AIRCRAFT
In this chapter observations of C-band RFI made from NASA’s WB-57 highaltitude aircraft using the PSR/CXI system of NOAA/ETL and the CISR digital
backend of OSU/ESL is documented. Figure 4.1 shows a picture of the WB-57
aircraft used in this campaign. The C-band Agile Digital Detector (ADD) system of
the University of Michigan [24] was also included in the campaign. ADD system did
not record RF frequency channel information which led to a difficulty in analyzing the
data from this instrument; since a comparison of ADD data with that of the CISR
data was desired, an automatic algorithm is developed to sort out this problem.
Resulting comparisons using this algorithm will also be described in this chapter.
The observations described were performed during a test flight on Aug. 25th,
2005 beginning at Ellington Field, Houston, TX and including overflights of Dallas
and San Antonio as well as a flight segment over the Gulf of Mexico. This campaign
will be denoted as “WB-57 campaign” in the rest of this work. Only the vertically
polarized C-band measurements are discussed here. The PSR/CXI instrument also
recorded horizontally polarized and polarimetric C-band channels, as well as X-band
brightnesses in multiple frequency sub-bands, but these were not sampled by CISR
during the flight.
127
Figure 4.1: WB-57 high-altitude aircraft
Note that a similar earlier airborne experiment that involved CISR and the
PSR/CXI frontend was performed on October 8, 2004 over Wallops Flight Facility
(WFF) and was reported in [16]. Unfortunately, in that campaign, a reasonable calibration of CISR could not be obtained for most of the flight. Since WB-57 campaign
is similar to WFF campaign in terms of hardware used and processing techniques
applied for RFI detection, and results from the WB-57 campaign allowed a much
better comparison between different sensors, WFF campaign will not be described
further in this thesis.
4.1
4.1.1
Instrumentation
PSR/CXI
The PSR/CX instrument of NOAA/ETL has been deployed in several previous
airborne remote sensing campaigns [18], and provides well calibrated brightness measurements along with a variety of possible scanning patterns during flight operations.
128
The majority of the data to be presented were obtained from conically scanned observations (i.e. the antenna is rotated in azimuth at a fixed speed) at a fixed incidence
angle of 55 degrees from nadir (when in level flight). One portion of the dataset
near the end of the flight includes fixed scan angle sky observations during steep rolls
of the aircraft for calibration verification purposes. The antenna 3 dB beamwidth
(two-sided) is approximately 10 degrees for all observations.
The PSR/CX instrument includes four C-band sub-channels, with respective 3
dB bandpass frequency ranges of 5.8-6.2, 6.3-6.7, 6.75-7.1, and 7.15-7.5 GHz. These
sub-channels can provide some measure of RFI mitigation, but remain large analog channels (∼400 MHz) compared to the bandwidth of likely RFI sources. The
PSR/CXI instrument utilized in this flight is a modification of PSR/CX that includes a downconverter module so that tuned observations can be made throughout
C-band in a narrower bandwidth than the 400 MHz of the main channels. Figure 4.2 is a simplified schematic of the PSR/CXI, and shows that following initial
filtering and front-end amplification, a portion of the antenna power is coupled to
the downconverter section. The local oscillator (LO) utilized is capable of tuning
from 5.4 to 7.5 GHz, uses a 125 MHz IF center frequency, and provides > 20 dB
rejection of the upper RF side band. The IF signal is filtered to both 10 and 100
MHz bandwidths, and both of these bandwidths are passed through a logarithmic
amplifier/detector, power integrated, and recorded. The 100 MHz bandwidth signal
is also passed through a linear amplifier; a portion of this coherent signal is coupled
to the CISR instrument in the cabin rack of the P-3, while the remainder is detected,
power integrated, and recorded by PSR/CXI (PSR/CXI post-amplifier section not
shown in Figure 4.2). Tuned channel observations recorded by the PSR are reported
elsewhere [54], and are not considered further here.
129
To PSR/C direct
detection channels
(5.8−6.2, 6.3−6.7,
6.75−7.1, 7.15−7.50 GHz)
Antenna
5.75−
7.55 GHz
LNA
125
MHz IF
To PSR
Computer
120−
130 MHz
5.75−
7.55 GHz
75−
175 MHz
5.4−7.5 GHz
Trigger
pulse
To digital
receiver
Figure 4.2: Simplified schematic of PSR/CXI
For the CISR data to be reported, the LO was tuned so that 100 MHz channel observations were made at 5.5-5.6, 5.6-5.7, ..., 7.6-7.7 GHz; this is a set of 22
channels. The LO was swept continuously throughout the flight, with each channel
being observed for approximately 37 msec; a complete sweep of channels required
approximately 814 msec. The PSR/CXI LO is a YiG-tuned device, and therefore is
subject to hysteresis effects as well as temperature sensitivity. For this reason, the
accuracy of the RFI frequencies should be taken as ≈ ±2 MHz throughout this study.
Although this is acceptable accuracy for radiometric studies, possible inclusion of an
LO with improved tuning accuracy and stability deserves consideration for future
flights of PSR/CXI.
4.1.2
CISR
A simplified schematic of the CISR instrument is provided in Figure 4.3. It can be
seen that CISR is similar in design to LISR, but with some major modifications. The
CISR digital receiver backend measures the incoming 100 MHz bandwidth through
130
the use of two 200 MSPS A/D converters, each of which samples the band 125175 MHz. A “channel-selection” is therefore required to upconvert the 75-125 MHz
portion of the PSR/CXI IF signal to 125-175 MHz. Following A/D conversion, the
operation of the system is basically same with LISR; CISR instrument can perform an
APB operation to suppress temporally localized interference, as well as a 1024 point
FFT operation, followed by optional power integration or max-hold computations.
“Integration, APB off” and “Capture” modes were the modes utilized in WB-57
campaign.
In “Integration, APB off’ mode, an 1024 point FFT operation is performed, followed by power computation and an integration over a 1.3 msec time period (same
as Canton experiment in Chapter 2). The output in this mode is then a spectrum
of the power in 1024 ∼100 kHz bandwidth sub-channels within the tuned 100 MHz
spectrometer channel. The “capture” mode again refers to the direct recording of the
sampled 100 MHz channel at 10 nsec temporal resolution with no further processing.
CISR’s asynchronous pulse blanking (APB) algorithm was not used in this campaign due to the relative infrequency of pulsed interferers at C-band, as well as a
desire to optimize calibration of the “Integration, APB off” data. Results of Chapter
2 have clearly demonstrated the effectiveness of the APB algorithm at mitigating
pulsed source interference while maintaining accurate brightness temperature measurements.
4.1.3
Interface between PSR/CXI and CISR
Because it is the PSR/CXI data acquisition computer that controls oscillator tuning in the downconverter, the CISR and PSR/CXI computers must be interfaced. To
make this interface as simple as possible, a simple 1-bit “trigger” signal was used. This
131
Trigger in
IF/Trigger
Computer
from PSR
Digital data
PCI−
DIO32HS
Processor control
Atten control
Channel selection
Digital Receiver
ADC
75−125 MHz
125−175 MHz
Clock
250 MHz
DIF/
APB
FFT
SDP
Capture
6 dB
ADC
125−175 MHz
Figure 4.3: Simplified schematic of CISR
TTL-level pulse is sent from the PSR/CXI control computer to the CISR computer
whenever a tuning operation has been completed and measurements should begin.
In addition, both computers include highly accurate on-board clocks, synchronized
through the IRIG-B standard, so that knowledge of the oscillator frequency for a
particular CISR measurement is obtainable by matching up recorded trigger pulse
times in post-processing of a joint data set. In the data considered here, unambiguous matchups of the PSR/CXI and CISR measurements were possible utilizing this
process. Offsets in the two “timestamps” were within 3 msec for the majority of the
recorded data.
One issue in this time synchronization involves observations of the internal noise
diodes of the PSR/CXI system. These noise diodes are useful for verifying or improving system calibration stability between measurements of the external hot and cold
targets embedded in the PSR/CXI scanhead. The simple 1-bit trigger interface however does not allow specification of the noise diode switch state during a measurement.
132
To address this issue, a plan was developed wherein the noise diode measurements
were to be performed within a specified time interval following the trigger signal.
However, the time accuracy of the PSR/CXI instrument states was insufficient to
ensure an accurate noise diode measurement by CISR at each opportunity.
4.1.4
CISR modifications for the WB-57 aircraft
A few modifications to the CISR hardware were performed for this campaign
to improve survivability in the high altitude environment. The CISR hardware was
located in a unpressurized portion of the WB-57 aircraft, and therefore exposed to the
ambient (though inside the aircraft) air temperature and pressure at altitude. Figure
4.4 is a photograph of the CISR enclosure when installed within the “foreward racks”
in the WB-57 aircraft.
Atmospheric pressure issues
The low atmospheric pressure of approximately 0.1 atm at altitude (62500 ft)
complicates heat dissipation issues within the CISR enclosure, and also requires use
of modified hard drive systems for recording observed data. Though solid state hard
drives are desirable in such cases, the high data rate of CISR makes use of faster
magnetic drives preferable. In this campaign, the CISR operating system and source
executable were placed on a 4 GB solid state drive, while data was stored on an
80 GB sealed magnetic hard drive. In an earlier deployment of CISR on-board the
WB-57 aircraft (April 2005), the latter drives failed due to improper sealing by the
vendor. However these problems were corrected by the vendor prior to August 2005,
and no further issues were encountered.
133
Figure 4.4: Photograph of the CISR enclosure installed on the WB-57 aircraft
Temperature control
CISR was further modified to include simple temperature control systems. A set
of heaters capable of providing 175 W of heating were included, under the control
of a thermostat within the CISR enclosure which turned the heaters on when the
temperature of the thermostat declined below 10 C. A second thermostat was also
included at an alternate location within the enclosure to override the heater on state
if temperatures greater than 50 C were encountered at a second thermostat. Thermostats were also placed within the enclosure to turn CISR electronics off when the
internal temperature decreased below 0 C or above 50 C. These thermostats were
simple on/off bimetal thermostats, and their status was not monitored or recorded.
The CISR enclosure included substantial thermal insulation due to the low ambient
134
air temperature at altitude. Given this insulation and the existing heating provided
by the CISR electronics, it is not believed that the heating system was utilized during
flight.
The reduced amount of convective cooling available at low atmospheric pressure
increases the possibility of localized “hot-spots” within the CISR enclosure, even
though many portions of the enclosure may be cold. The only enclosure temperature
information recorded during the flight was obtained from a temperature, humidity,
and pressure monitoring card within the CISR PC-104 computer. Data from this
sensor indicated local temperatures approaching but not exceeding 62 C. While further consideration of CISR thermal transfer could possibly reduce the likelihood of
these high internal temperatures, rated operational temperatures for CISR electronic
components exceed 62 C, and no apparent problems were observed.
Condensation
A final modification was made in an attempt to reduce the amount of condensation
within the CISR enclosure as the aircraft returns from altitude. Though the enclosure
was not hermetically sealed, and in fact becomes an explosive hazard if pressurized
relative to the ambient environment, attempts were made to keep the enclosure reasonably air tight with the exception of a single air exchange opening. This opening
was routed through a tube desiccator, so that air taken into the enclosure upon descent would contain a reduced moisture content. The PSR and University of Michigan
systems utilized a similar approach to reduce condensation problems. Though quantitative information on the performance of this system was not recorded, no damage
due to condensation within the CISR enclosure was encountered.
135
ADD Subband:
Subband #1
Subband #2
Subband #3
Subband #4
Subband #5
Subband #6
Subband #7
Subband #8
Fullband
3dB
122.00
127.25
132.50
137.75
143.00
148.25
153.50
158.75
122.00
band edges
- 127.25 MHz
- 132.50 MHz
- 137.75 MHz
- 143.00 MHz
- 148.25 MHz
- 153.50 MHz
- 158.75 MHz
- 164.00 MHz
- 164.00 MHz
Table 4.1: ADD subchannel frequencies within the 75-175 MHz IF band
4.1.5
ADD system
Since comparisons with the ADD system will also be provided, a brief description
of ADD as configured for this campaign is given here. ADD backend system, like CISR
system, observed IF signals provided by PSR. The ADD system further filtered the
IF data into 8 5.25 MHz ADD subchannels, with results from all ADD subchannels
recorded simultaneously and continuously at 1.2 msec time resolution. The 3 dB
points of the ADD subchannel IF filters are shown in Table 4.1. The ADD also has
a fullband channel with band edges at 122 and 164 MHz.
Since the PSR computer controlled IF tuning throughout the experiment, information on the current channel state was not recorded by the ADD system with the
hope that channel identification for the ADD data could be obtained by reconciling recorded time information between the ADD and PSR systems. However, time
synchronization between the instruments was found insufficiently accurate to achieve
this goal. Therefore, an alternative method was required for determining the PSR
channel corresponding to a given ADD observation.
136
Researchers at University of Michigan could only identify channel information of
a limited portion of the flight by using manual techniques. Since it was our goal to
compare the two sensors, an automatic method was developed at OSU to classify
ADD channel information. This method will not be described here since a thorough
description will likely disrupt the flow of this study. Once the ADD channels have
been identified, it is possible to calibrate the data. An initial comparison of calibrated
ADD brightnesses with those recorded by CISR is provided in Section 4.4. The ADD
dataset also allows the signal’s kurtosis to be computed and examined; however, this
data is not presented in this study since exact RFI detection procedure using kurtosis
statistic was not made available by University of Michigan.
4.1.6
Measurement process
PSR observation and spot Properties
As stated previously, the PSR/CXI LO was continuously tuned through the 22
bands of interest throughout flight operations, requiring approximately 814 msec to
complete a sweep. When in conical scan mode, the PSR/CXI antenna rotation period
was typically 40 seconds, so that 49.1 sweeps were performed per antenna “scan”
(i.e. rotation). PSR conical scan mode observations were performed in a total of 137
antenna rotations during the flight.
At the typical flight altitude of 62500 ft, and using the nominal PSR antenna 3
dB beamwidth (two-sided) of 10 degrees, the 3 dB footprint observed by the antenna
is approximately 10.3 km in diameter along track. At the typical flight speed of
205 m/sec, the 40 sec scan rate results in an along track sampling distance of 8.2
km, slightly less than the 3 dB footprint size. The cross-track 3 dB footprint is
approximately 4.75 km in diameter; this dimension represents approximately 1/36
137
of the circumference of the cross-track scan. Therefore each independent cross-track
footprint is observed for more than a second during the scanning process, allowing
more than a full sweep of the 22 tuned channels within each footprint.
As the PSR antenna rotates these 4.75 km by 10.3 km spots are swept in a
circle along the Earth surface of approximate diameter 64.3 km (between 3 dB spot
boundaries.) Thus, data from one PSR rotation represents observations comparable
in spatial dimensions to a typical C-band satellite footprint, although the total area
observed is only approximately half of the complete 64.3 km diameter circle. Based
on these properties, it appear reasonable to assume that RFI effects in single pixel
satellite observations are roughly comparable to those observed in a complete PSR
antenna rotation.
CISR operational process
CISR was configured to observe in “average, APB off” mode for 16 consecutive
sweeps of the tuned channels (around 13 seconds), followed by “capture” mode observations for 2 consecutive sweeps (around 1.63 seconds). CISR then paused for
approximately 4.6 seconds to write the recorded data to its internal hard drives. This
results in an approximate 19 second periodic pattern, completed approximately twice
per PSR antenna rotation. In “Integration, APB off” mode, a base temporal resolution of 1.3 msec was utilized, with 12 1024 point spectra obtained in 15.75 msec
following each PSR/CXI trigger. In the capture mode, a 128K capture (1.3 msec) was
recorded for each PSR/CXI trigger. The resulting CISR data rate is approximately
9.2 GB/hr; although this is certainly unacceptable for end-user radiometer applications, the goal of recording as much information on the observed RFI as possible
results in the high data rate for this study.
138
Calibration
The PSR main channel data to be reported was calibrated using the external
hot and cold loads of the PSR scan head; this procedure is expected to produce
brightnesses typically accurate to within 1-2 K for the obtained 20 msec observations.
Cal load observations were performed following 8 rotations of the PSR antenna for
the majority of the flight. Examination of the PSR main channel data following
calibration showed the expected 1-2 K standard deviations for PSR main channels 2
through 4, but an increased standard deviation of around 4 K in PSR main channel
1. This appears to have resulted due to a decreased system gain in main channel 1.
However it is possible to reduce these standard deviations further by performing an
additional integration over scan angle. This is because the standard PSR observation
occurs every 37 msec, which represents only 0.33 degrees of scan angle, compared to
the 3 dB spot of 10 degrees. PSR scan angle images to be illustrated have therefore
undergone an additional smoothing by taking an average over the 9 surrounding
pixels; this reduces the standard deviation by a factor of 3. The low gain of channel
one also reduces the accuracy of its absolute calibration relative to the other channels.
This offset among channels plays a role in decreasing the sensitivity that can be
achieved by the PSR four channel RFI mitigation algorithm.
Calibration of the tuned PSR/CXI and CISR channels is also available through
this process, and the slower scan rate and corresponding increased cal load observation
time result in greatly improved CISR calibration compared to [16]. However, as
with PSR main channel 1, calibration of CISR data below 6.2 GHz and above 7.5
GHz remain problematic due to the low apparent gain of the PSR front end in these
regions. For this reason, calibrated data will be illustrated for CISR channels between
139
6.2 GHz-7.5 GHz. No additional scan angle integration of these data are performed
due to the tuning process. Note that some evidence of corruption of the PSR main
channel calibration procedure due to strong RFI during cal-target observations is
observed in the campaign, although the majority of the cal load observations appear
to be RFI-free.
4.2
Experiment Conditions
Table one provides information on the flight plan. Figure 4.5 illustrates the geographical region observed; note major cities including Houston, Dallas-Fort Worth
(DFW), and San Antonio, are marked with circles. Figure 4.6 is a plot of the WB-57
altitude versus time. As can be determined from Table 1, the flight plan included
takeoff from Ellington Field (Houston area) at 17:14 along a West-Northwesterly
heading, followed by a turn to the North near Krebsville (a small town) at 17:32.
Ascent to a flight altitude near 62000 ft was completed by 17:51 as the DFW area
was approached. The flight path then included a turn to the South over DFW around
18:00, followed by observations over more rural Texas locations until an overflight of
San Antonio around 18:30. A turn to the Southeast over San Antonio then led to an
overflight of the Gulf of Mexico beginning at 18:49. The flight plan then included a
short circular flight at roll angle 30 degrees beginning at 18:54, followed by steeper
rolls to allow sky observations beginning at 18:59. These rolls were completed by
19:06, and all observations stopped at 19:11. The aircraft returned to Ellington field
at 19:42.
PSR and CISR were both powered on before takeoff, and both acquired data
until 19:11. However the analysis reported here focuses on conically scanned data
obtained after flight altitude was reached (17:51) and before the circle flights were
140
Flight path between 17:18:11 and 19:09:51
o
34 N
18:0
−
0
Fort Worth
Dallas
o
32 N
−
−
−
Austin
30 N
−
0
18:3
Houston
Krebsville
−
o
−1
7
:3
0
−
San Antonio
−
o
−
28 N
:0
0
−
19
Latitude (°)
Waco
o
26 N o
102 W
o
100 W
o
98 W
Longitude (°)
o
96 W
o
94 W
Figure 4.5: Flight path, including nearby Texas cities (circles)
141
Altitude of Plane vs. Time
65
60
55
Altitude (kfeet)
50
45
40
35
30
25
20
15
17:29
17:40
17:51
18:02
18:14
18:25
Time
18:36
18:47
Figure 4.6: Altitude of WB-57 aircraft versus time
142
18:58
19:09
Time
(UTC) Event
17:14 Takeoff from Ellington Field
17:17 Start PSR conical scans
17:23 Altitude 25 kft, ADD on
17:32 Turn Northward over Krebsville
17:51 Altitude 62 kft
17:59 Over DFW
18:30 Over San Antonio
18:49 Over Gulf of Mexico
18:52 Begin descent from flight altitude
18:54 Begin circle flight
18:59 Stop PSR conical scans
19:06 Finish cal rolls
19:11 PSR off
19:42 Land at Ellington Field
Comments
Depart Houston vicinity
Heading 280 degrees to 354 degrees
Approaching DFW
Turn toward San Antonio
Turn toward coast
roll 30 deg
Short rolls up to 60 deg
Table 4.2: Time history of flight on August 25th, 2005
begun (18:54). The ADD system was powered on after reaching altitude 25,000 ft
(17:23) and observed until all systems were powered off at 19:11. The dataset contains
observations in a variety of RFI environments, including urban, rural, and water
scenes. Images of PSR/CXI conically scanned data to be shown will be presented in
terms of time and scan angle; these should not be taken as geographic images given
possible variations in the aircraft heading, pitch, roll, and altitude.
RFI source information from the JSC database
A review of the NPOESS RFI source database obtained from the Joint Spectrum
Center (JSC) was performed to assist in preliminary interpretation of the datasets
to be described in the next sections. The JSC database utilized contains data not
updated since 1999, and therefore is subject to significant errors as well as source
omissions. Within the frequency ranges of interest, the JSC database is described
143
as including only 5.5-5.9 GHz and 6.2-7 GHz, and the database also omits classified
source information. Nevertheless, it is this dataset that has been utilized by the IPO
in performing RFI simulation studies, so that information on the accuracy of this
particular database is relevant. The dataset also provides some degree of information
on the spatial distribution of RFI sources.
Figure 4.7 plots RFI source locations in the frequency range of interest taken
from apparently valid records in the JSC database. The flight path is indicated by
the thick red line, and the Houston, DFW, and San Antonio areas are marked by
the large circles. The strong correlation of the source density with urban locations
is clear. The portion of the flight encountering the smallest source density is found
between the DFW and San Antonio areas. Note that some off-coast sources are also
included in the database.
A histogram of center frequencies for the sources included in Figure 4.7 is provided
in Figure 4.8. Overall the distribution is fairly uniform from 6.2-7 GHz, with the
exception of the region 6.4-6.6 GHz, which contains a smaller number of sources.
Note again that sources centered from 5.9-6.2 GHz are described as “not included”
in the database, so that the histogram should not be taken as accurate in this region.
Database information also reveals that many of the allocations below 5.9 GHz are
associated with radar or other pulsed sources, while those above 5.9 GHz are primarily
communication systems.
4.3
4.3.1
Comparisons with PSR
PSR scan images
The WB-57 campaign represents a very dynamic RFI environment, with differing
RFI sources observed as the tuned channel is swept through C-band and as the
144
36oN
34oN
32oN
30oN
o
28 N
102oW
100oW
98oW
96oW
94oW
Figure 4.7: Locations of C-band RFI sources within the JSC source database. Flight
path is indicated by the thick red line.
145
450
400
Number of Records
350
300
250
200
150
100
50
0
5400
5600
5800
6000
6200
6400
6600
Center Frequency (MHz)
6800
7000
Figure 4.8: Distribution of source center frequencies for the JSC database records
plotted in Figure 4.7. Note that records centered between 5.9-6.2 GHz are described
as “not included” in the database.
146
aircraft location changes. As seen in Figure 4.5, the flight path included major
urban centers, rural areas, and the Gulf of Mexico, all of which involve differing RFI
properties. Figure 4.9 plots PSR scan angle images for the foreward part of the PSR
conical scan, in all four PSR “main” channels. Obvious RFI is common in these
images, with brightnesses up to 10700 K, 2994 K, 2442 K, and 2660 K observed
in channels one through four, respectively. Of these channels, channel 1 generally
shows the largest degree of RFI corruption, and channel 4 the least, although obvious
RFI remains present in channel 4. Increase RFI associated with the Houston (prior
to 17:25), DFW (around 18:00), and San Antonio (18:30) portions of the flight is
apparent. The decreased brightness associated with observations over the Gulf of
Mexico is also obvious in the later portions of the images. Figure 4.10 plots the
average of the images in Figure 4.9 over scan angle, and shows behaviors consistent
with those discussed for Figure 4.9.
As described in [16, 18] and also in Chapter 1, it is possible to apply the PSR
four sub-channel RFI mitigation algorithm to these images in order to remove RFI
corruption. Use of the algorithm in this case eliminates the vast majority of the obvious RFI. However it is difficult to determine the amount of low-level RFI remaining
following this process. A particular problem results when the algorithm determines
that three or more channels contain RFI, so that the validity of the remaining channel is difficult to determine. Tables 4.3 and 4.4 provide summary statistics from the
PSR RFI mitigation algorithm for the time periods 18:08-18:23 (more rural locations)
and 17:54-18:08 (near DFW). Statistics are provided in terms of the percent of 21600
pixels determined to have a specified “interference level”. A “clean” classification
results for a channel only if it is determined to be RFI free and RFI was detected in
147
Figure 4.9: Calibrated brightnesses from PSR conical scans of the entire flight, in all
four PSR main channels
148
PSR Average Brightness vs. Time
1000
Ch1
Ch2
Ch3
Ch4
900
800
Brightness (K)
700
600
500
400
300
200
20
30
40
50
60
70
80
Minutes past 17:00 UTC
90
100
110
120
Figure 4.10: Average of Figure 4.9 over scan angle, versus time
less than 3 of the remaining channels. The classifications “1 channel” and “2 channel” for a particular channel indicate that that channel was determined to contain
RFI when only one or two of the four PSR channels were deemed corrupted. Cases
with “3 channels” however result in a classification for all PSR channels, due to the
limitations of the algorithm with regard to assessing the remaining channel in this
case.
While the percentages shown are influenced by PSR calibration issues and by
the particular algorithmic parameters utilized, overall the results clearly indicate a
significant RFI problem both in the DFW (53.1% classified as “3 channel”) and rural
149
PSR freq (GHz)
5.8-6.2 6.3-6.7 6.75-7.1
Clean %
39.6
41.8
49.1
1 Channel %
13.0
11.4
3.0
2 Channel %
19.3
18.8
19.7
3 Channel %
27.0
27.0
27.0
Algorithm failed %
1.3
1.3
1.3
7.15-7.5
65.2
0.6
5.9
27.0
1.3
Table 4.3: Statistics from PSR four sub-band interference suppression algorithm:
21600 pixels over rural Texas (18:08-18:23 UTC)
PSR freq (GHz)
5.8-6.2 6.3-6.7 6.75-7.1
Clean %
12.5
30.8
30.6
1 Channel %
10.7
3.6
1.5
2 Channel %
22.6
11.5
13.8
3 Channel %
53.1
53.1
53.1
Algorithm failed %
1.1
1.1
1.1
7.15-7.5
44.1
0.1
1.7
53.1
1.1
Table 4.4: Statistics from PSR four sub-band interference suppression algorithm:
21600 pixels near DFW (17:54-18:08 UTC)
150
(27.0% percent “3 channel’) observations. Other statistics show a decreased, but nonnegligible, presence of RFI in the highest frequency PSR channel, consistent with the
scan images of Figure 4.9.
Examination of CISR data can be used to help to reveal further the properties of
the RFI encountered, its distribution in frequency, and its temporal properties. Since
naturally emitted thermal noise is expected to vary slowly with frequency, and since
CISR has a very high spectral resolution, cross frequency mitigation using CISR data
should be very effective against RFI that is localized in frequency.
A cross-frequency mitigation technique slightly modified from the ones used in the
L-band campaigns was applied to calibrated data. For a particular observation, we
have brightness temperatures in 1024 frequency bins of approximately 97 kHz each
as a result of the FFT operation. The algorithm operates on a single set of 1024
frequency bins averaged to 15.75 msec time resolution, as follows:
1. First, an acceptable brightness temperature range is set depending on the scene
observed. Frequency bins with brightnesses outside the acceptable range are
marked as corrupted. For land observations, the acceptable range was set as
200 K to 400 K.
2. The standard deviation and mean (both over frequency) of brightnesses are
found for the lowest 85% (in terms of brightness) of the remaining frequency bins
(to avoid bias of the mean and standard deviation by RFI). Another threshold
test is then applied: frequency bins that are more than 4 standard deviations
from the mean over frequency are marked as corrupted. Neighboring frequency
bins within 4 frequency bins of the corrupted bins are marked as well.
151
3. Brightness temperatures of corrupted bins are replaced with the mean of the
remaining frequency bins.
This algorithm is relatively simple compared to other cross-frequency algorithms that
have been developed for a smaller number of channels [18], [16]. However it will
be shown to be successful in removing much of the observed RFI in what follows.
Simplicity in the algorithm is desirable in order to make future implementation in
digital hardware feasible. An algorithm very similar to the one described here will be
used to analyze the performance of cross-frequency detection algorithm theoretically
in Chapter 6.
While it is again possible ideally to predict the false-alarm rate of this algorithm,
the consideration of neighboring frequency bins in the cross-frequency blanking process complicates the analysis. A Monte Carlo analysis however showed the false alarm
rate for the specified algorithm to be ≈ 2%. These false alarms do not impact obtained mean brightnesses due to the replacement of detected bin brightnesses with
the average of the remaining brightnesses.
As an example of the algorithm’s performance, Figure 4.11 plots a comparison
of unmitigated and mitigated brightness temperatures versus UTC time for CISR
channels 8, 12, 16, and 20 during a one-hour portion of the flight (all over land).
These total channel brightnesses were calculated by taking the average of calibrated
brightnesses in the 1024 CISR frequency bins corresponding to a 100 MHz bandwidth.
Channel 8 represents observations from 6.2 to 6.3 GHz, channel 12 is 6.6 GHz to 6.7
GHz, channel 16 is 7 to 7.1 GHz, and channel 20 is 7.4 to 7.5 GHz. The brightness
temperature points shown are averaged over observations in the specified channel
during a 40 sec. time period; this is equal to one period of the antenna’s conical scan.
Time periods with no points in the figures correspond to times when the antenna was
152
CISR Channel: 8
CISR Channel: 12
500
450
500
CISR
CISR Mitigated
450
400
350
350
300
300
K
400
CISR
CISR Mitigated
250
17:29:52
18:00:27
18:31:03
250
17:29:53
CISR Channel: 16
18:31:04
CISR Channel: 20
500
450
18:00:28
500
CISR
CISR Mitigated
450
400
350
350
300
300
K
400
CISR
CISR Mitigated
250
17:29:53
18:00:29
UTC Time
18:31:04
250
17:29:53
18:00:29
UTC Time
18:31:04
Figure 4.11: Comparison of calibrated brightness temperatures vs. time between
original and mitigated data for CISR channels 8,12,16 and 20
153
not in conical scanning mode or when it observed external hot or cold load targets
for calibration purposes.
All four of these channels (as well as the CISR channels not plotted) show significant RFI corruption, with brightnesses averaged over 40 seconds exceeding 400
K in some cases, and the level of RFI corruption shows a significant variation over
time. In measurements around 18:00 UTC when the aircraft was over the Dallas-Fort
Worth area, all channels show large brightness temperatures. The higher frequency
channels show less RFI corruption compared to the lower frequency channels, which
is in accordance with the expected higher source densities at lower frequencies. However, all channels show at least 50 K of RFI contributions at some instant during
this interval. Results following application of the cross-frequency mitigation procedure show a greatly reduced impact of RFI. One interesting observation in general is
that, although all 22 CISR channels were mitigated separately, mitigated brightnesses
are very similar in each channel as should be expected for thermal noise measurements. Variations of the mitigated temperatures with time also are consistent with
expectations for observations over geophysical landscapes.
To make a more detailed analysis, CISR data from three different portions of the
flight will be inspected: one over DFW, one in rural Texas, and one over the Gulf of
Mexico.
4.3.2
CISR observations near DFW
A detailed time history of CISR data for channel 16 (7-7.1 GHz) (without any
averaging of the observed data over time) is presented in the lower half of Figure
4.12 for an approximately 90 sec. portion of the flight over the Dallas-Fort Worth
area. As this is an urban scene, it is not surprising that a high degree of RFI activity
154
with an apparent period of ≈40 sec. (the antenna rotation time) can be observed.
The mitigation algorithm significantly reduces this RFI. The results also show that
portions of the time history without obvious RFI have similar brightnesses before
and after the mitigation algorithm. However, RFI mitigations as large as 215 K
are observed in some cases. Spectrogram images of brightnesses for the original and
mitigated data for the same time period are also shown in Figure 4.12 (two upper
plots). The horizontal axis is UTC time and the vertical axis is frequency in MHz for
these images. The source of the periodic brightness increases in the time domain plot
can be seen clearly here, including apparent sources near 7010, 7040, 7060, 7070, and
7090 MHz. The mitigated data image shows that the algorithm developed removes
these contributions.
It is possible to match the sources observed in this overpass with licensed sources
from the FCC as well as the JSC database; the FCC’s license locator tool can be
used for this purpose [55]. RFI sources that are easily identifiable for this purpose
include two TDWR Doppler-Weather Radar systems located near the DFW airport;
these systems are pulsed radars operating between 5600 and 5650 MHz, and appear
to have been captured in the CISR dataset.
4.3.3
CISR observations in more rural Texas
To demonstrate cross frequency mitigation performance for weak RFI environments, a 45 sec. observation over a rural area between Fort Worth and San Antonio
is used. A comparison of original and mitigated CISR data for channel 17 (7.1 GHz7.2 GHz) is provided in Figure 4.13, again with the time history of total channel
brightnesses in the lower plot and brightness spectrograms in the upper plots. Narrowband interferers around 7107, 7135, 7145 and 7157 MHz are observed in original
155
Calibrated Brightness (K°)
Mitigated Brightness (K°)
7000
7000
400
7020
400
7020
MHz
350
7040
350
7040
300
7060
300
7060
250
7080
250
7080
200
7100
17:55:30
17:56:16
UTC Time
200
7100
17:55:30
17:57:03
17:56:16
UTC Time
17:57:03
CISR Channel: 16
700
CISR
CISR Mitigated
600
K
500
400
300
200
17:55:30
17:56:16
UTC Time
17:57:03
Figure 4.12: Spectrogram images and time history of brightness temperatures for
original and mitigated data over an urban landscape, CISR channel 16
brightness image, with 7145 MHz having the largest amplitude; the mitigated image
indicates removal of these sources.
The time history plot demonstrates that the contributions of these detected RFI
sources to the brightness of the 100 MHz channel are in the range 1-3 K, and comparable to the estimated radiometer brightness standard deviation of 1.5 K. Such low
level RFI is very difficult for a traditional radiometer to detect because it is within the
range of the instrument sensitivity as well the range of expected geophysical brightnesses; the high spectral resolution of CISR however allows these narrowband but
large amplitude sources (i.e. amplitudes more than four standard deviations from
the mean brightness over frequency) to be readily detected and removed.
156
Calibrated Brightness (K°)
Mitigated Brightness (K°)
7100
450
7100
450
7120
400
7120
400
MHz
350
7140
350
7140
300
7160
300
7160
250
7180
7200
18:10:06
18:10:28
UTC Time
18:10:51
250
200
7180
150
7200
18:10:06
200
18:10:28
UTC Time
18:10:51
150
CISR Channel: 17
310
K
305
300
CISR
295
CISR Mitigated
290
18:10:06
18:10:28
UTC Time
18:10:51
Figure 4.13: Spectrogram images and time history of brightness temperatures for
original and mitigated data over a rural landscape, CISR channel 17
4.3.4
CISR observations over the Gulf of Mexico
An example of observations in CISR channel 8 over the Gulf of Mexico is provided
in Figure 4.14. The plot of total channel brightness versus time (lower plot) is not
completely free of RFI even in these at-sea observations. For this case, sources near
6204 MHz and 6282 MHz are successfully mitigated, as shown in the spectrogram
images in the upper portion of the figure. Although the contribution of these RFI
sources to the total channel brightness is smaller than in the case of observations in
the Dallas-Fort Worth area, brightness increases near 45 K can still be observed.
157
Calibrated Brightness (K°)
6200
6220
MHz
Mitigated Brightness (K°)
300
250
6240
6200
300
6220
250
6240
200
6260
150
6280
6300
18:51:34
200
6260
18:52:18
UTC Time
18:53:02
100
150
6280
6300
18:51:34
18:52:18
UTC Time
100
18:53:02
CISR Channel: 8
250
K
CISR
CISR Mitigated
200
150
18:51:34
18:52:18
UTC Time
18:53:02
Figure 4.14: Spectrogram images and time history of brightness temperatures for
original and mitigated data over the Gulf of Mexico, CISR channel 8
4.4
Comparisons with ADD
As mentioned before, an automatic algorithm that is based on timing constraints
and known PSR passband characteristics is developed at OSU to recover channel
information for the ADD data. However, this algorithm is quite complicated and
a complete description would disrupt the focus of this thesis. In this section, a
comparison of the resulting ADD data with that obtained by the CISR system will
be provided. Agreement between the general characteristics of the two datasets is a
proof of success for the ADD channel identification algorithm.
158
One issue in comparing calibrated CISR and ADD brightnesses involves the fact
that the two systems observe antenna brightnesses over slightly different time intervals. CISR observed the antenna only for ≈ 15.75 msec out of the approximately
20 msec total antenna observation, while the ADD observed for a larger portion of
this interval. In both cases, the brightnesses to be shown are averages over the entire antenna observing time available. Although some differences between observed
brightnesses may result due to these differences, it is expected that the differences
would be largest for pulsed-type interferers, which are unusual at C-band over 5.9
GHz. In addition, CISR did not make measurements continuously because approximately 2 sec. was required to write data to the CISR internal hard drive following an
11 second measurement. In the results to be shown, ADD data is not plotted during
periods in which CISR did not make observations.
Figure 4.15 plots time histories of calibrated brightness temperatures in ADD
subchannel 6 (i.e. 148.25 to 153.25 MHz IF frequency) and CISR. Results from
channels 12 through 15 are shown here. The CISR results shown are an average
of calibrated brightness temperatures over the frequency bins corresponding to the
ADD subchannel 6 3-dB bandwidth. The results show highly correlated observations
in general, although significant differences are observed in some cases. The general
similarity of the datasets nevertheless verifies the procedure used for determining the
ADD data RF frequency.
Scatter plots of CISR vs. ADD brightness temperatures for the same time period
are shown in Figure 4.16. Again the high degree of correlation between the two
datasets is evident in these results. Appreciable differences between the two often
show larger ADD brightnesses rather than larger CISR brightnesses. One possible
explanation for this difference is the fact that CISR results were averaged only over
159
ADD (blue solid) and CISR (red cross) brightness temp., PSR ch.:12, ADD subch.:6
K
400
200
56
58
60
62
PSR channel:13
64
66
68
56
58
60
62
PSR channel:14
64
66
68
56
58
60
62
PSR channel:15
64
66
68
56
58
60
62
64
Minutes past 17:00 UTC
66
68
K
400
200
K
400
200
K
400
200
Figure 4.15: Time history of calibrated brightnesses for ADD and CISR, for ADD
subchannel 6 and tuned PSR channels 12 to 15
160
the 3 dB passband of ADD subchannel 6, whereas the true ADD subchannel 6 has a
non-ideal passband and may therefore include RFI contributions from outside the 3
dB passband.
To investigate this possibility, the CISR data was re-averaged over Kaiser-Bessel
windows with β = 3.2, order 27, and the 3 dB cutoff frequencies of ADD subchannel
6. Figure 4.17 illustrates the new scatter plots obtained via this procedure. The
results show that many of the outlier points in the previous scatter plots have been
eliminated. Further improvements in these results would be obtained if a more precise
description of the ADD subchannel filters were incorporated.
Another comparison of ADD and CISR calibrated brightnesses for PSR channels
12 through 15 is provided in Figure 4.18. These spectrogram images have frequency
on the horizontal axis and time on the vertical axis. Data corresponding to ADD subchannels 1 and 2 are not shown in the spectrogram due to the low IF gain of CISR in
the middle of the 75-175 MHz IF bandwidth,. The results show that CISR and ADD
locate very similar RFI sources, again verifying the ADD frequency determination
procedure.
4.5
Summary and Remarks
An examination of CISR and PSR data obtained from the August 25th, 2005
test flight on board the WB-57 aircraft has been provided in this Chapter. The
approximately 125 minutes of joint data contains observations of a variety of scenes,
including strong and weak RFI environments. The data clearly show the possibility of
significant corruption of measured brightnesses throughout C-band, with the higher
frequencies typically observing less but still appreciable corruption. It was shown that
161
ADD
Scatter plot of calibrated ADD and CISR brightness temperatures, PSR channel:12, ADD subchannel:6
1000
500
100
200
300
400
500
600
PSR channel:13
700
800
900
1000
200
300
400
500
600
PSR channel:14
700
800
900
1000
200
300
400
500
600
PSR channel:15
700
800
900
1000
200
300
400
700
800
900
1000
ADD
1000
500
100
ADD
1000
500
100
ADD
1000
500
100
500
600
CISR
Figure 4.16: Scatter plot for calibrated brightnesses of ADD vs. CISR, for ADD
subchannel 6 and tuned PSR channels 12 to 15
162
Scatter plot after filtering, PSR channel:12, ADD subchannel:6
ADD
1000
500
100
200
300
400
500
600
PSR channel:13
700
800
900
1000
200
300
400
500
600
PSR channel:14
700
800
900
1000
200
300
400
500
600
PSR channel:15
700
800
900
1000
200
300
400
700
800
900
1000
ADD
1000
500
100
ADD
1000
500
100
ADD
1000
500
100
500
600
CISR
Figure 4.17: Scatter plot after the filtering on calibrated CISR data, for ADD subchannel 6 and tuned PSR channels 12 to 15
163
Calibrated ADD Brightness
UTC, ch:12
Calibrated CISR brightness
400
17.95
18
18.05
18.1
300
200
UTC, ch:13
6620
UTC, ch:14
6640
300
200
6730
300
200
6830
17.95
18
18.05
18.1
300
200
200
6740
400
300
200
6820
300
6730
17.95
18
18.05
18.1
6840
400
6640
400
6720
400
6820
200
6630
17.95
18
18.05
18.1
6740
17.95
18
18.05
18.1
300
6620
400
17.95
18
18.05
18.1
6720
UTC, ch:15
6630
400
17.95
18
18.05
18.1
6830
6840
400
17.95
18
18.05
18.1
6920 6930 6940
Frequency (MHz)
300
200
6920 6930 6940
Frequency (MHz)
Figure 4.18: Comparison of brightness temperature spectrograms for CISR (left) and
ADD (right), tuned PSR channels 12 to 15
164
a cross-frequency algorithm combined with CISR’s high frequency resolution can be
used to remove most of the obvious RFI.
A cross-comparison of the CISR dataset with observations from the University of
Michigan’s C-band Agile Digital Detector is also performed; results were similar and
verified the channel sorting algorithm developed for the ADD data.
165
CHAPTER 5
A STUDY OF THE SHAPIRO-WILK TEST FOR THE
DETECTION OF PULSED SINUSOIDAL RFI
5.1
Introduction
It was mentioned in Chapter 1 that “tests for normality” can be used to test the
degree to which measured fields appear to be Gaussian random variables, as expected
for the thermal noise measured by a radiometer. These tests are more general (“omnibus”) compared to pulse detection and cross-frequency detection methods which
are effective against RFI sources that are temporally or spectrally “narrow”, respectively. Several such tests for normality of a data sample have been studied extensively
in the statistical literature (e.g. [34, 56–65]).
Among tests for normality, only the kurtosis test [56, 57] has been demonstrated
to date for radiometer RFI detection with the ADD radiometer which was described
to some extent in previous Chapters [24–27, 31, 66]. Studies to date have focused
on pulsed-sinusoidal RFI due to the potential impact of radar interference on L-band
radiometers, and the results of [24,25], [26,27,31,66] show the kurtosis test to provide
high sensitivity to low duty cycle pulsed-sinusoidal sources as well as some sensitivity
to continuous sinusoidal RFI. However, as mentioned in Section 1.2.3, a “blind spot”
for pulses having fifty percent duty cycle was also shown for which the algorithm is
166
insensitive to RFI, and reduced detection performance occurs for a broad range of
duty cycles around fifty percent. In general, the kurtosis algorithm is insensitive to
RFI sources that produce kurtosis values identical to those of a Gaussian random
variable. It is possible to alleviate this problem by using higher order moments or
time subsampling as described in Section 1.2.3, but these methods require additional
resources or post-processing of the data.
This chapter documents an examination of an alternate test for normality called
the Shapiro-Wilk test, and compares its performance with that achieved by the kurtosis test for pulsed and continuous sinusoidal RFI. The Shapiro-Wilk test was first
proposed in 1965 [58], and has been shown to be capable of detecting non-normality
for a wide variety of statistical distributions, including those with Gaussian kurtosis
values [59, 60]. It has been recommended as a powerful omnibus test of normality [61]. An initial study of the Shapiro-Wilk test’s performance in detecting pulsed
sinusoidal RFI in comparison with two similar omnibus normality tests, those of
Shapiro-Francia [62] and Chen-Shapiro [63], was provided in [34]. The results of this
study generally showed that performance among these three tests is similar, and all
three tests are based on order statistics of Gaussian distributions so that only the
Shapiro-Wilk test is examined here.
The discussion to follow considers only pulsed and continuous sinusoidal RFI
sources; other RFI types may produce distinct conclusions regarding performance.
In Section 5.2, a brief review of the Shapiro-Wilk test is provided. Given the interest
in performing the test in digital hardware on-board a radiometer system, test implementation is a key issue that is considered. Section 5.3 describes the Monte Carlo
simulation procedure used to examine test performance. Computation of kurtosis
statistics is described next in Section 5.3.3. Section 5.4 presents results obtained
167
from Monte Carlo simulations of the Shapiro-Wilk test. Results are compared with
those achieved by the kurtosis method, and the influence of a test size parameter is
also explored. Final remarks are presented in Section 5.5.
5.2
The Shapiro-Wilk test
In the following it is assumed that a microwave radiometer measures N samples
of a received field, written as y(i) for i = 1 to N . Such samples are assumed to be
separated by the Nyquist sample rate of the receiver so that no correlation among
samples is present in the absence of RFI.
The Shapiro-Wilk test is based on a correlation of sample “order statistics” with
those of a normal distribution. The use of order statistics implies that the data sample
must be sorted, with the sorted sample written in vector form as y0 = (y1 , ..., yN ) in
increasing values; here the prime
0
is used to note the transpose of a vector. The
Shapiro-Wilk test statistic W is defined as:
³P
W = PN
N
i=1
i=1 (yi
where
m1 =
ai y i
´2
− m1 )2
N
1 X
y(i)
N i=1
(5.1)
(5.2)
is the sample mean. W can be interpreted as a ratio of two estimates of the variance
of the sample, with the estimate in the numerator holding only if the sample is drawn
from a normal distribution since coefficients ai are calculated by linear regression to
the expected values of standard normal order statistics. It can be shown that W is
bounded by 0 and 1, and that the expected value of W converges to 1 for Gaussian
input data as the sample size is increased. The expected value of W becomes smaller
as the input signal becomes non-Gaussian.
168
5.2.1
Expressions for the ai coefficients
The vector of coefficients a0 = (a1 , ..., aN ) is normalized so that a · a0 = 1 and is
also antisymmetric, so that a1 = −aN , a2 = −aN −1 , etc. Analytical expressions for
the coefficients ai are provided in [58], but the original forms are somewhat computationally complex. Instead empirical forms for these coefficients have been determined
in terms of u = N −1/2 as [67]:
aN = −2.706056u5 + 4.434685u4 − 2.071190u3
−0.147981u2 + 0.221157u + cN
(5.3)
aN −1 = −3.582633u5 + 5.682633u4 − 1.752461u3
−0.293762u2 + 0.042981u + cN −1
(5.4)
ai = ²−1/2 m̃i
(5.5)
with the final equation holding for i = 3 to N − 2.
In equation (5.5), m̃0 = (m̃1 , ..., m̃N ) is a vector of expected values for the order
statistics of the standard normal distribution, approximated by [68, 69]:
m̃i = Φ−1 {(i − 3/8)/(N + 1/4)}
(5.6)
where Φ−1 denotes the inverse of the standard normal distribution function. Also in
equation (5.5),
m̃0 · m̃ − 2m̃2N − 2m̃2N −1
²=
1 − 2a2N − 2a2N −1
(5.7)
Finally, the ci values in equations (5.3) and (5.4) are determined from vector c0 =
(c1 , ..., cN ) by
c = m̃/(m̃0 · m̃)1/2
169
(5.8)
5.2.2
Implementation in digital hardware
Computation of the W statistic requires evaluation of the sample moments up to
second order (for the denominator), and also a weighted sum of the sorted sample
(for the numerator). It is assumed that floating point operations are not desired for
an on-board digital processor, so that the numerator and denominator are recorded
for computation of W in post-processing. The required sorting operation for the
numerator can be accomplished either through the use of existing sort algorithms for
a fixed sample size [70] or through a histogramming procedure in which counters are
used to record the number of occurrences of each possible sample value.
It is assumed here that a sorting operation is used, and that the value of N is
limited to a maximum of 4096 by hardware constraints. If it is desired to compute
W for datasets larger than 4096 samples, it is assumed that a larger sample of size
Q = IN is split into I sub-sets of size N , for which the square root of the numerator
and the denominator are separately computed. The I values for the sub-sets are
then averaged, the numerator term is squared, and the result divided by the averaged
denominator to obtain the W estimate for the entire sample. The impact of N in this
process will be examined in Section 5.4.2. The coefficients ai are fixed for a given N ,
so that computation of the W numerator is similar to a standard filter computation
once the data sort operation is completed. While these steps are somewhat more
complicated in terms of hardware implementation than purely moment based tests,
overall the Shapiro-Wilk test appears to be reasonably well suited for implementation
in hardware.
170
5.2.3
Quantization effects
When the yi data are rounded to integer values as in a fixed point digital processor,
Sheppard’s correction to the variance estimate is applied by subtracting (N − 1)/12
from the value in the denominator [71]. An examination of quantization impact on
the coefficients ai was also performed. Figure 5.1 plots a comparison of coefficients
before and after quantization using 8-bits for N = 4096. The anti-symmetric nature
of the ai coefficients is apparent in the Figure, along with the increasing nature of the
coefficients at large offsets from the array center. The lower plot of Figure 5.1 focuses
on the region near the center of the coefficient array, and reveals the quantization
more clearly. In the Monte Carlo simulations of Shapiro-Wilk test performance to be
presented in Section 5.4, tests showed that the ai coefficients for N = 4096 could be
quantized in as few as 8 bits without a significant loss in test performance, and eight
bit quantizations for these coefficients are used in all the results to be shown. The
impact of quantization of the yi data is further examined in Section 5.4.1.
5.3
Simulation Procedure
Because the W test statistic for normally distributed input data does not follow a
standard probability density function (pdf), the original presentation of the ShapiroWilk test [58] included empirical tables of this pdf for sample sizes up to 50. Later, an
approach was derived to transform W into a statistic that is approximately normal
in the RFI-free case [67, 72, 73] so that prediction of expected false-alarm rates is
possible. However, tests on the simulation results described in Section 5.4 showed this
transformation to be very sensitive to quantization effects, and the transformation is
not used for this reason. Instead a Monte Carlo simulation approach is applied; the
probability of false alarm and probability of detection are determined by counting
171
(a)
0.06
Coefficient
0.04
0.02
0
−0.02
Before quantization
After quantization
−0.04
−0.06
500
1000
1500
2500
3000
3500
4000
(b)
−3
1.5
2000
Index
x 10
Coefficient
1
0.5
0
−0.5
Before quantization
After quantization
−1
−1.5
1950
2000
2050
Index
2100
2150
Figure 5.1: Weight coefficients ai before and after quantization using 8-bit resolution
172
the number of realizations declared as RFI through a test on W in cases with no
RFI present and with RFI present, respectively. Monte Carlo simulations utilized
30000 realizations in all the results to be shown, and tests with larger numbers of
realizations showed negligible impact on the curves illustrated.
5.3.1
Signal model and notations
In a previous modeling study of the kurtosis test [26], received fields when pulsed
sinusoidal RFI is present were modeled as uncorrelated Gaussian random variables
with zero mean and standard deviation σ plus sinusoidal RFI with a specified amplitude and a uniformly distributed random phase. Although such an approach can
be acceptable when calculating statistics for large samples containing many pulses,
it does not realistically describe radars and other pulsed sources when resolved at
higher time resolutions. Another signal model suggested in [74] is instead used in
this study (which was in turn adapted from [75]), and is reviewed here.
In this model, two different time scales are used. The first scale, N , denotes the
number of samples on which the denominator and the square root of the numerator
of the Shapiro-Wilk test are computed (the N in equation (5.1)). The longer scale
consists of Q = IN samples, and represents the radiometer integration period in
samples. As described previously, I numerator and denominator values are obtained
from the Shapiro-Wilk computations during an integration period, and then averaged
and used to compute the W value for the entire sample.
For illustration, the sampling period of the digital radiometer system is assumed
to be 16 nsec, and the integration period Q is taken as 32768 samples so that the
total radiometer integration period is 524.288 µsec. The Shapiro-Wilk test length N
173
is set to 4096 samples typically (i.e. 65.536 µsec); the effect of varying N is examined
in Section 5.4.2.
For the case of pulsed sinusoidal RFI, sampled received fields can be written
as [74]:
xi [n] = A cos(2πf0 [(i − i0 )N + n] + φ)I(n, i) + wi [n]
n = 0, 1, · · · , N − 1, i = 0, 1, · · · , I − 1
(5.9)
where wi [n] refers to the independent identically distributed (i.i.d.) Gaussian measurements, with zero mean and standard deviation σ. The function
I(n, i) =
½
1 i0 N + Ns ≤ iN + n < i0 N + Ns + Np
0 otherwise
(5.10)
is an “indicator” function that locates RFI containing samples. Here, i0 is the pulse
arrival “frame”, A is the amplitude, f0 is the frequency, and φ is the phase of a single
sinusoidal pulse. It is assumed that if there are multiple pulses (i.e with arrival frames
i0 , i1 , · · · ) within the Q sized sample, they have identical amplitudes A and durations
Np , but the frequency f0 , phase φ, and arrival sample Ns are chosen independently.
These choices result in a duty cycle, d, of
d=
Npulse Np
IN
(5.11)
where Npulse is an integer.
As in [26], sine wave amplitudes in what follows are described in terms of the ratio
R of the average “signal-to-noise” power ratio (dA2 / (2σ 2 )) normalized by a factor
√
proportional to the uncertainty in the radiometer power estimate (σ 2 / N I). This
definition gives
dA2 √
R =
NI
s2
A =
2R
√
d NI
174
(5.12)
(5.13)
The parameter R thus describes the RFI power contribution to the integrated power
in units proportional to the brightness standard deviation (i.e. NEDT). Typically RFI
having R > 10 would be readily detectable without additional detection procedures,
so the primary interest is in detecting RFI having R < 10.
5.3.2
Cases considered
Monte Carlo simulations were performed for fixed pulse amplitudes (R), duration (Np ), and number of pulses (Npulse ) in a frame of 32768 total samples. Pulse
frequencies were uniformly distributed from 0 < f0 < 1/2 and independent from
pulse-to-pulse. Phases φ were also selected uniformly from 0 to 2π radians, and pulse
arrival frames i0 and arrival sample Ns were also selected uniformly within the constraints required by the specified number of pulses and pulse durations (i.e. pulses
were not allowed to overlap in time.)
The simulations performed were chosen to obtain a basic understanding of test
performance in semi-realistic RFI situations. First, a case having ten pulses of length
Np = 32 (0.512 µsec) within the integration period of 524.288 µsec was considered to
model very short pulse radar-like emissions at a very high pulse repetition frequency;
the resulting duty cycle is approximately 1%, and simulations were performed for R
values of 0 (false alarm case), 2.5, 5, and 10. Second, a case having a single pulse
of length Np = 16384 (i.e. 256.14 µsec) was used to obtain a 50% duty cycle which
is of interest due to the blind spot encountered by kurtosis at this duty cycle. Such
longer radar pulses are also produced by many radar systems, as evidenced by the
data reported in [66]. R values used for the 50% case were identical to those in
the 1% duty cycle case. Finally, a case with a 100 percent duty cycle was used to
175
model continuous RFI sources; in this case much larger R values of 0, 30, · · · 120 were
required, as will be discussed in Section 5.4.
5.3.3
Computation of the kurtosis statistics
Monte Carlo simulations are also used to compute the kurtosis statistics whenever
a performance comparison between the Shapiro-Wilk and the kurtosis tests is given.
To compute kurtosis statistics, second and fourth order central moments of the received fields were calculated; Received fields used were same for both Shapiro-Wilk
and the kurtosis tests and they were simulated in the manner described in Section
5.3.1. Note that ith central moment, mi , of a signal x is defined as:
E
D
mi = (x− < x >)i
(5.14)
where “<>” denotes the expected value operation. In practice, moments around the
origin (denoted as µi ) are used to calculate central moments. Obviously, µ1 = m1
while second and fourth order central moments are given as [26]:
m2 = µ2 − m21
(5.15)
m4 = µ4 − 4µ3 µ1 + 6µ2 m21 − 3m41
(5.16)
The kurtosis statistic was computed using the entire dataset of IN samples (i.e.
no sub-sampling.) Incorporating temporal subsampling into the kurtosis test may
produce different results [31, 66]; however, for the approximations in the calculation
of kurtosis test to hold at least 10000 samples is recommended ( [26] whereas number
of samples used for SW test is 4096, therefore such an approach is not considered
here. After second and fourth order central moments are calculated, kurtosis statistic
(denoted as Kn here) is found as:
176
Kn =
m4
m22
(5.17)
This definition of kurtosis does not include quantization effects; Sheppard’s correction is used to eliminate quantization effects in the kurtosis calculations as well [26].
5.4
5.4.1
Results
Histograms of W
The Monte Carlo simulation produces a set of W values for the cases described
in Section 5.3.2, which include both a false alarm simulation R = 0 and simulations
when RFI is present. Histograms of the W values obtained for the d ≈ 1% case
are presented in Figure 5.2. The upper plot (no-quantization of yi data) shows the
concentration of W near 1 in the RFI free case, as expected, although the mean of
the pdf when R = 0 is not exactly 1 [58] due to the finite sample size utilized in
the test. Non-Gaussian behavior is indicated in the Shapiro-Wilk test when the test
statistic value becomes significantly less than one; values of W less than a threshold
t are therefore declared as containing RFI. This basic behavior of W is apparent in
the Figure, where although the histograms of R = 0 and R = 2.5 are nearly identical,
the histogram means become smaller with increasing R. A larger variance of the test
statistic is also observed in this case as R is increased.
Histograms are also shown in the lower plot of Figure 5.2 when quantization
effects of the yi data are included. These data are generated by assuming that the
RFI free noise has a standard deviation σ = G and that the data are then rounded
to integer values. Larger values of G therefore should lead to reduced quantization
effects, although clipping would eventually become a problem for very large G values.
A rule of thumb to avoid clipping is to select the number of bits for the RFI-free
177
case such that signals with amplitudes at least 6 times G do not cause clipping. In
practice, the value of G is determined by the ratio of the observed thermal noise
voltage standard deviation to the bit width of the analog to digital converter used
in the radiometer. The results show negligible differences between the histograms
illustrated when quantization with G = 4 is considered. Quantization effects can
become significant for G < 4; however such effects are not considered further as
G ≥ 4 is a reasonable expectation for many radiometer systems.
Duty Cycle (d) ≈ 1%, No rounding
PDF of W
10000
R=0
R=2.5
R=5
R=10
1000
100
10
0.996
0.9965
0.997
0.9975
0.998
0.9985
0.999
0.9995
1
1.0005
W
Rounded, G = 4
PDF of W
10000
R=0
R=2.5
R=5
R=10
1000
100
10
0.996
0.9965
0.997
0.9975
0.998
0.9985
0.999
0.9995
1
1.0005
W
Figure 5.2: Histograms (scaled to correspond to probability density functions) of W
for non-quantized (upper Figure) and quantized data (G = 4, lower Figure), d ≈ 1%.
178
5.4.2
Receiver operating characteristic curves
Given the Monte Carlo set of W values, the probability of false alarm and probability of detection as a function of the threshold t in the test W < t can be computed.
Receiver operating characteristic (ROC) curves plotting the probability of detection
versus the probability of false alarm as the threshold t is varied can then be produced.
Figure 5.3 illustrates the obtained ROC curve for the d ≈ 1% case. The results
show that the test has poor performance at detecting RFI with R = 2.5, but improved
performance for R = 5. A further examination of quantization effects is also provided
by including the corresponding ROC curves using G values of 2, 4, and 8. The results
show a modest impact of quantization when G = 2 that becomes negligible for G ≥ 4.
Equation (5.13) shows that, for a fixed R value, the RFI sine wave amplitude A
is decreased as d is increased. This fact suggests that test performance may degrade
for higher duty cycle interference. To examine this issue, Figures 5.4 and 5.5 plot the
corresponding results for d = 50% and d = 100%, respectively. For the d = 100%
case in Figure 5.5, performance is relatively poor even with an R value of 60; other
tests such as Cross-frequency blanking whose practical implementation is described
in [23] as well as previous chapters of this work, would be preferable for continuous
sinusoidal interference. This will indeed be proven in Chapter 6. It is interesting to
note however that the d = 50% case of Figure 5.4 shows performance comparable
to that achieved for d = 1%. This is due to the fact that detection is not only a
function of d and R, but also of the relationship between the pulse duration (Np ) and
the local test size N . Simulations using a large number of shorter pulses to achieve
a 50% duty cycle showed that much higher R values were required to obtain good
test performance. Both Figures 5.4 and 5.5 again show quantization of the yi data to
179
have minimal impacts when G ≥ 4, although the impact is somewhat larger than in
the d ≈ 1% case.
Duty Cycle (d) ≈ 1%
1
0.9
0.8
0.7
Pd
0.6
0.5
0.4
No rounding, R=2.5
Rounded, G=2, R=2.5
Rounded, G=4, R=2.5
Rounded, G=8, R=2.5
No rounding, R=5
Rounded, G=2, R=5
Rounded, G=4, R=5
Rounded, G=8, R=5
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 5.3: ROC curves for d ≈ 1%
To examine further the impact of N , ROC curves were computed for fixed R
values as the sample size N was varied from 64 to 4096 (keeping the total integration
period N I constant at 32768 samples). The results are shown in Figures 5.6 and 5.7
for the d ≈ 1% (R = 2.5) and d = 50% (R = 5) cases, respectively. For d ≈ 1%, the
best detection performance is obtained when N = 64, and performance degrades as
N increases. This is not surprising since the pulses simulated in this case have length
Np = 32, so that the individual N = 64 sub-tests are more closely matched to the
duration of the RFI pulses. The d = 50% cases instead shows improved performance
180
Duty Cycle (d) = 50%
1
0.9
0.8
0.7
Pd
0.6
0.5
No rounding, R=2.5
Rounded, G=2, R=2.5
Rounded, G=4, R=2.5
Rounded, G=8, R=2.5
No rounding, R=5
Rounded, G=2, R=5
Rounded, G=4, R=5
Rounded, G=8, R=5
0.4
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
P
0.6
0.7
0.8
0.9
1
fa
Figure 5.4: ROC curves for d = 50%
as N becomes larger, again as might be expected since the RFI pulse in this case has
a duration of 16384 samples. These results show that it is desirable to choose the
sample size N to be matched to the duration of expected RFI pulses if this duration
is known a-priori. In the absence of such information, the parameter N should be
chosen as a trade-off between implementation complexity and test performance given
any knowledge of a potential range of RFI pulse lengths. The results of the next
section were computed using N = 4096 to represent the latter case.
5.4.3
Comparison of ROC curves for the Shapiro-Wilk and
kurtosis tests
Figures 5.8, 5.9, and 5.10 compare ROC curves achieved by the Shapiro-Wilk and
kurtosis tests for the d ≈ 1, 50, and 100 percent cases, respectively. ROC curves for
181
Duty Cycle (d) = 100%
1
0.9
0.8
0.7
Pd
0.6
0.5
0.4
No rounding, R=30
Rounded, G=2, R=30
Rounded, G=4, R=30
Rounded, G=8, R=30
No rounding, R=60
Rounded, G=2, R=60
Rounded, G=4, R=60
Rounded, G=8, R=60
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
P
0.6
0.7
0.8
0.9
1
fa
Figure 5.5: ROC curves for d = 100%
the kurtosis test were obtained from a Monte Carlo procedure under the same signal
model as that used for the Shapiro-Wilk analysis as explained in Section 5.3.3.
Results in Figure 5.8 show both tests to go from very poor performance to near
perfect performance as R is varied from 2.5 to 10. In between these values, the kurtosis
test achieves the better performance. The performance of the Shapiro-Wilk test can
be improved to exceed that of the full sample kurtosis test if N is selected through an
a-priori knowledge of the RFI pulse length, although kurtosis test performance could
also be improved if temporal sub-sampling were incorporated.
The situation is distinctly different in Figure 5.9 for duty cycle 50 percent, due
to the “blind spot” of the kurtosis test in this case. The Shapiro-Wilk test retains
sensitivity (although the results may be different for other choices of N and Np as
182
Duty cycle (d) ≈ 1%, R = 2.5
1
0.8
Pd
0.6
0.4
N=64
N=256
N=1024
N=4096
0.2
0
0
0.1
0.2
0.3
0.4
0.5
P
0.6
0.7
0.8
0.9
1
fa
Figure 5.6: ROC curves vs. test size N , R=2.5, d ≈ 1%
Duty cycle (d) = 50%, R = 5
1
0.8
Pd
0.6
0.4
N=64
N=256
N=1024
N=4096
0.2
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
Figure 5.7: ROC curves vs. test size N , R=5, d = 50%
183
1
explained in Section 5.4.1), while the kurtosis test is insensitive to the presence of
RFI.
Conclusions for the 100 percent duty cycle case in Figure 5.10 are similar to those
in the one percent duty cycle case in that both tests go from poor to near perfect
performance over a similar range of R values. Again the kurtosis test achieves better
performance over this range. However, the required R value for good performance is
quite high (up to 120) which suggests that the performance of both tests would likely
be exceeded by other approaches or by incorporating multiple frequency channels into
the radiometer system. In Chapter 6, cross-frequency detection will be shown to be
a viable alternative for the detection of CW signals.
Duty Cycle (d) ≈ 1%
1
0.8
Pd
0.6
0.4
SW, R=2.5
Kurtosis, R=2.5
SW, R=5
Kurtosis, R=5
SW, R=10
Kurtosis, R=10
0.2
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 5.8: Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d ≈ 1%
184
Duty Cycle (d) = 50%
1
0.8
Pd
0.6
0.4
SW, R=2.5
Kurtosis, R=2.5
SW, R=5
Kurtosis, R=5
SW, R=10
Kurtosis, R=10
0.2
0
0
0.1
0.2
0.3
0.4
0.5
P
0.6
0.7
0.8
0.9
1
fa
Figure 5.9: Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d = 50%
5.5
Summary and Remarks
The performance of the Shapiro-Wilk test for normality was analyzed in detecting
pulsed sinusoidal RFI. Results showed that the test can be successful in detecting
pulsed sinusoidal RFI, particularly for duty cycles of 50% or less, and simulations not
reported here over a larger range of duty cycles showed similar performance to those
illustrated in Section 5.4. A comparison of results with those achieved by the kurtosis
test showed that the two tests produce qualitatively similar results, with the kurtosis
test generally achieving better performance in most cases. However, the Shapiro-Wilk
test was shown not to suffer from the “blind spot” encountered in the kurtosis test
at duty cycle 50 percent. It was also shown that Shapiro-Wilk test performance can
be improved if a-priori expectations regarding RFI pulse lengths are available.
185
Duty Cycle (d) = 100%
1
0.8
Pd
0.6
0.4
SW, R=30
Kurtosis, R=30
SW, R=60
Kurtosis, R=60
SW, R=120
Kurtosis, R=120
0.2
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 5.10: Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d =
100%
186
Discussions of implementation of the Shapiro-Wilk test in hardware along with the
effects of the quantization were also provided. Simulations showed that performance
loss due to quantization is not significant if the radiometer thermal noise has a voltage
standard deviation of approximately four times the analog to digital converter bit
width. Implementation appears feasible, so that experimental tests should be possible
in the future.
187
CHAPTER 6
PERFORMANCE STUDY OF A CROSS-FREQUENCY
DETECTION ALGORITHM FOR PULSED SINUSOIDAL
RFI IN MICROWAVE RADIOMETRY
6.1
Introduction
In previous chapters, several earth observing radiometric systems including LISR/CISR,
ADD and PSR/CX that is capable of RFI detection have been described. Three main
methods used in these radiometers for RFI detection purposes were time-domain detection, cross-frequency detection and kurtosis detection methods.
A central question for future radiometric systems is the selection of a detection
method (or methods) that results in the best performance for a specific application.
Such a selection of course depends on the RFI environment that is to be observed, but
present knowledge of the RFI environment remains limited. It has been mentioned in
Chapter 5 that one RFI type that has been considered in previous studies is pulsed
sinusoidal RFI, due to the ability of this type to describe radar-like emissions (low
duty cycles) as well as continuous (i.e. very narrowband or “CW”) sinusoidal interference. Comparative studies of several algorithms for pulsed sinusoidal RFI have been
performed [32, 66, 74]. However, the performance of cross-frequency detection algorithms was not analyzed in the past. Cross-frequency detection can be performed for
188
radiometers having multiple frequency channels with only modest additional requirements on hardware or datarate, and also can be used together with other methods
such as kurtosis algorithms. Therefore, it is of interest for future systems to determine the detection performance of the cross-frequency approach, particularly in cases
where other detection methods perform poorly.
In this chapter, a theoretical performance model for cross-frequency detection
of pulsed sinusoidal RFI is reported with the motives outlined above. Results for
pulse-detection and subsampled kurtosis detection methods are also provided for
comparison purposes. In a practical implementation of the cross-frequency detector,
the detection threshold should vary with respect to the brightness temperature of the
observed scene in order to maintain approximately constant false alarm and detection
rates. A method for estimating this threshold and the impact of this estimation on
detection performance is also studied. Notations and the signal model used in this
Chapter is very similar to the one used in Chapter 5 for the analysis of the ShapiroWilk test; they will be briefly described in next section. Performance models used
for each of the three detectors considered will also be described. Results comparing
the performance of these detectors for different duty cycles and RFI strengths are
then presented in Section 6.3. Special attention is given to the CW case where the
pulse and subsampled kurtosis algorithms perform poorly. Effects of the system
temperature estimation on the cross-frequency detector are studied in Section 6.4,
and final remarks are provided in Section 6.5.
189
6.2
6.2.1
Formulation
Signal model and notations
The signal model employed in Chapter 5 (which was adapted from [74]) for the
RFI detection performance analysis of the Shapiro-Wilk test of normality is also used
in this chapter when analyzing the cross-frequency approach. This model is based
on a discretized representation of the fields observed by the radiometer, assuming
sampling at the Nyquist rate of the observed bandwidth. However, a simplified
signal model from [32] is used when comparing with the pulse and kurtosis methods
because modeling kurtosis algorithm performance is more difficult under the signal
model of [74]. The signal model is not reviewed further here save for the meaning of
parameters like I and N ; however, the simplifications of [32] will be explained when
the subsampled kurtosis method is discussed in Section 6.2.4.
In the model of Section 5.3.1, a radiometer integration period of Q samples is
divided into a set of I sub-sampling periods of length N each such that Q = IN . For
the cross-frequency detector, N corresponds to the length of an FFT operation (in
samples) used to produce the multiple frequency channels examined by the detector.
For illustration, Q is chosen as 768000 samples in this Chapter, and N values of
8, 16 and 32 are considered. For a radiometer sampling, for example, at 20 nsec (i.e.
the Nyquist rate of a 50 MHz bandwidth), the value of Q used results in a 15.36 msec
radiometer integration period.
It was mentioned in Section 5.3.1 that it is of interest to represent the amplitude
of the RFI sine waves in terms of the uncertainty in the radiometer power estimate
q
(which for an integration of Q samples is equal to Tsys Q2 where Tsys = σ 2 is the
radiometer system temperature).
190
For this reason, RFI amplitudes were described in terms of the exact ratio of the
average signal power (dA2 /2) normalized by the uncertainty in the radiometer power
estimate in a journal paper we submitted that was based on this study [76]. Note
that this ratio, which I will denote here as R0 , is equal to:
dA2
R0 =
2Tsys
There is an extra factor of
√
r
Q
2
(6.1)
2 in the definition of R0 compared to R given in
Equation 5.12 (ratio R was also defined as such in the journal paper we submitted on
the Shapiro-Wilk test for RFI detection; reference [35]). In this study, to be consistent
with the notation of the previous chapter, signal strength will be given in terms of R
√
as defined by Equation 5.12. Thus, reader should keep in mind the absence of 2 if
he/she were to compare the results of this chapter and the related journal paper [76].
There is not any real importance of using one definition of RFI strength over another
as long as the definition employed is clearly explained. It should be mentioned again
that RFI having R larger than approximately 10 would likely be detectable in the
original radiometer measurements; cases of interest therefore have R < 10.
6.2.2
Cross-frequency detection model
A schematic of the cross-frequency detection algorithm considered here is shown
in Figure 6.1. Incoming time domain field samples xi [n] are passed through a nonoverlapping N point FFT operation to obtain FFT outputs Xi [k]. The power in these
FFT outputs (|Xi [k]|2 ) is then computed; this quantity is called a power spectrum.
A total of I power spectra is obtained in the radiometer integration period of Q
samples; these I spectra are averaged to obtain the average power spectrum Y [k].
The cross frequency detector chooses the maximum over k (as described below) of
191
Y [k] and declares detection if the maximum exceeds a threshold value Tcross . This
threshold is related to the expected scene brightness temperature if it is assumed
that the detector is applied after calibration of the radiometer measurements. If the
scene temperature is not known a-priori (as is the case in radiometry), methods for
estimating the threshold can be applied as described in Section 6.4.
Figure 6.1: Schematic of the cross-frequency detector
In the absence of RFI, the FFT operation produces N complex Gaussian random variables for Xi [k]. N/2 − 1 of these N values are positive frequencies that
have corresponding conjugate negative frequency points; these negative frequencies
are discarded. The square of the amplitude of each of these N/2 − 1 positive frequency components is a chi-squared random variable with two degrees of freedom.
The FFT operation also produces DC (k = 0) and Nyquist (k = N/2) frequency
outputs that are purely real. For consistency with the positive frequency points, the
power in the DC and Nyquist frequency outputs is averaged to obtain an additional
chi-squared random variable with two degrees of freedom. As a result, N/2 such
random variables are obtained for each frame; the algorithm is described as using
192
N/2 “channels” henceforth. The averaging over I spectra then produces N/2 chisquared random variables with 2I degrees of freedom (Y [k]). If these N/2 random
variables are independent, which is the case in the absence of RFI when a rectangular
FFT “window” is used, the probability that the maximum of the Y [k]’s is less than
Tcross can be calculated by multiplying the probabilities that each of the N/2 random
variables is less than Tcross . The resulting quantity then allows computation of the
detector false alarm rate as a function of Tcross .
When RFI is present, the detector computes the maximum of N/2 non-central
chi-squared random variables with 2I degrees of freedom. The non-centrality parameters of these random variables can be obtained by computing the Fourier transform
of a specified pulsed source. These random variables remain independent so that
the maximum operation can be computed as in the false-alarm case. An average
probability of detection for a specified threshold can then be obtained by numerically
averaging the probability of detection over the frequency, phase, arrival sample, and
other properties of the RFI signal. This process includes the influence of “scalloping”
loss in the FFT (i.e. the RFI pulse frequency is not aligned with an FFT bin) as well
as partial filling of FFT N-point “frames” by an RFI pulse.
In comparison with the cross-frequency detection algorithms applied in postprocessing to LISR/CISR data that were described in the previous chapters, this
algorithm have two main differences:
• Passband response of the radiometer system is not taken into account.
• Spectral slope of the natural thermal radiation is not considered.
In other words, all frequency bins are assumed to have the same mean. System
passband response is device dependent; however, it can be said that to consider the
193
spectrum to be relatively flat where the variations in the passband response is small
in magnitude compared to the difference between the threshold and the assumed
mean level is a reasonable assumption. Spectral slope can be incorporated to a crossfrequency detection algorithm, such an algorithm is used for RFI mitigation in the
PSR/CX radiometer [18] as described in Chapter 4; but this effect is also not very
important if the total bandwidth considered is not on the order of GHz, and obviously
PSR/CX cross-frequency model is not suitable for a theoretical study or a real time
implementation of the cross-frequency method in hardware.
It should also be pointed out that in LISR/CISR systems a 1024 point FFT
operation is applied to the received fields. Although this high number of frequency
channels provide a great opportunity for the identification and mitigation of RFI,
such a high number of frequency channels is unlikely in a radiometer located in a
satellite where energy and datarate is at a premium. Hence, number of frequency
channels used in this chapter is limited by 32 as mentioned in Section 6.2.1.
6.2.3
Pulse detection model
A schematic of the pulse detection model considered in this chapter is shown in
Figure 6.2. In this model, it is assumed that the square of time domain measured
fields xi [n] are summed over N samples. This sum, denoted as P Di in the schematic,
is a chi-squared random variable with N degrees of freedom when there is no RFI.
When RFI is present, a non-central chi-squared random variable with N degrees of
freedom is obtained instead. The non-centrality parameter for this case can again
be computed for specified RFI pulse properties. During a radiometer integration
period of Q = IN samples, I such random variables are attained, and the detection
statistic is the maximum of these N random variables. The probabilities of false
194
alarm and detection for a specified threshold are calculated as the probability that
the test statistic is greater than the threshold (TP U LSE ) in the absence and presence
of RFI, respectively.
Figure 6.2: Schematic of the pulse detector
This algorithm is fairly similar to the APB algorithm described in Chapter 2.2.3.
Primary difference between the two pulse detection models is the fact that the APB
system is built for RFI “mitigation”, while the sole purpose of this model is pulse
detection. As a result, while the APB algorithm assumed that there could be several
pulses within an integration period and eliminated them in real time, in this particular
model data from the whole radiometer integration period of Q is classified as either
“corrupt” or ”clean” depending on the outcome of the test. Although this may seem
more restrictive compared to the APB algorithm at first, we might remember that
blanking window of APB had a certain width (denoted as NBLAN K ) and this width
was comparable to the expected pulse widths of the RFI sources. Therefore, it might
be concluded that if radiometer integration length Q is comparable to NBLAN K ,
mitigation performance of the two algorithms should not be very different.
195
6.2.4
Kurtosis detection model
A schematic of the “sub-sampled” kurtosis detector (different from Chapter 5 in
the sense that it allows multiple frequency channels and time subsampling) is shown
in Figure 6.3. In this case, N/2 frequency sub-channels are assumed to be obtained by
filtering in hardware rather than through an FFT operation. In the absence of RFI,
received fields in each channel are independent Gaussian random variables having
a variance that is reduced from that of the total channel (σ 2 ) by a factor of N/2.
Additionally, each of the kurtosis sub-channels originally has 2Q/N time samples to
maintain the Nyquist rate. This set of samples can further be split into a set of time
sub-samples, shown as tss in the schematic, if desired. The kurtosis estimator Kn
is computed for each time and frequency sub-sample by dividing the fourth central
moment of the received field (m4 ) by the field’s second central moment (m2 ) squared
as given by Equation 5.17. For a sufficiently large number of field samples in the
computation, Kn can be modeled as a Gaussian random variable with known mean
and variance values as given in [26] (note that quantization effects were assumed to
be negligible in these formulas). In the presence of RFI, the mean kurtosis can either
become larger or smaller than its mean value in the absence of RFI (which is three),
hence a two-sided test is used as shown in the schematic. The overall probabilities
of detection and false alarm are then computed as the probability that the kurtosis
values of at least one of the time/frequency subsamples exceeds the threshold of the
two-sided test.
The process used here is based on that described in [32], in which a simplified
version of the pulsed sinusoidal RFI signal model is used. In this model, pulsed RFI is
assumed to occur in only one of the frequency sub-samples (i.e. no scalloping effects.)
196
Figure 6.3: Schematic of the kurtosis detector
It is also assumed that RFI pulses, when present, begin at the beginning of the
observations, and only one RFI pulse is present in each radiometer integration period
(i.e. Npulse is 1). To provide a fair comparison among the detectors, this simplified
signal model is used in the results shown whenever the three detection methods are
compared. See [32] for additional information on the kurtosis detector performance
model. This signal model also results in considerable savings of computational time,
since a numerical integration is required to model the complete signal model with the
scalloping loss where at least 2N frequency points and 20 phase points are required
to obtain accurate results (in comparison with Monte Carlo simulations).
The results to be shown use four time sub-samples for the kurtosis algorithm
throughout. Other numbers of time sub-samples were also examined (including no
time sub-sampling) and only modest changes in the results, particularly for higher
duty cycle interference, were observed.
197
6.3
Detection Performance Results
Detector performance is presented using receiver operating characteristic (ROC)
curves. A ROC curve is a plot of the probability of detection (Pd ) vs. the probability
of false alarm (Pf a ) as the threshold is changed. A good detector achieves a high probability of detection with a low false alarm rate. ROC curves of the cross-frequency,
pulse, and kurtosis detectors are plotted for the case of CW interference (i.e. 100
percent duty cycle) in Figure 6.4 with an 8 channel radiometer (or N = 16 for the
pulse detector). ROC curves corresponding to R = 1, R = 2 and R = 3 interference
power levels are shown for the cross-frequency detector, and for R = 3 only for the
pulse and kurtosis detectors. ROC curves for the pulse and kurtosis detectors are
almost a straight line indicating that these detectors are insensitive to CW RFI at
this power level. In contrast, the cross-frequency detector obtains good performance
at R = 3.
In Figure 6.5, simulations of Figure 6.4 are repeated with the assumption of a
pulsed sinusoidal RFI source with 50% duty cycle. Performance of the cross-frequency
detector is virtually same with the CW RFI case. This is due to the fact that, for fixed
R, the non-centrality parameters in the cross-frequency algorithm depend primarily
on the total RFI power contained within a radiometer measurement, regardless of the
duty cycle. Pulse and kurtosis detectors remain insensitive for this duty cycle.
Figure 6.6 compares ROC curves for all three detectors for an RFI strength of 1
and a duty cycle of 0.1%. Cross-frequency performance remains the same but this
time the other two detectors perform better with the kurtosis detector obtaining the
best performance.
198
ROC curve comparison, 8 channels, d=1
1
0.8
Pd
0.6
0.4
Cross−freq R=1
Cross−freq R=2
Cross−freq R=3
Pulse detection R=3
Sub−sampled Kurtosis R=3
0.2
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 6.4: ROC curves for the cross-frequency, pulse, and subsampled kurtosis algorithms: CW RFI, 8 channels (N = 16 for the pulse detector). R = 1, R = 2 and
R = 3 cases shown for the cross-frequency case, R = 3 only for the pulse and kurtosis
cases.
Although previous three ROC plots gave an idea of the performance of these
three detectors versus RFI strength and duty cycle, a more complete description of
the detection performance is desired. Following [32], a single “area under the curve”
(AUC) metric can be used to achieve this goal; this quantity corresponds to the
area under the ROC curve. As seen in Figure 6.4, in the worst case the probability
of detection is equal to the probability of false alarm, yielding an area under the
ROC curve of 0.5. The best case produces perfect detection for any false alarm rate,
yielding an area under the ROC curve of 1. The AUC is defined such that the AUC
in the worst case is equal to 0 and equal to unity in the best case; this definition
199
ROC curve comparison, 8 channels, d=0.5
1
0.8
Pd
0.6
0.4
Cross−freq R=1
Cross−freq R=2
Cross−freq R=3
Pulse detection R=3
Sub−sampled Kurtosis R=3
0.2
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 6.5: Same as Figure 6.4, but for RFI with 50% duty cycle.
requires subtracting 0.5 from the true area under the ROC curve and multiplying the
result by 2.
Figure 6.7 presents an image of AUC values for the cross-frequency detector using
four frequency channels (i.e. N = 8). The horizontal axis of the image is the ratio
R that describes the RFI strength, and the vertical axis is the duty cycle of pulsed
sinusoidal interference on a logarithmic scale. The logarithmic color scale is defined
so that “blue” areas indicate AUC values near unity while “red” areas correspond
to low AUC values where the detector is ineffective. The image makes clear that
the performance of the cross-frequency algorithm depends only weakly on the RFI
duty cycle. The results show that it is possible to obtain good detection performance
(AUC ≈ 0.95 or more) for R ≥ 2.6 at all duty cycle values.
200
ROC curve comparison, 8 channels, d=0.001
1
0.8
Pd
0.6
0.4
Cross−freq R=1
Pulse detection R=1
Sub−sampled Kurtosis R=1
0.2
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 6.6: ROC curves for the cross-frequency, pulse, and subsampled kurtosis algorithms: 0.1% duty cycle, R = 1, 8 channels (N = 16 for the pulse detector).
Results from the cross-frequency detector (upper) are compared with those of
the subsampled kurtosis (middle) and pulse (lower) detectors in Figure 6.8 ( four
frequency subchannels and four time subsamples for the kurtosis detector and an
N = 8 sample integration for the pulse detector.) Although the pulse and kurtosis
methods provide a performance improvement for RFI with low duty cycles compared
to the cross-frequency detector, they become insensitive to RFI at higher duty cycles.
Kurtosis only becomes advantageous compared to cross-frequency detector when duty
cycle gets less than 1% while pulse detector, as it might be expected, performs the
best for extremely low duty cycles (less than 0.05%). Similar conclusions are obtained
as the number of time sub-samples in the kurtosis algorithm is varied.
201
Figure 6.7: AUC image for the cross-frequency detector vs. RFI strength and duty
cycle, four channels
Figure 6.9 provides a similar AUC image comparison for a 16 channel radiometer
(N = 32). The results show improved detection performance for the cross-frequency
algorithm compared to N = 8, such that good performance is achieved at R ≥ 1.5,
while the performance of the sub-sampled kurtosis method is also improved compared
to N = 8 but remains insensitive to higher duty cycle RFI. A modest increase in
performance of the pulse detector can be observed as well due to the increase in
integration time.
6.3.1
Effects of scalloping loss
The results presented to this point have used the signal model of [32], which
assumes that the RFI is centered in a single frequency channel. To quantify the
202
Figure 6.8: AUC image comparison for the cross-frequency (upper), subsampled kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and duty cycle, four
channels/N = 8
impact of non-centered RFI frequencies (i.e. scalloping loss), plots of the probability
of detection versus the RFI strength are provided in Figure 6.10 for a constant Pf a
of 1%; note the inverted and logarithmic nature of the vertical axis in the Figure.
Curves using the signal model of [32] and of Section 6.2.1 are compared for 8 and 16
frequency channels in the case of continuous sinusoidal interference. The results show
detector performance to degrade when scalloping loss is included. For example, a 99
percent probability of detection is not achieved until R ≈ 3.3 with 16 channels when
scalloping loss is considered, versus R ≈ 2 neglecting scalloping loss. These variations
are significant, but the subsampled kurtosis algorithm would also be affected in a
203
Figure 6.9: AUC image comparison for the cross-frequency (upper), subsampled kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and duty cycle, sixteen
channels/N = 32
similar manner when non-centered frequencies are considered, so that the relationship
between the algorithms is not expected to be significantly impacted.
As a further inspection of scalloping loss effects, AUC images of the cross-frequency
detection algorithm with (upper) and without (lower) scalloping loss are provided in
Figure 6.11 for sixteen channels. It can again be seen that scalloping loss causes a
degradation in performance: the RFI strength needed for a good detection performance (AU C ≈ 0.95) increases from R ≈ 1.5 to R ≈ 2.2. However, good performance
is still achieved, and the behavior of the cross-frequency detector’s performance versus duty cycle and RFI strength is very similar in both cases. In general, the cross
frequency algorithm appears to be very useful for detecting pulsed sinusoidal RFI
204
Pfa=1%
0
P
d
8 channels, with scalloping loss
8 channels, bin−centered
16 channels, with scalloping loss
16 channels, bin−centered
0.9
0.99
0
0.5
1
1.5
2
2.5
RFI Strength in R
3
3.5
4
4.5
Figure 6.10: Probability of detection vs. RFI strength for a constant Pf a of 1%, CW
RFI. Results when RFI is assumed to be centered in a channel (“bin centered”) are
compared with the general case of random RFI frequency (“with scalloping loss”) for
8 and 16 frequency channels.
with high duty cycles, and use of such an algorithm (in addition to other approaches)
in any radiometer system having multiple frequency channels is recommended.
6.3.2
A simplified method for the calculation of scalloping
loss
While the approach described in Section 6.2.2 for the calculation of scalloping
loss is general, the required numerical integrations can be computationally expensive
(although reasonable for desktop level computing resources in time scales of minutes
to hours.) Later analyses showed that a simplified approach without the complete
205
Figure 6.11: AUC image comparison for the cross-frequency algorithm with (upper) and without (lower) scalloping loss vs. RFI strength and duty cycle, sixteen
channels/N = 32
averaging procedure could yield predictions practically identical to those of the complete computation. This approach models the RFI as appearing in only two of the
FFT outputs Y [k] while the remainder are RFI free. The impact of scalloping loss
is considered by modeling the two corrupted frequencies as non-central chi-squared
random variables with 2I degrees of freedom; the non-centrality parameter is determined by pulsed RFI properties, and performance is averaged over a splitting of
the RFI power across two adjacent channels using weights determined by a channel
filter model. The channel filter used was that of the rectangular FFT window. It
is possible that other channel filters (or FFT windows) may also be describable in
the same way. However, note that when a window other than rectangular is used in
206
performing the FFT, the frequency outputs Xi [k] become correlated even in the RFI
free case, making the analysis much more difficult.
A comparison of this approximation and a complete solution of scalloping loss is
given in Figure 6.12. This figure is a plot of detection probability vs. RFI strength
for 8 and 16 frequency channels. The difference between the approximation and complete solution is negligible; thus, it could be concluded that this approximation can
be helpful to describe the effect of scalloping in similar simulations where power is
divided between frequency channels and computational efficiency is required. However, as mentioned above, computational time required for the complete solution was
reasonable of the purposes of this study and this approximation was not employed in
the results shown.
6.4
System Temperature Estimation Issues
The previous results have assumed a fixed threshold Tcross is used in the crossfrequency detection algorithm, corresponding to a fixed variance of the radiometer
thermal noise. In reality, the variance of the radiometer observed thermal noise can
vary significantly with the geophysical scene observed, so that some estimate of the
current thermal noise variance is required in order to maintain a known relationship
between the threshold value and the detector probability of false alarm. One method
for estimating this variance involves taking the mean of the observed brightnesses
versus frequency, excluding a specified number of the largest brightnesses in the computation of the mean. Detection can then be declared if any of the frequency channels
exceed some function of this system temperature estimate by a specified threshold.
207
Pfa=1%
0
P
d
8 channels, approximation
8 channels, complete solution
16 channels, approximation
16 channels, complete solution
0.9
0.99
0
0.5
1
1.5
2
2.5
3
RFI Strength in R
3.5
4
4.5
5
Figure 6.12: Probability of detection vs. RFI strength for a constant Pf a of 1%, CW
RFI. Results obtained with an approximation to the scalloping loss is compared with
the complete solution for 8 and 16 frequency channels.
To model the estimation process, consider a radiometer having M channels (M =
N/2 in the notation used) so that the cross frequency blanker considers M independent chi-squared random variables with 2I degrees of freedom in the RFI free case.
The estimation procedure requires sorting these measurements, so that M sorted measurements are obtained, and the largest Mdrop values are excluded in the computation
of the system temperature estimate.
One quantity of interest is the probability density function of the system temperature estimate, which is the mean of the smallest M − Mdrop of the chi-squared
random variables. This pdf in theory could be obtained using properties of order
statistics, but in practice is very difficult to evaluate since the computation involves
208
an M − Mdrop dimensional space. To simplify the computation, it is assumed that
the pdf of the system temperature estimate can be modeled as a Gaussian random
variable; this is likely to be reasonable so long as 2I and M − Mdrop are not small.
Once this approximation is applied, knowledge is required only of the mean and variance of the system temperature estimate. This mean and variance can be obtained
by combining means and covariances of the lower M − Mdrop order statistics; the
required means and covariances of these order statistics can be obtained using the
one and two point order statistic pdfs in [77]. It is also assumed that the pdf of
the system temperature estimate so obtained is applicable both in the presence and
absence of RFI; this is equivalent to assuming that the RFI lies entirely within the
discarded Mdrop largest channels.
As an illustration of this procedure, Figure 6.13 plots the mean and standard
deviation of the system temperature estimate with respect to the number of channels
discarded for the 4, 8, and 16 channel cases. Here it is assumed that the mean value of
Tsys is 590 K and its standard deviation is 0.95 K (corresponding to Q = 768000 from
Section 6.2.1) so that the results shown are in units of Kelvin. The negative bias in the
mean system temperature estimate is less than 1.5 K if 2 channels are discarded even
for a four channel system, and in addition this bias in the mean can be corrected
since it is fixed given the number of channels discarded. The dependence of the
standard deviation of the system temperature estimate on the number of channels
discarded is also modest, confirming that the effect of discarding channels in the
system temperature estimation should be small, especially as the number of channels
becomes larger.
To incorporate the system temperature estimation process into the detector performance model, the difference between the maximum brightness temperature (TM AX )
209
3
Change in mean/standard deviation (Kelvin)
2
1
0
−1
−2
−3
−4
Change in mean, 4 channels
Change in mean, 8 channels
Change in mean, 16 channels
Change in std, 4 channels
Change in std, 8 channels
Change in std, 16 channels
−5
−6
−7
0
2
4
6
8
10
Number of channels discarded
12
14
16
Figure 6.13: Mean and standard deviation of the system temperature estimate as
channels are discarded in the threshold estimation procedure of the cross-frequency
algorithm, for 4, 8, and 16 frequency channels.
and the mean of the smaller M − Mdrop frequency channels (TM EAN ) can be taken to
be a single random variable and compared with a fixed threshold. However, to model
this detector analytically, the joint pdf of TM AX and TM EAN must be known, but
this quantity is difficult to compute. One possible approach involves approximating
TM AX and TM EAN as independent, but since TM AX is always greater than TM EAN ,
this assumption is problematic.
An alternate approach creates a new random variable
TN = cTM AX + TM EAN .
210
(6.2)
and defines c so that the correlation between TN and TM AX is zero:
c=
−cov(TM AX , TM EAN )
var(TM AX )
(6.3)
Here, cov and var represent the covariance and variance operations respectively. The
new detection rule is to compare the difference between TM AX and TN to a threshold:
P (TM AX − TN > ∆TCROSS )
(6.4)
where P denotes the probability and ∆TCROSS is a fixed threshold. TM AX and TN
are now assumed to be independent, so that the pdf of TM AX − TN can be obtained
analytically. Note that while TN and TM AX are uncorrelated, they are clearly not
independent, however the assumption that TN and TM AX are independent appears to
be less restrictive than assuming that TM EAN and TM AX are independent.
Using the assumption of independence, the pdf of the difference random variable
can be evaluated as a convolution between the original pdf of the maximum brightness
temperature and a scaled version of the Gaussian pdf of the system temperature
estimate. The resulting random variable is then compared with a scaled version of
the original threshold.
To assess these approximations, Figure 6.14 compares ROC curves obtained with
this method to Monte Carlo simulations that include system temperature estimation
effects without approximation. CW RFI having R = 3 is considered, and the estimation procedure discards the 2 channels with the highest brightnesses. Comparisons
are shown for 4, 8, and 16 channels, and 16384 realizations were used in the Monte
Carlo simulation. The comparisons show that the accuracy of the approximation improves as the number of channels increases, and that the approximation is reasonable
for 8 channels and very good for 16 channels. Other simulations also confirmed that
211
results from this procedure can be used without a significant loss in accuracy if the
number of frequency channels is greater than or equal to 8.
R=3, 2 dropped channels
1
0.9
0.8
0.7
Pd
0.6
0.5
0.4
Approximation, 4 ch.
Approximation, 8 ch.
Approximation, 16 ch.
Monte Carlo, 4 ch.
Monte Carlo, 8 ch.
Monte Carlo, 16 ch.
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
Pfa
0.6
0.7
0.8
0.9
1
Figure 6.14: ROC curves obtained using the analytical approximation to threshold
estimation effects compared with results from Monte Carlo simulations. CW RFI
having R = 3, Mdrop = 2, and for 4,8 and 16 channels.
To show the effect of the system temperature estimation on detection performance,
the probability of detection versus CW RFI strength is plotted in Figure 6.15 for
Pf a = 1% when the system temperature is estimated with 2 channels discarded in
the 8 and 16 channel cases using the approximate model. For comparison, results
from Figure 6.10 when the system temperature is exactly known are also included.
These results confirm that estimation of the system temperature causes only a modest
degradation in detection performance for high duty cycle sinusoidal interference.
212
Pfa=1%
0
P
d
8 channels, no est. effects
8 channels, with est. effects
16 channels, no est. effects
16 channels, with est. effects
0.9
0.99
0
0.5
1
1.5
2
2.5
3
RFI Strength in R
3.5
4
4.5
5
Figure 6.15: Results of Figure 6.10 are compared with curves including system temperature estimation effects. CW RFI
6.5
Summary and Remarks
The performance of a cross-frequency detection algorithm was analyzed for pulsed
sinusoidal RFI and compared with that achieved by the pulse and sub-sampled kurtosis algorithms in this chapter. It was shown that the performance of the crossfrequency detector improves as the number of frequency channels increases, and that
its performance is only weakly sensitive to the RFI duty cycle at a fixed R RFI power
level. A reasonable sensitivity to pulsed sinusoidal RFI is achieved even with only
four radiometer channels. The performance of the pulse and sub-sampled kurtosis
methods was found to exceed that of the cross-frequency algorithm for low duty cycle
213
pulsed RFI, but the cross-frequency method offers significant performance improvements over these methods for higher duty cycle pulses including CW interference. A
method for modeling the impact of system temperature estimation effects on detector
performance was also provided, and it was shown that system temperature estimation
is expected to result in only a modest degradation for pulsed sinusoidal RFI sources.
A simple way to model scalloping effects was discussed as well.
It should be noted that, since pulsed sinusoidal RFI sources are localized in frequency by definition, it is not surprising that the cross-frequency detector performs
very well against this class of RFI. Because this class of RFI, especially the CW case
which represents any very narrowband emission, is expected to be encountered in
Earth observations, use of cross-frequency detection algorithms appears to be warranted in future Earth observing radiometer systems. Such use is compatible with
other algorithms such as the sub-sampled kurtosis or pulse methods that can provide
enhanced performance for low duty cycle pulses and/or other RFI source types.
214
CHAPTER 7
CONCLUSIONS
Several surveys on the severity of RFI at microwave frequencies have been published in the last ten years. Results suggested that RFI might cause a significant
degradation in the accuracy of the microwave radiometric observations. This thesis
documented studies of the RFI environment and of RFI mitigation methods with the
purpose of reducing the detrimental effects of the RFI problem. Contributions of this
research can be listed as follows:
• Using LISR and CISR digital radiometers that possess high spectral and temporal resolution, time and frequency properties of the existing RFI sources at
L- and C-band were investigated.
• Joint airborne and groundborne campaigns were conducted with other microwave remote sensing groups to intercompare the measurements of different
sensors. LISR/CISR have served as a ground-truthing device for the other radiometers participating in the campaigns. These experiments were devised in
part to test radiometric designs that are in consideration for future space based
microwave remote sensing missions like the Soil Moisture Active and Passive
(SMAP) mission of NASA. As a consequence, results of the campaigns have
provided a reference for the science teams of these radiometric missions.
215
• Effectiveness of the Asynchronous Pulse Blanker (APB) system in a practical setting was demonstrated with observations in the vicinity of an air route
surveillance radar.
• Pulse detection and cross-frequency detection algorithms that make use of the
high temporal and frequency resolution of the LISR/CISR sensors were developed and applied to the data obtained from the aforementioned campaigns in
post-processing. Considerable success was achieved in mitigating real world RFI
even against sources on the order of natural geophysical variations. Although
a system similar to LISR/CISR that operates in space and downlinks data at
such a high resolution is unlikely, proficiency of these methods in mitigating
RFI as well as the successful implementation of a real time RFI mitigation system in APB showed that it might be possible to have real time RFI mitigation
onboard the satellite and to downlink the clean data after further integration.
Downlinking a single flag that shows the presence/absence of RFI in a radiometer integration period was suggested as another alternative since it might not
be desirable to throw data out automatically.
• Shapiro-Wilk test of normality was proposed as a novel RFI mitigation technique. Results were compared to the kurtosis technique for the case of pulsed
sinusoidal RFI. It was shown that the Shapiro-Wilk test did not suffer from the
blind spot observed by the kurtosis algorithm. However, both the kurtosis and
the Shapiro-Wilk tests performed poorly against CW RFI. The Shapiro-Wilk
test was found to be suitable for implementation in a radiometric system. Considering the more omnibus character of this test cited in the statistical literature
216
compared to the kurtosis test, such an implementation of the Shapiro-Wilk test
in hardware might be a worthwhile endeavour.
• A theoretical comparison of detection performance was given between crossfrequency detection, pulse detection and kurtosis detection methods for the
case of pulsed sinusoidal RFI. Emphasis was on the cross-frequency detection for
this study and a very simple cross-frequency detection model that is suitable for
implementation in hardware was devised. It was shown that the cross-frequency
algorithm maintains a high (and almost constant) level of performance for this
type of RFI regardless of duty cycle while the other methods performed poorly
for the high duty cycle or CW RFI cases. Due to the abundance of narrowband
RFI (as also evidenced by experimental results of this thesis) and the relative
ease of implementing the cross-frequency algorithm in a multiband radiometer,
it was suggested that the cross-frequency algorithm should be considered in
radiometers with RFI suppression capability; possibly as an addition to a more
general method like kurtosis.
A summary of the thesis is as follows: In Chapter 1, a very brief overview of the
microwave radiometry is given. The severity of the RFI problem was described and
strategies used in literature for RFI mitigation were introduced.
Chapter 2 described an L-band groundborne campaign conducted in Canton,
Michigan. A review of the front-end and LISR back-ends were provided and solutions of several practical problems encountered in this campaign were discussed.
The capabilities of the LISR system at detecting and removing pulsed interference
in real-time were demonstrated. As a result of these capabilities, LISR system was
able to continue measurements even in the presence of very high levels of RFI. More
217
complicated algorithms that make use of the high spectral and temporal resolution
of LISR were also developed to mitigate RFI in post-processing .
Another L-band expedition was discussed in Chapter 3. This campaign was performed at the Jet Propulsion Laboratory (JPL) of NASA in Pasadena, California.
The durations of the observational states (e.g. antenna state) were very short in
this experiment which necessitated changes in the LISR design. Practical problems
encountered (like the noise in the state trigger channel) and their solutions were
described. Presence of RFI sources in the protected portion of the spectrum at Lband was shown using the high spectral resolution of the LISR system. These RFI
sources were successfully removed using pulse detection and cross-frequency detection
techniques in post-processing. Comparisons of LISR data with those measured by
Passive/Active L-S band (PALS) sensor of JPL and Agile Digital Detector (ADD)
sensor of University of Michigan were made. All three sensors showed periodic interference with RFI level closely correlated among sensors.
An airborne campaign at C-band was described in Chapter 4. CISR backend was
mounted to the WB-57 high altitude aircraft of NASA in this experiment. Polarimetric Scanning Radiometer (PSR) of University of Colorado provided the front-end
for the CISR and the ADD backends. Comparisons between the measurements of
these three sensors were provided. Flight path of this expedition included overflights
of major Texas cities, observations of the rural parts of Texas, and the observations
of the Gulf of Mexico. This variety in the observed RFI environments allowed a
demonstration of the CISR ’s capabilites in mitigating RFI with diverse amplitude
levels as well as distinct temporal and spectral characteristics.
In Chapter 5, possible use of the Shapiro-Wilk test as a detector of RFI was
analyzed. An overview of the Shapiro-Wilk test was given. As mentioned above,
218
detection performance was compared with that of the kurtosis test for the detection
of pulsed sinusoidal RFI. Feasibility of the Shapiro-Wilk test’s implementation in
hardware was discussed. Results suggested that the Shapiro-Wilk test is a viable
detector alternative for the RFI suppressing radiometer systems.
Chapter 6 provided detection performance comparisons between cross-frequency
detection, pulse detection, and kurtosis detection methods for the case of pulsed
sinusoidal RFI. Particular detection models used in this study were described. The
probabilities of detction and false alarm for each detector were given mathematically
for this particular type of RFI. As mentioned among the list of contributions, it was
shown that the cross-frequency method can sucessfully detect pulsed sinusoidal RFI
regardless of duty cycle.
Future research on this field should include further field experiments to better
understand the existing RFI environment and to quantify the effectiveness of RFI detection strategies against different RFI sources. Geolocating RFI sources and trying
to obtain a match with the known source databases might provide additional information that might be useful in removing RFI from earth remote sensing observations.
Tests of radiometers employing novel RFI detection strategies will likely continue in
the future. Shapiro-Wilk test or the simple cross-frequency blanking model described
in this thesis are candidates for implementation in hardware. Theoretical studies on
RFI detection techniques should also continue. Studies so far have focused on the case
of pulsed sinusoidal RFI. Simulations using models of different types of RFI sources
would enhance our knowledge about the capabilities and weaknesses of different RFI
detection strategies.
In conclusion, it can be said that although the history of the satellite microwave
remote sensing dates back to the late 1960s, the ramifications of RFI on the accuracy
219
of the obtained measurements have only been understood in this decade. There was a
significant need for learning more about the existing RFI environment, for developing
simple and effective techniques for RFI detection, and for analyzing and comparing
the performance of these techniques against different types of RFI sources. The studies described here contributed to the development of this relatively new research area
and provided a reference for future planned microwave satellite radiometry missions.
220
BIBLIOGRAPHY
[1] F. T. Ulaby, R. K. Moore, and A. K. Fung. The advanced theory of statistics,
Vol. 1. Artech House, Norwood, MA, 1981.
[2] J. R. Wang and B. J. Choudhury. Remote sensing of soil moisture content over
bare field at 1.4 GHz frequency. Journal of geophysical research, 86(C6):5277–
5282, June 1981.
[3] E. G. Njoku. Passive microwave remote sensing of the earth from space-a review.
Proceedings of the IEEE, 70(7):728–750, July 1982.
[4] R. H. Dicke, R. Beringer, R. L. Kyhl, and A. B. Vane. Atmospheric absorption
measurements with a microwave radiometer. Phy. Rev., 70(5-6):340, September
1946.
[5] A. E. Basharinov, A. S. Gurvich, S. T. Yegorov, A. A. Kurskaya, D. T. Matveyev,
and A. M. Shutko. The results of microwave sounding of the earth’s surface
according to experimental data from the satellite Cosmos 243. In Space Research
XI, Berlin, Germany, 1971.
[6] P. Gloersen and F. T. Barath. A scanning multichannel microwave radiometer
for Nimbus-G and Seasat-A. IEEE Journal of Ocean Engineering, 2:172–178,
1977.
[7] AMSR/AMSR-E home page.
index_ e. htm .
In http: // sharaku. eorc. jaxa. jp/ AMSR/
[8] A. Shibata, K. Imaoka, M. Kachi, and Y. Fujimoto. AMSR/AMSR-E sea surface
temperature algorithm development. In IEEE Geoscience and Remote Sensing
Symposium, volume 1, pages 672–673, July 2003.
[9] WindSat homepage. In http: // www. nrl. navy. mil/ WindSat/ index. php .
[10] D. Le Vine, C. Koblinsky, and F. Pellerano. The measurement of salinity from
space: sensor concept. In IEEE Geoscience and Remote Sensing Symposium,
volume 3, pages 1010–1012, July 2001.
221
[11] J. Font, G. S. E. Lagerloef, D. Le Vine, and A. Camps. Open issues for the
soil moisture and ocean salinity (SMOS) satellite mission salinity retrieval. In
Proceedings of SMOS Campaigns Workshop, pages ESA SP–525: 7–13, 2003.
[12] SMAP website. In http: // smap. jpl. nasa. gov/ .
[13] A. J. Camps, I. Corbella, F. Torres, J. Bara, and J. Capdevila. RF interference
analysis in aperture synthesis interferometric radiometers: application to L-Band
MIRAS instrument. IEEE Transactions on Geoscience and Remote Sensing,
38(2):942–950, March 2000.
[14] D. M. LeVine. ESTAR experience with RFI at L-band and implications for
future passive microwave remote sensing from space. In IEEE Geoscience and
Remote Sensing Symposium, volume 3, pages 847–849, 2002.
[15] L. Li, E. G. Njoku, E. Im, P. S. Chang, and K. M. St.Germain. A preliminary
survey of radio-frequency interference over the US in Aqua AMSR-E data. IEEE
Transactions on Geoscience and Remote Sensing, 42:380–390, 2004.
[16] J. T. Johnson, A. J. Gasiewski, B. Güner, G. A. Hampson, S. W. Ellingson,
R. Krishnamachari, N. Niamsuwan, E. McIntyre, M. Klein, and V. Leuski. Airborne radio frequency interference studies at C-band using a digital receiver.
IEEE Transactions on Geoscience and Remote Sensing, 44(7):1974–1985, July
2006.
[17] S. W. Ellingson and J. T. Johnson. A polarimetric survey of radio frequency
interference in C- and X-bands in the continental United States using WindSAT
radiometry. IEEE Transactions on Geoscience and Remote Sensing, 44:540–548,
2006.
[18] A. J. Gasiewski, M. Klein, A. Yevgrafov, and V. Leuski. Interference mitigation in passive microwave radiometry. In IEEE Geoscience and Remote Sensing
Symposium, volume 3, pages 1682–1684, June 2002.
[19] M. Basseville and I. V. Nikiforov. Detection of abrupt changes: theory and
applications. Prentice-Hall, Englewood Cliffs, NJ, 1993.
[20] P. A. Fridman and W. A. Baan. RFI mitigation methods in radio astronomy.
Astronomy and Astrophysics, 378:327–344, 2001.
[21] G. A. Hampson, S. W. Ellingson, and J. T. Johnson. Design and demonstration of an interference suppressing microwave radiometer. In IEEE Aerospace
Conference, volume 2, pages 993–999, March 2004.
[22] N. Niamsuwan, J. T. Johnson, and S. W. Ellingson. Examination of a simple
pulse blanking technique for RFI mitigation. Radio Science, 40, June 2005.
222
[23] B. Güner, J. T. Johnson, and N. Niamsuwan. Time and frequency blanking for
radio frequency interference mitigation in microwave radiometry. IEEE Transactions on Geoscience and Remote Sensing, 45(11):3672–3679, November 2007.
[24] C. S. Ruf, S. M. Gross, and S. Misra. RFI detection and mitigation for microwave
radiometry with an agile digital detector. IEEE Transactions on Geoscience and
Remote Sensing, 44(3):694–706, March 2006.
[25] S. Misra, S. S. Kristensen, S. S. Sobjaerg, and N. Skou. CoSMOS: Performance of
kurtosis algorithm for radio frequency interference detection and mitigation. In
IEEE Geoscience and Remote Sensing Symposium, number 23-28, pages 2714–
2717, July 2007.
[26] R. D. De Roo, S. Misra, and C. S. Ruf. Sensitivity of the kurtosis statistics
as a detector of pulsed sinusoidal RFI. IEEE Transactions on Geoscience and
Remote Sensing, 45(7):1938–1946, July 2007.
[27] J. R. Piepmeier, P. Mohammed, and J. Knuble. A Double Detector for RFI
Mitigation in Microwave Radiometers. IEEE Transactions on Geoscience and
Remote Sensing, 46(2):458–465, February 2008.
[28] R. Weber, C. Faye, F. Biraud, and J. Dansou. Spectral detector for interference
time blanking using quantized correlator. Astronomy and Astrophysics Supp.,
126:161–167, 1997.
[29] C. S. Ruf and S. Misra. Detection of radio frequency interference with the
Aquarius radiometer. In IEEE International Geoscience and Remote Sensing
Symposium, pages 2722–2725, July 2007.
[30] G. M. Nita, D. E. Gary, Z. Liu, G. J. Hurford, and S. M. White. Radio frequency
interference excision using spectral-domain statistics. The Publications of the
Astronomical Society of the Pacific, 119(857), July 2007.
[31] S. Misra and C. Ruf. Comparison of pulsed sinusoidal RFI detection algorithms
using time and frequency sub-sampling. In IEEE Geoscience and Remote Sensing
Symposium, 2008.
[32] S. Misra, P. N. Mohammed, B. Güner, C. Ruf, J. R. Piepmeier, and J. T. Johnson. Microwave radiometer radio frequency interference detection and mitigation
algorithms: a comparative study. submitted to IEEE Transactions on Geoscience
and Remote Sensing, 2008.
[33] R. DeRoo and S. Misra. Effectiveness of the sixth moment to eliminate a kurtosis
blind spot in the detection of interference in a radiometer. In IEEE Geoscience
and Remote Sensing Symposium, 2008.
223
[34] B. Güner, M. Frankford, and J. T. Johnson. Comparison of three omnibus
normality tests for the detection of pulsed sinusoidal RFI sources. In URSI
General Assembly, 2008.
[35] M. T. Frankford B. Güner and J. T. Johnson. A study of the Shapiro-Wilk
test for the detection of pulsed sinusoidal radio frequency interference. IEEE
Transactions on Geoscience and Remote Sensing, accepted for publication, 2009.
[36] National Research Council of the National Academies. Handbook of frequency
allocations and spectrum protection for scientific uses. The National Academies
Press, Washington DC, 2007.
[37] E. G. Njoku, P. Ashcroft, T. K. Chan, and L. Li. Global survey and statistics of
radio-frequency interference in AMSR-E land observations. IEEE Transactions
on Geoscience and Remote Sensing, 43(5):938–947, May 2005.
[38] S. W. Ellingson, J. D. Bunton, and J. F. Bell. Removal of the GLON ASS C/A
signal from OH spectral line observations using a parametric modeling technique.
The Astrophysical Journal Supplement Series, 135:87–93, July 2001.
[39] C. Barnbaum and R. F. Bradley. A new approach to interference excision in
radio astronomy: real-time adaptive cancellation. The Astronomical Journal,
115:2598–2614, 1998.
[40] P. Mohammed, J. Knuble, and J. Piepmeier. Analog radio-frequency interference
(RFI) suppression system for microwave radiometers. In American Geophysical
Union 2006 Fall Meeting, 2006.
[41] S. W. Ellingson and G. A. Hampson. Mitigation of radar interference in L-band
radio astronomy. The Astrophysical Journal Supplement Series, 147(1):167–176,
2003.
[42] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Characterization of L-band
RFI and implications for mitigation techniques. In IEEE Geoscience and Remote
Sensing Symposium, volume 3, pages 1745–1747, Toulouse, France, July 2003.
[43] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Design of an L-band
microwave radiometer with active mitigation of interference. In IEEE Geoscience
and Remote Sensing Symposium, volume 3, pages 1751–1753, July 2003.
[44] G. A. Hampson, S. W. Ellingson, and J. T. Johnson. Design of an L-band
microwave radiometer with active mitigation of interference. In IEEE Antennas
and Propagation/URSI Symposium, 2003.
[45] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Design of an L-band
microwave radiometer with active mitigation of interference. In NASA Earth
Science Technology Conference, 2003.
224
[46] J. T. Johnson, G. A. Hampson, and S. W. Ellingson. Design and demonstration
of an interference suppressing microwave radiometer. In IEEE Geoscience and
Remote Sensing Symposium, volume 3, pages 1683–1686, September 2004.
[47] N. Niamsuwan, J. T. Johnson, and S. W. Ellingson. Examination of a simple
pulse blanking technique for RFI mitigation. In Workshop in Mitigation of Radio
Frequency Interference in Radio Astronomy, 2004.
[48] J. T. Johnson, A. J. Gasiewski, G. A. Hampson, S. W. Ellingson, R. Krishnamarachi, and M. Klein. Airborne radio frequency interference studies at C-band
using a digital receiver. In IEEE Geoscience and Remote Sensing Symposium,
volume 3, pages 1687–1690, September 2004.
[49] N. Niamsuwan and J. T. Johnson. Sky observations at L-band using an interference suppressing radiometer. In IEEE Geoscience and Remote Sensing
Symposium, volume 8, pages 5562–5565, July 2005.
[50] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Digital receiver with interference suppression for microwave radiometry. In NASA Tech Brief GSC14776-1,
February 2005.
[51] S. Yueh, W. Wilson, E. Njoku, S. Dinardo, D. Hunter, Y. Rahmat-Samii, K. S.
Kona, and M. Manteghi. Compact, lightweight dual-frequency microstrip antenna feed for future soil moisture and sea surface salinity missions. In NASA
Earth-Sun System Technology Conference 2006, 2006.
[52] E. Njoku, W. Wilson, S. Yueh, S. Dinardo, T. Jackson, and V. Lakshmi. PALS
observations of soil moisture in SMEX02. In IEEE Geoscience and Remote
Sensing Symposium, 2003.
[53] MA-COM MD-169 mixer datasheet.
DataSheets/ MD_ MDC_ MDS-169. pdf .
In
http: // www. macom. com/
[54] E. McIntyre, A. J. Gasiewski, M. Klein, V. Leuski, B. L. Weber, and V. Irisov.
Experimental results from a spectral scanning technique for passive remote sensing of soil moisture. In U. S. National Radio Science Meeting, 2006.
[55] Federal Communications Commission Universal Licensing System search page.
In http: // wireless2. fcc. gov/ UlsApp/ UlsSearch/ searchLicense. jsp .
[56] R. Fisher. The moments of the distribution for normal samples of measures of
departures from normality. Proc. Royal Soc. A, 130:16–28, 1930.
[57] E. Pearson. A further development of tests for normality. Biometrika, 22:239–
249, 1930.
225
[58] S. S. Shapiro and M. B. Wilk. An analysis of variance test for normality (complete
samples). Biometrika, 52:591–611, 1965.
[59] S. S. Shapiro, M. B. Wilk, and H. J. Chen. A comparative study of various tests
for normality. Journal of the American Statistical Association, 63:1343–1372,
1968.
[60] E. Seier. Comparison of tests for univariate normality. Interstat Stat. Journal,
2002.
[61] F. F. Gan and K. J. Koehler. Goodness of Fit Tests Based on P-P Probability
Plots. Technometrics, 32:289–303, 1990.
[62] S. S. Shapiro and R. S. Francia. Approximate analysis of variance test for normality. J. Am. Statist. Assoc., 67:215–225, 1972.
[63] L. Chen and S. S. Shapiro. An Alternative test for normality based on normalized
spacings. J. of Stat. Comput. and Sim., 53:269–287, 1995.
[64] C. Jarque and A. Bera. Efficient tests for nonlinearity, heteroscedasticity, and
serial independence of regression residuals. Economics Letters, 6:255–259, 1980.
[65] A. Bera and C. Jarque. A test for normality of observations and regression
residuals. Int’l. Stat. Rev., 55:163–172, 1987.
[66] P. N. Mohammed, J. Knuble, and J. Piepmeier. Pulse and kurtosis detection
of radio-frequency interference (RFI): an experimental comparison. In IEEE
Geoscience and Remote Sensing Symposium, 2008.
[67] J. P. Royston. Approximating the Shapiro-Wilk W-test for non-normality. Statistics and Computing, 2:117–119, 1992.
[68] G. Blom. Statistical estimates and transformed Beta-variables. John Wiley and
Sons, New York, 1958.
[69] H. L. Harter. Expected values of normal order statistics. Biometrika, 48:151–165,
1961.
[70] J. Harkins, T. El-Ghazawi, E. El-Araby, and M. Huang. Performance and analysis of sorting algorithms on the SRC 6 reconfigurable computer. IEEE Field
Programmable Technology Con. Proc., 48:295–296, 2005.
[71] J. P. Royston. Correcting the Shapiro-Wilk W for ties. Journal of Computation
and Simulation, 31:237–249, 1989.
[72] J. P. Royston. An extension of Shapiro and Wilk’s W test for normality to large
samples. Applied Statistics, 31:115–124, 1982.
226
[73] J. P. Royston. A toolkit for testing for non-normality in complete and censored
samples. The Statistician, 42:37–43, 1993.
[74] J. T. Johnson and L. C. Potter. Performance study of algorithms for detecting
pulsed sinusoidal interference in microwave radiometry. IEEE Transactions on
Geoscience and Remote Sensing, 47(2):628–636, February 2009.
[75] S. M. Kay. Fundamentals of Statistical Signal Processing: Volume II, Detection
Theory. Prentice Hall, Upper Saddle Creek, NJ, 1998.
[76] B. Güner and J. T. Johnson. Performance study of a cross-frequency detection
algorithm for pulsed sinusoidal RFI in microwave radiometry. submitted to IEEE
Transactions on Geoscience and Remote Sensing, 2009.
[77] M. G. Kendall and A. Stuart. The advanced theory of statistics. Hafner, New
York, 1969.
227