Qu i ck Ti m e ™ an d a
YU V4 20 co de c de co m p re ss
ar e n ee de d t o s ee t h is pi ct ur
SHARC-II Data reduction
Workshop
Darren Dowell
Attila Kovacs
Colin Borys
Darek Lis
Min Yang
Jon Bird
SHARC-II DRW 11/08/2004
Outline



GOALS
Caltech success stories
Software Requirements and Installation





CRUSH vs SHARCSOLVE
General SHARC-II Calibration




Aperture Photometry
PSF Photometry
Example
Map tweaks and presentation






Opacity (tau) estimation
Calibrators at 350 micron
Stability of calibration
Calibrating your data




Overview of available software
Installation overview
Scripting with CRUSH
Pointing, Calibration, Coadding, Cropping, and Mosaicing
Making publishable images with IDL
Chopped data
Lessons learned
Tips for taking better data
Miscellaneous notes
SHARC-II DRW 11/08/2004
Goals
1. Transfer expertise from Caltech to other users
Why? To promote publication of SHARC-II data
2. Transfer expertise from other users to Caltech
Why? Users tend to be familiar with the issues involved.
3. Learn about difficulties users have with instrument/data
Why? To improve the system.
4. Improve data acquisition techniques
Why? Improve efficiency of SHARC-II observations
SHARC-II DRW 11/08/2004
Caltech Success Stories
The next few slides present results
from observations conducted by
members of the Caltech SHARC-II
group. They span a variety of flux
levels and redshifts, and are meant
to illustrate the full range of SHARCII’s abilities.
SHARC-II DRW 11/08/2004
SLUGS (z<0.05)
•
•
•
•
•
•
•
•
SHARC-II DRW 11/08/2004
Dunne et al. have
characterized the SED of 106
IRAS selected galaxies at
850m
Of those, only 17 were
detected by SCUBA at 450
m, and it was noted that the
data supported a 2component SED fit.
SHARC-II has detected
roughly 60/65 targeted so far.
They are so easy to detect
that they now are done as
poor-weather backup.
0.5h @ 225~0.06
1.0h @ 225~0.08
0.5 Jy< S350 < 3 Jy
crush -faint -compact
Spitzer HLIRG (z~1.5)
•
In follow-up observations of Spitzer
selected objects, we discovered an
object with an apparent luminosity
above 1013.5 L.
It has an SED similar to Arp220, but at
at a redshift of 1.5.
This object has sparked interest in
“Silicate Dropouts” as a way to select
high-z starbursts.
•
•
•
•
•
•
SHARC-II DRW 11/08/2004
0.5h @ 225~0.06
S350 = 226 ± 45 mJy
CHOPPED observing
sharcsolve reduction
Stanford sample (0.1<z<1.0)
2h @ 225~0.05
S350 = 44.1 mJy
Td = 40.9 K ,  = 1.5
•
•
•
The Stanford sample was compiled from cross-correlation of the faint-IRAS catalog and
the FIRST 21cm radio catalog
The sources are ULIRG’s lying within redshift range of 0.1 and 1; NIR morphologies of
these objects reveal they tend to be interacting systems
FIR/submillimeter fluxes were obtained for the first time on these targets, so were SED
fits in the FIR
SHARC-II DRW 11/08/2004
Fomalhaut Debris Disk
•
•
•
3.0h @ 225~0.039
SWEEP (Lissajous)
crush -deep reduction
•
Peak fluxes: 150
mJy/beam
Integrated flux: 1.2 Jy
Consistent with thin,
uniform dust ring
•
•
K. Marsh et al. (2004-5)
SHARC-II DRW 11/08/2004
Orion
Houde et al. (2004)
Johnstone & Bally (1999)
rms: 0.3
Jy/beam
rms: 0.3
Jy/beam
rms: 1
Jy/beam
rms: 0.3
Jy/beam
4 hr.
18 hr.
1.2 mm
PWV
mosaic of BOX scans
SHARC-II DRW 11/08/2004
1.0 mm
PWV
Low-z Interacting Galaxy Survey
•
•
•
•
Selected by IRAS 100 mm flux
and proximity using criterion of
Surace (2004).
Perhaps analogues to z>1
ULIRGs.
Lissajous scans in typically
t225×airmass = 0.05-0.10.
Observations of 14/42 sources
complete, to be reported by J.
Bird and D. Dowell.
5×1010 Lsolar
SHARC-II DRW 11/08/2004
Chopped High-z survey
•
•
•
•
•
•
SHARC-II DRW 11/08/2004
CHOP in azimuth 39″/1.39 Hz
plus slow SWEEP
parallactic angle rotation of
77° washes out negative
beams in sharcsolve
reduction
8.6 hrs, median tau225 = 0.044
rms = 10 mJy/beam in middle
2 sources are 80 mJy each
Definite and probable SCUBA
850 m sources marked with
stars
High-z SCUBA sources
•
•
3-4h @ 225<0.06
350 = 5 mJy
LE2
LE31
LE12
SHARC-II DRW 11/08/2004
LE21
Software and Requirements
CRUSH is Java based and written and maintained by A. Kovacs
Available on web page
Has been successfully used on:
•
•
•
•
Windows
Mac OSX
Linux
Solaris
SHARCSOLVE is C based and written by D. Dowell (phasing out?)
Available only by special request
Has been successfully used on:
•
•
•
Mac OSX
Linux
Solaris
Ancillary software written by C. Borys.
Available on web page
Has been successfully used on:
•
•
•
Mac OSX
Linux
GCC compatible, so in principle could be compiled on other platforms. (C. Borys)
Minimum requirements to run all the software are:
Java 1.4.1 (for CRUSH)
gcc version 3.2.3
cfitsio libraries (http://heasarc.gsfc.nasa.gov/docs/software/fitsio/fitsio.html)
Other nice software include:
IDL or graphic (for plotting)
DS9, GAIA (starlink) for displaying fits files.
SHARC-II DRW 11/08/2004
Overview of Ancillary software : 1
header_update : alters header keywords in maps generated by CRUSH.
boxscan : helps calculate BOX_SCAN parameters for SHARC-II observations.
•
See SHARC-II web page for more details
sharccal : applies a calibration to a reduced sharc2 signal and noise map
sharccal [-c scalefactor] [-o offset] [-v] [-u units] raw.fits calibrated.fits
If no options, it will use the builtin calibration factor (V2JY in fits file)
Output signal = scalefactor*(raw map + offset)
FITS keywords added or modified:
NAME
---V2JY
CALAP
OFFAP
OFFSET
BUNIT
value
----------scalefactor
T
T|F
value
unit
comment
-----------------calibration factor
calibration applied?
offset applied?
offset value (only if OFFAP=T)
Units of output image
sharcgap : tests a raw sharc2 data file for timing gaps
sharcgap startidx stopidx
Needs to be run from the directory in which data is stored.
It checks consecutive data points to see if they are spaced by more than 1% of the expected time.
Expected time is 36ms, but is calculated explicitly from the first HDU.
sharcsmooth : Performs a PSF fit to a reduced SHARC2 map.
This will be described later in the calibration section.
SHARC-II DRW 11/08/2004
Overview of Ancillary software : 2
sharclog : Gets basic information from a SHARC-II raw data file
sharclog startidx stopidx
Needs to be run from the directory in which data is stored.
It scans the header of each file to provide a summary of the data (similar to Darren’s)
sharcstat : computes basic statistics on a reduced SHARC2 map
sharcstat file.fits
> sharcstat zw247_2.fits
NX
NY
N
s_mean
118
76
5049
0.00049829
s_stddev
0.14564
rms_mean
0.023233
rms_stddev
0.0072745
N = number of pixels with data in them (checks for NaN).
S_ corresponds to mean and standard deviation of the SIGNAL map
Rms_ corresponds to mean and standard deviation of the NOISE map.
sharctau : uses Jon Bird's tau fits to estimate tau for a given SHARC2 file. You also need the taufit files
sharctau [-v] datafile taufile
Can be run from anywhere
> sharctau -v /home/bigdisk1/sharc2-012900.fits /scr/borys/sharc/tau225_sep2003.fit
# DD/MM/YYYY HH:MM UT/24 TAU225 FITTAU
# 30/09/2003 12:06 0.50 0.051 0.068
> sharctau /home/bigdisk1/sharc2-012900.fits /scr/borys/sharc/tau225_sep2003.fit
0.068
Note: tau225 is read from file. Fittau is at the frequency appropriate to the input taufile.
SHARC-II DRW 11/08/2004
Installation
• Create a convenient place for the CRUSH installation.
• Use logical links to point to the most recent version.
• CRUSH can ONLY be run from the directory in which it is installed.
istari (7:45am)
lrwxrwxrwx
1
drwxr-xr-x
3
drwxr-xr-x
3
drwxr-xr-x
3
drwxr-xr-x
2
drwxr-xr-x
3
drwxr-xr-x
2
[/scr/borys/sharc/attila] >ls -la
borys
cittgp
10 Nov 5
borys
cittgp
4096 Oct 3
borys
cittgp
4096 Sep 22
borys
cittgp
4096 Oct 6
borys
cittgp
4096 Aug 31
borys
cittgp
4096 Oct 28
borys
cittgp
4096 Feb 13
08:21
07:16
09:39
15:48
23:13
00:35
2004
crush -> crush-1.34
crush-1.33
crush-1.33b2
crush-1.34
data
devel
MaiTau
• Create a convenient place for the ancillary software.
• Add the directory to your PATH variable
• These programs can be run from anywhere.
istari (7:48am)
-rwxr-xr-x
1
-rwxr-xr-x
1
-rwxr-xr-x
1
-rwxr-xr-x
1
-rwxr-xr-x
1
-rwxr-xr-x
1
-rwxr-xr-x
1
-rwxr-xr-x
1
[/scr/borys/sharc/code/bin] >ls -la
borys
cittgp
6793 Aug 27 14:31
borys
cittgp
16548 Aug 27 14:10
borys
cittgp
784529 Mar 26 2003
borys
cittgp
766548 Aug 27 14:31
borys
cittgp
770786 Aug 27 14:31
borys
cittgp
770704 Aug 27 14:32
borys
cittgp
16164 Aug 27 14:13
borys
cittgp
766589 Aug 27 14:32
SHARC-II DRW 11/08/2004
boxscan
sharccal
sharcextract
sharcgap
sharclog
sharcsmooth
sharcstat
sharctau
Scripting
The necessity of running CRUSH from its install directory makes file management slightly
tricky. Ways of manipulating output name include:
1) -outpath= OR REDUCED_MAP_PATH in crush.cfg
• This will change the path in which the file is saved, but not alter the name itself. i.e. it
will keep the form OBJNAME.SCAN1.SCAN2…SCANN.fits
• This filenaming structure is sometimes inconvenient (e.g. GAIA)
2) -name=/path/to/mapdir/map.fits
• This will alter the name to one of your choosing. You can include a path here as well. If
used, outpath is ignored.
RECOMMENDATION: use scripts and the -name= option
#!/bin/csh
cd /scr/borys/sharc/attila/crush
echo “PROCESSING ic5634”
./crush -faint -compact -name=ic5634_1.fits 14176 14177 >! ic5634_1.log
./crush -faint -compact -name=ic5634_2.fits 14182-14185 >! ic5634_2.log
./coadd -out=ic5634.fits ic5634_1.fits ic5634_2.fits >! ic5634.log
sharcsmooth ic5634.fits ic5634_smooth.fits
mv -f ic5634* /scr/borys/sharc/projects/slugs/CRUSH/maps/.
Note the 2 different ways of specifying scans to analyze
SHARC-II DRW 11/08/2004
CRUSH
At this point in the workshop, Attila gave a presentation on CRUSH.
Download that separately and review it before proceeding.
SHARC-II DRW 11/08/2004
Important Differences between
CRUSH and SHARCSOLVE
• Pointing
– Different treatment of the case that the IRC Reference Pixel is not the
middle of the array (16.5, 6.5)
• Calibration
– CRUSH and sharcsolve use completely different units, so cannot mix.
– CRUSH corrects for dependence of detector gain on detector loading,
so resulting tau relations should look “normal” to SCUBA and SHARC
users: (SHARC II, 350 m) ≈ 25(225 – 0.01)
– sharcsolve does NOT correct for gain change, so tau scaling looks “too
big”: (SHARC II, 350 m) ≈ 32(225 – 0.01)
• Chopped reduction
– sharcsolve differences with respect to chopper as first step.
– CRUSH treats secondary chopping as merely another pointing offset.
– Relative advantages of two approaches under study.
SHARC-II DRW 11/08/2004
Tau and Calibration
• Calibration at short sub-mm wavelengths is challenging,
but necessary.
• In the next few slides, we present our procedure for
estimating the atmospheric opacity, and then discuss the
overall calibration uncertainty for SHARC-II
• Then we provide a more detailed example of how to obtain
the calibrated flux for a specific observation.
SHARC-II DRW 11/08/2004
SHARC-II Calibrators
• Availability of calibration sources has always been a problem in sub-mm observations,
particularly at shorter wavelengths (can’t use BLAZARS, etc)
• We use primary calibrators (Mars, Uranus, and Neptune) to bootstrap the calibration of the
secondary systems.
• For stationary objects, we can use repeated observations to derive averages.
• For solar system objects, we need to consider the changes in distance and solid angle
over time.
TB = T1AUr(-1/2)
S() = B(TB)
• T1AU is derived by evaluating TB given all the other parameters (r is the heliocentric
distance in AU,  is the solid angle as seen from Earth, and B is the Planck
function evaluated at the appropriate frequency (typically 350 micron). These
values are provided by the JPL Horizons System: http://ssd.jpl.nasa.gov/cgi-bin/eph
• We have used these relations to extrapolate the fluxes for all days between 2002 and
2010.
• These calibrations are available for download for the SHARC-II web page
SHARC-II DRW 11/08/2004
SHARC-II DRW 11/08/2004
Tau Fits
•
•
•
•
•
Tau Dippers are noisy by nature (single measurement
every ~10 min).
Fits for both the 225Ghz and 350 micron data exist for
every SHARCII night to date.
Least square polynomial fits over a large range of each
night (almost always covering the entire observing
time).
Images of these fits are located on the SHARCII
website (www.submm.caltech.edu/~sharc).
When reducing your data, observers should look at
these tau fits as a FIRST step, so that they may
determine the best fit to use and what was happening
in the atmosphere at the time of observation.
SHARC-II DRW 11/08/2004
Tau Fits
Typical 350
micron fit.
Residuals are
located on
bottom of plot.
Typical fit ranges
from 2 to 20
hours UT. Notice
that the X axis is
in fraction of a
day.
SHARC-II DRW 11/08/2004
Keep an eye on the 225GHz and 350 micron fits…they CAN differ
Fits from both tippers on the same night.
SHARC-II DRW 11/08/2004
CRUSH and Tau
• CRUSH uses the MaiTau server to obtain the fitted tau.
• Parses through fit table (see below). Available online in
conjunction with the tau fits.
• CRUSH’s output will inform you if a fit for your file was
found and what value was retrieved. “Got Mai-Tau!
tau(350um) = X”
By default, MaiTau looks at the 350 micron fits. Use “-taufit=”
option to choose which fit, or not to use a fit at all.
SHARC-II DRW 11/08/2004
Mai Tau success story on Mon R2
Below are maps made from individual scans of MonR2 (provided by D. Benford).
The raw tau recorded in the file was used.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Problem:
Much
variability.
Mai Tau success story on Mon R2
Below are maps made from individual scans of MonR2 (provided by D. Benford).
Opacity this time was provided by MaiTau.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
MaiTau helps!
SHARC-II and Calibration
Want to determine how stable that conversion
factor, and thus calibration, is over time.
• Perform aperture photometry on calibrators with “known”
fluxes.
• “Known” fluxes are obtained from HORIZONS.
• CRUSH’s default output is in Volts- constant Volts to
Janskys applied (crush.cfg).
• By comparing known flux with CRUSH reported flux, we
obtain a conversion factor.
SHARC-II DRW 11/08/2004
Plot shows conversion
factor of calibrators
taken during the
August-September
2004 run.
Calibration Stability
Conversion factor is
consistent to within:
21.3% for all
18.8% for Neptune
19.2% for Uranus
SHARC-II DRW 11/08/2004
Calibration is
consistent over a
wide range of
elevations.
Calibration Stability
You do not need to
take calibration scans
at the same elevation
as your science.
SHARC-II DRW 11/08/2004
Calibration/Tau summary
• Tau fits: Important for understanding what is
happening to the atmosphere during
observation.
• Always look at the tau fits as a first step towards
calibration and reduction.
• CRUSH calibration is now consistent to within
20% and improving.
• Calibration is consistent over the range of
telescope positions.
• In the next few slides, we concentrate on object
specific calibrations.
SHARC-II DRW 11/08/2004
Calibration : PSF
PSF Photometry is the most often used technique for point source extraction in SCUBA maps
(particularly high-z projects). It is mathematically equivalent to “convolving with the beam”, except it also
takes into account the pixel-to-pixel noise differences. The procedure is very straightforward. Start at a
given pixel (i,j), and calculate the following statistic:
2 
W
W
 
dxW dyW
A  PSF(dx,dy)  S(i  dx, j  dy)
2
N(i  dx, j  dy)2
S and N are the signal and noise maps respectively. This is simply a LLS fit, and it is easy to derive the
best fit value and error for the PSF’s amplitude, A. This can be extended to include an offset parameter
as well.
The ancillary
program sharcsmooth performs this function. It assumes a purely Gaussian, with a

default (but user settable) FWHM of 9”. (see imagetool as well)
The map answers the following statistical question: what is the best fit amplitude to a Gaussian centered
at a given pixel? There are consequences to this assumption. i.e., for pixels near the peak of a source,
the assumption that that given pixel is the center of a source is wrong.
Source extraction with a “smoothed” map is done by setting a SNR threshold to search for
sources in the field.
SHARC-II DRW 11/08/2004
Calibration : Aperture
Aperture Photometry is another popular choice, most often used on sources that are
readily visible in the map, or if some other astrometric marker is available on which the
aperture can be centered.
There are 3 circular radii to choose. In order of increasing value they are: source, inner sky,
and outer sky. The annulus defined by the last two radii are used to estimate the mean sky
level AND the scatter of pixels. The central aperture is used to sum up the flux contained
within it (after correction for the mean sky offset). The error on the flux estimate is related
to the RMS of the pixels in the sky annulus, and the number of pixels in both the aperture
and annulus. Aperture radii usually chosen via “curve of groth”, annuli choosen to minimize
noise while still providing a good estimate of sky background and RMS.
Given that the pixel to pixel errors are dominated by residuals in the sky estimation and not
shot and photon noise as they are in optical CCD work, the equations are simpler. Note
that CRUSH and SHARCSOLVE do provide a “noise” map, but I have always found
that the RMS scatter in the “signal” pixels is higher than what the weight map
implies. Thus I assume a uniform weight per map pixel, and calculate this weight via
the RMS of the signal map. (implications for PSF fitting…)
It is not yet clear to me that sky estimation is necessary. CRUSH and SHARCSOLVE do a
pretty good job of returning “zero” for a map mean when we look at blank sky. However,
experience has also shown that well detected sources sometimes have a negative “bowl”
around them.
SHARC-II DRW 11/08/2004
Calibration : Aperture
N
= Number of pixels in map under consideration. (not
necessarily the full mapÉone may exclude edges for example)
NA
= Number of pixels within the aperture
NS
= Number of pixels within the sky annulus
mi
= flux in a given map pixel, i
F
= estimated aperture flux
F
= associated noise estimate
S
= estimated sky background per pixel
I
= pixel index which runs over the relevant pixels in
the map
Important caveat: This procedure assumes that
the pixels are uncorrelated. This is NOT the
default procedure for CRUSH, and one has to use
-convolve=-1 to force this. Otherwise, the
RMS calculation will be lower then it is supposed
to be (you’ve essentially smoothed the map). If
you do not use the convolve flag, the RMS should
be increased by a factor of sqrt(N), where N = the
number of pixels that fall within the area of the
convolving function. By default, we use an 8”
beam, with noise calculated for ~4.8” pixels,
which therefore requires a sqrt(1.33)*(8/1.4) = 6.6
increase in RMS (and consequently the total error
budget).
SHARC-II DRW 11/08/2004

N

1
 mi  m
N i1
RMS 
2

N
S
1 S
 mi
N S i1
NA
F   mi  S 
i1
 N 
F  RMS  N A  1 A 
 N S 
1/2
Aperture vs. PSF
So which should you use, and what are some caveats?
POINT SOURCES
If the PSF is varying (ie, DSOS not functioning or not turned on), APERTURE is probably
the safer choice.
If CHOPPING, care must be taken to keep the annulus away from the offbeams (a concern
for SHARCSOLVE, not CRUSH), hence PSF might be a good choice.
What do I use? Aperture, almost exclusively, but use the PSF smoothed map for
presentation.
EXTENDED SOURCES
PSF, since it essentially gives you Flux/beam.
CAVEATS
In deep integrations, there are some issues related to correlated sky signal still in the map.
(more from Attila)
SHARC-II DRW 11/08/2004
Calibrating your data
The principles involved with calibrating SHARC-II data are applicable to all data from other
sub-mm telescopes.
Ingredients
• Good estimates of the atmospheric opacity for all science and calibration observations
• A decent collection of calibrators (different objects, airmasses, etc.)
• A CHOICE IN HOW YOU WILL EXTRACT FLUXES FROM YOUR DATA. What is done to
the science map must also be done to the calibration.
EXAMPLE: Reduction of a local IRAS galaxy: MRK 331
• Scans 9125-9127, taken on Jan 15, 2003, at UT 04:54
• PSF photometry to be used
SHARC-II DRW 11/08/2004
Calibration Example 1
Using sharclog, (or by some other log or by
looking at the header), I find the UT time and
date the data were taken, and then go to the
SHARC-II web page to get the tau-fit plot for
that night.
The data were taken at UT 04:30 (~0.20
fractional day). Fits look OK, so I will not
override MaiTau.
Next I run CRUSH to make the map
SHARC-II DRW 11/08/2004
Calibration Example 2
> ./crush -faint -compact -convolve=-1 -name=mrk331.fits 9125-9127 >! mrk331.log
> sharcsmooth mrk331.fits mrk331_smooth.fits
> cat mrk331.log
crush -- Comprehensive Reduction Utility for SHARC2
Author: Attila Kovacs <attila@submm.caltech.edu>
Version: 1.34-1
Scan 1: Reading /home/bigdisk1/sharc2/sharc2-009125.fits...
Got Mai-Tau! tau(350um) =
1.3088
83 HDUs, 16439 x 36ms frames -> 9.9 minutes total.
Filtering 13.4Hz on noisy pixels.
DownSampling -> 5479 frames
[MRK331] observed at 2003-01-15T04:54:01.949
RA = 23:48:54.0
DEC = 20:18:29.0 (1950.0)
= 23:51:26.7
= 20:35:10.1 (2000.0)
AZ = 277:44:54.5
EL = 57:01:57.9
RAO = 0.0
DECO= 0.0
AZO = 0.0
ZAO =-0.0
FAZO=-104.0 FZAO=-30.0
Rotator = 60.0 RotZero = 60.0
Pointing Center = 16.5,6.5 Rotation Center = 18.5,8.6
Parallactic Angle = 85.0
tau225 = 0.053 tau = 1.835
Plate Scale = 4.93"x4.77"
SHARC-II DRW 11/08/2004
Calibration Example 3
• Now I check the logs for that date to see which calibrators were done.
• In this particular case, I will only pick the one closest to the science observation, but you
should reduce ALL of them and ensure that they seem reasonable.
> ./crush -compact -convolve=-1 -name=oh231.fits 9140 >! oh231.log
> sharcsmooth oh231.fits oh231_smooth.fits
• The sharcsmooth program does a PSF fit to each pixel, so to calibrate, I load the image
in GAIA (or DS9) and determine the brightness of the peak. In this case it is 3.378 units.
•The true flux of OH231.8 is 19.4±1.9 Jy (10% calibration uncertainty).
•Hence the scale factor is: 19.4/3.4 = 5.7
•Now we need to scale our image:
> sharccal -c 5.7 -u Jy mrk331_smooth.fits mrk_cal.fits
• Finally I open up GAIA, and look at mrk_cal.fits. By looking at the Signal and RMS maps,
I see that the brightest pixel is:
1.80 ± 0.02 Jy
In this case the calibration uncertainty dominates the error budget, so I simply quote 1.8 ±
0.2 Jy.
SHARC-II DRW 11/08/2004
Map Tweaks
Making the maps is the first step, and
you may need/want to perform some of
the tweaks presented on the following
slides.
SHARC-II DRW 11/08/2004
Tweaks-Pointing Correction
• Two options for pointing correction:
– Apply knowledge of improved telescope pointing model at time
of running CRUSH:
• crush -FAZO=-120.0 -FZAO=40.0 …
• See “MEMO: SHARC II Pointing at Nasmyth Focus Using CRUSH
(Dowell, Nov. 2004)” on web page/Data Analysis.
– Align images after CRUSH*:
• Use header_update utility (on web page under “Software utilities”)
• header_update image.fits RAP 1.0
• header_update image.fits DECP -1.0
• Doesn’t change WCS of image; however, pointing corrections will
be applied in coadd
*Attila says: Use jiggle
SHARC-II DRW 11/08/2004
Tweaks-Calibration
• CRUSH defaults to producing FITS images in nV,
corrected for detector nonlinearity and atmospheric
absorption.
• Based on results for calibrator (reduced/analyzed the
same way), one can re-scale image before or after
coadd:
– sharccal -c 5.0 uncal.fits cal.fits
– sharccal is on web page / Software utilities
• One can also change units of image:
– sharccal -u Jy/beam input.fits output.fits
– Just updates BUNIT keyword.
– CRUSH recognizes: V, nV, Jy/beam, Jy/arcsec**2, and
Jy/sr
• CRUSH imagetool will do these operations in future.
SHARC-II DRW 11/08/2004
Tweaks-Coadding
• Use CRUSH’s coadd routine:
pushd .; cd crushdir
coadd \
../data/SGRASTAR.16938to16947.cal.fits \
../data/SGRASTAR.16948to16957.cal.fits \
../data/SGRASTAR.16960to16969.cal.fits
popd; mv crushdir/../data/SGRASTAR.coadded.fits .
• By default, images are weighted, but “zero levels” are
not adjusted. (This is likely to change in future.)
SHARC-II DRW 11/08/2004
Tweaks-Cropping
• Use imagetool to cut noisy edges off map:
– Example:
pushd .; cd crushdir
imagetool -minexp=0.25 ../data/SGRASTAR.coadded.fits
popd
– Modifies image rather than making a new copy, unless -out option is
used.
– imagetool is part of CRUSH.
– Note other options to imagetool (e.g., -clip).
SHARC-II DRW 11/08/2004
Tweaks-Mosaicing
• To mosaic many maps with signal in them (e.g., bright
Galactic clouds), I find that adjusting the zero levels
before coadd improves the appearance of the image.
• I find the mode of the image intensity distribution and
subtract it off:
– The mode can be found crudely with ds9.
– sharccal -c 5.0 -o 0.1 uncal.fits cal.fits
SHARC-II DRW 11/08/2004
Publishable Images
Everyone has their own way of turning maps into publication style images.
I use IDL and the astrolib library (http://idlastro.gsfc.nasa.gov/homepage.html)
IDL is not free, but it is very versatile.
; read in the highest resolution image first
img_vla=readfits('../J1428p3526.fits',hdr_vla)
CX=281 ; center of source in X dimension
CY=282 ; center of source in Y dimension
HW=100 ; half width of box to extract
HEXTRACT,img_vla,hdr_vla,CX-HW,CX+HW,CY-HW,CY+HW
img_350=readfits('../../059/set_16301_smooth.fits',hdr_350) ; read in sharc image
hastrom,img_350,hdr_350,img_350p,hdr_350p,hdr_vla,MISSING=0 ; make image match size/shape of VLA
loadct,0
; black and white color table
gamma_ct,1.0,/CURRENT
; normal gamma stretch
img_350p=bytscl(img_350p,min=-0.05,max=0.07)
; choose plotting range
set_plot, 'ps'
device , filename='sharc.eps', /encap, xsize=10., ysize = 10., $
yoffset = 1., BITS_PER_PIXEL=8, COLOR=1
imcontour, img_350p, hdr_350p, levels=0, xtitle=' ', ytitle=' ',$ ; plot the axis
charsize=1.2,charthick=3,/nodata,subtitle=' ',COLOR=0,$
XSTYLE=4,YSTYLE=4
tvimage,img_350p,/overplot,/keep_aspect; display the image
; overlay radio contours
imcontour, img_vla, hdr_vla, levels=[2e-4,3e-4,4e-4],xtitle=' ', ytitle=' ',$
charsize=1.2,charthick=3,/noerase,subtitle=' ',C_THICK=3,C_COLOR=0,COLOR=0, $
XTHICK=2,YTHICK=2
XYOUTS,10,10,'350 + VLA contours',charsize=1.5,charthick=3
DEVICE, /close
SHARC-II DRW 11/08/2004
Publishable Images
IDL> @sharc.idl
% READFITS: Now reading 561 by 561 by 1 by 1
array
% HEXTRACT: Now extracting a 201 by 201 subarray
% READFITS: Now reading 201 by 201 array
% HPRECESS: Header astrometry has been precessed
to 2000.0000
% LOADCT: Loading table B-W LINEAR
Other packages in use at Caltech:
• Graphic
• GAIA (A free starlink package)
More complicated example:
• Multiple contour sources
• Astronomical coordinates
• Object labelling
SHARC-II DRW 11/08/2004
Chopped vs. Unchopped?
• In principle a 2-beam chopping observation increases the noise by sqrt(2) because of
spending half the time on source. But this can be recovered by folding back in the flux from
the “off” beam, as long as it lands on the array.
• In general, we have had good success chopping and reducing the data with sharcsolve.
Attila only recently added chopped data support in CRUSH, though it seems to work. Once
tested more rigorously, we will likely phase out sharcsolve completely.
• We strongly encourage people who want to chop to discuss it with one of us.
Chopping is best suited for observations of point sources when the
atmosphere shows signs of strong variability.
We have not yet shown that chopping offers a substantial improvement.
SHARC-II DRW 11/08/2004
Comparison of Chopped Data
Stars denote sources detected by SCUBA, and 2 are well detected by SHARC-II. In this case,
CRUSH and SHARCSOLVE both do a good job recovering the same map.
Lessons learned (1)
• Poor choices for focusing telescope:
– Many protostellar/UCHII things, especially NGC2071. Even
IRAS16293-2422 is slightly elliptical.
– Saturn, Jupiter, Venus
– Better choices: Mars (usually), Uranus, Neptune, Callisto
(usually), Ganymede (often), Ceres, Vesta (sometimes), Pallas
(sometimes), CRL618, CRL2688, IRC10216, OH231.8, ARP220
• Poor choices for pointing: NGC 2071 (Use CRL618,
HLTAU, or OH231.8 instead)
• Poor choices for flux calibration: NGC 2071, blazars
• Moons: Callisto or Ganymede is usually observable.
Titan is hard to observe cleanly; don’t bother with it.
SHARC-II DRW 11/08/2004
Avoid NGC 2071
Don’t Focus/Point/Calibrate With It!
SHARC-II DRW 11/08/2004
Lessons learned (2)
• For best looking large maps, map full area in as
short a time as possible. Mosaics of fields under
different conditions and scan patterns tend to
have obvious “stitches”.
• The following projects have proven difficult;
embark on them at your own risk:
– High dynamic range
• CSO beam at 350 microns.
• Negative artifacts surrounding bright sources.
– Faint, widely extended emission, due to ripples in
reduced image.
– Be careful integrating total emission; use exactly the
same procedure for source and calibrator, and use a
“sky” aperture.
SHARC-II DRW 11/08/2004
Ripples and Mosaic Stitches in Box Scan of
Source with Low Surface Brightness
12′
• crush -deep gets rid of ripples, but also
some extended structure.
SHARC-II DRW 11/08/2004
M51
Problem:
Large scale low-level baseline
Bad Pixel flagging
• Bad pixels that are not properly flagged by CRUSH end up tracing the scan pattern on the final
map. Sometimes this is hard to see since by default CRUSH smooths the output.
• I recommend reducing each scan separately with -convolve=-1 in order to see if bad pixels are
corrupting any of the data. If so, remove them from the list of scans you use to make the
combined map. CRUSH 1.35 is meant to have a better algorithm for flagging bad pixels.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Right: Map of MonR2 with a bad
pixel clearly influencing the output.
(map courtesy of D. Benford)
Negative Halo Surrounding Bright Sources
• Negative intensity is ~3% of peak intensity.
SHARC-II DRW 11/08/2004
Tips for taking better data (1)
• DSOS:
– Use it!
• Beam shape at 350 m becomes
nearly independent of ZA.
– Check that you are using correct
procedure:
• Wait until telescope near outside
ambient temperature, then init.
(Assumes that telescope at that
time has the same shape as
during the holography.)
• Make sure you get into agent
mode before observing. Easy to
check with dsosm monitor
screen.
• When significantly changing ZA,
wait for new dish setting to
activate and settle (again with
dsosm). Otherwise, your
calibrator may have the wrong
PSF.
• Turn off DSOS at the end of the
night.
SHARC-II DRW 11/08/2004
Tips for taking better data (2)
• UIP source catalogs
– Enter before you get to 14,000 ft.
– Check the coordinates and equinox with VERIFY.
• Please use the existing CAL_* sources for extrasolar
calibrators. This makes it easier for the staff to identify
pointing/calibration scans to study, e.g., pointing model.
The CAL_* sources are now loaded automatically from
USER:[SHARC]SHARCCAL.CAT upon INST SHARC2.
• Get started quickly with the “Cheatsheet” on the web
page.
• Focus and pointing tend to drift early in the night. Check
every ~45 minutes until ~8 PM. After that, focus is
usually stable (check every ~2 hours). Still should check
pointing every ~1 hour, especially if observing faint
sources.
SHARC-II DRW 11/08/2004
Tips for taking better data (3)
• “SHARC Tau”: Use Bolometer Power Mean on Bolometer Means
window to convert to 225×airmass. This is useful in cases that skies may
be clearing rapidly early in night.
• Please report problems, even minor ones, to
sharc@submm.caltech.edu
SHARC-II DRW 11/08/2004
Miscellaneous Notes
In the next few slides, we present some information
that is related to the workshop and SHARC-II in
general, but was tangential to the material already
presented.
SHARC-II DRW 11/08/2004
Dependence of Gain on
Atmospheric Loading
Instrument status
• Nasmyth move was
successful, and we’ll
stay there for
foreseeable future.
• In early September, a
glue joint in the amplifier
for rows 9-12 separated,
breaking ~300 microwires in the process.
– The glue joint has been
repaired and reinforced.
– We’re now working on the
micro-wires, with
expectation of returning to
normal by the January
run.
– September run was
carried out in its entirety,
with ~2/3 mapping speed.
Sept 2004 Pointing T-Terms
• The move to Nasmyth forced a revisit
of how we calculate pointing
corrections. See the documentation on
the SHARC-II web page for a full
report.
• Top: as observed, (FAZO) = 3.7″
• Bottom: after model applied, (FAZO)
= 2.1″
• Model fit and method of correcting
data: web page memo
• Use CRUSH exclusively to apply the
model if not pointed on center of array
(16.5, 6.5).
SHARC-II DRW 11/08/2004
Temperature Effects on Pointing
(Hiroko)
SHARC-II DRW 11/08/2004
CSO/SHARC II Sidelobes (350 m)
~160″
• Sidelobe spots as bright as 5% of peak intensity.
SHARC-II DRW 11/08/2004
BOXSCAN
There are a number of constraints on the scan pattern:
1. It turns out that by starting the scan from the center of the map, the map size cannot be perfectly
square, otherwise the scan will hit a corner, and the algorithm will fail.
2. There is also a choice in how far apart the intersection points are. You want them to be less than
the size of the SHARC array (to ensure the whole map gets sampled), but having them too close
together means it will take longer to complete a scan.
3. The scan rate should be fast enough to close out the scan in a reasonable time, but not so fast as
to smear the beam. Also, faster scans mean it is harder for the telescope to handle rapid change
in direction at the boundaries.
Want a rectangular pattern instead of a box?
This is straightforward, but ask me for details.
SHARC-II DRW 11/08/2004
BOXSCAN
To help plan your scan, download, and compile the program boxscan.c.
Here is an example of it in action: I want to scan a 10x8 arcminute region, relatively finely.
I start with a 10 arcsec spacing, and a 40 arcsec/sec scan rate:
> boxscan 600 480 10 40
BOX_SCAN 600.000 471.429 40.0 45 :
23.33 minutes
# WARNING: scantime exceeds 15 minutes
Hmmm...that won't work. Scan length is too long. I'll try a larger spacing (20.0 arcsec)
> boxscan 600 480 20 40
BOX_SCAN 600.000 457.143 40.0 45 :
11.31 minutes
That's better, but now I wonder if 40arcs/sec is too fast. I'll try an even larger spacing and a
slower scan speed:
> boxscan 600 480 25 30
BOX_SCAN 600.000 494.118 30.0 45 :
13.20 minutes
Enter the scan command dirently into the UIP, and set the integration time as suggested (perhaps
adding on 0.2 minutes for overhead). Wondering what the "45" is in the "BOX_SCAN" output? It
is the angle at which the scan starts. This should always be 45. Consult with us if you want to try
other things.
SHARC-II DRW 11/08/2004
SHARC-II/CRUSH Simulations
Tom Tyranowski (visiting student) wrote a data simulator, which we use to test the
performance of CRUSH. If you are interested in using it, contact us.
100 mJy Ring surrounding
compact star in 1 hour
10' x 10'
Billiard Ball Scan.
Imperfect cleaning of
faint large scale structures
Source Fluxes Recovered within 1%
100 mJy Compact
Lissajous Sweep
in 1 hour
Imperfect cleaning of
faint large scale
structures
SHARC-II DRW 11/08/2004
To-Do list
•
•
•
•
•
•
•
•
•
•
•
•
Get rid of negatives around bright sources.
Find mode of an image
Calibration cheatsheet
example graphic link/script on web page
Improved CRUSH default -chopped reduction
Memory problem in 1.34? (z=6.5 source)
Bolometer scrambling test
CRUSH email exploder
Resume simulations; release simulation code.
sharcsolve (for Johannes): independent tau for each scan
Hoping for better pixel flagging in v. 1.35
Update CAL catalog in UIP.