Name
Mihai
Yongjun
Nathan
Shigeharu (Gary)
Sarma
Amy
Nick
Suzanne
Kevin
Prof. Lister
Email
mcara
chen43
nukwaste
kihara
sarma
alankey
nmellott
senichol
obrien
mlister
Status
Graduate student
Visiting scholar
Graduate student
Graduate student
Graduate student
Undergraduate
Undergraduate
Graduate student
Undergraduate
Professor
AIPS User #
11
333
410
660
1650
1937
137, 1025
If you do not have an AIPS number, please pick one between 100 and 2000.
Preliminaries/Setup
Environment variables that need to be set:
export DIFMAP_LOGIN=/home/mlister/.www/scripts/macros
export PGPLOT_DIR=/home/mlister/pgplot
export LD_LIBRARY_PATH=/home/mlister/pgplot
export PGPLOT_DEV=/xserve
export SCHED=/home/mlister/sched
export ARCHIVE=/home/sarah_1/2cmarchive
export GROUP=astrophys
export PATH=./:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:
/home/mlister/.www/scripts/:/project/miranda/mlister/perl/:/sbin
export PRINTER=knuth
export EDITOR=gedit
export TEXINPUTS=.:/usr/share/texmf/tex//:
/project/miranda/mlister/aastex52//:/usr/share/lib/texmf/tex//:/opt/local/share/teTeX/texmf/:
export
PERL5LIB=.:/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi:/home/mlister/.www/scripts:/proj
ect/miranda/mlister/perl/scripts:/project/mir
anda/mlister/perl
export WWW=/project/astro/www/MOJAVE/
If you are using bash shell, these need to go in your .bashrc or .profile (whatever
gets read automatically upon login). To find out what shell you are using, type
printenv SHELL
You also should be part of the 'astrophys' group. (Type groups to find out). If not,
let me know asap.
Computers/Disks/Tape Drives
Our group has six machines that can run AIPS:
oberon, ariel, miranda: in room 391.
miranda: my office
titania: Nate Cooper's desk
umbriel: Chen's desk
Don't run any large CPU tasks on the last three machines without asking their users first.
There is a DAT tape drive hooked up to ariel. To load from tape, login to
the ariel console, or log into ariel remotely and then type 'aips remote'.
Note that when you exit AIPS, it will spit out your tape automatically.
Each machine has a /project directory (/project/ariel, etc. ) that can be
used to store data files. cd to the directory and type
df -h
to show available disk space. If you don't have a subdirectory of your
own, create one, or ask me to do it for you. The /project areas for
umbriel and miranda will be appearing soon.
Most programs except AIPS will run remotely from any other linux machine on the
internet, or a windows machine with the right emulator software (note I have no
experience with the latter). For AIPS, it's a little trickier getting the TV window to
appear remotely, and it's usually painfully slow unless you're on the dept 1 Gb/s line.
You have to use 'aips tv=displaymachine:servermachine' . Note that each machine can
only host one TV server at a time. This means that you could potentially 'steal' the TV
from the person sitting at the host computer. Using 'aips remote' will give you a text only
AIPS session, but is very fast.
Loading VLBA data into AIPS from tape
Log in to ariel, put tape in drive that sits on top of ariel. Start aips with
linux> aips
(or aips remote if you aren't sitting at the ariel console)
AIPS> task 'mount';default; mount
AIPS> task 'vlbaload'; default;douvcomp false;
You need to set ncount to the number of files (N) listed on the paper label that
comes with the DAT tape.
AIPS>ncount N
Set OUTNAME to something sensible (usually the experiment VLBA code).
For experiments below 22 GHz, set CLINT = 0.5, Above 22 GHz, use 0.2
Pick your outdisk carefully. Most VLBA experiments are 1 to 2 Gb, check
disk space with
AIPS>free
If you fill up a disk, AIPS will crash, and other people might get unhappy.
Check the inputs with inp , then
AIPS> vlbaload
Eject the tape after you've loaded the data with
AIPS> dismount
Check the output in the message window to see that it loaded all the files listed on the
tape card. Another useful way of checking is to run VLBASUMM on the new file
that was created.
Loading data from disk
To obtain data from NRAO, go to archive.nrao.edu and use the advanced query tool
to locate the right experiment. Follow the instructions and download the data
to a /project directory with sufficient disk space. If you see a large single file with
your expt code and frequency, download it, otherwise, you'll have to download the
individual segments. Be careful, sometimes there are two versions of the exact same
segment listed, with the same time range. Pick the one with the more recent file
modification date. If the experiment consists of multiple segment files, check the time
listing carefully, and rename the files in chronological order FITS1, FITS2, FITS3, etc.
Once the files are downloaded, cd to the directory and type (in bash shell)
U=$PWD; export U
Start aips, then
AIPS>task 'fitld';default;douvcomp false; doconcat 1;wtthresh 0.7;clint 0.5
set CLINT,OUTNAME, OUTDISK as per directions for loading from tape.
set NCOUNT to the number of file segments to be loaded
AIPS> infil 'U:FITS
Review the inputs, then
AIPS> go fitld
Check the output to see if all files loaded correctly.
If you don't specify the U in infile, AIPS will assume that you are loading
from the default FITS directory, which is /project/ariel/AIPS-31DEC05/FITS
It is advisable to avoid using this directory, as the /project/ariel disk is always
close to capacity. An annoying feature of AIPS is that once you start aips,
it is too late to set any new environment variables like U. You must exit
AIPS with
AIPS>kleenex
, set the variables, then start aips again.
Note that the INFIL variable has a maximum of 48 characters, including the path
and filename. Leaving off the trailing quote in infil 'U:fits means AIPS will preserve
the lowercase, otherwise it will assume you are loading FITS1, FITS2, etc. For more
info within AIPS, type help outfile.
Processing non-MOJAVE VLBA data
-----------------------------------------------------Once the datafile is loaded, the procedure to follow is outlined in
http://www.aoc.nrao.edu/aips/cook.html#C09
The main steps in processing VLBA data are flagging bad data, calibrating the
amplitudes of the visibilities, and calibrating the phases of the visiblities. Here are
more details:
Making sure your file has the correct tables attached. Set indisk to the disk
which you loaded your file to, then type ucat. Then type getn X , where X is the
file number of your file. Type imhead to view details of the file and its tables. The
crucial tables are:
TY: system temperature table.
GC: antenna gain table
FG: flag table
PC: pulse calibration table
NX: index (scan listing) table
WX: weather table (only needed for experiments >= 15 GHz)
If your experiment is post year 2000, it will likely have all these files correctly
attached. (Note: there have been problems with CL&NX tables on files
downloaded from the archive, you'll have to delete these and recreate them with
INDXR). For experiments before this, or those that include non-VLBA antennas,
extra work is required to get correct tables. Details are in the AIPS cookbook.
For experiments below 8 GHz, a correction for ionospheric effects should be
considered.
For experiments involving phase-referencing (a special technique used to image very
weak sources), corrections need to be made to the earth orientation parameters.
Bad data need to be flagged (removed).
If your experiment has 2-bit sampling , run ACCOR . To find the sampling rate, go
to, e.g http://www.vlba.nrao.edu/astro/VOBS/astronomy/apr06/
change the apr06 as appropriate (an experiment observed in September 1996will
be sep96). Find your experiment code and click on it. The .sum file lists all the
gory details.
For experiments 15 GHz and above, you will need to enter the expected flux density
of each source using SETJY.
Run APCAL to establish the absolute flux density scaling for all antennas. For
experiments >= 15 GHz, you will need to set opcode='OPAC'.
Flag the resulting solution (SN) table using SNEDT and apply with CLCAL
If your experiment has full polarization or uses phase-referencing, you must run
VLBAPANG to correct the phases for parallactic angle rotation.
Apply the pulse-cal phase corrections using VLBAPCOR.
If your experiment contains all strong sources above 300 mJy, you should fringe fit
with VLBAFRNG
For full polarization experiments, calibrate the cross-hand phases with CRSFRING
Calibrate the bandpass with BPASS
Apply all calibrations with SPLIT
Run a point-source phase-only selfcalibration with CALIB on the splitted source
files.
Output all the self-cal'd files to linux disk with FITTP. See DIFMAP section.
Steps for reducing MOJAVE VLBA data
Because all MOJAVE experiments are identical, except for the observing frequency and
sources that are observed, a standard script can be used to reduce them. Only a few
modifications to the script are usually necessary.
If you are reducing MOJAVE data for the first time, start aips, enter your user number
(NNN) and then
AIPS> ehex NNN
to see what your user number is in extended hex form. If it returns less than 3 digits,
prefix the extended hex value with a zero, e.g. 1025 = 0SH in extended hex.
then type
linux>cp ~aips/RUN/MYPROC.0SH ~aips/RUN/MYPROC.EEE
where EEE is your extended hex user number.
1. Setting up linux directories
cd to a /project directory with at least 1 Gb disk space. The steps below assume you are
reducing an experiment BL137G:
linux> mkdir BL137G
linux> cd BL137G
linux> cp ~mlister/.www/scripts/BL137.TEMPLATE BL137G_U.SCRIPT
linux> cp ~mlister/.www/scripts/changes.txt .
linux>cp /project/astro/www/MOJAVE/keyfiles/bl137g.txt .
Edit the changes.txt file and change newsrc1, etc, to the source names listed in
bl137g.txt. Also, change newepoch to the epoch date listed at
http://www.physics.purdue.edu/astro/MOJAVE/data.html
change newexpcode1 to BL137G and newexpcode2 to bl137g
Save your changes, and then
linux>./changes.txt
Load the BL137G.SCRIPT file in a text editor, cd to your BL137G directory, then copy
and paste the commands under step 1 of the script into the linux prompt.
2. Loading the data
Obtain the data file from archive.nrao.edu and store it in your BL137G directory.
Comment out the appropriate sections of the step 2 section, change the outdisk if
necessary, start aips, and cut and past the commands under step 2 into the aips prompt.
3. Flagging and choosing reference antenna
After recreating the NX and CL tables by cutting and pasting the first few lines of step 3
into AIPS, you will need to flag data based on high system temperature values with
EDITA. Look for abnormally high values, and flag in all stokes/IFs. To pick a menu
option, highlight with the cursor, press 'A'. Check the AIPS message window for info.
To select a reference antenna, refer to the TSYS.PS plot, and the weather files
under http://www.vlba.nrao.edu/astro/VOBS/astronomy/apr06/
(change apr06 to the mmm/yy of your experiment). The weather plots are, e.g.
bl137ctsm.LA.ps.gz
A good reference antenna is one of FD,PT,LA , KP or NL, and experiences no rain or
snow, has less than 10 minutes downtime, no electronic/mechanical failures, and has
no abnormally high system temperatures in TSYS.PS. You should go through the
antennas in the order above until you find a suitable reference antenna.
Once you have selected a reference antenna, do a global search and replace
on antnum('FD') in the script where FD is your reference antenna code.
If you are reducing a 8 or 12 GHz experiment, do step 5, otherwise, do step 4.
4. (15 GHz only) A-priori amplitude correction with opacity correction.
Look up the flux density closest to this epoch for each source on the MOJAVE
webpages, and enter it into the script. If you can't find a flux, use 1.0 Jy.
http://www.physics.purdue.edu/astro/MOJAVE/MOJAVEsources.shtml
Paste the text from step 4 into AIPS
5. (Below 15 Ghz) A-priori amplitude correction with no opacity correction.
Paste the text from step 5 into AIPS
6. Paste the steps from step 6 into AIPS. Edit the SN table in the same
manner as you did with EDITA.
7. Run CLCAL by pasting the text from step 7 into AIPS. Temporarily exit out of aips
with ctrl-z and type gv $U/APCAL.PS to inspect the output SN table. Return
to AIPS by typing fg at the linux prompt.
8. Paste the text from step 8 to perform the parallactic angle correction.
9. For this step you need to identify a source in the experiment that is very bright, and
point-like. Check the flux densities and images of the sources on the MOJAVE pages.
Once you've identified the source, hit ctrl-z to return to linux, then type
grep 2134+004 VLBASUMM.TXT
(where 2134+004 is the source you've picked)
This will list a series of scan numbers and times that contain this source. Check
to make sure the time of the middle scan in the list doesn't coincide with any time
period you've previously flagged out for an antenna because of weather/electronics.
If so, pick another scan near the middle of the list and record its scan number. Go
back to AIPS with 'fg' and edit the script text under step 9 to change
TIMER=scantime(75) to the scan number you've selected. Copy the text of step 9
to the AIPS prompt. Check on the TV that it plots 9 baselines, and the data look
ok on all baselines.
10. Copy the text from step 10 into the AIPS prompt.
11a. Calibrating phases across frequency band
Copy the text from step 11 into the AIPS prompt. If you get an error message
here, it may be because you didn't delete the original NX file , or the PC table
is missing some info for some antenna, or the scan time you selected in step 10
doesn't contain data for all baselines.
11b. Global fringe fit
If all the sources are point-like and above 300 mJy, you may wish to run step 12.
Otherwise, you can skip this step entirely.
12. (8 Ghz only) If you are reducing an 8 Ghz MOJAVE experiment, you will have to
split the data at this point into 8.1 and 8.4 Ghz files, and run all the following steps on
each new file.
13. Crosshand phase adjustment
For this step, the source in your reference scan time (scantime) must be polarized –
check the MOJAVE page for at least 30 mJy of polarized flux). If not, you need to
choose a different scantime. Paste the text from step 13 into the AIPS prompt. Check the
output in the message window and make sure there are no failed solutions. If so, you
need to delete the SN and CL tables that were created by this task, pick a different
scantime, and rerun step 13. Check the POSSM_CROSS,PS plot to see if most
of the phases are flat.
14. Bandpass calibration
For this step, use the same scantime as you used in step 9.
15. Save tables and apply calibration
At this point save your calibration tables with the text in step 15, SPLIT the source
files from the main one, do a point source selfcalibration on them, and output to $U.
Make sure the filenames are correct, in that they use the correct frequency code (see the
script).
At this point, you are ready for DIFMAP. Take a well deserved break.
Self-calibrating and Imaging in DIFMAP
DIFMAP is a very useful program written by Martin Shepherd of Caltech for processing
and imaging inteferometer data. Although all the same functionality exists in AIPS,
DIFMAP is far faster and simpler to use.
To use DIFMAP, make sure you have your environment variables set up as described on
page 1, then cd to the directory containing your data, and type
linux>difmap
To load an interferometer data (uv) file, type
difmap> obs <filename>
If you're doing a MOJAVE epoch, you will want to load the uvf_raw_edt file.
To load a file that you've previously saved with 'save <filename>' in DIFMAP, type
difmap>@<filename>.par
1. Flagging bad data
Bad data can usually be recognized by having abnormally high amplitudes, bad phases,
or both. Typing
difmap>medt
will load up stokes V, and display a rad plot. Flag out the excessively high points
by typing 'c' with the cursor in the window, and click and dragging out a box.
Right clicking on the plot will load up Stokes Q, flag again, right click, etc.
When the display changes to the vplot display (of baselines), look for cases where
there has been a lot of flagging on the same scan at the same antenna. In many cases
it's safer to flag out the whole scan on that antenna. (Bad data are worse than no data)
2. Averaging the data
When you're satisfied with the flagging, average the data in time by typing
difmap>mav
3. Cleaning and self-calibration
The goal of self-calibration is to build up the best representation of the source, which
is used as a model to solve for the atmospheric effects at each antenna. The process is
iterative, and usually converges fairly quickly. To start selfcalibration, type
difmap>mapl
This displays the residual image (data – model). Put a small 'clean' box around the
brightest region in the image (about 10 x 10 pixels) by clicking and dragging the mouse.
Then right click, and type
difmap>clean 300,0.03
difmap>csm
The residual image will appear again. At this point, check out a previous image of
the source on the MOJAVE pages, to see where the real jet of the source lies. The object
here is to put clean boxes only in regions where the real jet is located, and where
you see emission in the residual image. Note that if the source is very large, you need to
use a bigger mapsize by setting mapsize 2048,0.1 at the difmap prompt. Add more clean
boxes if necessary, then right click and
difmap>csm
Repeat until you don't see any bright features significantly above the noise in the
region where you know the jet should be. Then right click, and
difmap> uvweight 0,-2
difmap>csm
Add clean boxes if necessary, and do csm again if necessary.
Once the residual image is smooth, you can type
difmap>fc 200
Add clean boxes if necessary, csm. At each stage, when the residual image is smooth,
type fc X, where X is ½ the previous number you used. You should stop when you
get to fc 0.5.
During this process, you can check on your model via
difmap>rad
hit 'm' while in the plot window to show the model, then '-' to display a plot of
data – model. Flag any discrepant points near the top of the graph with 'c'.
Once you're happy with the flagging and your model, type
difmap>@/home/mlister/.www/scripts/final_clean
This will save your existing windows to a file list.win, then clean the source
and display the final image.
Things to watch out for in your final image:
- does your image look a lot different than previous epochs on the MOJAVE page?
are there a lot of contour lines in sky areas that should be blank?
Is there a 'bump' on the other side of the core opposite the jet?
Is the rms above 0.5 mJy/beam? (create a clean box in a blank sky area, and hit 's' while
the cursor is inside it)
If you're happy with your image, create a hardcopy of it with
difmap>device <filename>.ps/vps
difmap>mapl cln
difmap>device “/xw”
Then save it with
difmap>save <filename>
If this is a MOJAVE epoch, also do
difmap>msave sourcename.freqcode.epoch
(e.g., msave 2134+004.u.2005_02_18)
Note that this msave command removes your calibration, and overwrites your
original uvf_raw_edt file.
Repeat these steps for every source in your experiment. Congratulations! You've
successfully made images from VLBA data.
Calibrating MOJAVE polarization data
Although the data are now calibrated for total intensity, polarization imaging requires
corrections to be made for feed leakage at the antenna.
17. Before entering aips, create a DTERMS and DTERMS/NOTUSED directory
according to the script directions. Paste the text for step 17 into the AIPS prompt.
18. Determining polarization leakage terms
Go back to the linux prompt, and cd to $U/DTERMS. Type
linux>process_AN.pl *AN
This will display a plot of leakage term solutions for each antenna and IF. Hopefully
there will be a clustering in each plot around the 'correct' value. The object here is to
note which sources give consistently bad dterms on most/all antennas. Move their
.AN files into the NOTUSED directory, and re-run process_AN.pl until you are
satisfied. Rename your DTERMS /DTERMS file to DTERMS/FINAL.DTERMS
19. Establishing the electric vector position angles
This step is only currently implemented for 15 Ghz experiments.
Copy your DTERMS/FINAL.DTERMS, to the master dterm file directory (listed in the
script). Run the new_process_EVPA.pl script to determine the correction to establish the
electric vector position angles on the sky.
20. Applying the polarization calibration
If you found an EVPA correction in step 19, multiply it by two and edit the text in step
20 appropriately. Otherwise, use clcorprm 0,0,0,0 . Return to aips with 'fg', then copy the
text in step 20 to the AIPS prompt.
21.Making final maps
This is fairly straightforward, and involves pasting lines from the script into the linux
prompt.
Modelfitting data with DIFMAP
Although extragalactic jets have extremely complex, time-variable emission structure,
their enormous distances, combined with the limited resolution capability of ground
based interferometers, means they appear rather 'blobby' . The positive side of this
is that the jets can be fit with a set of two-dimensional Gaussians whose centroids and
fluxes can be determined accurately.
To modelfit a source in DIFMAP, start difmap, load the calibrated uv file that you
wish to modelfit (likely ends in .uvf), and type
difmap>go
difmap>mapl
Click 'n' on the residual image to start a new gaussian component. Left click on
where you'd like the component to be centered, move the cursor a short distance, and
left click again, then hit 'n' again to make the component. Then type
difmap>mm
Add a new component if necessary, then 'mm' again. Repeat until satisfied.
Some guidelines:
be careful to only put components where you know there is real jet emission (consult the
MOJAVE pages maps)
if a component size becomes < 0.1 mas, remove it by moving the cursor over it and
hitting 'r'. Right click, run mapl again, and create a new delta-function component in
the same spot by typing 'n' , left clicking on the spot, then typing 'n' again.
components with fluxes less than 5 mJy are rarely worth fitting and can usually be
removed.
A powerful method of checking the reliability of fits is to fit the same source on
successive epochs. All of these jets change very slowly with time, so one epoch's
modelfit should be pretty close to the others. Load the model from the previous
epoch with rmod, then run mm.
Sometimes a jet will have a new component emerge very close to the core (the base of
the jet), in such cases, using an elliptical gaussian (left-click three times before
hitting 'n') might give better results.