TIPS/JIM January 19th 2011
£  News:
£ INS – Danny Lennon
£  Selection of the New COS/FUV Lifetime Position- Cristina Oliveira
£  Upcoming Enhancements to the HST Archive – Mark Kyprianou
£  Using Exchange Leave Calendars at STScI – Glenn Miller
Next TIPS/JIM February 16th
INS News - Danny Lennon
Welcome everyone to the first TIPS/JIM meeting of 2012.
Danny Lennon welcomes new staff to INS:
-----------------------------Stephen Holland to COS/STIS Team as Visitin Scientist
Dave Sahnow to COS/STIS Team as STScI Instrument Support Scientist
David Hickey to Telescopes Team as Instrument Engineer
Max Mutchler welcomed new RIAB members:
Josh Sokol
Svea Hernandez
Matthew Bourque
Andrew Colson
Sean Lockwood
Other News
---------Danny congratulated George Hartig on receiving the Roger Doxsey award.
Dave Soderblom congratulated Ron Gilliland on receiving the Beatrice M.
Tinsley Prize by the AAS.
TIPS/JIM January 19th 2011
£  Selection of the New COS/FUV Lifetime Position- Cristina Oliveira
£  Upcoming Enhancements to the HST Archive – Mark Kyprianou
£  Using Exchange Leave Calendars at STScI – Glenn Miller
Next TIPS/JIM February 16th
Selection of the New
COS/FUV Lifetime Position
Cristina Oliveira
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
1
•
•
COS FUV XDL is a photon-counting
micro-channel plate (MCP) detector
In COS FUV TIME-TAG mode every
photon is recorded with:
# of events
Gain-Sag Artifacts in COS FUV Spectra
–  position (x,y) and arrival time (t)
–  total electron charge generated/pulse-height
amplitude (0 ≤ PHA ≤ 31)
Gain-Sag Artifacts in COS FUV Spectra
•
For every
detector element, PHA
•!As the PHA distribution shifts to lower
distribution
changes with time, shifting
PHA values events reaching the
to lowerdetector
PHA values,
fewer
electrons
in areasas
with
low PHA
are no
can be extracted
from
the
MCP
with
longer recorded, effectively resulting
usage, the
in a so-called
flux loss. gain-sag effect (see
top figure)
•
PHA bin
•!As time progresses the flux loss can
Events reaching
detector
areas
become sothe
severe
that theinspectrum
with lowneeds
PHA to
are
longer
beno
moved
to a recorded,
new lifetime
effectively
resulting in a flux loss (see
position
bottom figure)
•
•!E.g., filtering out target photons with a
As time PHA
progresses
thehas
flux
loss can
value which
dropped
below
becomethe
sooriginal
severethreshold
that thevalue
spectrum
of 4,
needs toproduces
be moved
to a new artifacts
lifetime
absorption-like
in
positionthe extracted spectra corresponding
Gain sag holes
(Due to Lyα airglow)
to those detector regions more
Red:
PHA = [4,30] Blue: PHA = [2,30]
Lifetime
2
by airglow lines.TIPS Meeting - COS/FUV
•!Filtering with PHA threshold below 2
G160M/1577
data from program
not recommended
Blue
! PHA=[2,30]
Red ! 12424
PHA=[4,30]
Jan 19 2012
damaged
Timeline of Gain Sag Effects
Segment
B
B
B
B
A
B
B
Cenwave
FP-POS
1291
1309
1327
1318
1105
1300
1327
A
A
B
B
A
A
B
B
B
B
B
A
A
B
B
B
B
B
B
B
A
A
B
B
A
B
B
B
B
1105
Continuum
Continuum
1327
Continuum
1105
1291
1291
1309
Continuum
1318
1105
Continuum
1327
1309
1318
Continuum
1318
1291
1309
Continuum
Continuum
1300
1300
Continuum
Continuum
1300
Continuum
Continuum
Jan 19 2012
3
3
3
3
3
3
1
4
X=7000
X=1100
4
X=4500
1
1
4
1
X=3000
1
2
X=1500
2
4
4
X=4900
2
2
2
X=13000
X=11000
2
4
X=15000
X=8500
1
X=12000
X=14000
Days to PH=3
since 1/21/2011
161
208
292
372
425
428
441
446
495
519
536
540
547
575
577
591
601
618
624
628
636
653
671
690
711
719
748
764
787
800
826
826
830
907
1023
1240
Date
Jul-11 Rest of 2011
Aug-11
Nov-11
Jan-12 First half 2012
Mar-12
Mar-12
Apr-12
Apr-12
May-12
Jun-12
Jul-12 Second half 2012
Jul-12
Jul-12
Aug-12
Aug-12
Sep-12
Sep-12
Sep-12
Oct-12
Oct-12
Oct-12
Nov-12
Nov-12
Dec-12
Jan-13 2013
Jan-13
Feb-13
Feb-13
Mar-13
Apr-13
Apr-13
Apr-13
May-13
Jul-13
Nov-13
Jun-14 2014
TIPS Meeting - COS/FUV Lifetime
Gain sag holes currently
present in COS data
By July 2012 will loose part
of continuum in FUVA
HV
to be raised ~March to avoid this
By September 2012 will
lose part of continuum
in FUVB
Need to move to new
lifetime position before
September 2012
3
Moving to a New COS/FUV Lifetime Position:
What does it Mean?
Constraints on motion of aperture mechanism in Y direction prevents
PSA to be moved beyond +/− 6.0” from current position
Active area of the detector (in pixels)
Geocoronal
Lyα emission
Wavecal
location
Current Position at 0”
+6”
+3”
0”
−3”
−6”
COS FUV - Segment B (program 12678,G130M/1291) 17, 2012
JanJan
19 2012
TIPS Meeting - COS/FUV Lifetime
4 4
Parameters Considered in Choosing
New COS FUV Lifetime Position
In order to choose new lifetime position several factors were considered:
•
•
•
•
•
•
•
Resolution at each position
Projected lifetime at each position
Overall flat-field characteristics at each position
Effect of new position on subsequent positions
Use of wavelength calibration lamp spectra at each position
FUV dispersed-light target acquisition at each position
Impact of new lifetime position on aperture mechanism
•  Resolution and projected lifetime at each position were the most
critical parameters in choosing the new COS lifetime position
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
5
COS FUV Resolution:
Cross-Dispersion (Y) Offsets
•  Resolution at Y offsets inferred assuming R = 19,000 at
current position (0.0”)
•  Resolution at ±3.0” and ±6.0” measured from program
12678 (other values linearly interpolated from nearest
on-orbit measurements)
•  Ray-trace models and on-orbit data predict Y peak
between 0.0” and +3.0”
•  Positions above current position have better resolution
than similar positions below by ~ 20% (e.g., similar
resolution between +6.0” and −3.0”)
Assumes R0=19,000 at current pos. (0.0”)
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
X
Y
R
R/R0
0.0”
-6.0”
12000 0.63
0.0”
-5.0”
13000 0.68
0.0”
-4.0”
14000 0.74
0.0”
-3.5”
14500 0.76
0.0”
-3.0”
15000 0.79
0.0”
0.0”
19000 1
0.0”
+3.0”
18500 0.97
0.0”
+3.5”
17750 0.93
0.0”
+4.0”
17000 0.89
0.0”
+5.0”
15500 0.82
0.0”
+6.0”
14000 0.74
-3.0”
-3.0”
12000 0.63
+2.0”
-3.0”
15000 0.79
6
Resolution: Adding Dispersion (X) Offsets
•  X offset of 3.0” corresponds ~ to ½ FP-POS
-  Needed to interlace gain-sag holes due to Lyα airglow at each FP-POS
position
•  Mechanism soft stop in +X direction allows max offset of +2.0”
•  Data obtained at +2.0” and −3.0” offsets in X direction with a −3.0” offset
in Y (as part of characterization program 12678)
-  Resolution at (−3.0”,−3.0”) worse than (0”,−3.0”)
-  Resolution at (+2.0”,−3.0”) similar to (0”,−3.0”)
•  Ray-trace models support on-orbit results
-  In X, resolution peaks between 0.0” and +2.0”
-  Resolution with +2.0” offset in X direction similar to resolution with no offset
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
7
Resolution: Dispersion + Cross-Dispersion
Offset Examples
Effect of Y offsets in resolution
Effect of X offsets in resolution
(0.0”, 0.0”)
(0.0”,+3.0”)
(0.0”, −3.0”)
(−3.0”,−3.0”)
(0.0”, 0.0”)
(0.0”,−6.0”)
(0.0”, −3.0”)
(+2.0”,−3.0”)
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
8
Resolution: Summary
⇒  Peak of resolution between 0.0” and +3.0” in Y direction
•  Moves in Y > 0.0” lead to less resolution degradation than similar moves in
Y < 0.0” (by ~20%)
•  Ideally, would like to move to Y = +1.5” (peak as predicted by models).
•  Need to move further away from current position (at least 3.0”) to
avoid existing regions where gain has already sagged
•  Resolution at +3.0” similar to resolution at current location
•  Resolution at +6.0” similar to resolution at -3.0”
⇒  Peak of resolution between 0.0” and +2.0” in X direction
•  Offset of +3.0” slightly better to minimize gain sag effects in the first 1-2 yrs
•  Not feasible due to aperture mechanism constraints
•  Would likely lead to worse resolution
⇒  To optimize resolution best to move as close as possible to +3.0” in Y
(with or without X = +2.0” offset)
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
9
Projected Lifetime at Different Positions
The projected lifetime at each position is a function of the initial gain map
at the specific position, the detector usage, and how close the new
position is to the previous position
To determine the projected lifetime at each position:
–  Data obtained in 12676 were used to model the evolution of gain sag as a
function of time and position in the detector
–  Images were generated for pure Y moves and moves with a -3.0” and +2.0”
X offset
•  Histograms of the modal gain distribution were determined, for the regions where
the different gratings project the spectra, for each lifetime position
•  The % of pixels with modal gain ≤ 3 was determined for each case.
•  A modal gain of 3 corresponds to a 5% loss in flux
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
10
G160M/FUVB (0.0”,+3.5”)
G160M/FUVB (0.0”,+3.5”)
Jan 19
Dec
09,2012
2011
TIPS
COS
Meeting
Lifetime
- COS/FUV
Brainstorming
Lifetime
MODAL
GAIN
vs.
TIME
time = 1 yr (July 2013)
time = 3 yr (July 2015)
11
Summary: Analysis Modal Gain vs Time and Offset
•
Positions with Y > 0.0” allow us to mitigate gain sag effects for a longer
period of time than similar positions with Y < 0.0”
–  For FUVB, after ~3 years, the % of pixels with modal gain ≤ 3 is approx. the same
regardless of position, indicating that newly formed gain sag regions at that position
dominate other effects (except for G140L at -3.0”)
–  For FUVA, after ~3 years, the % of pixels with modal gain ≤ 3 is much larger for Y < 0.0”
than for similar Y > 0.0” positions
•  G160M spectra are narrower and fall lower in the detector than G130M, leading to asymmetric
gain sag structure from the currently used position
–  The evolution of the modal gain with time for positions at +3.5” or beyond is similar,
indicating that they are in a detector location not affected by the gain sag of the current
position
•  The position at +3.0” is not as pristine as that at +3.5”, and after ~3 years it has the double
amount of pixels with modal gain ≤ 3
•
Positions with X = -3.0” or X = +2.0” offsets are not significantly better at
mitigating gain sag than positions without X offsets
•  Slightly delay gain sag of the Lyα regions at the new position in the first 1-2 yrs
•  So not prolong the overall lifetime of the new position
–
In ~3 yrs the % of pix with PHA<3 is similar because gain sag at that location dominates
⇒  To optimize lifetime at the new position (due to gain sag) best to move to
+3.5” or beyond in the Y direction
(no huge benefit in moving in X direction)
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
12
Summary: Analysis of COS Lifetime Positions I
• To optimize resolution best to move as close as possible to +3.0” in Y (with or
without +2.0” X offset)
• To optimize lifetime at the new position (due to gain sag) best to move to +3.5” or
beyond in the Y direction (with or without X offset)
• X direction offset may be slightly advantageous only in the first 1-2 years because it
delays appearance of new gain sag holes
• Overall flat-field characteristics don’t change much as a function of Y position, but
positive moves are slightly preferred
•  In terms of bad pixel map, moves with Y > 0.0” are slightly preferred to moves with Y <
0.0”. However, no region is so badly affected as to have a significant weight in the
choice of the next lifetime position(s)
•  Positions in either the positive or negative Y directions lead to flat fields not
significantly different from that at current location
•  Geometric correction accuracy in X and Y comparable for lifetime positions between
-6.0” and +6.0” for both segments. No impact in choice of next lifetime position.
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
13
Summary: Analysis of COS Lifetime Positions II
•  To optimize the number of lifetime positions, best separation in Y between
positions is 3.0”-3.5”.
•  “Weighted” extraction can be used to reduce separation
•
Wavecal operations affected only for most extreme positions
•  Tagflash operations will have to be changed for lifetime positions beyond Y = +5.0”. All
other positions can use tagflash without modifications
•  Only lifetime positions at Y = -4.0” or lower will have to deal with WCA spectrum falling
on region of detector potentially affected by gain sag
•  Target acquisition algorithm will have to be modified for lifetime position at Y =
-6.0”.
•  Other lifetime positions require no modifications
•  The lifetime of the COS aperture mechanism does not constrain the lifetime
positions
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
14
Constraining the Next Lifetime Position
Y Positions (“)
-6
-5
-4
-3.5
-3
-2
-1
0
+1
+2
+3
+3.5
+4
+5
+6
Resolution
Projected
Lifetime
Flat Field
Subsequent
positions
Wavecal
Target Acquisition
Aperture
Mechanism
Best location for next
lifetime position at Y=+3.5”
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
15
New COS FUV Lifetime Position and Beyond
•  The following scenario could be considered for future positions (only next
position selected; future positions to be evaluated at later time):
Lifetime position 2 at (0”,+3.5”)
Lifetime position 3 at (0”,−3.0”)
Lifetime position 4 at (0”,+6.0”) – TAGFLASH changes needed to avoid FCA leak issue –
•  To extend lifetime of each position
-  Start at low HV and incrementally increase HV to overcome gain sag (modal gain
increases by ~ 3 PHA bins per 8 HV steps)
-  Decrease size of extraction boxes currently used
-  Use some sort of “weighted extraction” to avoid CALCOS throwing out entire columns
when only a few pixels at the edges of extraction box have sagged below the
threshold (due to overlap with previously sagged position)
•  These 3 additional positions will enable us to operate for additional ~ 9
years while mitigating gain sag effects
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
16
END
Jan 19 2012
TIPS Meeting - COS/FUV Lifetime
17
TIPS/JIM January 19th 2011
£  Upcoming Enhancements to the HST Archive – Mark Kyprianou
£  Using Exchange Leave Calendars at STScI – Glenn Miller
Next TIPS/JIM February 16th
Upcoming Enhancements to the HST Archive!
Mark Kyprianou!
Operations and Engineering Division!
Data System Branch!
Enhancements in DMS!
l
JWST requirements!
n  Define a better archive"
l
HST Mission Office!
n  Identify weaknesses/areas that could be improved"
n  Allocate resources to implement enhancements in a few areas"
l
A “win” for all missions!
n  Common code to support"
n  Common system to operate"
n  Better services to our customers"
l
Target areas for HST enhancements:!
n  Workflow Manager"
n  Reprocessing"
n  Online Cache"
n  Distribution/UI"
01/19/2012!
HST DMS Enhancements!
2!
"
"
"
Workflow Manager!
HST Workflow Manager – today itʼs OPUS!
l
OPUS Workflow manager in use since 1995!
l
All current HST level-0 -> level-2 data processing is performed
using OPUS pipelines.!
l
Why change? There are significant risks with using OPUS
throughout the remaining HST lifetime!
n
Reliability – OPUS GUIs (OMG, PMG) are “fragile”"
-  Susceptible to failure with additional network security"
-  Use Java “thick-client” application technology (NOT web-friendly)"
-  Have not been rebuilt for Windows platform in many years"
HST Workflow Manager - Alternatives!
n
A JWST Workflow Manager trade study has just been completed
and recommends a new workflow technology for JWST: Condor /
Open Workflow Layer (OWL)"
n
Transition HST from OPUS to Condor/OWL?"
-  Provides a “technology refresh” which should serve HST throughout its
remaining lifetime"
–  Condor has a huge world-wide user base and undergoes continuous
development and improvement"
-  Allows STScI pipeline operations team to focus on a single workflow
manager system, rather than learn to operate more than one"
-  Provides significant “upside” with flexibility for taking advantage of
distributed computing resources, both on-site and off"
"
n
Workflow Framework Trade Study (FOO - Future of OPUS)
https://trac.stsci.edu/trac/DMS/wiki/FutureOfOpus"
Reprocessing!
Rationale for Reprocessing!
l
Calibration improves over time!
n
Science instrument performance better understood"
-  Reflected in improved calibration algorithms and reference files"
l
Data better understood over time!
n
l
01/19/2012!
Additional keywords and improved data formats"
Pipeline software error corrections!
HST DMS Enhancements!
7!
"
"
"
On the Fly Recalibration: OTFR!
l
l
01/19/2012!
Advantages!
n
The user gets the benefit of the very latest data processing and
calibration enhancements at the time of their archive retrieval."
n
Less archive storage since calibrated data products not on disk"
n
Unpopular data do not get reprocessed"
Disadvantages!
n
Delay in retrieval while reprocessing, could be substantial if there is
a large retrieval queue"
n
No direct access to data"
n
All data not accessible through VO protocols"
n
Popular data get identically reprocessed many times"
HST DMS Enhancements!
8!
"
"
"
Reprocess on Change!
l
Advantages!
n
Rapid data retrieval through direct synchronous access or batch
request"
n
Data accessible through VO protocols"
-  Allows for data mining"
l
Disadvantages!
n
Requires development of more complex reprocessing software
system"
-  Logic needed for when to initiate reprocessing and where to start in the
pipeline"
01/19/2012!
HST DMS Enhancements!
9!
"
"
"
Reprocessing Concept (1/3)!
l
The Reprocessing System will automatically recalibrate affected
observations when updates to calibration reference files or the
calibration software are approved and released.!
n
l
The Reprocessing System will monitor changes in calibration
reference files and software.!
n
01/19/2012!
Other improvements to the quality of data products may trigger
reprocessing."
The Calibration Reference Data System will track changes to the
calibration reference files"
HST DMS Enhancements!
10!
"
"
"
Reprocessing Concept (2/3)!
l
The latest version of all data are stored in the archive.!
l
Reprocessed data products replace their previous version in the
primary archive. !
l
If the Archive User Interface indicates that the data being
requested do not have best calibration, archive users will be
notified prior to retrieval.!
n
Archive users accept existing calibration or wait until calibration is
updated."
"
01/19/2012!
HST DMS Enhancements!
11!
"
"
"
Reprocessing Concept (3/3)!
l
01/19/2012!
The order of data processing will take into account items such as: !
n
Data designated to be processed immediately.
n
Processing on initial receipt of data from the telescope.
n
Reprocessing of an observation less than one year from execution
requested by an archive user.
n
Reprocessing of an observation more than one year from execution
requested by an archive user.
n
Reprocessing of data less than one year from execution.
n
Reprocessing of data more than one year from execution.
HST DMS Enhancements!
12!
"
"
"
Archive User Decision Tree for Direct Download!
User waits"
01/19/2012!
HST DMS Enhancements!
13!
!
Data Storage:
Online Cache!
Storage Broker Concept!
l
Optimizes management of large scale distributed data storage
resources!
l
Provides a uniform interface to heterogeneous data storage
resources over a network !
l
n
ingest (adding files to the system)"
n
accessing files"
n
security"
Uses common metadata for file storage and location!
n
Utilizes a database schema for mapping of the logical file layer to
the physical disk locations on storage media."
l
Provides independence from the hardware platforms (mainframes,
intermediate systems, servers, PCs).!
l
Provides transparent use of public network protocols (SFTP,
HTTP, etc.)!
l
Simplifies file exchange between applications and mirror sites!
01/19/2012!
HST DMS Enhancements!
15!
"
"
"
Data Storage Key Features!
!
l
l
The Storage Broker (SB) supports:!
n
Internal archive RAID based disk storage for long term data
preservation (Primary Data Store)."
n
An online file storage of files for fast, immediate access."
n
An offline, offsite data backup of the file storage (Safestore)."
The SB provides online access to the latest version of the
processed data.!
!
01/19/2012!
HST DMS Enhancements!
16!
"
"
"
Distribution and Archive User Interface!
Data Distribution Concept!
l
l
l
01/19/2012!
There are two complementary concepts for data distribution.!
n
Batch distribution"
n
Direct distribution"
Batch distribution!
n
XML request generated by Archive User Interface and passed to
Distribution."
n
No further user interaction is needed once the request is submitted."
Direct distribution!
n
User has direct access to files through URL."
n
Supports VO services."
n
Necessary for data mining."
HST DMS Enhancements!
18!
"
"
"
Archive User Interface Concept!
l
The Archive Users Interface (AUI) will provide means to search for
data including Program/PI searches; spatial, time and wavelength
searches.!
l
After users identify data of interest the AUI will provide an option
of download method and prompt for authentication / authorization
information for use with proprietary data. !
l
AUI will provide the status of the requested data (e.g. best
calibration available or data are in reprocessing queue.) and
permit user to select if they want to wait for new data. !
l
Distribution shall record metrics for user transactions, such as IP
address, user ID, files selected, distribution mode and format, and
download size and time.!
01/19/2012!
HST DMS Enhancements!
19!
"
"
"
TIPS/JIM January 19th 2011
£  Using Exchange Leave Calendars at STScI – Glenn Miller
Next TIPS/JIM February 16th
Using Exchange Leave Calendars at STScI Glenn Miller 19 January 2012 Benefits of Leave Calendars *  Do not need VPN or web proxy *  New capability: *  Repeating events, e.g. every other Friday off *  Partial days, e.g. come in late or leave early *  Most important reason is that leave calendar is integrated with your personal calendar. *  1 click: there when you need it, gone when you don’t Old system is showing its age *  Sluggish performance *  Quirky: new leave requests occasionally fail *  Server to be decommissioned in June Transition *  Each division and office will set its transition date *  INS adopts new calendars as of 1 March 2012 *  Old system will be shut off in June Documentation/Help *  ITSD support web site:
http://www.stsci.edu/institute/itsd/collaboration/
exchange/leave *  Works with *
*
*
*
Outlook: Mac and Windows Outlook Web App iCal Thunderbird with Lightning add-­‐on (experimental) Demo *  Invite us to demo at your branch/team meeting