GSSC Data Servers for DC2 Tom Stephens GSSC Database Programmer

advertisement

GSSC Data Servers for DC2

Tom Stephens

GSSC Database Programmer

June 27-29, 2005 - DC2 Software Workshop - 1

Data To Be Served

LAT Photon Data (FT1)

LAT Pointing, Livetime and Mode History Data (FT2)

Gamma-ray Burst Data

LAT Source Catalog

Pulsar Ephemeredes (D4)

 

Calibration Database (CALDB) LAT Event Data (FT0) – maybe

June 27-29, 2005 - DC2 Software Workshop - 2

Data Access

Data access to the scientific data will be provided via the GSSC’s website at http://glast.gsfc.nasa.gov/ssc/data/access/DC2

Web page will provide

Link to download Source Catalog

Link to Burst Data website

– –

Link to download CALDB files Link to LAT Event, Photon, and Pointing database interface

Nothing there at the moment

June 27-29, 2005 - DC2 Software Workshop - 3

Source Catalog and Pulsar Ephem.

The LAT source catalog and Pulsar Ephemeredes database are each a single FITS file.

The link on the main data access page will point directly to the FITS file for downloading.

Users may download it and use it as they desire.

June 27-29, 2005 - DC2 Software Workshop - 4

Gamma-ray Burst Data

 

We will provide data for each burst “detected” “Detected” means that the burst crossed the GBM and/or LAT sensitivity threshold

 

For each burst a designation will be generated of the form “DC2-YYMMDDFFF” indicating that it is a DC2 burst and giving its time in the DC2 data set For each burst, a “Burst Data Package” will be constructed containing the GBM and LAT data for the time period and position of the burst

Information about each burst will be made accessible through a website listing the bursts and basic burst data

What would you like to see?

This will include a link to the data package.

June 27-29, 2005 - DC2 Software Workshop - 5

CALDB

The Calibration Database is a set of FITS data files stored in a predefined directory structure

This will be packaged into a single tarball and the link on the main data page will provide access to download the data

Once downloaded the user can unpack the data into the appropriate location in their science tools installation

For DC2 this will only be the LAT calibration files.

June 27-29, 2005 - DC2 Software Workshop - 6

LAT Photon & Pointing Data

Access to this data is provided by a set of custom data servers developed at the GSSC

The link on the main data access page will take you to the data server web interface at http://glast.gsfc.nasa.gov/cgi-bin/ssc/DC2/LATDataQuery.cgi

This page will allow you query the Photon and Pointing (and Event if it is available) data simultaneously

Allowed selection criteria are

Object name or position (RA, Dec)

Size of search area

– –

Time Energy

June 27-29, 2005 - DC2 Software Workshop - 7

Selection Criteria – Position

Select center point for search region

J2000, B1950, or galactic coordinates in comma separated list

Or enter name of object and choose Simbad name resolver

 

Choose shape of region

– –

Circle – must specify radius Box – must specify width, height and rotation angle in comma separated list

Default is circular region with 15 ° radius in J2000 coordinates

If nothing entered in position field, an all-sky search is performed

June 27-29, 2005 - DC2 Software Workshop - 8

Selection Criteria – Time

May choose from Gregorian date, MJD or Mission Elapsed Time formats. The default is Gregorian date.

May specify start time and optionally end time in comma separated list

If no end time specified, returns start time + 6 months or until end of data set which ever is less.

 

If no times specified, last 6 months of data returned Special keyword “START” and “END” provided representing the first and last data entries currently in the database.

June 27-29, 2005 - DC2 Software Workshop - 9

Selection Criteria – Energy & DB

Energy

Units are in MeV

Can specify a lower and optional upper energy limit in comma separated list.

If nothing is specified, will return all events from entire energy range of the LAT.

Database Choice

May select from any combination of Event, Photon and Spacecraft database via checkboxes at bottom of page

Searches are performed simultaneously

Default is Photon and Spacecraft

June 27-29, 2005 - DC2 Software Workshop - 10

Box Searches – How to Implement

 

Option 1 – Simple RA/Dec boxes

– –

Easy to implement as limits on the coordinates – width and height translate to simple offsets from the central position Not truly rectangular regions on the sky – degenerate to triangles at the poles

Implementing in other coordinate systems not as easy with current data file definitions (only RA & Dec in FT1 file)

Option 2 – Rectangular regions projected onto sky

More difficult to implement (but it’s just math and computers are fast)

Width and height would determine vertices of box and then project box onto celestial sphere to define region

Region are true spherical rectangles and would convert to a rectangular image when using a tangential projection to create maps.

Much easier to implement rotations and searches in other coordinate systems (i.e. galactic)

June 27-29, 2005 - DC2 Software Workshop - 11

Query Submission Screenshot

June 27-29, 2005 - DC2 Software Workshop - 12

Submission Response

Once the query parameters are submitted the system quickly returns with a response

Result depends on the type of query

All sky query with only a cut on time

• •

The page will indicate that the query is complete, give the query ID number and provide a link to the query results page The data for all-sky searches is pre-packaged in one week intervals and an all sky query returns a list of the files covering the time period requested.

All other queries

The page will indicate the estimated amount of time until the query completes, give the query ID number and provide a link to the query results page.

Will also provide an opportunity to enter an e-mail address to be notified when query is done (mainly for event data as the searches may be long)

June 27-29, 2005 - DC2 Software Workshop - 13

Submission Response Screenshot

June 27-29, 2005 - DC2 Software Workshop - 14

Query Results

Following the link on the previous page will take you to the results page

This page presents the following information about the query results

– –

The query ID The query parameters used to extract the data

– –

The estimated time remaining (if any) until the query is finished A table of the available data files from the query with their size and number of events contained and a link to begin downloading the files

It also provides

Another opportunity to enter your e-mail if the wait is still long

A field to enter the query ID of a different query to get its results.

Users will use the links provided to download the files to their computers.

June 27-29, 2005 - DC2 Software Workshop - 15

Query Results – What You Get

Time range only searches

List of files, each covering ~one week of all-sky data, that cover the time range specified by the selection criteria

There will be more data than you asked for at the beginning and end of the time period as no cuts are made on the data.

All other searches

List of files containing the data requested.

Broken up into smaller files if the data volume gets large

• •

Currently set to limit file size to ~200 MB – no real impact on DC2 Should this be set to a higher or lower limit?

June 27-29, 2005 - DC2 Software Workshop - 16

Query Results Screenshot

June 27-29, 2005 - DC2 Software Workshop - 17

Single CPU Performance

June 27-29, 2005 - DC2 Software Workshop - 18

Single CPU Performance

Searches on the Galactic Center Searches on Galactic Anti-center region June 27-29, 2005 - DC2 Software Workshop - 19

Event Data

  

The content of the event database is still ill-defined We may have an “event” database for DC2 but

It will not have 10-100x the number of events as the photon database

It will have the same entries but with more data than just the standard FT1 event summary data. Probably a cleaned up version of the Merit Ntuple

• •

Duplicates removed (i.e. don’t need Energy and log(Energy)) Variable stored in proper formats – not everything as a double

Access provided by the same interface as the Photon data but by checking the Event data box instead of or in addition to the Photon data box at the bottom of the page.

June 27-29, 2005 - DC2 Software Workshop - 20

Discussion

Additional data products

Contents of burst page data table

Additional search parameters for LAT photon and event database

Box search implementation

(no?) Need for Event database prototype

June 27-29, 2005 - DC2 Software Workshop - 21

Download