What is FITS?
• FITS = Flexible Image Transport System
• Invented ~1980 to transfer Radio Astronomy datasets
from one observatory to another
• Format now used by most astronomical software
• Widely used as data archive format
• FITSIO and other libraries developed to read/write FITS
efficiently using C, Fortran, Perl, Python, Java…
• Always big-endian (so portable)
• Format defined to the last bit in primary literature.
• Images and binary tables fairly space-efficient.
FITS Structure in outline
• Any number of Header-Data units (HDUs)
• Header has metadata: lines of 80 characters
NAME = value
/[units] other commentary
– A few reserved names, e.g. DATE-OBS, no rules otherwise
• Data section can be Image or Table
– Image:
• 8, 16, 32-bit integers, 32 or 64-bit floats
• Up to 999 dimensions (mostly 2 or 3 dimensions)
– Table
• up to 999 columns, up to 2 billion rows
• 8/16/32-bit integers, 32/64-bit floats, strings, logical, 64/128bit complex.
• Vector columns (optional variable length vectors)
• Null values can be defined
What’s wrong with FITS
• Flat structure for metadata, single name-space, only 8character names
• Blocked in units of 2880 bytes
• Length of data unit must be declared in header unit, so
cannot stream output
• No structure to HDUs, just a simple list
• No direct access to HDUs, have to decode each header
in turn to find size of data unit.
• Always big-endian, requires byte-swap on most popular
platforms.
• Not quite fully 64-bit clean
What’s wrong with VOTable
• Three alternative ways of including binary:
– Pure XML but very verbose, 4-byte real may turn into
~24 bytes, e.g.
<TD>-0.12345678E-04</TD>
– Blob encoded in Base64
– Pointer to separate FITS file
• Possible loss of linkage
• Duplication of header data
• Tabular datasets only, no format yet for images,
spectra, etc.
• Cannot use direct offset to required row or field
What we want
• Flexible header to contain metadata
– XML very suitable
• Data section to contain heterogeneous
data (table) or homogeneous (image etc)
but to be efficient, evenly spaced
– Binary formats as in FITS very suitable
– Endian-neutral
– Suitable for streaming data
VOTable++ or FITS++?
• Use MIME/DIME to have separate sections of
file, as proposed for BinX, and have:
– XML-based header, very similar to VOTable
– Binary data units to follow, similar to FITS
• Use BinX/DFDL in header to describe data unit.
• Can choose big or little-endian when file created
• Allow streaming format for tables, no pre-defined
length.
• Allow FITS HDU as special case of data unit for
compatibility.
• Make fully 64-bit clean for tables >2Grows.