Understanding Rasters
By Joseph Collins-Unruh
-1-
Table of Contents
Introduction and Statement of Purpose...........................................................................3
General Properties of Rasters..........................................................................................4
Height and Width........................................................................................................4
Data Type....................................................................................................................4
Vertical Order.............................................................................................................4
Endianness (Byte Order).............................................................................................5
Raster Graphics...............................................................................................................6
Bands..........................................................................................................................6
Color Model................................................................................................................6
Interleave.....................................................................................................................7
Geographical Rasters......................................................................................................9
Cell spacing.................................................................................................................9
Nodata Values (Null Values)......................................................................................9
Cell Origin..................................................................................................................9
Raster Origin.............................................................................................................10
Data Structures..............................................................................................................10
Continuous................................................................................................................10
Classified...................................................................................................................10
Compression.................................................................................................................11
Run Length Encoding (RLE)....................................................................................11
LZW Compression....................................................................................................11
Case Study: ENVI .hdr format......................................................................................12
Header File................................................................................................................12
Binary File................................................................................................................13
References.....................................................................................................................14
-2-
Introduction and Statement of Purpose
When I started working at Safe Software I had no idea what a raster Dataset was. I spent
hours scanning the internet searching for a precise definition of raster.
Unfortunately, resources were scarce and I never got beyond the basic definition, “A grid
of cells or pixels, with each pixel containing a single value.” While simple enough, this
definition taught me nothing about how those pixels were represented or what you could
do with them.
This guide is my attempt to save you trouble by giving you a solid reference to the terms
and ideas involved with rasters. It is an introduction to what a raster is so that you can
start using them without worrying about terminology.
Although the document was designed for programmers I’ve put a great deal of effort into
making it understandable by everyone. Non-programmers who read this can safely ignore
the points that refer to programming languages and techniques. Also, the ENVI .hdr
format case study in the last section will be of little use to non-programmers.
This is by no means a complete reference. It is comprised only of the main subset of what
I was exposed to while working at safe software, augmented significantly by research
conducted at the time of writing to ensure accuracy. There are certainly other raster
implementations and models beyond what I describe here. However, the vast majority of
raster formats that I worked with while at Safe Software could be described by this model
or an extension of this model with additional aspects such as compression methods or
color models.
-3-
General Properties of Rasters
Height and Width
The height and width of a raster is, quite simply, the number of
rows and columns contained in the grid. Height and width are
sometimes referred to as lines and samples respectively. The
individual entries of a raster are called cells or, in the case of
raster graphics, pixels.
Figure 1
Raster Data
Type
8-bit unsigned
Integer
8-bit signed
Integer
16-bit unsigned
integer
16-bit signed
integer
32-bit unsigned
integer
32-bit integer
32-bit real/float
64-bit real/float
Corresponding
C++ Data Type
unsigned char
char (character)
unsigned short
short
unsigned int
int
float
double
Data Type
A raster’s data type is the type of information
stored in each cell of the raster. Raster data types
are almost always numeric but practically any data
type is possible.
The standard for representing numeric data types
has the form:
[allocation]-bit [type]
Where the allocation is the number of bits that the
data type occupies and the type is one of: integer,
unsigned integer (integers with no negative
values), or real (numbers with decimals).
The table to the left shows many common numeric
raster data types and the associated C/C++ data
types.
Table 1
Common numeric raster data types
Vertical Order
The vertical order of a raster specifies what row of the raster you
start reading from. For a vertical order that is top-down you start
reading across the top row of the raster. When you’ve read all of
the columns of the top row you move on the second row down, and
so on. The bottom-up vertical order is exactly the opposite. Here,
you read all of the columns from the bottom row first then move
on to the second row up.
Figure 3
Bottom-Up Vertical
Ordering
-4-
Endianness (Byte Order)
The endianness of a raster describes how the raster data is stored in memory. Endianness
is not a property inherent to rasters, but rather a consequence of working with rasters on
different computer systems with different standards for storing numeric data. The two
most common endianness’ are ‘little endian’ and ‘big endian’. Little endiandiannes is
sometimes called ‘least significant byte’ and occurs most commonly on machines based
on the Intel architecture. Big Endianness is sometimes called ‘most significant byte’ and
occurs most commonly on machines based on the Motorola and SPARC architectures.
The technical details that are important for dealing with endianness programmatically
will not be dealt with here. Programmers should already know the specifics of endianness
and Non-programmers shouldn’t worry about endianness as it isn’t critical in
understanding rasters.
-5-
Raster Graphics
The most common type of raster is a digital image or graphical raster. Raster graphics are
a computers primary method for displaying images and pictures.
Most of the terms covered here are not limited solely to raster graphics. The ideas behind
terms like band and interleave are most easily represented in the context of images.
Bands
The number of bands that a raster has specifies the number of values for each cell or pixel
in the raster grid. For example, a RGB color image includes one band for each color
red(R), green(G) and blue(B). Another word that is commonly used for bands is layers.
Color Model
The color model describes how the color of individual pixels is determined from the
values at that point in the different bands. For single banded color models like grayscale,
the pixel value simply represents a range of light values. However, in order to produce
colors, more complex models are required.
Mono Color:
The most basic color model for rasters is mono color. With mono color each
pixel can be represented by a single bit. When the bit is 0 then the pixel is
black; when set to 1 the pixel is white.
Figure 4
Grayscale:
The next color model is grayscale. Grayscale images can use any data type with
the minimum value representing black and the highest value representing white.
However, almost all grayscale image formats use 8-bit unsigned integers with
some using 16-bit unsigned integers for a more precise representation of color. In
figure 5, to the right, the 8-bit unsigned integer data type is used for simplicity. So,
in this case black is represented by 0 and white by 255. (starts counting from 0)
RGB (Red Green Blue):
Figure 5
The RGB color model requires that the raster have 3 bands, with one
band for each of the three RGB colors; Red, Green and Blue. Like
Grayscale each color typically uses 8-bit unsigned integers to represent
the color rage. The difference is that each cell ranges from 0 being
black and 255 being the associated color. The resulting pixel uses
additive color mixing to obtain the final color, as shown in
figures 6 and 7.
Figure 6
Additive Color Mixing
Figure 7
-6-
RGBA (Red Green Blue Alpha):
The RGBA color model is exactly like that of RGB, except there is an additional band
representing the pixel’s alpha value or transparency. For example, if the alpha value for a
pixel is 0 then the pixel will not be visible at all and you should see what is behind the
image at that pixel. If, on the other hand, the alpha value is 255, then the pixel will be
opaque and it will appear as a RGB pixel would.
CMYK (Cyan Magenta Yellow Black):
Like RGB and RGBA, CMYK is another type of color mixing color
model, except CMYK uses subtractive color mixing instead. RGB
color is analogous to the mixing of colors of light, with black
representing the absence of light. CMYK on the other hand, is
analogous to the mixing of pigments, where black represents all
pigments combined.
Figure 8
Subtractive Color Mixing
Interleave
The raster’s interleave specifies how the pixels and bands in a
raster are arranged. Different interleaves allow you to arrange the
data non-contiguously to optimize certain methods of access.
Interleave is only relevant in multi-banded rasters; the cell values
for rasters with one band are written sequentially. The main
interleaves are BIL, BIP, BSQ and Tiled, which are defined below.
Figure 9
Base image for
interleave examples
To the right, Figure 7 is a 4x4 image which is the basis for
demonstrating how each of the interleaves is organized with the
RGB color model. In each case the overall size of the image will triple since all three
bands of each pixel will be shown. Also, the values should all be read from left to right,
top to bottom.
Band-Interleaved-by-Pixel (BIP): This
is the most intuitive interleave, where for
a given pixel, the values for all bands are
side by side. This interleave is most
efficient for determining all of the values
for a given pixel since they are all side
by side.
Figure 10: BIP
Each block outlined in white represents one
pixel. Each line is one line of the image.
-7-
Band-Interleaved-by-Line (BIL): With
BIL each raster row is made up of 3
band rows, one for each color
component. BIL interleave is most
efficient for accessing all bands of a
given row sequentially.
Figure 11: BIL
Each block outlined in white represents one
band of one row. Each line represents one line
of the image.
Band-Sequential (BSQ) Interleave: With BSQ interleave the entirety
of the first band is written first, followed by the second band, and so
on. BSQ Interleave is most efficient for accessing each band
sequentially. This was the most commonly used interleave that I
encountered while working with geographic data since it efficiently
accesses all of the values for a single band.
Tiled Interleave: With this interleave a raster is
divided into some number of equal sized blocks,
each written sequentially from left to right and top
to bottom. Typically, each tile is interleaved with
BSQ. For the example on the right, figure 9 was
divided into 4 2x2 blocks.
Figure 12: BSQ
Each block outlined in white
represents one band of the image
Figure 13: Tiled Interleave
Each block outlined in white
represents one 2x2 tile
-8-
Geographical Rasters
Next to image rasters, geographical rasters are the most commonly encountered type of
rasters. A geographical raster essentially divides a real world place into a grid and assigns
some element of the real world to each cell of the grid. Often the data that is stored is the
elevation at the cells position, but it can be almost anything that varies continuously or
discretely over a geographical range. Other common geographical rasters measure
temperature, soil composition or even land ownership.
Cell spacing
A raster’s cell spacing describes the area that each cell covers in the real world. This cell
spacing indirectly determines the smallest size an object can be and still be visible in the
raster. For example, if a raster’s cell spacing measures 3 meters in the x and y direction
for a total of 9 square meters, in order for the feature to be observable it must take up
most of the area of the cell or be at least 4.5 meters square. A raster’s cell spacing is often
described as the resolution of the raster since it determines how much detail can be
observed.
It’s important to note that there is no requirement that the area represented by cell has to
be square. There are many formats, such as GeoTiff and ENVI .hdr, which is described
below, where the area covered by a cell can be defined as rectangular.
Nodata Values (Null Values)
Often when measuring values over a geographical range it is not possible to get data for
every cell of the grid. For example, when measuring soil composition in the vicinity of a
lake it’s often not possible or relevant to get the data from below the surface of the lake.
In those cases, you would enter a nodata value in that position. Typically, if a raster
format supports nodata values it will reserve one extreme value that is rarely used to be
the nodata value. For example, USGS DEM format uses -32768, which is the smallest
possible integer value as its nodata value.
Nodata values are often used in images as well, for an example of this all you have to do
is look at your computers desktop. There, the icons only have image data in some areas.
All of the areas where you can see the background visible within icon, like the middle of
the e in the Internet Explorer® icon, are actually nodata values.
Cell Origin
The raster origin describes where within the cell the values are taken. Often when you are
taking measurements the cell values of a raster, you will only take data from one position
within each cell. That position is known as the cell origin. Other times, value contained
within a cell will be the average value of the area the cell covers. In these cases the cell
origin is less important.
-9-
Raster Origin
The raster’s origin is where, geographically, the raster is located. It’s what allows you to
map the cell’s values to a location in the real world. The rasters origin is taken relative to
some position in the raster. Often that position is the lower left corner of the raster, but it
varies depending on the format.
Data Structures
Continuous
This is the most basic and straightforward raster representation. It is
also the structure. Here, the value of each cell is sequentially written
one right after the other. This structure has a flexibility advantage
over other data structures since the values each cell can have aren’t
limited and is very simple to locate the value for a specific cell.
However, it is not very space efficient without compression.
Figure 14
Continuous Raster
Classified
Figure 15
Classified version of figure 14
With a classified data structure a map of possible cell
values in the raster is first created. Then, the raster body
is created with the actual cell values replaced by the
index of those values in the map. Classified rasters are
space efficient if there are few differing cell values. You
also maintain the ability to quickly locate the value of a
given cell. Web formats like PNG and GIF use
classified rasters with compression since it allows them
to put 24-bit or 32-bit images into 8-bit or 16-bit space.
- 10 -
Compression
Compression is used to reduce the final size of a raster by applying an algorithm to the
raster that allows you to change the way that the data is represented. With rasters that are
sufficiently large it is often vital that the raster be compressed. It can take a great deal
more time load an uncompressed raster from a hard drive or the internet into memory
than it takes to uncompress a raster already in memory.
Compression algorithms are often divided into two types, lossy and lossless. With lossy
compression algorithms the correctness of the data is often sacrificed in order to
compress a raster in to as small a space as possible. Lossless algorithms, on the other
hand, ensure that the data remains completely intact and because of this can’t approach
the compression ratios you can achieve with a lossy algorithm.
The majority of efficient algorithms, both lossy and lossless, used these days require a
strong mathematical background to understand, much less implement. Unfortunately,
because of this, many widely used and important compression algorithms won’t be
included here. Below, a few relatively simple lossless compression algorithms are
explained
Run Length Encoding (RLE)
Run Length Encoding is probably the most simple and naïve compression algorithm there
is. With run length encoding, instead of specifying a value at every single location in the
raster, you specify values as they appear and report how many occurrences of that value
there are consecutively. RLE encoding is can save a great deal of space if many
sequential cells contain the same values.
LZW Compression
LZW compression is a type of compression that allows you to optimize for space based
on sequences of bytes occurring frequently. First a dictionary of the 255 possible 8-bit
bytes is created and a code is assigned to each one. Then it traverses the binary file
adding character sequences that it finds that are not already in the dictionary and assigns
codes to each of those. The dictionary does not have to be transmitted with the
compressed data since the decompressor can build it in the same way the compressor did.
This compression algorithm is used by the GIF format.
- 11 -
Case Study: ENVI .hdr format
There are a number of features of the ENVI header format that make it a good
introductory raster format. ENVI has a simple structure; there is no compression or other
derivation from the standard continuous representation. This makes it easy to find
individual cell values within the binary file. It also has a lot of flexibility in the kinds of
data that can be stored. EVNI can have one of 6 data types for storing numerical data, it
can also store color images in RGB format. There is even the option to specify
geographic location information.
An ENVI raster is divided into two separate files. One is a header file which stores all of
the raster properties; such as its dimensions, data type and interleave. The second is the
binary file which stores all of the actual raster data.
Header File
The header is a plain ASCII text file that has the .hdr extension. Each property of the
raster is placed on a separate line in the format:
[keyword] = [value]
Below is a table of keywords and their possible values.
ENVI .hdr Keyword
samples
lines
bands
interleave
data type
byte order
(optional)
map info
(optional)
Desciption
Number of columns in raster
Number of rows in raster
Number of bands. If set to 3 or 4 most readers
will interpret as RGB or RGBA color.
Value can be one of bsq, bil or bip
Possible values:
1 – 8-bit unsigned integer
2 – 16-bit signed integer
3 – 32-bit signed integer
4 – 32-bit real
5 – 64-bit real
12 – 16-bit unsigned integer
13 - 32-bit unsigned integer
Possible values:
0 – little endian (default)
1 – big endian
This keyword provides geographical positioning.
See next page for complete description.
Table 2
ENVI .hdr header file keywords
- 12 -
Map Info Keyword:
The map info keyword will have the following format:
map info = {[projection], [reference cell x coordinate],
[reference cell y coordinate], [cell easting], [cell northing],
[x cell size], [y cell size], [projection zone], [direction]}
[projection]: This is the projection system that the raster uses, will almost always be
UTM in ENVI .hdr files.
[reference pixel x/y coordinate]: These two fields specify the grid location of the
first pixel in the binary file. This will be 1,1 for top-down vertical order or 1,<number
of columns> for bottom-up.
[pixel easting]: This is the number of meters east of the zone origin the reference
pixel is located.
[pixel northing]: This is the number of meters north of the projection zone origin the
reference pixel is located.
[x/y pixel size]: This is the size/resolution of each cell in meters.
[projection zone]: This is the projection zone that the raster is located in.
[direction](UTM only): Because there are two of every UTM projection zone – one
on each side of the equator – you must specify which one by entering North or South.
Binary File
After reading the header file you now know all of the raster’s properties relevant to
reading the binary image data. The ENVI .hdr format is a standard continuous format, so
accessing individual cells of the raster is straightforward.
Below, I’ve included a table of formulae for accessing the individual cell values for a
given row, column and band from the binary file based on the header properties. The
table assumes that row, column and band are counted starting from zero.
Interleave
One band, no
interleave
BIP
BIL
BSQ
Formula for file position
(row*samples + column)*allocation
(row*samples + column)*allocation*bands+band
(row*bands+band*column)*samples*allocation
(lines*band*samples+row*samples + column)*allocation
Table 3
- 13 -
References
1. Complete description of LZW Compression algorithm in Dr. Dobb’s Journal by
Mark Nelson. Created October 1989. Accessed January 2006.
http://www.dogma.net/markn/articles/lzw/lzw.htm
2. Reference in creating the ENVI .hdr Case study:
RSI Documentation, “ENVI Reader for ArcGIS® User’s Guide”, March 2003,
RSI Inc.
3. Reference for information about color spaces and color models.
Unknown Author
http://en.wikipedia.org/wiki/Color_space
Updated January 11, 2006: Accessed January 11, 2006
4. Geographical raster information.
http://www.sli.unimelb.edu.au/gisweb/GISModule/GIST_Raster.htm
All text and images in this document were created by Joseph Collins-Unruh.
- 14 -