Brief overview of the SASRI Weather
Data Acquisition and Processing System
July 2013
P. Sithole, Agrometeorologist, SASRI
Introduction
Weather information is important for the everyday agronomic crop management and for agricultural
research. The SASRI met office collects and processes weather data from various automatic weather
stations (AWS), manual met stations (MWS) and rainfall stations (RS). The variables recorded at AWS
are rainfall, air temperature, relative humidity, solar radiation and wind speed. MWS record
everything recorded by AWS, but record sunshine duration instead of solar radiation. MWS also
record soil temperature and Class A Pan evaporation while only rainfall is recorded at RS. Variables
like soil water content, reference evapotranspiration, reference sugarcane stalk growth and many
others are derived from the primary records.
Most of the AWS and MWS belong to SASRI, but a significant number are also owned by our
collaborators, notably the Agricultural Research’s Institute for Soil, Climate and Water (ARC-ISCW)
and individual sugarcane farmers.
The objective of this document is to provide users of the SASRI weather website (here after referred
to as WeatherWeb) with an overview of the weather data acquisition and processing, including data
quality assurance, as well as explain the codes used when any of the primary records is an estimate.
Data Acquisition
AWS data are downloaded daily and records updated on the WeatherWeb at about 10.30 am. MWS
and RS data are manually captured and uploaded once a month. The latest MWS and RS data
available on the WeatherWeb are therefore for the previous month, and in some cases MWS data
can be up to two months behind due to the extra time required to process sunshine cards.
Data Quality Assurance
The accuracy and relevance of weather data based predictions, information or decisions are
influenced significantly by the quality of the input data. Efforts are therefore made to ensure the
integrity of data posted on the WeatherWeb. This includes on-site sensor maintenance and
calibrations by trained SASRI technicians and a computer based automated data quality check and
validation system developed to identify missing, erroneous and suspect data. Such data are
automatically replaced by data derived from related climatic variables, neighbouring sites or from
historical records. A similar system is used to estimate missing data.
1
Field calibrations and maintenance by SASRI technicians
All new (or repaired) sensors or instruments are calibrated prior to installation at the weather
stations, and thereafter every six months (field checks) against standard instruments. Any sensors
showing larger than expected differences from the standard are immediately replaced, unless
correctable on site. These field calibration checks (Fig. 1) allow early detection of slight sensor drift
which would otherwise be difficult to detect with our computer based quality control measures.
Figure 1: Field calibration and maintenance work by SASRI technicians at one of the automatic
weather stations
Computer based quality control and validation system
These are automated and run daily with every data upload. The main purpose is to flag and replace
(patch) erroneous data (e.g. due to sensor malfunction), missing and suspiciously high or low values.
There are two main levels that data has to pass to be considered valid.
1. Acceptable range
Upper and lower limits are set for all variables based on one or a combination of theory, historical
records and specifications of the recording instrument. This limits all values to within an acceptable
range e.g. relative humidity has to be within the range of zero to 100 %. Long term historical records
may be applied to further refine the range. Missing data also get flagged for patching.
2
2. Internal and spatial consistency
Data are checked for consistency at a given site to ensure that there is agreement between
related/associated variables, for example minimum temperature has to be always less than
maximum temperature. The rate of change of values between successive days is also checked to
make sure it’s plausible. Similarly, trends between neighbouring sites are also validated for
consistency (may be done for daily, weekly or monthly values depending on the variable in
question).
Patched data codes
The following codes are used to indicate what primary data have been patched (estimated) when
the ‘standard variables’ file format option is selected:
Patched raw data codes:
Code
Variable
-1
-2
-4
-8
- 16
- 32
solar radiation
air temperature
relative humidity
wind speed
rainfall
pan evaporation (MWS only)
The codes are cumulative, i.e. if more than one variable is missing then the code is the sum of the
individual codes as illustrated in the example in Fig 2 under the ‘MIS’ column. The value -15 is the
sum of codes for the patched records on the day, i.e. solar radiation (-1), air temperature (-2)
relative humidity (-4) and wind speed (-8). Similarly the -31 indicates that all records have been
patched for the given day for this AWS. The positive number is the selected station’s identity
number, 489 in this case, which shows that none of the records were patched.
Figure 2: Examples of patched data codes
Please note: isolated cases of inaccurate data may be experienced due to a temporary system failure
or otherwise. We will appreciate your feedback (see contact details on home page).
3