Design and Implementation of an XML-based Universal Mobile
Data Acquisition System
Huang Zhiyi12, Chang Qingling12, Shen Zhihong1, Zhou Yuanchun1, Yan Baoping1, Liu Lei1
（1, Computer Network Information Center CAS, Beijing 100190; 2, Graduate University of Chinese Academy
of Sciences, Beijing 100190）
Abstract
Presently a lot of mobile data acquisition
systems have been applied in outdoor data
acquisition work in all walks of life. But
since they are specially designed and
developed for a certain application
independently, which have disadvantages of
long development period and poor
expansibility. Aimed at these disadvantages,
we designed and implemented an xml-based
universal mobile data acquisition system.
The innovation of this system is that it can
be used for different applications. By simple
customization, it can meet the needs of
different users, without developers' design
and development work time after time. To
take the data acquisition work about plants
and animals in Qinghai Lake Nature
Reserve as an example, this paper
introduces the design and implementation of
this universal mobile data acquisition
system.
1 Introduction
Advances of mobile data acquisition
system based on Personal Digital Assistant
are making data acquisition faster, easier,
and more accurate than paper-based data
collection methods[1][2]. At present, the
mobile data collection systems have been
widely applied in many professions:
medicine
[3],
transportation
[4][5],
agriculture[6][7], geography[8], and so on.
But since they are specially designed and
developed for a certain application
independently, which have disadvantages of
long development period and poor
expansibility. At the same time, through the
investigation into informationization in
many institutes of Chinese Academy of
Sciences, we see widely and pressing
demand of mobile data acquisition system.
For example, Qinghai lake nature reserves,
Institute of Marine, Institute of Botany and
Northeast Institute of Geography and
agricultural ecology, all of them need
mobile data acquisition system to change
their traditional way: paper-based data
collection.
So
many
application
requirements drive us to design and
implement a universal mobile data
acquisition system.
Through analysis and comparison of
these demands of mobile data acquisition
system, we found that they have a lot in
common. Most of them have database
application systems, the staff will input the
recorded data on paper through a web form.
Or they input data to Excel file firstly, and
then import to database by program. Most of
the data recorded on paper are digital, short
text, or GPS data. Based on these things in
common, we design a universal mobile data
acquisition system. Through simple
customization, it can meet the demands of
different users. So this system not only can
avoid developers' repeated development
work, but also can reduce the complexity of
the maintenance. If the demand changes,
user only need to change the configuration,
without the development again. To take the
data acquisition work about plants and
animals in Qinghai Lake Nature Reserve for
an example, this paper introduces the design
and implementation of this universal mobile
data acquisition system in detail.
As China's biggest inland saltwater lake
and international wetlands, Qinghai Lake is
the most abundant biodiversity treasure of
Tibet Plateau. The reserve workers shall
periodically do investigation of birds,
wetland vegetation, and endangered species.
Birds investigation include three activities:
survey around the lake twice each year,
route survey many times each year, every
day observation in some location. Workers
write down the numbers of every kind of
birds (including young, male, female
quantity, and nests quantity, etc), and the
numbers of birds of different behaviors in
different living surroundings. These survey
data provides valuable scientific data not
only for reserve workers to better protect
ecology, but also for plants and animals
scientists to do scientific research and
experiment.
2 Structure and function of the
system
In order to realize the generalization, we
make use of customization by user and
dynamic interpretation by program to
replace the developers’ design and coding
work according to the requirements of
certain application. The key problems are
that how to provide a simple, universal way
for user to custom, how to preserve the
customized information, and how to
generate correct data acquisition forms on
PDA as users want.
Considering that the data acquisition
applications generally need to establish the
corresponding relation database for data
analysis in the next step, and by extracting
table structure metadata of the database, the
system can automatically get part of custom
information, so we base on relational
database, take table for data acquisition unit
(one table in database correspond to one
data acquisition form) to provide users with
simple customization mode.
The universal mobile data acquisition
system consists of three modules: custom
module, data acquisition module, data
synchronization module (figure 1). Users
use this system as follows: firstly, users
specify requirements of data acquisition
based on the well-built relational database,
(including giving title, setting edit style and
data source, et al for each data acquisition
field, setting the acquisition menu items on
PDA, and setting the menu item
corresponding which data acquisition form).
The custom module will save the custom
information to an XML file. Then use the
data synchronization module to get
dictionary data from the database, and copy
the XML file and dictionary data to PDA.
Finally, users start the program on the PDA,
and they will obtain the correct data
acquisition system. The data inputted by
users is stored in the SQLite database(an
embedded database), and then be export to
background database through the data
synchronization module (applicable to
Oracle, SQL server, MYSQL database) etc.
Custom module provides users with a
web interface. Firstly users configure the
connection parameters to background
database. Then the custom module connects
to the database and extracts metadata
information. Based on these information,
custom module provides users with
customize pages to input custom
information about tables, fields, and tables’
relation aroused by foreign key. At last, the
system will store the metadata information
and user customized information to an XML
file.
The data acquisition module is the
program running on PDA. It consists of
three sub modules: XML parsing module,
data acquisition forms generation and data
access module, GPS receiving module.
XML parsing module Reads and analyzes
the XML file, and supply information for
data acquisition forms generation and data
access module. The latter module
dynamically create input widgets and data
access button for the data acquisition forms,
rather than the design and development for a
certain application independently, the input
widgets and buttons are designed statically
by developers. If the data is GPS type, GPS
receiving module will automatically access
and preserve data.
Data synchronization module consists of
two sub modules: data import module and
data export module. It is desktop software.
It builds up a bridge between the PDA and
background database. Because each
application system may have some static
data, such as the plants’ and animals’ name
in the Qinghai lake nature reserve database,
so the data import module can extract these
static data from some table specified by
users in the background database. These
data will reduce the input data for users.
Data export module is used to upload the
data recorded by users on the PDA to the
background databse. Before uploading,
users can browse, edit the data to ensure the
accuracy of the data.
Custom module
GPS aerial
Metadata
extract
Custom
page
Write
XML
XML file
Web
Data acquisition module
XML
parsing
Dictionary
data
Background
database
Forms generation
and data access
GPS
receiving
SQLite database
Data synchronization
module
Windows Mobile
Data import
Data export
Recorded
data
Desktop software
Figure 1 System structure
generate correct data acquisition forms. In
order to clear explain the solutions to these
3 The key technology research
two issues, we take birds investigation
around Qinghai Lake for example. An
The
key
problems
to
realize
investigation around Qinghai Lake takes 1-2
generalization by customization based on
weeks. Investigators travel around the lake
the relational database tables are that how to
according to the plan. They stop at some
design a reasonable, universal XML schema
sites, and record the numbers of very kind
to preserve custom information and how to
of birds. So we design three relational tables:
use custom information to dynamically
Circle Lake Survey (circleLakeID, start date,
end date, captain, participants); Places
Survey (placeSurveyID, placeID, time,
weather, investigators, methods, latitude,
longitude, altitude, circleLakeID); Birds
Survey (birdsSurveyID, birdID, young
quantity, subadult quantity, adult female
quantity, adult male quantity, nest amount,
placeSurveyID). placeID, birdID both are
foreign key pointing to dictionary table:
Places(placeID,
name).
name),
Birds(birdID,
3.1 Design of XML schema
The structure of XML file is as figure 2.
The root element is <dataset> which
contains <repository>, <table>, <relation>,
<view> four kinds of child element.
<repository> is one and only, the others are
repetitive elements.
r eposit ar y
t able
t able
t able
dat aset
r elat ion
view
view
r elat ion
view
view
Figure 2 Structure of XML schema
<dataset>
Users can name the PDA application
<uri>cn.csdb.qinghai</uri>
corresponding to the title that shows on
<title>Qinghai Lake plants and animals</title>
PDA which is store in <dataset>.<title>
<repository>
element. The first step for users to
<productName>mysql</productName>
customize is configure database connection,
<productVersion>5.0</productVersion>
namely fill in database product type, the
<userName>root</userName>
version number, the IP address and port,
<userPass>1</userPass>
user name and password, database name.
<host>localhost</host>
These
data
are
stored
in
<port>3306</port>
<repository>element and will be used by
<databaseName>qinghailakedb</databaseName>
synchronization
module
to
extract
</repository>
dictionary data and import recorded data.
Users need to customize each relation
table. Firstly, set the title and primary key
for the table. Secondly, set titles, edit style
(single-line text widget, multiple line text
widget, drop-down box, or radio buttons)
for each field. All of the information is
stored in the XML file as the list below. The
primary key is signed by <identifier> and
<idGenerator> indicate whether the key is
automatically growth. The type of field can
be integer, real, enumeration, string, GPS
type, collection type. If it is enumeration
type, the user is required to specify the
enumeration data through character string
with comma interval, and the string is kept
in <options>. If the PDA has GPS aerial, the
GPS type field will be filled automatically
by the PDA. The field which is Collection
type is a virtual field refers to another
relation table due to the foreign key.
<table>
<title>CircleLake Survey </title>
<tableName>circlelake</tableName>
<uri>cn.csdb.qinghai.circlelake</uri>
<identifier>
<idGenerator>autoinc</idGenerator>
<field>cn.csdb.qinghai.circlelake.id</field>
</identifier>
<field>
<title>Primary Key</title>
<type>Long</type>
<columnName>ID</columnName>
<editorStyle>LongEdit</editorStyle>
<editorHeight />
<uri>cn.csdb.test.circlelake.id</uri>
</field>
<field>
……
<type>Enum</type>
<options>sunny,rain,cloudy </options>
</field>
</table>
Maybe you have found that <dataset>,
<table>, <field> all have <uri> child
element. This is the only identifier for each
object. It is used to mapping. Take the
foreign key mapping for example. Users
need to configure the foreign key
relationship. For example, in table “Birds
Survey”, birdID is a foreign key refers to
table “Birds”. So the inner text of the
element <relation>.<foreignKey> is the
value of <uri> of field birdID in talbe “Birds
Survey”. The value of element <target> also
is <uri> of the table “Birds”.
<relation>
<target>cn.csdb.qinghai.birds</target>
<foreignKey>cn.csdb.qinghai.birdsSurvey.birdID
</foreignKey>
</relation>
The element <view> is used to tell the
PDA data acquisition module three
information: what tables constitutes the
menu of the PDA data acquisition system;
What tables are the dictionary tables; for
each table what fields should be edited on
PDA and how they are arranged. Take the
view below for example, because the inner
text of <source> element is the <uri> of
<dataset> and the name of the view is
“pdaEdit”, so it is about what tables
constitutes the menu of the PDA data
acquisition system. It has two <item>,
corresponding to the table “Circle Lake
Survey” and “Sample Route Survey”, so the
PDA data acquisition system has two menu
items. If the inner text of <source> is <uri>
of a table, it means what fields should be
edited on the PDA. The configuration page
is as figure 3. For table “Circle Lake
Survey”, the primary key circleLakeID is
auto generate, so users do not need to edit
on PDA.
<view>
<source>cn.csdb.qinghai</source>
<name>pdaEdit</name>
<item>cn.csdb.qinghai.circlelake</item>
<item>cn.csdb.qinghai.sampleRouteSurvey
</item>
</view>
3.2 Dynamic generation of mobile
data acquisition system
The data acquisition module's mission is
to use the custom information saved in
XML file to create menu items and the
corresponding data acquisition forms, and
preserve users’ input data to SQLite
database. There are two technical difficulties.
One is how to dynamically create menu and
data acquisition forms as users want. The
second is how to show two or more forms
together which are related because of
foreign keys.
How to solve the first problem? Firstly,
XML parsing module provides enough
interfaces about custom information for
forms generation module. With the interface
DataStore.getEditTablesList(), we can get an
tables arrayList corresponding to menu
items on PDA. Traverse the arrayList, use
table.getTitle() to obtain the title of the
forms. When the user clicks one menu item,
the system use table.getEditFieldsList() to
get edit fields list of the corresponding table.
And then we use Field.title, Field.type, and
Field.editStyle to create different types of
input widget. For example, the weather field
in table “Place Survey” of Qinghai Lake
database. It is an enumeration type. If the
edit style configured by users is radio button,
the inner text of <editStyle> in XML file
will be “EnumRadioEdit”, the system will
create a series of radio Buttons. If users
select style of drop-down box, <editStyle>
is “EnumSelectEdit”, the system will create
Combobox.
So-called parent-child relation tables, it is
two tables: the primary key of table A is
table B’s foreign key, and table B should be
designed with table A in the same form as
two Tab page. Because the value of foreign
key in table B should be added by the
program rather than users. For example, the
three tables about birds survey in Qinghai
Lake constitute two levels parent-child
relation. The table “Circle Lake survey” is
the parent table of “Places Survey”, and
“Places Survey” is parent of “Birds Survey”.
How users customize such multiple tables
cascade show? In the configuration page of
PDA edit view of the parent table A, you
only need to select the virtual field which
refer to table B into the edit fields list. As
shown in figure 3, because of the foreign
key “circleLakeID” in table “Places Survey”,
a virtual field is added to table “Circle Lake
Survey”. If the virtual field was added to the
edit list, there will be a child tab page for
table “Birds Survey”. With the interface
table.GetEditChildTableList(), the forms
generation module will get the information
about the child tables. As shown in figure 4,
there are three tab pages on PDA
corresponding to three tables of birds survey
in Qinghai Lake.
Figure 3 configuration page of PDA edit view
Figure 4 parent-child forms
4
System
development
and
implementation
At the present time, there are multiple
operating systems on different mobile
equipment, including Symbian of Nokia,
Windows Mobile system of Microsoft, and
PALM system supported by PalmOne
Company. Because that Windows Mobile
system is the same as the Windows system
on PC, and much easy than any other
operating systems, so we chose Windows
Mobile5.0 as our PDA experimental
platform.
We use JSP language with eclipse and
resin server as development environment to
develop the custom module. The other two
modules including data acquisition module
on PDA and desktop data synchronization
module are developed with c#. The
development environment includes Visual
Studio 2005, Windows Mobile 5.0 Pocket
PC SDK, .net Compact Framework 2.0,
SQLite3 and ActiveSync and 4.5.
References
[1] Paul D. Greene. Handheld Computers as Tools
for Writing and Managing Field Data, Field
Methods, May 2001; 13: 181 - 197.
[2] Linda A. Fletcher, Darin J. Erickson, Traci L.
Toomey, et al. Handheld Computers: A Feasible
Alternative to Paper Forms for Field Data
Collection. Eval Rev, Apr 2003; 27: 165 - 178.
[3] Fischer S, Stewart TE, Mehta S, et al. Handheld
computing in medicine. J Am Med Inform Assoc
2003; 10(2):139–49.
[4] Vivoda, J.M., Eby, D.W. 2006. Using Personal
Digital Assistants (PDAs) for the Collection of
Safety Belt Use Data in the Field. Behavior
Research Methods, Instruments, and Computers
vol. 38, p. 158-164.
[5] Zou Guoping, Huang Zheng, Hao Guochang.
Design and Implementation of PDA-based
Highway
5 Conclusion
Maintenance
Data
Acquisition
System. Traffic Standardization, 2007(9)：
97-101.
XML-based universal mobile data
acquisition system provides an automatic
mode for outdoor data acquisition workers
to record data and import data into
background database. It is more convenient
and effective than paper-based data
collection methods. Furthermore, through
simple customization, this system can be
applied to many different applications, not
only reduces the workload of developers,
but also makes the system be easy expanded.
Now this system is used by the Qinghai
Lake Nature Reserve workers, and will be
used by Institute of Marine and Institute of
Botany. Next, we plan to design
cross-platform mobile data acquisition
system, and diversification of data
collection, including images, video,
temperature data, etc.
[6] Otuka.
A.,
& Yamanaka,
T.
(2003). An
application for insect field surveys using a
handheld computer. Agricultural Information
Research, 12(2), 113-124.
[7] Louis George Clark, Jr., Donald Romaine
Gummow, et al. Hand-held GUI PDA with
GPS/DGPS receiver for collecting agronomic and
GPS position data. U.S patent, No:5699244,
Dec 16, 1997.
[8] John F. Alexander, Jr. Handheld portable digital
geographic
data
manager.
No:6083353, Jul 4, 2000
U.S
patent,