A WEB-BASED SYSTEM FOR CONTINUOUS DATA STREAM MANAGEMENT,
MEASURING THE CARQUINEZ BRIDGE MOTIONS USING RTK-GPS
Xing H. Liu
B.S., California State University, Sacramento, 2009
PROJECT
Submitted in partial satisfaction of
the requirements for the degree of
MASTER OF SCIENCE
in
SOFTWARE ENGINEERING
at
CALIFORNIA STATE UNIVERSITY, SACRAMENTO
SPRING
2011
A WEB-BASED SYSTEM FOR CONTINUOUS DATA STREAM MANAGEMENT,
MEASURING THE CARQUINEZ BRIDGE MOTIONS USING RTK-GPS
A Project
by
Xing H. Liu
Approved by:
__________________________________, Committee Chair
Ahmed Salem, Ph.D.
__________________________________, Second Reader
Ying Jin, Ph.D.
____________________________
Date
ii
Student: Xing H. Liu
I certify that this student has met the requirements for format contained in the University format
manual, and that this project is suitable for shelving in the Library and credit is to be awarded for
the Project.
__________________________, Graduate Coordinator
Nikrouz Faroughi, Ph.D.
Department of Computer Science
iii
________________
Date
Abstract
of
A WEB-BASED SYSTEM FOR CONTINUOUS DATA STREAM MANAGEMENT,
MEASURING THE CARQUINEZ BRIDGE MOTIONS USING RTK-GPS
by
Xing H. Liu
In 2004, Caltrans installed a monitoring system (GPS sensors) at Carquinez Bridge
(Highway I-80) to obtain real-time displacement measurements of bridge motions under
traffic, wind, and seismic loads. However, since the data file is huge, approximately 1 GB
a day, it takes a long time to download the text file for analysis, and common tools such
as Excel cannot open these large files.
The Web-based system will have three main features (Capture Real-Time Stream,
Identify Special Events, and Record History records). First, it will capture the constant
stream of sensor data generated by RTK-GPS processing software (RTKNav) and filter
the data based upon an assessment of data quality. Second, the system will calculate
relevant statistical measures (e.g. mean, standard deviations) to identify significant
measurements (e.g. during earthquakes, windstorms). Third, the system will store
average hourly and daily measurements in a database and maintain full time history
records.
iv
The system is built on a WAMP stack (Windows, Apache, MySQL and PHP). Once the
system is complete, it will be deployed to a server located at Carquinez Bridge(Highway
I-80) toll plaza.
_______________________, Committee Chair
Ahmed Salem, Ph.D.
_______________________
Date
v
ACKNOWLEDGMENTS
I would like to take this opportunity to thank all the people who have been involved in
this project. Without their professional knowledge and encouragement, this project
would not have been finished successfully.
First, I would like to say deeply thanks to my project advisor Dr. Ahmed Salem for his
professional advice on my project and his encouragement from begin to end for me to
accomplish this project successfully.
I would like to thank Dr. Ying Jin for agreeing to be my second reader and helping me
with the project.
I would like to give thanks to Mr. Loren Turner, Senior Transportation Engineer,
Division of Research and Innovation – Caltrans. He has been supporting me on the
project from begin to end, providing me great feedbacks on the client’s perspective on the
system.
Finally, I would like to thank my family and friends for their continuous support.
Especially, I want to thank my mother and father for supporting behind the scene and
giving me encouragements when I needed to accomplish this project as well as my
degree.
vi
TABLE OF CONTENTS
Page
Acknowledgments.............................................................................................................. vi
List of Tables ..................................................................................................................... ix
List of Figures ..................................................................................................................... x
Chapter
1. INTRODUCTION .......................................................................................................... 1
1.1
Background and Related Work ....................................................................... 1
1.2
Problem ........................................................................................................... 4
1.3
Scope………………………………….. .................................................... ….5
2. TECHNOLOGY ............................................................................................................. 6
2.1
Apache HTTP Server ...................................................................................... 8
2.2
PHP……………………………………. ........................................................ 8
2.3
MySQL ........................................................................................................... 9
2.4
JQuery Flot Chart............................................................................................ 9
2.5
Google Earth API.......................................................................................... 10
3. FUNCTIONAL REQUIREMENTS ............................................................................. 11
3.1
Functional Requirements .............................................................................. 12
3.2
Scenarios ....................................................................................................... 19
4. SYSTEM DESIGN ....................................................................................................... 22
vii
4.1
Database Design ........................................................................................... 24
4.2
Data Dictionary ............................................................................................. 26
4.3
System Sequence Diagram ........................................................................... 29
5. SYSTEM IMPLEMENTATION .................................................................................. 33
5.1
System Architecture ...................................................................................... 33
5.2
System User Interface ................................................................................... 34
5.3
Components Implementation ........................................................................ 40
5.3.1
Client-Side Components Implementation ..................................................... 40
5.3.2
Server-Side Components Implementation .................................................... 41
6. CONCLUSIONS AND FUTURE WORK ................................................................... 42
6.1
Conclusions ................................................................................................... 42
6.2
Future Work .................................................................................................. 43
Appendix A - JQuery Plot Load ....................................................................................... 44
Appendix B - Server Side Script ....................................................................................... 47
Appendix C - Server Environment Setup ......................................................................... 53
Bibliography ..................................................................................................................... 54
viii
LIST OF TABLES
Page
1. Table 4.1 real_time_data table ...................................................................................... 26
2. Table 4.2 data_archive table ......................................................................................... 27
3. Table 4.3 special_event table ........................................................................................ 28
4. Table 4.4 system_variables table .................................................................................. 28
5. Table 4.5 list_marker table ........................................................................................... 29
ix
LIST OF FIGURES
Page
1. Figure 1.1 Tacoma Narrows Bridge Collapse [2] ........................................................... 1
2. Figure 1.2 Carquinez Bridge (Highway I-80) [6] ........................................................... 3
3. Figure 2.1 The Existing System Architecture ................................................................. 6
4. Figure 2.2 The Proposed Web-based System Architecture ............................................ 7
5. Figure 3.1 Use Case Diagram for GPS Data Stream .................................................... 11
6. Figure 3.2 View Streaming Data Page .......................................................................... 12
7. Figure 3.3 Near Real-Time Plot .................................................................................... 13
8. Figure 3.4 Google Earth API ........................................................................................ 14
9. Figure 3.5 View Full Time-History Record .................................................................. 15
10. Figure 3.6 Highlighting Preferred Area on the Graph ................................................ 16
11. Figure 3.7 Special Event Display................................................................................ 16
12. Figure 3.8 State Transition Diagram for Data Streaming System .............................. 17
13. Figure 3.9 Configure System Variables ...................................................................... 18
14. Figure 3.10 Configure Marker Attributes ................................................................... 18
15. Figure 4.1 The Existing System Architecture ............................................................. 22
16. Figure 4.2 The Proposed Web-based System Architecture ........................................ 23
17. Figure 4.3 System Flow Diagram for Web Based Sensor Streaming ......................... 24
18. Figure 4.4 Data Stream Tables.................................................................................... 25
19. Figure 4.5 Administrative Database Tables ................................................................ 25
20. Figure 4.6 System Sequence Diagram for Near Real-Time Data Streaming ............. 30
x
21. Figure 4.7 System Sequence Diagram for View Full Time-History Data .................. 31
22. Figure 4.8 System Sequence Diagram for View Special Event Data ......................... 32
23. Figure 5.1 Web Pages Map ......................................................................................... 33
24. Figure 5.2 Home Page................................................................................................. 34
25. Figure 5.3 History Page .............................................................................................. 35
26. Figure 5.4 Special Event ............................................................................................. 36
27. Figure 5.5 System Setup Page .................................................................................... 37
28. Figure 5.6 List Google Earth Marker Page ................................................................. 38
29. Figure 5.7 System Variables Page .............................................................................. 39
xi
1
Chapter 1
INTRODUCTION
This project is a web-based system for continuous data stream management and
measuring the Carquinez Bridge Motions. The main focus of this project is to automate
all post-processing and visualization of the bridge sensor data.
1.1
Background and Related Work
In 1940 the Tacoma Narrows Bridge (Figure 1.1) collapsed during a heavy wind storm.
Since then, Structural Health Monitoring [1] has become very important tool in detecting
early signs of bridge damage. One of the commonly used techniques in Structural Health
Monitoring is Global Positioning Systems (GPS), because it can provide centimeter
precision deformation data at high data acquisition rates.
Figure 1.1 Tacoma Narrows Bridge Collapse [2]
2
Many related studies on the GPS sensors have been published since Structural Health
Monitoring was introduced. For example, Tsinghua University of China published a
study on “real-time monitoring of large-span bridge using GPS” in 2002 [3], which
compares the GPS sensors results between Tsing-ma Bridge of Hong Kong and Humen
Bridge in Guangdong China. In another related study done by University of
Nottingham, UK, the researchers suggested to use GPS sensors for structural deflection
monitoring and natural frequency detection. [4] Although these related studies explains
very well on the benefits and applications of bridge GPS sensors, these papers lack
mentioning how to automate post-processing and visualization of the bridge sensor data.
In 2004 Caltrans installed a monitoring system on the Carquinez Bridge on Interstate 80
north-east of San Francisco (Figure 1.2), to obtain near real-time displacement
measurements of bridge motions under traffic, wind, and seismic loads. Measurements
are obtained using autonomous Real-Time Kinematic Global Positioning System (RTKGPS) technologies. [5] Three sensors were installed on the structure and provide
centimeter accurate position information at 20 samples per second. The intent of the
system was to provide near real-time data to key Caltrans personnel for assessment of
design and maintenance issues.
3
Figure 1.2 Carquinez Bridge (Highway I-80) [6]
At the time of installation, the three sensors were wirelessly transmitting position data to
a RTK-GPS processing PC housed at the Carquinez Toll Plaza building at the north-east
end of the bridge. The RTK-GPS processing PC runs two applications, the RTK-GPS
processing software, RTKNav [7], from Waypoint Consulting, and a data management
and FTP server application developed with National Instrument’s LabView [8].
RTKNav is used to processes the raw GPS data and convert the data into ASCII string
format, while LabView runs as a parallel application to filtering the data based on the
quality of the measurements. In addition, LabView creates small archive files of the
4
positioning data, transferring the data to a File Transfer Protocol (FTP) server, and
making the data available to Caltrans engineers for review.
1.2
Problem
Although, the RTK-GPS sensor technology provides very useful data for bridge
engineers, the current implementation for data delivery and visualization presents
significant barriers to routine use of this information. Specifically:

Real-time visualization and notification is not available.

Transferring the large files generated by the system is impractical.

Significant post-processing is required by the end user, and most common
software tools cannot handle the large files.
The RTK-GPS processing software, RTKNav, stores data in a delimited text file format.
Although the text file can be easily interpreted, the size and quantity of the data files
generated daily makes it difficult for bridge engineers to quickly analyze the data and
generate reports. Currently, the only way to view the real-time streaming data is to sit in
front of the laptop at the Carquinez Toll Plaza building, or remote into that laptop. It is
possible to FTP the data to the bridge engineer's laptop for analysis. However, since the
data file is huge, approximately 1 GB a day, it takes a long time to download the text file,
and common desktop applications, such as Excel, cannot open these large files.
5
1.3
Scope
The main focus of this project is to automate all post-processing and visualization of the
bridge sensor data. The proposed web-based system will have three main features,
specifically:
 Capture near real-time stream
 Identify special events
 Record history records
First, it will capture the constant stream of sensor data generated by RTKNav and filter
the data based upon an assessment of data quality. Second, the system will calculate
relevant statistical measures (e.g. mean, standard deviations) to identify significant
measurements (e.g. during earthquakes, windstorms). Third, the system will store
average hourly and daily measurements in a database and maintain full time history
records.
6
Chapter 2
TECHNOLOGY
In the original bridge monitoring installation, three sensors transmit information to a
RTK-GPS processing PC, installed at the Carquinez Toll Plaza building (Figure 2.1).
The RTK-GPS processing PC runs two applications, the RTK-GPS processing software,
RTKNav from Waypoint Consulting, and a data management and FTP server application
developed with National Instrument’s LabView.
Figure 2.1 The Existing System Architecture
RTKNav is used to processes the raw GPS data and convert the data into ASCII string
format, while LabView runs as a parallel application to determine the quality of the
measurements. In addition LabView creates small archive files of the positioning data
and transfer the data to a File Transfer Protocol (FTP) server.
7
The goal of this project is to migrate the data processing and storage functions from the
PC at the Carquinez Toll Plaza Server to a new and more efficient web-based data
streaming system off-site. This system will connect over the internet to the Carquinez
Toll Plaza Server to receive raw GPS data. The processing, filtering, storage, and
presentation of the data will be carried out by the new off-site server, thereby eliminating
the overhead of transferring huge data files FTP, while providing near real-time analysis
and presentation of the bridge sensor data. (Figure 2.2)
Figure 2.2 The Proposed Web-based System Architecture
The project is developed in an Apache – PHP – MySQL stack on a Windows XP
workstation (WAMP). This framework was chosen because of three main factors –
portability, extensibility, and the open source framework.
8
2.1
Apache HTTP Server
The Apache HTTP Server Project is an effort to develop and maintain an open-source
HTTP server for modern operating systems including UNIX and Windows NT. [9] The
Apache server is easy to set up and provides platform independency for the project.
2.2
PHP
PHP is a general-purpose scripting language that was originally designed for web
development to produce dynamic web pages. For this purpose, PHP code is embedded
into the HTML source document and interpreted by a web server with a PHP processor
module, which generates the web page document. PHP is available as a processor for
most modern web servers and as a standalone interpreter on most operating systems and
computing platforms. [10]
The portability of PHP on multiple platforms is the main reason for selecting PHP as the
development language. Currently, the web-based sensor project is developed in
Windows operating system running Apache web server. However, if there is a need to
move to a UNIX machine, project source code can be transferred freely without any sort
of code conversion. It is platform independent.
9
2.3
MySQL
MySQL is a relational database management system (RDBMS) that runs as a server
providing multi-user access to a number of databases. [11] There are several benefits for
using MySQL database in this project as listed below.
1. Open Source – MySQL is free under the terms of General Public License. In order to
save the cost for the state government, Open Source MySQL is the ideal choice for
the government to cut the cost and provide similar RDBMS services comparing to the
commercial RDBMS.
2. Platform Independent – Similar to PHP, MySQL can be run on different kinds of
operating systems, such as Windows, Linux, and Sun Solaris. [12] Therefore, if there
is a need to migrate the web-based streaming system to other operating system, there
is no database conversion needed.
2.4
JQuery Flot Chart
JQuery Flot Chart is a pure Javascript plotting library for jQuery. It produces graphical
plots of arbitrary datasets on-the-fly client-side. [13] First of all, Flot is lightweight. The
main class file is only 33KB in size, which will provide all the basic plotting function to
the web browser such as scaling and adding multiple lines in a chart. Second, Flot works
well under the AJAX framework, which provides updates on the chart without refresh the
whole page.
10
2.5
Google Earth API
The Google Earth Plug-in and its JavaScript API let you embed Google Earth, a true 3D
digital globe, into your web pages. [14] The reason to use Google Earth API in this
project is because Google Earth API provides markers drawing functionality and 3D
model for the bridge, which will help the user visualize the locations of the sensors and
get more information through marker information box.
11
Chapter 3
FUNCTIONAL REQUIREMENTS
The main functional requirements for the GPS Data Stream System are captured in the
use case diagram shown in Figure 3.1. The main end users for this system are the Bridge
Engineer and the Administrator. The Bridge Engineer interacts with the system to
access, view, and analyze bridge sensor data with specific use cases corresponding to
viewing streaming near real-time sensor data (UC1), viewing historical trends (UC2), and
analyzing specific significant events (UC3). The Administrator interacts with the system
to setup, configure, and maintain the operational components (UC4).
Figure 3.1 Use Case Diagram for GPS Data Stream
12
3.1
Functional Requirements
This section provides detailed description of each functional requirement.
UC1: View Streaming Data
Description: The View Streaming Data function allows user to view near real-time plots
of bridge motions at each sensor location. (Figure 3.2)
Figure 3.2 View Streaming Data Page
The user is presented with a set of three time-history plots, (Figure 3.3) each set
representing north/east/altitude measurements from each of the three different sensors. In
all plots the x-axis represents time (hour:minute:second), while the y-axis represents the
current sensor position (meters) with respect to a fixed location base station. Besides
13
displaying the near real-time data, the plots also display the moving average and the
threshold for identifying significant movements based on a standard deviation. The
moving average, or mean, is represented by red line, while the threshold is represented by
upper and lower gray boundary lines.
Figure 3.3 Near Real-Time Plot
UC1.1: View Sensor Location
Description: As shown in Figure 3.4, the 3D model from Google Earth API allows user
to visualize the position of the sensor and provides spatial context for the corresponding
sensor data.
14
Figure 3.4 Google Earth API
UC2: View Historical Data
Description: This function displays time-history records (Figure 3.5) that are stored in the
system based on a filtered set of the data. Similar to the real-time plot, the x-axis of the
graph represents the time, while the y-axis represents the current sensor position in
meters with respect to the base station. In addition, the user has an option to choose
which time period of the history graph to show by using the calendar filter on top of the
graph.
15
Figure 3.5 View Full Time-History Record
UC2.1: Record Hourly Average
Description: This function records the average value from each sensor hourly, which
means, in a typical day, there are 24 average value points that are archived into the
system.
UC2.2: Filter History Results
Description: This function allows user to customize the history results by picking
different periods from a pop up calendar. (Figure 3.5) The calendar will be initialized to
the current month. In addition, user can filter the result by highlighting preferred area on
the graph. (Figure 3.6) The graph will zoom into the user defined period.
16
Figure 3.6 Highlighting Preferred Area on the Graph
UC3: View Special Event
Description: This function lets the user select the preferred event from an event table
(Figure 3.7). When user clicks on the “Graph” link, a corresponding graph will show up
below the event table.
Figure 3.7 Special Event Display
17
UC3.1: Record Special Event
Description: Record Special Event function monitors the incoming GPS sensor data
stream. When the stream data exceeds the threshold three consecutive times, the system
will consider the stream as special event stream and enter special event state (Figure 3.8).
On the other hand, in order to exit the special event state, the current data has to be under
the threshold level for three consecutive times. By default, the system will record five
minutes of data before the special event occurs, the current special event data, and five
minutes of data after the special event occurs.
Figure 3.8 State Transition Diagram for Data Streaming System
UC4: Configure System
Description: Configure System function allows administrator user to setup, configure,
and maintain the operational components (Figure 3.9).
18
Figure 3.9 Configure System Variables
UC4.1: Configure Marker
Description: Configure Marker allows administrator to make changes to the sensor’s
information on Google Earth. Administrator can add, delete, and edit sensor’s
information including Name, Latitude, Longitude and Altitude. (Figure 3.10) Once done,
the updated markers will be displayed on the Google Earth interface shown in Figure 3.4.
Figure 3.10 Configure Marker Attributes
19
UC4.2: Configure Time Interval
Description: This function lets administrator determine how long the system should be
keeping the record of the near real-time data. By Default, this value is set to five
minutes, which means, when special event such as earthquake or windstorm occurs, the
system will record the buffer of previous five minutes of data.
UC4.3: Start/Stop Server
Description: This function gives administrator control on the web HTTP server to start or
stop listening to the sensor UDP data stream.
UC4.4: Adjust Threshold Variable
Description: This function provides flexibility to change the threshold value of the system
in the future. Since the threshold value is determined by bridge engineers, it’s better to
have it flexible to change.
3.2
Scenarios
This section will list the scenarios for the main use cases UC1, UC2, UC3, and UC4.
Each scenario will represent a set of steps for each use case. Scenarios will be identified
by upper case letters “SC”, from 1 to 4.
20
SC1: View Streaming Data (use case: UC1)

Open View Data Stream page.

A Google Earth interface and a set of three time-history plots will be presented.
(Figure 3.2)

Bridge engineer chooses preferred sensor from the top of the time-history plots.

The time-history plots will be updated based on user’s choice.
SC2: View Historical Data (use case: UC2)

Open View History Data page.

A set of three time-history plots, which is set to current month by default, shows
up. (Figure 3.5)

Bridge engineer choose specific date from a calendar filter on top of the plots.

The full time-history plots will be updated based on user’s date selection.
SC3: View Special Event (use case: UC3)

Open View Special Event page.

A table displays a list of special events occurred, with the sensor number and date
information. (Figure 3.7)

Bridge engineer clicks on the “Graph” link of each row.

A corresponding plot shows up below the event table.
21
SC4: Configure System (use case: UC4)

Open Configure System page.

A system variables page is presented. (Figure 3.9)

Administrator changes the value of threshold factor.

Click “Save” button.

Database is updated based on administrator’s change.
22
Chapter 4
SYSTEM DESIGN
In the original bridge monitoring installation, three sensors transmit information to a
RTK-GPS processing PC, installed at the Carquinez Toll Plaza building (Figure 4.1).
The only way to analyze sensor data is for the user to be either physically present at the
Carquinez Toll Plaza or transfer a huge set of sensor data using FTP. This is
inconvenient if bridge engineers need immediate analysis on bridge movements.
Figure 4.1 The Existing System Architecture
The goal of this project is to migrate the data processing and storage functions from the
PC at the Carquinez Toll Plaza Server to a new and more efficient web-based data
streaming system off-site. (Figure 4.2)
23
Figure 4.2 The Proposed Web-based System Architecture
Instead of transferring sensor data through a user-initiated FTP to the client’s machine,
the RTK-GPS processing PC will stream sensor data using the User Datagram Protocol
(UDP) to an Apache HTTP server. This will shift the work load of data transfer and postprocessing from the user’s machine to a backend server. The reason to use UDP
streaming instead of TCP is because UDP provides a faster transmission rate of data
packets compared to TCP, which requires handshaking to ensure reliability. As such,
UDP provides the best performance and fulfills the requirement for near real time
streaming. Future implementation of TCP might need to be considered for the “View
Special Event” function, because TCP is more reliable and will ensure to capture every
data points in the significant event.
24
In addition, to further shifting the work load of post-processing of sensor data for bridge
engineer, there needs to be a place to store sensor data. There are two options to store
sensor data. One is to store in a text file format, and retrieve data from the file every time
client requests to view the data. However, this happens to be insufficient since there is no
way to filter the data other than programmatically running through the whole data set
from the file. This leads to the second method, which is to store the data in a database.
(Figure 4.3) The database tables to store sensor data are independent with each other.
The backend logic of the system will determine which table to use at a specific given
time.
Figure 4.3 System Flow Diagram for Web Based Sensor Streaming
4.1
Database Design
The database design for this project is divided into two categories of tables:

Data Stream tables (Figure 4.4) handle storage of real time data, full time-history
data, and special event data. These tables are independent with each other since
the backend logic handles which table to use at a specific given time. (Figure 4.3)
25

Administration tables (Figure 4.5) store system variables and Google Earth
marker configuration attributes. This set of tables is for administrative purposes,
which will improve the maintainability of the system.
Figure 4.4 Data Stream Tables
Figure 4.5 Administrative Database Tables
26
4.2
Data Dictionary
This section presents attributes from each database table, including MySQL type and
description, as shown in Table 4.1 to Table 4.5. These tables are captured in Figure 4.3
System Flow Diagram.
Real_time_data table stores sensor stream real time data information.
Attribute
MySQL Type
Description
id
remote_number
east
INT(10)
DOUBLE
DOUBLE
north
DOUBLE
up
DOUBLE
lat
lon
qf
DOUBLE
DOUBLE
INT(10)
dateTime
DATETIME
millisecond
INT(10)
Primary Key on real_time_data
This indicates the sensor remote number
East coordinate value with respect to master
station (meters)
North coordinate value with respect to master
station (meters)
Vertical coordinate value with respect to master
station (meters)
Latitude in degrees
Longitude in degrees
GGA quality factor
 0 – No solution
 1 – Single Point Solution (SPS)
 2 – DGPS solution
 3 – PPS DGPS solution
 4 – RTK with fixed integers (KAR valid)
 5 – Float RTK
Solution time in UTC format (yyyy-mm-dd
HH:MM:SS)
Solution time in millisecond (00 – 99)
Table 4.1 real_time_data table
27
Data_archive table stores full time-history data information.
Attribute
MySQL Type
Description
id
remote_number
east
INT(10)
DOUBLE
DOUBLE
north
DOUBLE
up
DOUBLE
lat
lon
qf
DOUBLE
DOUBLE
INT(10)
dateTime
DATETIME
millisecond
INT(10)
Primary Key on data_archive
This indicates the sensor remote number
East coordinate value with respect to master
station (meters)
North coordinate value with respect to master
station (meters)
Vertical coordinate value with respect to master
station (meters)
Latitude in degrees
Longitude in degrees
GGA quality factor
 0 – No solution
 1 – Single Point Solution (SPS)
 2 – DGPS solution
 3 – PPS DGPS solution
 4 – RTK with fixed integers (KAR
valid)
 5 – Float RTK
Solution time in UTC format (yyyy-mm-dd
HH:MM:SS)
Solution time in millisecond (00 – 99)
Table 4.2 data_archive table
28
Special_event table stores special event data information. Each table row is identified by
event_id column.
Attribute
MySQL Type
Description
id
remote_number
east
INT(10)
DOUBLE
DOUBLE
north
DOUBLE
up
DOUBLE
lat
lon
qf
DOUBLE
DOUBLE
INT(10)
dateTime
DATETIME
millisecond
event_id
INT(10)
INT(10)
Primary Key on special_event
This indicates the sensor remote number
East coordinate value with respect to master
station (meters)
North coordinate value with respect to master
station (meters)
Vertical coordinate value with respect to master
station (meters)
Latitude in degrees
Longitude in degrees
GGA quality factor
 0 – No solution
 1 – Single Point Solution (SPS)
 2 – DGPS solution
 3 – PPS DGPS solution
 4 – RTK with fixed integers (KAR
valid)
 5 – Float RTK
Solution time in UTC format (yyyy-mm-dd
HH:MM:SS)
Solution time in millisecond (00 – 99)
Identify a specific event
Table 4.3 special_event table
System_variables table stores system variables.
Attribute
MySQL Type
Description
SysVarID
DisplayFactor
INT(10)
FLOAT
SystemState
INT(10)
Threshold
DOUBLE
Buffer_minute
FLOAT
Primary Key on system_vaiables
This indicates the number of points displayed in
the real time plot
This indicates whether the system is listening to
the UDP port or not.
 1 – Listening the port
 0 – Idle
This threshold factor is multiplied by the
standard deviation from the real time plot.
This indicates how long the real time plot should
store the data prior to the current time.
Table 4.4 system_variables table
29
List_marker table stores Google Earth’s marker information.
Attribute
MySQL Type
Description
List_marker_id
Lat
Lon
Altitude
Name
Html
INT(10)
DOUBLE
DOUBLE
DOUBLE
VARCHAR(100)
VARCHAR(200)
Primary Key on list_marker
The latitude of Google Earth marker
The longitude of Google Earth marker
The altitude of Google Earth marker
The name of Google Earth marker
This holds HTML format information about the
marker.
Table 4.5 list_marker table
4.3
System Sequence Diagram
The System Sequence Diagram describes the behavior of a system. The System
Sequence Diagram presented here for Near Real-Time Data Streaming (Figure 4.6) is
captures the use cases presented earlier in chapter 3 for (UC1) View Streaming Data,
(UC2.1) Record Hourly Average and (UC3.1) Record Special Event. This diagram also
shows how the system integrates the GPS sensor data stream over UDP.
30
Figure 4.6 System Sequence Diagram for Near Real-Time Data Streaming
The System Sequence Diagram for View Full Time-History Data (Figure 4.7) is related
to use cases described in chapter 3 as UC2 View Historical Data. This diagram shows
how the bridge engineer accesses and views full time-history graph information.
31
Figure 4.7 System Sequence Diagram for View Full Time-History Data
The System Sequence Diagram for View Special Event Data (Figure 4.8) is related to use
cases described in chapter 3 as UC3 View Special Data. This diagram shows how bridge
engineer request special event information for analysis.
32
Figure 4.8 System Sequence Diagram for View Special Event Data
33
Chapter 5
SYSTEM IMPLEMENTATION
5.1
System Architecture
The web based sensor streaming system is comprised of four main pages (Figure 5.1),
including Home, History, Special Event, and System Setup. The Home page is the top
level page and provides navigation links to the three other pages. In addition, the Home
page presents the near real-time plots as well as the Google Earth interface.
Figure 5.1 Web Pages Map
34
5.2
System User Interface
This section describes each of the four main pages, their primary purpose, and key
features.
Figure 5.2 Home Page
Figure 5.2 shows the home page for the system. A Google Earth API interface is
embedded in the top half of the page showing the locations of the sensors on the bridge.
When the user clicks an icon, the average sensor position for the previous 24 hour will be
shown. The lower half of the page presents a series of x-y time history plots,
35
corresponding to a sensor location. The user can toggle the display of different sensors
using the radio buttons below Google Earth interface.
Figure 5.3 History Page
Figure 5.3 shows the History page. The user can specify the historical time period to
display by selecting the begin and end dates using a calendar-based selection tool.
Similar to the Home page, the user can toggle the display of different sensors using the
radio buttons near the top.
36
Figure 5.4 Special Event
Figure 5.4 shows the Special Events page. A list of special events is displayed on top of
the page. The user can choose specific event by clicking “Graph” link on each row of the
special event list.
37
Figure 5.5 System Setup Page
Figure 5.5 shows the System Setup page. This page is provided for an Administrator to
setup and maintain the system, including customizing the presentation in the Google
Earth interface and controlling the stream of data over UDP. The Administrator can stop
the streaming server by clicking “Stop Listening to Stream” button. Also, administrator
can go to List Google Earth Marker page (Figure 5.6) and System Variables page (Figure
5.7).
38
Figure 5.6 List Google Earth Marker Page
Figure 5.6 shows List Google Earth Marker page. The Administrator can add, edit and
delete Google Earth Marker through this page.
39
Figure 5.7 System Variables Page
Figure 5.7 shows System Variables page. Administrator can modify system variables
through this page. Streaming points displayed determines the number of points shown in
the real time plot. Threshold factor is used to identify significant events by comparing
the current streaming data and the multiplication of threshold factor with standard
deviation. Real time buffer indicates how many minutes the system should keep the
record of current streaming data. In a case of signification event, this buffer of data will
be stored into the special event table for analysis.
40
5.3
Components Implementation
There are two main categories of components implementation in this system, client-side
components and server-side components. Below is the listing of these components:
Client-Side Components

Google Earth Interface

JQuery Plot Interface

Request sensor data display
Server-Side Components

Handles client-side’s display request

Listening to sensor data stream

Record full time-history data

Identify significant event
5.3.1
Client-Side Components Implementation
The client-side components on this system are designed to be thin-client oriented, which
means client-side components only involve in the display of sensor data without any
calculation workload. In the client-side machine, a web browser with Javascript enabled
is required because Google Earth and JQuery plot interface both use Javascript for
display of the sensor data. In addition, the other reason to use Javascript in the client-side
41
is that Javascript can update portion of the page without reloading the whole page. In the
case of near real-time streaming, data points are changing rapidly every second. If the
page is refreshed every time the data points change, it will slow down the client’s web
browser. Therefore, it’s more suitable to use Javascript implemented feature in the
client-side for this project. More detailed implementation code of client-side components
will be available in Appendix A.
5.3.2
Server-Side Components Implementation
The server-side components are responsible for handling client-side’s data display
request and the logics to handle the sensor data. The programming language used in the
server-side is PHP scripting language. The reason to use PHP scripting language in this
project is because it’s open source, and PHP has key supporting functions such as
MySQL query functions and UDP socket functions for stream handling.
There are two main implementations in the server-side, sensor data display request and
UDP stream handling. First, when the client-side data display request arrives, the PHP
will query the MySQL database and return the result data back to the client-side. On the
other hand, an UDP socket PHP script will run in the background once the administrator
starts the server. This UDP socket script will be responsible to handle near real-time
bridge sensor data, record full time-history data, and identify significant event. More
detailed implementation code of server-side components will be available in Appendix B.
42
Chapter 6
CONCLUSIONS AND FUTURE WORK
6.1
Conclusions
This web-based bridge sensor streaming system will have the following benefits and
contributions:

Provide bridge engineers with near real-time bridge displacement data

Alert bridge engineers when displacements exceeds potentially damaging
thresholds

Facilitate access to monitoring data through any Internet capable device

Eliminate cumbersome downloading / post processing of data files.
The web-based bridge sensor streaming system is designed to be a thin-client layer
application. Clients, in this case bridge engineers, can access the web-based system
anywhere anytime as long as they have access to a computing device with a web browser.
The workload of calculating, querying, and generating meaningful way for presentation is
shifted to a server on the backend. Because of the portability of this open source PHPMySQL-Apache framework, this system can be deployed to other operating systems
quickly as needed. From this project, I have gained valuable experience on bridge sensor
software RTKNav, Google Earth API, JQuery plotting API, PHP and MySQL
technologies.
43
6.2
Future Work
The main purpose of this project is to automate all post-processing and visualization of
bridge sensor data. There are a few improvements that could be made to improve system
functionality in the future:

Stream the sensor data through mobile applications such as Android/IOS 4.

Use reliable TCP to record significant events data points.

Use an email notification system for bridge maintenance engineers under
significant events (eg. earthquake, windstorm).
44
APPENDIX A - JQuery Plot Load
The displayData function is responsible for displaying sensor data on a JQuery plot. This
function also involves in an AJAX HTTP request call for data in the backend. Once the
server sends respond data result back to the displayData function, the function will
construct data, average, and threshold arrays. After the preparation of sensor data is
finished, displayData function will use JQuery plotting function $.plot to display bridge
sensor data on the real time graph. Finally, the function will check if the display variable
is still true. If it’s true, then recursively call the displayData function; otherwise, this will
exit the function.
function displayData()
{
//Initialize HTTP object call
xmlHttp2=GetXmlHttpObject();
var url="get_report_string_3.php?remote_number="+remote_number;
xmlHttp2.onreadystatechange=function(){
if (xmlHttp2.readyState==4)
{
var result_str = xmlHttp2.responseText; // Get the return text from the server side
var result_array = result_str.split(";");
var now = new Date();
document.getElementById("time").value = (now.getMonth()+1)+"/ " +now.getDate()+"/ "
+now.getFullYear()+" " +now.getHours() + ":"+now.getMinutes()+ ":"+now.getSeconds();
//JQuery function to update the plot
$(function () {
//Initialize variables to store sensor position data
var east = [];
var north = [];
var height = [];
var north_avg_value = result_array[result_array.length-6];
var east_avg_value = result_array[result_array.length-5];
var height_avg_value = result_array[result_array.length-4];
45
var east_avg = [];
var north_avg = [];
var height_avg = [];
var north_threshold = result_array[result_array.length-3];
var east_threshold = result_array[result_array.length-2];
var height_threshold = result_array[result_array.length-1];
var east_thres_top = [];
var east_thres_bottom = [];
var north_thres_top = [];
var north_thres_bottom = [];
var height_thres_top = [];
var height_thres_bottom = [];
for (var i = 0; i < result_array.length-6; i ++)
{
var ele = result_array[i].split(",");
//constructing sensor data array
north.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10), ele[0]]);
east.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10), ele[1]]);
height.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10), ele[2]]);
//construct average array
east_avg.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
east_avg_value]);
north_avg.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
north_avg_value]);
height_avg.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
height_avg_value]);
//construct threshold array
east_thres_top.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
(parseFloat(east_avg_value) + parseFloat(east_threshold))]);
east_thres_bottom.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
(parseFloat(east_avg_value) - parseFloat(east_threshold))]);
north_thres_top.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
(parseFloat(north_avg_value) + parseFloat(north_threshold))]);
north_thres_bottom.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
(parseFloat(north_avg_value) - parseFloat(north_threshold))]);
height_thres_top.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
(parseFloat(height_avg_value) + parseFloat(height_threshold))]);
height_thres_bottom.push([Date.UTC(ele[6],ele[7]-1,ele[8],ele[9],ele[10],ele[11],parseFloat(ele[12])*10),
(parseFloat(height_avg_value) - parseFloat(height_threshold))]);
}
46
var options = {
xaxis: { mode: "time" }
};
//Update the JQuery Plot data
$.plot($("#placeholder_north"), [{label: "Northing",data:north, color:"blue"}, {label:
"Mean",data:north_avg, color:"red"}, {label: "Threshold",data:north_thres_top,
color:"#999999"},{data:north_thres_bottom, color:"#999999"}],options);
$.plot($("#placeholder_east"), [{label: "Easting",data:east, color:"green"},{label: "Mean",data:east_avg,
color:"red"}, {label: "Threshold",data:east_thres_top, color:"#999999"},{data:east_thres_bottom,
color:"#999999"}],options);
$.plot($("#placeholder_alt"), [{label: "Height",data:height, color:"orange"}, {label:
"Mean",data:height_avg, color:"red"}, {label: "Threshold",data:height_thres_top,
color:"#999999"},{data:height_thres_bottom, color:"#999999"}],options);
});
if(display == true) // if display of plot is still on
displayData(); //Recurively call the displayData function until display variable is true
}}}
xmlHttp2.open("GET",url,true);
xmlHttp2.send(null);
}
47
APPENDIX B - Server Side Script
This server side script is responsible for handling real time streaming of sensor data,
recording full time-history data, and identifying significant events. First, the script will
initialize a stream socket to connect to a UDP port. The system will listen to the data
stream through the UDP port. Once it’s detected there is incoming data from the UDP
port, the server side script will determine whether the data quality is good for storage by
checking the quality factor attribute $qf from the sensor data stream. If the data is good
enough, the script will go through three more steps. First step, store the current data into
a real-time table. Second, store the hourly average of data if needed. Third, check
against the threshold to see if current data exceeding the system threshold.
<?php
require_once("include_session.php");
$DBConn = new DB();
$socket = stream_socket_server("udp://10.160.173.154:6003", $errno, $errstr, STREAM_SERVER_BIND);
if (!$socket) {
die("$errstr ($errno)");
}
$rtvec_set = 0; //format flag
$gpgga_set = 0; //format flag
ob_start();
$count = 1;
$current_date = "1990-1-1";
$previous_time = 0;
$current_time = 0;
$init = 0;
48
$result_event_id = $DBConn->Query("select event_id from special_event group by event_id order by
event_id DESC");
if(mysql_num_rows($result_event_id)==0)
{
$event_id = 0; //For special event identification
}
else
{
$event_id = mysql_result($result_event_id, 0 , 'event_id')+1; //generate a unique event key
}
$consecutive_threshold_north = 1; // For consecutive points in nothing that passes threshold line
$threshold_count_north = 0; //Count how many points in nothing to pass threshold consecutively
$consecutive_threshold_east = 1; // For consecutive points in easting that passes threshold line
$threshold_count_east = 0; //Count how many points in easting to pass threshold consecutively
$consecutive_threshold_up = 1; // For consecutive points in altitude that passes threshold line
$threshold_count_up = 0; //Count how many points in altitude to pass threshold consecutively
$recorded_previous = 0; //Recorded the previous buffered values
$aftermath_time = 0; //The time that starts the after math
$special_mode = 0; //Toggle between specail event mode or normal mode
/*
0: Initial State
1: Record aftermath data
2: Record previous and current exceeding data
*/
$special_remote = 0; // Record the special remote id
$sys_var = $DBConn->Query("select * from system_variables ");
$buffer_size = mysql_result($sys_var,0,"buffer_minute");
$display_factor = mysql_result($sys_var,0,"DisplayFactor");
$north_sd_array = array();
$east_sd_array = array();
$height_sd_array = array();
$north_avg_array = array();
$east_avg_array = array();
$height_avg_array = array();
echo "Server now listening to port: 6003";
do {
$pkt = stream_socket_recvfrom($socket, 1024, 0, $peer);
ob_flush(); flush();
$info = split(",",$pkt);
if($rtvec_set==0)
49
{
$remote_number = "";
$east = "";
$north = "";
$up = "";
}
$lat = "";
$lon = "";
$qf = "";
$current_time = "";
$sum_north = 0;
$sum_east = 0;
$sum_up = 0;
$sum_lat = 0;
$sum_lon = 0;
//for ($i = 0; $i < sizeof($info) ; $i++)
//{
if($info[0]=='$RTVEC')
{
if($info[2]=="R1")
{
$remote_number = 1;
}
if($info[2]=="R2")
{
$remote_number = 2;
}
if($info[2]=="R3")
{
$remote_number = 3;
}
$east = $info[3];
$north = $info[4];
$up = $info[5];
$rtvec_set = 1;
}
if($info[0]=='$GPGGA')
{
$current_time = $info[1];
$time = explode(".", $current_time);
$hour = substr($time[0], 0, 2);
$minute = substr($time[0], 2, 2);
$second = substr($time[0], 4, 2);
$millisecond = $time[1];
50
$year =date("Y");
$month = date("m");
$day = date("d");
// echo "<br>$init -- $previous_time <br>";
if($init ==0)
{
$result = $DBConn->Query("select * from real_time_data where
remote_number = ".$remote_number." order by id ASC");
//echo " Greater than the previous minute <br>";
if(mysql_num_rows($result)>0)
{
//echo "select * from real_time_data where remote_number =
".$remote_number." order by id ASC";
$previous_time = strtotime(mysql_result($result, 0, "dateTime"));
}
else
{
$previous_time = strtotime("$year-$month-$day
$hour:$minute:$second");
}
$init = 1;
}
$current_time = strtotime("$year-$month-$day
$hour:$minute:$second");
$lat = $info[2];
$lon = $info[4];
$qf = $info[6];
$gpgga_set = 1;
}
if($rtvec_set == 1 && $gpgga_set == 1 && $remote_number!="" &&
$east!=""&& $north!=""&& $up!=""&& $remote_number!=""&& $lat!=""&& $lon!=""&& $qf!="" &&
$qf>=4)
{
$DBConn->Query("Insert into real_time_data(
remote_number,
east,
north,
up,
lat,
lon,
qf,
dateTime,
millisecond
51
) VALUES(". $remote_number. "," . $east."," . $north."," . $up."," . $lat."," . $lon."," .
$qf.",STR_TO_DATE('".$month.",".$day.",".$year." ".$hour.":".$minute.":".$second."','%m,%d,%Y
%h:%i:%s'), $millisecond )");
$sum_north += $north;
$sum_east += $east;
$sum_up += $up;
$sum_lat += $lat;
$sum_lon += $lon;
if($second == "30" && $millisecond =="00")
{
$DBConn->Query("Insert into data_archive(
remote_number,
east,
north,
up,
lat,
lon,
qf,
dateTime,
millisecond
) VALUES(". $remote_number. "," .($sum_east/$count)."," . ($sum_north/$count)."," .
($sum_up/$count)."," . ($sum_lat/$count)."," . ($sum_lon/$count)."," .
$qf.",STR_TO_DATE('".$month.",".$day.",".$year." ".$hour.":".($minute).":".($second-30)."','%m,%d,%Y
%h:%i:%s'), $millisecond )");
$sum_north = 0;
$sum_east = 0;
$sum_up = 0;
$sum_lat = 0;
$sum_lon = 0;
$count = 1;
}
$count++;
$result = $DBConn->Query("select * from real_time_data where
remote_number = ".$remote_number." order by id ASC");
if(((float)($current_time - $previous_time))/60 > $buffer_size)
{
52
$DBConn->Query("delete from real_time_data where id =
".mysql_result($result,0,"id"));
$result = $DBConn->Query("select * from real_time_data where
remote_number = ".$remote_number." order by id ASC");
$previous_time = strtotime(mysql_result($result, 0, "dateTime"));
}
$rtvec_set = 0; //format flag
$gpgga_set = 0; //format flag
}
} while ($pkt !== false);
?>
53
APPENDIX C - Server Environment Setup
The web-based sensor runs on the following server setup:

Windows XP/Vista/7 machine

Apache HTTP server 2.0

PHP 5.2.9

MySQL 5.1.33
The location of the root web pages can be set from the Apache configuration file
http.conf. The file is located at the path: apache\conf\httpd.conf
DocumentRoot "/xampplite/htdocs"
DocumentRoot indicates the directory out of which you will server your documents.
<Directory "/xampplite/htdocs">
Directory location should be changed to whatever you set DocumentRoot to.
Since the system use a lot of JQuery plotting, there are some includes necessary for the
JQuery to run. At the top of the page where JQuery plot is running, includes the
following:
<script language="javascript" type="text/javascript" src="flot/jquery.js"></script>
<script language="javascript" type="text/javascript" src="flot/jquery.flot.js"></script>
In addition, in order to connect a PHP page to MySQL server, the following include is
needed:
require_once('include_database.php');
Within the include_database.php, you can set up preferred MySQL connection:
$this->host
$this->db
$this->user
$this->pass
$this->dbtype
= "localhost";
= "project";
= "root";
= "";
= $GLOBALS["DB_TYPE_MYSQL"];
54
BIBLIOGRAPHY
[1] Wikipedia. “Structural health monitoring” [Online].Available:
http://en.wikipedia.org/wiki/Structural_health_monitoring
[2] Civil Engineering Portal. “Tacoma Narrows Bridge Collapse” [Online].Available:
http://www.engineeringcivil.com/tacoma-narrows-bridge-collapse.html
[3] JIANG, Jianjing, LU, Xinzheng, GUO, Jingjun. (2002). “Study for Real-time Monitoring
of Large-Span Bridge Using GPS” [Online].Available:
http://luxinzheng.net/publications/11.pdf
[4] Meng, Xiaolin. (2002). “Detecting Bridge Dynamics with GPS and Triaxial
Accelerometers” [Online].Available:
http://etheses.nottingham.ac.uk/279/1/Meng%27s_full_thesis.pdf
[5] Turner, Loren. (2003). “Continuous GPS: Pilot Applications – Phase II” California
Department of Transportation. Sacramento. CA. [Online].Available:
http://www.dot.ca.gov/research/researchreports/2002-2006/2003/fhwa-ca-ir-200305.pdf
[6] Et-arts.com “Carquinez Bridge” [Online].Available: http://www.et-arts.com/bridge/
[7] Novatel. “RTKNav Real-Time Processing Software” [Online].Available:
http://www.novatel.com/products/waypoint-software/real-time-processing-software/
[8] National Instruments. “NI LabVIEW” [Online].Available:
http://www.ni.com/labview/
[9] Apache. “Apache HTTP Server Project” [Online].Available: http://httpd.apache.org/
55
[10] Wikipedia. “Wikipedia: PHP” [Online].Available:
http://en.wikipedia.org/wiki/PHP
[11] Wikipedia. “Wikipedia: MySQL” [Online].Available:
http://en.wikipedia.org/wiki/MySQL
[12] MySQL. “Download MySQL Community Server” [Online].Available:
http://www.mysql.com/downloads/mysql/
[13] Google Code. Flot. “Flot – Attractive Javascript plotting for jQuery”
[Online].Available: http://code.google.com/p/flot/
[14] Google Code. Google Earth API. “What is the Google Earth API”
[Online].Available: http://code.google.com/apis/earth/