Running head: Lab 3 – Current ITS Prototype Test Plan
Lab 3 – Current ITS Prototype Test Plan
Red Team
Brian Dunn
CS411W
Professor Brunelle
11/20/2012
1
Lab III – Current ITS Prototype Test Plan
2
+
Table of Contents
1. OBJECTIVES ............................................................................................................................. 4
2. REFERENCES ........................................................................................................................... 6
3. TEST PLAN................................................................................................................................ 7
3.1 Testing Approach .................................................................................................................. 8
3.2 Identification of Tests ........................................................................................................... 9
3.3 Test Schedule ...................................................................................................................... 21
3.4 Fault Reporting and Data Recording .................................................................................. 22
3.5 Resource Requirements ...................................................................................................... 23
3.6 Test Environment ................................................................................................................ 24
4. TEST RESPONSIBILITIES ..................................................................................................... 25
5. TEST PROCEDURES .............................................................................................................. 26
5.1 Test Case Name and Identifier .............................................Error! Bookmark not defined.
6. TRACEABILITY TO REQUIREMENTS ................................Error! Bookmark not defined.
List of Figures
Figure 1: Classroom Layout........................................................................................................ 24
Lab III – Current ITS Prototype Test Plan
3
List of Tables
Table 1: Test Case Outline ............................................................................................................ 20
Table 2: Test Schedule .................................................................................................................. 22
Lab III – Current ITS Prototype Test Plan
4
1. Objectives
Current ITS is a product designed to enhance communication and real-time information
delivery platforms in light rail systems. The initial prototype, under design by the Old Dominion
University (ODU) CS411 Red Team, will focus on The Tide light rail system in Norfolk, VA. In
operation by Hampton Roads Transit (HRT), The Tide is currently lacking any form of real-time
tracking and alerts for both riders and administrators. Current ITS will solve this with a web
based solution using HyperText Markup Language (HTML) and Hypertext Preprocessor (PHP)
for most of the code in addition to Cascading Style Sheets (CSS) and JavaScript for some
features.
The prototype of Current ITS is intended to validate these features using a Graphical User
Interface (GUI) that presents each user with modules that allow interaction with the system.
Through this, riders, business owners, and HRT operators will be served data from simulated
sensors and provided with a realistic emulation of the product in a real world scenario. All three
of the GUI’s layouts will be similar, but key differences in the level of detail and available
modules will make them distinct. This difference in approach for each user will be important, as
the GUI for riders will emphasize quick access with upfront data, administrators will want more
in depth information with more control, and business users will fall in between. Through the
simulated use of each GUI, it will be displayed that the modules are intuitive and relatively
simple to use.
The second objective of the prototype will be to demonstrate the Decision Engine and
Database Server capabilities that provide functionality to various modules in the system. The
highlight of the Decision Engine will be the trend analysis algorithm that will search historical
Lab III – Current ITS Prototype Test Plan
5
data to pick up on trends that impact ridership levels. These trends will be used to both highlight
past ridership issues and predict future ridership levels, which will complement the real-time data
with past and future data. The on-time performance report will perform a similar role, but
instead of ridership levels it will track whether trains are arriving on schedule or not. While the
prototype will not automate this data into practical applications, it will still demonstrate the
potential use by HRT in regards to planning for varying levels of train ridership.
The third set of objectives will be to demonstrate the alert and feedback systems, which
will be the key to enhancing communication within light rail systems like The Tide. The alert
system will start with the HRT operators being able to submit alerts to the system through a
simple web form. This will then present the information to all three user GUIs, as well as the
Short Message Service (SMS) phone alert system. In order for users to receive these alerts on
their phone, they must opt-in by submitting their phone number, to which the GUI alert delivery
mechanisms will automatically display the messages. The feedback system will demonstrate the
ability for riders and business owners to communicate their opinions to HRT administrators
using another basic web form submission that will automatically mail the feedback to a
configurable e-mail address in real-time.
The final objective of the Current ITS prototype will be to display the overall
functionality of the system through the use of a test harness. Since all of the data will be
simulated instead of using actual train sensors and running the test through multiple weeks, the
test harness will serve as a method to manipulate this data in real-time. This will allow Current
ITS to display its ability to handle various scenarios as they happen, ranging from advertisement
campaigns ending to bizarre events such as train derailments. The test harness will have its own
Lab III – Current ITS Prototype Test Plan
6
GUI to facilitate easy testing, but its impact will be seen in the actual prototype and generate
behavior mirroring real world functionality.
2. References
Lab I – Current ITS Product Description. Version 2. (2012, September). Current ITS. Red Team.
CS411W: Brian Dunn.
Lab II – Current ITS Prototype Product Specification. Version 1. (2012, October). Current ITS.
Red Team. CS411W: Brian Dunn.
Lab III – Current ITS Prototype Test Plan
7
3. Test Plan
This section will introduce the procedures to test the Current ITS prototype, including the
general approach that will be used to facilitate testing. The bulk of this section will include an
outline of the specific tests that will be used to check the functionality of each module, interface,
and engine. In addition, the section will include details regarding scheduling, fault reporting,
requirements, and the test environment. Through this section, the full plan for the Current ITS
testing phase will be established.
(This space intentionally left blank.)
Lab III – Current ITS Prototype Test Plan
8
3.1 Testing Approach
The general approach to the testing phase of Current ITS will be to simulate the various
scenarios anticipated for each level of user. The experience for a business user or a general rider
will be very different, so while the functionality of many modules will be tested in similar ways,
it will still require each GUI to be tested. Aside from the GUI testing, there will be testing
performed on the backend of the system, namely the Decision Engine and Database Server.
Finally, to emulate changes in the simulated environment, the test harness will control variables
outside of the standard GUI such as readings from the sensors and manipulating timestamps to
mimic changes in time.
(This space intentionally left blank.)
Lab III – Current ITS Prototype Test Plan
9
3.2 Identification of Tests
Test cases for the Current ITS prototype are listed in Table 1. These test cases have been
labeled with three different categories: unit, integration, and system. On top of that, they have
been assigned subcategories based on the component that they are intended to test. A more
detailed breakdown of these tests can be found in Section 5 of this document.
Test
Category
Description
Case
Description
Unit
Database
1.1
Formats
Objective
Visually examine the entries for date
(YYYYMMDD) and time (HHMMSS)
formats.
1.2
1.3
Read/Write
Test create, read, update, and delete
Procedures
stored procedures on the database.
Refresh
Times
1.4
Read/Write
Triggers
Test updateable views for functionality
and response/refresh time.
Test triggers for proper functionality on
various create, read, and delete
commands.
1.5
Schema
Efficiency
Test tables for proper schema,
functionality, and efficiency and ease of
use, and for proper data values.
Lab III – Current ITS Prototype Test Plan
10
Database
Decision
Test Current ITS database connection
2.1
Connectivity
Integration
Engine –
Test ability to query the database and
DB Interface
2.2
Select Ability
necessary tables
Input
Validate Input of date, time range, stop id
2.3
Validation
Interval
Test future/past date determination
2.4
Validation
Decision
Test average of embark/disembark for
Function Test
past 15 days
Past Event
Test past Event Detection
2.5
Engine –
Unit
Average
Ridership
2.6
Trend
Test
Analysis
Future Event
Test future Event Detection
2.7
Test
2.8
Ridership
Test accuracy of variance between
Variance
established disembark/embark averages
Function Test
and past event values
Lab III – Current ITS Prototype Test Plan
11
Ridership
2.9
Integration
Output
Test output of embark/disembark to
Ridership Trend Report function
Validation
Input
Validate input of GPS coordinates and
Validation
dates
Train
Test ability to determine if a train is
Activity Test
active or not
Delay
Test accuracy of average variance from
Average
the schedule
2.10
2.11
Decision
2.12
Calculation
Engine –
Test
Unit
Delay Impact
Calculator
2.13
Alert
Test ability to correctly identify active
Detection
alerts
Test
Alert Delay
Test accuracy of alert severity level delay
Interval
interval on variance average
2.14
Application
Test
Lab III – Current ITS Prototype Test Plan
2.15
12
Total
Test accuracy of comparison between
Calculated
calculated expected time-of-arrival and
Delay Test
schedule
Delay
Test output of delay time to Train Data
Estimate
Report Module
2.16
Integration
Output
Validation
Input
Validate input of Date range and Stop ID
2.17
Validation
Unit
Decision
Engine –
2.18
Ontime
Test accuracy of variance between past
Accuracy
arrival times and schedule times
Ontime
Test
Performance
Ontime
Test output of variance from the schedule
Reporting
Performance
to the Train Data Report Module
2.19
Integration
Output
Validation
Test
GPS Data
Harness
tester
3.1
GPS stop
Verify each virtual stop has an
tester
associated GPS coordinate.
Lab III – Current ITS Prototype Test Plan
3.2
3.3
13
GPS format
Verify GPS coordinate is in the correct
test
format
GPS route test
Verify the GPS route list contains a
proper GPS coordinates
3.4
GPS train test
Verify a train’s coordinate is updated
and valid.
Ridership
Ridership
Test virtual rider generation for each
Data Control
Data
stop
tester
generation test
Ridership
3.5
3.6
Data Control
Ridership
Ensure realistic proportions of riders are
Data test
generated conforming to variable
tester
thresholds which can be changed by the
user
Train control
3.7
tester
3.8
3.9
3.10
Train GPS
Verify each active train return assigned
test
GPS coordinate
Train sensor
Verify each train has the ability to
failure test
simulate sensor failure
Train outage
Verify the ability for a train to no return
failure
a GPS coordinate.
Ridership test
Verify each train has the ability to return
the current amount of riders on board
Business ad
control tester
3.11
End date test
Verify access to each advertisement’s
end date
Lab III – Current ITS Prototype Test Plan
3.12
14
Start date test
Verify access to each advertisement’s
start date
3.13
Advertisement Verify Ability to assign and view
stop test
3.14
Advertisement Verify access to each advertisement’s
start time
3.15
3.16
Interface
3.17
3.18
3.19
start time
Advertisement Verify access to each advertisement’s
end time
Test harness
advertisements assigned at each stop
end time
Train property Verify interface has the ability to display
GUI
each trains properties
Train settings
Verify interface has the ability to edit
test
different train settings
Stop property
Verify interface has the ability to display
test
ridership at each stop
Stop Property
Verify interface has the ability to edit
edit test
different ridership numbers at each stop.
Verify an alert can be
Manage
created/modified/closed within the HRT
4.1
Alerts
System
GUI
Alert Module
Demonstrate alerts are viewable in
4.2
View Alerts
module
Lab III – Current ITS Prototype Test Plan
Unit
Feedback
15
Submit
Verify a rider or business user can submit
Feedback
feedback via web form
Retrieve Stop
Verify DB interface provides results for
Information
stops & vehicles in operation
4.3
Module
System
System
4.4
Overview
Module
Demonstrate stop & vehicle information
4.5
Map Overlay
is displayed on a dynamic map
Google Maps
Google Maps
Unit
Verify Google Maps search is performed
4.6
Web
Web Form
for direction request
Redirection
Demonstrate Events are viewable within
4.7
View Events
the module
Calendar
System
Verify that Events can be added, edited
Event Module
Manage
or removed within the Business & HRT
4.8
Events
GUI
Webpage
Verify user a able to insert name, desired
Layout
user name, email address and password.
Username
Ensure username is validated and return
Validation
error if username exists.
5.1
User
Unit
Management
5.2
Lab III – Current ITS Prototype Test Plan
16
Username
Prove username is provided after
5.3
5.4
Retrieve
validation method.
User Self
Confirm user is able to reset password.
Password
Reset
Administrati
5.5
on
Validate administrator can change user
information, groups, and passwords
Application
User
5.6
Information
Verify user can edit personal
information.
Update
Group
5.7
Administrati
Demonstrate the administrators ability to
create and manage groups
on
Application
Confirm user is able to access application
5.8
Authenticatio
Unit
Access
through one-factor authentication.
Token
Validate proper identification token
5.9
n Module
Generation
generation.
Access
Prove access is granted only after token
Control
validation.
6.0
Lab III – Current ITS Prototype Test Plan
17
Confirm user is unable to access the
Security
system after a predetermined time of
6.1
Time-out
non-use.
Account
Validate user account is locked after
Locking
multiple login failure attempts.
6.2
Verify recording of a users login time,
6.3
Logging
location, authentication success or failure
and the page requested.
Group
6.4
Administrati
Demonstrate the administrators ability to
create and manage groups
on
Database
Confirm application opens access to the
6.5
Connection
database for data transfer
Validate query and result transfer
Authenticatio
Integration
Query
between the database and the Web
6.6
n Module
Transfer
Application Engine modules.
Demonstrate capability to prevent a SQL
6.7
Security
injection attack.
Lab III – Current ITS Prototype Test Plan
6.8
18
Decision
Verify data is sent between the Decision
Engine
Engine and the Web Application Engine.
Connection
Validate data is sent between the Test
Test Harness
Harness and the Web Application
6.9
Connection
Engine.
List
7.1
Verify listing of advertisements.
Advertiseme
nts
Create
Unit
Business Ad
7.2
Advertiseme
Campaign
nt
Module
Edit
7.3
Advertiseme
Verify submission of advertisement input
fields.
Verify modification of advertisement
input fields.
nt
Integration
Database
Verify interface with database for data
Interface
input/output.
Display
Verify display of default ridership trend
Default
report.
7.4
Ridership
Unit
8.1
Trend Report
Report
Lab III – Current ITS Prototype Test Plan
8.2
19
Display
Verify display of custom ridership trend
Detailed
report.
Report
8.3
Request
Verify input fields for custom report
Custom
query.
Report
Decision
Integration
8.4
Engine
Verify interface with Decision Engine for
data retrieval.
Interface
9.1
Display
Verify display of default train data
Default
report.
Report
Display
Verify display of custom
Train Data
Unit
9.2
Detailed
Report
Report
9.3
Request
Verify input fields for custom report
Custom
query.
Report
Lab III – Current ITS Prototype Test Plan
20
Decision
Integration
9.4
Engine
Verify interface with Decision Engine for
data retrieval.
Interface
Display
10.1
Verify display of modules on Rider GUI.
Rider
Modules
Graphical
Unit
10.2
User Interface
Display
Verify display of modules on Business
Business
GUI.
Modules
Framework
Display HRT Verify display of modules on HRT GUI.
10.3
Modules
Module
Verify interface with all Web
Interface
Application Engine modules.
10.4
Table 1: Test Case Outline
Lab III – Current ITS Prototype Test Plan
21
3.3 Test Schedule
For the presentation of the Current ITS prototype, the team will have 45 minutes to
demonstrate the testing procedures. The first 10 minutes will be dedicated to the feasibility
portion of the presentation, covering both the setup and the scope of Current ITS in the prototype
stage. After this, the algorithms and GUIs will be tested to display their functionality through
simulated scenarios. The presentation will conclude with an opportunity for the audience to ask
questions of the Current ITS team members.
Start Time
Duration
Description
Test Cases
(minutes)
(minutes)
0:00
10
Feasibility
0:10
5
Database Demo
1.0
0:15
10
Algorithm Unit
2.0,3.0
Covered
Tests
0:25
10
Integration Tests
4.1 – 4.7
-HRT
-Business
-Rider
0:35
10
System Tests
-HRT
-Business
-Rider
0:45
15
Q&A
5.0-10
Lab III – Current ITS Prototype Test Plan
22
Table 2: Test Schedule
3.4 Fault Reporting and Data Recording
Current ITS members from the Red Team will be recording results of each test,
specifically whether it succeeds or fails to meet the expected results. The test components
include the Database Server, Decision Engine, the GUI, and test harness. The procedure to
identify test outcomes will vary with each test, but consist of viewing GUI output or checking
the database through phpMyAdmin to determine if the appropriate result occurred. These results
will be recorded on paper so that the log of results can be used later for further development if
necessary.
(This space intentionally left blank.)
Lab III – Current ITS Prototype Test Plan
23
3.5 Resource Requirements
The resource requirements for the Current ITS prototype presentation will be minimal
due to the nature of the product. Since the prototype can be hosted in its entirety on a single web
server, the machine provided by the ODU Computer Science Department will suffice for
equipment to run Current ITS. In order to access the prototype, a computer with a recent version
of Mozilla Firefox, Internet Explorer, or Google Chrome with Javascript enabled will be
required. To continually monitor the phpMyAdmin interface throughout the testing, a second
laptop will be used during the presentation of the prototype. Finally, scripts and prepopulated
data will be used for input to the test harness to generate scenarios for the testing.
(This space intentionally left blank.)
Lab III – Current ITS Prototype Test Plan
24
3.6 Test Environment
The presentation will take place in the room in which CS411 is being held during the Fall
2012 semester. This room is found in the Gornto Center on the campus of Old Dominion
University in Norfolk, VA. The room features Teletechnet capabilities so that distance students
will be able to watch the presentation and participate in the Q&A session along with the live
audience. The team will present from the front of the room where the desk and desktop
computer are placed to allow for optimal viewing and recording.
Figure 1: Classroom Layout
Lab III – Current ITS Prototype Test Plan
25
4. Test Responsibilities
Test responsibilities for Current ITS will be based off of the same roles used for
development of the components in the prototype. Dean Maye developed the Database Server
and will be managing phpMyAdmin throughout the presentation. Akeem Edwards is in charge
of the test harness and will be using it during the presentation to manipulate the testing scenarios.
Nate Lutz created the algorithms used in the Decision Engine and will be using it to display the
system’s ability to process real-time data. Chris Coykendall is in charge of the modules for
Events, System Overview, Feedback, Alerts, and the Google Maps Web Form. Brian Dunn
developed and is responsible for testing the modules for Ridership Trend Report, Train Data
Report, Business Advertisement Campaign, and the overall GUI Framework. Finally, CJ Deaver
is the developer for the User Management, Authentication, and the Integration modules for the
Database Server and Decision Engine.
(This space intentionally left blank.)
Lab III – Current ITS Prototype Test Plan
26
5. Test Procedures
The following sections will expand further upon the previously described testing plan. In
Section 5.1, a fully detailed list of the test cases developed for the Current ITS prototype is
shown. The two most important sections of the prototype test cases are the test inputs and
expected results. However, in order to make sure testing is recorded efficiently and setup
properly, other information has also been included. Using this test list, the Current ITS team can
test the functionality of the product during the prototype presentation.
(This space intentionally left blank.)