Running Head: Lab III – Current ITS Prototype Test Plan/Procedure Outline
Lab III - Product Specification Outline
CS 411W Lab III
Prototype Test Plan/Procedure Outline
For
Current ITS
Prepared by: Chris Coykendall, Red Team
Date: 11/20/2012
1
Lab III – Current ITS Prototype Test Plan/Procedure Outline
2
Table of Contents
1
2
3
OBJECTIVES ......................................................................................................................... 3
REFERENCES ....................................................................................................................... 4
TEST PLAN............................................................................................................................ 5
3.1
Testing Approach ............................................................................................................ 5
3.2
Identification of Tests ..................................................................................................... 6
3.3
Test Schedule ................................................................................................................ 12
3.4
Fault Reporting & Data Recording ............................................................................... 13
3.5
Resource Requirements ................................................................................................ 14
3.6
Test Environment .......................................................................................................... 14
4
TEST RESPONSIBILITIES ................................................................................................. 15
5
TEST PROCEDURES .......................................................................................................... 15
6
TRACEABILITY TO REQUIREMENTS ........................................................................... 15
List of Figures
Figure 1: Current ITS Prototype Major Functional Component Diagram ...................................... 6
Figure 2: Demonstration Room Layout ........................................................................................ 14
List of Tables
Table 1: Test Case Outline ............................................................................................................ 12
Table 2: Test Schedule of Current ITS Demonstration ................................................................ 13
Table 3: Fault Reporting & Data Recording ................................................................................. 13
Lab III – Current ITS Prototype Test Plan/Procedure Outline
3
1 OBJECTIVES
This document details the testing plan and procedures of the Current Intelligent
Transportation System (ITS) prototype. To ensure an accurate depiction of the real-world
product, a framework is presented here for ensuring that the Current ITS prototype meets all
functional requirements set forth in the prototype description (Lab II). The testing plan
framework is comprised of a suite of tests which encompass all use-case-driven requirements, as
well as subsequent non-functional quality requirements to demonstrate proof of concept.
Lab III – Current ITS Prototype Test Plan/Procedure Outline
4
2 REFERENCES
American Public Transportation Association. "Ridership Increases In Third Quarter." Policy
Development and Research Program at APTA, Dec. 2011. Web.
<http://apta.com/resources/statistics/Documents/Ridership/2011-q3-ridershipAPTA.pdf>.
Lab I – Current ITS Product Description. Version 2. (2012, October.). Current Intelligent
Transportation System. Red Team. CS411W: Chris Coykendall.
Lab I I – Current ITS Product Specification. Version 1. (2012, November.). Current Intelligent
Transportation System. Red Team. CS411W: Chris Coykendall.
Press Release Distribution. "Global Light Rail Market to Reach $7.5 Billion by 2015, According
to New Report by Global Industry Analysts, Inc." 14 July 2010. Web.
<http://www.prweb.com/releases/light_rail/light_rail_transit/prweb4253534.htm>.
Lab III – Current ITS Prototype Test Plan/Procedure Outline
5
3 TEST PLAN
The Current ITS testing plan described here provides the processes and procedures
necessary to facilitate a thorough quality assurance testing strategy, as well as a sufficient
demonstration of prototype functionality. The processes and procedures in this section are the
result of an in-depth analysis by the development team of the Current ITS prototype design and
subsequent implementation. They incorporate the use of best practices such as black-box, whitebox, and usability testing.
3.1 Testing Approach
The test cases for the Current ITS prototype are comprised of three basic categories: Unit
Tests, Integration Tests, and System Tests. Unit testing will provide coverage of testing modular,
independent blocks of software, while integration tests will ensure that these small blocks fit
together in the appropriate manner. System testing will incorporate the larger functionalities
(referred to as “stories” in many development models) that are driven by use cases of the
potential customer, Hampton Roads Transit (HRT). Each major component of the Current ITS,
described in the Current ITS Product Specification (Lab II) will contain various tests from these
three categories.
To facilitate the automation of mundane tasks such as inserting, updating or deleting
records, the Current ITS prototype testing will be automated through a Test Harness. This Test
Harness will provide the capability to trigger tests and specific events in all of the major
components of the prototype, include the Web Application Engine (WAE), Decision Engine
(DE), and Database (Figure 1). For demonstration purposes, the Test Harness will also have a
Lab III – Current ITS Prototype Test Plan/Procedure Outline
6
front-end GUI which can make these changes on the fly. The first tests to be completed will be
the Unit Tests, followed by Integration and then subsequently System Tests. There are
interweaved dependencies between the latter two types, which are defined in Section 5 of this
document.
Figure 1: Current ITS Prototype Major Functional Component Diagram
3.2 Identification of Tests
The overall outline of the Current ITS prototype test suite is shown in Table 1 of this
document. The development team has identified the individual tests in this outline as a result of
an in-depth analysis of the requirements specification (Lab II), and collaboration amongst the
group and mentors. The tests identified here are organized according to the appropriate
requirement or major functional component, and are described in full technical detail in Section
5 of this document.
Lab III – Current ITS Prototype Test Plan/Procedure Outline
Category
Unit
Integration
Description
Database
Decision
Engine –
DB Interface
Test
Case
1.1
Read/Write
Procedures
1.3
Refresh Times
1.4
Read/Write
Triggers
Test triggers for proper functionality
on various create, read, and delete
commands.
1.5
Schema
Efficiency
Test tables for proper schema,
functionality, and efficiency and ease
of use, and for proper data values.
Test Current ITS database connection
2.1
Database
Connectivity
2.2
Select Ability
2.4
Unit
2.5
2.6
2.7
2.8
Integration
Unit
2.9
Decision
Engine –
Delay Impact
Calculator
Objective
Visually examine the entries for date
(YYYYMMDD) and time (HHMMSS)
formats.
Test create, read, update, and delete
stored procedures on the database.
1.2
2.3
Decision
Engine –
Ridership
Trend
Analysis
Description
Formats
7
2.10
2.11
Input
Validation
Interval
Validation
Average
Function Test
Past Event Test
Future Event
Test
Ridership
Variance
Function Test
Ridership
Output
Validation
Input
Validation
Train Activity
Test
Test updateable views for functionality
and response/refresh time.
Test ability to query the database and
necessary tables
Validate Input of date, time range, stop
id
Test future/past date determination
Test average of embark/disembark for
past 15 days
Test past Event Detection
Test future Event Detection
Test accuracy of variance between
established disembark/embark averages
and past event values
Test output of embark/disembark to
Ridership Trend Report function
Validate input of GPS coordinates and
dates
Test ability to determine if a train is
active or not
Lab III – Current ITS Prototype Test Plan/Procedure Outline
2.12
2.13
2.14
2.15
Integration
2.16
2.17
Unit
Integration
Test
Harness
Decision
Engine –
Ontime
Performance
Reporting
GPS Data
tester
2.18
2.19
3.1
8
Delay Average Test accuracy of average variance from
Calculation
the schedule
Test
Alert Detection Test ability to correctly identify active
Test
alerts
Alert Delay
Test accuracy of alert severity level
Interval
delay interval on variance average
Application
Test
Total
Test accuracy of comparison between
Calculated
calculated expected time-of-arrival and
Delay Test
schedule
Delay Estimate Test output of delay time to Train Data
Output
Report Module
Validation
Input
Validate input of Date range and Stop
Validation
ID
Ontime
Test accuracy of variance between past
Accuracy Test
arrival times and schedule times
Ontime
Test output of variance from the
Performance
schedule to the Train Data Report
Output
Module
Validation
GPS stop tester Verify each virtual stop has an
associated GPS coordinate.
3.2
GPS format
test
Verify GPS coordinate is in the correct
format
3.3
GPS route test
Verify the GPS route list contains a
proper GPS coordinates
3.4
GPS train test
Verify a train’s coordinate is updated
and valid.
Ridership Data
Control tester
3.5
Ridership Data
generation test
Test virtual rider generation for each
stop
Ridership Data
Control tester
3.6
Ridership Data
test
Ensure realistic proportions of riders
are generated conforming to variable
thresholds which can be changed by
the user
Lab III – Current ITS Prototype Test Plan/Procedure Outline
Train control
tester
Business ad
control tester
Test harness
Interface
9
3.7
Train GPS test
Verify each active train return
assigned GPS coordinate
3.8
Train sensor
failure test
Verify each train has the ability to
simulate sensor failure
3.9
Train outage
failure
Verify the ability for a train to no
return a GPS coordinate.
3.10
Ridership test
Verify each train has the ability to
return the current amount of riders on
board
3.11
End date test
Verify access to each advertisement’s
end date
3.12
Start date test
Verify access to each advertisement’s
start date
3.13
Advertisement
stop test
Verify Ability to assign and view
advertisements assigned at each stop
3.14
Advertisement
start time
Verify access to each advertisement’s
start time
3.15
Advertisement
end time
Verify access to each advertisement’s
end time
3.16
Train property
GUI
Verify interface has the ability to
display each trains properties
3.17
Train settings
test
Verify interface has the ability to edit
different train settings
Lab III – Current ITS Prototype Test Plan/Procedure Outline
System
Unit
System
Unit
3.18
Stop property
test
Verify interface has the ability to
display ridership at each stop
3.19
Stop Property
edit test
Verify interface has the ability to edit
different ridership numbers at each
stop.
System
Overview
Module
Google Maps
Web Form
Manage Alerts
4.2
View Alerts
4.3
Submit
Feedback
Verify a rider or business user can
submit feedback via web form
4.4
Retrieve Stop
Information
Verify DB interface provides results
for stops & vehicles in operation
4.5
Map Overlay
Demonstrate stop & vehicle
information is displayed on a dynamic
map
4.6
Google Maps
Web
Redirection
Verify Google Maps search is
performed for direction request
4.7
System
Calendar
Event Module
4.8
5.1
5.2
5.3
Unit
User
Management
Verify an alert can be
created/modified/closed within the
HRT GUI
Demonstrate alerts are viewable in
module
4.1
Alert Module
Feedback
Module
10
5.4
5.5
5.6
Demonstrate Events are viewable
within the module
Verify that Events can be added, edited
Manage Events
or removed within the Business &
HRT GUI
Verify user a able to insert name,
Webpage
desired user name, email address and
Layout
password.
Username
Ensure username is validated and
Validation
return error if username exists.
Username
Prove username is provided after
Retrieve
validation method.
User Self
Confirm user is able to reset password.
Password
Reset
Administration Validate administrator can change user
Application
information, groups, and passwords
User
Verify user can edit personal
Information
information.
Update
View Events
Lab III – Current ITS Prototype Test Plan/Procedure Outline
5.7
5.8
5.9
6.0
Unit
Authentication
Module
6.1
6.2
6.3
6.4
6.5
6.6
Integration
Authentication
Module
6.7
6.8
6.9
7.1
Unit
Business Ad
Campaign
Module
Integration
7.2
7.3
7.4
8.1
Unit
Ridership
Trend Report
8.2
Group
Administration
11
Demonstrate the administrators ability
to create and manage groups
Confirm user is able to access
Application
application through one-factor
Access
authentication.
Token
Validate proper identification token
Generation
generation.
Prove access is granted only after
Access Control
token validation.
Confirm user is unable to access the
Security Timesystem after a predetermined time of
out
non-use.
Account
Validate user account is locked after
Locking
multiple login failure attempts.
Verify recording of a users login time,
Logging
location, authentication success or
failure and the page requested.
Group
Demonstrate the administrators ability
Administration to create and manage groups
Database
Confirm application opens access to
Connection
the database for data transfer
Validate query and result transfer
Query Transfer between the database and the Web
Application Engine modules.
Demonstrate capability to prevent a
Security
SQL injection attack.
Decision
Verify data is sent between the
Engine
Decision Engine and the Web
Connection
Application Engine.
Validate data is sent between the Test
Test Harness
Harness and the Web Application
Connection
Engine.
List
Verify listing of advertisements.
Advertisements
Create
Verify submission of advertisement
Advertisement input fields.
Edit
Verify modification of advertisement
Advertisement input fields.
Database
Verify interface with database for data
Interface
input/output.
Display
Verify display of default ridership
Default Report trend report.
Display
Verify display of custom ridership
Detailed
trend report.
Report
Lab III – Current ITS Prototype Test Plan/Procedure Outline
8.3
Integration
8.4
9.1
Unit
9.2
Train Data
Report
9.3
Integration
9.4
10.1
Unit
10.2
Graphical
User Interface
Framework
10.3
10.4
Request
Custom Report
Decision
Engine
Interface
Display
Default Report
Display
Detailed
Report
Request
Custom Report
Decision
Engine
Interface
Display Rider
Modules
Display
Business
Modules
Display HRT
Modules
Module
Interface
12
Verify input fields for custom report
query.
Verify interface with Decision Engine
for data retrieval.
Verify display of default train data
report.
Verify display of custom
Verify input fields for custom report
query.
Verify interface with Decision Engine
for data retrieval.
Verify display of modules on Rider
GUI.
Verify display of modules on Business
GUI.
Verify display of modules on HRT
GUI.
Verify interface with all Web
Application Engine modules.
Table 1: Test Case Outline
3.3 Test Schedule
For demonstration of the Current ITS prototype, the project team will be allotted :45
minutes to present before a panel audience. In order to fully appreciate the prototype
implementation, the demonstration will be performed bottom-up, beginning at the back-end
Database tests all the way through System tests. The test schedule of the presentation is shown in
Table 2. The conclusion of the demonstration will be followed by a Q&A session for the panel.
Start Time
(minutes)
0:00
0:10
Duration
(minutes)
10
5
Description
Feasibility
Database Demo
Test Cases
Covered
1.0
Lab III – Current ITS Prototype Test Plan/Procedure Outline
0:15
10
0:25
10
0:35
10
0:45
15
Algorithm Unit
Tests
Integration Tests
-HRT
-Business
-Rider
System Tests
-HRT
-Business
-Rider
Q&A
13
2.0,3.0
4.1 – 4.7
5.0-10
Table 2: Test Schedule of Current ITS Demonstration
3.4 Fault Reporting & Data Recording
During the test schedule, the Current ITS prototype development team will record the
results of tests performed. Some of the tests performed will be verified through logging, while
most of the tests for demonstration purposes will be conducted visually to engage the audience.
The fault reporting and data recording process is described in Table 3.
Component
Recording Process
-
Database
-
Test Harness GUI
-
Decision Engine Algorithms
-
Web App Engine GUI
-
Report failures through visual inspection of
database records using SQL queries
Document using paper forms
Report failures through visual inspection of
output
Document using paper forms
Report failures through visual inspection of
output in log
Document using paper forms
Report failures using visual inspection of GUI
screens in Web App Engine and in Test Harness
Document using paper forms
Table 3: Fault Reporting & Data Recording
Lab III – Current ITS Prototype Test Plan/Procedure Outline
14
3.5 Resource Requirements
The Current ITS prototype employs a number of boilerplate software packages residing
within a virtual machine. This software includes Apache Web Server, PHP, MySQL,
phpMyAdmin, CentOS, and more. Obtaining access to these resources necessitates having access
to a desktop workstation computer with Internet connectivity to the Old Dominion University
Computer Science Department network. The Current ITS Web Application Engine is located at
https://www.odin.cs.odu.edu:8080/.
3.6 Test Environment
In order for the audience to adequately view the screen, the demonstration will also
require larger monitors available in the room. For this purpose, the demonstration will take place
at the Gornto Teletechnet building on the Old Dominion University campus. The layout of the
demonstration room is depicted in Figure 2.
Figure 2: Demonstration Room Layout
Lab III – Current ITS Prototype Test Plan/Procedure Outline
4 TEST RESPONSIBILITIES
Please see collaborative document Lab III Sections 4-6.
5 TEST PROCEDURES
Please see collaborative document Lab III Sections 4-6.
6 TRACEABILITY TO REQUIREMENTS
Please see collaborative document Lab III Sections 4-6.
15