Assignment 13: Test Plan
Write a high-level test plan for the ATM Simulation by Russell C. Bjork. This includes testing scope, testing
approach, testing criteria and quality checkpoints, and tools and automation strategy.
Author(s)
William Rozzo and Jonathan Wesel
Summary
...
Significant Findings or Learning
We found it difficult to complete the test plan without a proper Requirements Specification. The ATM
Simulation had requirements, but they were very informal. In order to make sense of it, we had to
reorganize it and assign requirement IDs. This made it easer to reference them in the scope and throughout
the plan.
We also found it tricky to think in terms of how things would be done since the true development was
already completed, and it was not actually done by us. Once in the right mindset however, we found it a good
exercise of the way one might want to approach a project with respect to testing. It was helpful to point out
the testing goals and the focus of the testing in terms of what we were mostly concerned with. This is a very
important thing to do in a real world application of a test plan. For example, in a very large-scale system with
over 900 source code files and thousands of lines of code it would be a very daunting task to test each and
every piece of the code. It is much more realistic to have a focal point and goals to consider and mold the
testing against.
Honor pledge
I pledge on my honor that I have not given or received any unauthorized assistance on this
assignment/examination. I further pledge that I have not copied any material from a book, article, the
Internet or any other source except where I have expressly cited the source.
Detailed Analysis
Part 1: Testing Scope
Testing the ATM Simulation will place focus on end user facing functions. This includes the four main options
given to the customer, Deposit, Withdrawal, Transfer, and Balance Inquiry. It will concern the software
dealing with bank transactions, customer accounts, and card verification.
Top priority will be given to any paths that result in funds being added or removed from accounts through
deposits and withdrawals. The reasoning behind this is that problems in these areas hold the most risk for
key stakeholders. It could be costly for the bank customer to fix and frustrating for the ATM user.
The table below describes the requirements that will be included in the test plan.
Req ID
2.1
2.2
2.3
3.1
3.2
3.3
3.5
3.6
3.7
3.8
3.9
4.1
5.1
6.1
7.1
Requirement Description
The ATM will service one customer at a time
A customer will be required to insert an ATM card and enter a personal identification number (PIN)
The customer will be able to perform one or more transactions.
A customer must be able to make a cash withdrawal from any suitable account linked to the card,
in multiples of $20.00.
Approval must be obtained from the bank before cash is dispensed.
A customer must be able to make a deposit to any account linked to the card, consisting of cash
and/or checks in an envelope.
Approval must be obtained from the bank before physically accepting the envelope.
A customer must be able to make a transfer of money between any two accounts linked to the
card.
A customer must be able to make a balance inquiry of any account linked to the card.
A customer must be able to make a balance inquiry of any account linked to the card.
A customer must be able to abort a transaction in progress by pressing the Cancel key instead of
responding to a request from the machine.
The ATM will communicate each transaction to the bank and obtain verification that it was allowed
by the bank.
If the bank determines that the customer's PIN is invalid, the customer will be required to re-enter
the PIN before a transaction can proceed.
If a transaction fails for any reason other than an invalid PIN, the ATM will display an explanation of
the problem, and will then ask the customer whether he/she wants to do another transaction.
The ATM will provide the customer with a printed receipt for each successful transaction, showing
the date, time, machine location, type of transaction, account(s), amount, and ending and available
balance(s) of the affected account ("to" account for transfers).
The test plan will not deal with the mechanics of the simulation. The simulation will be tested as if it were a
real ATM and will not include any physical functions such as filling the machine with cash, inserting the credit
card, or dispensing money. The purpose of the simulation is to ensure the ATM software functions correctly,
and testing the simulation portion would not be beneficial in the view of key stakeholders such as the bank
customer.
Part 2: Testing Approach
Test Approach
The ATM simulation is a system in which the need for security and reliability are most important. Based on
the type of system the ATM simulations is, a set of testing goals and objectives were created. The goals and
objectives are based on what are considered the key factors for ensuring a successful product, most
importantly security and reliability.
Goals:
 Remove as many security related flaws from the system as possible
 Increase overall system reliability



Increase defect prevention
Remove existing defects
Increase the probability the system will adhere to requirements
Objectives:
 Ensure that the system performs all security related tasks reliably
 Ensure that the system performs its core functionality in a timely manner
 Ensure that the system performs all core functionality correctly
 Automate testing where possible to save resources
In the process of establishing a test plan that will lead to a successful product, key project members got
together and came up with a list of risks that have a direct impact on product quality. The most important
risks considered were schedule, staffing, development bottlenecks such as software waiting on hardware,
and training necessary for engineers on tools. The plan for testing has been developed with these key risks in
mind. For example, the tools for testing and development were chosen based on existing experience of test
and development engineers to decrease the need for training which could negatively affect the schedule.
Additional risk details are outlined in the table below.
Number
1
Risk
Schedule
Description
Any slips in meeting
project milestones
on time.
2
Staffing
Loss of key team
members. At any
time someone may
get sick or leave the
company, this must
be taken in to
consideration.
3
Bottlenecks
4
Training
This could be a
bottleneck with
respect to an
imbalance in
engineer experience
(others cannot
proceed without the
experience of
another), or one
group such as
software waiting on
hardware before it
can continue.
All engineers must
have the proper
training in all areas
Effect
Direct impact on
budget, cost increases,
customer satisfaction
decreases.
Direct impact on budget
and schedule due to
improper manning. This
could also cause
additional training to be
needed if an untrained
replacement must be
attained.
Schedule slips due to
waiting on a single
group or engineer.
Decreased customer
satisfaction and trust.
Negative effect on
budget.
Mitigation Strategy
Add some grace to the schedule
estimates to compensate for
possible slips.
Tool training can take
weeks depending on
how many different
Select as many tools as possible that
engineers already have experience
using to reduce the need for
From the start ensure there is the
proper number of engineers for all
tasks throughout the project. Lay out
a backup plan for any missing
engineer in case this occurs.
Ensure all engineers have proper
training in areas necessary to avoid a
single point of failure. Mitigation for
proper scheduling should ensure the
least amount of bottlenecking with
respect to different groups waiting
on others before productivity can
continue.
he or she is
responsible.
tools are used and the
current experience of
engineers with such
tools. This could push
schedule back and
hence have a negative
budget impact. Training
for tools will cost
money as well.
additional training.
Acceptance of the ATM system is based on the following criteria:
 Formal Qualification Testing (FQT) is conducted
o All system requirements are tested against, if any requirement fails it must be reviewed with
the customer to decide impact and whether system can be accepted with the failure.
 All requirements regarding system reliability pass
 All requirements regarding system security pass
 All trouble reports closed/resolved
Development for the ATM system follows the Waterfall lifecycle model. Based on well known statistics
regarding defect removal efficiency, the testing team has decided to test the system early and as often as
resources permit. This means that for each phase of development testing will be conducted.
Requirements Phase:
The main deliverable from this phase of development is the requirements specification. Static testing will be
conducted on the requirements to ensure that they are complete and accurate. System requirements will
also be tested to ensure they adhere to the overall goals and objectives of the system.
Design Phase:
This phase of development will produce an overall system design, which will be a guideline for the system
implementation. To ensure the overall system design is sufficient, experts from both inside and outside of the
project team will review it. Reviewers will focus on the systems goals and objectives when reviewing the
design. This will allow further development phases to build upon a strong foundation.
Implementation Phase:
For each major ATM capability unit testing will be conducted on the security of critical components.
Additionally, after each capability is developed and unit tested, it will be added to the overall system and go
through integration testing.
Verification Phase:
Once all major capabilities are complete, as bugs are found and fixed regression testing will be conducted.
Also, initially the system will be alpha tested in house. Once alpha testing completes and the system is
nearing deployment, it will go through beta testing at select sites before it is officially deployed. Lastly, formal
qualification testing against the system requirements will be conducted.
Maintenance Phase:
Regression testing will continue throughout the maintenance phase as bugs are found or new functionality is
introduced to the system.
Part 3: Testing Criteria and Quality Checkpoints
The Test Plan begins with the Requirements Phase. The entrance criteria of this phase are simply that the
project begins. A request has been given by the stakeholders to develop the system through a statement of
work or request for proposal. The exit criteria are the production of the System Requirements Specification
(SRS). A quality checkpoint will include a review of the SRS where test team members are present.
The Design Phase begins when there is an SRS that has been agreed upon by all key stakeholders. The test
team will determine which requirements need to be tested. They will focus efforts on understanding design
documents dealing with these requirements. This phase ends once the design documents are completed and
reviewed. Checkpoints for this phase include the Preliminary Design Review (PDR) and the Critical Design
Review (CDR). These reviews will include a representative of the test team along with other key stakeholders.
Implementation begins upon successful completion of the CDR. The exit criteria are met when the system has
been fully integrated. The checkpoints would include the completion of each major development phase and
handed off for integration and test.
Upon completion of integration and test, the Verification Phase begins. This phase is complete when
regression tests are done and beta testing has been thoroughly carried out. These are the exit criteria.
The entry criteria for the Maintenance Phase are the completion of the beta testing and delivery of the
system to the customer. The exit criteria are simply the end of the products lifecycle.
Below is a table summarizing the entry & exit criteria of each phase along with checkpoints.
Phase
Requirements
Design
Implementation
Verification
Entry Criteria
RFP
Complete SRS
Successful CDR
Integration Complete
Exit Criteria
SRS
CDR
Integration Complete
Delivery
Maintenance
Delivery
End of Product Life Cycle
Quality Checkpoints
SRS Review
PDR & CDR
Build Phases
Regression & Beta tests,
FQT
Part 4: Tools and Automation Strategy
The ATM simulation utilizes a known dedicated platform for all ATMs on which the system is deployed. As
such, the test team does not have to be concerned with automated tests not covering different system
software configurations. The test teams goals for test automation is to both reduce resources required to
conduct tests and to allow ease of test repeatability. Testing facilities will include hardware identical to what
will be in the field so that tests can be conducted on the same hardware that the system will be finally
deployed onto.
Code will be developed using the Java programming language. To save funding and resources, Eclipse is the
chosen development environment as all test engineers have experience using this tool. Additionally Eclipse
provides debug configurations that allow test repeatability with minimal effort. All source and configuration
items created throughout development will be tracked using Subversion via the Subclipse Subversion plugin
for Eclipse. All bugs found during development will be tracked using the Mantis bug-tracking tool. Any test
scripts that can aide test automation and ease of repeatability will be developed using the Python scripting
language. Python was chosen due to its strength as a language, the fact that it is extremely easy to learn and
the speed at which test scripts can be developed when compared to other scripting languages such as Perl.
The following is a summary of the tools to be utilized for test automation efforts.
Activity/Test
Development Environment
Configuration Management
Defect Tracking
Automated Test Scripts
Tool
Eclipse
Subversion, Eclipse Subclipse Plugin
Mantis
Python, Eclipse configurations