LANGUAGE TEACHER (LT)
TEST PLAN
February 23, 1998
Prepared for:
University of Massachusetts Boston, Harbor Campus
100 Morrissey Boulevard
Boston, MA 02125-3393
Professor William Campbell, Ph.D.
Laboratory in Software Engineering I, II (CS 611, CS 613)
Prepared by:
Language Teacher (LT) Project Team
Satit Phanichyakarn, Project Lead
Contributors: James R. Hill, William Perry, Malisetti Sagar
TABLE OF CONTENTS
1.0
INTRODUCTION
1.1
GENERAL
1.2
PURPOSE
1.3
SCOPE
Page
1
1
1
1
2.0
APPLICABLE DOCUMENTS
2
3.0
VERIFICATION METHODS
3.1
“WHITE BOX”
3.2
“BLACK BOX”
3.3
INSPECTION
3.4
ANALYSIS
3.5
DEMONSTRATION
3.6
TEST
3.7
REQUIREMENTS VERIFICATION CROSS-REFERENCE MATRIX
3
3
3
3
3
3
4
4
4.0
TEST PROCEDURE DEVELOPMENT
4.1
SPECIFICATION COMPLIANCE SCHEDULE
4.2
TEST PROCEDURE FORMAT
7
7
7
5.0
TEST CONDUCT
5.1
TEST CONDUCT
5.2
FAILURE REPORTING
5.3
DOCUMENTATION CONTROL
5.4
TEST RESPONSIBILITIES
5.5
TEST REPORTS
10
10
10
10
10
11
ii
LIST OF FIGURES
Page
Figure 3.1-1
Requirements Verification Cross-Reference Matrix
5,6
Figure 4-1
Sample Test Procedure
8
Figure 4-2
Test Data Sheet
9
Figure 5-1
Bug Report
12
Figure 5-2
Preliminary Test Report Format
13
iii
1.0
1.1
INTRODUCTION
GENERAL
This plan describes the concepts, practices, policies, and procedures that the Language Teacher team will
use to test a LT system. The plan is designed to meet industry level test standards and is tailored to meet
specific requirements of the Language Teacher system as described in our Engineering Requirements
Specification, dated December 16, 1997.
1.2
PURPOSE
The purpose of the test plan is to provide assurance to Henry Batterman and William Campbell that the
developed Language Teacher (LT) product meets the technical requirements as a suitable training aid for
use by language students.
1.3
SCOPE
This test plan presents the test structure that is specific to the Language Teacher system. It identifies the
categories of tests and methods of verification, and it also prescribes a breakout for implementing the
defined test program. The plan also documents the responsibilities and procedures for implementing
unit, subsystem, and system tests in accordance with good engineering practices.
1
2.0
APPLICABLE DOCUMENTS
Language Teacher (LT) Project Plan; November 5, 1997, revised November 17, 1997; The
Language Teacher Project Team
Engineering Requirements Specification (ERS) of Language Teacher (LT); December 16, 1997;
FINAL, The Language Teacher Project Team
Language Teacher (LT) Prototype User’s Guide; January 20, 1998; The Language Teacher
Project Team
“Language Teacher Project Presentation;” February 2, 1998; The Language Teacher Project
Team
Object-Oriented Software Engineering, A Use Case Driven Approach; 1992; Addison-Wesley
Publishing Company; Ivar Jacobson
2
3.0
VERIFICATION METHODS
Compliance with the technical requirements for the Language Teacher (LT) system will be
verified in two modes and by four methods. The two modes employed are “White Box” testing
and “Black Box” testing. The methods are: inspection, analysis, demonstration, and test. Each
of these methods will provide a mean of indicating pass/fail with respect to LT characteristics
being tested and evaluated. Descriptions follow.
3.1
“WHITE BOX”
“White Box” testing is conducted with full knowledge of the internals of the system under test.
That is the tester has schematics, code, and/or other documentation by which critical decision
points can be determined and code sequences that can be divided into independent block for
individual testing. The majority of our unit tests will be conducted in this mode.
3.2
“BLACK BOX”
“Black Box” testing is conducted without knowledge of the internals of the system under test.
Performance/requirements specifications, User Guides, or other documentation is used describing
the behavior of the intended system is used to focus the test efforts. Many of our subsystem and
our system test will be conducted in this mode.
3.3
INSPECTION
A method of verifying acceptability of hardware, software or technical documentation by
determining the compliance to requirements by visual examination of condition or content. The
criteria for examination are obtained from standards, schematics, or documentation consisting of
static test measurements, inventories or conformance features. The acceptable criterion is
pass/fail.
3.4
ANALYSIS
A method of verifying requirements for hardware, or software or configuration item requirements
by technical or mathematical representation (mathematical models, algorithms, equations) or
otherwise (matrix, charts, graphs, data reduction or representative data) with know scientific and
technical principles, procedures or practices. Results of the comparison are used to estimate the
capability of the system configuration to meet vision requirements. Verification criterion is
pass/fail.
3.5
DEMONSTRATION
3
A method of verifying subsystem/system or configuration item requirements by observing their
functional response to dynamic exercising. This qualitative evaluation is to be made by proper
operation of the item under specific condition. No pre-established results are specified. Actual
data need not have to recorded. An evaluation criterion is pass/fail.
3.4
TEST
A method of verifying performance requirements of subsystem/system or configuration items by
quantitative measurement of controlled functional or environmental stimuli under all appropriate
conditions. These dynamic measurements are made using standardized procedures, laboratory
equipment, or other services, that include collection, analysis and evaluation of resulting data to
determine the subsystem/system compliance. Pre-established results are required. The test
compliance criterion is pass/fail.
3.5
REQUIREMENTS VERIFICATION CROSS-REFERENCE MATRIX
The Requirements Verification Cross-Reference Matrix is shown in Figure 3-1. The matrix
relates the requirements of the Language Teacher (LT) system as found in reference document
Engineering Requirements Specification (ERS) and the verification mode and method of
verification to be used to the listed test event (test procedure) during which the verification will
be performed. By identifying the associated verification event, this matrix provides a test
schedule that ensures verification of all listed requirements for a LT system. The tests are
conducted at the level identified by the matrix.
4
Figure 3-1 Requirements Verification Cross-Reference Matrix
5
Figure 3-1 Requirements Verification Cross-Reference Matrix
6
4.0
4.1
TEST PROCUDURE DEVELOPMENT
SPECIFICAITON COMPLIANCE SCHEDULE
The specification compliance schedule has been development and is furnished in reference
document Language Teacher Project Plan. You will find that schedules for unit testing,
subsystem testing, and system testing are provided. That schedule provides the direction for test
procedure preparation. The Requirements Verification Cross-Reference Matrix shown in the last
section itemizes all LT requirements that must be verified. The Language Teacher project team
will assign the identified requirements to individuals to prepare the test procedures and conduct
the tests.
A log will be kept of all tests developed and conducted. The log will be used to document all
pertinent activity, whether test have passed or failed or passed with comments. The log will be
maintained as a current record and as a progress report item.
4.2
TEST PROCEDURE FORMAT
Test procedures (TP), which include data sheets, are prepared for all unit, subsystem, and system
tests. Each test procedure will contain the signatures of test teams and will be kept as part of the
system documentation. The test procedures will be submitted to the project manager for
approval as required. The test procedures will be prepared according to the standards and format
contained in this document and tailored to meet the needs of the LT project. A sample Table of
Contents for a test procedure is shown as Figure 4-1 Sample Test Procedure.
The Test Data Sheet (Figure 4-2) contained in each test procedure will be used to record and
document test results. The original of these sheets along with a copy of the applicable procedure
will be preserved as a record of the tests performed on the Language Teacher (LT) and will
provide the basis of the project test reports. A copy of all test documentation will be provided to
the customer as required.
7
SAMPLE TEST PROCEDURE
TABLE OF CONTENTS
1.0 IDENTIFICATION
1.1
1.2
1.3
1.4
1.5
1.6
Test Procedure Number
Test Requirement Identification (Reference)
Test Name
Location
Test Author
Test Date
2.0 TEST DESCRIPTION
2.1 Purpose
2.2 Test Method
2.3 Constraints
3.0 TEST PROCEDURE REQUIREMENTS
3.1
3.2
3.3
3.4
Test Duration
Materials
Test Order
System Version
4.0 TEST PROCEDURE
4.1 Test Steps
4.2 Descriptions of Expected Results
4.3 Test Data Sheet
Figure 4-1 Sample Test Procedure
8
TEST DATA SHEET
TEST PRCEDURE NUMBER:
REQUIREMENT (REFERENCE):
TEST TITLE:
DATE COMPLETED:
TEST TYPE:
TEST METHOD:
DESCRIPTION:
ANOMILIES, DISCREPANCIES, FAILURES:
STATUS:
PASS______
FAIL_____
COMMENTS:
LANGUAGE TEACHER SIGNATURE:__________________________________
Figure 4-2 Test Data Sheet
9
5.0
TEST CONDUCT
A detailed step-by-step set of instructions for each test is furnished as part of the test
procedure. These will be used for the conduct of the test. They will include all test
actions, sequence of actions and the proper response to the actions.
5.1
TEST LOG
A Test Log will be maintained by the Language Teacher project team, which will conduct
the tests. This log will provide a chronological narrative record of significant events
during test and will be used to record both equipment operation and test operations. Its
purpose is to provide a central record of test and system status and to provide a means of
re-examining or correlating data with respect to test events. In all the unit, subsystem,
and system tests a project person will validate the results by signature.
5.2
FAILURE REPORTING
Failures that occur during the performance of any test will be logged by the team and
recorded on the appropriate forms, namely the Test Data Sheets, Bug Reports, Test Log,
etc. A sample Bug Report is shown as Figure 5-1.
Failure that affect the serviceability of the product will be investigated by the Language
Teacher project team and project manager (B. Campbell) who will assist in making a
judgement as to whether the test should repeated at a later time after resolution.
5.3
DOCUMENTATION CONTROL
The original copy of all completed and signed test procedures, reports, and forms will be
maintained in the project documentation file.
5.4
TEST REPONSIBILITIES
The following are the responsibilities of individuals conducting tests on the Language
Teacher (LT) system:
a) Maintain the Test Log of all testing events, details of testing will be recorded
on the appropriate documentation and entered in the Test Log;
b) Ensure that the test is being conducted in the appropriate environment with the
current version of the software baseline;
c) Maintain test data sheets marked with test identification, date, and time;
d) Generate test reports to document results of individual tests;
e) Accumulate test data for generation of a final test report; and
f) Participate in meetings to review the status of testing.
5.5
TEST REPORTS
Upon completion of all tests through system testing the Language Teacher project team
will provide a test report that reflects the actual condition of the system at the termination
of the test. A preliminary test report format is included as Figure 5-2.
11
BUG REPORT
DATE:
PROJECT:
BUG NUMBER:
BUG NAME:
TITLE:
HARDWARE ENVIRONMENT (E.G., SUN WORKSTATION, PC, ETC.):
SOFTWARE ENVIRONMENT (E.G., UNIX, WINDOWS95, DOS, APPLICATION VERSION):
ACTIVITY (CHECK ONE):




DEVELOPMENT
DEBUG
INTEGRATION
TEST
PROBLEM (describe inputs, system behavior, conditions):
ADDITIONAL MATERIAL ATTACHED? YES___
NO___
DOCUMENTATION AFFECTED? YES___
NO___
SUGGESTED SOLUTION:
BUG REPORTED BY:___________________________________
Figure 5-1 Bug Report
12
PRELIMINARY TEST REPORT
COVER PAGE
Test Title:
Date:
Test Conductor’s Name:
DESCRIPTION
Brief description of the test, conditions, and results obtained. Reference
attached material for specifics. Identify any major problems uncovered that
affect the serviceability of the Language Teacher (LT) product.
ATTACHMENTS
A. TEST PROCEDURE (COPY)
B. TEST DATA SHEET
C. BUG REPORTS
Figure 5-2 Preliminary Test Report Format
13