TEST AND D EPLOYMENT PHASE
9.0
63
TEST AND DEPLOYMENT PHASE
To enter the Test and Deployment Phase, a completed development test
baseline must be submitted. The development test baseline contains the
first delivered build of the system and configuration deemed ready for
independent testing. During the Test and Deployment Phase, the
developed system undergoes independent testing, the project team and
production support teams prepare for the system release, and the system
is deployed to the production environment.
Exhibit 32 summarizes the entry criteria, objectives, and exit criteria of
the Test and Deployment Phase.
Exhibit 32: Test and Deployment Phase Overview
Phase Entry Criteria
Phase Objectives
??D e v e l o p m e n t T e s t
Baseline
??Verify satisfaction of
system acceptance criteria
??Test Readiness Review
(TRR) approval
??Evaluate operational
security and data integrity
??Approval to begin phase
as determined by work
pattern
??Certify and accredit system
??Determine whether to
accept system as delivered
??E s t a b l i sh p r o d u c t b a s e l i n e
??Deploy system to
production environment
Phase Exit Criteria
??R e q u i r e d p h a s e d o c u m e n t
artifacts submitted to the
ICE Electronic Library
??System successfully
p a ss e d i n d e p e n d e n t
testing
??System certified and
accredited
??S y s t e m p r o d u c t b a s e l i n e d
??Release Readiness
Review Approval
Certification signatories
concur that the project
team has satisfied the
review exit criteria
??S y s t e m O w n e r a p p r o v a l
and Technical
A r c h i t e c t u re a n d
Integration concurrence
to conclude the Test and
Deployment Phase
ICE System Lifecycle Management Handbook
October 2004
TEST AND D EPLOYMENT PHASE
64
9.1
Independent Testing Process and Capabilities
In preparation for independent testing, the independent test team reviews
the documents and code artifacts developed by the project team and
creates an independent test plan. This test plan identifies the testing
scope, methodology, and schedule; specifies the testing environment and
configuration; and defines the test procedures. As appropriate, the
independent testing team may present for selection to the IT Project
Manager and System Owner testing options of varying scope, duration and
risk.
The independent test team may, as determined by the testing scope,
conduct one or more of the following types of testing:
??System Acceptance Testing - Verifies the functional
requirements as described in the Requirements Document to
determine if the product performs the business functions as
documented
??Security testing - Validates implementation of security
requirements and controls in the system and identifies potential
intrusion or sensitive data exposure vulnerabilities
??Interoperability Testing - Assesses the compatibility and
potential impact of multiple software components coexisting on the
infrastructure
??Performance testing—Detects any performance and capacity
limitations by generating system load that emulates the behavior of
users conducting business trans actions, determines when
performance begins to degrade, and identifies bottlenecks across
the system application and infrastructure
??Facilitated User Acceptance Testing - Provides a controlled
environment for actual users to test systems before the system
becomes operational.
At the conclusion of independent testing, the independent testing team
prepares a report on the testing results, which is delivered to the System
Owner, IT Project Manager, project team, and other stakeholders for
review. Detailed information regarding the scope and processes of these
tests can be found in the Architecture Test and Evaluation Plan (ATEP).
October 2004
ICE System Lifecycle Management Ha ndbook
TEST AND D EPLOYMENT PHASE
9.2
65
Test and Deployment Phase Activities
During the Test and Deployment Phase, the project team typically performs
the key activities identified in Exhibit 33. Each activity is subsequently
decomposed into the main tasks that the project team should execute to
ensure that it successfully and efficiently accomplishes the activity. The
activities and tasks are listed in the recommended order of completion;
however, the actual order will depend on the particular project.
Exhibit 33: System Test and Deployment Process
Development
Test and Deployment
Submit
New Build
Key Test
(9.2.1.)
(9.2.2.)
(9.2.3.)
(9.2.4.)
Activities:
Certify independent test lab
Support independent testing effort
Prepare the Certification and Accreditation Package
Prepare for production deployment
Disapprove
Release
Readiness
Review
Approve
Key Deployment Activities:
(9.2.5.)
Deploy system to production environment
Operations and
Maintenance
Activities
ICE System Lifecycle Management Handbook
October 2004
TEST AND D EPLOYMENT PHASE
66
9.2.1 Certify independent test lab
1. Maintain contact with TAP Test Lab Engineer to complete independent test
lab configuration and certification.
2. Approve and sign Certification of Independent Testing Environment.
9.2.2 Support independent testing effort
1. Coordinate with ICE Architecture Assurance to identify the independent
testing to be performed and to determine the preliminary testing schedule.
The independent testing conducted in the ICE Independent Testing Facility
includes:
2. Maintain contact with the independent test teams throughout the independent
testing effort, providing technical assistance as required.
3. Review daily and weekly status reports.
4. Determine corrective action for test problem reports (TPR) found during the
testing effort. (Note: A TPR is created during testing whenever actual results
do not match expected results and the discrepency could not be resolved or
satisfactorily explained.)
a. Review the TPR and prepare a fix or workaround or postpone the fix until
a future release. If the fix is postponed, the problem is recorded in an
independent test report and becomes a system change request (SCR).
b. If a fix or workaround is prepared, deliver the code to Version Manager for
retesting by the independent test team.
5. Review the results of the independent testing effort.
9.2.3 Prepare the Certification and Accreditation Package
1. Coordinate with the ISSO and Security Engineer to assemble the Certification
and Accreditation (C&A) Package.
2. Review the System Security Plan, Security Risk Assessment, and
Contingency Plan for accuracy and completeness and update if any changes
needed.
3. Include the results obtained from the independent Security Test and
Evaluation performed on the system.
4. Coordinate with the ISSO and Security Engineer to submit the C&A Package
to the DAA for accreditation after the ISSM signs the Certification Letter.
5. Initial implementation provides a six month Interim Authority to Operate
(IATO) after which an operational ST&E is conducted. The operational ST&E
is necessary to achieve full accreditation.
October 2004
ICE System Lifecycle Management Ha ndbook
TEST AND D EPLOYMENT PHASE
67
9.2.4 Prepare for production deployment
1. Validate the accuracy of system and user manuals and update if necessary.
2. Coordinate with the Help Desk, a component within Service Delivery, to
provide the information required for this release.
3. Confirm SLA agreements.
4. Finalize training curriculum and update the Training Plan.
5. Train end users
6. Coordinate with production teams to identify the optimal time for deploying
the system release into production in order to minimize disruption to the
existing production environments.
7. Construct and distribute the broadcast e-mail to notify system users of the
expected downtime and when the system will be available. The notice should
include the deployment schedule, a brief synopsis of the benefits of the new
system, the difference between the old and new system, responsibiliti es of
end user affected by the deployment during this phase, and the process to
obtain system support, including contact names and phone numbers.
9.2.5 Deploy system to production environment
1. Execute the Conversion Plan.
2. Deploy system
3. Arrange for Operational Security Test and Evaluation to be conducted on the
production system.
ICE System Lifecycle Management Handbook
October 2004
TEST AND D EPLOYMENT PHASE
68
9.3
Test and Deployment Phase Document Artifacts
The project team records the evidence of its work in a set of document
artifacts, the possible range of which is identified in Exhibit 34. The
approved project work pattern determines the required artifacts, which
compose the set of phase document artifacts and are identified in the
project work breakdown structure (WBS).
Exhibit 34: Test and Deployment Phase Document Artifacts
Test and Deployment Phase Document Artifacts
New
??Certification and Accreditation Package
Updated if needed
??V e r s i o n D e s c r i p t i o n D o c u m e n t
??U s e r G u i d e
??Deployment Plan
??Conversion Plan
??S y s t e m A d m i n i s t r a t i o n a n d M a i n t e n a n c e
Manual
??R e q u i r e m e n t s T r a c e a b i l i t y M a t r i x
??O p e r a t i o n s M an u a l
??T r a i n i n g P l a n
??P r o j e c t M a n a g e m e n t P l a n
Artifacts produced by independent testing
??Independent Test Plans
??Independent Test Reports
Note: Some of these artifacts are updated, expanded, and refined continually throughout the lifecycle
as the project evolves; provide the appropriate level of detail commensurate with the complexity of the
project and the information available during this phase.
9.4
SLM Milestone: Release Readiness Review
The Rele ase Readiness Review (RRR) ensures that the system is ready for
deployment. Specifically, this SLM milestone verifies that the project
team satisfies the system acceptance criteria, establishes the product
baseline to be deployed to the production environment, ensures that any
necessary initial end user training has been conducted, confirms that
production support teams are ready to deploy the system to production,
and establishes the deployment date and time. Exhibit 35 provides a
guide to the Release Readiness Review.
October 2004
ICE System Lifecycle Management Ha ndbook
TEST AND D EPLOYMENT PHASE
69
Exhibit 35: Release Readiness Review Guide
Guide to the Release Readiness Review
Pre -Review Activities
??Verify that all outstanding
Development Phase issues
have been resolved
??S u b m i t t h e T e s t a n d
Deployment Phase
document artifacts to the
ICE Electronic Library
??S e n d e -mail notification to
S L M R E V I E W @ D H S. G O V t o
schedule the Release
Readiness Review
??Prepare a system release
readiness overview and
project status update
briefing to present at the
review
??P o s t c o d e i n V e r s i o n
Manager (VM), verify
accuracy of assigned label,
a n d s e n d e -m a i l m e s s a g e t o
C M T O O L S@ D H S . G O V n o t i f y i n g
ICE Configuration
Management of posted code
in VM and identifying the
l a b e l a s s i g n e d to code
Review Activities
Post-Review Activities
??Review the readiness of the system
for release and project status: The
project team presents a summary of
the system release preparation results
and readiness and updates the project
status
??R e s o l v e a n y o u t s t a n d i n g
Release Readiness
Review issues
??R e v i e w i n d e p e n d e n t t e s t i n g r e s u l t s :
ICE Architecture Assu r a n c e reports
o n a n y o u t s t a n d i n g i s s u e s and raises
any questions
??R e v i e w T e s t a n d D e p l o y m e n t P h a s e
d o c u m e n t a r t i f a c t s : ICE
Architecture Assurance confirms that
the Test and Deployment Phase
artifacts are in the ICE Electronic
Library and reports the docume nt
assessment results
??Incorporate any required
changes or additional
content into the Test and
Deployment Phase
d o c u m e n t s a n d s u b mit
the updated documents
to the ICE Electronic
Library
??D e p l o y t o p r o d u c t i o n
environment
??R e c o r d R e l e a s e R e a d i n e s s R e v i e w
r e s u l t s : The chair/facilitator
completes the Release Readiness
Review Approval Certification form
with input from the appropriate
stakeholders and notes any issues
??S i g n t h e R e l e a s e R e a d i n e s s R e v i e w
Appr o v a l C e r t i f i c a t i o n : Th e
signatories concur that the project
team has satisfied the review exit
criteria, the system owner indicates
the level of system acceptance, and
the designated Technical Architecture
and Integration representative
c o n c u r s t h a t t h e project team can
deploy the system to the production
environment and begin system
operations
Typical Review Attendees
??T A P S L M R e v i e w F a c i l i t a t o r
??S e r v i c e D e l i v e r y R e p r e s e n t a t i v e
??S y s t e m O w n e rA
??IT Business Management Representative
??I T P r o j e c t M a n a g e rS
??TAP Quality Assurance Analyst
??C o n t r a c t T e a m P r o j e c t M a n a g e r
??TAP Technical Architect
??Technical Architecture and Integration
Representative S
??Information System Securi t y O f f i c e r
??T A P I n d e p e n d e n t T e s t a n d E v al u a t i o n
??OISSM Representative
??IT Engineering Services Representative
S
Signatory
A
Approval Authority
??TAP Configuration Management Analyst
S
S
ICE System Lifecycle Management Handbook
October 2004
TEST AND D EPLOYMENT PHASE
70
9.5
Test and Deployment Phase Stakeholder Roles
Exhibit 36 identifies the stakeholders and their roles in the completion of
the activities during the Test and Deployment Phase.
Exhibit 36: Test and Deployment Phase Stakeholder Roles
Matrix
P = Performer
SLM Roles
R
Prepare the Certification
and Accreditation
Package
R
P
Prepare for production
deployment
R
P
R
P
Deploy system to
production environment
R
P
R
P
Coordinate and
consolidate document
assessments
User Group
P
USCIS Records Services
R
ISSO
Support independent
testing effort
Help Desk
R
Training Services
P
Service Delivery
R
Systems Engineering
Contractor Project Team
IT Business Management
IT Engineering Services
Architecture Engineering
Archite cture Assurane
IT Project Manager
Certify independent test
lab
ISSM
System Owner
Activities
R = Resource and
Guidance
IT Business Council
C = Collaborative Partner
P
R
R
P
R
P
R
R
P
Release Readiness Review
October 2004
ICE System Lifecycle Management Handbook