Evaluation Plan
Geographical Auto-Delivery System
Evaluation Plan
Version:
Date:
Author:
Status:
1.0
13 December 2010
Ben Cawrse
Final
The purpose of the evaluation plan is to provide a method for determining the success or failure
of efforts during a particular phase of the project. Additionally, the evaluation plan is used by the
Project Manager to determine whether a particularly phase of development is complete and if
progress should be started on the next phase. Each phase is evaluated by four criteria: time,
budget, scope and evaluation markers specific to that phase. The time allotted to each phase is
specified in the Work Break-down Structure (WBS), Appendix H-K. The scope of the project
includes all outlined solution features specified by the Major Functional Components Diagram
(MFCD), found in section 2 of the Development Plan. Upon tentative completion of a phase, the
project manager will review the time, budget, scope and evaluation markers for that phase to
determine if the phase has been successfully completed utilizing Table E-1. Once all MFCD and
evaluation markers for a particular phase are completed, the Project Manager will indicate the
actual time and budget required and advance the project to the next phase.
Time (Days)
Budget
Scope
MFCD
Phase Allowed Actual Allowed Actual
0
1
2
3
Table E-1
Evaluation
Markers
Used by the project manager to determine whether a phase of development is complete or not
G1
Evaluation Plan
Geographical Auto-Delivery System
Phase 0 Evaluation Markers
Phase 0 is the base line project planning phase. During phase 0, all planning documents are
iteratively generated and approved by review board. All aspects of the project’s plan are
addressed and rigorously reviewed for consistency. The scope of the problem and the proposed
solution is defined by the Feasibility presentation. The Milestone and Final presentations outline
specifics of the solution plans, including marketing, funding, risk and evaluation plans. After
Final presentation board approval, submission for an NSF grant is made to provide initial phase 1
project funding. To evaluate phase 0, five evaluation markers have been established.
Web site
The web site will form the first impression for all future clients and investors and care should be
taken to ensure that it is of the highest quality. It should be reviewed for design, consistency, link
flow and accessibility. A document repository on the web site should be established and all phase
0 deliverables should be easily accessible from the site. The web site should also contain a team
biography. After work on the web site has completed, the Project Manager should review the
web site to ensure if meets all criteria specified in Table E-2.
Criteria
Participants’ description
Project description
Project presentation
Project WBS by phase
Project minor deliverables
Project plan
Project SBIR/budget links
Project risks
Project bibliography
ODU and CS dept links
Page link validity
Single displayed page
Link organization
Page creativity
Project Manager Approved
Table E-2
Represents a website check-off list for the Project Manager
Feasibility acceptance
The Feasibility presentation approval is dependent on approval by the review board. The
Feasibility presentation defines the scope of the problem and the proposed solution. The
Feasibility presentation outlines the characteristics of the problem and supports these with
specific and sufficient data. A general outline of the proposed solution and process flow
improvements are generated to demonstrate how the proposed solution solves the problem. After
G2
Evaluation Plan
Geographical Auto-Delivery System
completion of the Feasibility presentation the board will approve or disapprove the project based
on the criteria specified in Table E-3.
Criteria
Problem defined
Problem evaluated
Customer Identified
Market evaluated
Reference support
Competition identified
Computer components (hardware/software) defined
Benefits of solution
Problems with solution
Presentation introduction
Presentation organization
Presentation conclusion
Table E-3
Board Approval
Defines the criteria for feasibility approval
Milestone acceptance
The Milestone presentation approval is dependent on approval by the review board. The
Milestone presentation expands on the Feasibility presentation, providing more details on the
problem and the proposed solution. Specific comments and revision suggestions from the review
board from the Feasibility presentation are used to form the basis of changes in the Milestone
presentation. Though not a specific objective of the Milestone presentation, draft versions of the
marketing, funding, evaluation, management and risk plans are generated and reviewed by the
board. Additionally, the Milestone presentation includes a major functional component break
down (MFCD) that outlines the specific hardware and software components required for the
solution. After completion of the Feasibility presentation the board will approve or disapprove
the project based on the criteria specified in Table E-4.
Criteria
Problem & solution defined
Market defined and analyzed
Objectives evaluated
Description of what solution does
Description of what solution doesn’t do
Major technical issues
Major management method issues
Major risks
Major resource issues
Presentation organization
Presentation conclusion
Table E-4
Board Approval
Defines the criteria for milestone approval
G3
Evaluation Plan
Geographical Auto-Delivery System
Final board approval
The Final presentation represents the culmination of all the efforts of the Feasibility and
Milestone presentations and is the final document produced during phase 0. The Final
presentation is the final expansion of Feasibility and Milestone presentations. Specific comments
and revision suggestions from the review board made during the Milestone presentation are
integrated into the Final presentation. The Final presentation includes the final versions of the
marketing, funding, evaluation, management and risk plans. Additionally, the Final presentation
includes a Work Break-down Structure (WBS), outlining the tasks required to create the
hardware and software components of the solution.
Criteria
Identification of problem, identification of customer
Description and supporting data
Product Goal and Objectives
Product Design and Approach / Milestones
Innovative aspect(s)/ Research Potential
Prototype Design and Approach
Prototype Milestones
Staff & Resource Requirements (Phase 1,2,3)
Marketing Customer Evaluation/Initial Customer
Marketing Price Point
Marketing Return on Investment
Presentation organization
Funding potential
Table E-5
Board Approval
Defines the criteria for the final, Phase 0 approval
Phase 1 Evaluation Markers
Phase 1 is the prototype stage. During phase 1, a software prototype and an initial Alpha Testing
Plan is created. The software prototype should contain all features specified in the Major
Functional Components Diagram (MFCD), found in section 2 of the Development Plan.
Additionally, an initial Alpha Testing Plan is generated to test the functionality of each prototype
MFCD component.
Prototype design approved
A software prototype is developed for all MFCD components specified in section 2 of the
Development Plan. The specific tasks required to accomplish this are outlined in the Work
Break-down Structure (WBS), Appendix H-K. Utilizing Table E-6, the Project Manager should
ensure that as each component from the MFCD is completed, it is evaluated against the Alpha
Testing plan to ensure any deviations from testing acceptance are handled as early as possible.
G4
Evaluation Plan
Geographical Auto-Delivery System
Phase 1 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Computer System
Smartphone SDK
Simulated GPS
Simulated Internet
Validated
Table E-6
Used by the Project Manager to ensure each of the MFCD components used in Phase 1 are incorporated.
Alpha Testing Plan approved
The Alpha Testing plan is the document used to test and approve the functionality of each major
MFCD component in the prototype phase. The document should be generated simultaneously
with the prototype to ensure that as components of the MFCD are completed they are tested
immediately after development. Utilizing Table E-7, the Project Manager should ensure that as
the testing plan for a particularly MFCD component is generated that the appropriate box is
checked.
Phase 1 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Computer System
Smartphone SDK
Simulated GPS
Simulated Internet
Validated
Table E-7
Identical to Table E-6
Used by the Project Manager to verify testing of all MFCD components during Phase 1
Prototype function validated
Once all MFCD components have been completed and approved by the Alpha Testing Plan, the
project may proceed to phase 2. Utilizing Table E-8, the Project Manager should indicate in the
table once all MFCD component has been completed.
G5
Evaluation Plan
Geographical Auto-Delivery System
Phase 1 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Computer System
Smartphone SDK
Simulated GPS
Simulated Internet
Validated
Table E-8
Identical to Table E-6
Used by the Project Manager to verify all MFCD components for Phase1 are completed
Funding approved
After the prototype is completed and validated against the Alpha Testing Plan, an application for
NSF funding can be submitted. Once approval from NSF is granted, the project begins phase 2.
Phase 2 Evaluation Markers
Phase 2 is the implementation phase. Funding is provided by an approved NSF grant, proposed
and approved during phase 1. The approved and alpha tested prototype from phase 1 is revised
and expanded to meet the full functionality specified in the Major Functional Component
Diagram (MFCD). Additionally, a beta testing plan is developed to provide testing criteria to
prove the functionality of all software components. A case study is created consisting of one or
more clients. These case study clients are provided with a copy of the beta version of the
software for testing and feedback. Their feedback is used to help find software bugs and specific
minor feature improvements. Testing should be conducted simultaneously with development to
ensure that specific software components operate correctly.
Code completion
The work break down structure contains the list and order of all coding objectives. As coding
objectives are achieved, the Project Manager should be notified and the Beta Testing Plan
implemented to ensure that the software component operates correctly. Only once the component
has been verified as operating correctly as per the Beta Testing plan should progress be started on
the next component. Utilizing Table E-9, the Project Manager should indicate in the table once
an MFCD component has been completed.
G6
Evaluation Plan
Geographical Auto-Delivery System
Phase 2 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Third Party Server
Computer System
Smartphone
GPS
Internet
Validated
Table E-9
Used by the Project Manager to ensure each of the MFCD components used in Phase 2 are incorporated.
Beta Testing Plan approved
A Beta Testing Plan should be created to specify testing tasks to be performed to verify the
operation of each coding objective specified in the work break down structure. This testing plan
should enumerate all testing tasks in a check off table format. As coding objectives are achieved
the testing plan’s check off list can be referenced and the specific testing tasks performed to
ensure that the software meets objectives. Testing plan approval is contingent on Project
Manager review. Utilizing Table E-10, the Project Manager should indicate in the table once the
Beta Testing Plan for that component has been completed.
Phase 2 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Third Party Server
Computer System
Smartphone
GPS
Internet
Validated
Table E-10
Identical to Table E-9
Used by the Project Manager to verify testing of all MFCD components during Phase 2
Code function validated
Code function should be validated utilizing the Beta Testing Plan as soon as possible after
development on a specific design goal is met. Incremental testing ensures that any deviations
from the testing plan are detected and corrected as early as possible. Utilizing Table E-11, the
Project Manager should indicate in the table once an MFCD component has been validated
against the Beta Testing Plan.
G7
Evaluation Plan
Geographical Auto-Delivery System
Phase 2 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Third Party Server
Computer System
Smartphone
GPS
Internet
Validated
Table E-11
Identical to Table E-9
Used by the Project Manager to verify all MFCD components for Phase1 are completed
Case study client acceptance
A case study of one or more clients should be selected. The clients selected should be
representative of either low or high LEP population states. Weekly meetings should be conducted
with the clients to ensure that all bugs are properly reported and to document any client feedback
on minor feature improvements. Utilizing Table E-12, the Project Manager should indicate case
study client acceptance of specific MFCD components as they are validated.
Phase 2 MFCD Components
Server
Input Handler
Database
Notification Subsystem
Third Party Server
Computer System
Smartphone
GPS
Internet
Validated
Table E-12
Identical to Table E-9
Used by the Project Manager to verify all MFCD components for Phase 2 are validated by the customer
during a client case study.
Phase 3 Evaluation Markers
Phase 3 is the culmination of all efforts of previous phases. The completed and validated GAS
software is fully tested and commercially viable. The focus of phase 3 is increasing market and
revenues though marketing to new clients and retention of current clients. A marketing goal of an
initial 60,000 end-users, and 1000 chain locations, with 1000 new end-users and 50 small
businesses per month with a break-even point within six months. To ensure retention of current
clients, the customer service team will provide technical support to customers and accept client
feedback on bugs and feature requests. The customer service team will forward all bug and
G8
Evaluation Plan
Geographical Auto-Delivery System
feature request feedback to the development team. A Version Control Plan is used by the
development team to ensure that new versions of GAS are uniformly distributed to clients as
patches are made available fixing software bugs and implementing new features. Additionally,
reaching recruitment and staff retention goals is necessary to facilitate all operations.
Client Feedback and Version Control Plan acceptance
To ensure retention of current clients, the customer service team will provide technical support to
customers and accept client feedback on bugs and feature requests. The customer service team
will forward all bug and feature request feedback to the development team. A Version Control
Plan is used by the development team to ensure that new versions of GAS are uniformly
distributed to clients as patches are made available fixing software bugs and implementing new
features. Utilizing Table E-13, the Project Manager should indicate that the client feedback
process and Version Control Plan have been completed.
Client Feedback Process
Customer Service Team
Team Lead
Service Representatives
Customer support script
Customer bug report feedback form
Customer feature request feedback form
Initial training conducted
Version Control Plan
Customer bug report database
Customer feature request database
Source control version database
Patch database
Table E-13
Completed
The Project Manager uses this table to indicate the client feedback process and version control plan have
been completed
Financial “break even” reached
The focus of phase 3 is increasing market and revenues though marketing to new clients and
retention of current clients A marketing goal of an initial 60,000 end-users, and 1000 chain
locations, with 1000 new end-users and 50 small businesses per month with a break-even point
within six months.
Full staffing achieved
For the human resources team, reaching recruitment and staff retention goals is necessary to
facilitate all operations. Utilizing Table E-14, the Project Manager should indicate as specific
team members are recruited.
G9
Evaluation Plan
Geographical Auto-Delivery System
Team Member
Project Manager
Marketing Director
Finance Director
Technical Director
Human Resources Director
Documentation Specialist
Hardware Manager
Software Manager
Web Programmer
Database Administrator
Software Engineer
Server Integration Specialist
Procurement Manager
Customer Support Specialist
Position Filled
Table E-14
Utilized by the Project Manager to verify that full staffing has been achieved
G1
0