Software Proof of Concept Best Practices
A software selection process is often described
by project managers and business analysts as
one of the most difficult endeavors an
organization can undertake. The costs alone of
bringing in vendors, pulling employees off their
current job duties, sitting through canned
presentations, comparing notes, and selecting a
product creates such an enormous burden that
“No Selection” is often the conclusion to the
process. Creating buy-in from end users only
adds to the time, costs, and efforts of the
process. The indirect costs associated with an
evaluation can often be up to 25% of the cost of
the product. Studies (DeGrace, 1990) indicate
that those organizations that do select a
product often set expectations for a failed
implementation by creating requirements that
are not fully understood before the project
begins, by educating users on requirements
only after having seen an initial version of the
software, by changing requirements during the
software
configuration
process,
or
incorporating methodologies that make the
implementation of best practices impossible. A
proof of concept is used as an implementation
milestone. It gets a buying committee past the
“canned sales” presentations to a true
understanding of the capabilities of both the
company and the product. A proof of concept
phase of the buying cycle enables an
organization to incorporate the concerns and
facilitate adoption of the end user community,
create change, incorporate best practices, and
set the tone for a successful implementation.
What is a Proof of Concept?
A proof of concept is a test(s) of a software
application made by building a prototype
architected from detailed process design
documents. It is the first milestone in the
implementation process that clarifies the
requirements process and prototypes the
configuration requirements.
The proof of
concept requires bringing the vendor's product
in-house to ensure that it will work in the
proposed environment and function the way it
was promoted by the vendor. The outputs from
a proof of concept includes design documents
that detail current business and best practice
processes, in depth tests and analysis of the
capabilities of the product and the company,
communication and training plans that facilitate
end user adoption, architecture documents that
specify data mapping and conversions as well as
any interfaces to third party external systems.
Defining a proof of concept as an
implementation milestone educates users on
requirements.
When to Perform a Proof of Concept
Due to the scope and complexity, the proof of
concept occurs at the end of the sales cycle and
continues as the first milestone of the
implementation. As a result, an organization is
better off working with one vendor in a proof of
concept for a myriad of reasons: the high cost
and effort involved in understanding the
product capabilities, the time spent on
gathering requirements, the re-allocation of
human resources, the money spent on travel,
the efforts to generate best practice design
documents, the depth of testing and the level of
involvement from the vendor. Conversely, an
organization would have to perform the first
implementation milestone twice should they
use a proof of concept as part of the vendor
selection process. The end result would be
much greater costs, efforts, and resource
utilization that create a greater probability of
“No Selection”. Due to the scale of the proof of
concept as the first milestone in an
implementation, best practices indicate that the
organization complete the vendor selection
process with one vendor. It may be in the
organizations best interest to inform the second
place vendor of their status in the event that
the primary vendor does not succeed in the
proof of concept. Defining the proof of concept
as a milestone in the implementation enables
an
understand
requirements
in
a
methodological manner that sets the tone for a
successful implementation.
Creating a Proof of Concept
As mentioned earlier, a proof of concept is a
test of a software application architected from
detailed process design documents.
The
timeframes for a proof of concept should be
well defined and in proportion to the time of
the implementation. For instance, an eight
month software implementation should take no
longer than two months for the proof of
concept. As it is the first milestone in the
implementation, the first step is design
documents that include project plans and
charters,
developing
product
training
workshops, conducting design sessions that go
beyond functional requirements that include
data conversions and interface mappings,
provisioning environments, and generating test
scripts. The outputs should include product
specific project plans with clear roles and
responsibilities, product training, functional and
technical requirement documents, a production
and development environment, fit-gap analysis,
and testing documents with measurable
pass/fail results. Project plans are often
produced by the vendor and managed by the
organization. Product training enables the
organization to assist with configuration and
train end users. A fit-gap analysis is often
performed after product training and may be
develop from a request for proposal (RFP) or
request for information (RFI). It is important to
have a development environment as part of the
system to test processes prior to migration to
the production environment. The major forms
of testing include user acceptance, system and
parallel testing. User acceptance testing occurs
when a particular feature is used with a result
of pass or fail. For instance, can a product save
account information: the result is either yes it
can or no it cannot. User acceptance testing is
also used to generate end user buy in especially
when performed by a well respected peer(s).
Stress tests measure the speed and
responsiveness of the system. Parallel tests
have two inputs, one in the legacy system and
one in the proposed system and the outputs of
the tests should be identical. Last, the proof of
concept should include tests about the vendor.
For instance, does the vendor have the
knowledge and expertise to manage change in
an organization? Do they understand how to
implement best practices? What level of
experience do they bring to the table? A key
consideration in the proof of concept is
whether a vendor has the experience to assist
in the development of the workflows. Defining
the proof of concept as an implementation
milestone is a means of solidify requirements
during configuration of the software.
Conducting a Proof of Concept
The proof of concept typically occurs at the
location of the customer. A project room
should be allocated with enough space for all
the end users to fit comfortable for the duration
of the implementation. The users should be a
cross sample of both “inter” and “intra”
departments: inter-departments are those
areas that may be affected either through the
elimination of informational silos or through
interfaces; intra-department are interested in
end users acceptance. The resources assigned
should be well respected within the community
as their acceptance to the system correlates to
end user adoption. Depending upon the
contractual agreements, the responsibilities of
the organization will be to serve as the overall
manager of the project. This includes managing
the scope of the proof of concept, time and
budget. The organization should also be
available to provide input to support the design
documents, facilitate communication with end
users to generate excitement and adoptions,
input into product configuration, and testing.
The organization will sign-off on the measures
used for success of the proof of concept, the fit
gap analysis, and the results of any tests. The
responsibilities of the software vendor include
building out the design documentation,
developing best practices, configuring the
system for the proof of concept, resolving
issues, and coordinating testing activities.
Defining the proof of concept as an
implementation milestone is a means of
adopting best practices.
Conclusion
A proof of concept as an implementation
milestone is the first step towards a successful
implementation. As such, it should not be used
as part of the evaluation process, but rather to
develop a deeper understanding of both the
product and the company, and ultimately
reduce business risk. It will educate users on
requirements, solidify requirement, enable best
practices and set the tone for a successful
implementation.