Keys to Successful Beta Testing
Presented by
Ron Dinwiddie, Product Director, Blackboard Commerce Suite™
Jessica Finnefrock, Product Director, Blackboard Academic Suite™
April 12, 2005
A word from our lawyers…
•
Any statements in this presentation about future expectations, plans
and prospects for Blackboard and other statements containing the
words "believes," "anticipates," "plans," "expects," "will," and similar
expressions, constitute forward-looking statements within the meaning
of The Private Securities Litigation Reform Act of 1995. Actual results
may differ materially from those indicated by such forward-looking
statements as a result of various important factors, including the factors
discussed in the "Risk Factors" section of our most recent 10-K filed
with the SEC. In addition, the forward-looking statements included in
this press release represent the Company's views as of April 11, 2005.
The Company anticipates that subsequent events and developments
will cause the Company's views to change. However, while the
Company may elect to update these forward-looking statements at
some point in the future, the Company specifically disclaims any
obligation to do so. These forward-looking statements should not be
relied upon as representing the Company's views as of any date
subsequent to April 11, 2005. Blackboard, in its sole discretion, may
delay or cancel the release of any product or functionality described in
this presentation.
Introduction
• Session Objectives:
–
–
–
–
Define Blackboard® Beta programs
Describe Blackboard Beta client selection process
Explain the process of a Blackboard Beta program
Highlight the benefits of a Blackboard Beta
program
– Hear from *Real* Beta Clients
• Innovation
– Beta v. Early Availability
• Results/Outcomes
– Product Quality
What Does Beta Mean at
Blackboard?
• Definition
– The purpose of the Beta Program is to verify that
Blackboard’s software and/or hardware performs as
specified by real users at customer sites using their data
and equipment
• Beta today = GA of old
– Product does not go to Beta until it meets strict QA exit
criteria
– Beta clients are selected to ensure coverage of diverse
environments and resolve those issues before GA
– GA timeline is not fixed—it is dependant on quality as
verified with our Beta partners
What is Early Availability?
• When Beta starts, Blackboard announces the Early
Availability release of that product
• Beta clients receive code to install on their test
servers and then move into production
• Other clients interested in test-sever-only copies of
the application can sign up via a web form to get a
preview copy of the release before GA
– This preview copy will be released midway through the Beta
program
How are Beta Clients Selected?
•
Client profile
–
–
–
–
•
Risk-tolerant
Interested in exploring the new features and functions in the release
Willing to update their production system with the Beta software
Willing to provide feedback (not necessarily positive)
Recruitment
– Initial communication with potential clients
• Sales
• Support
• Services
– Provide overview of product
– Select candidates based on beta requirements
•
Agreements
– Beta Contract
• Defines commitment of client
• Defines tasks needed to perform test
• Defines acceptance of risk associated with beta
How Do Beta Clients Prepare?
• Early Access to Product and Documentation
– Access to demo server hosted by Bb
– Overview sheets per feature (1-3 pages)
– Draft documentation
• Client installation
– Hardware
• Site Survey
• Outside contractors (electrical, etc…)
• Order equipment
– Software
• Blackboard Commerce Suite
– Support installs software
• Blackboard Academic Suite
– Client downloads software from central server
How do Beta Clients
Communicate with Blackboard?
• Issue Submission
– Standard Ticket Process
– Web forum
– Blackboard Beta Program
Manager
• Feedback
– Blackbaord Beta Program
Manager
– Web forum
– Updated Release Notes with new
builds
• Status
– Blackboard Beta Program
Manager
– Weekly team meeting with all beta
clients
What Tests Do Beta Clients
Perform?
• Project Plan
– Defined by Blackboard Beta Program Manager
– Specific tasks to perform
• Test Questions
– Use Case Results
• Independent Testing
– Additional testing defined by client
• Specialized Testing
– Unique Hardware or Software configurations
When is Beta Over?
• All Beta Test Criteria have been
accomplished
• No Critical Bugs
– All critical issues resolved by GA announcement
• Changes made based on beta feedback
– Changes are prioritized and implemented
dependant on time frame
What Are the Benefits of
Participating?
• Close working relationship with Blackboard Product
Development
• A Blackboard coffee mug
• Fast-tracked escalation and resolution of issues
• A Blackboard coffee mug
• Assurance that the application will perform with high
quality in their environment
• A Blackboard coffee mug
• First-access to new features and functionality
• A Blackboard coffee mug
• Opportunity to influence the future direction of the
product
• Did we mention…a Bb coffee mug?
Beta Client Perspective
• Donna Wicks, Course Delivery Technologies
Specialist, Kettering University
• TBD (Commerce Suite)
Closing Slide
• Innovating Together in ‘05:
– Understanding a Beta
– Committing to the Beta Plan
– Feedback that improves our GA release
• Follow up Contact(s):
– Darren Sapper – Associate Director of Client Programs,
Product Development (dsapper@blackboard.com)
– Ron Dinwiddie – Product Director, Blackboard Commerce Suite
(rdinwiddie@blackboard.com)
– Jessica Finnefrock – Product Director, Blackboard Academic Suite
(jfinnefrock@blackboard.com)
• IF YOU ONLY REMEMBER 1 THING:
– A successful Beta Program insures a successful release of a
quality product