MSF Testing
• Introduction
• Functional Testing
• Performance Testing
MSF Testing Strategy
• Requires a focus because there are many possible
test areas and different types of testing available
for each one of those areas.
• Because there are always resource constraints—
whether these are time, people, or money—
prioritization of the test areas and the type and
level of testing to be done are very important
decisions, and are the focus of preliminary test
planning.
Test Types
• Functional testing ensures that the system provides
the functionality described in the functional specification
document.
• Regression testing checks whether or not the
identical actions performed using an earlier build of a
product function the same on a new build of the
product.
• Security testing guarantees that only users with the
appropriate authority are able to use the applicable
features of the system.
• Performance testing ensures that the application
responds in the time limit set by the user.
• Stress testing verifies that the application responds
appropriately with many users and activities happening
simultaneously.
• Automated testing can be used for regression and
functional testing.
Test Areas
• User Interface Testing – References
specification documents to checks form and
consistency, including screen appearance as
well as checks on data validations for all of
the fields in all of the forms.
• Business Logic Testing
• Backend Testing
Test Areas (Continued)
• Platform testing certifies that the application runs
successfully on the operating system and browser
combinations agreed upon in the master test plan.
• Internet service provider (ISP) smoke testing
confirms that the application responds to requests made
over an ISP connection.
• End-to-end interface testing checks all of the inputs
and outputs as well as the system.
• Input and boundary testing guarantees that the
system allows only valid input.
• Windows/Internet GUI standards testing verifies
that the application has a standardized look and feel.
• Localization testing guarantees that the application
will work properly in different languages.
• Euro-compliant testing is used when an application
will receive monetary values from the Economic and
Monetary Union (EMU).
Test Areas (Continued)
• Conversion testing checks any data that must be
converted to ensure that the application works properly.
• Installation/upgrade testing checks the
setup/upgrade routine to ensure that the application
can be installed over an existing copy.
• Usability testing ensures that the application is easy
to work with, limits keystrokes, and is easy to
understand.
• Ad hoc testing is done to test the system with
unstructured scenarios to ensure that it responds
appropriately.
• Environment security testing guarantees that the
application installs and runs in the production
environment.
• Network testing determines what happens to the
application when different network latencies are applied.
Test Areas (Continued)
• Disaster recovery (backup/restore) testing is done to
ensure that adequate procedures are in place for restoring the
application and its data store in the event of a disaster.
• Application-based failover functionality testing ensures
that application-based failover works in documented failure
situations.
• User acceptance testing is typically performed by those
who are similar in skill set and background to the target
audience.
• Out of memory and memory leaks testing ensures that
the application runs in the amount of memory specified in the
technical documentation.
• Migration testing of applications from earlier versions of the
operating system ensures that the application runs after a
later version of the operating system is installed.
• Help testing is done to ensure that the details provided in
Help are relevant and provide a solution to the problem faced.
Testing Levels
• High – Very important to thoroughly test
this area
• Medium – Perform standard testing
• Low – Test if time allows
Functional Testing
• Should carefully test each build to
ensure that it provides the functionality
described in the application functional
specification.
• Involves ensuring that the application
behaves in the expected manner when
each of the user scenarios identified in
the application design occurs.
Functional Testing Methodology
Test Plan
• Sample detailed test plans in this
section address the following
functionality: Registration and
authentication
• Scenarios described are a selection
from the full detailed test plan
used in the development of the Ecommerce Web site
Scenario 1 – Registration Using Microsoft Passport
Description
A new user registers using Passport authentication.
Objective
Determine whether a new user can register on the site
using Passport credentials.
Priority
High
Steps:
1.1
Access the home page, and then click Register Now.
1.2
Click the Passport link on the registration page.
1.3
Use valid Passport credentials (e-mail address and
password) to complete all data fields on the page.
1.4
Click the Register button.
Scenario 2 – Logon Using Passport
Description
A user logs on to the site using Passport
authentication.
Objective
Determine whether a user with valid Passport
credentials can access the site. Depending on the
user's browser settings, check whether cookies are
deposited on the user's computer.
Priority
High
Steps:
2.1
Access the home page, and then click the Passport
link.
2.2
Enter a valid Passport credentials (e-mail address and
password).
2.3
Click the Sign In button.
Detailed Test Case Legend
Column
Heading
Definition
No.
Number of the specific test case in this document.
DTP Ref.
Detailed Test Plan Scenario on which this test case was built.
Priority & Test
Type
Relative priority of the test case (high, medium, or low) and type of
test performed—automated (A) or manual (M)
Condition to be
Tested
Specific functionality to be tested
Execution
Details
Steps to complete the test case
Expected Result
What should happen
Actual Result
What actually happened
Test OK?
Did the application function as expected? (Y=Yes; N=No).
Derived Test Cases
No.
DTP
Ref.
Priority
& Test
Type
Condition to
be Tested
Execution
Details
Expected
Result
1
DTP 1
High
(M)
User registers
as a new user
using Passport
authentication.
Visit the test
site.
Click the
Register Now
link for
Passport.
Complete all
fields.
Click Register.
The new
user
should be
able to
register
on the
site.
2
DTP 2
High
(M)
User logs on
using Passport
authentication.
Visit the test
site.
Click Passport
Sign in.
Enter a valid
user name and
password.
Click Sign In.
The user
should be
able to
log on
and have
access to
all
available
features
on the
site.
Actual
Result
Test
OK?
Performance Testing
• Response Time – measures the interval
between a user request and the response
from the application.
• Throughput - measures the ability of the
application to cope with the load placed on
it by multiple concurrent users.
Performance Testing Tools
• Microsoft Windows 2000 System
Monitor
• Netmon
• SQL Server Profiler
• System log files such as those
generated by IIS
• Microsoft Web Application Stress
(WAS) tool
WAS Tool
• Can be used to simulate the load placed
on your application by a number of
concurrent users.
• Can record a sequence of HTTP
requests to your site and play those
requests back for a specified number of
concurrent users.
• Gathers response time and throughput
statistics, which can then be used to
evaluate your application's
performance.
Performance Testing Methodology
•
•
•
•
•
Preparing for the analysis.
Creating a stress script.
Executing the test.
Analyzing the results.
Documenting and delivering the results.
Example Testing Infrastructure
Questions?