Test Automation
Using
SilkTest
1
Test Automation Rules
• Product feature understanding, test
planning, test case documentation, test
bed setup, defect tracking, progression
tests are all manual
• Test automation success depends on test
cases
• Every manual step has commands in test
script
• If it is not worth, do NOT automate
2
Tools - Comparison
•
•
•
•
•
•
•
•
•
WinRunner, QTP are from Mercury Interactive – USD 1bn company and market
leader; Robot from Rational IBM - USD 100bn company’ Silk from Segue – USD
50mn company
Script here is object oriented and close to QTP script. Winrunner TSL is like C and
Robot SQA Basic is like Visual Basic
GUI to create verification points are better in WinRunner, QTP and Robot
Silk is more towards moving scripts on MS Windows and Unix; whereas Robot or
WinRunner scripts cannot be moved to other OS as those tools do not support
Rational does not provide GUI specification in a separate file; WinRunner, QTP and
Silk provide that
You need to buy Java Plugin for WinRunner; Silk and Robot come with Java plugin
Record, replay, verification method, data driven tests, database connectivity, bitmap
verification are all same at concept level and differ only at script syntax level
Wizards are less in SilkTest, compared to other tools
Cost-wise, QTP and WinRunner are most expensive; Silk stands in between
WinRunner and Rational Robot
3
4
Architecture
•
•
•
•
•
•
•
•
Project .vtp (for maintenance)
Testplan .pln (organizing the tests)
Test Frame* .inc (AUT Window details)
Scripts .t (test scripts)
Data driven tests .g.t (scripts with many data)
Include file .inc
Suite .s (for batch runs)
Result .res
5
Test Automation Principle
•
•
•
•
•
•
•
•
•
Make the tool learn about AUT GUI
Record steps based on mouse and key events
Organize steps into test case
Organize test cases into test plans
Run test cases
Batch different sets and run
Analyze results
Do NOT try to code application logic into testing code
Whenever possible, generate code thru the tool and not
hand-code
• Read STUSER Help file (user guide for SilkTest)
6
Test Process
• Manage requirements using SilkCentral Test
Manager
• Manage test plans and cases using Silk test
• Associate test plans/cases to requirements
using test manager
• Declare GUI declarations using SilkTest
• Develop scripts for atomic activities first so that
they can be reused later
• Develop scripts using SilkTest and run them
• Track defects using Defect Tracker
7
Test Frames
•
•
•
•
•
•
•
•
•
Before test case recording, do this step
This creates an .inc file that has GUI declarations
Identifies GUI controls as classes
Label or caption is the best way to identify objects; if not available,
object index is used (available tags are caption, prior text, index,
window id, coordinates)
Class and tag are the key elements in declarations
Record window declarations is the way to store test frames
When objects are not identified properly, use different tags to
uniquely identify the objects. This will vary from application to
application
If object X,Y coordinates are to be used, declare constants and then
use those constants for further declarations
To declare new objects within a window, it is better to use AnyWin or
CustomWin classes. Use ~ActiveApp to specify the coordinates
8
9
Test Frames
• Along with window declarations, we can declare methods that are
specific to that window and objects
• The method declarations within a window object is similar to object
oriented programming
• Uses can use these method declarations as wrapper classes for
executing a series of steps that are recorded
• To the extent possible, do not have any hard coded values in the
GUI Window methods
• While recording new test cases, ensure that you update window
declarations as well; but check the frame.inc file, not to have
duplicate declarations for same window
• Default is frame.inc; but we can have any number of include files
• Unless the window declarations are in scope, the test script will not
recognize the controls
10
Recording
• Recording will automatically generate statements on every user
action
• To capture the mouse click coordinates and keyboard events, we
can use low level enabler for every control in recording options
• We can record the base state as well as test steps
• Soon after recording, ensure the test frame declarations, for any
changes or duplicates
• Recording can be done for window declarations or test case or
actions
• Every recorded script must start from a known base state and must
return back to the same known base state
• BaseState and DefaultBaseState can also be specified using
recording (appState is a function that ensures the pre-condition state
for a test case)
11
12
13
Test Cases
•
•
•
•
•
•
•
•
•
•
•
Each test case must be independent
Initialize (pre-requisite setup), run and verify, cleanup
One test case must have One purpose
Establish application state for every test case. Record the base state
Test Case contains commands for every test step. Record the steps
and edit if needed
Establish verification points for every test case
Record the clean up steps as well
Sequencing of test cases is very important when it comes to
automation
These are also functions, with the type as “testcase”
We can use parameters for testcase as well
A testcase is also a function by itself, but it can have base states
14
4Test Elements
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Variables – INTEGER, STRING, BOOLEAN, ANYTYPE, REAL, LIST
Variables do have scope – public or private
Const is for constants
Regular arithmetic and string operations hold good
2 types of for statements
Foreach and While loops are also available
IF, select-case and SWITCH-case are available for branching
Do-except is for exception handling
Exception functions will be useful for tracing exception number and details
Test steps will have window.object.method or window.object.property style
of commands
User defined functions with in, out, inout parameters
Functions can return values as well
Use built-in functions for list and array manipulations
Print and Printf commands are used for sending output to results file
15
Built-in Functions, Classes
• For debugging purpose, we can use print function. It helps in
documentation as well as to track the test run status, at any given
point of time
• Refer to General Core Classes section in SilkTest Help (PARTNER
file) – AnyWin, MainWin, CheckBox, ComboBOx, DialogBox,
ListBox, Menu, PushButton, RadioButton, StaticText, TextField,
StatusBar are the most used classes
• Refer to Functions by Programming Task section in SilkTest Help
(PARTNER file) – Datatype commands, Database commands, List
and string Manipulations, Window Information commands are the
most used ones
• System related functions can be used to carry out any DOS
commands to set the test environment (Sys_Execute, Sys_Kill)
• String functions such as ltrim, rtrim, etc are used extensively to
manipulate test data; arithmetic functions are not used often
• User defined functions must be called from main function
16
17
18
AnyWin Properties & Methods
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
CaptureBitmap
ClearTrap
Click
DoubleClick
Exists
GenerateDecl
GetAppId
GetArrayProperty
GetBitmapCRC
GetCaption
GetChildren
GetClass
GetContents
GetEverything
GetHandle
GetHelpText
GetIDGetIndex
GetInputLanguage
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
GetManyProperties
GetName
GetNativeClass
GetParent
GetProperty
GetPropertyList
GetRect
GetTag
InvokeMethods
IsActive
IsArrayProperty
IsDefined
IsEnabled
IsOfClass
IsVisible
InvokeJava
MenuSelect
MoveMouse
MultiClic
19
AnyWin Properties & Methods
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
MultiClick
PopupSelect
PressKeys
PressMouse
ReleaseKeys
ReleaseMouse
ScrollIntoView
SetArrayProperty
SetInputLanguage
SetProperty
SetTrap
TypeKeys
VerifyActive
VerifyBitmap
VerifyEnabled
VerifyEverything
VerifyText
VerifyProperties
WaitBitmap
•
•
•
•
•
•
•
•
•
•
•
•
•
bActive
AppId
sCaption
lwChildren
Class bEnabled
bExists
sID
iIndex
sName
wParent
Rect
hWnd
WndTag
20
Method examples
•
•
•
•
•
•
•
•
•
•
•
Find.OK.Click
Find.MatchCase.Check
Find.FindText.SetText(“abc”)
Notepad.CloseWindows
Find.Up.VerifyValue(TRUE)
Fonts.FontName.verifyContents(…)
Notepad.Editor.SelectAll() // this is a custom method to select all
contents. This needs to be declared in the windows declarations
User defined functions can have in, out, inout and optional
parameters
ClipBoard functions are used to manipulate clipboard text
Cursor class methods can be used to manipulate coordinates
Property prefixes – b for boolean, I for int, ls for list, s for string
21
Get, Set, Verify for Controls
•
•
•
•
•
•
•
•
•
For every object, Get methods are used to obtain attribute values into a
variable. After getting to a variable, we need to manipulate
For every object, Set methods are used to set attribute values on the
screen. This will work only when the object cooperates with SilkTest
operations
For every object, Verify methods are used to check the attribute values –
implicit Get and an if-then-else condition. It must have an expected value as
argument
Whatever properties that can be used for get statement, can be used for set
and verify statements
Verify methods evaluate to pass or fail status
Any other custom methods can be used, if they are already declared in
window declarations
Most of the properties and methods are inherited from AnyWin to all other
GUI classes
Dlls can be loaded using dll declaration and the functions can be invoked
from 4Test
Capture and Compare bitmap commands are used for image checks
22
Utility Elements
•
•
•
•
•
•
•
•
•
•
•
•
•
Array functions are used to manipulate arrays (ArraySize, ArrayFind)
List functions are used to manipulate lists (ListSort, ListAppend)
DateStr, TimeStr, Sleep, StartTimer, StopTimer are used for timer functions
FileOpen, FileClose, FileWriteLine, FileReadLine are used for file
operations
Wild cards - ? And *; MatchStr command is used for expressions
Multitags enable an object to be identified with many labels
Registry Operations are available – not used widely (Reg_CreateKey)
LogError, LogWarning, Print to write to results file
Timer commands to create, start, stop, destroy and get the interval
Date format and functions to manipulate date fields
DLLs can be loaded and then functions can be executed; but this needs to
be kept at minimum
To get script, environment and browser related information, we can use the
respective built-in functions (GetArgs, GetBrowserType)
Date related functions are available; but it is advised not to use inside script
to manipulate dates
23
Database Test
• Database tests follow the verification of GUI
verifications
• Database commands start with DB_
• Connect to DB (DSN needed)
• Execute Query (give SELECT)
• Fetch the records and compare with baseline
• Finish DB operation (release resources)
• Disconnect after every database operation
• Do not keep the database connections for a long
time in 4Test scripts
24
Verification
• Object properties are the key elements
• Control state, value, label, text, count, size,
location are the most common properties
• Feed the expected result to the verify command
• Bitmaps can be verified using CaptureBitmap
and VerifyBitmap
• Use Bitmap tool to capture baseline image
• Is* methods evaluate to a boolean value
(IsActive, IsEnabled)
• LinkTester checks the links in web applications
25
Data Driven Tests
• Same script, variety of data, less coding
• Use Data Driven work flow for a standard
recorded test case
• Create a test database and configure DSN
• Replace hard coded recorded values to data
driven variables (using find/replace option)
• When you run the test, it runs for all rows of data
in the data set
• A new test case with DD_ prefix will be created
26
27
Run Tests
• One can run a test case or test plan or test suite
• Suite contains many test cases in it
• Using distributed run mode, one can run scripts in
another machine that has Silk Agent
• We can run in run mode or debug mode
• Debug mode enables us to step, step into, step out,
declare breakpoints etc
• All other hosts run the Agent
• Spawn, Parallel, Rendezvous are used in distributed
testing – but unless required, distributed testing is not
widely used
• One must connect and disconnect to target machines
28
Traceability
• Requirements to test traceability is thru Test Manager
• Test case to tester traceability is again part of test
manager
• We need to define naming conventions by ourselves for
test plans, test cases, requirements
• Suggested guideline is to name the test case with the
test case ID itself and have the test case description as
comments
• Use base states for capturing test pre-requisites
• Version control is integrated with PVCS for check-in,
check-out and labeling features
29
Recovery Scenarios
• Recovery system enables the test to come
to the Base State
• Record application state is used to specify
the recovery scenarios
• Exception handler itself is script recovery
feature
• StopRunning() method bypasses all
recovery settings
30
Test Extensions
• These are used to help SilkTest to identify new
objects and their properties
• Silktest can automatically enable extensions for
open applications
• Java, .Net, AOL Browser, Netscape Navigator,
Internet Explorer require extensions
• For browsers, enabling DOM or Virtual Object
mode would be configured to identify objects
31
32