Contents
Introduction.............................................................................................3
Why SoapUI?..................................................................................................... 3
Functional testing........................................................................................ 3
What is an API?................................................................................................. 4
What is a Web Service?..................................................................................... 4
SOAP.......................................................................................................... 5
REST.......................................................................................................... 5
Testing APIs....................................................................................................... 5
No graphical user interface......................................................................... 5
Request and response................................................................................ 6
Exploring............................................................................................................ 6
Getting started........................................................................................7
Installing............................................................................................................. 7
Your first project................................................................................................. 8
Add a test suite................................................................................................ 13
Add a test......................................................................................................... 14
Add an assertion.............................................................................................. 17
Run the test..................................................................................................... 19
Run the test suite............................................................................................. 19
Further exploration........................................................................................... 20
Create tests..........................................................................................20
Verify a range................................................................................................... 21
Verify response time........................................................................................ 25
Verify schema compliance............................................................................... 25
Go pro: Point and click testing......................................................................... 26
Verify a range............................................................................................ 26
Data driven testing...............................................................................28
Go pro: Data source out of the box.................................................................. 29
Create a data source................................................................................ 29
Create a test with parameters................................................................... 32
Loop over the data source........................................................................ 33
Execute the test case................................................................................ 35
Further exploration.................................................................................... 36
Debugging............................................................................................36
Manual............................................................................................................. 36
Go pro: Test debugging.................................................................................... 36
Multiple environments..........................................................................38
SoapUI Pro
by SMARTBEAR
2
Change the target environment....................................................................... 38
Go pro: Multi-Environment Support................................................................. 39
Create a test report.......................................................................................... 44
Organize your work..............................................................................46
Workspaces..................................................................................................... 46
Version control................................................................................................. 46
Why........................................................................................................... 46
What.......................................................................................................... 47
How........................................................................................................... 47
Open source or an enterprise system?..................................................... 48
Automation...........................................................................................48
Run SoapUI test suites without SoapUI........................................................... 48
Command line.................................................................................................. 48
Maven.............................................................................................................. 49
Add a repository for SoapUI...................................................................... 50
Add and configure the SoapUI plugin....................................................... 51
Execute a test from Maven....................................................................... 52
Properties.................................................................................................. 53
A complete example.................................................................................. 53
More details.............................................................................................. 55
Automate the execution................................................................................... 55
Conclusion............................................................................................56
SoapUI Pro
by SMARTBEAR
3
Introduction
Why SoapUI?
If you’re unfamiliar with SoapUI, we’ll begin with a short introduction
to what the tool actually does. SoapUI is an API testing tool that’s both
free, open source, and cross-platform. With over 1 million downloads it
has become the de facto standard for functional testing of APIs, allowing you to easily and quickly create automated functional, regression,
compliance, and security tests.
SoapUI is also available as SoapUI Pro, which includes several timesaving features aimed at making your testing faster and your testing life
easier.
Functional testing
While SoapUI’s functionality includes functional testing and security
testing, as well as service simulation, functional testing is definitely the
most popular.
Functional testing means checking that the behavior of the system
under consideration matches the business expectations in terms of
functionality. SoapUI enables users to perform functional testing easily.
This is the main functionality of SoapUI.
With SoapUI, you can also perform security testing and service simulation.
You would set up SoapUI to send in requests and specify assertions
to verify that the response is correct. You would also specify broken
requests to be able to verify that the application doesn’t blow up when it
receives that strange and possibly faulty request. These tests can then
be used as one-off, unit tests, regression tests, etc.
Load testing should be done using LoadUI, which is integrated as a
component in SoapUI.
SoapUI Pro
by SMARTBEAR
4
What is an API?
An Application Programming Interface, API, specifies how some software components should interact with each other.1
While many people associate APIs with software libraries, the definition
is broad enough to include Web Services and all types of messaging
that SoapUI supports: SOAP, REST, JMS, JDBC, AMF and HTTP.
What is a Web Service?
The W3C defines a “Web Service” as:
A Web Service is a software system designed to support interoperable
machine-to-machine interaction over a network. It has an interface described in a machine-processable format (specifically WSDL[sic]). Other
systems interact with the Web Service in a manner prescribed by its
description using SOAP-messages, typically conveyed using HTTP with
an XML serialization in conjunction with other Web-related standards.2
In other words, a Web Service is a service available on a network, that
will allow other systems to communicate with it, using a defined protocol. The Web part indicates that the service is using transport protocols
designed for the World Wide Web, i.e., it uses HTTP to communicate.
Other systems can be Web applications, apps in phones and similar.
APIs come in two main flavors, SOAP and REST. Generally, we have
noticed that when discussing APIs and Web Services, the two terms
have different connotations, though they technically define the same
thing. “Web Service” is generally used when discussing SOAP-based
services, while API is used in a broader sense, encompassing SOAP,
REST, and the rest of the API protocols. Therefore, we simply use the
abbreviation API to refer to the service henceforth.
1http://en.wikipedia.org/wiki/Application_programming_interface
2http://www.w3.org/TR/2004/NOTE-ws-gloss-20040211/#webservice
SoapUI Pro
by SMARTBEAR
5
SOAP
A SOAP API is defined as a receiver of an XML document and is also
expected to respond with an XML document. The documents should
follow a format standardized by W3C.
All parameters that the receiver needs to be able to respond to should
be a part of the XML document sent.
Testing a SOAP API means manipulating a request XML and interpreting a response XML.
REST
REST or RESTful APIs encode all parameters in the request. A RESTful
service is never stateful. The server should not keep any state about a
client.
Testing a REST API means manipulating the request URI and interpreting the answer. The answer may be of many different formats. XML is
possible, but formats like JSON or plain text are also possible.
Testing APIs
There are some initial challenges when testing an API. There is no
graphical user interface, and, you must understand a response that
hasn’t primarily been created to be read by humans.
No graphical user interface
APIs are primarily intended for computer-to-computer communication.
Computers don’t have any use of a user interface designed for a human. This is a challenge if you are a tester who is accustomed to testing
software designed for end-users, with a graphical interface you can
click, drag-and-drop, collapse, close, and in general manipulate on the
screen.
A solution is to use testing tools that will assist you as much as possible
to create requests and view the responses.
SoapUI Pro
by SMARTBEAR
6
SoapUI and SoapUI Pro are two tools that will make your life as a tester
less complicated, by providing interfaces to enter your request, assertions to validate your response, various tools to create and maintain
your tests, and reports to print and share with your colleagues. The tools
also give you a user interface to both call APIs, and analyze answers
from APIs. Hence the “UI” ending in SoapUI, meaning User Interface.
Request and response
SOAP-based APIs use XML as a communication protocol. It must provide an XML descriptor (WSDL), that in machine terms, clearly defines
all types of request elements the SOAP API will accept, and all types of
response elements it will send back. This means that you, as a tester,
must be able to read, understand, create and update XML documents to
be able to test a SOAP API.
In REST-based services, the request is much simpler, for the most part
only involving sending a URL (or URI) to the service. The responses
returned are usually in the format JSON or XML. These are, again, not
formatted to be consumed by human beings, i.e. you.
This means testing the service may be a challenge without a good tool.
Once again, SoapUI and SoapUI Pro will make your life as a tester
easier. You will be able to do small changes to an XML request and get
a response back from the server that you can validate. SoapUI will assist you with response validations, response format, and response times
so that expected SLAs can be asserted.
Exploring
A common situation when SoapUI is used, is when an unknown API is
explored.
With the API computer-to-computer communication, the API will act as
the host and you need to have another piece of code that would act
as an API client. You can set up SoapUI as the API client. This is very
convenient and saves a lot of time. This is common among developers
SoapUI Pro
by SMARTBEAR
7
who need to write a client for an unknown service. Exploring a service
is probably less common among testers, as they tend to have an API
that is known and should be verified. The exploring and discovering has
already been done.
Getting started
You need to do a few things to be able to use SoapUI. This section will
help you
¿¿ Set up SoapUI
¿¿ Get started with your first project
¿¿ Add a test suite
¿¿ Add a test
¿¿ Add an assertion
¿¿ Run a test suite
I will use baby steps to allow for a beginner to follow. I hope it isn’t too
detailed for you.
Let’s begin at the beginning and install SoapUI.
Installing
Download SoapUI from http://www.soapui.org/. Follow the download link
to get the version you want. Both SoapUI and SoapUI Pro are available
at the same location.
SoapUI is available for many different operating systems. Just follow the
installation instructions for your operating system:
¿¿ Windows: www.soapui.org/Getting-Started/installing-onwindows.html
¿¿ Mac: www.soapui.org/Getting-Started/installing-on-mac.html
SoapUI Pro
by SMARTBEAR
8
¿¿ Linux: www.soapui.org/Getting-Started/installing-on-linuxunix.
html
SoapUI is installed. Let’s continue with a small example project.
Your first project
Let’s create a project.
There is an API that will take two currencies and return the currency
conversion. Let’s use that for our example. It has very few moving parts,
so it will be a good starting point.
¿¿ Start SoapUI.
¿¿ Create a new project, File | New soapUI Project.
¿¿ Give it a descriptive name. I choose to name it “Currency
converter.” Enter the URL to a description of the service in
the field “Initial WSDL/WADL”: http://www.webservicex.net/
CurrencyConvertor.asmx?WSDL
¿¿ The checkbox, “Create Requests” is checked by default. Leave
the defaults and click “OK.” SoapUI will now call the API to get
its details.
SoapUI Pro
by SMARTBEAR
9
A new project should have shown up in the Navigator section in top-left
section of SoapUI. There are two versions of the service: “CurrencyConverterSoap” and “CurrencyConverterSoap12”. The difference
between these is that the first one follows an older standard of SOAP
compared to the latter one. Which one we use in this example doesn’t
matter, so I will remove “CurrencyConverterSoap12”. Mark it and press
the Delete button, or right click on “CurrencyConverterSoap12” and
choose Remove.
SoapUI Pro
by SMARTBEAR
10
Let’s explore the remaining service. Expanding the tree to the left in
the project explorer reveals four things: the project name, “Currency
converter,” below it the name of the interface, “CurrencyConverterSoap,”
below it, the method name, “ConversionRate,” and finally, a sample
request to the service, called “Request 1.” “Request 1” is a name that
SoapUI has generated for us. We want to change it to a name that is
more descriptive. I chose “Euro To US Dollar.” Right click on “Request 1”
and set the new name.
SoapUI Pro
by SMARTBEAR
11
Double click on the “Euro To US Dollar” request. A sample request will
show up in the desktop. The content should look like this:
<soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/
soap/envelope/” xmlns:web=”http://www.webserviceX.NET/”>
<soapenv:Header/>
<soapenv:Body>
<web:ConversionRate>
<web:FromCurrency>?</web:FromCurrency>
<web:ToCurrency>?</web:ToCurrency>
</web:ConversionRate>
</soapenv:Body>
</soapenv:Envelope>
SoapUI Pro
by SMARTBEAR
12
This is an XML message that will be sent to the server where the API
runs. Sending it as it is will result in an error. We need to change two
things before we can send it and expect something useful in return.
The two question marks are placeholders for two parameters that must
be set properly. The names of the parameters are “FromCurrency” and
“ToCurrency.” Replace the question mark for “FromCurrency” with the
symbol for Euro, EUR. Then replace the question mark for “ToCurrency”
with the symbol for US Dollar, USD. The result should look like this:
<soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/
soap/envelope/” xmlns:web=”http://www.webserviceX.NET/”>
<soapenv:Header/>
<soapenv:Body>
<web:ConversionRate>
<web:FromCurrency>EUR</web:FromCurrency>
<web:ToCurrency>USD</web:ToCurrency>
</web:ConversionRate>
</soapenv:Body>
</soapenv:Envelope>
SoapUI Pro
by SMARTBEAR
13
We should get the conversion rate between Euro and US Dollar when
we send this to the server. The response will be a similar XML document, where we will locate an element that contains the answer. It will
look like this:
<soap:Envelope xmlns:soap=”http://schemas.xmlsoap.org/soap/
envelope/” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”>
<soap:Body>
<ConversionRateResponse xmlns=”http://www.webserviceX.NET/”>
<ConversionRateResult>1.318</ConversionRateResult>
</ConversionRateResponse>
</soap:Body>
</soap:Envelope>
The element “ConversionRateResult” contains the value 1.318, which
should be interpreted to mean that it will cost you 1.318 US Dollars to
get one Euro. I will add an assertion later that will verify this value. Adding an assertion in this case may be a bit risky, as the exchange rate
between Euro and US Dollar isn’t fixed. But let’s ignore that problem for
now.
Add a test suite
The sample request we made earlier should be included in a test suite
that contains one or more test scenarios. Each scenario should also
contain one or more assertions. We will execute this test suite to do a
functional test of an API.
SoapUI Pro
by SMARTBEAR
14
Let’s add the test suite.
¿¿ Right click on “CurrencyConverterSoap.”
¿¿ Click on “Generate TestSuite.”
¿¿ A dialog box will show up where you can customize the
generation. Leave the defaults as they are and click “OK” to
generate a test suite.
Call the test suite “Euro to Dollar TestSuite” and click “OK.”
A test suite is generated for us. It has been populated with one test
case. Let’s move on and have the test case do something useful.
Add a test
Expand the newly created test suite. A test case has been created for
us. Our next step is to tweak it so we can verify the conversion rate.
¿¿ Expand the tree until the test steps have been unfolded and the
test step “ConversionRate” is visible.
¿¿ Double click on the test step. A sample request should appear in
the request editor.
SoapUI Pro
by SMARTBEAR
15
Replace the two question marks in the XML with EUR and USD. The
result should look like this:
<soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/
soap/envelope/” xmlns:web=”http://www.WebserviceX.NET/”>
<soapenv:Header/>
<soapenv:Body>
<web:ConversionRate>
<web:FromCurrency>EUR</web:FromCurrency>
<web:ToCurrency>USD</web:ToCurrency>
</web:ConversionRate>
</soapenv:Body>
</soapenv:Envelope>
¿¿ The “web:FromCurrency” tag should contain EUR and the
“web:ToCurrency” should contain USD.
SoapUI Pro
by SMARTBEAR
16
This is all we need to do to create our first test case. We can execute
this test case now.
The result after an execution will look similar to this:
Unfortunately, we will not get any indication if it failed or passed if we
execute it now. We need to add at least one assertion that will warn us if
there was a problem that failed this test. Let’s do this as the next step.
SoapUI Pro
by SMARTBEAR
17
Add an assertion
Add our first assertion.
¿¿ Click on the label “Assertions” at the bottom of the request
editor.
¿¿ This will expand the assertions editor. It is empty.
Click on the small +-sign at the top of the assertions editor.
Select “Property Content assertions.” The first one in the list is a Contains assertion. Let’s use that one. Click on the “Contains” box and then
click on “Add” to add it to the test case.
SoapUI Pro
by SMARTBEAR
18
Add 1.318 in the next dialog box.
Click “OK” to add it.
We have one assertion to start with. We will probably want to add more
assertions later, but let’s start small and just have one. We will add more
moving parts when this first case has been verified. Hardcoding the
conversion rate between two currencies like this is brittle. We will have
to do something about that later, also.
SoapUI Pro
by SMARTBEAR
19
Run the test
The first thing we want to do is to run a single test case.
¿¿ Click on the small, green arrow at the top left corner of the
request editor.
¿¿ This will send the request to the server. The assertion will be
green if the value 1.318 is available in the response.
Next step is to run the entire test suite. It doesn’t contain more than a
single test at the moment. The result should be exactly the same as running a single test.
Run the test suite
Executing the entire test suite can be done like this:
¿¿ Double click on the test suite “Euro to Dollar TestSuite.”
¿¿ Click on the small, green arrow at the top of the test suite editor.
All test steps will be executed and they should be green. There is only
one in this case, but one is enough to verify the execution.
SoapUI Pro
by SMARTBEAR
20
Further exploration
Now that you have created a test suite, it is time to add more test steps
to it or add more assertions to the step we just created. The basic infrastructure is in place. Now it is just more of the same.
Create tests
Creating tests is mostly about adding proper assertions to a test case.
We have our test step that will verify the conversion rate between Euro
and US Dollar. The verification we added earlier is brittle. The conversion rate between two currencies is seldom fixed. Other verifications
we can add are verifying if a response is received fast enough and thus
answer the question if the service we are testing is following a given
service level agreement, SLA.
SoapUI Pro
by SMARTBEAR
21
Let’s add verifications to our test case and discover more about how to
verify the response.
Verify a range
It is easy to verify that an answer contains a value. This may be great if
you want to verify a value that is not expected to change. Some values
change. We have to create a test that can handle a range instead of a
fixed value. Suppose that we can assume that the response will contain
a value that is greater than something else. With this, we can verify a
range.
Let’s assume that the conversion rate between Euro and US Dollar is
larger than 1 and lower than 2. How could that be expressed?
We need to start with locating the element that contains the value.
When this is done, we need to compare the value with our assumption.
¿¿ Click on the label “Assertions” at the bottom of the request
editor.
¿¿ Click on the small +-sign at the top of the assertions editor.
¿¿ Select “Property Content.”
SoapUI Pro
by SMARTBEAR
22
¿¿ Select “XPath match” and click “Add.”
¿¿ Click “Declare” in the XPath editor, SoapUI declared two name
spaces for you. They can be called anything. The two name
spaces that were declared are called soap and ns1.
¿¿ Rename ns1 to something more descriptive. I choose Web
since it is the same name space that is used in the request.
Web is perhaps not much more descriptive than ns1, but at least
it is not an abbreviation for Name Space 1.
¿¿ The next step is to add an XPath3 expression that will search
for the element that contains the conversion rate.
//Web:ConversionRateResult
3http://www.w3schools.com/xpath/
SoapUI Pro
by SMARTBEAR
23
¿¿ Verify that the syntax is correct by pressing the “Select from
current” button. You should get the value available in the
response. I got 1.318 when I tried.
¿¿ When we are able to locate the element, let’s continue with
verifying that it is larger than one. Rewrite the XPath expression
like this:
//Web:ConversionRateResult > 1
SoapUI Pro
by SMARTBEAR
24
¿¿ Verify the result again by pressing “Select from current.” I got
the result true.
¿¿ Press the Test button to execute the test. I got a dialog box that
told me that the test was a success.
¿¿ Save the current verification by pressing “Save.”
To verify that the conversion rate is lower than 2, create a similar assertion and change the XPath to
//Web:ConversionRateResult < 2
An easy way to do this is to clone the XPath assertion. Right click on it,
select “Clone” and do the appropriate changes.
SoapUI Pro
by SMARTBEAR
25
Verify response time
Verifying the response time is often important. A slow API is a problem
waiting to emerge. Customers will probably start to complain when you
have a lot of traffic and they don’t get their response quickly enough.
Let’s prepare for this by requiring the service to respond within 1 second.
¿¿ Add a new assertion.
¿¿ Select “SLA” and “Response SLA.”
¿¿ Add it.
¿¿ Specify the desired response time. The default value is 200 as
in 200 ms. We want to allow one second, so set the new value
to 1000 ms and press “OK.”
Adding a minimum response time like this is easy. A more difficult problem may be to actually deliver such short response times. But that is not
our current focus; we are just verifying that the service is fast enough.
Verify schema compliance
APIs should follow their own schema definition. We used the schema
definition when we created the request to the service. We need to add
verification if we want to be sure that the response follows the schema it
should be following.
¿¿ Add a new assertion
¿¿ Select “Compliance, Status and Standards”
¿¿ Select “Schema Compliance”
¿¿ Add it
¿¿ Define the schema, the response should follow. The suggested
schema is the same schema as we used to create the service.
This is the schema we want to use in this example.
¿¿ Click “OK.”
SoapUI Pro
by SMARTBEAR
26
All requests now have to follow its schema definition. If they don’t, the
test will fail. This is great if we want to catch a bug introduced to the
service.
Many more tests can obviously be added, but the ones I have added
here should get you started.
Go pro: Point and click testing
SoapUI Pro has been extended to support pointing and clicking when
you create tests or add assertions. This is a real time saver and is
something you will really appreciate when you are working under time
constraints.
Creating an XPath assertion can be tricky when you need to get the correct data. It is very easy to do using SoapUI Pro where you can click on
the data element in the response and get a proper XPath for it.
SoapUI Pro also makes moving data from a response to the next
request easy by assisting with the XPath expression you want to use to
retrieve the correct data.
Verify a range
Let’s revisit verifying that the response is within a certain range. We
did that by writing an XPath validation manually. SoapUI Pro has
implemented a wizard that will remove a lot of the details you have to
remember and allow you to focus on the actual problem: verifying that
the conversion rate between Euro and US Dollar is larger than one.
¿¿ Add a new assertion.
¿¿ Select “Property Content.”
¿¿ Select “XPath Match.”
¿¿ Add it.
¿¿ Click on the small node selector to the left of the “Declare”
button.
SoapUI Pro
by SMARTBEAR
27
Select the node you are interested in verifying. An XPath expression will
be created for you and will locate that specific element. This is a read
only field.
¿¿ Click “OK.”
¿¿ Change the XPath created for you from
//ns1:ConversionRateResponse[1]/
ns1:ConversionRateResult[1]
to
//ns1:ConversionRateResponse[1]/
ns1:ConversionRateResult > 1
SoapUI Pro
by SMARTBEAR
28
¿¿ Click “Select from current” and notice that the “Expected Result”
area gets set to true.
¿¿ Run the test by clicking the “Test” button. It should be a success.
¿¿ Save the test.
We just added the same verification, but we didn’t have to understand or
know how to write the XPath to locate the value we wanted to verify.
Spending less time writing XPath expressions and more time focusing
on what really needs to be tested is a good reason for going pro.
Data driven testing
Verifying an API using different test data is common. A service should
perhaps be able to translate between many different currencies. Verifying each combination by hand can very quickly be too much to cope
with.
SoapUI doesn’t have any built in support for reading test data from
a source and using it as a parameter to a test. It is possible to implement using a Groovy script that reads from the data source and passes
parameters to a test case. This is, unfortunately, out of scope for an
introduction to SoapUI, so it will not be covered here.
SoapUI Pro
by SMARTBEAR
29
Another solution is to use SoapUI Pro that has support for reading from
a data source and passing the result to a test case out of the box.
Go pro: Data source out of the box
Setting up a test to use a data source is available out of the box for
SoapUI Pro. The setup is done in the following steps:
¿¿ Create a data source.
¿¿ Create a test that uses values from the data source.
¿¿ Add a loop to iterate over the values found in the data source.
Let’s add a data source to our example.
Create a data source
The data source is created as a test step. We need some kind of data
before we can make any use of it. SoapUI Pro supports these types of
data sources:
¿¿ Data connection – connect to a data source and use SQL to
extract your test data
¿¿ Directory – read a set of files from a directory and use their
content as test data
¿¿ Excel – read an Excel sheet and use its content as test data
¿¿ File – read a separated file and use as test data
¿¿ Grid – define a grid in SoapUI Pro that will hold all test data
¿¿ Groovy – create a Groovy script that generates the test data
¿¿ JDBC – connect a data source and use SQL to extract the test
data. This one is very similar to the Data connection above
¿¿ XML – read test data using XPath expressions
I want to keep it simple, yet useful, so I will use a comma-separated text
file where each row will be test data for one test.
SoapUI Pro
by SMARTBEAR
30
Create a text file with this content:
EUR, USD
EUR, GBP
SEK, PLN
I saved it as test-data.txt. This will be our first data source. Our goal is
to iterate through it and verify that we get conversion rates for the three
different cases.
The next step is to connect to it.
¿¿ Add a new test step to the test steps, select the type
“DataSource.”
¿¿ Specify a name for it, I used ConversionsDataSource.
¿¿ Define the type of Data source in the drop down DataSource.
Select “File.”
SoapUI Pro
by SMARTBEAR
31
¿¿ Browse to the file you just created and select it.
¿¿ The default setting is that the file will be comma-separated,
leave that as it is.
¿¿ The default value for “Charset” is also ok.
¿¿ The checkbox Trim values is set and that is also something
we want to keep. The values we read from the file should be
stripped of whitespace, EUR should be read without the spaces
surrounding it.
¿¿ Our values are not quoted, so leave that checkbox unset.
¿¿ Name the two columns in the data source. Click on the +-sign in
“Properties” editor to the left.
¿¿ Call the first property “FromCurrency” to indicate that the first
column in our test data file will be the currency we want to get
the conversion from.
¿¿ Call the next column “ToCurrency” to indicate that the second
column in the test data file is the currency we want to get the
conversion to.
SoapUI Pro
by SMARTBEAR
32
This is all we want to set for our data source. Next step is to use it in a
test case.
Create a test with parameters
The test step is similar to the one we have created before. The only actual difference will be that instead of defining the currencies, we will use
variables. These variables will be set with values from the data source
and we will get many different tests. This is how to do it:
¿¿ Add a new test step to test steps, select the type “Test Request”
and call it “Conversion Test Request.”
¿¿ Select the operation to invoke for this request. There is only one
available.
¿¿ Define some settings in the next dialog box.
¿¿ “Add SOAP Response Assertion” is selected, leave it selected. It
will verify that we actually receive a SOAP response.
¿¿ I leave the rest of the suggested assertions unchecked and
Click “OK.”
¿¿ Connect the parameters.
¿¿ Right click on the drop-down “FromCurrency.”
¿¿ Select “Get Data.”
¿¿ Navigate to the step “ConversionDataSource.”
¿¿ Select “Property [FromCurrency].”
¿¿ Do the same for “ToCurrency.” Right click -> Get Data ->
ConversionDataSource -> Property [ToCurrency].
That was all we had to do to connect the data source with the test case.
Next step is to iterate, or loop if you want, over the data source, so we
can execute the test case for each row that is found in the data source.
SoapUI Pro
by SMARTBEAR
33
Loop over the data source
Iterating over the data source is done using a DataSource Loop.
¿¿ Add a new test step to test steps, select the type “DataSource
Loop” and call it “Conversion DataSource Loop.”
¿¿ Double click on the new test step to bring up the editor where
we can define which step to use as the data source for this loop,
and define which step will be executed.
¿¿ Select “ConversionDataSource” as Datasource Step.
¿¿ Select “Conversion Test Request” as Target Step.
¿¿ Click “OK.”
The data source and the test request have been connected. The next
step is to execute a test run and verify that we send three different
requests and get valid answers back.
SoapUI Pro
by SMARTBEAR
34
Execute the test case
We have prepared a test case. Let’s execute it and see if we get something interesting.
¿¿ Double click on the “ConversionRate TestCase” to bring up the
TestCase overview.
¿¿ The test steps should be listed for execution.
¿¿ Click on the small green arrow in the left, upper corner, to
execute the steps.
SoapUI Pro
by SMARTBEAR
35
Eight steps were executed. The first was our previous test step, the
second read the data source, the next executed the test request, and
the fourth read the next value from the data source. The last two steps
where repeated until all rows in the data source were read.
Try adding one more row in the file test-data.txt and execute the test
case again. I added USD, EUR. Adding one row resulted in one more
test case that was executed. Verifying more combinations is just a matter of adding them to the data source file.
Further exploration
A simple file was used as a data source in this case. It is common to
have other data sources. Many of them can be connected out of the box
with SoapUI Pro.
Debugging
Writing a test perfectly the first time is difficult. Debugging and testing
are common activities that you usually will spend some time doing until
the test you are creating is perfect, or at least good enough.
SoapUI Pro
by SMARTBEAR
36
Manual
Problem: You have a test suite with some steps, and you are not sure
that the steps are correct.
Solution: Disable all steps, except the first step. Rerun the suite and
verify the outcome. When the first step delivers the expected result,
enable the next step. Adding one step at a time, in a controlled way, is a
great way to build up confidence for the suite.
A general problem-solving technique to debug anything is half-interval
search4. You remove half of the steps and check to see if the problem
you are trying to locate is present. If it isn’t, then you know that your
problem is in the other half. Dividing the steps into smaller and smaller
parts will eventually, often quite soon, tell you exactly where the problem
is.
Go pro: Test debugging
Easily follow your test flows step by step, and improve the quality of
your tests with Test Debugging. There are times when you want to
examine the behavior of your tests in detail – this is called debugging.
If you suspect errors in your tests or the services you’re testing, Test
Debugging will help you diagnose it. With the debugging support, you
can execute your TestSteps step by step. Alternatively you can add
breakpoints and then run the test to those set breakpoints and view the
current value of the SoapUI properties. The Test Debugging Interface
simplifies following Test Flow, Variables, Properties, Requests, Context,
and much more, making test creation and improvement more streamlined.
4http://en.wikipedia.org/wiki/Binary_search_algorithm
SoapUI Pro
by SMARTBEAR
37
Let’s walk through a small debugging session to show how it can be
done using SoapUI Pro.
¿¿ Double click on the “ConversionRate TestCase” in the Navigator.
¿¿ Select “TestCase Debugging” in the test case editor to the right.
¿¿ The steps are now listed and can be executed by clicking on the
small, green arrow above the test step list. Doing so now will
execute them in a sequence. You must add breakpoints to be
able to step through them.
¿¿ Add breakpoints. Click just to the left, in the column named BP,
of each test step. That will toggle a breakpoint on or off. Set
breakpoints on all steps.
¿¿ Run the tests and notice that the execution stops at each
breakpoint.
¿¿ The current step is indicated with a small, green arrow, just to
the right of the step name.
¿¿ Click on the solid green arrow to run one step at a time.
SoapUI Pro
by SMARTBEAR
38
Debugging a simple case like this may not be very interesting, but it
prepares you for more complicated cases.
Debugging test is only available in SoapUI Pro and is a good reason for
going pro.
Multiple environments
The test you have created should be possible to use for a system test
environment as well as an acceptance test environment. The only thing
you should change is the address, also known as the endpoint, to the
server where the system is hosted.
Change the target environment
The environment you test against is defined in the service endpoints.
Change it like this:
¿¿ Double click on the Service definition,
“CurrencyConverterSoap.”
¿¿ Select the “Service Endpoints” tab.
¿¿ Click on the endpoint you want to change in the list and change
the address.
http://www.webservicex.net/CurrencyConvertor.asmx to http://acc-www.
webservicex.net/CurrencyConvertor.asmx if you want to test the same
service on the host acc-www.
¿¿ Press enter when you are done with your change.
¿¿ Execute the test step. My execution failed, there is no host
called acc-www.webservicex.net
It is possible to use properties to define the endpoint, and this could be
another way of setting the service endpoints in such way that they are
fast to change. More information about this is available on soapui.org5
5http://www.soapui.org/Functional-Testing/structuring-and-running-tests.html
SoapUI Pro
by SMARTBEAR
39
Go Pro: Multi-Environment Support
If you’re testing different environments, such as Staging and Production,
this feature will be a major time-saver. Manually changing all the different connections and end-points of your tests when you need to switch
environment is cumbersome. With SoapUI Pro you can choose which
environment you want to apply through a simple drop-down menu.
SoapUI Pro then automatically changes the necessary parts of your
tests to enable it to run in the new environment. All that’s needed is a
one-time setup of your different environments, and you are good to go.
Let’s walk through an example on how you can set up different environments so you can switch between them quickly.
¿¿ Double click on the project.
¿¿ Select the “Environments” tab to the far right.
¿¿ Click on the small +-sign to add a new environment.
¿¿ Name it “Acceptance.”
¿¿ Keep the default values and do not copy from the test case. We
will add a good endpoint manual in a short while.
¿¿ Click “OK.”
SoapUI Pro
by SMARTBEAR
40
¿¿ All requests and their endpoints are listed in the dialog box
below the Environment name.
¿¿ Double click on “CurrencyConverterSoap.”
¿¿ Add a good endpoint for it. I use:
http://acc-www.webservicex.net/CurrencyConvertor.asmx
¿¿ Test it by executing the test case and selecting “Acceptance” as
environment.
¿¿ It failed, there is no host called http://acc-www.webservicex.net.
¿¿ Create another environment by following these steps again, call
it “Production” and use the endpoint.
http://www.webservicex.net/CurrencyConvertor.asmx
It should work when you execute the test and use the environment
“Production.”
SoapUI Pro
by SMARTBEAR
41
When you change the environment you test, a database holding the
current data set for the environment will often have to be changed. This
can be done at the same time. Let’s extend the example a little bit by
adding support for different databases at the same time as you change
environment.
We don’t have a database connection in our example, so let’s add one.
¿¿ Double click on the project.
¿¿ Select the tab “JDBC Connections.”
¿¿ Add a new data connection by clicking on the small +-sign in the
left corner.
¿¿ Specify a name for the data connection. I use “Customer” to
indicate that this is our customer data connection.
¿¿ Click “OK”.
¿¿ Select the correct database and connection details. I set up a
MySql connection.
We just created a data connection. Now let’s make sure that we connect
to different databases when we change environment.
SoapUI Pro
by SMARTBEAR
42
Open up the “Environments” tab again.
We have an environment called “Acceptance,” let’s add another one
called “Stage.” Click on the small +-sign above the environment list and
call it “Stage.”
Keep the setup defaults empty and click “OK.”
Click on “Acceptance” and notice that we have an endpoint set.
Click on “Stage” and notice that we haven’t an endpoint set. Set it to
something appropriate.
Click on the “JDBC Connections” tab and check the value set for “Acceptance” and then the value set for “Stage.”
Acceptance has the same connection details as we specified earlier.
Stage doesn’t have any Connection string defined yet.
Double click on the “Customer JDBC” connection for Stage and set up
the connection string.
The result will be that when we test against Acceptance, we will use the
endpoint defined for SOAP Services and the data connection defined in
JDBC Connections for Acceptance. When we change environment to
SoapUI Pro
by SMARTBEAR
43
Stage, both of these settings will be changed. This allows us to change
between different environments very quickly.
Using different environments could be a good reason to go pro.
Reporting
Reporting the status of your test runs is important. A test report is often
required before you are allowed to deploy a product into production.
There is no built-in support in SoapUI to generate reports from a test
execution. This is only available in SoapUI Pro.
Create a test report
A test report is created based on a test suite execution. This means that
we must start by running a test suite before a report can be generated.
¿¿ Execute the test suite Euro To Dollar TestSuite.
¿¿ Click on the “report” button in the toolbar to generate a report. It
is the icon just to the left of the environments drop-down.
¿¿ Select a format. I select TestSuite Report.
¿¿ Click “OK.”
¿¿ A preview of the report shows up.
¿¿ Save it by clicking on the floppy icon to the far left.
SoapUI Pro
by SMARTBEAR
44
¿¿ A “save” dialog box shows up where you can define the name
for the report as well as the format. I choose RTF, Rich Text
Format, because I want to be able to edit it later, using Microsoft
Word.
SoapUI Pro has an extensive set of prepared reports that supports
these formats:
¿¿ PDF – Portable Document Format
¿¿ RTF – Rich Text Format
¿¿ ODT – Open Office
¿¿ HTML
¿¿ Single sheet XLS – Excel
¿¿ Multiple sheets XLS – Excel
¿¿ CSV – Comma-separated file for import to Excel
¿¿ XML
You can change the content of the reports, but that is out of scope for
this SoapUI introduction.
Organize your work
How you organize your work is obviously not something that should be
dictated by a tool. At the same time, the tool you are using may support
working in different ways. SoapUI will force you to organize your work
using workspaces. It will give you a choice on how you want to save
your project so it can be shared with other testers. Let’s start with workspaces and then discuss version control and sharing.
SoapUI Pro
by SMARTBEAR
45
Workspaces
SoapUI uses workspaces6 where you do all your work. A workspace
contains all your current projects. This means that you can work on
more than one project in the same SoapUI session. When you want
to change to another project either add it to the current workspace or
switch to another workspace.
Workspaces can be shared in the team. I am not sure that sharing the
workspaces are the best idea though. Sharing workspaces will force
everyone to work with the same set of projects all the time. This may not
suit how you do the work.
Version control
Storing your projects in a version control system is considered the best
practice. It will enable you to go back to a known version. It will make it
easier to share files with other testers.
Why
A shared resource on a file system is a fragile way to share your work.
People can overwrite files that another person is working on and
changes can easily get lost. File history is hard to keep track of. Tracking who made which changes to the project is not possible. Therefore,
sharing files through a version control system is often used in development projects.
The tests you have created with SoapUI have a significant value and
should be protected as much as possible. Storing them in a version
control system is therefore something you’ll want to do.
What
What should be stored then? Your workspace may or may not be
interesting for other testers. The workspace reflects how you personally
6http://www.soapui.org/Working-with-soapUI/managing-workspaces.html
SoapUI Pro
by SMARTBEAR
46
group different projects. Sharing them with the other testers is something you need to discuss and then decide upon.
You should store your project files. They are the most important outcome from your testing work. They should be used and maintained for
all testers in your team.
SoapUI Pro support saves the project as a composite. This means that
each part of the project will be saved in separate files instead of saving the entire project in the same file. This doesn’t have any impact on
anything more than the possibility to cooperate and allow more then one
person to work on the same project. But cooperating and sharing is the
key to scalability. One person can only do so much; two persons can do
a lot more.
Combining composite projects and storing them in a version control
system allows the testing effort to scale and more tests can be created
and maintained, compared to not using composite projects.
How
There are many different version control systems. Some of them are
large enterprise solutions, others are open source and freely available.
They all solve the same business problem by providing a single repository for your files and versioning the changes.
Open source or an enterprise system?
This is a question that depends a lot on the organization you work with.
Some organizations want to buy tools, others support open source tools
and just buy the support they need when they need it.
Open source solutions are used to store some very high profile systems.
The Linux kernel, for example, is stored using an open source version
control system called Git. SoapUI is also version-controlled using Git.
SoapUI Pro
by SMARTBEAR
47
Automation
Automated testing is an area where many companies put in a lot of
effort. They want to eliminate the mundane, expensive and manual testing that used to be the only way you could verify a system. They also
want the developers to get fast feedback as soon as they have done a
change to a system.
How can SoapUI be used for test automation? The trick is to execute
test suites developed using SoapUI, without using the user interface of
SoapUI, and instead trigger it from some kind of automation build tool.
There are many different tools that could trigger the execution. Most of
them have in common that they are good at triggering anything from a
command line prompt. This could be a bat script in Windows, a shell
script in Unix, or it could be Maven project in a Java build environment.
Run SoapUI test suites without SoapUI
The first example will show how SoapUI can be executed from a command line. The second example will show how Maven can be used to
run SoapUI.
Command line
Command line tools are good if you want to be able to execute something from a script that you have created yourself. SoapUI has support
for executing from a command line.
Execute it like this:
testrunner project_file
where project_file is a SoapUI project file that could look like this c:\
projects\my-soapui-project.xml if you are on Windows.
The command line tool takes a lot of parameters that will allow you to
control the execution. They are described at the SoapUI Web site.7
7http://www.soapui.org/Test-Automation/functional-tests.html
SoapUI Pro
by SMARTBEAR
48
It turns out that it is actually the command line tool that is used when a
test is executed from the TestRunner. If you want help with running the
exact same execution from a command line that you executed from the
TestRunner, then try this:
¿¿ Execute the test you want from the test runner.
¿¿ Scroll back in the execution log until you find the launch of the
test runner in the beginning of the log.
¿¿ Copy the command and use it in your own script or from a
command line.
Maven
Maven is a popular build tool for building Java. Running a SoapUI test
from Maven is similar to running it from a command line. The difference
is that you need to set up a Maven project and then use Maven. This
is a good alternative if Maven is already building the rest of the project.
Many Java projects are using Maven. Let’s set up a Maven project that
will run SoapUI.
Maven is very good at handling dependencies. There are repositories
on the Internet where dependencies that could be used in a Maven project are stored. When Maven realizes that it doesn’t have a local copy,
it will download the resources it misses. This is a feature we can use
when retrieving the SoapUI plugin.
Maven uses a file called pom.xml for all settings. We need to extend an
existing pom file for your project or create a new one. Going through all
the details for a Maven project is out of scope here. I will just focus on
the parts that are SoapUI specific and leave the rest of the details to be
explained by other resources8 9 on the Internet.
8http://maven.apache.org/
9http://thomassundberg.wordpress.com/2013/02/24/maven-the-simplest-possible-introduction/
SoapUI Pro
by SMARTBEAR
49
The SoapUI specific things that must be added to a pom are
¿¿ Specify a repository where the SoapUI plugin can be found
¿¿ Add and configure a plugin that will execute the test
Add a repository for SoapUI
The SoapUI plugin for Maven must be found and downloaded. It lives in
a repository that must be specified specifically. Add this to your pom:
<pluginRepositories>
<pluginRepository>
<id>eviwarePluginRepository</id>
<url>http://www.soapui.org/repository/maven2/</url>
</pluginRepository>
</pluginRepositories>
The value of the id tag could be set to almost anything, as long as it is
unique. A descriptive name is, as always, preferable.
Now that we can locate the SoapUI plugin, let’s continue with adding
and configuring it.
Add and configure the SoapUI plugin
Add the following snippet to the build section of your pom:
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>4.5.1</version>
<configuration>
SoapUI Pro
by SMARTBEAR
50
<projectFile>simple-test-soapui-project.xml</
projectFile>
</configuration>
</plugin>
</plugins>
This will specify that we want to use a plugin called maven-soapui-plugin and that we want to use version 4.5.1 of the plugin.
The project we want to execute is specified in the configuration section
and it is called simple-test-soapui-project.xml.
You can specify more things, but this is a small start with just a few moving parts.
The next step would be to execute a test.
Execute a test from Maven
Maven is a command line tool that is executed with the command
mvn
You want to specify at least one goal that Maven should execute. The
simplest possible could be
mvn validate
It will validate that the format of your pom is correct.
SoapUI Pro
by SMARTBEAR
51
To actually execute the test we specified above, run this instead:
mvn eviware:maven-soapui-plugin:test
We specify that we want to execute the SoapUI plugin and the goal test.
The plugin has other goals that you could use. Executing load test is
one of them.
These were just a few snippets. Let’s put them in context in a complete
example.
Properties
You can define properties in your test. You need to add a section called
projectProperties in the configuration section. Then add the values you
want as name value pairs.
<configuration>
<projectFile>simple-test-soapui-project.xml</projectFile>
<projectProperties>
<value>message=Hello World!</value>
</projectProperties>
</configuration>
There should not be any space after the equals sign. It would place a
space in front of your value. That could be a subtle bug that you would
like to avoid.
SoapUI Pro
by SMARTBEAR
52
A complete example
A small example can look like this:
<?xml version=”1.0” encoding=”UTF-8”?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.smartbear</groupId>
<artifactId>soapui-example</artifactId>
<version>1.0-SNAPSHOT</version>
<pluginRepositories>
<pluginRepository>
<id>eviwarePluginRepository</id>
<url>http://www.soapui.org/repository/maven2/</
url>
</pluginRepository>
</pluginRepositories>
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-pro-plugin</artifactId>
<version>4.5.1</version>
<configuration>
<projectFile>simple-test-soapui-project.
xml</projectFile>
</configuration>
</plugin>
</plugins>
</build>
</project>
SoapUI Pro
by SMARTBEAR
53
I have defined the necessary plugin repository and configured the
soapui plugin. I used the pro version here. If you would like to do the
same thing for SoapUI, then use the plugin called maven-soapui-plugin.
To execute this example, run Maven like this:
mvn eviware:maven-soapui-pro-plugin:test
from a command line prompt.
You can define properties that will be set from the pom and you can
run the tests as an integrated part of the Maven build. That is more of a
Maven discussion than a SoapUI discussion, so I will leave that out to
minimize the number of moving parts.
More details
The Maven plugin can be used to do more than what I described here.
The details can be found at the SoapUIWeb site. 10
Automate the execution
After creating your SoapUI functional tests with test suites and connecting SoapUI tests to your build tool, it is time to execute them, as soon as
a change has occurred.
There are many different build tools available. One popular tool is Jenkins.11. Jenkins will be able to listen for a change on a url and kick off a
test job. This would make it possible for you to start testing as soon as
there is something new to test.
All build servers handle Maven projects out of the box, so setting up a
build that executes SoapUI tests is just a matter of setting up a Maven
build. Describing that is, however, out of scope here.
10http://www.soapui.org/Test-Automation/maven-2x.html
11http://jenkins-ci.org/
SoapUI Pro
by SMARTBEAR
54
Conclusion
You should now be able to verify and test APIs using a tool like SoapUI.
Your chances of securing the delivery of high quality APIs will be drastically improved, compared to when testing the services using your own,
homegrown API clients. You will save a lot of time and be able to focus
on the real problem, testing, instead of the problem of creating your test
tool.
Using SoapUI Pro will give you additional benefits. You will be able to
save even more time, and therefore money, by using the built-in features of SoapUI Pro that are specifically targeted on speeding up common tasks. Reporting, Data driven testing and simplified test creation
are just three examples.
SoapUI Pro
by SMARTBEAR
55
Haven’t gone PRO yet?
Try functional testing
in SoapUI Pro today!
About SmartBear Software
More than one million developers, testers and operations professionals use SmartBear tools to ensure the quality and performance of their
APIs, desktop, mobile, Web and cloud-based applications. SmartBear
products are easy to use and deploy, are affordable and available for
trial at the website. Learn more about the company’s award-winning
tools or join the active user community at http://www.smartbear.com, on
Facebook or follow us on Twitter @smartbear and Google+.
SmartBear Software, Inc. 100 Cummings Center, Suite 234N Beverly, MA 01915
+1 978.236.7900 www.smartbear.com
©2013 by SmartBear Software, Inc. Specifications subject to cha
SB-SUI-20130925-V1-SoapUI101