WORKSHOP – WEBSITE USABILITY – HOW TO SELECT RIGHT KPI’S AND COLLECT
USER EVIDENCES
Are you a stranger here …But only once!
LEOKADIJA SVIRIDOVA
Heinrich-Hertz-Institut, Einsteinufer 3, 10587 Berlin
Email address: leokadija.sviridova@hhi.fraunhofer.de
Abstract. The workshop is mainly directed to scientists and experts who are interested to get deeper
understanding of online-users behavior and seeking for proper technical tools.
1. Gathering user data
Web is constantly gaining importance as a platform for interactive applications. Recent tendency to go
away from simple HTML-Page layouts to the complex user interfaces and delivering more and more
functions to the client cannot be overseen. Enabling the analyzes how users behave and interact with
complex web applications is important for variety of reasons: profiling users and user groups,
development, debugging and optimizing web applications, revealing in-depth usage patterns for marketing
or business processes. To analyze the behavior of web user in the first row statistics need to be collected.
Variety of metrics with detailed tracking of user interaction can be collected using standard web
technologies. Tracking of user activity takes place either on the client-side, the server-side or combined.
Some procedures of user data collection are outlined below.
1.1 Client-side tracking
Client-side tracking by placing cookies at user’s device is one of the options to collect user data. The
cookie is a file, which is left at user’s device while user is browsing the website. Cookie helps website
operator to recognize the visitor and keeps track of his activity. The cookie is saving information and
profiling user. Information of diverse kind is collected: user preferences, customize data, last visit,
personal browsing habits, remember unique visitors or to keep track of items in an order.
Common are two types of cookies - HTTP-cookie and Flash-cookie. While HTTP-cookie is well known to
the most of users, Flash-cookie stays rather unknown. This helps the website operator to secure that the
information about the user action keeps being collected, even if the user explicitly rejected the cookie
placement in data privacy protection in browsers. If the cookie is deleted, user tracking is halt.
“Third-party” cookies – are cookies that belong to domains different from the one shown in the address
bar. Web pages are placing content from third-party websites. These opens up an option for tracking of
user's browsing history. E.g. while visiting a website, third-party websites - advertising websites send
“third-party” cookies to visitor. That allow advertising websites operator to track the sites visitor is
visiting, that use that same advertising network. This information can be used in turn to advertise targeted
products for user’s apparent interests. Explicit examples of these applications will be shown during the
workshop.
Inline or history sniffing – is sniffing of browsing history to track users activity outside the advertising
network. History sniffing uses the technique that browsers display links differently depending on whether
link was visited or not, by including of software code on a website that displays hidden links to web
addresses. When a visitor accesses the website, his browser will display these hidden links in one of two
colors. The software code can see differ the color and save the visitor’s browsing history accordingly.
History sniffing allows an advertiser to track activity on websites outside its network. This practice can’t
be prevented by simply deleting cookies. To provide an analogy for history sniffing - visitor is walking
into a Esprit store and he is told by the salesperson that they've been monitoring him and know that he
recently visited the H&M, Primark and so on.
1.2 Server-side tracking
Web server log – is a server side way to collect usage statistics. The information is logged on the web
server of the website operator. Web server log contains the history of page requests including client IP
address, request date/time, page requested, HTTP code, bytes served, user agent, and referrer. Combined
into a single file, or separated into distinct logs, such as an access log, error log, or referrer log. However,
server logs typically do not collect user-specific information. A statistical analysis of the server log may
be used to examine traffic patterns by time of day, day of week, referrer, or user agent. Applications like
Elasticsearch, Logstash and Kibana will be shown in the workshop.
1.3 Hybrid methods - combined server and client side tracking
HTTP proxy tracking – before delivering HTML pages to a client an HTTP proxy adds JavaScript
modification to these HTML pages. This JavaScript code intercepts all traffic between server and client
and outputs log data with details about any requests sent to servers as well as the replies that a server
sends back to the client. This code snippet collects data of user actions, such as navigation between pages,
actions on a page like mouse movements or scrolling, user input like clicks or text.
2. Measuring user activity with web analytics
Web Analytics – collection and analyses of user actions on the website with a goal to improve website
performance and monitor the efficiency of the website and its campaigns.
With management approach of setting goals, objectives and define metrics and key performance indicators
web analytics works with such tasks as e.g. how to attract new visitors in order to expand traffic, how to
transform visitors into users, and increase registrations, and how to build a loyal audience and make
visitors come back. Metrics collected over server-side and browser-side techniques are transformed to
measurements within tree important dimensions: recency, frequency und engagement of users.
The procedure of collecting metrics is described here in brief: A visitor follows referrer link arrives at
website. After the landing on a web page, visitor either bounce or request additional pages. Visitor may
complete a transaction, converting them from a lead into – a customer, a user, a member, or a contributor.
The visitor may abandon that transaction and exit the website. Visitor has many external attributes –like
browser they’re using, or geographical location where they are surfing from. With this information users
can be grouped into segments. The goal of analytics is to maximize conversions by optimizing website,
often by experimenting with different content, layout and campaigns; also analyzing the results of those
experiments on various internal and external segments.
2.1 Goals and KPI‘s
Goals in online business shall not be in contradiction with overall company goals. There are some
examples, where operative goals looks like going in opposite direction, e.g. free of charge online services
vs. profit maximization. However considering overall orientation the strategic online goal has to match
with company goals. So will the use of free of charge online service reduce the load on the call center.
Consequently this will lead to the reduction of overall costs and result in higher profit. KPI is a benchmark
that shows the progress of an organization’s efforts toward a specific goal. KPI‘s are almost always rates,
averages, percentages, or ratios. KPI on example of content website: media/content-websites directed on
B-2-C information flow. Such kinds of websites are delivering current and desired content for private user.
Therefore it is crucial to explore the quality of the content.
Here are listed some examples of KPI for content websites: average page view duration, number of visits
or page views, number of page views per visit, number of new vs. repeat visitors, number of subscriptions
(Newsletter, RSS, Forum, Chat), conversion rate from visitor to subscriber, conversion per advertisement.
Further KPI examples for online shops or social media websites will be given during the workshop.
3. Web usage mining and understanding of visitors behavior
Making use of collected information and extracting information related to user experience and usability
and establishing relationship between behavior and website usability are concerns of web usage mining.
User activity can be divided in to two ways of interaction, implicit and explicit. While during an explicit
interaction on the website, like filling a form or requesting a service user is aware of his action, implicit
interaction is happening usually subconscious. The user might think for a while before filling a form field,
because he might be uncertain about an answer.It is hard to distinct between implicit and explicit
interaction. However the experience shows that users do interaction without thinking much about the
small steps in the process. Nevertheless having an overview of implicit interaction helps to assess and
understand usage. E.g. speed, how users are typing, clicking through the service or pointing precision is an
indicator proficiency level of a user and frequent use of application. Time-based metrics such as average
time spent on a page, amount of time the mouse pointer hovers over elements, and time of usage of
elements shell be analyzed in deep with focus on detection of cognitive load while usage of element.
Actions with mouse input, like absolute mouse position and identifying the elements under the mouse
pointer can point on familiarity of with the task or other way around unfamiliar use. Intentional and
straight mouse movements are a sign of familiarity with the task. This information is not intentionally
given by user, operating an application user provides inevitably such parameters.
3.1 Web data mining
The reason why a user types in a certain field much slower than the others may be due to the fact that he
needs to think longer on this question, but on the other side it could also mean that the user withdrawn
from the process by a phone call. To sort out such cases and find clear dependencies it is important to
monitor the data over longer period of time. The aim in web usage mining is to discover and retrieve
useful and interesting patterns from a large dataset. Methods and tools will be shown during the workshop.
References
Arvind K. Sharma, P.C. Gupta, “Enhancing the Performance of the Website through Web Log Analysis and Improvement”,
International Journal of Computer Science and Technology (IJCST) Vol.3, Issue 4, Oct-Dec 2012.
Cooley, R., “Web Usage Mining: Discovery and Application of Interesting Patterns from Web data”, 2000,
http://citeseer.nj.nec.com/426030.html.
http://en.wikipedia.org/wiki/HTTP_cookie, Nov 2013