KP-LAB
Knowledge Practices Laboratory
Usability Expert Evaluation on SMAT
Due date of Report: xx/07/2008
Actual submission date: xx/08/2008
Start date of project:
1.2.2006
Duration: 60 Months
Integrated Project
Information Society Technologies
Organisation name of lead contractor for this report: EVTEK
Revision [1.0]
Product name and version:
Date of the test:
Date of the report:
Editor:
Contact name(s):
1
SMAT (Semantic Multimedia Annotation Tool)
10-07-2008
25.07.2008
Mary Nyamor
maryn@evtek.fi
________________________________________________________________________
Contents
Page
Executive Summary .................................................................................................................................................................3
Introduction .............................................................................................................................................................................3
Background Information ..........................................................................................................................................................3
Objective..................................................................................................................................................................................3
Methodology............................................................................................................................................................................4
Computing Environment..........................................................................................................................................................4
Results .....................................................................................................................................................................................4
Problems Encountered .............................................................................................................................................................4
Summary..................................................................................................................................................................................5
2
________________________________________________________________________
Executive Summary
The report is a heuristic evaluation that was done as a part of an Expert evaluation in EVTEK
University of Applied Sciences. The evaluations was executed by one of the experts
This report presents a summary of the expert evaluation and testing that was executed in KP-Lab
during the summer 2008. The report includes an evaluation of the Semantic Multimedia Annotation
tool (SMAT) which was carried out in EVTEK University of Applied Sciences.
The testing derived results and preliminary suggestions on improvements. The report concentrates
mainly on the usability problems.
The test user of the test was Mary Nyamor; a student in Department of Media Engineering, studying
at EVTEK University.
Before the expert evaluation she had been using Shared Space Application herself, therefore, she
had deeper knowledge on the purpose and possible domain usages of the application, as well as,
about the date of start of the release.
The document mainly points out the problems that have been encountered as well as the
functionality of the Semantic Multimedia Annotation tool.
Introduction
SMAT is a versatile tool for video annotation which allows a user to establish and perform its own
annotation scenario whereby a wide variety of scenarios are possible.
The current version of SMAT supports video streaming from the Adobe Flash Media Server (FMS).
Due to this, only video encoded within an FLV format can be streamed.
Background Information
In this particular test, there was one student that was a user of the system (Shared Space
The student used the following browsers:
 Mozilla Firefox
 Internet Explorer
Objective
The main objective of this test is to find out the functionalities of the semantic multimedia
annotation tool and have a visual look of the annotations as well as the general usability standards
The method used was expert evaluation according to Nielsen guidelines for conducting heuristic
evaluation. The heuristics that were used were the KP-Lab heuristics. Also the previous release
M12 Shared Space Application help-files were available for refreshing the memory of the
application and its purpose. The demo of the to-do list was available for testing
3
________________________________________________________________________
Methodology
The method that was used to carry out the test is through the use of a demo of the semantic
multimedia annotation tool (SMAT)
Computing Environment
URL of tested website:
http://kplab.enseeiht.fr:8081/SMAT/SMAT.html
Computer platforms:
Browser tested:
Screen resolution:
Operating system:
Connection speed:
(Intel) Dell Pentium 4 CPU , 1.00GB RAM
Mozilla Firefox 2.0.0.6
1024 X 768
Windows XP
100.0Mbps
Results
The results in the reports were formulated based on the notes the students took while executing the
evaluation. The results are presented according to the KP-Lab heuristic points.
Problems Encountered
Some of the problems that were encountered include the following
Visibility of system status
Problem
File uploads works fine but once the
upload is completed nothing really
happens. There is no video clip on
the sidebar therefore one cannot be
able to drag play the video
Help and documentation
Problem
The connect button remains inactive
throughout
4
Suggestions
It would be good to be
able to allow file upload
(performance) of the
system and at least
inform users of what is
going on through
appropriate feedback
within reasonable time
Image
Suggestions
Image
More information regarding the
button should be given. The placing
of a mouse over an inactive button
should give some description for the
button to enable users know what the
button is for
________________________________________________________________________
Match between system and the real world
Problem
Suggestions
Image
The new user button is meant to
Concepts should be
create a new user ;the user clicks the made familiar to the user
new user button, he or she should
so as to make the
therefore proceed to the input of data information appear
after which he or she can then press
natural and in logical
the login button
order
User control and freedom
Problem
This tool does not allow recovery
from unwanted states. For example,
when uploading a video. It is not
possible to stop the action by for
example pressing a cancel button.
When a user is waiting and thinks it
takes too long, or if he selected the
wrong file he or she should be able to
cancel the process and go on with
another task.
Suggestions
Having a way of being
able to revert or cancel
the process will users to
leave the unwanted state
without having to go
through an extended
dialogue. Support undo
and redo. or having to
wait for the process to
finish
Image
Summary



5
Login works fine
Ease in use
Consistency and use of colours
________________________________________________________________________