27490
KP-LAB
Knowledge Practices Laboratory
Integrated Project
Information Society Technologies
M 32 Tag Vocabulary editor
Revised version: 17.01.2009 (final)
Start date of project: 1.2.2006
Product name and version:
Organisers of the test:
Date of the test:
Date of the report: Editor:
Contact name(s):
Duration: 60 Months
M32 release of Tag Vocabulary editor
Metropolia
Early December and 10.01.2009
17.01.2009
Merja Bauters
Merja.bauters@metropolia.fi
Executive summary
Provide a brief level overview of the test (including purpose of test)
Name the product
M32 release of Tag Vocabulary editor
Purpose/objectives of the test
Field trial testing – testing in actual use
Method 1
Self-documenting
Number and type of participants
1
Tasks (if task are used)
Basic task: such as adding of vocabulary,
adding, deleting, changing and organising
tags
The present testing process aimed at examining of the usability of the core functionalities
1
of the Tag Vocabulary editor and how the integration with the tagging of content items
works, i.e., when adding tags in Tag Vocabulary editor do these appear as suggestion
when placing tags onto the content items and when adding (free) tags to content items are
these available for organising and managing them in the Tag Vocabulary editor. Two selfdocument periods where executed: 1) early December setting up a vocabulary for the
course of “Semiotiikan Metodologia” (Semiotic methodology) and 2) the tenth of January
to continue the editing of the previously made vocabulary.
Main results:
Positive:
 The main functions of the Tag Vocabulary work. These functions are:
o Adding, changing, deleting, and organising tags
o Writing descriptions and saving the added vocabulary
o The visual elements are clear, i.e., what to do and what they mean
o It also looks like an easy tool to use
o The tags added into the vocabulary are presented as suggestions when
tagging a content item,
o Hierarchical organisation of the tags in the Tag Vocabulary works
Problems:
 The user cannot write alternative tags
 The tags users defined (free tags) for the content items do not appear in the right
side of the Tag Vocabulary editor
 The added vocabulary do not appear items in the “select vocabulary” drop down
list of the form for tagging a content item
 When using the Tag Vocabulary editor it should Not make the background (the
use of the content view) inactive.
Other use problems:
 The loading of the Tag Vocabulary editor was somewhat slow
 Link descriptions were presented a suggestions when tagging a content item
 The OK-button in the form of tagging content items keeps being inactive until the
user changes something in the title or description
 The tags are not displayed in the info tab when selecting a content item active
 There are too many separate parts in the info tab, if there is a lot of information
they do not fit into one screen. Suggestion: use less arrows, i.e., more information
combined under one arrow (see
http://kplab.evtek.fi:8080/wiki/Wiki.jsp?page=M24GeneralImprovements)
Table of contents
Introduction Product Description ............................................................................... 3
Method Participants .................................................................................................. 3
Context of Product Use in the Test ........................................................................ 3
Results ........................................................................................................................ 4
References ...................................................................................................................... 6
2
Introduction
Product Description
VM32 release of Tag Vocabulary editor
The Tag Vocabulary Editor is an important tool for supporting users possibility to
form own conceptualisations of the concepts, knowledge objects etc., and organise
them according to the subject matter. The Tag Vocabulary editor allows the users to
insert new tags, modify and organise the tags hierarchically. The idea is it to be easy
and fast to use to avoid the common problem of current ontology editors that require
extensive knowledge on the ontology creation to be useful and usable. The Tag
Vocabulary editor is highly integrated allowing the tags of the content items to appear
in the Tag Vocabulary and the added tags in the Tag Vocabulary to appear as
suggestion when tagging the content items.
The basic functionalities of the Tag Vocabulary are the following:

Add/Modify vocabulary metadata

Add a new term

Modify an existing term

Delete a term

Publish the Vocabulary for use inside a shared space

Attach a note to a term (description metadata field of a term/concept)

Modify an existing term title
The testing process covered the basic functionalities of the tool as well as observing
the integration to the content item tagging procedure. It aimed at examine how these
functionalities work in individual use.
Method
Participants
One participant tested the Tag Vocabulary editor in a real life situation. The
participant was a teacher of the course “Semiotiikan metodologia”. The participant
had previous knowledge on the Tag Vocabulary editor. Therefore, the testing aimed at
finding out if the functions work and if it works in the manner that was envisioned.
The ease of use is somewhat speculative since the participant knew before hand how
the tool is supposed to work. The participant also has quite good technological skills
and can be said not be “scared” of just trying out things. The documentation followed
the self-documentation manner, e.g., the participant wrote down problems, irritations,
and positive points when they occurred in the use.
Context of Product Use in the Test
The above-mentioned functions were used since they reflect the core actions in Tag
Vocabulary editor and are also crucial from the perspective of trialogical approach for
supporting users conceptual framework creation. Each task was regarded as
successfully performed if its completion led to the supposed outcome visible to the
participant performing them.
3
The contextual setting was a course at Helsinki University on Semiotic methodology.
In the course it would be essential to learn to use different concepts, be able to
organise them easily so that it does not disturb the other assigned work on the course.
The tagging of content items and organising the concepts is a supporting activity in
the course. The idea was that the teacher would make a start by tagging some
material, notes etc., and from those terms form a common vocabulary, which the
students would then use and evolve further.
Participants’ Computing Environment
The computer use for the course activities by the participants was: Mac OS X, version
10.5.6, processor 2.4 Ghz Intel Core 2 Duo
The browser used for the activities was: Flock, Mozilla/5.0 (Macintosh; U; Intel Mac
OS X; en-US; rv:1.8.1.18) Gecko/20081107 Firefox/2.0.0.18 Flock/1.2.7
Procedure
The protocol followed the self-documentation manner (see Laitakari et al.2003). The
participant wrote down, and if necessary took screenshots of the problems
encountered as well as noted the positive aspects of the use when completing the tasks
what the participant would do to follow her course organisation. The participant also
placed preliminary suggestion for improve the tool.
Results
The following tables summarise the results of the self-documentation.
Period: Early December
Task/functionality
Add a vocabulary
Status of completion
Succeeded
Filling the form for new
vocabulary
Adding terms/tags
Succeeded
Organising terms/tags
Succeeded
Delete term/tag
Did not succeed
Save the vocabulary
Did not succeed
Succeeded
Observations
First looked at tools menu;
the Tag Vocabulary editor
was not there. Then rightclicked to the background
of the content view – Yes
hit.
Clear
Problems the place to add
the term did not erase the
guiding text, the user had
to write on top of text –
annoying (has been fixed
already)
Had to try couple of times
the system was a bit slow
in reacting to the user
inputs e.g., dragging of the
terms on top of each other
to make “sub-terms”.
The trash was still missing
(has been fixed already)
It seemed to be saved.
4
There was a statement that
the saving did succeed.
But the Tag Vocabulary
content item did not
appear in the content view,
which would have allowed
opening the just created
Tag Vocabulary.
Opening the Tag
Succeeded
It was possible to open the
Vocabulary again
Tag Vocabulary the same
way as adding a new one.
But it is misleading since
the user has to select “add
vocabulary” which s/he is
not doing when opening
the vocabulary s/he just
added.
Decision: Participant decided not to introduce the Tag Vocabulary editor yet to the
students, too many things were still not working properly. In addition, the tags did not
appear as suggestion when tagging content items and the already given tags of the
content items did not appear in the Tag Vocabulary in the right side (Free tag
vocabulary) for using them in the structuring of the added vocabulary as well as there
was no way of seeing the given tags of the content items anywhere, i.e., they did not
appear in the Info Tab.
Positive aspect: the drag and drop is nice and the visual look is pleasant, it does not
look like a hard tool to use. Can be seen to lower the thresh hold to use it.
Date: 10th of December 2009
Task/functionality Status of
completion
Open previously
Succeeded
added vocabulary
Managing
terms/tags
Succeeded
Writing description
to terms/tags
Writing an
alternative term/tag
Succeeded
Saving the changes
made
Succeeded
Changing metadata
of the vocabulary
Did not
succeed
No Success
Observations
After last time figuring out where to open the
vocabulary again, it was faster now, still
though annoying (the content item Type:
Vocabulary should be soon implemented).
No problem of adding more tags in the
vocabulary, deleting terms/tags and organising
them hierarchy.
No problem in that - clear.
The field to type the Alternative term/tag was
not active, i.e., the user could not write there
anything
There was a pop up stating that the saving
succeeded and the changes were visible after
closing the vocabulary and opening it again.
When opening the vocabulary again the fields
of the name and description of the vocabulary
are inactive. But when in the Tag Vocabulary
editor the user could activate the title of the
vocabulary and edit it, but the description that
5
Using the Free term
vocabulary (user
defined terms)
No success
Tagging Content
items with the tags
from the made
vocabulary
Success
Deleting tag/terms
Succeeded
from Vocabulary –
the term does not
appear as suggestion
Selecting a
No success
vocabulary when
tagging a content
item
Tagging the content Succeeded
item
was given to the vocabulary was not in the
description text field. Thus there is no way of
editing the description after the vocabulary has
been created.
It was not possible to use the tags/terms that
were placed to the content items using the free
tag vocabulary. These tags/terms do not
appear on the right side of the Tag Vocabulary
editor as they should for adding them to the
tag hierarchy of the made vocabulary.
Thus, the integration seems not to work
properly.
The tags added in the Tag Vocabulary editor
do appear as suggestions when tagging a
content item. However, it also provides the
link descriptions as suggestions, which should
not come for the tagging of content items.
The tags/terms that are deleted from the
vocabulary are also eliminated for the
suggestion list when tagging a content item.
If the user wants to select the vocabulary s/he
made, it does not appear in the form’s drop
down list for selecting what vocabulary to use
for the tagging.
But the tags are not visible anywhere, e.g.,
they do not appear in the Info tab. Also it is
nor possible to see the previously added tags
of the content item the user is tagging in the
field for tags (see suggestion for
improvements
http://kplab.evtek.fi:8080/wiki/Wiki.jsp?page=
M24ImprovementsOfTagging)
General comments: The loading of the Tag Vocabulary editor is sometimes somewhat
slow. It would be good to provide a feedback for the user that the system is active and
that the Tag Vocabulary editor is being loaded;
The OK button still keeps inactive in the form where the tagging of the content items
occurs unless the user changes something on the title, or description;
When adding and managing the tags in the Tag Vocabulary editor the background
should not become dimmed, it would enhance the use of the Tag Vocabulary editor if
the user could in the same time tag the content items and work in the content view. In
similar manner that the peer-to-peer chat works; There are too many separate parts in
the info tab, if there is a lot of information they do not fit into one screen. Suggestion:
use less arrows, i.e., more information combined under one arrow (see
http://kplab.evtek.fi:8080/wiki/Wiki.jsp?page=M24GeneralImprovements)
Validity – reliability
Validity of the test can be seen to be good. Since the functionalities and their expected
outcomes where know, going through them and seeing if they work or not, measures
6
well success of the functionalities. The self-documentation manner is well suited to
take into account the feelings and ease of use.
Reliability can be said to be good in respect to the testing of the functionalities, since
these were basic functionalities and it is clearly visible if they work or not. Anyone
who has been provided the list of the functions and their expected outcome could do
the test. However, in expressing the feelings or ease of use remains subjective and can
be generalised only to similar users than the tester and even then with caution.
References
Huotari, Petteri, Laitakari-Svärd, Ira, Laakko, Johanna & Koskinen, Ilpo. 2003.
Käyttäjäkeskeinen tuotesuunnittelu: Käyttäjätiedon keruu, mallintaminen ja arviointi.
Taideteollisen korkeakoulun julkaisu B 74. Helsinki: Taideteollinen korkeakoulu.
7