Product Interface Evaluation Based on task accomplishment
Discussion User Kansei State as Context of Use
Lei SHI*, Akira HARADA**
*University of Tsukuba Graduate School of Comprehensive Human Sciences 1-1-1 Tennodai Tsukuba-shi Ibaraki 305-8574
leishi1008@hotmail.com
** University of Tsukuba C/o Institute of Art & Design1-1-1 Tennodai Tsukuba-shi Ibaraki 305-8574
aharada@kansei.tsukuba.ac.jp
Abstract
With the wildly applying of integrated circuits and digital technology in consumer products, most consumer products
become more ease to be used whenever and wherever user would like to. Product functions are also brought into
greatly play by user. All these changes require product must to face more complex situations such as user’s state, task
condition and environment information, in a word, context of use. This paper shows that user’s Kansei states should be
also attached important as an aspect of context of use. In the paper we try to find a way to evaluate various factors
influenced task accomplishment. Further, we discussed that it is validity to carry out the evaluation on task
accomplishment to establish the relationship among task accomplishment, product and user’s mood, both operation and
kansei states.
We took digital camera as typical product in our research. Task scenario was designed and played to investigate
which states between user and product influenced the task accomplishment under the given task. We used picture
protocols to compare results of performed tasks with given goals. Questionnaire was set out, and I class quantification
was used to analyze the states that interfered the tasks accomplishment.
In the research we evaluated task accomplishment from objective of picture quality and subjective of user’
self-satisfaction. Through evaluation on task accomplishment and analysis, we discussed the interfering factors to task
accomplishment in user’s states, product’s states and kansei states. We concluded that kansei context should be taken as
important information for product interface as well as user, task and environment.
Keyword: context of use, kansei, task accomplishment,
1.
Introduction
1.1.
Background of Research
In the last decade years, context of use (CoU) analysis has played important roles in product interface
evaluation. Obviously these have made much progress on improving products usability. With the wildly applying
of integrated circuits and digital technology in consumer products, most consumer products become more ease to
be used whenever and wherever user would like to. Product functions are often been brought into greatly play by
user. All these changes require products must to face more complex user’s mood, task condition and environment
information. At this meaning, capability of answering various contexts of use positively becomes more important
for product.
It is more likely that the usability specification would be used to make product easier to use. ISO
9241-11(Guidance on usability) provided the definition of usability. Guidance is given on how to describe the
context of use of products and the measures of usability in an explicit way [1] [6]. It is described that Usability
Context Analysis (UCA) should deal with three important issues: Who are the intended users, what are they tasks
and what are the technical and environment constraints. Plenty research works concentrated on how to define the
context of use. An example of attributes gathered by UCA was given including user’s characteristics as well as
details of equipment and environment [2]. Obviously items, such as user skills and knowledge, personal attributes,
product basic description, and environment structure, etc., are considerably identified as context of use.
It is clear that user’s kansei state was seldom mentioned in former researches. According to research by
Harada, kansei exits beneath in everybody’s mental and behavior. It is interaction of intuition and intellectual
activities, and keeps reaction with external stimulus [3].
1.2.
Objective of the Research and method
Would kansei states influence interactions between user and product? In our research, we got the way to evident
that Kaisei states should be identified as a context of use, which cannot be ignored in evaluation of product
interface.
In the research, we aimed consumer product as the study object and took the practical example of digital
camera. If we classify goals of camera by shot targets, we can get two sorts of task contexts. One sort is taking
photos for still, such as landscape, still life. The other is for moving, such as human’s action, expression, or
moving object. Application of CCD (charge-coupled devices) has reduced skill requirements and made user much
easier to take nice photos in almost any kind of lighting conditions. CCD technology also makes photographic
process so freely [4]. Benefited from the revolution of the core technology, more and more persons like to use
camera to take memory in their daily life even including instant actions and emotions of anytime and anywhere.
All these changes broaden context of use, which include user kansei states facing different using conditions. We
can see this kind of changes based on technical innovation wildly appear in consumer products.
In our experiment we used evaluation on task accomplishment to check that if product interface is efficient in
using, and to find the reasons of gap among certain states and task accomplishment. First, we designed a specified
task named Picture on Instant Image. The moving images were shot by video beforehand and replayed to subjects.
In order to collect the related contexts under the condition of specified task, we design the task scenario in which
subjects were asked to finish the given task. Then they were required to answer a questionnaire including possible
conditions, which classified in user skill states, camera’s states (product technology) and user kaisei states.
I class quantification analysis was used to conclude the factors that interfered in the task accomplishment and
how much the correlation among task accomplishment and three context groups. The evaluation of task
accomplishment was one key point here. We evaluated it from two aspects of objective and subjective. In the
objective evaluation we used picture protocols to compare the taken pictures with the given target image and
marked it 1-10 levels, while in subjective evaluation we let the subjects to self-evaluate the pictures taken by
themselves and marked it 1-10 levels.
By using the example of digital camera, the paper shows that user kansei state should be identified as context of
use as well as user skill states, product states.
2.
Experiment method
In the experiment, we must design appropriate task for subjects, the task accomplishment should be recorded
and evaluated availably and correctly.
Figure 1, illustration of experiment scene.
2.1 Task scenario design
As mentioned above, we took one 40
Please answer the following question by “Yes” or “No”.
States
1. I was tensed up
seconds period video of one’s moving actions
2. The action of my finger was too slow to take pictures on time.
and expressions as target sample. Then we
3. I pushed the shutter button so hard that the camera shuttered.
4. I made some mistakes in operating.
marked two parts of instant images with red
dot in the video. 40 students of two groups
5. I could not get used to the camera.
User
skill
states
were invited to take part in experiment. Each
9. I have no habit to focus before taking picture.
10. I pushed the shutter-button intentionally early in order to take the pictures.
images with red dot by a designated camera
11. Pushed the shutter-button weakly.
12. I had no time to do focus.
while watching the video (figure 1, illustration
13. Camera reaction was too slowly to take the picture.
of experiment scene). In the video, before each
marked some images with green dot so that the
7. I could not understand the position of half-push the shutter button.
8. I forgot procedure.
subject was required to take the picture of
part of target image with red dot, we also
6. I could not understand the meaning of function buttons.
14. It was difficult to half-push the shutter button.
15. It was difficult to understand the function of focus.
Camera 16. It was difficult to watch from the finder window.
states
17. It was difficult to watch from the Liquid Display.
18. It was difficult to hold camera.
subjects could be aroused attention to make
19. The position of shutter-button isn’t good.
20. It was difficult to see if the focus was done.
some prepares (for example focus, get prepare
Among the following feelings,
Please answer by “Yes” or “No” for those you had when you check the pictures.
21. Joyful
to push the shutter button) for taking photos.
Each subject was asked to take two pictures of
22. A sense of fulfillment
designated images in the video.
23. Fun
24. Surprise
The process of each subject operating was
also recorded by video in order to analyze the
subjects’ action.
User
Kansei
states
25. Shameful
26. Disappointment
27. Regret
28. Dislike
29. Impression
30. Hope to try again
2.2 Questionnaire Design
31. Loss of self-confidence
Figure 2. Formal Questionnaire
There were two pieces of questionnaire in the experiment. As we said before, 40 subjects were divided into two
groups, each group with 20 persons. For the first group, after finished the task, we asked everybody some
questions on task processing by a simple questionnaire. Based on the subjects’ answers and video records of
process, we collected the related states and formed into a formal questionnaire table including 31 questions. The
questions were divided into three parts, such as user’s skill states, camera states and user’s Kansei states (Figure 2,
Formal Questionnaire).
For the second group, subjects were asked to finish the same task as the first group. After the task, they were
required to answer the formal questionnaire only by Yes or No. Additionally we asked each subject to make
self-evaluation on the two self-taken pictures.
3.
Data analysis method and discussion
3.1 Evaluation on task accomplishment
Evaluation on task accomplishment is the basic point in our analysis. Here we evaluated task accomplishment
from two aspects, objective evaluation and subjective evaluation by subject. We deem that user’s self-satisfaction
on task accomplishment was one standard to evaluate product, so we used method of user’s self-evaluation to
definite extent of self-satisfaction.
Objective evaluation is about quality of taken-pictures. We used picture protocols to compared taken pictures
with the target images in the video. Evaluation contents include how many time slots (Timing) between the taken
picture and target image with red dot, and if the picture was definitely taken (Definition). 10 levels from 0 to 10
were ruled on each evaluation contents. In addition to objective evaluation, each subject was asked to make
self-evaluation on self-taken pictures. 1-10 levels were ruled, and the selected level by subject was recorded as the
value of self-evaluate (Table 1, evaluation on task accomplishment).
In the experiment every subject was required to take two pictures on the target images marked with red dot.
Considering the first taken-picture is the training operation for the subjects, we just took the second taken-picture
of each subject as sample. Furthermore we think that, after taking first picture in the process of experiment,
neither succeeded nor failed, subject could be induced into certain kaisei states.
Subjects P01 P02 P03 P04 P05 P06 P07 P08 P09 P10 P11 P12 P13 P14 P15 P16
Objective
Timing (level 0-10) 7
4
2
9
9
6 10 7
5
0
9
6
3
8
5
9
evaluation Definition (level 0-10) 9
7
7 10 8
6
8
1
8
7
3
0
6
4
8
3
Self-evaluate (level 1-10)
6
1
1
6
8
3
7
6
6
5
4
4
7
4
3
4
P17
0
9
1
P18 P19 P20
8
9
9
7
5
7
7
8
6
Table 1, Evaluation on the second taken-pictures
From Table.1 we can see, just 5% subject (10 level of Timing) took the picture of target image, 30% subjects (9
level of timing) just took the picture near to target image. 5% subjects (10 level of Definition) took the definition
picture, and 10% subject (9 level of Definition) took the relatively clear picture. Just 10% subjects (8 level of
self-evaluate) satisfied on task accomplishment relatively.
The problem is what users took is not what they wanted, but why? Maybe somebody would say that it’s
because there is shutter time log in present digital cameras. But as we think, in addition to the time log, there must
be some more conditions that cannot meet the task requirement between user and camera.
3.2 Total of questionnaire data
Based on answers of the formal questionnaire, we made the data table (Table 2, Total table of formal
Questionnaire) by defining the answers of ‘Yes’ as 1, ‘No’ as 0. From the table 2, we can see that 100% subjects
deemed it was difficult to see if the focus was done.
State (YES/1, NO/0)
Camera states
14 15 16 17 18
0 1 1 1 1
0 0 0 1 0
1 1 0 0 0
0 1 1 0 0
0 1 0 0 0
0 1 0 0 0
0 1 1 1 0
1 1 0 0 0
1 1 1 1 1
1 1 0 1 0
0 0 0 0 0
0 0 1 1 0
1 0 0 1 0
0 1 0 0 0
0 0 0 0 0
0 0 0 0 0
1 1 0 0 1
1 1 1 1 0
1 1 0 1 0
1 0 1 1 1
User skill states
5 6 7 8
1 1 0 0
1 0 0 0
1 0 0 1
1 1 1 0
1 1 0 1
1 0 1 0
1 0 0 0
0 0 0 0
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
1 1 1 1
0 0 0 0
1 0 0 0
1 1 1 0
1 0 1 0
1 0 1 0
0 0 0 0
User kansei
Subjects 1 2 3 4
9 10 11 12 13
19 20 21 22 23 24 25 26
P01 0 1 0 0
0 0 0 0 1
0 1 0 0 0 0 0 0
P02 1 1 1 0
0 1 0 1 0
0 1 0 0 0 0 1 1
P03 0 1 1 1
1 0 0 1 1
0 1 0 0 0 0 0 0
P04 1 1 0 0
1 0 0 1 1
0 1 0 0 1 0 1 0
P05 0 1 0 0
1 1 0 1 0
0 1 0 0 1 0 1 0
P06 0 1 0 0
1 1 0 1 1
0 1 0 0 1 0 0 1
P07 0 1 0 0
1 0 1 1 0
1 1 0 0 1 1 0 1
P08 1 1 0 0
1 0 1 1 0
0 1 0 0 0 0 0 1
P09 0 1 0 0
1 1 0 1 1
1 1 0 0 1 0 1 1
P10 1 1 1 0
0 1 0 1 1
0 1 0 0 1 0 1 0
P11 1 1 0 0
0 0 0 1 1
0 1 0 0 1 1 1 1
P12 1 1 0 0
0 1 0 1 0
0 1 0 0 0 1 0 1
P13 1 1 1 0
1 0 0 0 1
0 1 0 0 0 0 0 0
P14 1 0 0 0
0 1 0 1 0
0 1 0 0 1 0 1 0
P15 0 0 0 0
0 1 0 0 1
0 1 0 0 0 0 0 0
P16 0 1 1 0
0 0 1 1 0
0 1 0 0 1 0 0 1
P17 1 1 0 1
1 0 0 1 1
0 1 1 0 1 0 0 0
P18 0 1 0 1
1 0 1 0 1
0 1 0 0 1 1 0 0
P19 1 1 0 0
0 1 1 0 0
0 1 1 0 1 0 0 0
P20 0 1 0 1
0 0 0 0 1
0 1 1 0 1 0 0 0
YES answering
Rates (%) 50 80 25 20 85 30 35 20 50 45 25 70 60 45 65 35 50 20 10 100 15 0 65 20 35 40
states
27 28 29 30 31
0 0 0 1 0
1 0 0 1 0
1 0 0 1 0
1 0 0 0 1
0 0 0 1 0
1 0 0 1 0
1 0 0 1 1
1 0 0 1 0
1 0 0 1 0
1 0 0 1 1
1 0 0 1 0
1 0 0 1 0
0 0 0 0 0
1 0 0 0 0
0 0 0 0 0
1 0 0 1 0
0 0 0 1 0
0 0 0 1 0
1 1 1 1 0
0 0 1 0 0
65
5
10 75 15
Table 2. Total table of formal Questionnaire
3.3 Date Analysis and discussion
In order to analyze the relationship between task accomplishment and various states, I class qualification analysis
was respectively used among 6 groups of data statistic, objective evaluation (value of Timing plus value of
Definition) and user skill states, objective evaluation and camera states, objective evaluation and user kansei states,
self-evaluation and user skill states, self-evaluation and camera states, self-evaluation and user Kansei states. 20
subjects of the second group were taken as samples in the analysis. We aligned each subject’s values of evaluate
(object evaluate or self-evaluate) on the vertical axis and various states on horizontal axis as items (Appendix,
Statistic table for I class qualification analysis). We got the results under list.
3.3.1 Relationship between objective value and user skill states
We analyzed relationship between pictures’ qualify and user sates based on the date of statistic table (Appendix).
The analytic result is Table 3.
Items (User skill states)
Range
1. I was tensed up.
3.02
6. I could not understand the meaning of function buttons.
2.96
3. I pushed the shutter button so hard that the camera shuttered.
2.60
4. I made some mistakes in operating.
1.75
Category scores
Y
-1.51
N
1.51
Y
2.07
N
-0.89
Y
-1.95
N
0.65
Y
-1.40
N
0.35
Multiple correlation of coefficient
0.75999298
Coefficient of determination
0.57758933
Table 3. Analytic result of objective evaluation / user skill states
According to the value of Multiple Correlation of Coefficient, user skill states greatly influenced qualify of
taken-pictures in the experiment [5]. Under the condition of instant task, subjects presented tension, hand shutter,
and couldn’t understand the operation method, even made some operation mistakes. All these skill states would
result in users’ failure of taking photos with time slot or un-definition.
3.3.2 Relationship between objective value and camera states
We analyzed relationship between pictures’ qualify and camera sates based on the date of statistic table
(Appendix). The analytic result is showed as Table 4.
Items (Camera states)
Range
16. It was difficult to watch from the finder window.
3.17
14. It was difficult to half-push the shutter button.
2.75
Category scores
Y
2.06
N
-1.11
Y
1.24
N
-1.51
Multiple correlation of coefficient
0.635506819
Coefficient of determination
0.403868917
Table 4. Analytic result of objective evaluation / camera states
According to the value of Multiple Correlation of Coefficient, pictures qualify were also bonded with camera
states [5]. We consider that some technical constraints cannot satisfy the task accomplishment. For example, focus
function and shutter function are integrated into one shutter button. We supposed that user usually couldn’t finish
two actions in instant time, either or, the half-push operation manner interfered in the instant of taking picture.
Besides that, according to the total date in Table 2, 100% subjects considered that it was difficult to see if the
focus was done, maybe it is not good way through liquid display or AF lighting to let users see if the focus was
done.
3.3.3 Relationship between objective value and user kansei states
The analytic result is showed as Table 5.
Items (User Kansei states)
Range
21. Joyful
6.75
29. Impression
5.96
27. Regret
5.58
28. Dislike
4.62
Category scores
Y
-5.74
N
1.01
Y
5.37
N
-0.59
Y
-1.95
N
3.63
Y
4.39
N
-0.23
Multiple correlation of coefficient
0.730897662
Coefficient of determination
0.534211392
Table 5. Analytic result of objective evaluation / user kansei states
We can see from the value of Multiple Correlation of Coefficient in the Table 5, user kansei states greatly
influenced value of objective evaluation in the experiment [5]. We supposed that negative emotions, such as regret,
unhappy or dislike, were helpful to task accomplishment. Deeply mental impressions were also helpful for
subjects here.
3.3.4 Relationship between users’ self-evaluation and user skill sates
The analytic result is showed as table 6.
Items (User skill states)
Range
12. I had no time to do focus.
3.23
4. I made some mistakes in operating.
2.48
11. Pushed the shutter-button weakly.
2.39
Category scores
Y
-0.96
N
2.26
Y
-1.98
N
0.50
Y
1.79
N
-0.60
Multiple correlation of coefficient
0.878234432
Coefficient of determination
0.7712957178
Table 6. Analytic result of self-evaluation / user skill states
See from the value of Multiple Correlation of Coefficient in the table 6, It is obviously that the value of
self-evaluation was strongly bonded with user skill states [5]. The states, such as no time to do focus, made
mistakes, were more easily to lead users into mood of un-satisfaction.
3.3.5 Relationship between users’ self-evaluation and camera sates
The analytic result is showed as Table 7. According to the table, value of self-evaluation was also bonded with
camera states [5].
Items (Camera states)
Range
16. It was difficult to watch from the finder window.
3.16
14. It was difficult to half-push the shutter button.
2.75
Category scores
Y
2.06
N
-1.10
Y
-1.51
N
1.24
Multiple correlation of coefficient
0.635506819
Coefficient of determination
0.403868917
Table 7. Analytic result of self-evaluation / camera states
3.3.6 Relationship between users’ self-evaluation and kansei sates
The analytic result is showed as Table 8.
Items (User kansei states)
Range
21. Joyful
6.03
28. Dislike
5.91
29. Impression
4.97
27. Regret
3.88
Category scores
Y
-5.12
N
0.91
Y
5.61
N
-0.30
Y
4.47
N
-0.50
Y
-1.36
N
2.52
Multiple correlation of coefficient
0.825249164
Coefficient of determination
0.681036183
Table 8. Analytic result of self-evaluation / user Kansei states
Seeing from the analytic result, we can know that user kansei state strongly influenced the value of
self-evaluation [5].
According to the Table 8, users’ self-satisfaction would come from the strong feelings during
task process. Either happiness or dislike, the stronger feeling subjects got, the more self-satisfaction subjects
would got from task process.
4.
Conclusion
In this research, we defined task accomplishment from two aspects of objective and user’s subjective, and we
analysis the relationships between the task accomplishment and various states under the condition of specified
task. From the experiment, we found that few persons succeed in the task. Task accomplishment is greatly
interfered by user kansei states as well as user skill states and product technical states. Comparing the values of
Multiple Correlation of Coefficient among the 6 groups date statistic, both user skill states and kansei states
appeared stronger influence on task accomplishment than product technical states in some extents.
We consider that analysis based on task accomplishment is an available method to find out items connecting
with product usability. Through the research we suggest that user kansei states should be settled into context of
use together with items of user, task and technology or environment. ISO 9241-11 described three important
issues that Usability Context Analysis (UCA) should deal with: Who are the intended users, what are they tasks
and what are the technical and environment constraints. Could we add an element as, how are users feeling with
operation processing.
This paper gave the simple evident of kansei states being context of use. The research is just an initial step in
the study of kansei context. How to attribute user kansei states and how user kansei states influence operation
process, these will be the next steps of our research.
References
1. User centered design standards, http://www.usability.serco.com/trump/resources/standards.htm, (2003)
2.
Stella Mills, The importance of task analysis in Usability Context Analysis-designing for fitness for purpose,
Journal of Behaviour & Information Technology, Vol.19, No.1, PP57-68 (2000)
3.
Harada Akira, The definition of kansei, Report of Modeling the Evaluation Structure of Kansei, 49-56 (1998)
4.
How digital camera works, http://www.howstuffworks.com/digital-camera.htm, (2003)
5.
Excel ni yoru Tyousa bunseki Nyuumon-Kikaku・Design no tame no Tool syuu, Kaibundo, 56-61 (1996)
6. Yamayasu.yamamoto, Usability Kougaku to kokusaikikaku, IPSL Magazine Vol.44 No.2 Feb.2003, 130-131
Appendix
Statistic table for I class qualification analysis
Items
Cammera states
subjects
objective evaluate
self-evaluate
Value
P01
P02
P03
P04
P05
P06
P07
P08
P09
P10
P11
P12
P13
P14
P15
P16
P17
P18
P19
P20
16
11
9
19
17
12
18
8
13
7
12
6
9
12
13
12
9
15
14
16
7
4
2
9
9
6
10
7
5
0
9
6
3
8
5
9
0
8
9
9
User skill states
1
2
3
4
5
6
7
8
9
User Kansei states
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
YNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYNYN
01 10 01 01 101 00 10 10 101 01 01 10 011 01 01 01 001 10 01 01 010 10 10 10 10 101 10 01
10 10 10 01 100 10 10 10 110 01 10 01 010 10 11 00 101 10 01 01 010 11 01 01 00 101 10 01
01 10 10 10 100 10 11 01 001 01 10 10 101 00 10 10 101 10 01 01 010 10 10 11 00 101 10 01
10 10 01 01 101 01 00 11 001 01 10 10 011 01 00 10 101 10 01 01 100 11 00 11 00 101 01 10
01 10 01 01 101 00 11 01 010 01 10 01 011 00 10 10 101 10 01 01 100 11 00 10 10 101 10 01
01 10 01 01 100 11 00 11 010 01 10 10 011 00 10 10 101 10 01 01 100 10 11 01 00 101 10 01
01 10 01 01 100 10 10 11 001 10 10 01 011 01 01 00 110 10 01 01 101 00 11 01 00 101 10 10
10 10 01 01 010 11 00 11 001 10 10 01 101 00 10 10 101 10 01 01 010 10 11 01 00 101 10 01
01 10 01 01 101 01 01 01 010 01 10 10 101 01 01 01 010 10 01 01 100 11 01 01 00 101 10 01
10 10 10 01 100 10 10 10 110 01 10 10 101 00 11 00 101 10 01 01 100 11 00 11 00 101 10 10
10 10 01 01 100 10 10 10 101 01 10 10 010 10 10 10 101 10 01 01 101 01 01 01 00 101 10 01
10 10 01 01 100 10 10 10 110 01 10 01 010 11 01 00 101 10 01 01 011 00 11 01 00 101 10 01
10 10 10 01 100 10 10 11 001 01 01 10 100 10 11 00 101 10 01 01 010 10 10 10 10 101 01 01
10 01 01 01 101 01 01 00 110 01 10 01 011 00 10 10 101 10 01 01 100 11 00 11 00 101 01 01
01 01 01 01 010 10 10 10 110 01 01 10 010 10 10 10 101 10 01 01 010 10 10 10 10 101 01 01
01 10 10 01 100 10 10 10 101 10 10 01 010 10 10 10 101 10 01 01 100 10 11 01 00 101 10 01
10 10 01 10 101 01 00 11 001 01 10 10 101 00 10 11 001 10 10 01 100 10 10 10 10 101 10 01
01 10 01 10 100 11 00 11 001 10 01 10 101 01 01 00 101 10 01 01 101 00 10 10 10 101 10 01
10 10 01 01 100 11 00 10 110 10 01 01 101 00 11 00 101 10 10 01 100 10 10 11 01 010 10 01
01 10 01 10 010 10 10 10 101 01 01 10 100 11 01 01 001 10 10 01 100 10 10 10 10 110 01 01