991219
http://www.ul.ie/~infopolis/methods/index.html
Ergonomics Methods and Tools
This section is intended for practitioneers such as designers, engineers, project managers and
ergonomics practitioneers in the public transport domain - with or without previous task analysis
experience - who need to carry out a traveller task analysis.
It includes and explains several methods of task analysis with an ergonomic point of view. It shows
how and when various task analysis, task description and task simulation methods can be used.
Task Analysis Methods
Task analysis is the study of identifying how users do certain tasks or operate different
products. This is usually done before a product is designed, or when a product needs to be
redesigned based on usability testing results. Task analysis is an important aspect of user
interface design, ensuring that the end product is usable and practical.
Techniques used to collect data on actual or proposed task performance:
[Critical Incident Technique] [Group Techniques] [Interview Techniques] [Observational
Techniques] [Questioning Techniques] [ Verbal Protocols]
Task Description Methods
Task Description Methods are techniques which structure the information collected into a
systematic format (e.g. the data from observation studies may be recorded by the use of flow
charts, or operational sequence diagrams). Such formats may then serve either as reference
material to enhance the understanding of the human-system involvement, or may be used
more directly (e.g. the hierarchical task analysis technique can be used to identify user
needs)
Representing such data in a pre-specified format:
[Charting and Network Techniques] [Hierarchical Task Analysis] [Task Decomposition
Techniques]
Task Simulation Methods
In addition to classical data task collection methods, potential users can be placed in mockup or prototype versions of the system that simulate the proposed equipment and 'walk
through' the planned task sequences.
[Walkthrough and Talkthrough Methods] [Mock-ups and Simulators]
Selecting a Method to Evaluate a User Interface
When deciding on the method (or methods) to use it is important to consider:





The exact purpose of the evaluation;
The external limitation imposed on the evaluation process, such as the time constraints of
the development cycle, cost and availability of equipment or expertise;
Biases in the data collected: how much the evaluation affects the situation being evaluated
(known as the ecological validity), the reliability of the methods and so on;
Practical questions such as: Are the data easy to gather? How long will it take? How much
will it cost? Are appropriate resources available?
How well particular methods can be assimilated with industrial and commercial practices
(for example, expert evaluation and interview methods can be integrated easily whereas
observational techniques are more difficult).
The features of the main evaluation methods are summarized in a table : Which method to use and
when ? (adapted from Preece et al, 1994).
Home Page I Menu I Site Map I Contact I About the Project
Project Scope I Overview of Existing Systems I Traveller Needs I Ergonomics Methods I Mock-ups I
Guidelines
Search I Library I Bibliography I Links
http://www.ul.ie/~infopolis/methods/index.html
© Infopolis 2 Consortium
User Interface Evaluation : Which Method
to Use and When ?
Method
Observational
Evaluation
Experiments
Interviews
Contextual
Inquiry
Heuristic
Evaluation /
Cognitive
Walkthroughs
Purpose
Understanding
the real world,
Comparing
designs and
engineering
towards a
target
Standards
conformance,
comparing
designs,
benchmarking
Users’
opinion,
requirements
identification
Understanding
natural usage,
requirements
identification
Engineering to
a target,
conformance
testing
Interface
development
Any stage of
development
At least a
working
prototype
Any level of
development
Any level of
development
Early at
specification or
mock-up stage
User
involvement
Yes, some
control of tasks
by users
Yes, little or
no control
Yes, often no
control
Yes,
considerable
control
No
Type of data
Quantitative
and qualitative
Emphasis on
quantitative
Quantitative
and
qualitative,
but more
emphasis on
quantitative
Qualitative
Some
qualitative but
emphasis on
quantitative
Practical
considerations
Special
equipment
useful but not
essential
Laboratory
conditions
preferred
None
Little or no
equipment
needed.
Video/audio
may be used
No equipment
needed
Some of the Key Differences Between Laboratory
Conditions and Normal (Work) Environments




Work - For example, typical word processing benchmark tasks are six pages long,
whereas, in practice, working with documents 30 to 50 pages long is not unusual;
Time - Experiments generally have a prescribed time context, whereas in work
environments people tend to have more influence on what they do and when they do it;
Motivation - In an experimental context the experimenter controls the situation, whereas
usually in the work context there is scope for some negotiation;
Social factors - In the work environment there is normally a social network of support that
does not exist in an experimental context.
Task Analysis Methods: Critical Incident
Technique
Overview
The Critical Incident Technique consists in collecting information on incidents occurring over a
specific period of time, for instance a trip, in order to make corrections and prevent accidents.
These incidents are events or features which could have an effect on system objectives. It is a way
of identifying events that may have a positive or a negative influence on the system. The first
principle of the technique is that critical incidents will be memorable to those using a system.
Taking an inventory of incidents, these can be classified and analysed. It should then be possible to
identify what changes can be made in the system to ameliorate problems, or to ensure positive
components of system success.
Application
Generally, the critical incident technique is most useful in the early stages of a larger task or activity
analysis, because its strength lies in the correctly rapid separation of problem areas within a
system. It is also most interesting that the critical incident technique is applied to existing systems.
The technique can be seen as a very open ended one, where operators are simply asked to
respond to questions. A much more systematic approach of interviewing operators on a daily basis,
or getting them to fill in a weekly or monthly report on critical incidents can also be used (see also
Deliverable 3 : "Needs of travellers: an analysis based on the study of their tasks and activities" Infopolis 2 project; see page Library of this web site to download this document).
Advantages
The critical incident technique is inexpensive and provides rich information.
This technique is helpful in emphasising the features that will make a system particularly
vulnerable.
The technique is especially appropriate for the identification of unusual events, which may
not be picked up in other techniques that rely on users reporting their everyday activities.
It may also have a particular interest to system users who are interested in reporting on
near-misses, etc.
Disadvantages
A first problem comes from the type of the reported incidents. The critical incident
technique will rely on events being remembered by users and will also require the accurate
and truthful reporting of them.
There is also the problem of reliability in terms of memory, because many critical incidents
may be forgotten or distorted by events.
The last point to highlight is that the technique will emphasise only rare events, and more
commonplace events will be missed.
Type of Critical Incident Technique
Micro-incident analysis
This method aims at analysing how complex systems can recover small errors but also how global
reliability is obtained (this objective is analysed deeply in critical safety situations, in which complex
systems play a major part and where it is important to ensure their reliability such as Air Traffic
control or Urgency Medical Centre).
Task Analysis Methods: Group Techniques
Overview
A potential weakness of methods such as interviews or questionnaires is that they are based on the
activities and views of individuals on their own rather than as part of a wider social-technical
situation. Also group techniques are useful for getting opinions efficiently from a set of people. The
main principle is that each participant can act to stimulate ideas in the other people present, by a
process of discussion. It is well-known that the collective view is greater than the individual
contributions (see also Deliverable 3 : "Needs of travellers: an analysis based on the study of their
tasks and activities" - Infopolis 2 project. Not published yet)
Application
There are various means of managing group sessions. The simplest is to have the group meet and
discuss all aspects of the problems being addressed. Alternative approach are, for instance, Focus
groups, Philip 66, Brainstorming, … In order to encourage discussions, the group members should
be assured that their contributions will remain confidential. If any recording is made, members
should also be assured that this will remain confidential. In the domain of information systems
groups may be of use in understanding social issues affecting the system and the design and the
evaluation of possible design solutions.
Advantages
Groups techniques allow the rapid collection of a wide variety of views from a range of
people.
Group discussion helps to summarise the ideas and information held by individual
participants.
Meetings can be recorded for future analysis.
Disadvantages
Some shy participants may be inhibited by other group members.
One member of the group may appear highly confident and persuasive and may influence
the opinions of the other group participants.
Some people may also not think creatively in a group setting and prefer to be interviewed
or to complete a questionnaire.
Types of Group Techniques
Focus group
A focus group brings together stakeholders and / or experts to help refine the requirements of a
system or to evaluate a particular system. Views are elicited by a facilitator on relevant topics. A
focus group is useful at the early stage of the system development: it can assist identification of
problems for a target users population, refinement of requirements for systems, and assessment of
the early prototype. It helps to identify issues which may need to be tackled and provides a multifaced perspective on them (see also Deliverable 3 : "Needs of travellers: an analysis based on the
study of their tasks and activities" - Infopolis 2 project. Not published yet).
Group discussion
Group discussions are based on the idea of stakeholders within the design process discussing new
ideas when relevant to the design process. Group discussions help to summarise the ideas and
information held by individual members.
Brainstorming
Brainstorming, developed by Osborn, is probably the oldest and best known group method. This
technique has been designed to help creative thinking in initial system development. The principle
is to bring together a range of people to inspire each other in the creative, idea-generation phase of
the problem-solving process. The members of the brainstorming group should have a range of
experience but should consist not only of experts. The advantage of this technique is that it is a
rewarding experience for the participant, and it is a quick and easy method to apply. The
disadvantage is that user's expectations of the new system's capabilities may be heightened to
unrealistic levels and the users may therefore be disappointed later.
Task Analysis Methods: Interview
Techniques
Overview
The interview technique is a systematic collection of verbal information. It consists in asking about
users opinions and attitudes to get basic information with prepared questions asked by the
interviewer. The answers are either written or recorded. The interviews can be structured or
unstructured. The terms structured interviews implies that the content of the interview, in terms of
the questions and their sequence, is predefined. Because of the structuring the interview offers the
opportunity for more systematic collection data. The unstructured interview is more free, and the
interviewee develops the themes proposed by the interviewer
Application
Interviews can be used at any stage during a comprehensive task analysis activity. They can
usefully be applied early on in an investigation for collecting basic information about the situation.
An interview with the user will provide a deeper understanding of the requirements for the system
and of the user tasks involved. It can also assist in finding out more about individual user
perceptions (see also Deliverable 3 : "Needs of travellers: an analysis based on the study of their
tasks and activities" - Infopolis 2 project. Not published yet).
In order to encourage frank answers, the participant should be assured that his contribution will
remain confidential.
If any recording is made, the interviewer should also assure that this will remain confidential.
Newman and Lamming (1995) describe a number of elements of a successful interview:
Determine some basic domain knowledge before the interview so that time is efficiently utilized
during the interview.
Clearly state to the interviewee the purpose of the interview at the outset.
Enumerate all user activities with general and follow-up specific questions.
Find out how user activities are performed.
Trace interconnections with other users.
Uncover issues that determine and affect the performance of the user tasks.
Follow up on exceptions, the unusual activities which are unlikely to occur during
observation.
Advantages
Interview techniques are useful for identifying possible areas for more detailed analysis.
Interviews are easy to conduct and direct, the unstructured interview can generate
interesting points, statistical analysis can be run on the users answers.
The data collected provides information about general rules and principles and is faster
than observational techniques.
Interview techniques are useful for investigating events which occur infrequently.
The interviews can be recorded for a future analysis.
Disadvantages
Respondents are not committed to give correct answers and may often be influenced by
what they believe the interviewer requires, or what they themselves wish to portray.
The interviewer may need to acquire domain knowledge in order to know what questions to
ask.
There is a range of considerable bias due to the understanding by the users of the
questions, and the subjective collected information might be misleading or inaccurate.
The critical aspects are the choice of the place for the interview and how to conduct it.
Types of Interview Techniques
Unstructured interview or depth interview
In an unstructured interview, the respondent must develop the theme proposed by the
interviewer. The investigator does not impose a questionnaire and must follow the thoughts and
reactions of the respondent. Unstructured interviews are good for investigating potential emotional
and / or sensitive personal issues.
Semi-structured interview
Semi-structured interviews should only be carried out in a situation where broad issues may be
understood, but the range of respondents' reactions to these issues is not known or suspected to be
incomplete. This type of interview is mostly applicable in situations where both qualitative and
quantitative feedback are required.
Structured interview
Structured interviews are useful in situations where the respondents range of replies may be
estimated and there is a need to clarify details, opinions or ideas. Structured interviews work well
when the assessment goals are clear.
Task Analysis Methods: Observational
Techniques
Overview
Observational techniques are a general class of techniques whose objectives are to obtain data by
directly observing the activity or behaviour under study. A wide range of observational techniques
are commonly used and combined according to the particular requirements of a study.
Observations may be direct observations, where the investigator is actually present during the task
(visual observations, remote observations via closed-circuit television), or indirect observations,
where the task is viewed by some other means such as video recording or time-lapse photography.
Application
Perhaps the most obvious method by which the user can be analysed is to observe him or her
performing the task in an operational situation. Although observational techniques can capture the
details of many individual tasks, they may not elicit the work role. Thus, observational techniques
are most appropriate when the information of primary interest is of visual or auditory form. Thus,
observation methods can be especially useful for recording physical task sequence, or verbal
interactions between several people. One method which can augment the information gathered is
that of current verbal accounts. Here the user is asked to 'think aloud' and describe the activities he
or she is performing as they are undertaken.
Advantages
Observational techniques produce information which cannot be collected in any other way.
Indeed, detailed physical task performance, social interactions, surrounding influences can
be recorded.
Observation methods allow the observer to become more familiar with the task. So they
are very appropriate for exploratory studies because they allow the investigator to decide
what to look for.
Observation studies can be used to identify and develop explanations of individual
differences in task performance.
Observational techniques provide data which can be compared and contrasted with
information collected by another investigator, or by another method.
Disadvantages
Real time observation requires trained ergonomists.
Observational data is the widest possible form of information, for this reason the effort
which must be expended on classification and analysis is considerable.
Direct observation can be an intrusive technique. The user behaviour might be influenced
by the presence of the observer,. Therefore the investigator must estimate whether the type
of observation selected will significantly alter the task under study.
Observational situations which produce rich context data are rarely the ones which
produce precise and controlled data. So data will be incomplete, inconsistent and sometimes
unusable
Observational techniques cannot provide information about underlying thought-processes,
and so they will be of very little use for highly cognitive activities.
The equipment needed for producing high quality observational data can be expensive and
difficult to install, and video recording followed by later analysis is time-consuming.
Type of Observational Technique
Passive observation
At the simplest level passive observation, where the person conducting the observation
quietly observes and records the user activity, can be used to record some details of the
processes which the user undertakes. Passive observation has the advantage that it
minimises the intrusion which the user experiences. One problem with passive observation is
the practical difficulty in recording sufficient detail of the activity. This can be overcome
through the video recording of user activity which, after transcription into textual form, can
provide a highly detailed analysis.
Participant observation
The investigator takes part in the tasks alongside other operators. It may be a useful data
collection approach if skilled performance is such that actions are "semi-automatic" in which
case users may find it difficult to verbalise how they are achieving the task goals. This type
of observation can also be useful if aspects of team performance are being investigated to
understand how the team members are organised and perform their tasks.
Communication and conversational analysis
Investigation of the patterns in the organisation of people's interaction (it would be possible
to use the principle of analysing a conversation between two travellers, or one traveller and
one transport enquiry office operator in real situations, in order to identify the needs for the
traveller in specific contexts, what are the difficulties encountered, how a system could
resolve these difficulties,... ). One of these methods is the language / action approach which
considers the language as a means by which people act. Advantage of this method : it
provides a complete and logical conceptual frame in order to investigate all kinds of
conversation. Disadvantage of this method : many situations are characterised by subtle
communication processes not taken into account by the model. There is also a difficulty in
labelling an interaction and a message especially if they do not fit into the request or promise
categories.
Task Analysis Methods: Questioning
Techniques
Overview
Questionnaires are sets of predefined questions arranged on a form and typically answered in a
fixed sequence. The questionnaire is usually completed by the individual under study, but it is also
possible for the questionnaire to be administered by a second party. There are essentially two types
of questions : closed questions (respondents are asked to select an answer from a choice of
alternatives) or open questions (the respondent is free to provide his own answer).
Application
Questionnaires can be used in a very broad series of applications. They can be used for evaluating
specific features or issues of a system, and they can also be used in order to investigate different
opinions, perceptions, knowledge or attitudes towards this system. However, designing a
questionnaire is a much more complex task than it might appear. It is important to make the
questionnaire easy for the user to complete. There are two quite obvious reasons for this. First in
order for the analysis to be statistically significant it is necessary to generate a high return rate. If
users find difficulty in completing the questionnaire they might not complete it, and only a relatively
small proportion will be returned. The second reason is that it is important to avoid ambiguity in the
questions which are posed. Ambiguous questions generate unreliable answers. For both these
reason, it is usual to trial the survey instrument with a small proportion of the intended population so
that any design problems can be addressed before the main survey is carried out.
Advantages
The main advantages are the area and the flexibility of administration. They can be filled in
at any time by any number of individuals. Because it is relatively easy to administer a large
number of questionnaires, they are especially useful for comparing the opinions or
understanding of a large number of users.
The use of closed questions can prevent digressions.
The use of multiple choice items and rating scales can produce answers that can be
weighted for their reliability and consistency.
Careful decisions have to be made concerning the sample of user population and whether
they are representative of the whole group of interest.
Another advantage of this technique is to gather a large amount of information from many
people, relatively rapidly and at low cost.
Disadvantages
Respondents do not necessarily give correct answers and may often be influenced by what
they believe the analyst requires, or what they themselves wish to portray, especially for
personal information. Anonymity and having the questionnaire completed remotely from the
analyst can reduce distortions.
Loss of information or subjective insights may be introduced by inappropriate perceptions
of what the question is asking. The correct choice of target population minimizes this risk.
There is room for considerable bias in what questions are asked and how the answers are
interpreted, mainly if open-ended questions are incorporated.
Response rates for posted questionnaires can be very low (30 % or less is considered as
normal).
Types of Questioning Techniques
Multiple Choice Items
Multiple Choice items provide two or more specific responses from which respondents must
choose an item which is most representative of their opinions, perceptions, knowledge or
attitudes.
Rating Scales
Rating scales can be used to obtain subjective information from respondents which gives an
indication of both the nature and magnitude of their opinions about certain aspect of a task.
Bipolar alternatives
Bipolar alternatives is a type of rating scale, where respondents must make a choice
between two alternatives. Ranking Ranking requires respondents to order several items
according to precise criteria. Open-ended questions The respondents are free to write their
own comments, answer or opinions on the questions
Task Analysis Methods: Verbal Protocols
Overview
Verbal protocols are the recorded verbalisations of users as they carry out their tasks. This
technique uses users' spoken comments, who verbalise how they use the system, explaining what
they are trying to do and the type of problems they experience. Verbal protocols may be collected
by video or simply audio tape recorder, direct notes can be also used. These self-commentaries are
made while the task is being carried out in order to avoid the inevitable distortion or forgetting that
could occur if the reporting were left until afterwards. These protocols are particularly useful for
gaining information about the psychological or cognitive process for users' actions, which cannot be
directly observed, such as the knowledge requirements or mental processing. However, the think
aloud protocol, during the performance of the task, can impose an additional task-load on the user,
significant when the task is difficult. This load can alter the way the user performs the task. The
post-event protocol is based upon the user's memory after running the task. The self-confrontation
is a protocol where users are asked to comment on their actions in front of the video recording after
the system use.
Application
It is important that verbalisations should not interfere in any way with task performance and that
users should freely report on what they are doing without any direction. Also, verbal protocols are
limited by the ability of subjects to freely and honestly state why they are undertaking particular
actions, without first making a conscious effort to explain these actions. When to use : To identify
the likely errors that users make while learning how to use the system. To identify the expectations
and preconceptions that users may bring to an interaction, revealing the internal model developed
by the user concerning the way the system operates. This technique is useful for activity analysis of
existing tasks, for system evaluation and for teasing out any misunderstanding or misconceptions a
user has when operating a system for the first time.
Advantages
The main advantage is that it helps to find out what the user is thinking when performing a
task or operating a prototype system.
Verbal protocols provide a basis for investigating the underlying mental process of complex
tasks which cannot be studied in other way.
Data collection is rapid, because very few special arrangements need to be on-site, and
data analysis will usually be conducted off-site.
The results of applying the technique will be credible and comprehensible to non-specialist
observers or users.
Disadvantages
Verbalisations may be biased because they interfere with the basic task, for instance,
changing the speed or the execution method. This is mainly the case if the activity
necessitates the use of short term memory.
Verbal protocols may reflect only the results of cognitive process, rather than the process
itself. Thus the depth of knowledge which can be obtained is often limited, and it may be
necessary to do additional investigations.
Language may be an inadequate descriptive technique for the process being studied.
Many mental processes depend on visual, auditory, or mathematical imagery for their
execution. Therefore it may be difficult to verbalise such events.
Another disadvantage is that it is timeconsuming to analyse audio and video tapes
afterwards.
Types of verbal protocol
Personalised Task Representation (PTR)
This method is used to investigate user procedures that are not well documented. A
standard set of questions can be used to probe activities in a flexible manner : discussions
with the task expert are about the activity they perform, the reasons for each task action, and
how the action is carried out. · Advantages of this method : it can be applied to a range of
knowledge and decision based processes. · Disadvantages of this method : the data
captured can produce complex data diagrams that will need to be structured afterwards into
a logical form.
Task Description Methods: Charting and
Network Techniques
Overview
Charting and network techniques are graphic representations produced from the analysis of a
system which can be used to describe tasks within a system. The principal aim of charting and
network techniques is to use a formal graphical representation of the task which is more easier to
understand than a textual description.
Application
These techniques provide a systematic approach to describe tasks and offer a graphical
description, which allows an adaptable means of analysis. The techniques have a range of
applications because the techniques try to represent interaction between human and machine or
system. These techniques can also be useful for describing the action plans of a hierarchical task
analysis.
Advantages
Charting and network techniques are able to offer a very clear representation of the tasks
(tasks into clear units and diagrams which link units) and provide information in a format that
can be directly used for making decision, producing requirements or creating operational
procedure.
The charts can be used to represent human activities, tasks and equipment in a common
format.
Flow charts and network diagrams can incorporate the representation of the flow of time, in
order to describe the conditions under which a task is performed.
Disadvantages
Charting and network techniques become less satisfactory if cognitive content is increased.
Only a limited amount of information can be represented in one diagram, too much
information lead to complicated charts.
Types of charting and network techniques
Input-output diagrams
This technique, developed by Singleton, provides a representation of a task as an "inputoutput diagram". The technique involves a first selection of tasks or steps, and then
identifying all inputs and outputs which are necessary to execute these tasks or steps. Inputoutput diagrams are useful for understanding the role of the user in a system, showing what
information and actions are necessary to realised the task.
Process charts
Process charts are a 'top-down flow line' which links a sequence of activities or events.
Moreover, information about time taken or distances moved can be shown on the process
chart. This technique is mainly used for individual task.
Functional flow diagrams
Functional flow diagrams are block diagrams which illustrate the relationships between
different functions. Usually, these functions are expressed by a "noun-verb-number"
combination, in order to identify each function and the sequences within the process.
Decision-Action-Information diagrams
The Decision-Action-Information diagrams, also called information flow charts, depict the
process through a system in terms of the decisions and actions which must be performed.
Decision blocks are usually represented as diamonds and the possible solutions are labelled
on the exit lines and blocks.
Scenario analysis
Another method for representing data in a pre-specified format is the scenario analysis. This
method allows the identification of classes of typical scenarios (like road accidents
scenarios). The analysis can be made from archive documents and interviews. The
advantages of this method are: introduction of cognitive factors in the model in order to
understand incident genesis, all stakeholders participation in the accident process are
represented in the scenario, strategies of risk management and environment characteristics
are represented.
Task Description Methods: Hierarchical
Task Analysis
Overview
Hierarchical task analysis, also called HTA, is a broad task analysis method, which describes the
task in terms of a hierarchy of operations and plans based on structure chart notation. The
hierarchical task analysis prompts the analyst to establish the conditions when various sub-tasks
should be carried out in order to meet a system's goals. This method produces a hierarchy of three
levels of task analysis: Goals (external task): system state that the human wishes to achieve,
Tasks: structured set of activities in some sequence to achieve goals, Operations or actions:
different things that a person must do within a system; simple tasks having no control structure.
Application
Hierarchical task analysis can be used to deal with specific issues, such as interface design or work
organization in order to help the designer articulate how tasks should be executed. Hierarchical task
analysis is a useful framework in which other task analysis techniques can be applied effectively.
The main characteristics of the HTA :
The goal decomposition is based on a person's verbalisation (interview, thinking aloud
methods) and thus on his mental representation of the activity.
Each sub-goal and activity may be described by a verb and a name
Each sub-goal or activity has a number
The plan at each level of decomposition and on each branch of the tree indicates how the
activities are done (sequential, parallel, simultaneous) and their characteristics (optional,
conditional, loop).
Each plan has a number.
The decomposition stops when it is not possible to break down the cognitive activity any
more by the person, or when the analyst considers that more decomposition would not be
useful for the application.
Advantages
This method is easy to learn and to use.
The hierarchical structure of this task analysis approach allows the analyst to concentrate
on crucial aspects of the task within the context of the overall task. Also other specific
techniques of task analysis may be applied.
This method is best developed as a collaboration between the task analyst and user
involved in operations. Thus the analyst should operate in accordance with the perceived
needs of people who are users of the system. · This method can form the basis of many
other assessments, such as the communications analysis.
The hierarchical task analysis is commonly used and widely accepted in cognitive task
analysis.
The HTA is very powerful because it can be applied to different types of physical and
mental activities and different domains of applications.
Disadvantages
The hierarchical task analysis requires a lot of time, skill, and effort to use.
The method must be carried out with the collaboration of managers, engineers and
operating staff, and this collaboration involves agreement, time and effort from a lot of
people.
Errors and " unforeseens ", inevitable in the performance of a task, invalidate a part of the
plans.
It is difficult to represent in the plan goals which apply to every activity, interrupted activities
or 'ad hoc' activities
The hierarchical task analysis applies only to procedural activities and not to heavily
parallel activities.
Task Description Methods: Task
Decomposition Techniques
Overview
The task decomposition is a structured way of developing the information from a task description
into a series of more detailed declarations about particular issues. It starts from a set of tasks
descriptions which describe how each element within a particular task is carried out. Next, further
information is collected about each task element, for example, displays used or time taken. Then
this information is presented using a sub-heading, so that the total information for each step is
boken down into a series of statements. Decomposition is aimed at determining the control and
information requirements of each step so that it is possible to ensure that the necessary interfaces
for a task will be provided.
Application
The task decomposition is an information collection technique which is used to systematically
expand upon the basic description of the activities which must be performed in each task element.
The task decomposition includes three steps: obtain a task description, choice of decomposition
categories, collection and presentation of decomposition information. This method is particularly
useful as a basis for the preparation of control or information requirements lists (see also
Deliverable 3I : "Needs of travellers: an analysis based on the study of their tasks and activities" Infopolis 2 project ).
Description of the Technique
There are three steps which must be undertaken in any task decomposition: obtain a task
description, choice of decomposition categories, collection and presentation of decomposition
information. Obtain a task description The starting point for a task decomposition is a set of clear
task descriptions for all the task elements which are associated with a particular task. Normally,
these descriptions would be derived from a hierarchical task analysis, but for short task sequences,
or certain highly proceduralised tasks, these task descriptions could be produced as a simple linear
sequence of task elements representing the order in which they would be undertaken.
Choice of Decomposition Categories
This technique, developed by Miller (in "A method for man-machine task analysis"), proposes that
each element is decomposed into the following categories:
Description
Sub-task
Cues initiating action
Controls used
Decisions
Typical errors
Response
Criterion of acceptable performance
Feedback
The above decomposition approach still has relevance to the analysis of tasks today. However, this
categorisation does not cover all issues, and so it will be necessary to develop other decomposition
categories. Collection and presentation of decomposition information The decomposition categories
are listed on a set of information collection forms, using a set of clear sub-headings, and starting
with the task description.
Example from Infopolis 2 (Delivrable 3 - WP 3.2: Traveller Needs Not published yet)
Task
Purpose Cue (for Decision Action
element
element)
travel
steps:
Optimum
way with
rapidity
Frequency
Route
Networks
Pre-trip
as criteria
Vehicles
On-trip
Display Control Likely
errors
Looking for Timetable
information (screens,
leaflets,
PAT, … )
Vehicles
To miss
arrival and one’s
travel time connection,
To get lost,
…
Lines
End-trip
Schedule
Advantages
The task decomposition technique can be applied to a general or specific task descriptions.
This technique assures that the issues of interest are systematically considered for every
task situation or element. The decomposition information is particularly useful for deriving
control and information requirement lists.
Disadvantages
The main difficulty is that it may take a considerable amount of time and effort to obtain all
the necessary information and to evaluate eventual conflicts
Task Simulation Methods: Walkthrough
and Talkthrough Techniques
Overview
A walkthrough, as the name implies, involves a user walking through a task by demonstrating it. So
a walkthrough is a step by step method that requires users to undertake and explain some
demonstration of a task in a realistic environment, without necessarily undertaking that task. A
talkthrough method is very similar, but it is undertaken more remotely from the normal task location,
so that the tasks are verbalised rather than demonstrated. So, walkthroughs and talkthroughs have
a close relationship to verbal protocols.
Application
Walkthroughs and talkthroughs are generally used to describe and verify the principal observable
components of tasks. Walkthroughs can take place in real time with a minimum of comments from
the user who is performing the task. Walkthrough and talkthrough methods can be considered as
lying on a continuum which represents the degree of accuracy with which a task and its interfaces
are represented. These techniques can be used in their own right to assess whether a task is
feasible and can be accomplished in time.
Advantages
Although it is not necessary to perform a task, only to demonstrate it, a walkthrough is
effectively an actual performance of a real task. This is also true of talkthroughs, since the
user is describing observable events and behaviours rather than inferred mental processes.
In the case of a talkthrough, or if the walkthrough is not executed in real-time, it is possible
to interrupt the process with a period of questioning.
Task description methods can be used to further describe the data collected in a more
useful way. They can be constructed from sufficiently comprehensive walkthrough or
talkthrough methods.
Disadvantages
The technique requires an experienced user. The exception to this is if the system is
intended either for casual and untrained users. In such cases, the user's expectations of the
way in which tasks should be performed will be a focus of interest.
In the case of interrupted walkthrough, the analyst must to have a good knowledge of the
system, in order to ask the right questions.
The effort which must be devoted to data analysis is considerable.
Types of Walkthrough and Talkthrough techniques
Talkthroughs
The main advantage of talkthroughs is that they do not require any special task surrounding,
thought it is helpful to have some technical documentation available, such as procedure or
control / display panel drawings. This gives the advantage that talkthroughs can be
undertaken early during the design stage before any mock-ups, simulators or real plant have
been produced. In such situations, it is more profitable to utilise the knowledge of several
individuals and to make a group discussion. Another important use of talkthroughs is as a
means of getting an orientation before undertaking a more extensive walkthrough.
Detailed Walkthroughs
These are walkthroughs in which the accent is placed upon gathering information about
particular aspects of a task, so that is paced by the rate at which responses are given, which
will not necessarily correspond to the normal rate of task performance.
Real-time Walkthroughs
In real-time walkthroughs the main aim for the user is to closely imitate the way in which a
task would be undertaken. An essential element of this is that the walkthrough should be
undertaken at a realistic pace. And so for most tasks the depth of the comments which the
respondent is able to provide will be limited. In order to enable a person to undertake a real
time walkthrough at a realistic progression, it will be necessary to undertake such a
walkthrough in a relatively highly realistic task environment. The main reason for undertaking
a real time walkthrough will be to determine whether the workload which is imposed by a
particular task will be feasible for a user, and to assess the effects of dynamic factors upon
task performance.
Team Walkthroughs
These are real time walkthroughs which are undertaken by a team of users who are either
undertaking the same task, or a group of highly related tasks. Such walkthroughs are useful
for investigating the interactions and communications between individuals in complex multiperson tasks.
Task Simulation Methods: Mock-ups and
Simulators
Overview
Under the names mock-up and simulator a range of techniques can be considered which involve
the development and use of some form of simulation of the system. The main objective is to
observe user activity and performance in a representation of the task environment. In an evaluation
of the mock-ups real potential users of the system are involved in implementing typical tasks they
are likely to encounter when the system is in operational use.
Application
Simulators and mock-ups will usually be used when the real system is unavailable or expensive for
study, or in order to study systems prior to the real system being designed. The simulation would
typically be used to finalise appropriate working methods, ergonomics of display / control layout and
design, identification of possible sources of error, or to derive learning recommendations (see also
Deliverable 4: "Mock-ups: different versions of different systems" - Infopolis 2 project. Not published
yet) .
Advantages
The simulators and mock-ups offer the possibility to assess user activities in a situation which is
not normally easily observable.
Disadvantages
The behaviour observed in simulation may not be fully realistic, because of the user's motivation
toward the simulator or mock-up and because the simulation may not contain all the features of the
real task.