The Individually Prioritised Problem Assessment: A Clinically Useful
Instrument for Measuring the Effectiveness of Assistive Technology
MacLean, L.
INTRODUCTION
In the current climate of accountability with the focus on evidence-based
practice, it is essential that clinicians measure the effectiveness of assistive
technology provision in order to inform professional decisions, justify
resources and enhance outcomes for clients.
“Assistive technology outcomes can be defined in terms of changes produced
by assistive devices in the lives of users and their environments, including
less dependence on other people and a variety of societal resources” (Fuhrer,
Jutai, Scherer, and DeRuyter, as cited in Lenker, Scherer, Fuhrer, Jutai and
DeRuyter, 2005, p. 8). “Efficacy of an assistive technology device is
determined by the effect resulting from its use in comparison to the effect
claimed beforehand” (Gelderblom and de Witte, 2002, p. 91).
There are many standardised instruments available to clinicians in general
practice, however, as the field of assistive technology is relatively young,
there are few sound instruments available specifically designed to measure
the complex and diverse nature of assistive technology outcomes. Assistive
technology outcome measurement is commonly associated with a number of
conceptual domains, including: device usability, user satisfaction, quality of
life, social role participation, functional level and cost (Lenker et al., 2005).
Most instruments focus on one or more of the above domains to measure a
specific type of outcome of the application of assistive technology. The
challenge for clinicians is in identifying the most appropriate tools for their
clients and service.
BACKGROUND
TASC Consultative Services (Technology solutions for computer Access,
Seating and Communication) is a division of Technology Services of The
Spastic Centre in NSW, and provides specialist clinical information for people
with complex assistive technology needs across NSW and the ACT. TASC
consultants work in a collaborative-consultative model with local support
teams to provide advice and information, comprehensive assessments, and
assistive technology trials and recommendations. Historically, outcomes of
TASC consultant’s assessments and recommendations were reliant on
clinical observations and subjective reports from local support teams and the
clients themselves. A need was identified for more objective measures to
evaluate the effectiveness of TASC consultant’s recommendations.
In order to determine appropriate outcome measures for TASC’s specific
service delivery needs, a focus group was formed (including a research fellow
of The Spastic Centre), and a literature review undertaken to identify possible
measures. Key elements identified by the focus group were that the measure
needed to be: generic to be able to evaluate a range of technology
interventions, relevant to a diverse client group in age and disability, simple to
administer, cost effective, and sensitive to show specific individual changes in
relation to the provision of the assistive technology. As a client centred service
it was felt the most important construct to measure was whether
recommendations made by TASC consultants were impacting on individual
client’s specific needs and goals, and that the service was not attempting to
evaluate specific pieces of technology or compare individuals.
THE INDIVIDUALLY PRIORITISED PROBLEM ASSESSMENT
The Individual Prioritised Problem Assessment (IPPA) was identified as a
suitable outcome measure to meet TASC’s needs. The IPPA is a generic
effectiveness measure that can be used to assess the effectiveness of any
kind of assistive technology provision for individuals with a disability. “It
assesses the extent to which problems identified by an individual assistive
technology user in his or her daily activities have been diminished as a result
of the provision of assistive technology” (Persson et al., as cited in Wessels et
al., 2002, p. 141). The IPPA is client centred in that it assesses effectiveness
in relation only to those activities considered relevant by the individual service
user. It is available free of charge with reference to the authors.
The aim of the IPPA is to assess change over time. It involves a clinician
conducting a simple face-to-face initial interview, a follow-up telephone
interview some time (several months recommended) after provision of
assistive technology and completion of relevant forms (See Appendix 1).
Limited training is required for the clinician in the administration and scoring of
the IPPA. The administration of the initial interview takes between
approximately 10-30 minutes to complete and the follow-up interview less
than 15 minutes.
The client is asked at initial interview to “identify the problems that he or she
experiences in everyday life and that he or she hopes are eliminated or
diminished as a result of an assistive technology provision” (Wessels et al.,
2002, p. 142). This is an interactive process and it is important that problems
are identified in relation to concrete daily activities. It is preferable that this
interview takes place as early in the service delivery process as possible.
The client identifies up to seven problems and for each of these problems, an
IPPA form is completed (see Appendix 1). Scores are assigned (using a 5
point Likert scale) to each identified problem relative to the importance of the
activity and the level of difficulty associated with performing the activity.
A total IPPA score is calculated by multiplying each ‘importance’ score and
‘difficulty’ score then summing the problem scores and dividing by the total
number of problems. “This score represents the total average perceived
inconvenience experienced by the client with respect to the problems
associated with daily activities” (Wessels et al., 2002, p. 142). The higher the
‘IPPA score’, the more an individual perceives their life to be inconvenienced
by these problems.
The IPPA score is re-calculated at the follow-up interview by re-assigning a
‘difficulty’ score to each problem which is then weighted by the original
‘importance’ score. The difference between the total IPPA score before and
after provision of assistive technology is considered to represent the
‘effectiveness’, “thus indicating the degree
to
which
the perceived
inconvenience with respect to the problems has diminished” (Wessels et al.,
2000, p. 107).
In addition, at follow-up both the client and the clinician answer a separate
question regarding the degree to which the assistive technology has fulfilled
their expectations (see Appendix 1). This question is answered separately
and without consultation. This provides an ‘expectation fulfilment score’ for
both the client and the clinician, and this is done by summing the expectation
scores and dividing by the number of problems.
PILOT STUDY
Following identification of the IPPA as a suitable outcome measure, the focus
group developed a pilot study to trial the instrument and determine its clinical
utility for TASC consultants’ service delivery model.
All consultants
underwent an initial training session with the research fellow and were
provided with literature detailing the instrument. Assessment forms were
developed (see Appendix 1) and protocols determined for the implementation
of the instrument. All new clients entering the service were to complete the
IPPA interview process with a consultant at the initial assessment. Problem
identification needed to be restricted to areas that would be addressed by
TASC Consultative Services. A database was developed to track results and
to highlight timeframes for follow-up interviews once equipment was provided.
RESULTS
Since commencement of the pilot study, the following results have been
obtained over an 18-month period:
Initial IPPA

57 IPPA Initial Interviews have been completed

Mean total Initial IPPA Score was 19.47 (out of a maximum of 25),
indicating a high level of perceived inconvenience

131 problems were identified

Average of 2-3 problems identified by most clients

Highest number of problems identified was 5, lowest was 1

Most clients indicated a high level of importance for all problems
identified (score of 4 or 5)
In a breakdown of identified problems, the following common problems were
identified in three main areas:
Communication:

Access issues to a device or system

Communicating basic needs

Communicating with unfamiliar conversation partners

Communicating more complex information / conversations

Making choices

Speaking on the phone

Initiating communication

Having independent communication

Frustration with communication attempts
Computer use:

Access issues to standard mouse or keyboard

Access to the Internet or email

Access to leisure activities and games

Access to the curriculum

Access to software to develop literacy, numeracy and other basic skills

Inability to handwrite

Pain or fatigue with typing or mouse use

Difficulties with switch access
Environmental controls:

Control over infrared appliances e.g. television/DVD/stereo

Access to the phone

Control over other appliances e.g. lights, beds, doors, alarm systems
Follow-up IPPA
Of the 57 clients completing initial IPPA Interviews:

26 were recommended technology to address identified problems

14 are still investigating technology solutions

17 were not recommended technology
Of the 26 clients who were recommended technology:

9 have received their equipment

4 have been using their technology for a sufficient time period (several
months as recommended) and have participated in IPPA follow-up
interviews

1 client who received equipment died before a follow-up interview

1 client has been uncontactable for a follow-up interview

3 clients have been using their technology for less than 1 month,
therefore a follow-up interview is not indicated as yet

17 are still awaiting equipment provision – the longest wait period from
assessment to date being 12 months
Of the 4 clients who have participated in a follow-up interview the following
results were obtained (see graph):
Scores
Initial and follow-up IPPA data
Initial IPPA
Follow-up IPPA
LD
JH
KJ
MB
Clients

Client LD showed a significant reduction in total IPPA score and both
client and clinician indicated the technology (alternate mouse and
switch for computer access) had solved the problems ‘more than
expected’.

Client JH showed a slight reduction in total IPPA score and indicated
the technology (voice-output device) had solved the problems ‘less
than expected’, the clinician ‘as expected’. Subjective reports indicate
that this client requires more time to develop skills in using this device.

Client KJ showed a moderate reduction in total IPPA score and
indicated the technology (alternate mouse and keyboard for computer
access) had solved the problems ‘as expected’, the clinician ‘less than
expected’. Subjective reports indicated this client was not satisfied as
additional equipment requested had not been funded (Internet
connection) therefore all identified problems had not been addressed.

Client MB showed an increase in total IPPA score and both client and
clinician indicated the technology (range of equipment for computer
access and environmental control) had solved the problem ‘much less
than expected’. This client has a degenerative condition that
progressed rapidly, and therefore technology was no longer accessible
or appropriate.
DISCUSSION
The results, or lack thereof so far, indicate the diverse and difficult nature of
assistive technology outcome measurement. Many clients assessed as
needing technology are waiting lengthy time periods for allocation of funding
for equipment. During this wait period, client’s needs may change and also
their identified problems and expectations. At this stage of the study TASC is
unable to show statistically significant results to indicate effectiveness of
overall technology recommendations for all clients assessed. However, initial
results have been obtained to show small, significant changes for individual
clients in diminishing their identified problems, and the technology has been
shown to be effective in reducing perceived inconvenience related to
problems in most cases.
The process of implementation of outcome measurement has been valuable
for the service and the IPPA has been found to be a clinically useful tool. The
process of problem identification has helped TASC consultants to clarify goals
of the client, families and support teams and their expectations of the service.
Initial results obtained indicate the measure will provide clinically useful
information and continued use is warranted.
The IPPA was found to be clinically useful as demonstrated by the following
five areas (Law, 1987):

The format and layout is clear, concise, complete and simple in
presentation.

Limited training was required, involving only reading the literature, and
the measure was simple to administer and appropriate for the clinical
situation.

The cost of the measure was effective in terms of monetary costs (free
to obtain and maintain) and staff time to administer and score the
measure (10-30 minutes).

The measure is acceptable to the client as it is easy to understand,
appropriate for age and developmental level and relevant to a range of
disabilities. The measure is client-centred as problems are identified
and rated by the client.

The measure is able to provide information that can be used for clinical
management of the client as it demonstrates if identified problems have
diminished in the eyes of the client and if their expectations of
intervention have been met.
CONCLUSIONS
The IPPA has been found to be a clinically useful instrument for TASC
Consultative Services. Further results need to be obtained and analysed
before statistically significant conclusions can be made on the effectiveness of
TASC consultants’ technology recommendations in diminishing clients’
identified problems. However, initial results show small, significant reductions
in perceived inconvenience related to identified problems for individual clients
as a result of technology provision. Future results will be shared and further
discussion is required on the clinical usefulness of the IPPA for assistive
technology outcome measurement.
References
Gelderblom, G.J., & de Witte, L.P. (2002). The assessment of assistive
technology outcomes, effects and costs. Technology and Disability, 14, 91-94.
Law, M. (1987). Measurement in occupational therapy: Scientific criteria for
evaluation. Canadian Journal of Occupational Therapy, 54(3), 133-138.
Lenker, J.A., Scherer, M.J., Fuhrer, M.J., Jutai, J.W., DeRuyter, F. (2005).
Psychometric and administrative properties of measures used in assistive
technology device outcomes research. Assistive Technology, 17(1), 7-22.
Wessels, R., de Witte, L., Andrich, R., Ferrario, M., Persson, J., Oberg, B.,
Oortwijn, W., VenBeekum, T., & Lorentsen, Ø. (2000). IPPA, a user-centred
approach to assess effectiveness of Assistive Technology provision.
Technology and Disabilty, 13, 105-115.
Wessels, R., Persson, J., Lorentsen, Ø., Andrich, R., Ferrario, M., Oortwijn,
W., VanBeekum, T., Brodin, H., & de Witte, L. (2002). IPPA: Individually
Prioritised Problem Assessment. Technology and Disability, 14, 141-145.
APPENDIX 1
Individually Prioritised Problem Assessment (IPPA)
Client name:
DOB:
Assessment date:
Follow-up date:
Clinician(s) completing form:
IPPA SCORES
Number of identified problems:
Total IPPA score at assessment:
Sum of problem scores
(A)
# of problems
Total IPPA score at follow-up:
Sum of problem scores
# of problems
(A) – (B)
Effectiveness:
Sum of Expectation scores
Expectation fulfilment of client:
# of problems
Sum of Expectation scores
Expectation fulfilment of consultant:
# of problems
(B)
IPPA ASSESSMENT FORM
Client name:
Problem no:
Date:
Problem:………………………………………………………………………………
1. How do you rate the importance of this problem?
(circle the most appropriate number)
1
2
3
4
5
Not important at all
Not so important
Somewhat important
Quite important
Most important
2. How do you rate the level of difficulty you have with this problem in
everyday life? (circle the most appropriate number)
1
2
3
4
5
Not difficulty at all
Little difficulty
Quite some difficulty
A lot of difficulty
Too much difficulty
to perform the
activity at all
PROBLEM SCORE (Question 1 x Question 2) =
Problem no:
Date:
Problem:………………………………………………………………………………
1. How do you rate the importance of this problem?
(circle the most appropriate number)
1
2
3
4
5
Not important at all
Not so important
Somewhat important
Quite important
Most important
2. How do you rate the level of difficulty you have with this problem in
everyday life? (circle the most appropriate number)
1
2
3
4
5
Not difficulty at all
Little difficulty
Quite some difficulty
A lot of difficulty
Too much difficulty
to perform the
activity at all
PROBLEM SCORE (Question 1 x Question 2) =
IPPA FOLLOW-UP ASSESSMENT FORM
Problem no:
Date:
Problem:………………………………………………………………………………
1A. How do you rate the level of difficulty you have with this problem in
everyday life? (circle the most appropriate number)
1
2
3
4
5
Not difficulty at all
Little difficulty
Quite some difficulty
A lot of difficulty
Too much difficulty
to perform the
activity at all
2A. Has using the assistive device solved this problem as much as you
expected it would? (circle the most appropriate number)
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
PROBLEM SCORE (Question 1A x Question 1 initial assessment) =
Problem no:
Date:
Problem:………………………………………………………………………………
1A. How do you rate the level of difficulty you have with this problem in
everyday life?
(circle the most appropriate number)
1
2
3
4
5
Not difficulty at all
Little difficulty
Quite some difficulty
A lot of difficulty
Too much difficulty
to perform the
activity at all
2A. Has using the assistive device solved this problem as much as you
expected it would? (circle the most appropriate number)
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
PROBLEM SCORE (Question 1A x Question 1 initial assessment) =
IPPA CLINICIAN OVERALL SCORE
Client name:
Clinician name:
Date:
Has using the device solved problem 1 as much as you expected it would?
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
Has using the device solved problem 2 as much as you expected it would?
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
Has using the device solved problem 3 as much as you expected it would?
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
Has using the device solved problem 4 as much as you expected it would?
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
Has using the device solved problem 5 as much as you expected it would?
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
Has using the device solved problem 6 as much as you expected it would?
-2
-1
0
1
2
Much less than
expected
Less than expected
As expected
More than expected
Much more than
expected
SUM OF SCORES =