446650
POI37110.1177/0309364612446650Robinson and FatoneProsthetics and Orthotics International
2012
INTERNATIONAL
SOCIETY FOR PROSTHETICS
AND ORTHOTICS
Expert Clinical Viewpoint: Invite from the Editor
You’ve heard about outcome measures,
so how do you use them? Integrating
clinically relevant outcome measures
in orthotic management of stroke
Prosthetics and Orthotics International
37(1) 30­–42
© The International Society for
Prosthetics and Orthotics 2012
Reprints and permission:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0309364612446650
poi.sagepub.com
Christopher Robinson and Stefania Fatone
Abstract
In today’s healthcare environment it is increasingly important to be able to quantify the amount of change associated
with a given intervention; this can be accomplished using one or more appropriate outcome measures. However, the
selection and integration of outcome measures within clinical practice requires careful consideration. This includes
identification of the measure construct which can be assisted by the International Classification of Functioning, Disability,
and Health; selection of outcome measures based on need, appropriateness and feasibility; and careful use in regular
clinical practice including data collection, analysis and re-assessment of the process. We describe this process, focusing
on orthotic management of stroke, in particular the improvement of mobility as a common goal.
Clinical relevance
The growing emphasis on improved documentation of patient care and outcomes requires that clinicians integrate
clinically relevant outcome measures into their practice. We suggest a process to assist clinicians integrate outcome
measures into clinical practice with a particular emphasis on the orthotic management of stroke.
Keywords
Orthotics, stroke, outcome assessment, gait, mobility
Date received: 22 October 2011; accepted: 3 April 2012
Introduction
Stroke occurs in approximately 795,000 individuals per
year in the USA, accounts for 1 in every 18 deaths, and
is the leading cause of serious long-term disability.1
Stroke commonly occurs in the elderly, a population that
is rapidly growing and will contribute to potentially doubling the incidence of stroke by 2050.2 Among stroke
survivors, 15–30% are permanently disabled and 20%
require institutional care.1 Furthermore, 50% of stroke
survivors over age 65 have some form of hemiparesis,
30% are unable to walk without some form of assistance,
and 26% are dependent in activities of daily living six
months after stroke.1 Many stroke survivors are unable
to walk independently within the community, which
explains why recovery of mobility after stroke is valued
highly by patients.3,4 Additionally, mobility limitations
experienced post-stroke can lead to loss of independence, restrictions in activities of daily living, and reduced
quality of life.5
Orthotic management is intended to address lower limb
impairments in the post-stroke population, with ankle foot
orthoses (AFO) being the most commonly provided orthosis by certified orthotists practicing in the USA.6 Recent
literature reviews on the orthotic management of stroke
indicate that AFOs can: (1) improve walking in the poststroke population, especially when plantar flexion motion
is restricted by the orthosis7; (2) improve gait kinematics
during both stance and swing phases7; (3) directly affect
function of the foot and ankle complex and indirectly affect
the knee, hip and trunk7-9; and (4) may make a positive contribution to quality of life.8,9 These reviews indicate that a
Northwestern University, Chicago, USA
Corresponding author:
Stefania Fatone, Northwestern University Prosthetics-Orthotics Center,
680 N Lake Shore Drive, Suite 1100, Chicago IL 60660, USA.
Email: s-fatone@northwestern.edu
31
Robinson and Fatone
Table 1. Example resources for outcome measures relevant to rehabilitation and stroke.
Resource Name
Website
Rehabilitation
Measures Database
www.rehabmeasures.org
StrokEngine-Assess
StrokeCenter
Evidence Review of
Stroke Rehabilitation
Description
A comprehensive outcome measures database compiled by the
Center for Rehabilitation Outcomes Research (CROR) offering
clinicians and researchers information on outcome measures
appropriate for many rehabilitation populations.
www.medicine.mcgill.ca/
Provides scientific evidence about stroke assessments created by a
strokengine-assess/
group of experts in stroke rehabilitation.
www.strokecenter.org/
Provides health professionals with multiple tools and educational
professionals/stroke-diagnosis/
presentations about stroke assessment, stroke treatment and
stroke-assessment-scales/
management.
www.ebrsr.com/reviews_details. A resource for clinicians that compiles stroke literature from around
php?Outcome-Measures-9
the world into summaries referred to as evidence reviews. Includes a
chapter on outcome measures.
primary goal of orthotic management of stroke is the
improvement of mobility.
In today’s healthcare environment it is increasingly
important to be able to quantify the amount of change associated with a given intervention. This can be accomplished
using one or more appropriate outcome measures.10 An outcome measure is a standardized instrument used in clinical
and research settings to evaluate change in health status of
an individual, group or population that is attributable to an
intervention or series of interventions.11-16 They include
self-reported, professional-reported, and performancebased measures. Self-report measures are ones where the
patient or a proxy (e.g. family member or care giver)
respond to items. While self-report measures are more subjective in nature, they are relatively easy to administer and
can reflect the patient’s perspective not only on the benefits
of treatment but also satisfaction with it. Professionalreported measures are ones where the clinician responds to
items, permitting rigid use of defined item characteristics.
They can also be performed rather quickly but are subject
to clinician bias, which may be particularly problematic
when the clinician rating the patient is also treating the
patient. Performance-based measures are usually administered by trained professionals and are more objective in
nature although they may only reflect physical ability on a
particular occasion in an artificial setting (i.e. the clinic).
Outcome measures may also be generic and therefore useful
for comparisons across populations, or condition-specific
and suitable for use only within a specific population.
While outcome measures can provide important information, their routine use in clinical practice can be challenging.17,18 Administrative and respondent burden are
often cited as impediments to use of outcome measures in
clinical practice.18,19 Random errors may occur during
any part of the measuring process as a consequence of
inattention, fatigue or inaccuracy.20 Practice, floor and
ceiling effects may also confound the use of outcome
measures.21 Practice or learning effects refer to
improvements in performance of a measure with each
attempt. The consequence of a practice effect is that an
improvement in the outcome measure may be mistakenly
attributed to the intervention. Floor and ceiling effects
refer to the inability of a measure to capture the individuals lowest and highest levels of ability, respectively. The
consequence of a floor effect is that deterioration in the
outcome measure may be missed and, for a ceiling effect,
improvement may be missed.21 Adoption of outcome
measures into clinical practice requires cognizance of
these issues.
Use of outcome measures in the management of stroke
is difficult due to varying stroke etiologies, symptom heterogeneity and severity, and, during the acute phase, spontaneous recovery.22 An additional challenge is not
necessarily the availability of measures, but the quantity of
measures from which to select.23,24 To simplify the process
of selecting an outcome measure experts have written systematic reviews of measures.3,22,24-26 recommended core
batteries of measures,23 and created online repositories of
measures. Examples of outcome measure repositories pertinent to the stroke population are shown in Table 1. These
repositories typically contain information about the available measures, including descriptions, populations for
which the measures have been validated, cost for use, training and equipment required for administration, and psychometric (i.e. measurement) properties. Psychometric
properties of outcome measures include reliability, validity, responsiveness and sensibility or feasibility.21,22
Reliability refers to the stability of a measure (the extent to
which it is free from random error) when used under identical conditions; validity refers to the degree to which the
measure actually measures what is intended; responsiveness refers to the ability of a measure to detect true change
in the area it is designed to measure; and sensibility/feasibility refers to the overall appropriateness, importance and
ease of use of the measure.21,22 Table 2 provides more
information about these common psychometric properties.
32
Prosthetics and Orthotics International 37(1)
Table 2. Definitions of some common psychometric properties of outcome measures.
Reliability
Test/Retest reliability
Intra-rater reliability
Inter-rater reliability
Validity
Face validity
Content validity
Criterion validity
Predictive validity
Concurrent Validity
Construct validity
Discriminant validity
Convergent validity
ensitivity to change
S
(Responsiveness)
Minimal detectable change
The degree of stability when a measure is performed under identical conditions.43
There are different types of reliability, some of which are mentioned below.
The administration of an instrument consecutively to the same subject, measuring the agreement
between the scores of each measurement.53
The administration of an instrument by the same tester consecutively to the same subject and the
agreement between the scores of each measurement.53
The administration of an instrument by different testers and the agreement between the scores of
each tester.53
The degree to which the measure actually measures what it is intended to. There are different types
of validity, some of which are mentioned below.
The measure superficially appears to measure what it is intended to measure. This is the lowest
relative measure of validity.54
The items that make up an instrument adequately represent all possible items that compose the
construct being measured.43
The performance or accuracy of an instrument based on comparison to a gold standard.20 Timing
component of criterion validity is divided into concurrent and predictive validity.
How well the outcomes of an instrument predict a future state or outcome.43
How two measures compare when the two measures are taken at the same time.20,43
The ability of an instrument to measure an abstract concept and the degree to which the instrument
reflects the theoretical components of that concept.43 Includes convergent and discriminant validity.
The degree to which two or more measures, assessing theoretically different constructs,
demonstrate a difference in outcomes.43
The degree to which two measures, assessing theoretically similar constructs, demonstrate similar
results.43
The ability for an instrument to detect true and significant change in the area it is designed to
measure.20
The amount of change required to indicate a true difference is observed versus an error in
measurement.20
It is important when selecting a measure to ascertain to
what extent the psychometric properties of the measure
have been evaluated for the population of interest, in this
case stroke.22
So, how might we define the basic tasks of daily life,
such as mobility, that may be assessed using outcome
measures? The International Classification of Functioning,
Disability, and Health (ICF) provides a conceptual framework for description, measurement and policy development
related to health and function.27 Figure 1 depicts the main
components of the ICF model and their relationship to one
another. Using the ICF model, health conditions are conceptualized by interactions between impairments in body
structure and function, activity limitations, and participation restrictions, as well as their relationship with both
environmental and personal factors. Outcomes may be
measured at any of these levels, allowing the type of information assessed and their clinical relevance to be determined.21,22,24 The ICF provides a common approach linking
the consequences of a specific health condition, such as
stroke, to relevant ICF categories of functioning, e.g. the
ICF Core Set for Stroke.28 These categories, including
mobility, can then be operationalized by linking them to
outcome measures.3,21
Figure 1. Diagram of the ICF Model (taken from the World
Health Organization27).
Despite the important role that orthotic management
may play in addressing mobility limitations following
stroke, orthotists infrequently assess the effectiveness of
their interventions. This is not surprising given the challenges inherent in selecting and integrating outcome
33
Robinson and Fatone
Table 3. Mobility tasks defined by the ICF that refer to walking (adapted from World Health Organization27).
Activities and Participation
d4 - Mobility
d450 – Walking
d455 – Moving around (other
than walking)
d460 – Moving around in
different locations
d4500 – Walking short distances
d4501 – Walking long distances
d4502 – Walking on difference
surfaces
d4503 – Walking around
obstacles
d4509 – Walking unspecified
d4551 – climbing (stairs, steps,
curbs)
d4552 – running
d4553 – jumping (hopping,
skipping)
d4600 – Moving around within
the home
d4601 – Moving around within
other buildings
d4602 – Moving around outside
home and other buildings
d465 – Moving around using
equipment
measures into routine clinical practice. Therefore, the aims
of this paper were to: (1) define the domain of mobility
using the ICF; (2) describe a process for selecting and integrating clinically relevant outcome measures into clinical
practice; and (3) illustrate how this process may be used in
the orthotic management of stroke.
Defining the mobility domain
Mobility has been defined as the ability to move independently from one point to another.29 As such, mobility serves
as a fundamental part of many activities of daily living and
may affect quality of life.23,30 It has even been proposed that
improved quality of life and independence are a non-biomechanical effect of orthotic use post-stroke,8 suggesting
that mobility may affect both biomechanical and non-biomechanical outcomes. However, mobility is a broad domain
and for the purpose of selecting an outcome measure it is
helpful to use a framework like the ICF to further define it.
This makes it easier to match an outcome measure to the
treatment goal of improving mobility.
The mobility domain (assigned the descriptor d4 by the
ICF) falls within the level of ‘Activity and Participation’
and is defined by the ICF as ‘moving by changing body
position or location or by transferring from one place to
another, by carrying, moving or manipulating objects, by
walking, running or climbing, and by using various forms
of transportation’ (p. 150).31 Mobility is broken down into
tasks such as ‘changing and maintaining body position’
(assigned descriptors d410–d429), ‘carrying, moving, and
handling objects’ (d430–d449), ‘walking and moving
around’ (d450–d469), and ‘moving around using transportation’ (d470–d489). The mobility tasks that refer to walking (d450–d465)3 and are most often the focus of orthotic
intervention are shown in Table 3.
Using these different mobility tasks, we can identify
specific components of walking that may be affected by
orthotic intervention. For example, ‘walking’ (d450) can be
broken down into subtasks such as ‘walking short distances’ (d4500) or ‘walking long distances’ (d4501). Recent
literature reviews have shown that the outcome measures
used most frequently to evaluate walking ability after stroke
are those that assess walking over a short distance at a selfselected walking speed or at a fast pace.3,32 Although
informative, such outcome measures provide a relatively
limited assessment of the ability of the individual to ambulate in the everyday environment.
Outcome measures that assess walking under different conditions, although used sparingly in stroke
research, reflect more recent attempts to measure the
influence of the environment on mobility.3,4 As defined
by the ICF, environmental factors encountered in the
home and community include, ‘walking over different
surfaces’ (d4502) and ‘walking around obstacles’
(d4503). Of particular relevance to the orthotic management of stroke, the ICF defines a mobility task (d465)
that acknowledges the use of assistive devices such as
orthoses and walkers. Given that the orthotist’s primary
modality falls under the category of ‘moving around
with equipment’ (d465), an ideal outcome measure for
assessing the orthotic management of stroke would
include assessment of that specific task.
34
Proposed process for integrating
outcome measures into clinical practice
Outcome measures are not an event but a process involving
the careful selection of measures, correct and consistent
administration of those measures, analysis of results, and
use of the results to inform clinical practice. Outcome
measures allow a clinician to track an individual’s clinical
care over time, as well as compare clinical outcomes across
patient groups.20 While the administration of outcome
measures may add time to the patient visit, this added time
may not be as substantive as is often thought if the outcome
measures replace subjective, ad-hoc methods of patient
assessment or provide helpful information. We suggest the
following process to assist clinicians when integrating outcome measures into clinical practice.
It is important to identify at the outset why an outcome
measure is needed by identifying what value it will bring to
clinical practice. Common reasons for choosing to use outcome measures include the need to: (1) estimate prognosis;
(2) create treatment guidelines; (3) measure the effect of
treatment; and (4) provide information about care to third
parties.3,20 Defining why an outcome measure is to be used
helps determine what is to be done with the collected data.
For example, is it enough to have the outcome data documented in each patient file, or should the data be aggregated across patient populations?
The next step is to identify the specific information to
collect based upon treatment goals common in the particular patient care setting. With orthotic management of stroke,
treatment often addresses impairments that result in
decreased mobility. In particular, it is often intended that an
orthosis improve the individual’s ability to ambulate within
the community. As such, the specific mobility tasks to be
addressed by the orthosis must be defined and assessed.
Outcome measures exist that address a single mobility task
or multiple mobility tasks. Outcome measures can also be
used in combination to ensure comprehensive assessment
of all relevant outcomes although this may add to the
administrative and respondent burden, challenging feasibility of administration in a busy clinical environment. For
example, if the goal of orthotic management is solely to
improve walking speed the Ten-Meter Walk Test (10mWT)
may be an appropriate and sufficient outcome measure.
This is a single-task outcome measure that assesses the time
a patient takes to walk 10 meters at a self-selected walking
speed.10 However, if the orthotic intervention has multiple
goals (e.g. improving walking speed and walking endurance), the test will be insufficient to assess both outcomes
and multiple single task outcome measures may be needed.28
In a study by Geboers et al.33 both the 10mWT (to assess
walking speed) and the Six-Minute Walk Test (6MWT) (to
assess endurance) were administered to subjects with paralytic equinus walking with, and without, an AFO. The
authors reported that significant improvements were
Prosthetics and Orthotics International 37(1)
observed when walking was measured with the 6MWT but
not the 10mWT. A plausible explanation for these results
offered by Stevens et al.34 was that individuals with paralytic equinus are able to compensate effectively over short
distances, but not over extended periods of exertion. This
study illustrates the importance of clearly defining the specific mobility tasks of interest and selecting outcome measures best suited to assess the desired task.
Where the goals of the orthosis are aimed at improving
multiple aspects of mobility including walking speed, ability to transfer, and ability to navigate environmental obstacles, a multi-task test, such as the modified Emory
Functional Ambulation Profile (mEFAP),35 may be appropriate. The mEFAP consists of five individually timed
mobility tasks: a 5-Meter Walk Test (5mWT) on a hard
floor; a 5mWT on a carpeted floor; the Timed Up and Go
(TUG) test (rising from a chair, a 3-meter walk, and returning to a seated position); navigating a standardized obstacle
course; and ascending and descending five steps.35 Three of
the mEFAP tasks are based on the 5mWT36,37 and the TUG
test,38–41 which are standardized outcome measures in their
own right and can be administered and scored independently. Individually, the scores for each of the mEFAP tasks
offer unique information relevant to the specific task, while
the sum of the scores is intended to provide an overall
assessment of functional ambulation.35 The ability to
administer some subtasks independently may be beneficial
where the patient or facility does not have the means to
perform the entire set of tasks at every patient encounter,
making the mEFAP quite versatile. However, it is only
when the five tasks are performed and scored as a whole,
that the measure can be called the mEFAP.
If walking endurance were also an outcome of interest in
addition to walking speed, ability to transfer, and ability to
navigate environmental obstacles the mEFAP alone would
not be sufficient. As demonstrated by Hung et al.,42 the
6MWT could be used in conjunction with the mEFAP to
assess all these mobility tasks. Hung et al.42 used both these
measures to assess the effect of an anterior AFO on functional
mobility and walking endurance of persons post-stroke.
Once the type of information that needs to be collected
has been identified, the timing of assessment should be considered. Decisions regarding timing of assessment are based
on both what the clinician is interested in assessing and the
feasibility of frequent assessments. Some measures may be
too involved to feasibly employ at all office visits and may
therefore be best suited for initial and summative assessments, while other shorter measures may be readily performed at every patient encounter. Additional issues, such as
accommodation to orthotic intervention, may need to be considered. For example, if allowing accommodation to the
orthosis is important to the particular outcome of interest, a
clinician may administer an outcome measure at the initial
visit and at some time point after device delivery (i.e. a follow-up visit). On the other hand, if the immediate effect of
Robinson and Fatone
the orthosis is of interest, then the measure would be administered upon delivery of the orthosis and the results compared
to the initial visit or a no-device assessment at device delivery. Administering the outcome measure at all three time
points: initial visit, orthosis delivery and follow-up; would
allow assessment of immediate change as well as additional
changes following accommodation to the orthosis.
Once the need to adopt outcome measures has been
defined, the type of information to be collected identified,
and the timing of assessment determined, the search for one
or more outcome measures that best meets these needs
must begin. When choosing an outcome measure it is prudent for the clinician to assess its suitability by asking questions such as the following (adapted from Mudge and Stott3
and Stokes20):
•• Does the purpose of the outcome measure (i.e. what
is measured) match the aim of the intervention?
•• Was development of the outcome measure published
in a peer-reviewed journal? Answering ‘yes’ to this
question suggests that the measure has undergone
some scrutiny and therefore has some credibility.
•• Are instructions available describing how to administer and score the outcome measure and are they
clear and easy to use? This makes standardized
application and scoring of the measure easier to
accomplish.
•• How much time is required to perform the outcome measure? How is it administered? Is there a
cost or copyright associated with use of the outcome measure? Is special training or equipment
required to administer or score the outcome measure? The answer to these questions may inform
feasibility of adopting particular measures into
clinical practice.
•• To what degree does the measure have favorable
psychometric properties (e.g. validity and reliability)? In simple terms, can it reasonably assess the
property of interest in the population of interest, and
can it do so in a consistent manner?
•• Can a patient wearing an orthosis or using an assistive
device be assessed with this outcome measure? Does
the score account for use of an orthosis or assistive
device, especially if these are changing over time?
Table 4 presents an example of using the above questions to
assess a small selection of outcome measures that have
been used to asses mobility post-stroke. The information
provided in the table is intended to be illustrative of applying the questions described above rather than an exhaustive
review of each measure. As such, our primary source of
information was the Rehabilitation Measures Database43
identified in Table 1 as it allows the user to search for measures by criteria such as population (i.e. diagnosis),
35
construct (i.e. area of assessment), length of test and cost,
as well as the name of a measure or key word.
Once an outcome measure has been selected, it must be
carefully integrated into clinical practice. A plan should be
developed for data collection and analysis. All personnel
engaged in the use of the measure need to be properly
trained so that the measure can be administered accurately
and consistently. Even if the selected measure does not
require formal training, it is important to ensure that all
testers are confident that they understand how to apply the
measure and can do so consistently. Assuming that an
appropriate measure is selected, collecting, recording and
assessing outcome measures in a consistent manner will
help to ensure that good quality information about patient
care is obtained.
Failure to integrate the information derived from collecting outcome data into clinical practice will greatly
diminish the value of data collection efforts. In order to
ensure outcome measures are being used effectively, the
clinical practice must periodically re-assess how the information collected is being used to inform patient care. Does
the chosen measure collect information that is useful to the
clinical practice and, if not, where is it falling short? If the
measure is not proving to be useful, despite proper administration, then it might be prudent to re-evaluate practice
needs and select a more appropriate measure or an additional measure.
It is also important that clinicians are aware of how to
interpret outcome measure scores. Interpretation of scores
is facilitated where information such as normative data
(normal values for specific variables within a population),
cut-off scores (used to classify individuals into groups),
minimal detectable change (the minimum amount needed
to be sure the change is not the result of measurement
error), and minimally clinically important difference (the
smallest amount of change that might be considered important) are available in the literature for the population of
interest.43 For example, self-selected walking speed has
been established as a good predictor of community walking
status in persons with stroke, with cut-off scores of <0.4
m/s predicting household walking, 0.4 to 0.8 m/s predicting
limited community walking, and >0.8 m/s predicting
unlimited community walking.44-46 Additionally, progression between categories is clinically meaningful because it
has been associated with gains in self-reported function and
quality of life.45 For the 10mWT, the minimal detectable
change in chronic stroke patients has been reported as 22%
for self-selected comfortable walking speed43 and the minimally clinically important difference as 0.1 m/s47 (0.16 m/s
for acute stroke patients).48
The literature on lower limb orthotic management of
stroke suggests that one outcome consistently observed
with AFO use is an increase in walking speed.7 If clinicians
instituted clinic-wide data collection of walking speed and
found that stroke patients’ speeds rarely improved after
Yes,
Mobility, balance,
walking ability, and see 3, 39, 43
fall risk
[walking short
distances - d4500]
Rivermead Functional mobility Yes,
see 3, 43
following stroke
Mobility
Index (RMI) (gait, balance,
transfers)
[walking short
distances - d4500,
walking on different
surfaces - d4502,
climbing stairs d4551, running
- d4552), moving
around within the
home - d4600,
moving around
outside home and
other buildings d4602]
Timed Up
and Go
(TUG)
Yes, clear
and easy
to use
Yes, clear
and easy
to use
3–5 mins
<3 mins
approx.
10 mins
None
None
None
Physicianreported (includes
one performancebased item)
Performancebased measure
Performancebased measure
None
None
None
Yes, clear
and easy
to use
None
Walking endurance Yes,
6-Minute
see 3, 43
Walk Test [walking short
distances - d4500]*
(6MWT)
Performancebased measure
None
<5 mins
Yes, clear
and easy
to use
Yes,
see 3, 43
10-Meter Walking speed
Walk Test [walking short
(10mWT) distances - d4500]
What is
Measured
[ICF Domains
of Walking
Addressed]
Training
Required
Name of
Measure
Instructions Time to
Type of
Cost of the
PeerAdminister Administration Instrument
reviewed Available
References
Available
Table 4. An example of assessing outcome measures that assess mobility post-stroke.
None
(form only)
Standard
armchair
(~ 46cm tall),
stopwatch
Measuring
wheel,
stopwatch
14-m walkway,
tape measure,
stopwatch
Equipment
Required
Validated
for Stroke
(Continued)
Does not
assess for
improved
mobility via
assistive
devices/
orthoses
Assistive
devices may
be used
but are not
accounted
for in scoring
Assistive
devices may
be used
but are not
accounted
for in scoring
Assistive
devices may
be used
but are not
accounted
for in scoring
Responsiveness Accounts
for Stroke
for Use of
Orthoses
Not Found
Not Found
Test/Retest
= Excellent
(Chronic); Inter/
Intra-rater
= Excellent
(Chronic)
Criterion = Not Found
Test/Retest
Established
= Excellent
(Chronic)
(Chronic &
Acute),
Inter/Intra-rater
= Adequate
(Acute) /
Excellent
(Chronic)
Not Found
Criterion
Test/Retest
= Excellent
= Excellent
(Chronic)
(Chronic)
Inter/Intra-rater Content =
Based on Get
= Not Found
Up and Go
Test
Established
Criterion/
Test/Retest
Convergent (Acute)
= Excellent
= Excellent
(Chronic)**,
Inter/Intra-rater (Acute)
= Excellent
(Acute)**
Reliability
for Stroke
36
Prosthetics and Orthotics International 37(1)
What is
Measured
[ICF Domains
of Walking
Addressed]
Performancebased measure
approx. 20
mins
None
None
None
Training
workshop
recommended
Training
Required
Reliability
for Stroke
Standard
armchair
(~ 46cm tall),
stopwatch, tape
measure, 10m
walkway, 40
gallon trash can
Scoring can
be weighted
depending
on level
of manual
assistance or
device used
Use of
assistive
devices/
orthoses
affect
scoring of
instrument
Responsiveness Accounts
for Stroke
for Use of
Orthoses
Criterion = Not Found
Excellent with
a range of
tests (Acute),
Construct =
Excellent with
Fugl-Meyer
Assessment
(Acute)
Validated
for Stroke
Concurrent Good
Test/Retest =
responsiveness
Excellent (Acute = Excellent
(Acute)
(Acute),
& Chronic),
Convergent =
Inter-rater =
Excellent (Acute) Excellent with
10mWT and
Rivermead
Mobility Index
(Acute)
Test/Retest =
Adjustable
table, standard Excellent
armchair, floor (Acute),
Inter/Intra-rater
mat, pillows,
pitcher of water, = Excellent
measuring cup, (Acute)
a ball (2.5’
diameter), foot
stool, 2m line on
floor, stopwatch
Equipment
Required
*The ICF considers any distance less than 1 km to be a short distance. Mudge and Stott3 indicated that most patients with stroke are unable to ambulate 1 km in the time allotted. **For the entire instrument,
individual components have also been validated.
Yes, clear
and
reasonably
easy to use
Performancebased measure
45–60 mins
Yes, clear
but may not
be as easy to
use
Instructions Time to
Type of
Cost of the
PeerAdminister Administration Instrument
reviewed Available
References
Available
Chedoke- Physical impairment Yes,
see 3, 43
McMaster and disability in
patients with
Stroke
Assessment stroke and other
neurological
problems
[walking short
distances - d4500,
walking on different
surfaces - d4502,
climbing stairs d4551, moving
around within the
home - d4600,
moving around
outside home and
other buildings d4602]
Yes,
Functional
Modified
ambulation in adults see 3, 20, 35
Emory
Functional with neuromuscular
Ambulation disease
[walking short
Profile
distance - d4500,
(mEFAP)
walking on different
surfaces - d4502,
walking around
obstacles - d4503,
climbing stairs d4551, moving
around using
equipment - d465]
Name of
Measure
Table 4. (Continued)
Robinson and Fatone
37
38
treatment with an AFO, it would be prudent to evaluate
why results were contrary to what might be expected based
on the available evidence. However, beware that there are
many possible explanations for unexpected results including, but not limited to, issues with the measurement process, insufficient sensitivity of the chosen measure, the
intervention not affecting the construct being measured,
and ineffectiveness of the intervention. Exploring potential
reasons for unexpected results may lead to re-evaluation of
the approach to clinical care or the outcome measure.
There are many ways that an outcome measure can be
used to inform clinical practice. On an individual patient
basis, measures can be used to guide decisions regarding
care, helping to confirm whether or not clinical care is
consistent and has the intended effect.20,21 Measures can
be used to improve communication with the patient about
the effect of treatment or their progress with treatment
over time. This may encourage compliance with treatment especially if progress is slow and perspective as to
the improvements made hard to maintain. Similarly, it
may help to detect when undesirable changes have
occurred, alerting the clinician to potential problems.
Using standard assessment tools and documenting individual patient outcomes can also improve communication between care providers20,21 and provide information
to support medical necessity. When information from
outcome measures are aggregated across patients and
analyzed, it may be used to review quality of care or services21 and for benchmarking performance.20 Such information may be useful as part of quality improvement
activities,49 to achieve accreditation standards,50 or to
market services.
Application of outcome measures in the
orthotic management of stroke
Some of the above issues are illustrated in the following
hypothetical example. A busy orthotic practice decides that
they wish to assess walking speed with all post-stroke
patients being managed with lower limb orthoses because
the available evidence suggests that walking speed is one
outcome that should routinely improve with orthotic management post-stroke.7 The 10mWT is selected because it is
an appropriate measure of walking speed for post-stroke
patients, valid and reliable for the intended use, feasible for
routine use in clinical practice, and can be assessed with
and without the orthosis. A form is developed for inclusion
in the patient charts so that clinicians can readily track
changes in performance over time using this measure. A
training session is held to ensure that all clinicians are comfortable administering the measure and can do so consistently. Stopwatches are placed in an easily accessible
communal location and a 10-m walkway is permanently
marked along the clinic hallway to facilitate efficient and
consistent administration of the measure.
Prosthetics and Orthotics International 37(1)
All clinicians diligently perform the 10mWT with every
patient at every office visit and the results are documented
on the form. A staff member is tasked with entering the
results into a spreadsheet for review. A review meeting is
held six months after beginning data collection. All clinicians indicate that they appreciate having information that
documents their patient’s progress over time because they
can use it to communicate with their patients, referral
sources and third party payers, improvements made in
walking speed as a result of orthotic management. However,
they are frustrated that the 10mWT does not capture concerns some of their patients have with fatigue during walking. After some discussion, it is decided to review the
available outcome measures for a measure that assesses
endurance during walking. Following review, it is agreed
that the 6MWT will be administered at initial and summative visits for patients the clinicians identified as having
issues or potential issues with fatigue, in addition to the
10mWT, which would continue to be administered at every
visit. It was also agreed to hold another review meeting in
six months to assess if the added measure is helpful in
addressing the concerns raised and if the time to administer
the 6MWT is manageable.
In another hypothetical scenario, clinicians may be
interested in documenting more than just walking speed
and endurance, and look for a measure that can assess the
effect of orthotic management on the ability to walk in the
community where different environmental conditions may
be encountered. The clinician searches the available literature and outcome measure repositories to identify potential
measures that meet this need. From among the measures
provided in Table 4 as examples, the clinician may consider
the mEFAP, RMI, and Chedoke-McMaster Stroke
Assessment as potentially suitable measures because they
incorporate assessment of ICF mobility tasks related to the
negotiation of various environments and environmental
obstacles. Additionally, the clinician is able to ascertain that
development of all these measures has been described in
peer-reviewed journals; that with the exception of the
Chedoke-McMaster Stroke Assessment, they are free to
use, require no special training or equipment, but take varying times to administer; that the RMI is a physician-reported
measure and that the other two are performance-based
measures; that they all have reasonable psychometric properties and, with the exception of the Chedoke-McMaster
Stroke Assessment, they are valid for use in both acute and
chronic populations post-stroke; and that they all have
straightforward scoring procedures, although only the
mEFAP scoring accounts for changes in use of an orthosis
or assistive device over time.
The mEFAP appeals to the clinicians because they are
already familiar with administration of timed walking tests
and the TUG test and are intrigued by how the scoring may
account for changes in device use over time. For the
mEFAP, sub-tasks such as the 5mWT and TUG Test can be
39
Robinson and Fatone
Figure 2. Sub-scores and total mEFAP scores for three studies
of subjects ambulating without orthoses at different times post
stroke: subjects >90 days post-stroke from Sheffler et al.51;
subjects >1 year post-stroke from Liaw et al.52; and subjects a
mean of 33.5 months post-stroke from Hung et al.42
Figure 3. Sub-scores and total mEFAP scores for two studies
of post-stroke subjects walking with and without AFOs: subjects
>90 days post-stroke from Sheffler et al.51 and subjects a mean
of 33.5 months post-stroke from Hung et al.42
administered independently, potentially increasing flexibility of use in clinical practice. Even when the mEFAP is
administered as a whole, sub-task scores may be evaluated
individually along with the total score to provide additional
insights into performance. Using results from three studies
of patients at different times post-stroke ambulating without orthoses,42,51,52 differences in mEFAP performance by
sub-task and total score are discernible (Figure 2). These
data suggest that there is a hierarchy in the sub-tasks
wherein the 5mWT is relatively easy to perform and the
obstacle test relatively hard. The mEFAP has been used to
demonstrate improvements in mobility tasks with use of an
AFO42,51 (Figure 3) as well as to compare the effects of different orthotic interventions on functional ambulation poststroke51 (Figure 4).
The ability to account for use of an orthosis when scoring a measure is worth considering when selecting a
measure to assess orthotic management of stroke,
Figure 4. mEFAP sub-scores comparing the effects of both an
AFO and peroneal nerve stimulator (ODFS) to walking without
either device (from Sheffler et al.51). Total mEFAP scores shown
were calculated from the sub-scores presented in the paper by
Sheffler et al.51
especially if tracking changes in orthotic management
over time is important. When scoring the mEFAP, clinicians are able to account for changes in both use of
orthoses and manual assistance over time in a single
patient. Imagine a scenario where a sub-acute stroke
patient requires a solid AFO when the mEFAP is initially
administered, but uses a posterior leaf spring AFO when
the mEFAP is administered a year later. However, the
patient performs the mEFAP on each occasion in the same
amount of time. This could lead to the interpretation that
no improvement has occurred even though the patient has
progressed in terms of level of orthotic assistance needed
when performing the mobility tasks. In this situation, the
mEFAP allows for a standardized weighting of the score
to account for the fact that performance of the measure
was in fact completed with less assistance on the second
occasion.35
Conclusion
The selection and successful integration of outcome measures within clinical practice requires careful consideration.
To facilitate use of outcome measures in the orthotic management of stroke, we described a process for the judicious
identification of the measure construct, selection of outcome measures, and careful use in regular clinical practice.
Key points
•• A primary goal of orthotic management of stroke
is the improvement of mobility.
•• Using the ICF to identify mobility tasks can help
clinicians identify appropriate outcome measures
for the orthotic management of stroke.
40
Prosthetics and Orthotics International 37(1)
•• Selection and integration of outcome measures
into clinical practice is a process.
•• The process described can be used to select,
administer and review the results of clinically relevant outcome measures in the orthotic management of stroke.
6
7
Existing knowledge and what has been
learnt
Orthotists are under increasing pressure to evaluate the
effects of their interventions. A primary goal of orthotic
management of stroke is the improvement of mobility.
The ICF framework can be used to identify outcome
measures appropriate for the assessment of mobility
tasks after stroke. This is one step in a process required
to select, administer and utilize the results of outcome
measures in a clinical setting.
What remains to be accomplished
The usefulness of the proposed process to clinicians
should be evaluated along with the effect on patient
care of increased use of outcome measures in the
orthotic management of stroke.
8
9
10
11
12
Acknowledgements
The ideas presented in this paper would not have been possible
without many informative discussions with colleagues including
Beth Halsne (certified orthotist/prosthetic resident), Brian Hafner,
PhD, at University of Washington, and Kjell-Åke Nilsson,
Swedish orthotist.
13
Funding
14
This work was funded by the National Institute on Disability and
Rehabilitation Research (NIDRR) of the US Department of
Education under Grant No. H133E080009.
15
References
1 Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD,
Brown TM, et al. Heart disease and stroke statistics-2011
update: A report from the american heart association.
Circulation 2011; 123: e18–e209.
2 Olsen T. Stroke – understanding the problem. In: Condie E,
Campbell J and Martina J. 1st ed. Copenhagen, Denmark:
International Society for Prosthetics and Orthotics, 2004.
3 Mudge S and Stott NS. Outcome measures to assess walking
ability following stroke: A systematic review of the literature. Physiotherapy 2007; 93: 189–200.
4 Pollock C, Eng J and Garland S. Clinical measurement of
walking balance in people post stroke: A systematic review.
Clin Rehabil 2011; 25: 693–708.
5 Weerdesteyn V, de Niet M, van Duijnhoven HJ and Geurts
AC. Falls in individuals with stroke. J Rehabil Res Dev 2008;
45: 1195–1213.
16
17
18
19
20
Whiteside SR, Allen MJ, Barringer WJ, Beiswenger WD,
Brncick MD, Bulgarelli TD, et al. Practice analysis of certified practitioners in the disciplines of orthotics and prosthetics. www.abcop.org/certification/OrthotistsProsthetists/
Documents/PracticeAnalysis_SS04.pdf (2004, consulted
August 30, 2011).
Condie E, Campbell J and Martina J. Report of a consensus
conference on the orthotic management of stroke patients.
1st ed. Copenhagen, Denmark: International Society for
Prosthetics and Orthotics, 2004.
NHS Quality Improvement Scotland. Best practice statement
– August 2009: Use of ankle-foot orthoses following stroke.
www.healthcareimprovementscotland.org/programmes/
cardiovascular_disease/programme_resources/ankle-foot_
orthoses_stroke.aspx (2009, consulted August 30, 2011).
Bowers R and Ross K. Development of a best practice statement on the use of ankle-foot orthoses following stroke in
scotland. Prosthet Orthot Int 2010; 34: 245–253.
Stevens PM. Clinimetric properties of timed walking
events among patient populations commonly encountered
in orthotic and prosthetic rehabilitation. J Prosthet Orthot
2010; 22: 62–74.
Jette A and Haley S. Contemporary measurement techniques
for rehabilitation outcomes assessment. J Rehabil Med 2005;
37: 339–345.
Fitzpatrick R, Davey C, Buxton M and Jones D. Evaluating
patient-based outcome measures for use in clinical trials: A
review. Health Technology Assessment 1998; 2: 1–74.
Smith R, Darzins P, Steel C, Murray K, Osborne D and
Gilsenan B. Outcome measures in rehabilitation. Australia:
National Ageing Research Institute, 2001. www.health.
vic.gov.au/subacute/outcome_phase1.pdf (Accessed on:
December 26, 2011).
Gailey R. Predictive outcome measures versus functional
outcome measures in the lower limb amputee. J Prosthet
Orthot 2006; 18: 51–60.
Rehabilitation Services Plan Implementation Committee.
Outcome measurement for rehabilitation services.
Rehabilitation Services Plan Implementation Committee,
Canada, 1996. www.gnb.ca/0383/pdflibrary-e/Outcome_E.
pdf (Accessed on: December 26, 2011)
Wade DT. Outcome measurement and rehabilitation. Clin
Rehabil 1999; 13: 93–95.
Jette D, Halbert J, Iverson C, Miceli E and Shah P. Use of
standardized outcome measures in physical therapist practice: Perceptions and applications. Phys Ther 2009; 89:
125–35.
Dunckley M, Aspinal F, Addington-Hall JM, Hughes R and
Higginson IJ. A research study to identify facilitators and
barriers to outcome measure implementation. Int J Palliat
Nurs 2005; 11:218–225.
Kane R and Radosevich D (eds). Conducting health outcomes
research. Sudbury MA: Jones & Bartlett Learning 2011.
Stokes EK. Rehabilitation outcome measures. 1st ed. London:
Churchill Livingston Elsevier, 2011.
41
Robinson and Fatone
21
Kucukdeveci AA, Tennant A, Grimby G and Franchignoni
F. Strategies for assessment and outcome measurement in
physical and rehabilitation medicine: An educational review.
J Rehabil Med 2011; 43: 661–672.
22 Barak S and Duncan PW. Issues in selecting outcome measures to assess functional recovery after stroke. NeuroRx
2006; 3: 505–524.
23 Lennon S and Pomeroy VM. Outcome measures for orthotic
intervention in stroke rehabilitation. In: Condie E, Campbell
J and Martina J. Report of a Consensus Conference on
the Orthotic Management of Stroke. 1st ed. Copenhagen,
Denmark: International Society for Prosthetics and Orthotics,
2004.
24 Salter K, Jutai JW, Teasell R, Foley NC, Bitensky J and Bayley
M. Issues for selection of outcome measures in stroke rehabilitation: ICF activity. Disabil Rehabil 2005; 27: 315–340.
25 Salter K, Jutai JW, Teasell R, Foley NC and Bitensky J.
Issues for selection of outcome measures in stroke rehabilitation: ICF body functions. Disabil Rehabil 2005; 27: 191–207.
26 Salter K, Jutai JW, Teasell R, Foley NC, Bitensky J and
Bayley M. Issues for selection of outcome measures in stroke
rehabilitation: ICF participation. Disabil Rehabil 2005; 27:
507–528.
27 World Health Organization. Towards a common language
for functioning, disability and health (ICF). www.who.int/
classifications/icf/en/ (2002, consulted August 30, 2011).
28 Geyh S, Cieza A, Schouten J, Dickson H, Frommelt P, Omar
Z, et al. ICF core sets for stroke. J Rehabil Med 2004; 44
Suppl: 135−141.
29 Patla AE and Shumway-Cook A. Dimensions of mobility:
Defining the complexity and difficulty associated with community mobility. J Aging Phys Act 1999; 7: 7–19.
30 McDougall J, Wright V and Rosenbaum P. The ICF model
of functioning and disability: Incorporating quality of life
and human development. Dev Neurorehabil 2010; 13:
204–211.
31 World Health Organization. International classification of
functioning, disability and health: Children and youth version ICF-cy. Geneva: World Health Organization, 2007.
http://site.ebrary.com.turing.library.northwestern.edu/lib/
northwestern/docDetail.action?docID=10227085 (Accessed
on: 02/23/12).
32 Geyh S, Kurt T, Brockow T, Cieza A, Ewert T, Omar Z et al.
Identifying the concepts contained in outcome measures of
clinical trials on stroke using the international classification
of functioning, disability and health as a reference. J Rehabil
Med 2004; 44 Suppl: 56–62.
33 Geboers JF, Drost MR, Spaans F, Kuipers H and Seelen
HA. Immediate and long-term effects of ankle-foot orthosis
on muscle activity during walking: A randomized study of
patients with unilateral foot drop. Arch Phys Med Rehabil
2002; 83: 240–245.
34 Stevens P, Fross N and Kapp S. Clinically relevant outcome
measures in orthotics and prosthetics. The academy today,
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
p.A4–a11. www.oandp.org/AcademyTODAY/2009Feb/2.
asp (2009, consulted August 30, 2011).
Baer HR and Wolf SL. Modified emory functional ambulation profile: An outcome measure for the rehabilitation of
poststroke gait dysfunction. Stroke 2001; 20: 429–437.
Salbach NM, Mayo NE, Higgins J, Ahmed S, Finch LE and
Richards CL. Responsiveness and predictability of gait speed
and other disability measures in acute stroke. Arch Phys Med
Rehabil 2001; 82: 1204–1212.
Collen FM, Wade DT and Bradshaw CM. Mobility after
stroke: Reliability of measures of impairment and disability.
Int Disabil Stud 1990; 12: 6–9.
Mathias S and Isaacs B. Balance in elderly patients: The
‘get-up and go’ test. Arch Phys Med Rehabil 1986; 67:
387–389.
Podsiadlo D and Richardson S. The timed ‘up & go’: A test
of basic functional mobility for frail elderly persons. J Am
Geriatr Soc 1991; 39: 142–148.
Andersson AG, Kamwendo K, Seiger A and Appelros P. How
to identify potential fallers in a stroke unit: Validity indexes
of 4 test methods. J Rehabil Med 2006; 38: 186–191.
Flansbjer UB, Holmback AM, Downham D, Patten C and
Lexell J. Reliability of gait performance tests in men and
women with hemiparesis after stroke. J Rehabil Med 2005;
37: 75–82.
Hung J-W, Chen P-C, Yu M-Y and Hsieh Y-W. Long-term
effect of an anterior ankle-foot orthosis on functional walking ability of chronic stroke patients. Am J Phys Med Rehabil
2010; 90: 8–16.
Heinemann A. Rehabilitation measures database. www.
rehabmeasures.org (2010, consulted August 20, 2011).
Perry J, Garrett M, Gronley JK and Mulroy SJ. Classification
of walking handicap in the stroke population. Stroke 1995;
26: 982–989.
Schmid A, Duncan PW, Studenski S, Lai SM, Richards L,
Perera S, et al. Improvements in speed-based gait classifications are meaningful. Stroke 2007; 38: 2096–2100.
Bowden MG, Balasubramanian CK, Behrman AL and Kautz
SA. Validation of a speed-based classification system using
quantitative measures of walking performance poststroke.
Neurorehabil Neural Repair 2008; 22: 672–675.
Perera S, Mody SH, Woodman RC and Studenski SA.
Meaningful change and responsiveness in common physical performance measures in older adults. Journal of the
American Geriatrics Society 2006; 54: 743–749.
Tilson J, Sullivan K, Cen S, Rose D, Koradia C, Azen S,
et al. Meaningful gait speed improvement during the first
60 days poststroke: Minimal clinically important difference.
Physcial Therapy 2010; 90: 196–208.
Heinemann AW, Fisher W and Gershon R. Improving
healthcare quality with outcomes management. J Prosthet
Orthot 2006; 18: 46–50.
American Board for Certification in Prosthetics Orthotics
and Pedorthics. ABC Facility Accreditation Standards
42
2011. Alexandria, VA: American Board for Certification in
Prosthetics Orthotics and Pedorthics.
51 Sheffler LR, Hennessey MT, Naples GG and Chae J.
Peroneal nerve stimulation versus an ankle foot orthosis for correction of footdrop in stroke: Impact on functional ambulation. Neurorehabil Neural Repair 2006; 20:
355–360.
52 Liaw L-J, Hsieh C-L, Lo S-K, Lee S, Huang M-H and
Lin J-H. Psychometric properties of the modified emory
Prosthetics and Orthotics International 37(1)
functional ambulation profile in stroke patients. Clin
Rehabil 2006; 20: 429–437.
53 Siegle D. Instrument reliability - educational research.
www.gifted.uconn.edu/siegle/research/Instrument%20
Reliability%20and%20Validity/Reliability.htm (2011, consulted 8/23/11, 2011).
54 Toro B, Nester CJ and Farren PC. The status of gait assessment among physiotherapists in the united kingdom. Arch
Phys Med Rehabil 2003; 84: 1878–1884.