Sarah Durham
1
LOUISIANA STATE UNIVERSITY HEALTH
School of Allied Health Professions
DEPARTMENT OF REHABILITATION SCIENCES
PROGRAM IN OCCUPATIONAL THERAPY
OCCT 6625
Occupation Based Practice I
Spring 2014
Evidence Based Practice Assignment: Due the day of your assigned topic. See Article List for assigned
topic.
Article Analysis Format: Provide the following information on the article assigned. Use your own words
to answer the questions. Present your article findings during class at the appointed time.
Reference
Wuang, Y,P, Su, C,Y, & Huang, M,H. (2012). Psychometric comparisons of three measures for
assessing motor functions in preschoolers with intellectual disabilities. Journal of
Intellectual Disability Research, 56, 567-578.
Briefly state the objective(s) of the study. Why research this topic?
Children with intellectual disabilities (ID) are characterized by deficits in intellectual
functioning and delays in the development of adaptive behaviors. Children with ID may also
exhibit motor and perceptual dysfunctions that contribute to poor limb control, difficulty
executing purposeful movements, and problems initiating tasks. These motor deficits can disrupt
a child’s ability to perform activities of daily living independently, participate in educational
activities, and develop social relationships with peers. Therefore, therapeutic treatment in early
childhood is important to improve motor performance and facilitate participation in meaningful
occupations. However, in order to evaluate the effectiveness of intervention, therapists need to
use reliable and sensitive measures that provide consistent results and detect subtle changes in a
child’s motor performance.
Clinicians and researchers use a variety of instruments to assess motor function and
evaluate the effectiveness of intervention. The Bruininks-Oseretsky Test of Motor ProficiencySecond Edition (BOT-2), the Movement Assessment Battery for Children- Second Edition
(MABC-2), and the Peabody Developmental Motor Scale-Second Edition (PDMS-2) are the
most commonly used measures to asses motor function in children with disabilities. The
psychometric properties of reliability, validity, and responsiveness are not well known for each
of these measures for assessing a cohort of preschool aged children with ID. Therefore, the
objective of this study was to compare the reliability, validity, and responsiveness of the BOT-2,
MABC-2, and PDMS-2 for assessing motor function in a single group of preschoolers with ID.
Sarah Durham
2
What type of research was performed? What did the researchers do?
A single group of 141 children aged 3 to 6 with ID participated in a pediatric rehabilitation
program at least one day a week for six months. The therapy program offered motor training
intervention techniques including motor learning, sensory integration, and neurodevelopment
treatment. The researchers assessed the group of children with the BOT-2, MABC-2, and
PDMS-2 three different times throughout the study. Before the treatment program began, two
baseline measurements (T1 and T2) were performed within a one week interval; therapy
interventions did not occur between T1 and T2. After six months of participating in the pediatric
rehabilitation program, a follow up assessment was conducted for each of the three measures
(T3). Before conducting the re-test, the experimenters did not review the first assessment to
decrease the possibility of experimenter bias. Testing for each individual took place at the child’s
school, home, or pediatric occupational therapy unit at the hospital; testing occurred at the same
time of the day for T1, T2, and T3. In addition to the BOT-2, MABC-2, and PDMS-2, the study
used the physical tasks performance scale (PTPS) of the School Function Assessment-Chinese
version (SFA-C) as an external criterion-referenced assessment to detect clinically significant
change in function. The PTPS was conducted at T1 and T3.
The data for each measure was analyzed using a variety of psychometric tests. The score
distributions of each measure were analyzed for ceiling and floor effects. Reliability of each
measure was determined by examining the internal consistency, test-retest reliability, and
standard error of measurement (SEM). The internal consistency test determined the degree to
which a set of items in each instrument measured the same trait. Test-retest reliability was used
to test the stability of the three measures between T1 and T2, and SEM was used to determine
the precision of the three measures’ subtests and composite scores.
Construct validity of each measure was assessed by analyzing the concurrent validity and
predictive validity. Concurrent validity assessed the degree to which the outcomes of the BOT2, MABC-2, and PDMS-2 correlated with one another at the three time points of measurement
using the Spearman ƿ correlation. Predictive validity tested each measure’s ability to predict
future performance by correlating the outcome of the BOT-2, MABC-2, and PDMS-2 at T1 with
the scores of the PTPS at T3.
Responsiveness was evaluated by four different analyses: effect size (ES), standardized
response mean (SRM), minimal detectable change (MDC), and minimal clinical important
difference (MCID). ES was used to detect the magnitude of change between the initial and third
measurement time points. SRM indicated if the measures exhibited improvement or
deterioration. The MDC determined the smallest change in score that represented real change
rather than measurement error for each assessment. The MCID indicated the smallest difference
in a measured variable that represented clinically significant change in a participant’s
performance.
Sarah Durham
3
Describe the strength of the evidence. Level of research (use table in “Resource”)
The study focused on one, nonrandomized group of preschool aged children with ID and
implemented a pretest and posttest design to test the validity, reliability, and responsiveness of
the BOT-2, MABC-2, and PDMS-2; therefore, the study’s level of evidence was Level III.
Describe the study participants if appropriate (if not state, N/A and why it is not
appropriate)
A total of 141 children participated in the entire study. Originally, 191 children met the
inclusion and exclusion requirements, but 50 dropped of the study due to various reasons. Of the
141 children, 37% were females, and the average age was 42.9 months. 101 children were
described as having a mild ID and 40 had a moderate to severe ID. The primary caregivers of the
children had an average of 14.4 years of education, and the occupations of the caregivers were
classified into four major categories: professional or central administration, semi-professional
workers, technical workers, and semi-technical or non-technical workers.
The participants in the study were from southern Taiwan and were collected from five
public special schools, two child development centers, and three hospitals. To be selected for the
study, the children had to be between the ages of 3 and 6. The children had a diagnosis of ID
defined by a full-scale intelligence quotient less than or equal to 70 with limitations in adaptive
functioning. The children did not exhibit serious emotional or behavioral disturbances and had to
participate in occupational or physical therapy programs during the time of research. The
exclusion criteria included coexisting diagnoses of autism, cerebral palsy, blindness, and
deafness. Children with previous traumatic brain injury, muscular dystrophy, epilepsy, or other
neurological disorders were also excluded.
Describe the outcome measured
The PDMS-2 exhibited less ceiling and floor effects than the BOT-2 and MABC-2;
therefore, the PDMS-2 evaluates a wider range of motor functions. Reliability statistics revealed
the BOT-2, MABC-2, and PDMS-2 had excellent internal consistencies, which implied that
homogeneity of the subscales of each measure were sufficient. The test-retest analysis indicated
excellent reliability of each measure, and the SEM values revealed acceptable measurement
precision for the BOT-2, MABC-2 and PDMS-2.
The Spearman ƿ correlations for each pair of the three measures at the three time points
indicated high concurrent validity. The predictive validity of each measure was satisfactory as
the scores on three measures at T1 were moderately correlated the scores on the PTPS at T3;
therefore, the study showed that the BOT-2, MABC-2, and PDMS-2 were satisfactory measures
to predict future motor performance.
The PDMS-2 demonstrated the highest responsiveness of the three measures. The ES
and SRM values for the three measures indicated that each measure had moderate
responsiveness; the detected changes between T1 and T3 in each measure were significant. The
MDC values were used as reference values to determine if changes in a score on a measure for
Sarah Durham
4
an individual child represented real improvement. The study found the MDCs of each measure
were less than 20% of each measure’s high score, designating an acceptable level of
measurement error. To detect real change, the total standard score needed to be greater than 8
points for the BOT-2, greater than or equal to 2 for the MABC-2, and greater than or equal to 8
points for the PDMS-2. MCID values showed that the BOT-2, MACB-2, and PDMS-2 had
acceptable discriminative properties to detect significant clinical change. Therefore, the
responsiveness analyses indicated each measure’s ability to accurately detect subtle changes in
the motor performance of children with ID in a 6 month interval.
Overall, the BOT-2, MABC-2, and PDMS-2 exhibited acceptable levels of reliability,
validity, and responsiveness; however, the PDMS-2 is recommended as an assessment for
preschoolers with ID because it was more sensitive in evaluating change over time and exhibited
more acceptable levels of measurement error. The PDMS-2 gains advantage over the BOT-2 and
MABC-2 as it provides quantitative and qualitative analysis of a child’s motor function. The
PDMS-2 also allows the examiner to start testing at the measurement item that is closest to the
child’s developmental skills.
What are the merits of the study?
Knowledge of the psychometric properties of the BOT-2, MABC-2, and PDMS-2 is
important to allow occupational therapists to trust that the outcomes of each measure accurately
reflect a child’s level of motor function. These measures often assist in determining a child’s
educational diagnosis, which affects the child’s ability to receive special education services.
Therefore, being able to accurately assess a child’s current skill level affects the prediction of
future performance skills and the development of appropriate goals and interventions. Also,
knowledge of the reliability of each instrument is important as occupational therapists often use
the BOT-2, MABC-2, or PDMS-2 as a baseline measure and follow up measure to evaluate the
effectiveness of treatment. Because each instrument has moderate responsiveness, the therapists
can expect that changes in motor function are clinically significant. Therefore, therapists can
analyze the changes in the client’s performance in order to modify intervention to meet the
specific needs of the child or discharge the child based on optimal improvement in occupational
performance.
What are the study’s limitations?
One limitation of the study was that the sample only included children 3 to 6 years old.
The study did not include children representative of the ages the three measures intended to
assess. Therefore, to fully analyze the reliability, validity, and responsiveness of the BOT-2,
MABC-2, and PDMS-2 , school-age children and adolescents with ID should be included in
future studies.
Sarah Durham
5
Glossary (list any unfamiliar or operationally defined terms from the article)
Ceiling effect- a measurement limitation of an instrument in which the scale cannot determine
increased performance above a certain level.
Construct validity – the degree to which a test is measuring what it was designed to measure
Criterion referenced assessment –measure of level of performance skills in relation to welldefined domain of content. Provides description of specific skills each child can perform.
Floor effects- a measurement limitation of an instrument in which the scale cannot determine
decreased performance below a certain level.
Psychometric properties- quantifiable attributes that detect the statistical strength and weakness
of a measurement
Reliability- degrees of an assessment tool to produce consistent and stable results
Responsiveness- describes the ability of an assessment instrument to detect significant changes
in performance over time
Spearman ƿ correlation- used to identify and test the strength between two sets of data