Laboratory Measures of
ADHD
Adam B. Lewin
November 19, 2003
What are Laboratory Measures?
Laboratory Measures
• Techniques where behavior is observed
under standardized conditions
– Usually involving stimuli designed to evoke
the specific behavior of interest
• Excluded:
– Naturalistic observations in unstandardized
conditions
– Techniques designed primarily to elicit & observe
physiological responses.
(Frick, 2000 – J Clin Child Psychology)
Laboratory Measures
• Direct, systematic behavioral
observations conducted in a clinic or
research setting where efforts have
been made to approximate more
naturalistic situations (e.g., school,
home, etc.).
(Barkley, 1991 – J Ab C Psychology)
• Often limited ecological validity
Laboratory & PerformanceBased Measures
• A Broader Perspective:
– Techniques suitable for use in
research settings
– Often limited implementation clinically
• Time consuming
• Non-diagnostic
– Despite clinical utility
Ecological Validity
• The extent to which predictions
based on a laboratory measure can
be extended to naturalistic setting
– e.g., Does the result of a laboratory
task reflect actual attentional
problems?
Ecological Validity: Barkley’s
Criteria
• 1. Does the laboratory measure show differences
between ADHD & control groups?
• 2. Correlations with other laboratory measures
with well-established ecological validity
• 3. Sensitivity to experimental manipulations know
to affect the criterion
• 4. Correlations with ecological criteria
Laboratory Measures
• 142 studies comparing ADHD & normal
controls
– 439 comparisons
– Variety of tasks & neurocognitive tests
• Plethora of instruments employed
– CPT, WISC-R, WCST, MMFT, Stroop
» (Rapport et al., 2000)
Laboratory Measures
• Reliable vs. unreliable instruments for
detecting group differences
– Reliable: CPT, WISC-R Coding & Arithmetic,
Visual recall tasks
– Unreliable: Tapping, pegboard, Trails A, WISC-R
Mazes
• Distinguishing characteristics:
–
–
–
–
Involve recall and/or recognition
Require use of the phonological loop
Pacing – experimenter controlled
Response stimulus not continuously displayed
» (Rapport et al., 2000)
MEASURES OF ATTENTION
Continuous Performance Tasks
• Originally designed to detect lapses in
attention during seizures.
» (Rosvold et al., 1956)
• Usually computerized assessments of
sustained attention
• Can be visual, auditory, numerals,
characters, shapes
• Subject must respond to target
embedded in a series of distracter
stimuli
Continuous Performance Tasks
• Three General Models:
» (Rapport, 1993)
• X-Version – respond to a target stimulus
– Reverse X-Version: inhibit response
to a stimulus
• AX-Version – respond to a target
stimulus only when it is preceded by a
different target
• Double Letter Version – respond only to
an immediately repeated stimulus
» (Friedman et al., 1978)
Continuous Performance Tasks
• Assessment of sustained attention:
– Number of Correct Responses
– Errors of Omission (EO)
• Number of Target Stimuli Missed
• Believed to assess sustained attention & impulse
control:
– Errors of Commission (CE)
• Responding after a non-target stimulus
» (Sostek, Buchsbaum, & Rapoport, 1980)
• Weakly correlated with error scores from the MFFT
• Both CEs and OEs significantly correlated with
CPRS & CTRS Hyp & Inattn scales
» (Barkley, 1991)
Conners’ CPT
•
•
•
•
Computer-assisted assessment of attention
14 minutes
X & Reverse X Versions; AX Version
6 Trial Blocks; 3 Sub-blocks per trial
• 20 Trials Each
• Interstimulus interval varies from 1, 2 or 4 seconds
Conners’ CPT
• Numerous output data:
– Correct responses, OEs, CEs, reaction times,
Index score
– CPT-II: Clinical Confidence Interval
• Norms for general population and children 418 diagnosed with ADHD
• Low false positives and negatives (<10-15%)
• Practice effects are minimal
• Sensitive to pharmacological treatment
changes
Conners’ CPT
• Some Advantages/Disadvantages
– Avoids false negatives by frequent target presentation
– Chance of impulsive errors maximized due to the
continuous level of responding
– Questionable ecological validity
• Correlates with analogue measures of attention (.25.35)
• Relates to parent & teacher ratings of inattention &
hyperactivity
– CPRS/CTRS & CBCL
» (Barkely 1991)
Conners’ CPT II
• Normative sample of 2,686 clinical and nonclinical
subjects.
• T-Scores & the following classifications:
– markedly, moderately or mildly atypical, within the
average range, and good or very good performance.
• New confidence index that is the percentage out of
100 clients that would be correctly classified based
on a profile.
• CPT-II provides an overall index, for research
comparisons with the CPT I
Ages 6 years and older
– (Kiddie CPT for ages 4-5).
TOVA
• Test of Variables of Attention (TOVA)
» (Greenberg & Waldman, 1993)
• Two 11-minute computerized tasks (one for
children under age 5)
• Easily discriminated visual stimuli
– Square with a small square adjacent either to the
top (target) or bottom
• Attempts to eliminate confounds due to
learning difficulties
TOVA
• First task presents the target infrequently (1:3
½)
– Designed to elicit boredom & thus measure
sustained attention
• Second task presents the target frequently (3
½ :1)
– Designed to measure impulsivity
• Normative data in 2-year intervals for children
4-19 (10 year intervals for adults)
• Diagnostic utility not well documented
Auditory CPT
• Tape of a 96-word list, of 20 different
monosyllabic words, read 6 times
• Target word is “dog”
• 20 times per 96-word trial
• Respond by giving “thumbs up”
• Test-retest only .67-.84
• Scoring is difficult
Gordon Diagnostic System
• Gordon Diagnostic System (GDS)
» (Gordon, 1979)
• Portable machine
• Visual
– Vigilance (numerical AX task)
– Distractibility (Random numbers flash in proximity to
the target)
– Delay (points awarded for delaying response at least 6
seconds)
• Measure of response inhibition
• Correlates with hyperactivity ratings by parent &
teachers
• Not proven sensitive to medication effects
GDS
• Auditory
– Vigilance Task: Subject responds to
numbers that are heard instead of seen
– Interference Task: Random number
through the headphones. The subject
performs on the standard Vigilance (or
Distractibility) tasks while having to contend
with the confusing auditory input.
IVA
• Intermediate Visual & Auditory CPT
(IVA)
• Half the targets are visual (the
characters are a "1" and a "2“) and half
are presented audibly through the
computer's speaker
• 13 minutes
Other CPT Tasks
• Children’s Checking Task (CCT)
» (Margolis, 1972)
– Paper & Pencil
– “Cancellation Task”
– Mark numbers listed in rows on a page as they are
read on a recording
– Circle discrepancies between the list & recording
• 30 minutes; Scores include OE & CE
• Strong correlations with other measures of
attention
• Better ecological validity than other CPTs
Other CPT Tasks
• Matching Familiar Figures Test (MFFT)
» (Kagan, 1966)
• Measure of attention and impulse control
– Match-to-sample task
– Identify the identical matching target picture from an
array of six similar stimuli
– 12 or 20 trial versions
– Measures of response latency and errors
• Fails to differentiate ADHD from controls &
medication effects
• Adolescent norms unavailable
• Not recommended for clinical use
TEA-Ch
• Test of Everyday Attention for Children (TEACh; Manly et al., 1999)
• Norms for ages 6-16
• 9 Game-like subtests
• 3 Domains
– Selective Attention (2 subtests)
– Sustained Attention (5 subtests)
– Attentional control/shift (2 subtests)
• Approximately 2 hours to complete; subtests
can be administered individually
TEA-Ch
• Children with ADHD show significant
impairment on sustained attention &
attentional control tasks (compared to
clinical controls)
• Differences on selective attention tasks
not significant.
» Heaton et al., 2002
Neuropsychological Tests
Neuropsychological Measures
• Stroop Word-Color
– Timed test measuring the ability to inhibit or suppress
automatic responses
– High % of false negatives (53%)
» (Barkely & Grodzinsky, 1994)
• Trail Making Test
– Trails B – Attentional Shift
– Very high false negatives (80-82%)
– Overall classification <54% correct.
» (Barkely et al., 1992; 1994)
• Mixed results
• Not consistent in identifying group differences
Neuropsychological Measures
• Wisconsin Card Sorting Task (WCST)
– Computerized/manual administration
– Participant must correctly sort a series of colored
geometric shapes according to an set of rules
unknown to the subject.
– After each “sort,” the only feedback is
correct/incorrect
– Rules must be deduced from this feedback
– Rules change on each successive trial
– Requires an ability to shift attention
» (Mirsky et al., 1991; Heaton et al., 1993)
– Not recommended for diagnostic use.
• False negative 61-89%;
MEASURES OF
ACTIVITY
Measuring Activity Level
•
Two primary classifications:
1. Binary Devices – Respond in an “all or
nothing” manner when movement
exceeds a threshold value
2. Proportional Devices – Measure motor
activity in direct proportion to the
magnitude of movement.
(Tyron, 1984)
Measuring Activity Level
Binary Devices
• Mercury Switches
–
–
Position change sensors
The “wiggle chair”
–
–
•
Pedometers
–
•
Not consistent in identifying group differences
Not related to parent hyperactivity ratings
Activated by the impact of the foot & ground
Photoelectric cells
Actometers
Proportional Devices
• Actometers – ankle or wrist
– Modified self-winding watches
– Movements of the limb corresponds to
movement of the watch’s hands
– Sensitive to stimulant drug effects
– Laboratory actometer ratings not significantly
related to parent ratings of hyperactivity at
home
– Ankle actometers relate to CPT CEs.(.37)
»
(Barkely et al., 1975; Ullman et al., 1978)
Parent Rating Scales
Rating Scales
• Should address aspects of the following:
–
–
–
–
core features of ADHD
symptom severity and development
level of impairment
comorbid conditions
• Advantages:
–
–
–
–
–
Standardized
Decreases subjectivity
Cost-effective method for multiple informants
Can be completed prior to evaluation
Access to infrequently displayed behaviors that
may be missed in observation periods
» (Anastopoulos 2001)
Rating Scales
• Potential limitations:
– Assume informant is familiar enough with the
subject to accurately complete the measure
– The informant must be able to understand the
questions
– Adult psychopathology may distort parent
perceptions of the child
– Parental or teacher tolerance of behavior may
influence ratings.
» (Anastopoulos 2001; Sattler 2002)
Rating Scales
• Conners’ Rating Scales
– Parent & Teacher revised versions
• Children 3-17
• Parent 80 items (27 on the Short-form)
• Teacher 59 items (28 on the Short form)
• Short form has limited scales – focus on ADHD/ODD
symptoms
– Self-report for adolescents 12-17
• Conners-Wells Adolescent Self Rating Scale
• 87-item (27 on the short form)
– Male & female norms in 3 year intervals
– Rating on a 4-point scale
Rating Scales
•
•
•
•
Conners’ Rating Scales
Ratings based on the previous month
Excellent psychometric properties
Simple comparisons between teacher & parent
versions
• ADHD Index (parent form): 12 items
• Hyperactivity Scale moderately related to total
hyperactivity score during analogue
observation
» Barkely 1991
Rating Scales
• Behavioral Assessment Scale for Children
(BASC)
» (Reynolds & Kamphaus, 1992)
• Parent, teacher, and self ratings scales;
Student observation system
• Ratings over the previous 3 months
• Preschool, child and adolescent versions
• 130 items on a 4 point scale (parent & teacher)
• 170 True/False items for the self-report form
Rating Scales
• BASC
• Scales include: adaptability, aggression,
anxiety, attention problems, atypicality,
conduct problems, depression,
hyperactivity, leadership, learning
problems, social skills, somatization,
study skills, & withdrawal
• Provides index of Adaptive Skills
Rating Scales
• BASC
• Excellent psychometrics; correlates highly
with the CBCL & Conners’; moderately
with the PIC.
• More predictive of ADHD status than the
CBCL
• 88% of the sample correctly identified as
ADHD (using the Attention subscale)
» (Ostrander et al., 1998)
Rating Scales
• Achenbach Child Behavior Checklist (CBCL)
• Recently revised with new normative sample
• Parent, teacher, & self-rating form of behaviors
over the past 6 months
• Externalizing, Internalizing, and Total Problem
Scale
– Attention & hyperactivity profiles
• Perhaps the most frequently used broadbanded measure in research
» Anastopoulos & Shelton, 2001)
Rating Scales
• DSM-IV SNAP-IV ADHD Checklist
» (Swanson, 1992)
• For parents, teachers, caregivers
• DSM-IV symptoms on a 4 point rating
scale
• Does not include:
– rating on impairment in function
– information of symptoms across setting
– symptom chronicity ratings
– symptom onset data
Rating Scales
• SNAP-IV
• Sample items:
– Does not seem to listen when spoken to
– Often “on the go” or acts as if “driven by a
motor”
– Often has difficulty waiting for a turn
• Research screener for ADHD
• Adult version is available
Rating Scales
• Personality Inventory for Children
– 280/420 item parent rating form (True/False)
– Lacks an inattentive subscale
• Devereux Scales of Mental Disorders
– Caregiver rating form
– 5 point scale based on the previous month
– Only 4 inattention items, 3 impulsivity items & 3
hyperactivity items
– No hyperactivity or impulsivity subscale
– Modest psychometrics
Rating Scales
• Behavior Rating Inventory for Executive
Function (BRIEF; Gioia et al., 1996)
• Designed to assess several aspects of
executive functioning
– Inhibition, Shift, Emotional Control, Working
Memory, Planning/Organization, Organization
of materials, Monitoring
• For children ages 5-18
• 86 items; Parent & Teacher versions
Rating Scales
• BRIEF
• May help differentiate ADHD subtypes
» (Barkely, 1997)
• May be useful in identifying difficulites associated
with ADHD (e.g. poor behavioral initiation;
planning, organization)
• Working Memory & Inhibit Scales moderately
predictive of ADHD diagnosis (Predominately
Inattentive or Combined Type)
Structured
Interviews
Structured Interviews
• Diagnostic Interview Schedule for Children IV
(DISC-IV)
» NIMH 1997
• Developed for use in epidemiological studies of
childhood behavioral disorders
• 6 major sections; 24 diagnostic modules
– 30 DSM-IV diagnoses may be generated
• Graded question format
–
–
–
–
Questions on symptom onset, severity, duration
Stem questions asked of everyone
Contingent questions
Primarily responded to by yes or no
Structured Interviews
• DISC-IV
• Administered to parents of children 6-17
– Youth version for ages 9-17
•
•
•
•
45-90 minutes administration time
Clinician administered, computer entered
Good reliability & validity
Gathers information on ADHD symptoms &
comorbid disorders
• May result in an overestimation of psychiatric
symptoms
Structured Interviews
• Diagnostic Interview for Children &
Adolescents-IV (DICA-IV)
» (Reich et al., 1996)
• Screening measure for psychiatric disorders
• Revised to address major child and adolescent
DSM-IV diagnoses
• Clinician administered, computer entered
• Parents of children age 6-17 years
• 28 Diagnostic Categories; 5-20 minutes each
• 6 “High Risk” areas, e.g., CD, alcohol & street
drug use, MDD, suicide, PTSD
Structured Interviews
•
•
•
•
DICA-IV
Two child forms available
Allows for probing beyond simple yes/no format
Requires more clinical skill & expertise than the
DISC to administer
• Clinicians have the option to not administer
certain categories
• Critical item screen
Structured Interviews
• Schedule for Affective Disorders and
Schizophrenia (K-SADS)
» (Puig-Antich & Chambers, 1978)
• Semi-structured interview for parents and
children
• Children ages 6-17
• Typically takes 30-90 minutes
• K-SADS-PL – version most useful for ADHD
assessment
» (Kaufman et al., 1997)
Structured Interviews
• K-SADS
• Unstructured introductory interview
– Demographics, present complaints, development,
prior psychological treatment, family/peer
relationships, academic functioning & hobbies.
• Screening Interview (82 symptoms divided
into 20 diagnostic areas)
– Rate symptoms with regard to current & most
severe past occurrence
– 27-28 questions on ADHD section
• Diagnoses based on both parent & child
report
Structured Interviews
• K-SADS
– Reliability good for ADHD (.63 for present
diagnosis; .55 for lifetime diagnosis) but slightly
less than structured interviews.
• Assesses age of onset, impairment, crosssituation criteria, academic achievement.
• No absolute guidelines for resolving informant
discrepancies
• Requires substantial training relative the
DISC & DICA
Behavioral
Observation
Systems
Behavioral Observation
• Why observe?
– Parent evaluations may be biased due to
distress or other psychological factors
– Intense behaviors may be perceived as
more frequent
– Increased objectivity
• Ecological validity vs. experimental
control
Behavioral Observation
• Direct observation of a child’s behavior
– Analogue/laboratory settings
– Naturalistic observation
• “Checklist” vs. Anecdotal Observation
– ABC data is useful in understanding overall behavioral
patterns
– More intensive for the observer
• Data collection
–
–
–
–
Interval
Frequency
Duration
Latency
Behavioral Observation
• Requires clear operational definitions of
target behaviors
– Objective, observable characteristics
– Replicable (clear; unambiguous)
– Discriminating information; eliminate judgment
» (Baer, Wolf & Risley, 1968)
• Observer training and agreement
Behavioral Observation
• BASC Student Observation System
• Coding of classroom behaviors
• 15 minute observation period
– Checklist of 65 adaptive and maladaptive
behaviors
– Occurrence of behaviors during 30 3-second
intervals
– Additional observations (e.g .teacher reaction)
Behavioral Observation
• BASC Student Observation System
– Good for initial assessment
– No Normative Data
– Brief sampling window
– Difficult to examine behavioral contingencies
Behavioral Observation
• ADHD Behavioral Coding System
» (Barkely; 1990)
• Laboratory-based
• Child is observed completing an academic
task
– Instructed to work on the assignment
– Stay seated
– Do not touch toys & stimuli in the room
Behavioral Observation
• ADHD Behavioral Coding System
– 15 minute observation
– Off-task behavior, fidgeting, leaving the seat,
vocalizing, or playing with the toys
– 30-second interval recording
– Can be used to code behavior during the CPT
Behavioral Observation
• ADHD Behavioral Coding System
– Good inter-observer agreement (.77-.85)
– Children with ADHD tend to score higher
– Sensitive to medication effects
– Lacks normative data
– Questionable ecological validity
– Interval/frequency recording
General Conclusions
- When incorporating laboratory measures
into clinical assessments, keep in mind
ecological validity.
- In case of disagreement, consider the most
ecologically valid source of information
- Diagnosis and treatment planning should
not be based on a single piece of
information.
END