Figuring out whether that’s working for your child Single subject designs for families Theodore A. Hoch, Ed.D., B.C.B.A. George Mason University Goals of today’s talk Help you to be an informed and discerning consumer Give you ideas as to how to figure out whether the therapy options you’ve chosen for your child are working Autism is diagnosed when … At least six symptoms, including: At least two of these: A) nonverbal behavior impairment; B) relationships impairment; C) lack of spontaneous interaction; D) lack of social reciprocity And, at least one of these: A) delay in onset or lack of speech; B) difficulty initiating or sustaining a conversation; C) stereotyped or repetitive language; D) lack of varied, imaginative, imitative play And, restricted, repetitive, and stereotyped interests, to include at least one of these: A) stereotyped interests; B) inflexible routines and rituals; C) stereotyped mannerisms; D) preoccupation with parts of objects And onset of delayed functioning prior to 3 years of age in at least one of these: A) social interaction; B) conventional social communication; C) symbolic or imaginative play So, At least 7982 combinations of presentations that can correctly be called “autism” This is where we get a spectrum! With 7982 different presentations: Is there an “average kid with autism?” Is your child “the average kid with autism?” Will what works for “the average kid with autism” work with your child? How to find what will work for your child Look for empirically supported interventions Where? Internet POAC website and listserv Pediatrician / Neurologist Friends / neighbors / family TV / Radio Empirically Supported Interventions Catch-phrase for the 2000s? Backed by well-done research How to access it? Google, web pages Professionals People you trust Original sources GMU library, other libraries Why is research done? To answer questions, such as: What happens if … Which of these is better … Research is not done to … Find a predetermined solution Produce results that will support a particular intervention When should research be done? Before a practitioner recommends or prescribes a particular course of treatment for another person Exception: the course of treatment is experimental, and the person who is the recipient of the treatment has given fully informed consent So I go to one of those places and find some research. How do I know I have an empirically supported intervention? Validated by group design research Sufficient sample size Random sampling Random assignment Well-chosen, well-implemented experimental design and controls Reliable measurement Well-chosen, well-conducted and interpreted statistical analysis Replication in peer-reviewed journals How do I know I have an empirically supported intervention? (continued) Group design research Where to find? Outcome? Types of questions answered Which works better, on average? Which affects whom, on average? Who says / does / makes / chooses what? Questions about groups Applicability to your child Feasibility for families? Okay. So I go to a different source. How do I know I have an empirically supported intervention? Validated by single subject design Procedure described so you can do exactly what the authors did Authors describe where they did the study, so you can compare their setting with yours. Authors measure and report data on how well they actually implemented their interventions. Data and design indicates intervention (and not other factors) changed the behavior Findings replicated At least 5 studies in peer-reviewed journals, At least 3 different sets of researchers At least 20 different participants. How do I know I have an empirically supported intervention? Single subject design Can involve one, a few, or many participants Focus always on individual participant Typically, no statistics Where to find? Outcome? Types of questions answered Which works better for this person? How does this work for this person? What happens if I do this, with this person? Questions about the individual and the intervention Applicability to your child Feasibility for families? Controversial interventions Controversial = produces lively discussion and disagreement Good? Bad? Work? Don’t work? In peer-reviewed journals? Less scholarly sources? Examples Can be effective, for some individuals (just like any other intervention) Types of interventions ABA / VB Auditory Integration Attachment / holding Typical special ed Dietary Specialized special ed Medication Occupational therapy Chelation Music therapy Vitamin / megavitamin Speech therapy Sensory Integration Hippotherapy Facilitated Communication Aquatic therapy Others? Empirically Supported v. Controversial Any therapy can be one, the other, or both. Will an empirically supported intervention work for your child? Will a controversial intervention work for your child? Does it really matter, as long as it works? How do I know whether it’s working for my child? Go by my impressions Do the research Using Research Designs to Evaluate Your Child’s Intervention Group design Random selection Random assignment Sufficient number of subjects Statistical Analysis Single subject design Work with who you have Interested in who you have Your subject is her or his own control Visual inspection of graphs – no statistics analysis (usually) Using single subject designs to evaluate your child’s intervention Interested in effects on behavior of individuals – not the average person! Your child provides Baseline data = data collected before you start / in absence of the intervention Intervention data You compare your child’s baseline data with your child’s intervention data Draw conclusions about how it works with your child Using single subject designs to evaluate your child’s intervention Participants and settings described thoroughly Were research participants similar to your child? Was procedure done in a setting similar to your child’s? Were procedures implemented by people similar to those who’ll implement your child’s? Were data collection procedures and equipment similar to those you’ll use? Were professional supports used similar to those you can access? Using single subject designs to evaluate your child’s intervention Dependent variable – what you’re interested in changing Usually, your child’s behavior Operationally defined What to do if professional opposes this? Operationally defining dependent variable Example: Intervention is semi-monthly IM injection and daily subcutaneous injection of Lupron to decrease symptoms of autism Which symptom(s)? Impulsivity What does person do that gets you to say he’s behaving impulsively Hits, kicks, pinches, runs away when asked to do something, destroys things, takes others’ things Dependent variable is these behaviors Operationally defining dependent variable Example: Intervention is brushing protocol to decrease tactile defensiveness How do we know tactile defensiveness is happening? Removes clothing when it is not time to do so Target is percentage day dressed, number of times clothing removed per day, duration dressed, etc. Operationally defining dependent variable Example: Intervention is a proton pump inhibitor taken to decrease irritability, vomiting, and food refusal that may be related to gastroesophageal reflux disease How do we know irritability is happening? Crying, whining, complaining Target is crying, whining, complaining, proportion of meal eaten, presence / absence of vomiting Operationally defining dependent variable Example: Intervention is a differential reinforcement procedure to improve compliance How do we know compliance is happening? Child initiates doing what parent or teacher asked within 5 sec of parent or teacher asking Target is count / percentage of times child did what was asked within 5 sec of having been asked Operationally defining dependent variable Describe exactly what you see and hear when the thing you’re interested in is happening Using single subject designs to evaluate your child’s intervention Dependent variable Repeated measurement Need at least three measures for a trend No conclusions based on a single measure Can differ, in medical situations Blood levels, range of motion measures, videoesophagram observation, counts of aggression, cold probes of correctly answering “Wh” questions How to Measure Choose a system you can do Well Regularly In the settings where it will need to be done By the people who’ll need to do it With the resources you have When you need to do it Reliably How to Measure Direct measures Count – how many Duration – for how long Latency – started when, in relation to what Extensity – how big / over what area Rate – how fast / slow / many per ________ Intensity – with how much force Interepisode time (interresponse time) – how much time in between How to measure Indirect measures – samples, estimates, or products of phenomenon of interest Partial interval sampling Behavior too frequent to count, competing responsibilities preclude counting Momentary time sampling Behavior too frequent to count, partial interval sampling not feasible Permanent products Behavior produces distinct products, nothing else produces those products How to measure – Partial interval sampling Decide on overall observation time Divide overall observation time into equal observation intervals Observe Record The first time the dependent variable occurs during an interval Whether or not the dependent variable occurred during an interval Convert to percentage intervals of occurrence How to measure – Partial Interval Sampling Dependent variable = rumination 16 hour observation time, 72 fifteen minute intervals Record R in cell if rumination was observed during that time, N if no rumination was observed during that time Convert to percentage intervals occurrence How to Measure – Partial Interval Sampling How to measure – Momentary Time Sampling Decide on overall observation time Divide overall observation time into equal observation intervals Set countdown timer to signal end of interval Timer sounds Observe Record if dependent variable is happening at that moment Do not record what is happening between times when the timer sounds! Reset timer, continue Partial Interval Sampling versus Momentary Time Sampling Estimate Estimate Potentially more effortful, more accurate than momentary time sampling Said to overestimate Potentially less effortful, less accurate than momentary time sampling Use when purpose is to decrease dependent variable, more rigorous procedures not feasible May underestimate Use when more rigorous data collection procedures are not feasible Converting Partial Interval Sampling to Scatterplots How to Make a Scatterplot Use Table feature in Word (or just draw a grid) Horizontal Axis – Day of the Week (or month) Vertical Axis – Time of Day (equal intervals) Color code dependent variables Shade in cell with relevant color when dependent variable happens Examine for patterns of occurrence (and nonoccurrence) Draw phase dividers, label conditions, when you change conditions Continue data collection, inspection, and decision making Using single subject designs to evaluate your child’s intervention Dependent variable Ensure reliability Define clearly Train data collectors Conduct reliability observations Take corrective / congratulatory action based on reliability observations Using single subject designs to evaluate your child’s intervention Independent variable – the intervention Describe clearly – put it in writing and refer to it often! Train implementors until you see them actually doing it correctly Check periodically to make sure it’s still being done correctly Introduce at the right time Your data tell you when … 25 19 17 15 Day 23 Intervention Baseline 21 Intervention 13 Baseline 11 16 14 12 10 8 6 4 2 0 Number of X by Day 9 Make them usable and keep them handy! Cum # Responses 4 11 14 15 17 1456 3813 6032 8391 11062 13746 16214 18138 18151 18153 18157 20135 22820 25318 28116 31010 33958 36333 36523 36525 36529 36532 38456 41134 43880 46558 49533 52545 52632 52637 52640 54685 57549 60342 60406 62898 62951 65699 65768 68236 68308 70989 71057 73924 7 Your data and your graphs are tools! # Responses 4 7 3 4 2 1439 2357 2219 2359 2671 2684 2468 1924 13 2 4 1978 2685 2498 2798 2894 2948 2375 190 2 4 3 1924 2678 2746 2678 2975 3012 87 5 3 2045 2864 2793 64 2492 53 2748 69 2468 72 2681 68 2867 5 Condition Saline Saline Saline Saline Saline Dzp Dzp Dzp Dzp Dzp Dzp Dzp Sailine Saline Saline Saline Dzp Dzp Dzp Dzp Dzp Dzp Dzp Saline Saline Saline Saline Dzp Dzp Dzp Dzp Dzp Dzp Saline Saline Saline Dzp Dzp dzp Saline Dzp saline dzp Saline dzp saline Dzp Saline Dzp 3 Date 1.21.05 1.22.05 1.23.05 1.24.05 1.25.05 1.26.05 1.27.05 1.28.05 1.29.05 1.30.05 1.31.05 2.1.05 2.2.05 2.3.05 2.4.05 2.5.05 2.6.05 2.7.05 2.8.05 2.9.05 2.10.05 2.11.05 2.12.05 2.13.05 2.14.05 2.16.05 2.17.05 2.18.05 2.19.05 2.20.05 2.21.05 2.22.05 2.23.05 2.24.05 2.25.05 2.26.05 2.27.08 2.28.05 3.1.05 3.2.05 3.3.05 3.4.05 3.5.05 3.6.05 3.7.05 3.8.08 3.9.05 3.10.05 3.11.05 1 Session 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 Number Graphing your data – Why? Graphing your data – How? Draw a grid (or use graph paper, or use Excel) Horizontal axis = days or sessions, with equal intervals Vertical axis = dependent variable measure, with equal intervals Plot data point as soon as you get it Connect the dots (but not across phase dividers) Phase dividers and condition labels Examine your data for: Level Treatment Baseline Number of Responses by Session Baseline Treatment Baseline Treatment 18 14 12 10 8 6 4 2 Session 29 27 25 23 21 19 17 15 13 11 9 7 5 3 0 1 Number of Responses 16 Examine your data for: Trend 18 Number of Responses by Day Treatment Baseline 16 14 Number of Responses by Day Treatment Baseline 120 Number 12 10 8 6 4 2 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Day Number Number of Responses by Day 60 Treatment Baseline 40 12 20 10 8 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Day Number Responses 0 80 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Day Examine your data for: Variability Baseline Number of Responses by Day Treatment Number of Responses 120 100 80 60 Less variability more stable 40 20 More variability Less stable 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Day What’s happening – Level, Trend, and Variability Treatment Baseline Number of Responses by Session Baseline Treatment Baseline Treatment 18 14 12 10 8 6 4 2 Session 29 27 25 23 21 19 17 15 13 11 9 7 5 3 0 1 Number of Responses 16 What’s happening – Level, Trend, and Variability Number of Responses by Day Number of Responses 30 Baseline Treatment Baseline Treatment 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Day What’s happening – Level, Trend, and Variability Number of Responses by Day Baseline Treatment Baseline Treatment 35 30 Number 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Day What’s happening – Level, Trend, and Variability Number of Responses by Day Baseline Treatment Baseline Treatment 25 Number 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Day Using a design Why? Demonstrating functional control Determining whether there’s been a change (and how much) Determining whether the intervention produced the change Determining how much of what you’re doing is necessary Determining how much of what you’re doing is the right amount Did the intervention produce the change? Three criteria Prediction If, by doing what I’m about to do, I’m actually doing something different, then I should see a different pattern from my dependent variable. If, by doing what I’m about to do, I’m really not doing anything different from what I was already doing, then I should see the same pattern from my dependent variable Prediction Prediction criterion not satisfied Prediction criterion satisfied Number of X by Day Baseline Intervention 14 14 12 12 10 8 6 8 6 4 2 2 0 0 2 3 4 5 6 Day 7 8 9 10 11 Intervention 10 4 1 Baseline 16 Number Number 16 Number of X by Day 1 2 3 4 5 6 Day 7 8 9 10 11 Did the intervention produce the change? Verification If the change that came about is attributable to the intervention, then when I stop doing the intervention, my dependent variable should return to (or trend toward) its state. If the change that came about is attributable to something else, then when I stop doing the intervention, the dependent variable should continue as though nothing had changed. Prediction and Verification Predication and Verification Satisfied Prediction Satisfied, Verification Not Satisfied Number of X by Day Baseline 16 Intervention Number of X by Day Baseline 14 Number Number 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Day Intervention Baseline 18 16 14 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 Baseline 9 10 11 12 13 14 15 16 17 18 Day Did the intervention produce the change? Replication I did my intervention, my dependent variable measure changed. I stopped doing the intervention, and my dependent variable returned to its baseline state. I’m doing the intervention again. If the dependent variable changes as it did before, the intervention produced the change. If the dependent variable measure doesn’t change, then something else produced the change before. Prediction, Verification, and Replication Prediction, Verification, and Replication Criteria Satisfied Prediction and Verification Criteria Satisfied, Replication Criterion not Satisfied 25 23 19 17 15 Day 21 Intervention Baseline 13 11 9 Intervention 7 Baseline 5 25 23 21 19 17 13 11 9 7 5 15 Day 16 14 12 10 8 6 4 2 0 3 Intervention Baseline 1 Intervention Number of X by Day Number Baseline 3 16 14 12 10 8 6 4 2 0 1 Number Number of X by Day AB Design A = Baseline B = Intervention – introduce after you have: Stable baseline (no trend) Baseline data trending in direction opposite to that which the intervention is intended to move the data Repeating variability in the data So, if goal is to increase behavior, when should I introduce the intervention? Dependent Variable Measure by Day Dependent Variable Measure by Day 40 50 A Dependent Variable Measure Dependent Variable Measure 50 30 20 10 40 B 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Day Day Dependent Variable Measure by Day Dependent Variable Measure by Day 40 50 C Dependent Variable Measure Dependent Variable Measure 50 30 20 10 0 40 D 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Day Day If goal is to decrease the behavior, when should I introduce the intervention? Dependent Variable Measure by Day Dependent Variable Measure by Day 40 50 A Dependent Variable Measure Dependent Variable Measure 50 30 20 10 40 B 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Day Day Dependent Variable Measure by Day Dependent Variable Measure by Day 50 40 C Dependent Variable Measure Dependent Variable Measure 50 30 20 10 0 40 D 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Day Day Introduce intervention (or change conditions) when: Data are stable (e.g., flat trend, minimal variability), or Data are trending in direction opposite to that which the next condition would be intended to move it, or Variability is reliably repeating AB Design •Prediction? •Verification? •Replication? •Functional relation? •Questions answered •Controlling for potential confounding variables •Good enough? AB Design Prediction? Verification? Replication? Functional relation? Questions answered Controlling for potential confounding variables Good enough? When might you want to use an AB Design? (Behavioral and not behavioral examples) ABA Design A = Baseline B = Intervention Same rules for introducing intervention or changing conditions apply ABA Design Prediction? Verification? Replication? Functional relation? Questions answered Controlling for potential confounding variables Good enough? Ethical or other concerns? When might you want to use an ABA Design? BAB Design A = Baseline B = Intervention Intervention implemented without getting baseline data first as the person’s current condition is sufficiently bad as to render taking baseline data undesirable Same rules for introducing intervention or changing conditions apply BAB Design Percentage Bites and Drinks Accepted by Meal Intervention Baseline Intervention 100 90 80 Percentage 70 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Meal •Prediction? •Verification? •Replication? •Functional relation? •Questions answered •Controlling for potential confounding variables •Good enough? When might you want to use a BAB design? (Behavioral and not behavioral examples) ABAB (Withdrawal) Design A = Baseline B = Intervention Same rules for introducing intervention or changing conditions apply ABAB (Withdrawal) Design •Prediction? •Verification? •Replication? •Functional relation? •Questions answered •Controlling for potential confounding variables ABAB (Withdrawal) Design •Prediction? •Verification? •Replication? •Functional relation? •Questions answered •Controlling for potential confounding variables ABAB (Withdrawal) Design •Prediction? •Verification? •Replication? •Functional relation? •Questions answered •Controlling for potential confounding variables ABAB (Withdrawal) Design •Prediction •Verification •Replication •Functional relation between intervention and dependent variable? ABAB (Withdrawal) Design •Prediction •Verification •Replication •Functional relation between intervention and dependent variable? When might you use an ABAB Design? (Behavioral and not behavioral examples) Multiple Treatment Withdrawal Design A = Baseline B = Treatment 1 C = Treatment 2 D = Treatment 3, etc. Same rules for introducing intervention or changing conditions apply Multiple Treatment Withdrawal Design •Prediction •Verification •Replication •Functional relation between intervention and dependent variable? •Be aware of possibility of sequence effects (or multiple treatment interference effects) When might you want to use a multiple treatment withdrawal design? (Combined medication and behavioral treatment, possibly. Others? Component Analysis with a withdrawal design A = Baseline B = a treatment package, made up of multiple components After stable data pattern attained during treatment package, remove one or more components at a time, with whole package implemented in between, comparing level of dependent variable between conditions Determines which of the treatment components are necessary and sufficient to maintain the improvements (and which you can drop) Component Analysis with a withdrawal design •Prediction •Verification Dependent Variable Measure by Day BL 60 •Functional relation? 50 •How do you know what to keep and what to drop? Dependent Variable Measure •Replication 1, 2, & 3 1& 2 1,2,&3 1 & 3 1,2,&3 2 & 3 1, 2, & 3 1& 2 40 30 20 10 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 Day When might you want to use a component analysis? Considerations with Withdrawal Designs Reactivity between baseline procedure and dependent variable History effects Sequence effects Irreversibility of some behaviors (e.g., bicycle riding, reading, behaviors coming under control of “natural contingencies”) Failure to return to baseline level Ethical considerations “Bad idea” behaviors “He’s doing so well, why withdraw treatment?” Multiple Baseline Designs A = Baseline B = Intervention Good when you need to work on One dependent variable, but in multiple settings One dependent variable, but with multiple “therapists” More than one dependent variable And you can’t get to it all at once! Multiple Baseline Design Across Settings Baseline Percentage Meals Consumed by Day Intervention 100 90 •Prediction 70 Percentage •Verification 80 60 50 40 30 20 •Replication 10 0 100 •Functional relation? 90 80 Percentage 70 60 50 40 30 20 10 0 100 90 80 Percentage 70 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 Day 12 13 14 15 16 17 18 19 20 21 Multiple Baseline Design Across Academic Behaviors •Prediction •Verification •Replication •Functional relation? Multiple Baseline Design Across Academic Behaviors •Prediction •Verification •Replication •Functional relation? Stacked Graphs do not make a Multiple Baseline Design! Stacked Graphs with a Squiggly Line Don’t Make A Multiple Baseline Design! When might you want to use a Multiple Baseline Design? What works (or will work) for your child? Up to you to decide. Empirically supported versus controversial Sometimes a false dichotomy Possibly irrelevant What works (or will work) for your child? • Be an informed consumer •Magazines versus journals •Second hand versus original reports •The speaker or author versus the procedure and the data •Skeptical consideration of the design, data, and conclusions drawn from them •Understand that selecting an empirically supported intervention does not obviate need to evaluate the intervention with your child! Single subject research designs are tools to help decide. Single subject research designs are tools to help decide whether or not what you’re doing is helping The data are right, and the child is right Impressions are often inaccurate Designs provide a way to more objectively examine how things are going, and with which to make differently informed decisions for our children