Measuring Intervention Results for Infants, Toddlers, and Preschoolers Using Individual Growth and Development Indicators (IGDI's) Charles R. Greenwood & Dale Walker Juniper Gardens Children’s Project University of Kansas http://www.igdi.ku.edu 1 Current IGDI Development Workgroup Charles Greenwood, Judith Carta, Dale Walker, Jay Buzhardt and Barbara Terry Original development team: Judith Carta, Charles Greenwood, Dale Walker Jane Atwater, Gayle Luze, Deborah Linebarger, Carol Leitschuh, Ken Parsley, Annessa Staab, Gabe Cline, and Susan Higgins ©Juniper Gardens Children’s Project, University of Kansas 2 Today’s Presentation What are IGDIs for infants and toddlers? Why use them? What outcomes do they measure? What website support is available? How are they used at the Individual, Program, and State Levels? How can they guide intervention and monitor progress over time? How can you start using them through an IGDI partnership Discussion and implications ©Juniper Gardens Children’s Project, University of Kansas 3 What are IGDIs? Measures that provide helpful information about children's and parents' growth toward socially valued outcomes and that guide intervention decision making ©Juniper Gardens Children’s Project, University of Kansas 4 The most well known and widely used IGDIs are Pediatric Growth Charts • Reflect progress toward desired outcomes • Chart individual progress • Evaluate individual progress relative to benchmarks • Used by medical practitioners and parents ©Juniper Gardens Children’s Project, University of Kansas 5 An IGDI Example: Growth in Early Expressive Communication Child’s Observed Trajectory 36 Mos Expectation Normative Trajectory ©Juniper Gardens Children’s Project, University of Kansas 6 Early Communication Indicator General Outcome: “The child uses gestures, sounds, words, or sentences to convey wants and needs or to express meaning to others” OSEP Outcome: “Use of appropriate behaviors to meet needs” Constructs: Prelinguistic Communication Spoken Language Key Skills: Gesture Vocalization Single Words Multiple Words ©Juniper Gardens Children’s Project, University of Kansas 7 General Administration Administration Procedures Toy-play setting with familiar adult as play partner 6-minute testing sessions Play partner’s role is to facilitate play and follow child’s lead Set-up/clean-up/put away Toy Form A: House Toy Form B: Barn Alternate Toy Forms Observational Recording Procedures ©Juniper Gardens Children’s Project, University of Kansas 8 ©Juniper Gardens Children’s Project, University of Kansas 9 Why use IGDIs? Linked to important outcomes (e.g., OSEP outcomes) Standardized Authentic Quick and easy to administer by practitioners Reliable and valid Repeatable Identify children in need Monitor intervention results Key skills rather than comprehensive skills measured Promote excellent communications with parents Website support, data entry, and reporting Individual data roll up to program- and state-level summaries of progress ©Juniper Gardens Children’s Project, University of Kansas 10 IGDIs are alternative to existing measures that are: Not direct measures of child performance Based on conceptual frameworks that do not embrace early intervention Not designed for early interventionists; require specialized expertise, Too time intensive Insensitive to short-term growth, the very information that early interventionist can act on Insensitive to early intervention effects Difficult to understand by early interventionists and parents ©Juniper Gardens Children’s Project, University of Kansas 11 Outcome Indicators for Infants and Toddlers Early Communication Indicator (ECI) Early Movement Indicator (EMI) Early Social Indicator (ESI) Early Problem Solving Indicator (EPSI) Indicator of Parent-Child Interaction (IPCI) www.igdi.ku.edu ©Juniper Gardens Children’s Project, University of Kansas 12 Outcomes Infant/Toddler IGDIs Address Communication Movement “The child moves in a fluent and coordinated manner to play and participate in home, school, and community settings.” Social “The child uses gestures, sounds, words, or sentences to convey wants and needs or to express meaning to others." “Child interacts with peers and adults, maintaining social interactions and participating socially in home, school, and community." Problem Solving “Child solves problems that require reasoning about objects, concepts, situations, and people.” ©Juniper Gardens Children’s Project, University of Kansas 13 What OSEP outcomes do they measure? Child Outcome 1: Positive social emotional skills (including positive social relationships) Child Outcome 2: Acquisition and use of knowledge and skills (including early language/communication [Part C] and early literacy [Part B preschool]) Child Outcome 3: Use of appropriate behaviors to meet their needs Parent Outcome 1: Know Parent Outcome 3: Help their children develop and learn ©Juniper Gardens Children’s Project, University of Kansas 14 Individual Growth and Development Indicators (IGDIs) Infants and toddlers (www.igdi.ku.edu) Preschoolers (http://ggg.umn.edu) Early elementary school (http://dibels.uoregon.edu) ©Juniper Gardens Children’s Project, University of Kansas 15 What Does the Literacy Connection Look Like? ©Juniper Gardens Children’s Project, University of Kansas 16 What website support is available? ©Juniper Gardens Children’s Project, University of Kansas 17 The Public IGDI Home Page Automated password update Descriptions of all IGDI measures. What they measure, how they’ve been tested, and how to use and score them Printable forms for conducting and scoring assessments Training requirements for certification, training videos, PowerPoint presentations, etc. Reports and presentations of IGDI research, descriptions, and training materials Send us questions using an online form or give us a call ©Juniper Gardens Children’s Project, University of Kansas 18 What You Can Do With the Password Protected Data System: Data Entry & Assessor Personnel Edit and add children to the system Edit, add, browse, ECI data (Primary and Reliability) Child reports Update your account ©Juniper Gardens Children’s Project, University of Kansas 19 What You Can Do With the Data System: Program Coordinator/Director Child and Program Reports Edit program Add, modify, delete users ©Juniper Gardens Children’s Project, University of Kansas 20 What You Can Do With the Data System: Project Director (Multiple Programs) Child and Program Reports Add/Edit all programs Add, modify, delete users ©Juniper Gardens Children’s Project, University of Kansas 21 How are they used? Individual Reports Program Level Reports Project (State) Level Reports ©Juniper Gardens Children’s Project, University of Kansas 22 Individual Child Report Click “Reports” Link ©Juniper Gardens Children’s Project, University of Kansas Select a Report 23 ©Juniper Gardens Children’s Project, University of Kansas 24 Expected Rate of Progress ©Juniper Gardens Children’s Project, University of Kansas 25 Delayed and Unresponsive ©Juniper Gardens Children’s Project, University of Kansas 26 Key Skill Elements ©Juniper Gardens Children’s Project, University of Kansas 27 Key Skill Elements Contin. ©Juniper Gardens Children’s Project, University of Kansas 28 Full Program Report Click “Reports” Link ©Juniper Gardens Children’s Project, University of Kansas Select a Report 29 Program Description ©Juniper Gardens Children’s Project, University of Kansas 30 Program Description ©Juniper Gardens Children’s Project, University of Kansas 31 Program Description Proficiency Status ©Juniper Gardens Children’s Project, University of Kansas 32 Program Description ©Juniper Gardens Children’s Project, University of Kansas 33 Program Description ©Juniper Gardens Children’s Project, University of Kansas 34 Full Project Report Select a Report Click “Reports” Link ©Juniper Gardens Children’s Project, University of Kansas 35 Within Project, Program Comparisons 25 Total Communication Rate 20 15 KSEHS Original Beacons EPIC 10 5 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 Age at Test ©Juniper Gardens Children’s Project, University of Kansas 36 Project Description Proficiency Status Proportion of Children on Target ©Juniper Gardens Children’s Project, University of Kansas 37 Using IGDI Data for DecisionMaking about Individual Children 38 IGDIs Lead to Improvements in Intervention Can give interventionists a focus for their efforts Can help interventionists “see” when they’re making a difference Can help interventionists know more quickly when a change is necessary Can help administrative staff understand when programs need improvements ©Juniper Gardens Children’s Project, University of Kansas 39 Use within an Intervention Decision-Making Model Individual Growth & Development Monitor Indicator (IGDI) Identify/Validate Need Generate Options Exploring Solutions Families AssessEvaluate Implement ment Intervention Intervention Monitor ©Juniper Gardens Children’s Project, University of Kansas What Individual Monitoring Achieves? Doing quarterly assessments on each child allows progress monitoring Seeing if any individual falls above or below age benchmark Taking action if child falls below the benchmark ©Juniper Gardens Children’s Project, University of Kansas 41 During quarterly monitoring If child scores… And…. Then… Slightly below average No other concerns Continue quarterly monitoring Slightly below average (-1 SD) Other concerns Begin monthly monitoring Below average (-1.5 SD) No other concerns Begin monthly monitoring Below average (-1.5 SD) Other concerns Go to Exploring Solutions On Target No other concerns Continue quarterly monitoring 42 ©Juniper Gardens Children’s Project, University of Kansas Deciding to Change a Child’s Intervention Make decisions on more than one assessment point Use other data and information from parents and other caregivers Check that assessment is representative of the child’s functioning Consider that child may already be receiving Early Intervention (Part C) ©Juniper Gardens Children’s Project, University of Kansas 43 ©Juniper Gardens Children’s Project, University of Kansas 44 During monthly monitoring If child… And…. Then… Continues to score slightly Below Average No other concerns Continue monthly assessments Continues to score Slightly Below Average Other concerns Go to Exploring solutions Continues to score Below Average No other concerns Go to Exploring Solutions Returns to On Target No other concerns Return to quarterly ©Juniper Gardens Children’s Project, University of Kansas 45 Using the ECI to monitor before and after intervention ©Juniper Gardens Children’s Project, University of Kansas 46 Implementing and Evaluating Interventions ©Juniper Gardens Children’s Project, University of Kansas 47 Promoting Communication: Arranging the Environment Designing the environment to promote communication Having books to read in quiet, comfortable area Regular routines Displaying pictures at child’s eye level Provide opportunities for exploring environment ©Juniper Gardens Children’s Project, University of Kansas 48 Following Child’s Lead Noticing and talking about the child’s interest, activity, or materials Using child’s interest to provide opportunities for communication Adult: “ I’ll dance with you.” Describe child’s actions Allow child to direct activities Give choices ©Juniper Gardens Children’s Project, University of Kansas 49 Commenting and Labeling Describing actions, toys, activities Children hear talk about actions and hear labels Comment on what the infant is playing with Talk about the pictures in a book ©Juniper Gardens Children’s Project, University of Kansas Teacher: Mmm. Milk Teacher: Keysha is swinging. 50 Caregiver’s self evaluation of using these language promoting interventions Complete self evaluation form to document use of intervention ©Juniper Gardens Children’s Project, University of Kansas 51 ©Juniper Gardens Children’s Project, University of Kansas 52 Evaluating and monitoring intervention effectiveness Continue administering ECI monthly Look at Key Skill Elements and Total Communication to check that they are moving toward benchmark Look at the slope of communication progress to determine if the slope is increasing Look for evidence that intervention has increased communication? If progress is not at expected rate, examine intervention and make necessary changes ©Juniper Gardens Children’s Project, University of Kansas 53 Key Skill Element: Single Words Marks Intervention Start ©Juniper Gardens Children’s Project, University of Kansas 54 Using ECI Data for DecisionMaking about Programs 55 For groups of children, ECI data can tell us… How children in our program are performing relative to benchmark on communication outcome. Whether our program is moving children toward important outcomes. When to consider modifications in program. When we institute a change in the program, whether it makes a difference in percentage of children who are on-target. ©Juniper Gardens Children’s Project, University of Kansas 56 Which children are currently off-target? Child Name Risk Status Randall, Donnie Slightly Below Tarantino, Quentin Slightly Below Pitt, Brad Below Average Leary, Timothy Below Average ©Juniper Gardens Children’s Project, University of Kansas 57 Identify children who are not on-target Are there more/less than the previous month? What might account for this? What strategies are in place for offtarget? Language promoting strategies? ©Juniper Gardens Children’s Project, University of Kansas 58 Examine the percentage of children who are on-target Possible questions... How does this vary by age group? What might account for differences by age? What types of changes might you make as a result of this information? ©Juniper Gardens Children’s Project, University of Kansas 59 Looking at program data by age group: Age Group On Target Slightly Below Average 1-11 Months 11 100% 0% 12-23 Months 16 94.1% 24-35 Months Below Average Total 0% 11 100.0% 1 5.9% 0% 17 100.0% 19 90.5% 0% 2 9.5% 21 100.0% 36-47 Months 7 77.8% 1 11.1% 1 11.1% 9 100.0% Overall 53 91.4% 2 3.4% 3 5.2% 58 100% ©Juniper Gardens Children’s Project, University of Kansas 60 How does program group data change from one month to the next? 61 Looking at program data quarterly Age Grp Percentage of ON-TARGET Qtr 1 Qtr 2 Qtr 3 Qtr 4 1-11 Months 11 100% 11 100% 11 100% 11 100.0% 12-23 Months 16 94.1% 17 95.9% 17 95.9% 17 100.0% 24-35 Months 19 90.5% 20 95.5% 20 95.5% 21 100.0% 36-47 Months 7 77.8% 7 77.8% 7 77.8% 9 100.0% Overall 53 91.4% 57 94.5% 57 94.5% 67 100% ©Juniper Gardens Children’s Project, University of Kansas 62 Questions to ask if program is not meeting your expectations: Are high quality language-promoting strategies being implemented? Is the program being delivered at the expected levels of intensity, duration? Are children/families participating at appropriate levels? Are additional levels of supports, resources, training necessary? ©Juniper Gardens Children’s Project, University of Kansas 63 We want you…. As an IGDI Partner! ©Juniper Gardens Children’s Project, University of Kansas 64 How can you start using them? Talk to us about establish an IGDI partnership ©Juniper Gardens Children’s Project, University of Kansas 65 Expectations of the Partnership 1. 2. 3. 4. 5. Programs will serve as a place to gather data for instrument validation, norming, and scaling up Programs are recipients of results gathered through the Website. Program staff will be interactive partners to help shape instruments through feedback Programs will become proficient in the use of infant/toddler IGDIs. Programs are knowledgeable about how data can improve and inform their practice. ©Juniper Gardens Children’s Project, University of Kansas 66 Discussion and Implications IGDIs are measures for interventionists They have good technical adequacy The are scalable through website technology We are testing the uses and benefits of individual, program, and state level progress data We are planning and conducting intervention sensitivity studies We are planning predictive validity (precursor skills) studies (e.g., the literacy connection) We are planning studies of the effects of using IGDIs on child outcomes ©Juniper Gardens Children’s Project, University of Kansas 67 For More Information carta@ku.edu greenwood@ku.edu walkerd@ku.edu http://www.igdi.ku.edu/ ©Juniper Gardens Children’s Project, University of Kansas 68