Indicator 7:
Measuring Preschool Outcomes
Entry Data Collection Using the COS
Process
Sarah Geldart – MA ESE
Sarah.Geldart@doe.mass.edu 781.338.3364
Additional Contact:
Lauren Viviani – MA 619 Coordinator
lviviani@doe.mass.edu 781.338.3372
Agenda
Overview of State Performance Plan/Annual
Performance Report (SPP/APR)
Indicator 7
Completing the Child Outcomes Summary
Process
Identifying entry ratings
2
SPP/APR Information
17 indicators of performance or compliance in
special education
State identified targets for performance
indicators
MA ESE set targets for Indicator #7 based on
data & feedback from stakeholders
3
Indicator 7: Preschool Outcomes
Percent of preschool children aged 3 through 5 with
IEPs who demonstrate improved:
A. Positive social-emotional skills (including social
relationships);
B. Acquisition and use of knowledge and skills
(including early language/communication and
early literacy); and
C. Use of appropriate behaviors to meet their needs
(20 U.S.C. 1416(a)(3)(A))
4
Outcome A: Children Have Positive
Social Relationships
Involves:
 Relating with adults
 Relating with other children
 For older children- following rules related to groups or
interacting with others
Includes areas like:
 Attachment/separation/ autonomy
 Expressing emotions and feelings
 Learning rules and expectations
 Social interactions and play
5
Outcome B: Children Acquire &
Use Knowledge & Skills
Involves:
 Thinking, reasoning, remembering, problem-solving
 Using symbols and language
 Understanding physical and social worlds
Includes:
 Early concepts – symbols, pictures, numbers, classification,
spatial relationships
 Imitation
 Object permanence
6
 Expressive language and communication
 Early literacy
Outcome C: Children Take Appropriate
Action to Meet Their Needs
Involves:
 Taking care of basic needs
 Getting from place to place
 Using tools
 In older children, contributing to their own health and safety
Includes:
 Integrating motor skills to complete tasks
 Self-help skills (e.g., dressing, feeding, grooming, toileting,
household responsibility)
 Acting on the world to get what one wants
7
Indicator 7: Preschool Outcomes
Five Progress Categories for Each Outcome
The percent of children who:
a) did not improve functioning
b) improved functioning but not sufficient to move nearer
to functioning comparable to same-aged peers
c) improved functioning to a level nearer to same-aged
peers but did not reach it
d) improved functioning to reach a level comparable to
same-aged peers
e) maintained functioning at a level comparable to sameaged peers
8
Summary Statements
 OSEP has taken the five categories and collapsed
them into two summary statements
 Summary Statement 1: Of those children who
entered the program below age expectations in each
Outcome, the percent who substantially increased
their rate of growth by the time they exited the
program.
 Summary Statement 2: The percent of children
who were functioning within age expectations in each
Outcome by the time they exited the program.
9
The Child Outcomes Summary
Process
Presentation developed with the assistance of
the Early Childhood Technical Assistance Center
What is the Child Outcomes
Summary (COS) Process?
A team process for reviewing child assessment
data from different sources culminating in the
‘rating’ of a child’s functioning on a scale of 1-7
Considers the child’s functioning across
situations and settings
At home, in a play group, at the library, in child
care, at community events, in preschool, etc.
Compares child’s functioning to same age
peers
11
Why was the process developed?
For federal reporting on child outcomes
No method to synthesize child outcomes data
from multiple sources
Different programs would be using different
assessment instruments, and outcomes data
would need to be aggregated across programs
12
Purposes of the COS
It is not an assessment tool
It uses information from multiple sources,
including assessment tools, observations, and
family interview to get a global sense of how
the child is doing at one point in time
13
Features of the COS
7-point rating scale
Rating is based on the child’s functioning:
What the child does across settings and situations
Compared with what is expected given the child’s
age
Ratings look at the child in two ways
How is the child functioning compared to ageexpected functioning?
Has the child gained at least one new skill in this
area?
14
Why two COS questions?
Progress data for federal reporting are
calculated from the COS ratings (entry versus
exit) plus the yes/no progress question
The yes/no progress question differentiates
between children who made absolutely NO
progress or regressed -- and children who
gained at least one new skill
15
Timing of the COS Process
In order to ensure that the entry ratings
represent a accurate snapshot of a child’s
current functioning, please complete the COS
Process within 4-6 weeks of a student
entering a program or beginning services and
within 4-6 weeks of a student exiting a
program or ending services.
16
The Child Outcomes Summary (COS) Process
Children ages 3-5 start special education services
(Aug. 1 – Dec. 31)
Data is collected about functioning across settings and
situations (including from families)
The COS team meets to discuss functioning, complete
the COS Form, and assign a rating for each outcome
(ideally within 4-6 weeks of starting services)
January-Entry data from the COS Forms is entered
into the data collection spreadsheet and submitted to
ESE
June– Exit data is collected as students exit the
program and reported annually to ESE in June
17
Sources of Information for
the COS Process
18
Summary ratings are based on…
Types of Information
 Curriculum-based
assessments (e.g., HELP)
 Norm-referenced
assessments (e.g.,
BDI-2)
 Developmental
screenings (e.g.,
Ages & Stages)
 Observation and
report
Sources of Information
 Routines Based
Interview (family
information)
 Service providers
 Therapists
 Physicians
 Child care providers
 Teachers
 People familiar with
the child in all of the
settings and
situations he’s in
19
Gathering Information from
Families
 Family input about the child’s functioning is critical
 Family members see the child in situations that
professionals do not
 Need to ask family members about what the child does at
home
 The team will need a way to learn what family
members know about the child
 There is no expectation that family members will be
able to determine whether what they are seeing is age
appropriate
0
2
Essential Knowledge for
Completing the COS
Among them, COS team members must:
 Know about the child’s functioning across settings and
situations
 Understand age-expected child development
 Understand the content of the three child outcomes
 Know how to use the rating scale
 Understand age expectations for child functioning
within the child’s culture
21
Important point
It is not necessary that all team members be
knowledgeable in all 5 areas
Especially, there is no expectation that parents
understand the rating scale or typical child
development…
But the professionals have to!
22
Why age anchor?
To answer the two COS questions:
a. To what extent does this child show ageappropriate functioning, across a variety of
settings and situations, on this outcome? (Rating:
1-7)
b. Has the child shown any new skills or behaviors
related to [this outcome] since
the last outcomes summary? (Yes-No)
Age Anchoring Resources:
http://ectacenter.org/eco/pages/training_resources
.asp#ChildDevelopment
23
The Child Outcomes Summary (COS) Process
Children ages 3-5 start special education services
(Aug. 1 – Dec. 31, 2014)
Data is collected about functioning across settings and
situations (including from families)
The COS team meets to discuss functioning, complete
the COS Form, and assign a rating for each outcome
(ideally within 4-6 weeks of starting services)
January 2015 – Entry data from the COS Forms is
entered into the data collection spreadsheet and
submitted to ESE
June 2015-June 2017 – Exit data is collected as
students exit the program and reported annually to
ESE in June
24
The Form
Cover page + three outcome pages
On each outcome page:
Two questions per outcome
Space to document the basis for the rating
Accessing the form:
http://ectacenter.org/eco/pages/outcomes.asp#
COSFormandInstructions
2
5
COS Form – Cover Page
26
COS Form- Page 2
27
COS Form – Page 2 Continued
28
COS Form – Page 2 Continued
This question will be answered
when completing the exit rating.
29
The Process for Answering
Questions 1a, 2a, 3a
For each outcome…
1. Discuss the child’s current functioning in this outcome
area across settings and situations
2. Identify areas where the child’s functioning is age
appropriate
3. If not all functioning is age appropriate, identify areas
where the child’s functioning reflects immediate
foundational skills
4. Decide which rating best describes the child’s current
functioning
30
Selecting a Rating
Looking at Development…
Foundational
Skills
Foundational
Skills
Immediate
Foundational
Skills
AgeExpected
Skills
32
7 – Completely
Child shows functioning expected for his or her
age in all or almost all everyday
situations that are part of the child’s life
Functioning is considered appropriate for
his or her age
No one has any concerns about the child’s
functioning in this outcome area
33
6 – Between completely
and somewhat
 Child’s functioning generally is considered
appropriate for his or her age but there are
some significant concerns about the child’s
functioning in this outcome area
 These concerns are substantial enough to suggest
monitoring or possible additional support
 Although age-appropriate, the child’s functioning
may border on not keeping pace with age
expectations
34
5 – Somewhat
Child shows functioning expected for his or her
age some of the time and/or in some
settings and situations
Child’s functioning is a mix of age-appropriate
and not age-appropriate behaviors and skills
Child’s functioning might be described as like
that of a slightly younger child
35
4 – Between somewhat
and nearly
Child shows occasional age-appropriate
functioning across settings and situations
More functioning is not age-appropriate
than age-appropriate
36
3 – Nearly
 Child does not yet show functioning expected of a
child of his or her age in any situation
 Child uses immediate foundational skills,
most or all of the time across settings and
situations
 Immediate foundational skills are the skills upon
which to build age-appropriate functioning
 Functioning might be described as like that of a
younger child
37
2 – Between nearly and not
yet
 Child occasionally uses immediate
foundational skills across settings and
situations
 More functioning reflects skills that are not
immediate foundational than are immediate
foundational
38
1 – Not yet
 Child does not yet show functioning expected of a
child his or her age in any situation
 Child’s functioning does not yet include immediate
foundational skills upon which to build ageappropriate functioning
 Child functioning reflects skills that developmentally
come before immediate foundational skills
 Child’s functioning might be described as like that of a
much younger child
39
Age-expected development
Entry
Exit
40
41
http://ectacenter.org/eco/assets/pdfs/Decision_Tree.pdf
Checking for Quality - Ratings
Ratings on all three outcomes should be
reported for every child
Ratings are needed in all areas even if:
No one has concerns about a child’s development.
A child has delays in one or two outcome areas, but
not in all three outcome areas
42
Checking for Quality Documenting the Rating
On the form, documentation needs to include:
What evidence led to the selected rating, evidence
of …..
Age expected functioning?
Immediate foundational skills
Skills and behaviors that will lead to immediate
foundational skills
Who participated in the conversation and the
decision
Documentation provides a record of the
rationale for the rating decision
43
Why is it important to document
the rating?
Evidence can be reviewed to see whether
people are using the system properly (i.e.,
rating similar children in the same ways)
Documentation helps identify needs for future
training and technical assistance
Documentation may be useful for new team
members reviewing the file
44
Checking for Quality - The
Progress Questions (1b, 2b, 3b)
Apply only if a Child Outcomes Summary
Form has been completed previously
Do not complete at entry
Has the child shown any new skills or
behaviors in the outcome area since the last
rating? (Yes or No)
Small steps of progress count!
Most will select “Yes”
45
Checking for Quality –
Completeness of the Form
 All information is
completed (i.e. name,
DOB, people involved,
family input)
 Evidence for all ratings,
that support each of the
ratings
 Ratings complete
 Progress question
answered at exit
46
The Child Outcomes Summary (COS) Process
Children ages 3-5 start special education services
(Aug. 1 – Dec. 31)
Data is collected about functioning across settings and
situations (including from families)
The COS team meets to discuss functioning, complete
the COSF, and assign a rating for each outcome
(ideally within 4-6 weeks of starting services)
January– Entry data from the COSFs is entered into
the data collection spreadsheet and submitted to ESE
June– Exit data is collected as students exit the
program and reported annually to ESE in June
47
Resources: Early Childhood
Technical Assistance Center
 Assessment Crosswalks
 ECTA Center has “crosswalked” assessment tools to the
Outcomes
 Crosswalks show which sections of assessment are related to
each Outcome
 The number of items addressing an Outcome does not
necessarily mean that the assessment captures functioning
across settings
 The COSF & information on how to complete it
 Other resources on Early Childhood Outcomes
www.ectacenter.org
48
Resources: Early Childhood
Technical Assistance Center
 Outcomes Measurement PD Resources:
 http://ectacenter.org/eco/pages/training_resources.asp
 Overview of the COS Process from ECTA:
 http://ectacenter.org/eco/assets/docs/COSF_overview.rtf
 The Child Outcomes Step-by-Step Video:
 http://ectacenter.org/eco/pages/videos.asp
 Outcome Rating Definitions:
 http://ectacenter.org/eco/assets/pdfs/Definitions_Outcome_Ratings.pdf
 The Ratings Decision Tree:
 http://ectacenter.org/eco/assets/docs/Decision_Tree.doc
49
Contact Information for Indicator 7
Primary Contact:
Sarah Geldart
Sarah.geldart@doe.mass.edu – 781.338.3364
Additional Contact:
Lauren Viviani
lviviani@doe.mass.edu – 781.338.3372
50