Child Outcomes Quality Assurance Presentation (with Playlist)

January 8, 2010

Presented by

Sherlock Center on Disabilities at RI College and

The RI Department of Human Services

For more information contact Maureen Whelan:

MWhelan@ric.edu

Today’s Learning Outcomes

 Understand how the Child Outcomes Summary

Forms (COSF) gets translated and reported to

OSEP (the Office of Special Education Programs)

 Learn ways to review your own program data

 Discover how to improve the validity and reliability of your data and

 Begin to use the data for quality improvement

The Sherlock Center on Disabilities at RI College and RI Department of Human Services

Outcome Statements

1. Children will demonstrate positive social emotional development (including positive social relationships)

2. Children will acquire and use knowledge and skills, including early literacy skills.

3. Children will use appropriate behavior to meet their needs.


Federal Reporting: Accountability

• Moving from Compliance to Performance

• Summary Statements

• Target Setting

• RI’s Current Process: COSF


Next Steps


Outcome Statements

1. Children will demonstrate positive social emotional development (including positive social relationships)

2. Children will acquire and use knowledge and skills, including early literacy skills.

3. Children will use appropriate behavior to meet their needs.


Outcome 1:

Children will demonstrate positive social emotional development

(including positive social relationships)



Involves:



Relating with Adults







Relating with Children

Following and understanding rules

Regulating emotions and behaviors



Includes areas like:



Attachment/separation/autonomy







Expressing emotions and feelings

Learning rules and expectations

Social interactions and play


Outcome 1:

Children will demonstrate positive social emotional development

(including positive social relationships)

 Regulate emotions



A 3 – 6 month old child can be comforted or soothed by caregiver when over stimulated





A 12 – 18 month old may use a security object to soothe, may have small tantrums but can be easily directed

A 24 – 36 month old begins to verbalize feelings, can wait for short periods of time, can transition between activities and follows directions with little resistance


Outcome 2:

Children will acquire and use knowledge and skills, including early literacy skills.





Involves:







Thinking

Reasoning

Remembering







Problem Solving

Using symbols and language



Understanding physical and social worlds

Includes areas like

Early concepts – symbols, pictures, numbers









Imitation

Object permanence

Expressive language and communication

Early literacy


Outcome 2:

Children will acquire and use knowledge and skills, including early literacy skills.

 Memory



A 3-6 month old will begin to repeat an enjoyable activity (i.e. shaking a rattle)





A 12-18 month old will show object permanence and remember social games or actions

A 24-36 month old can retell stories, remember simple rules, and remembers simple life events


Outcome 3:

Children will use appropriate behavior to meet their needs.





Involves:





Taking care of basic needs

Getting from place to place





Using tools (e.g. fork, toothbrush, crayon)

Health and Safety

Includes areas like:



Integrating motor skills to complete tasks







Self-help skills

Using appropriate communication to ask for help when needed

Acting on the world to get what one wants


Outcome 3:


 Communicating and satisfying hunger



A 3-6 month old will have a different cry to mean hungry, and will begin to hold bottle





A 12-18 month old may have a few words or signs for familiar foods, can use hands to feed self, and explores with utensils

A 24-36 month old can ask for food using familiar phrases and simple sentences, feeds self independently with utensils


Functional Skills are

INTEGRATED

Have Positive

Social

Relationships

Acquire & Use

Knowledge &

Skills

Take Appropriate

Action to Meet

Their Needs

NOT domain-specific


The Ratings: Things to Remember

When assessing children for the outcomes:

•

• Involve multiple sources

Involve multiple measures

•

•

•

Ask yourself:

What does the child typically do?

What is the child’s actual performance across settings and situations?

How does the child use skills to accomplish tasks?


The Ratings: Things to Remember

Outcomes are NOT:

•

• The child’s capacity to function under unusual or ideal circumstances

Necessarily the child’s performance in a structured testing situation

MOST IMPORTANTLY!!!

What are the typical functional skills and behaviors for this child across a variety of settings?


What do we need to know when filling out the COSF?











Understand the contents of the three outcomes

Understand age-expected child development

Understand age-expectations for a child functioning within the child’s culture

Know about the child’s functioning across settings and situations

Know how to use the rating scale


COSF Ratings

7

Completely



The child shows behaviors and skills expected in all or

almost all everyday situations that are part of the child’s life



The child’s functioning is considered appropriate for age



No one has significant concerns about the child’s functioning in this outcome area


Outcome 3:


Example: 28 month old child

7 Completely

Child is independently mobile in all settings. Child dresses, brushes teeth, puts away own clothes, and eats independently. Child is able to communicate with a variety of people wants, needs, and asks for help using verbal language. Child remains safe in all settings, and demonstrates an understanding of rules at home and at childcare.


COSF Ratings

6 Between Completely and Somewhat



The child’s functioning is generally considered

appropriate for age, but there are significant concerns about the child’s functioning in this outcome area


Outcome 3:



 6 Between Completely and Somewhat

Child is independently mobile in all settings. Child dresses, brushes teeth, puts away own clothes, and eats independently.

Child is able to communicate with a variety of people wants, needs, and asks for help using verbal language most of the time.

Child requires constant supervision at home due to safety concerns (jumps off furniture and throws objects). Child is able to follow rules, but this is more consistent when in a highly structured environment.


COSF Ratings

 5 Somewhat



The child shows functioning expected for age some of the time and/or in some situations





The child’s functioning is a mix of age-appropriate and not-appropriate function

The child’s functioning might be described as like that of a slightly younger child


Outcome 3:



 5 Somewhat

Child is independently mobile in all settings. Child is able to take clothes off independently, but needs help to finish dressing. Child can brush teeth with help, and eats independently with utensils. At times, the child needs a reminder to use verbal language to communicate with a variety of people his wants, needs and to ask for help. Child remains safe in all settings, and is more likely to demonstrate an understanding of rules at child care than at home.


COSF Ratings

 4

Between emerging and somewhat



Child shows some age appropriate functioning some of the time or in some situations or settings, but most of the child’s functioning would be described as not yet age appropriate



The child’s functioning might be described as that of a younger child


Outcome 3:



 4 Between Somewhat and Emerging

Child is independently mobile in all settings, although child tends to trip a lot when an object is in his way. Child dresses and undresses with adult help, brushes teeth with help, and eats most of a meal using utensils before using his fingers to self-feed. At times, the child needs a reminder to use verbal language to communicate with a variety of people his wants, needs and to ask for help. Child requires much supervision at home and at child care due to safety concerns. Child is beginning to follow some daily routine activities, but requires a lot of assistance and guidance with other rules and non-routine directions within his daily routine both at home and at child care.


COSF Ratings

 3 Emerging







The child does not yet show functioning expected of a child his/her age in any situation

The child’s behaviors and skills include immediate

foundational skills on which to build age appropriate functioning

The child’s functioning might be described as like that of a younger child


Outcome 3:



 3 Emerging

Child has recently become stable with walking long distances, but still has times where he does fall especially if an object is in his path. Child needs assistance with dressing and undressing although he likes to try it on his own first. Child finger feeds to eat, is beginning to use utensils, but is very messy at meal times. Child has a limited vocabulary, but is using some single words to request favorite foods. Child will tug on his mother’s clothing to indicate he wants her help, but will usually resort to crying when he needs help at child care. Child can follow basic routine directions, and is beginning to follow other simple directions and commands with some guidance from an adult.


COSF Ratings

 2 Between Emerging and Not Yet



The child does not yet show functioning expected of a child his/her age in any situation





The child’s behaviors and skills have some of the

immediate foundational skills on which to build age appropriate functioning, but these are not displayed often.

The child’s functioning might be described as that of a younger or much younger child


Outcome 3:





2 Between Emerging and Not Yet

Child is stable most of the time while walking, but will revert to crawling when he is tired. Child needs assistance with dressing and undressing although he likes to try it on his own first. Child finger feeds to eat, is beginning to use utensils, but is very messy at meal times. Child has a limited vocabulary, but is using some single words to request favorite foods. Child will usually cry or fuss when he needs help, but sometimes will tug on an adult’s clothing to gain their attention for help. Child will follow basic routine directions and simple commands if the adult pairs the command with a physical gesture.


COSF Ratings

 1 Not Yet





The child does not yet show functioning expected of a child his/her age in any situation

The child’s skills and behaviors also do not yet

include any immediate foundational skills on which to build age-appropriate functioning





The child’s functioning might be described as like that of a much younger child

Child with 1 ratings still have skills, just not yet an immediate foundational skill


Outcome 3:



 1 Not Yet

Child currently crawls to get from place to place and has pulled to stand on his sofa a few times. Child requires full assistance with dressing. Child finger feeds to eat at times, but is usually fed by an adult. Child drinks from a bottle, and is not able to drink from an open cup. Child will sometimes point to what he wants, but usually will cry, fuss, or tantrum when he is not understood by others.


Decision Tree

Child: Julie

Outcome: 3

Age: 30 months

Summary: Julie is able to walk from place to place, but will fall if there is an object in her way. She will often bump into things and is a bit unbalanced on uneven surfaces (sand, grassy areas). Julie requires assistance to climb up and down stairs, and up an down from chairs that are not toddler height. Due to this, Julie is sometimes either unsafe in areas that are cluttered or at times unwilling to even approach an area that is cluttered or challenging. Julie can drink from an open cup, use utensils, change her clothes, and brush her teeth with adult assistance. Julie seems to be more independent with these skills at home than at her child care center as reported by her mother and child care teacher. Julie uses some intelligible single words to request foods (drink, cookie, more, banana) and to request favorite activities (book, dora, swing-to mean playground, nana-to mean grammas house). When she is not understood by others, or if she becomes frustrated while trying to do something for herself, she will immediately fall to the floor and tantrum.


Progress Y/N

 The ECO Center defines progress as “the acquisition of at least one new skill or behavior related to the outcome”

 Compares child to her/himself over time (not to same-age peers, as with the 1-7 ratings)

 HINT: The answer should almost always be YES, unless the child has a degenerative or extremely disabling condition



“Impossible” – 1-7 ratings indicate growth but ‘no progress’ indicated


Where Do We Go From Here?



COSF data needs translation to interpret



Entry (1-7) + Exit (1-7) + Progress =

Categories a-e


Category “a”

 Child did not improve functioning from entry to exit



Child didn’t gain any new skills while in the program OR



Child regressed during the program



This category should include only those children with degenerative conditions or very significant disabilities


Child “a”: Sally

 Sally was enrolled in EI at age 2 due to recent loss of some single words but was pointing to request preferred foods and toys

 Sally was diagnosed with Rett Syndrome at age 25 months

 At exit, Sally had no spoken language and had lost her ability to point to request things


Developmental Trajectory for “a”

7

Level of

Development

Entry

Age in Months

Exit


6

5

4

3

2

1

Category “b”

 Child who improved functioning but not sufficient to move nearer to functioning of same-age peers



Child who acquired new skills but grew at the same rate throughout their time in the program



Child made gains but did not change their rate of growth


Child “b”: Timmy

 Timmy entered EI at age 2, and he was using a few single words to request, but his parents often could not understand his speech

 By the time Timmy was 3, he was using more words at home but unfamiliar people still found his speech difficult to understand


Developmental Trajectory for “b”

Level of

Development

Entry

Age in Months

Exit


7

6

5

4

3

2

1

Category “c”

 Children who improved functioning to a level nearer but not equal to same-age peers



Child who accelerated their rate of growth during their time in the program



Child made progress toward “catching up” to peers but still functioning below age expectations



“Narrowed the gap”


Child “c”: Kareem

 Kareem was enrolled in EI at 9 months of age because he was not sitting independently or crawling

 By age 3, Kareem was walking skillfully but still required support to manage uneven surfaces when playing with peers in community settings (parks, library)


Developmental Trajectory for “c”

Level of

Development

Entry

Age in Months

Exit


7

6

5

4

3

2

1

Category “d”







Children who improve functioning to reach a level comparable to same-age peers

Child who was functioning below age expectations when they entered program but are age-appropriate when they leave program

Child “caught up” while in the program


Child “d”: Maria

 Maria lives in a bilingual home and entered EI at age

2 with a few Spanish single words, but was mostly pointing or getting things for herself

 By age 3, Maria was a bit shy with new people until she became comfortable, but was consistently using

2- and 3-word phrases in both Spanish and English


Developmental Trajectory for “d”

Level of

Development

Entry

Age in Months

Exit


7

6

5

4

3

2

1

Category “e”

 Children who maintained functioning at a level comparable to same-age peers



Children who were functioning at age expectations when they entered the program and kept up with age expectations throughout their time in the program


Child “e”: Alex

 Alex was enrolled in EI at 3 months of age due to a diagnosis of neurofibromatosis and possible risk for vision problems and developmental delays

 Alex was discharged from EI at age 15 months of age because he had not developed vision problems and was age-appropriate in all areas of development


Developmental Trajectory for “e”

Level of

Development

Entry

Age in Months

Exit


7

6

5

4

3

2

1

Converting COSF data to OSEP progress categories

Tammy

Patty

Jonas

Phoenix

Angela

Juan

Terry

Leroy

Maria

Tony

Child's Name Entry Rating

2

1

4

1

7

6

3

3

4

5

Exit Rating

4

1

4

1

7

7

5

5

7

3

Progress?

yes no yes yes yes yes yes yes yes yes

OSEP Category

Tutor:

Conversion of Child Outcomes Summary Form (COSF) data to OSEP Reporting Categories

Entry

Outcome

Exit

Outcome

Instructions

Entry: Enter rating number for outcome (1 through 7)

Exit: Enter rating number for outcome (1 through 7)

Indicate whether or not progress was made since Entry

(Yes or no: format "y" or "n")

Reporting category will appear in "OSEP Category" column a - Children who did not improve functioning b - Children who improved functioning but not sufficient to move nearer to functioning comparable to same age peers c - Children who improved functioning to a level nearer to same-aged peers but did not reach it d - Children who improved functioning to reach a level comparable to same-aged peers e - Children who maintained functioning at a level comparable to same-aged peers

Progress?

- No determination of progress has been provided yet (please fix)

Impossible - The combination entered could not possibly occur. For example, a child cannot go from a 5 to a 7 and show no progress (please fix)

Progress

OSEP

Category

--

State and Individual

Program Data


State Entry Data

What do you see?

6’s and 7’s

Outcome 1 = 46%

Outcome 2 = 36%

Outcome 3 = 26%

6’s and 7’s are lowest in Outcome 3

Does what you see make sense?


Guidance From the Early

Childhood Outcomes Center

• “Few children receiving services would be expected to be considered functioning as typically (few scores in the typical range) at entry”

• “Functioning in one outcome area will be related to functioning in the other outcome areas”


Program Entry Data

What do you see?

• Do you have high 6’s and 7’s?

Add up the % of 6’s and 7’s

Then compare your % with the State as a whole

• Does Outcome 3 have a lower % of 6’s and 7’s than Outcomes 1 and 2?

• Compare % of 6’s and 7’s across all Outcomes



Red Flags

 Look for examples of all 6’s and 7’s at entry

 Look for 2 Outcomes entering at 6 or 7 and

1 Outcome entering at lower rating

 Look for wide discrepancies in ratings across Outcomes


State Exit Data

What do you see?

• 6’s and 7’s increase with all Outcomes

• Increase in 6’s and 7’s for Outcomes 2 and 3

• Decreases in 5’s



Program Exit Data

What do you see?

• Do 6’s and 7’s increase with all Outcomes?

• Compare 6’s and 7’s at exit to entry ratings in all Outcomes

• Compare exit and entry of other ratings?



Red Flags

 Look for examples of all 6’s and 7’s at exit in all/or 2 Outcomes areas for children who exited to Part B


State Data Scores

Impossibles

Missing Information

Progress Questionable

What do you see?

• % of e’s is higher than all other categories in

Outcome 1 and 2

• Low % of a’s

• % of d is higher in Outcome 3 compared to

Outcomes 1 and 2


Program Data Scores

What do you see?

• Does your program have a high % of e’s

• Is there a difference in % of e’s between

Outcomes?

• Is there a difference in the % of d’s between

Outcomes?

• Is there another score category that is high? %b?

• Does your program have missing data?

Why do you think this is?


More Red Flags

 Look at e’s…Do you have records with 6’s and 7’s at entry and exit in 2 or more Outcomes

 Look at a’s… Is the progress question correct?

 Look at b’s…Does the written summary support a rating that has stayed the same?

 Look at records with Progress… Is the progress question blank or is it a data entry issue?

 Look at the Impossibles….Was the question understood?

 Look at Missing records…Is it a data entry problem or a systems issue (Note: Attaching the COSF to the discharge sheet is an effective way to lessen “Missing Data”


Summary Statements

Summary Statement 1:

Of the children who entered below age expectation, the % of children who substantially increased their rate of growth by exit c+d / a+b+c+d

Summary Statement 2:

The % of children who were functioning within age expectations by the time they exited d+e/ a+b+c+d+e


RI Process and Guidance

• Upon Entry and Exit for ALL children

• COSF’s for transfers

• In-State – use previous programs entry status

• Out-of-State – begin new entry status rating

• More than 1 EI experience – begin new rating upon each entry

• COSF Review Protocol


RI Process and Guidance

Rating Resources

•

Backside of COSF

•

RI Child Outcomes Guidance

•

ECO Decision Making Trees


Outcomes Summary Spreadsheet

What is it?

• Excel spreadsheet, found in the RI Early Intervention

Care Coordination System in Welligent - updated quarterly

• Contain information for all providers – no identifying data

• Can compare to statewide data or another provider

• Identify missing data

• Analyze program data for trends – identify training needs



Where is it?

Go to the communications center then click on downloads



How is it downloaded?

Click file to download Click save

Then select where to save the file then click save

Note: If your agency’s server does not allow you to save the file on your

PC, you may be able to save the file to a thumb drive instead of your PC



Main components of the spreadsheet

Definition tab – describes what the spreadsheet contains and the meaning of terms

Missing Data tab – review data to find what children have missing or invalid data

Main Entry tab – contains tables and graphs to review each outcome entry data *

Main Exit tab – contains tables and graphs to review each outcome exit data *

Main Scores tab – contains tables to review each outcome scoring data and outcome statements *

Raw Tab – contains all the raw data

Data Patch – contains data to make sure the outcome statements work correctly

* Providers may change parameters on this sheet to analyze different aspects of the data



What are parameters?

Parameters are drop-down lists that the user may change to view filtered information. The default is “all” meaning all data is used for the calculations. The user can change the following drop-downs to filter the out-come data.

• Discharge SFY –

• Discharge year –

Select one state fiscal year (July – June) based on discharge date

Select one calendar year (January-December) based on discharge date

• Disc – Select one Discharge Status

• SFY-

• Year -

Select one state fiscal year (July- June) based on referral date

Select one calendar year (January- December) based on referral date

• Eli – Select one Eligibility Category

• Provider- Select one EI Provider

• Group Months in EI – Select a length of stay in EI based on referral date

• Group Months from entry – Select a length of stay in EI based on the COSF entry date

• Group (or Entry) More Than 6 – Older versions of excel only allow the user to select one option from each drop down list. If you are using a older versions this drop down allow you to select children who have been in EI (based on referral or COSF entry) “less” than six month or “More” than six months. Users with a higher version of excel do not need this option as then can select one or more age groups from the option above.


data


To work with any tableright mouse click on the table and click refresh

Tips

Click on View then Zoom to display the tab in a larger or smaller print!

Change any the parameters fields by using the drop down lists

Click on just the graph to print the graph only

Click on chart and then press {shift}{ctrl}F to format the chart. This works if you are using Microsoft

Excel 2000. For high or lower versions of Excel you may need to format manually or create a macro.



Missing Data Guidelines

The missing data tab will assist in identifying the IDs of children with incorrect or missing data. This spreadsheet only contains children enrolled in EI who have been discharged. If the child has been enrolled, providers, in most cases, should be able to fill out an exit outcomes form based on their knowledge of the child.

There will be exceptions to the rule when a provider has not served the child.

The parameters in the missing data tab can help providers sort through problems with missing or incorrect data.

Incorrect data must be corrected

• Even for children who have been in EI less then six months the form should be filled out – However the parameter “Group months in EI” can be used to identity children over or under six months in EI.

• Certain discharge categories may indicate the reasons for missing data such as unable to contact but this would also depend on how long the provider has served the family.


EI TUNES

Outcome Essentials


Child Outcomes Quality Assurance Presentation (with Playlist)

Today’s Learning Outcomes

Outcome Statements

Federal Reporting: Accountability

Next Steps

Outcome Statements

Functional Skills are

INTEGRATED

NOT domain-specific

The Ratings: Things to Remember

When assessing children for the outcomes:

Ask yourself:

The Ratings: Things to Remember

What do we need to know when filling out the COSF?

COSF Ratings

COSF Ratings

COSF Ratings

COSF Ratings

COSF Ratings

COSF Ratings

COSF Ratings

Decision Tree

Progress Y/N

Where Do We Go From Here?

COSF data needs translation to interpret

Entry (1-7) + Exit (1-7) + Progress =

Categories a-e

Category “a”

Child “a”: Sally

Developmental Trajectory for “a”

Category “b”

Child “b”: Timmy

Developmental Trajectory for “b”

Category “c”

Child “c”: Kareem

Developmental Trajectory for “c”

Category “d”

Child “d”: Maria

Developmental Trajectory for “d”

Category “e”

Child “e”: Alex

Developmental Trajectory for “e”

State and Individual

Program Data

State Entry Data

What do you see?

Does what you see make sense?

Guidance From the Early

Childhood Outcomes Center

Program Entry Data

Does what you see make sense?

Red Flags

State Exit Data

Program Exit Data

Red Flags

State Data Scores

Program Data Scores

More Red Flags

Summary Statements

RI Process and Guidance

RI Process and Guidance

Backside of COSF

RI Child Outcomes Guidance

ECO Decision Making Trees

Outcomes Summary Spreadsheet

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib