BUILDING AN OUTCOMES MEASUREMENT SYSTEM
QUESTIONS TO CONSIDER
Draft 9-7-05
Activity 1 of Framework (What key decisions does a state make in the early stages of
developing an outcomes measurement system?)
1. What are existing efforts and structures that need to be considered?
Within state
 What are existing efforts in your state that relate to measuring outcomes?
 Are there existing Early Childhood accountability initiatives?
 Are there existing stakeholder groups with missions that may overlap?
 What assessment tools are currently being used at the local level?
Outside of state
 What are the federal requirements?
 What have national organizations recommended?
 What have other states done to develop an outcome measurement system and
what are their lessons learned?
2. What will be the decision-making process in the development of the outcomes system?
Within that, how will stakeholders be involved and how will information, decisions, etc. be
communicated to others?
 Who is the leadership? Who are decision-makers/Who gets the ultimate say?
 How will you get input from stakeholders? (who are the stakeholders; how were
they defined; what is their role?)
 How will you communicate decisions to others? (To whom do/will you
communicate? what information? when/how often? What is your marketing and
communication plan)?
3. What is the purpose of the outcomes measurement system?
e.g.,


a. To report to OSEP only
b. For improvement and decision-making
c.
What are the reasons for choosing one way vs. another (e.g. EC system structure,
program values and beliefs, utility)?
What are the implications for choosing one way vs. another? Pros and cons?
4. What children will be included in the outcome measurement system?
e.g,.
a. Children with IFSPs/IEPs
b. All children in state-funded EC Programs
c.


What are the reasons for choosing one way vs. another? (e.g. EC system
structure, program values and beliefs)
What are the implications for choosing one way vs. another? Pros and cons?
1
5. What outcomes will be measured?
e.g.
a. OSEP SPP/APR outcomes
b. OSEP SPP/APR outcomes plus other outcomes (e.g. health)
c.


What are the reasons for choosing one way vs. another (e.g. EC system structure,
program values and beliefs)?
What are the implications for choosing one way vs. another? Pros and cons?
6. Will your state further specify the outcomes into sub-outcomes or indicators?


What are the reasons for choosing one way vs. another? (e.g. EC system
structure, program values and beliefs)
What are the implications for choosing one way vs. another? Pros and cons?
7. Will (and if so how will) the outcomes be related to Early Learning Guidelines/
Standards or other standards?
e.g.
a. EL Guidelines/Standards serve as basis and outcomes are related
b. Outcomes are basis and ELG/S are related
c. No relationship



What are the reasons for choosing one way vs. another? (e.g. EC system
structure, program values and beliefs)
Would cross-walking be useful as a tool to help examine the relationship between
outcomes and guidelines/standards??
What are the implications for choosing one way vs. another? Pros and cons?
Activity 2 of Framework (What key decisions does a state make related to getting information
about child progress?)
8. How will the state get information about child progress? Within that…
8a. What sources of data will be used to measure the outcomes?
e.g., a. Published/Formal assessment instruments only
b. Published/Formal assessment instruments plus other input (such as
family report, provider report, clinical opinion)
8b. What type of assessment instrument will be used?
e.g., a. Norm-referenced
b. Criterion-based or curriculum based for ongoing progress monitoring
c.

What will be the criteria for choosing assessment instruments?
(e.g. program values and beliefs; consistency with policy; content related to
outcomes/cross-walking content to outcomes; cost/resources need;
appropriateness for children with special needs)
8c. Will the state require the use of a specific (or limited number of) assessment
measurement tools to measure child outcomes?
2


What are the reasons for choosing one way vs. another? (e.g. EC system
structure, program values and beliefs)
What are the implications for choosing one way vs. another? Pros and cons?
9. How will the state convert/transform data points into reportable data formats?
9a. How are multiple measures converted into one score for each outcome area? or
How is information from multiple sources combined to determine a level of
functioning for a child that is in relation to same aged peers?
9b. At what level should conversions be done?
e.g., a. Individual IFSP/ISP
b. Local programs
c. State level
Activity 3 of Framework (What key decisions does a state make in determining what and when
to pilot/field test data collection?)
10. What kinds of pilot efforts should be conducted (processes, outcomes, instruments,
data analysis, etc)?
e.g., a. Processes
b. Outcomes
c. Instruments
d. Data analysis
e.



What would be the purpose of the pilot?
How would the pilot be designed?
What would the results look like?
Activities 4, 5, and 6 of Framework
11. How will the state ensure accuracy and reliability of the data across multiple settings?
e.g., a. Training
b. Technical assistance
c. Monitoring
d.

What are the challenges/issues?
[MORE QUESTIONS TO BE ADDED]
3