Early Learning Standards,
Measurement and Regulation
Parallel Session C:
August 28, 2012
Speakers
• Prof Nirmala Rao, Professor, Faculty of Education, The
University of Hong Kong
– Using Early Learning and Development Standards in the East
Asia Pacific Region
• Dr. Moni Day, Visiting Associate Professor CECED, AUD
– Early Learning Standards: Indian Case
• Dr. Maxine Bernstein, Professor Emeritus, School of
Education, Tata Institute of Social Sciences, Hyderabad
– Emergent to Early Literacy
Introductory Remarks
• Why is the assessment of early development
and learning important?
• What are Early Learning and Development
Standards (ELDS)?
Why is the assessment of early
development and learning important?
• Monitor child development in the context of
poor school readiness and learning outcomes.
• Track the development of vulnerable and atrisk children in parts of Asia, sub-Saharan
Africa
4
What are Early Learning and
Development Standards?
• Statements of expectation for “what children
should know and be able to do”
• Five Broad Domains of School Readiness have
been widely accepted:
•
•
•
•
•
Physical Health, Well-Being, and Motor Development
Social & Emotional Development
Approaches Toward Learning
Language, Literacy & Communication
Cognition and General Knowledge
Standards Language:
Terms & Definitions
•
There are four levels:
–
Level 1: Domain
•
–
–
–
Level 1.A. Sub-Domain
Level 2: Standard
Level 3: Indicator
Level 4: Preparatory Learning Activities
Example of an Early Learning Standard for 4-yr child
Domain: Language, Literacy & Communication
Sub-domain: Communication
Standard: Children should be able to listen to and
understand language
Indicator: Child is able to follow directions that involve a two or
three step sequence of actions
Preparatory Learning Activity: Give oral directions and play a
game like “caregiver says”
Make the children give simple directions to each other
(Kagan & Britto, 2006)
Example of an Early Learning Standard for 4-yr child
Domain: Physical Health & Well-being & Motor Development
Sub-Domain: Health & Personal Care
Standard: Children should be able to practice basic care routine
Indicator: Child gets drink of water without assistance
Preparatory Learning Activity: Offer plenty of opportunities to
the child to take care of self
(Kagan & Britto, 2006)
Using Early Learning and Development
Standards in the East Asia-Pacific Region
Nirmala Rao
Faculty of Education
The University of Hong Kong
South Asia Regional Conference on Early Childhood Care and Education Policies and Practices:
Towards 2015 and beyond. August 27 – 29, 2012, New Delhi
Overview
1. ELDS in the East Asia-Pacific Region
2. ARNEC project: Method & Findings
3. Conclusions
10
ELDS in the East Asia-Pacific Region
• 7 countries in the East Asia-Pacific Region had
developed and/or validated ELDS for their
countries with the support of UNICEF
11
11
ARNEC Project
• ARNEC commissioned HKU team to do project on
ELDS: Nirmala Rao, Pat Engle, Sun Jin, Marie Ng,
Yvonne Becher, Diana Lee, Carrie Lau and Zhang Li
• Develop draft Early Child Development Scales based
on the Early Learning Development Standards (ELDS)
of countries of the East Asia and Pacific Region
• The Scales should have dimensions of child
development which all countries consider valuable
12
ARNEC Project
Phase I
Select items to be used in the scale (desk review)
Phase II
Pilot test in small samples in as many countries as
possible. Revise and redo test based on results
Phase III
Representative sample in as many countries as possible.
Final revision and creation of age-based norms
14
Phase I
• We set out to develop the scales for the specific
purposes (not a screening test)
• Created a data file with all of the indicators, domains,
standards, country and age
• Decided on common categories and sub-categories
across countries based on analysis of data file
• Established reliability on definitions of categories
• Selected most common indicators for creating items
for pilot testing
15
Preparation of the database
Converted indicators Form a 100-item measure
to items
Selected indicators
in each category
To construct a 100-item
regionally-sensitive measure
Determined # of indicators Based on the proportion of the total # of
indicators in each category
in each category
Developed categories +
sub-categories
Established an
Indicator Database
7 categories
1738 indicators for children aged 3, 4,
and 5 from seven countries’ ELDSs
Focus on rationale and process
Preparation of the database
Stage 4 – Selecting indicators
• Established and applied decision rules and
criteria for including or excluding indicators
• We decided to select 100 indicators from the
list of 1710 indicators. Why 100 items ?
17
Categories
Approaches to learning
Cognitive Development
Cultural Participation and
Knowledge
Language and Emergent Literacy
Motor Development
Health, Hygiene and Safety
Socio-Emotional Development
Total
# of subcategories
12
Total # of
indicators
97
14
473
15
215
17
284
16
146
13
152
15
343
102
1710
18
Categories
Approaches to learning
Cognitive Development
Cultural Participation
and Knowledge
Language and Emergent
Literacy
Motor Development
Health, Hygiene and
Safety
Socio-Emotional
Development
Total
# of subcategories
12
% of total
Total # of
indicators in
indicators
each category
97
5.67
14
473
15
215
17
284
16
146
13
152
15
343
102
1710
27.66
12.57
16.61
8.54
8.89
20.06
100
19
20
EAP-ECDS Structure
1. Approaches to Learning
2. Cognitive Development
9 (5.67% of total indicators )
(8 separate; 1 incorporated in
another item)
22 (27.66% of total indicators)
3. Cultural Knowledge and Participation 10 (12.57% of total indicators)
4. Language and Emergent Literacy
5. Motor Development
6. Health, Hygiene and Safety
7. Socio-Emotional Development
Total
15 (16.61% of total indicators)
9 (8.54% of total indicators)
(8 separate; 1 incorporated in
another item)
9 (8.89% of total indicators)
16 (20.06 % of total indicators)
90
21
Stage 5 – Converting indicators to items
Code Sub-category
LA16
Indicator
selected
Countries
Draws a human
figure (head,
Draws/writes
eyes, mouth,
Philippines(4);
without an
trunk, arms, legs,Cambodia(5);
example
etc.) without
Thailand(4;5);
prompts
Ages
4, 5
22
Materials
1. A piece of paper; A pencil
Procedure
Assessor asks child to draw a complete picture of him
/ herself.
Instruction Here is a pencil and paper. I would like you to draw a
complete picture of yourself standing.
Scoring
Head and/or trunk plus
□ 0
□ 1
Criteria and one other feature.
Comments Recognizable arms and/or
legs.
□ 0
□ 1
Recognizable hands
and/or feet.
□ 0
□ 1
One recognizable facial
feature (eyes, mouth,
nose, or ears).
□ 0
□ 1
At least one additional
recognizable facial
feature (eyes, mouth,
□ 0
□ 1
nose, or ears)
23
Stage 5 – Converting indicators to items
Code Sub-category
Indicators
Selected
Countries
Age
Tells how to pay
respect to elders
(Ex. clapping
hands, bowing,
etc.).
SS01
Cambodia,
Laos,
Shows or uses
Mongolia,
respectful
Philippines,
Uses polite words
behavior
Thailand,
with adults (by
Vanatu,
answering thank
Vietnam
you, hello, good
bye etc.).
3;4;5
24
Materials
Two pictures of:
1. A sitting child and a standing adult in bus (can be
changed by country).
2. An adult giving a nicely wrapped gift to a child
(only two persons in picture)
Procedure Assessor shows one picture at a time and asks child
what the child in the picture should say.
Instruction 1. (for Picture 1) Look at this picture. Look at this
child. Here is a grandmother. What would you say
or do if you were the child?
2. (for Picture 2) In this picture, the adult is giving a
very nice gift to this child. What would you say to
the adult if you were this child?
Scoring
Correctly answered
□ 0
□ 1
Criteria and Question 1.
Comments
Correctly answered
□ 0
□ 1
Question 2.
25
• Revised feedback on draft categories, subcategories,
indicators and items at January workshop
• HKU team developed items, scoring criteria and
instructional manual. How we did this? Iterative process
• Panel of experts reviewed drafts materials and
appropriateness of methods and suggested changes
26
Summary (Phase I)
• Bottom-up process of defining instrument from
existing concepts – based on existing ELDS from EAP
countries
• Created a data file with all of the indicators, domains,
standards, country and age
• Decided on common categories and sub-categories
across countries based on analysis of data file
• Obtained reliability on definitions of categories
• Selected most common indicators for creating items
for pilot testing
27
Phase II: Pilot studies
1. To field test items in three diverse countries
2. To modify the Scales based on the results and
feedback
3. To develop a final version of the Scales which can
be used in many countries in Phase III
28
Adapted EAP-ECDS for each
country
Pilot study
Mongolia
Fiji
China
Modification
Field trials
Sent draft to Translation
&
countries
material checking
Field Consultancy Support
Specific Suggestions
• Use testing materials which are familiar to the child
• Use language which child can understand
• Follow the standard instructions and procedures
• Give children different ways to respond : actions not
always words
• Establishing and maintaining rapport
30
Field training workshops
31
Enhance reliability and validity
•
•
•
•
Structure equivalence
Inter-assessor reliability
Representative sampling
Test administration
– In preschool or in home depending on context
– Tester records notes regarding items and administration
– Parental questionnaire
32
Schedule for pilot studies
Oct. &
Forward
1. Preparation of
Aug. -Sep.
data collection
in China
Test
2. Data Entry in
Jun. – Aug.
administration in
Mongolia and
Mongolia and Fiji
Fiji
Jun.
May
Countries
received the
test package
1. Data collection
and field
support in China
2. Data analyses of
Mongolia and
Fiji data and
report writing
Consultations
and support to
Mongolia and
Fiji
33
Samples in three sites
Urban
Rural
Total (in school)
Boys
Girls
Boys
Girls
Boys
Girls
3 years
11
10
10
9
21(13)
19(16)
4 years
10
10
13
9
20 (20)
22(17)
5 years
9
11
9
9
20(15)
18(17)
3 years
6
7
12
8
13(7)
20(9)
Fiji
4 years
7
12
4
10
19(5)
14(9)
(Total: 120)
5 years
11
10
20
13
21(20)
33(15)
3 years
10
10
10
10
20(20)
20(20)
China
4 years
10
10
10
10
20(20)
20(20)
(Total: 120)
5 years
10
10
10
10
20(20)
20(20)
(Total: 120)
34
Duration of assessment
3 year olds
(two sessions)
4 year olds
5 year olds
90100m
3045m
4050m
Mongolia
6080m
4050m
45m
6080m
45m
30m
Fiji
China (urban)
35
Questions for Phase II validation
• Can this scale measure achievement of
standards/competence/ability?
– Are the items appropriate?
• Discriminative
• Difficulty level
• How do the countries differ?
– Scoring on the scales
– Difference across age groups
36
Strategy
Step 1
• Model Selection
• Compare the fit by different models
Step 2
• Item Analysis
• Item discrimination
• Item variance (difficulty)
Step 3
• Country comparisons
• Responses pattern
• Scores
37
Model Selection
• Can this scale measure ability/attainment of
ELDS?
• Item Response Theory (IRT)
– Assume a latent ability variable
– Latent variable measured by observed variables
– Observed variables are dichotomous or
polytomous
38
Models considered
• Rasch Model
• Two-parameter logistic latent trait model
• Birnbaum’s three-parameter model
39
Model selected
AIC
LRT
P-Value
Rasch
2756.64
-
-
Logistic (2 parameter)
2689.19
119.45
<0.001
Birnbaum’s
2727.36
15.88
0.955
40
Item Difficulty & Discrimination:
Cognitive Development
CD12_2
CD22_4
CD12_1
CD3_1
CD22_3
CD22_2
CD5_2
CD17_3
CD5_3
CD5_4
Top 5 Easiest Items
Sat on the chair
Correctly put the toy under the chair
Brought the block (or other small item) to assessor
Correctly put one block inside the box
Correctly put the toy on the chair
Top 5 Hardest Items
Correctly put the toy on the left side of the chair
Correctly picks 7 blocks on the paper
Correctly arranged all the pictures in right order
Correctly picks 15 blocks on the paper
Correctly picks 29 blocks on the paper
41
Item Difficulty & Discrimination:
Cognitive Development
42
Total Information:
Cognitive Development
43
Item Difficulty & Discrimination:
Socio-Emotional Development
SED1_1
SED80_1
SED25_2
SED24_4
SED26_2
SED24_5
Top 3 Easiest Items
Knows full formal name
Consistently followed the rules during assessment
Correctly point to the HAPPY face
Top 3 Hardest Items
Correctly answered “what help” in the losing mummy
situation
Correctly gave an answer to help stop crying
Correctly gave a second answer for “what help” in the losing
mummy situation
44
Item Difficulty & Discrimination:
Motor Development
MD36_1
MD35_1
MD41_1
MD38_4
MD38_2
MD38_3
Top 3 Easiest Items
Poured water and sipped from an open cup independently
Walk forward: fell off or side-stepped three times or less
Successfully strung large beads
Top 3 Hardest Items
Hit the target on third attempt
Hit the target on first attempt
Hit the target on second attempt
45
Item Difficulty & Discrimination:
Language and Emergent Literacy
LEL45_4
LEL43_5
LEL45_2
LEL43_1
LEL45_1
LEL56_2
LEL56_3
LEL48_3
LEL46_3
LEL54_3
Top 5 Easiest Items
Correctly identified playing (with ball)
Correctly identified cleaning teeth
Correctly identified reading
Correctly identified combing hair
Correctly identified running
Top 5 Hardest Items
Copied triangle
Copied rectangle
Mentioned another one or more events or happening of one
event in logical order
Linked all four pictures with some cohesive evidence
Drew recognizable hands and/or feet
46
Item Difficulty & Discrimination:
Health, Hygiene and Safety
HHS58_1
HHS58_2
HHS60_2
HHS59_1
HHS66_2
HHS66_1
Top 3 Easiest Items
Could pretend to clean his/her face and hand
Could pretend to brush his/her teeth
Could take the smock off
Top 3 Hardest Items
Mentioned washing hands after using the toilet
Correctly identified confectionary as unhealthy food
Correctly identified chips as the unhealthy food
47
Item Difficulty & Discrimination: Cultural
Knowledge and Participation
CKP74_1
CKP74_2
CKP74_3
CKP70_1
CKP70_2
CKP71_3
Top 3 Easiest Items
Correctly clapped one line according to the rhythm of the
song
Correctly clapped one more line
Correctly clapped another one ore more lines and refrain
Top 3 Hardest Items
Mentioned informing an adult
Mentioned returning the toy to the owner
Correctly named another festival
48
Item Difficulty & Discrimination:
Approaches to Learning
ATL84_1
ATL83_1
ATL84_4
ATL83_6
ATL83_5
ATL83_3
Top 3 Easiest Items
Showed positive emotion
Beginning: stays on task
Spontaneous positive interaction with assessor
Top 3 Hardest Items
End: not want to stop tasks
End: not easily distracted
Beginning: not want to stop tasks
49
Country Comparisons:
Cognitive Development
50
Discussion
• Psychometric issues
– Limited data
– Factor analysis difficult and unreliable
– Time consuming and challenging to develop
culturally appropriate items
51
Items localized
• Changed the task: Item 44
Phoneme awareness task 
syllable detection task
(Shu, Hong, & McBride-Chang, 2008)
• Changed the pictures: Item 64
Vegetable  Fruit
• Changed the pictures: Item 65
French fries Twisted fried bread
52
Items to be revised
Item 17:
• The pictures are not clear enough.
• The logic in the story is not specific.
Item 19:
• The pictures are not clear enough.
• The word “weather” is a bit abstract for
children.
• The instructions should be simplified.
53
Items to be revised
Item 22:
The instructions are ambiguous.
54
Difficulties in Assessment
• Children felt tired during the process of the assessment.
 Let children go out of the test room if they feel tired.
• The assessors’ conversations with the children beyond the
instructions prolonged the whole testing.
Avoid too many conversations and always keep children focused on
the test.
• The assessors unintentionally revealed the recording sheet to
children which made children nervous.
• How to motivate children during assessment?
 Give children enough oral encouragement, but not small gifts
55
Conclusions
• Test based on culturally sensitive ELDS; first
effort in the region
• Test is reasonably valid and reliable and can
meet need for a tool to monitor child
development at the population level
• Valuable effort to achieve both cultural
appropriateness and item equivalence in
cross-cultural measurement
56
56
THANK YOU!
57
57