Preparing IRA/CAEP
Program Reports to Earn
National Recognition
Dr. Diane Kern, IRA SPA Coordinator
CAEP Conference
Fall, 2014
1
Agenda
 Introductions
 Overview of program report and IRA 2010 Standards for Reading
Professionals
 Requirements of the assessment system
 IRA specific guidelines
 Literacy coaching tips and coaching continuum
 Sample report sections
 Resources, Contacts and Q & A
2
Overview of program report
 Institutions must provide evidence of meeting IRA standards based on
6-8 assessments (recommendation: 8 assessments).
 Assessments and rubrics must align with the IRA 2010 Standards for all
three roles of the reading specialist/literacy coach:
 Interventionist/reading specialist
 Literacy coach
 Reading program supervisor/leader
 Assessment system must build a case that all standards are met
through utilizing the elements and potential sources of evidence =
“preponderance of evidence”.
 Program level assessments must be administered to all candidates and
should span across the program.
3
Format: What Option?
Option
Description/Standards
Option A
NCATE Report Format: 6-8 Assessments,
Five Designated Assessments: Content (2)
including State Licensure Data (if required);
Professional and Pedagogical Knowledge
(2); P-12 Student Learning (1) Additional
Assessments (1-3) /
Option B
Choose Your Own Assessments: Up to 8
Assessments including 1) State Licensure
Data (if required); 2) Impact on Student
Learning; 3) Additional Content,
Pedagogical Knowledge Assessments
4
IRA 2010 Standards for Reading Professionals
1. Foundational knowledge
2. Curriculum and instruction
3. Assessment and evaluation
4. Diversity
5. Literate environment
6. Professional learning and leadership
5
IRA specific guidelines
 Program must incorporate an equivalent of 21-27
graduate semester hours in reading, literacy, language
arts, and related courses.
 Coursework must include a supervised practicum
experience – typically the equivalent of 6 semester
hours.
 Supervision may include observations, conferences,
audiotape or videotape review of lessons, etc.
 Academic and practicum experiences must develop
candidates for all three roles: interventionist, literacy
coach, and literacy program leader
6
Literacy Coaching Tips
 Coaching initiatives should include
experiences in the program (such as the
reading clinic) AND also opportunities to
support teachers and/or paraprofessionals in
authentic school settings.
 Institutions should implement a
developmentally appropriate continuum of
coaching experiences across the program.
7
Coaching Activity: 3 Levels of Intensity
(Bean, R.M., 2004)
8
Sections of the Program Report
 Cover Sheet
 Section I: Context
 Section II: List of assessments (chart)
 Section III: Relationship of assessments to standards (matrix)
 Section IV: Evidence for meeting standards (attached files using CAEP
required format)
 Section V: Use of assessment results to improve program (narrative)
 Section VI: For revised reports or response to conditions reports only
9
Section I: Context
Narrative or attachments, including the following:
 Any state or institutional policies that influence your program?
 Describe supervised clinical/practicum and other experiences
 Attach program of study
 Table with three years of data on candidates enrolled in the
program; separate if multiple sites
 Faculty information
10
Section II: List of Assessments
11
Section III: Matrix
12
Section IV: Sample Assessments #1 & #2
Content Knowledge
 Licensure exams or professional examinations of
content knowledge (must be #1 if required by state,
institutions must report results)
 Comprehensive Examinations
 Research Reports
 Child Studies/Action Research
 Portfolio Tasks (that all candidates perform)
 Evidence Base for Professional Development
Initiatives
13
Sample Assessment #3
Ability to plan instruction
 Developing lesson plans and units of study
 Selecting, using, and evaluating assessment tools
 Selecting, using, and evaluating curriculum,
instructional techniques, and appropriate materials
 Establishing and evaluating literate environment
 Designing and implementing instruction
 Portfolio tasks
 Literacy coaching components utilizing assessments
above
14
Sample Assessment #4
Assessment of internship, practicum, clinical
experience
 Reflective practitioner observation protocol
 Coaching cycle observation, reflection and evaluation by
supervisor
 Final school report, parent conference
 Analysis of video segment with peer review, coaching and
supervisor feedback
 Practicum portfolio
15
Sample Assessment #5
Impact on Student Learning
 Case studies
 Action research with a pretest-posttest design
 Analyzing student work samples
 Designing and implementing instruction and/or
intervention
 Portfolio tasks
 Literacy coaching and leadership experiences
utilizing assessments above
16
Sample Assessments #6, 7 & 8
Additional Assessments
Eight assessments are strongly recommended, although only six are
required
Diversity, technology, creating literate environment, coaching and
leadership experiences are typical areas to further develop
 Diagnostic portfolio for diverse learners at the elementary
and secondary levels
 Three levels of literacy coaching in the clinic and in school
settings
 Data reports for whole school reading program and Report
Night
 Supporting teachers to select and evaluate materials and
instructional methods to meet
 Professional development series, book study leadership
17
Section IV: Evidence for Meeting
Standards
Two-page narrative of each assessment includes:
a)Brief description of assessment and its use in the
program
b)Description of how assessment aligns with IRA
standards
c) Brief analysis of data findings
d)Interpretation of how data provide evidence for
meeting specific standards by IRA standard
number, title, and/or wording.
2) One attachment for each assessment, which includes
the assessment tool itself (the detailed
directions given to candidates), scoring
guide/rubric, and data table disaggregated
by TWO applications (and if multiple sites, by
18
each site)
Section V: Assessments/data to inform
program improvement
 Narrative with no more than 12,000 characters
 Summarize principal findings from the assessment results,
faculty interpretation of data, and how candidate
performance informs plans to strengthen the program
 Organized by three headings:
 Content knowledge
 Professional and pedagogical knowledge, skills and
dispositions
 Impact on P-12 student learning
19
Section VI: For Revised programs only
 Response to conditions narrative—Organized by the
conditions—what changes did you make? Only one
application of data required
 Revised reports narrative: Describe changes or additions
have been made to address the IRA Standards
 TIP—Even if IRA did not specifically ask for something, you
can include any (all!) sections of the program report to make
your case to earn national recognition.
20
Resources
 Fall IRA/CAEP Workshops and Individual 30 min.
Consultation at Association of Literacy Educators and
Researchers (ALER) annual meeting (Sat., Nov. 1, 2014—
Delray Beach, FL)
 Spring/Summer IRA/CAEP Workshops and Individual30 min.
Consultation at International Reading Association (IRA)
annual meeting (Fri., July 17, 2015—St. Louis, MO)
 IRA Research Division, Accreditation website
http://www.reading.org/Resources/ProfessionalDevelopment/
Accreditation.aspx
21
Resources, cont.
 CAEP (NCATE) SPA Library
http://www.ncate.org/Accreditation/ProgramReview/ProgramRevi
ewResources/SPAAssessmentLibrary/tabid/460/Default.aspx
 IRA CAEP Guidelines for Program Writers and Reviewers
http://www.reading.org/Libraries/resources/ira-guidelines-forprogram-writers-and-reviewers.pdf
 Consultation with member of IRA Audit Team when much
support is needed; contracted service with IRA, either on-site or
email/telephone
22
Q&A
Contacts
IRA and CAEP want your program to align to IRA
Standards, to prepare 21st century reading
specialists/literacy coaches, and to be Nationally
Recognized!
Dr. Diane Kern, IRA SPA Coordinator:
dkern@reading.org
Elizabeth Vilky, CAEP, Director of Program
Review elizabeth.vilky@caepnet.org
Dr. Stevie Chepko, CAEP, Senior Vice
President, Accreditation
stevie.chepko@caepnet.org
CAEP telephone: 202-223-0077
23