Education Department
M. Ed. in Reading
CIP Code: 13.1315
Program Code: 620
Fall 2010
Program Quality Improvement Report 2009-2010
1
Learning Outcomes
1. Demonstrate knowledge of the foundations of reading and writing
processes and instruction (B. T. Knowledge & Comprehension)
2. Use a wide range of instructional practices, approaches, methods, and
curriculum materials to support reading and writing instruction; (B. T.
Application)
3. Use a variety of assessment tools and practices to plan and evaluate
effective reading instruction (B. T. Application , Analysis, Evaluation)
4. Create a literate environment that fosters reading and writing by
integrating foundational knowledge, use of instructional practices,
approaches and methods, curriculum materials, and the appropriate
use of assessments (B. T. Application & Synthesis)
5. View professional development as a career-long effort and responsibility.
(B. T. Application)
*
**
***
Fall 2010
Bloom’s taxonomy for Academic programs only
Outcomes may be carried over from one assessment cycle to the next based on program needs
All program objectives are required by International Reading Association which is our accrediting
agency.
Program Quality Improvement Report 2009-2010
2
Alignment of Learning Outcomes
Program learning outcomes as they relate to the students at
Cameron University and in southwest Oklahoma and
alignment with Cameron University Mission Statement
:
Mission Statement: Cameron University provides a diverse and dynamic
student body access to quality educational opportunities; fosters a
student-centered academic environment that combines innovative
classroom teaching with experiential learning; prepares students for
professional success, responsible citizenship, life-long learning, and
meaningful contributions to a rapidly changing world; and is a driving
force in the cultural life and economic development of the region.
Alignment: One of the specific functions assigned to Cameron University
by the State Regents for Higher Education is to provide programs,
involving both course work and practical experiences, designed to assist
in the development of effective early childhood, elementary, secondary
or elementary-secondary teachers. Because reading specialists are
licensed to work with children from kindergarten through 12th grade, the
M. Ed. in Reading supports this specific function by providing a program
to assist in the development of specialists who can work at all levels.
Fall 2010
Program Quality Improvement Report 2009-2010
3
Program learning outcomes relationship to Cameron
University strategic plan
• #1: Demonstrate knowledge of
the foundations of reading and
writing processes and instruction.
• #2: Use a wide range of
instructional practices,
approaches, methods, and
curriculum materials to support
reading and writing instruction.
• #3: Use a variety of assessment
tools and practices to plan and
evaluate effective reading
instruction.
•
PLAN 2013: Maintain and enhance
Cameron’s commitment to providing
programs of the highest quality in
instruction, research, and service to better
meet the needs of the citizens of the
region.
•
PLAN2013: Assure efficient, effective
course delivery in multiple formats.
PLAN 2013: Make effective use of existing
and new facilities.
•
•
PLAN 2013: Maintain and enhance
Cameron’s commitment to providing
programs of the highest quality in
instruction, research, and service to better
meet the needs of the citizens of the
region.
Program learning outcomes relationship to Cameron
University strategic plan
•
•
#4: Create a literate environment that
fosters reading and writing by
integrating foundational knowledge,
use of instructional practices,
approaches and methods, curriculum
materials, and the appropriate use of
assessments.
•
PLAN 2013: Assure efficient, effective
course delivery in multiple formats.
•
PLAN 2013: Increase educational
partnerships with common education,
career technology centers, community
colleges, and other Oklahoma
Universities.
PLAN 2013: Maintain and enhance
Cameron’s commitment to providing
programs of the highest quality in
instruction, research, and service to
better meet the needs of the citizens of
the region.
#5: View professional development as
a career-long effort and responsibility.
•
Alignment of Department Mission with
Learning Outcomes
• The Department of Education believes that competent, caring, and
committed educators are successful in their careers.
– Educators who are competent use pedagogical and content knowledge to support
learning for all, provide instruction based on standards and student needs, and use
assessment and strategies for active engagement so that all can learn.
– Educators who are caring are responsive to individual needs and create learning
environments that promote positive social interactions and motivation.
– Educators who are committed collaborate with others by using effective communication
skills while being reflective decision makers and lifelong learners who are willing to
change in order to continuously improve.
• Alignment
– We believe that all of the above qualities are fostered through standardsbased coursework with coordinated field experiences.
– Department dispositional forms measure these 3 characteristics
– Dispositional forms are completed at program entry, after 9 hours, after 21
hours, and at program completion
Fall 2010
Program Quality Improvement Report
2009-2010
6
Measures of Student Learning
• Direct Measures (5 point scale)
– Portfolio artifacts scored in Chalk and Wire *
– Dispositional Assessment *
• Remediation Plan
– * Candidates failing to receive satisfactory scores (3 or better) on the portfolio
artifacts and/or dispositional assessments are asked to complete a
remediation plan. They work with a professor from the M.Ed. in Reading
program to improve those measures.
• Direct Measure - OSAT (Oklahoma Subject Area Test)
• 300 points possible
• 240 needed to “pass”
• Indirect Measures (5 point scale)
– Exit Survey
Fall 2010
Program Quality Improvement Report
2009-2010
7
Program Actions Based upon 2008/2009 Action Plan
•
IRA 2.2 Literacy coaches use a wide range of instructional practices, approaches, and
methods, including technology-based practices, for learners at differing stages of
development and from differing cultural and linguistic backgrounds.
– Action: Because 2008/2009 average was 3.5/5 we agreed implement a strategy file this
semester. Strategy file will contain practices, approaches and methods for learners at
differing stages of development and from differing cultural and linguistic backgrounds
•
Chalk and Wire data
•
2008/
2009 Average
N
2009/
2010 Average
N
3.5/5
12
4.0/5
20
* Course taught Fall 2009 and Fall 2010 – strategy file implemented Fall 2010
Fall 2010
Program Quality Improvement Report
2009-2010
8
Program Actions Based upon 2008/2009 Action Plan
IRA 4.2 Candidates use a large supply of books, technology-based information and non-print
materials representing multiple levels, broad interests, and cultural and linguistic backgrounds.
IRA 4.3 Model reading and writing enthusiastically as valued lifelong activities.
• Actions: We taught both Reading Practicum I and Practicum II classes in Fall 2009 and again in
Spring 2010. The specificity added to the assignments of lesson plans and narrative seem to
have increased candidate ability to provide the information needed to determine whether or
not these program objectives are being met.
•
Chalk and Wire Data
Fall 2010
IRA
2008/
2009
Average
N
2009/
2010
N
4.2
3.9/5
7
4.7/5
6
4.3
3.9/5
7
5/5
6
Program Quality Improvement Report
2009-2010
9
Program Actions Based upon 2008/2009 Action Plan
• IRA 5.1 Candidates view professional development as a career-long effort and
responsibility.
• IRA 5.3 Candidates work with colleagues to observe, evaluate, and provide feedback on
each other’s practice.
• Action: This assignment was revised to make it more specific which resulted in somewhat
higher scores for 5.3 and excellent scores for 5.1.
•
Chalk and Wire Data
Fall 2010
IRA
2008/
2009
Average
N
2009/
2010
Average
N
5.1
3.5/5
12
4.7
20
5.3
3.5/5
12
4.0/5
20
Program Quality Improvement Report
2009-2010
10
Program objective and measurement
PROGRAM GOAL: Use a wide range of instructional practices, approaches, methods, and curriculum
materials to support reading and writing instruction
MEASUREMENT OF PROGRAM OBJECTIVE
PROGRAM
OBJECTIVE 2.2
IRA 2.2 Literacy
coaches use a wide
range of
instructional
practices,
approaches, and
methods, including
technology-based
practices, for
learners at differing
stages of
development and
from differing
cultural and
linguistic
backgrounds.
Fall 2010
CURRICULUM
AREA OR TARGET
AUDIENCE
Reading
Measurements
Direct
measurements:
OSAT (Oklahoma
Subject Area Test)
Portfolio
Dispositional
Assessments
Indirect
measurements
Exit survey
Follow-up Survey
IDEA
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
OSAT - Normreferenced scores
SPA- approved
rubrics for
portfolio reviewed
by Faculty and
Advisory Board
Inter-rater
reliability
Inter-rater
reliability
Compared to
previous years
Program Quality Improvement Report 2009-2010
Unable to
determine
Schedule for
measurements
Graduation
At the end of
each course
Entry, 9 & 21
hours, graduation
Graduation
1 year after
graduation
IDEA – after each
course
11
Priority # 1 - Program Objective # 2
Oklahoma Specialty Area Test (OSAT)
N
Below 240
240 – 269
Above 269
Average
instructional Practices/
selected responses
3
0
0
3
282.33
Instructional
Practices/constructed
response
3
2
1
0
246
Total of instructional Practices
3
2
1
3
264.17
Trend Analysis: Program Objective 2
Selected Response
Constructed Response
6
5
4
3
2
1
0
6
5
2007 - 2008 N = 1
2008 - 2009 N = 8
2009 - 2010 N = 3
Below
240
Fall 2010
240 - 269
Above
269
4
2007 - 2008 N = 1
3
2008 - 2009 N = 8
2
2009 - 2010 N = 3
1
0
Below 240
240 - 269
Program Quality Improvement Report
2009-2010
Above 269
12
Priority # 1- Program Objective # 2
Portfolio Assessments
SPA element
Portfolio
Assignments
N
unacceptable
%
acceptable
%
Exemplary
%
%
Meeting
Standard
IRA 2.2 Candidates
support classroom teachers
and paraprofessionals in
the use of a wide range of
instructional practices,
approaches and methods,
including technologybased practices for learners
at differing stages of
development and from
differing cultural and
linguistic backgrounds.
Literacy
Coaching
20
0
0
10
50
10
50
100
Classroom
Handbook
7
1
14
4
57
2
29
86
27
1
4
14
52
12
44
96
Totals
Trend Analysis: Program Objective 2.2
(Percent Meets and/or Exceeds standard)
100
Meets
50
Exceeds
0
Fall 2008
Fall 2010
Fall 2009
Program Quality Improvement Report
2009-2010
13
Priority #1 – Program Objective #2
Dispositional Assessment (Scale of 1 – 5)
N=3
Disposition
1
2
3
4
5
Average
#3: Is sensitive to student needs
0
0
0
0
3
5
#4: Values subject matter
0
0
0
0
3
5
#5: Appreciates multiple teaching strategies
0
0
0
0
3
5
#6: Uses fair and consistent practices
0
0
0
0
3
5
Trend Analysis: Program Objective 2
5
4.5
4
3.5
3
2.5
2
1.5
1
2008 - 2009 N = 9
2009 - 2010 N = 3
Question 3
Fall 2010
4
5
Program Quality Improvement Report
2009-2010
6
14
Priority # 1 - Program Objective # 2
Exit Survey (N = 4)
1
2
3
4
5
Average
4. Knowledge of instructional strategies
0
0
0
0
4
5.00
5. Ability to present instruction in clear & meaningful ways
0
0
0
0
4
5.00
6. Ability to integrate technology in instruction
0
0
0
2
2
4.50
7. Ability to apply professional and pedagogical knowledge when
observing classroom instruction.
0
0
0
0
5
5.00
9. Ability to use research to improve practice
0
0
0
0
5
5.00
24. Reflect upon your practices
0
0
0
1
3
4.75
35. Develop a classroom/school climate that values diversity
0
0
0
0
4
5.00
Trend Analysis: Program Objective 2
5.5
5
4.5
2008 - 2009 N = 8
4
2009 - 2010 N = 4
3.5
Ques 4
Fall 2010
5
6
7
9
24
Program Quality Improvement Report
2009-2010
35
15
Action Plan – Priority # 1
Priority Goal
Action Plan
IRA 2.2 Candidates
use a wide range of
instructional
practices,
approaches, and
methods, including
technology-based
practices, for
learners at differing
stages of
development and
from differing
cultural and
linguistic
backgrounds.
We are instituting the Fall 2010
addition of a strategy
file to EDUC 5663
Content Area Reading.
This will be measured
on Chalk and Wire.
This is the same as last
year because the
course is Fall only. By
the time we had data,
it was too late to
make the change last
year.
Fall 2010
Time Line
Program Quality Improvement Report
2009-2010
Needed Resources
None – instructor
has made that
change in the
syllabus
16
Program objective and measurement
PROGRAM GOAL:
View professional development as a career-long effort and responsibility
MEASUREMENT OF PROGRAM OBJECTIVE
PROGRAM
OBJECTIVE 5.1
IRA 5.1
Candidates view
professional
development as
a career-long
effort and
responsibility.
CURRICULUM
AREA OR TARGET
AUDIENCE
Reading
Measurements
Direct
measurements:
OSAT (Oklahoma
Subject Area Test)
Portfolio
Dispositional
Assessments
Indirect
measurements
Exit survey
Follow-up Survey
IDEA
Fall 2010
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
OSAT - Normreferenced scores
SPA- approved
rubrics for
portfolio reviewed
by Faculty and
Advisory Board
Inter-rater
reliability
Inter-rater
reliability
Compared to
previous years
Program Quality Improvement Report 2009-2010
Unable to
determine
Schedule for
measurements
Graduation
At the end of
each course
Entry, 9 & 21
hours, graduation
Graduation
1 year after
graduation
IDEA – after each
course
17
Program objective and measurement
PROGRAM GOAL: View professional development as a career-long effort and responsibility
MEASUREMENT OF PROGRAM OBJECTIVE
PROGRAM
OBJECTIVE 5.3
IRA 5.3 Candidates
work with colleagues
to observe, evaluate,
and provide feedback
on each other’s
practice.
Fall 2010
CURRICULUM
AREA OR TARGET
AUDIENCE
Reading
Measurements
Direct
measurements:
OSAT (Oklahoma
Subject Area Test)
Portfolio
Dispositional
Assessments
Indirect
measurements
Exit survey
Follow-up Survey
IDEA
Methods used to
determine validity
of measurement
instruments
Methods used
to determine
reliability of
measurements
OSAT - Normreferenced scores
SPA- approved
rubrics for
portfolio reviewed
by Faculty and
Advisory Board
Unable to
determine
Inter-rater
reliability
Inter-rater
reliability
Compared to
previous years
Program Quality Improvement Report 2009-2010
Schedule for
measurements
Graduation
At the end of
each course
Entry, 9 & 21
hours, graduation
Graduation
1 year after
graduation
IDEA – after each
course
18
Priority # 1 - Program Objective # 2
Oklahoma Specialty Area Test (OSAT)
Subcategory 5
Professional
Development
N
Below 240
240 – 269
Above 269
Average
3
2
1
0
246
Trend Analysis: Program Objective 5
Percent
6
4
2007 - 2008 N = 1
2
2008 - 2009 N = 8
0
Below
240
Fall 2010
240 - 269
Above
269
Program Quality Improvement Report
2009-2010
2009 - 2010 N = 3
19
Priority # 2 & 3 - Program Objective # 5
Portfolio Assessment
SPA element
IRA 5.1 Candidates view professional
development as a lifelong effort and
responsibility.
IRA 5.3 Candidates work with
colleagues to observe, evaluate, and
provide feedback on each other’s
practice.
A 5.3
Portfolio
Assignment
Literacy
Coaching
Literacy
Coaching
N
unacceptable
%
acceptable
%
Exemplary
%
20
0
0
10
50 %
10
50 %
% Meeting
Standard
100 %
20
0
0
10
50 %
10
50 %
100 %
Trend Analysis: Program Objective # 5
(Percent Mets and/or Exceeds standard)
80
60
40
20
0
Meets
Exceeds
2008/2009
Fall2010
2009/2010
Program Quality Improvement Report
2009-2010
20
Priority #2 & 3 – Program Objective #5
Dispositional Assessment
N=3
Disposition
1
2
3
4
5
Average
# 1: Exhibits a belief that all students can learn
0
0
0
0
3
5
#2: Demonstrates strong interpersonal skills
0
0
0
0
0
5
Trend Analysis: Program Objective 5
5
4
3
2
1
2008 - 2009 N = 8
2009 - 2010 N = 3
Disposition 1
Fall 2010
Disposition 2
Program Quality Improvement Report 2009-2010
21
Priority # 2 & 3 - Program Objective # 5
Exit Survey (N = 4)
Question
1
2
3
4
5
Average
2. Comprehension of professional standards in your area
25. Field experiences and/or clinical practices were sufficiently
extensive and intensive
26. Field experiences and/or clinical practices faculty gave regular
support
Trend Analysis: Program Objective 5
5
4.5
4
3.5
3
2.5
2
1.5
1
2008 - 2009 N = 8
2009 - 2010 N = 4
Question 2
Fall 2010
25
26
Program Quality Improvement Report
2009-2010
22
Action Plan, Priority # 2 & 3
Priority Goal
Action Plan
Time Line
Needed Resources
IRA 5.1 Candidates
view professional
development as a
career-long effort
and responsibility.
We are moving the assignment to
a different course where all class
members are working toward
reading specialist certification.
That way we can target the
assignment more specifically to
their needs.
Spring 2011
None – instructor is
agreeable
IRA 5.3 Candidates
work with
colleagues to
observe, evaluate,
and provide
feedback on each
other’s practice.
We are moving the assignment to
a different course where all class
members are working toward
reading specialist certification.
That way we can target the
assignment more specifically to
their needs. This will be measured
on Chalk and Wire in EDUC 5623
Theoretical Foundations of
Reading.
Spring 2011
None – instructor is
agreeable
Fall 2010
Program Quality Improvement Report
2009-2010
23
Ancillary Actions
• Our Specialized Professional Organization (SPA)
which is International Reading Association has
revised their standards for reading professionals. As
of Fall 2010, we are expected to follow the new
standards. This requires us to revise all assessments
and rubrics. We are currently working on that. By
next year, we hope to have all the new assessments
and rubrics in place and have some data from Fall
2010 & Spring 2011 to report.
Fall 2010
Program Quality Improvement Report
2009-2010
24
Published information on graduates
2008 - 2009
2009 - 2010
Total
Fall 2010
Working In Discipline
8
3
11
Program Quality Improvement Report 2009-2010
Unknown
0
1
1
25