2a and b. Changes made to courses, programs, and unit responses to data gathered; evidence of
actions taken as a result of spa review
In order to provide clear examples across initial and advanced programs, we have grouped the data
analyses and the resultant changes in courses and programs in response to those data as (a) Content
Knowledge, (b) Pedagogical and Professional Knowledge; and (c) Impact on Student Learning changes.
These are examples of changes, as indicated in the description of the required exhibit. Two additional
tables present changes related to the program and unit operations improvement cycle and the p-12
student learning positive outcomes.
Content Knowledge: Initial Programs
Data
Special Education: Praxis II Scores indicated
that candidates at the satellite campus
scored lower on the Praxis II consistently;
several students needed to retake
assessments in order to continue in the
program
Special Education: On the candidate
assessments of the program, one of the
introductory courses was consistently related
as redundant
Special Education: On evaluations,
candidates reported a discomfort with
reading strategies needed in inclusive
settings
Middle Childhood: Candidates have
struggled with required content knowledge
Praxis II tests, particularly in mathematics
and science
Early Childhood Education: Though
differences were not significant, trends of
cohort differences were noted on the Praxis
II PLT Communication Techniques (short
answers) and Knowledge based core items
Relationships with Families and Professionals
Change in response to data
Candidates demonstrated consistently poorer
performance; though grounded in performance based
budgeting, discontinued the satellite program to insure
consistency of pre-cohort preparation.
Reviewed content of course, overlap was significant so
courses were merged
A more focused literacy course, Teaching Reading and
Writing in the content areas was developed and
embedded in the field
Collaborated with the A&S mathematics Department to
create and implement three shadow courses paired with
three mathematics courses; these shadow courses, are
taught by middle school program faculty memb3ers to
align the instruction of content with pedagogy; in the year
that this program has been implemented the pass rate on
Praxis II mathematics has increased to 90% ; in science
methods courses were redesigned to provide increased
content of critical and difficult concepts in the context of
appropriate, research-based pedagogy
Faculty teaching the courses in which these topics were
address met to discuss trends and determined strategies
to mediate differences
English/Language Arts Grades 7-12: English
Literature and Content Knowledge Test
revealed a need to increase focus on
linguistics, understanding critical literacies,
and media
World Languages K-12: Candidates’ oral
language proficiency was judged by mentor
teachers as poor; candidates struggled to
pass the Oral Proficiency Test in their target
languages
Reduction of linguistics coursework in the English
department in Arts and Sciences led to designing new
courses with the Literacy and Second Language Studies
program in the use of language and linguistics, language
acquisition, and language analysis; Content related to
critical literacies and media was added to Teaching
Literature in Secondary Classroom course; Reading
Improvement in the Secondary School course was
embedded in an after-school program "RAW" (Reading
and Writers) in the high needs high school across from the
college- all candidates engage with high school students in
literacy activities including Freedom Writers, Creating
Graphic Texts, Literature Circles, and Creative Writing
A new requirement was put in place that candidates must
pass the Oral Proficiency Test prior to entering student
teaching; when this rule was put in place, very few passed
the test in time to actually student teach; in order to
increase the likelihood that candidates would be orally
proficient, the program is being moved to a post-bacc
licensure build on a degree in one of the target languages.
A revised transcript analysis tool will monitor the rigor of
language classes taken in each candidates baccalaureate
program.
Pedagogical and Professional Knowledge, Skills, and Dispositions: Initial Programs
Data
Early Childhood, Middle Childhood,
Secondary, and Special Education
Programs: The Office of Assessment and
Continuous Improvement downloads
disposition data weekly and provides a list of
students with any scores at the “sometimes”
or “somewhat” or “inconsistent or contrary”
level; In using these data to develop action
plans, more specific information and
examples appeared necessary
Special Education: Candidate evaluation of
field experiences and university supervisor
evaluation of field placements revealed a
disconnect between field experiences and
courses
Change in response to data
A work group developed and programs piloted a researchbased, detailed dispositions assessment to be used when a
focus area was identified; for example, rather than just
indicating “rapport with students” it provides observables
such as “refers to each student by name” “welcomes
students at the door” “positive affect” “can include
appropriate personal information about students in
discussions”
Faculty moved the program to integrate increased field
experiences closely aligned with core reading and writing
courses; classes are themselves taught in high needs
schools, and are followed by a hands on experience, with
faculty members serving as models
Special Education: Candidate and university
supervisor evaluation of field placements
revealed a lack of knowledge and
implementation of co-teaching models in the
field
Courses taught in the high needs school are co-taught with
special education faculty, general education faculty, and
school faculty, providing a model for co-teaching through
explicitly describing and implementing models
Middle Childhood Education: : Candidate
and university supervisor evaluation of field
placements a "disconnect" between courses
and practices; structure of field experiences
was described by cooperating teachers,
faculty members, and candidates as
"dropping in" rather than being engaged in
the setting
In spring 2011, the Middle Childhood Program entered
into a partnership with Princeton City School District to
house the majority of coursework on-site; this provides an
environment of immersion in the middle school
philosophy and embedded field experiences in a large,
diverse, urban setting
Early Childhood Education: Consistently high Continue to engage with cooperating teachers in
program quality ratings by cooperating
orientations and communication
teachers
Early Childhood Education: Review of the
Blue Ribbon Panel for Field and Clinical
Experiences and Transformation Initiative;
analysis provided data for faculty discussion
English/Language Arts Grades 7-12:
Systematic analysis of PLT and candidates'
performance in methods class assignments
such as conceptual units, analysis of student
work, and differentiating instruction
revealed that candidates sometimes make
dangerous assumptions and
overgeneralizations, gave "lip service" to
designing accommodations and adaptations
for students with special needs, co-teaching
with intervention specialists, and only
superficial attention to scaffolding students'
reading and writing skills
Embedded field experiences in courses in science methods
and social studies methods held at Taft Elementary School;
implementing a first year field experience for freshmen
English/Language Arts Grades 7-12:
Improvement in scores on analysis of student
work protocols, assessment adolescents'
motivation to read, and research-based
strategies assignments
Reviewed changes in methodology and field placements
made in response to concerns in these areas; revised
conceptual unit, analysis of student work, and
differentiating instruction assignments to better
determine impact on student achievement.
English/Language Arts Grades 7-12: Results
of pilots of Teacher Performance Assessment
Provide feedback related to tasks and handbooks;
participate in revision of handbooks with Stanford;
increased number of raters at the University of Cincinnati
Rebuilt methods courses around the proposition that
successful teaching is based on knowledge of subject
matter and subject specific pedagogy, reflecting and acting
on evidence of effects of instruction on student learning,
and growing awareness and development of one's own
values, expectations, and cultural competency; Modeled
coteaching with special educators using research based
strategies identified by Friend & Cook (2003) in two
courses; paired English/language arts candidates and
special education candidates in an urban practicum
experience in both general and special education settings
Impact: Initial Programs
Data
Change in response to data
Special Education: Educator Impact Rubric
was confusing to students and though
research based was judged to lack reliability
Middle Childhood Education: Impact
measurers were inconsistently applied in
field experiences
English/Language Arts Grades 7-12:
Assessments did not provide specific enough
evidence that candidates reviewed student
work and made significant instructional
changes
Assessment replaced with embedded analysis of student
work throughout methods coursework; Teacher
Performance Assessment requires documentation os
student growth
Teacher Performance Assessment analyses of student
work will be used to prepared candidates to teacher,
assess, measure student impact, and re-teach
Methods courses refocused on analysis of student work
for differentiation; embedded field experiences allowed
for modeling and group work related to analysis of student
work
Content Knowledge: Advanced Programs
Data
School Psychology: Candidate performance
on Praxis II slightly lower in legal/ethical
foundations
School Psychology: Summarized candidate
performance on Masters' Comprehensive
Examination revealed inconsistent
performance on data-based decision making
Educational Leadership: Candidates
completed coursework successfully, "passed"
assessments, yet still were not consistently
ready for internship
Change in response to data
Added explicit content on legal/ethical issues across and
within many courses
Literacy and Second Language Studies: The
low number of candidates enrolled has made
it difficult to draw substantive conclusions
about program quality from Praxis II scores;
some previous content assessments and
accompanying rubrics were less precise in
capturing the range of elements associated
with a given IRA standard than those we
haveconstructed for our new program.
Replaced Grades in Coursework as an assessment, and
increased the precision of our other assessments and their
related rubrics in order to ascertain with specificity the
degree to which candidates have met each of the IRA
standards
Curriculum and Instruction: Data
generated by a vague rubric assessing the
literature review failed to provide data
needed for program improvement
Developed a more detailed rubric, aligned more clearly
with demonstrating content knowledge
Provided explicit content in teaching of applied behavioral
analysis course related to foundation knowledge and databased decision making
All candidates are required to receive a target score (2/2)
for each standard assessed in courses prior to internship;
increased rigor and consistency in scoring
Pedagogical and Professional Knowledge: Advanced Programs
Data
Change in response to data
School Psychology: Practicum Evaluation
and Intern Competency Checklist data
revealed gaps in candidates' application of
principles and practices on counseling cases
as compared to intervention cases
Educational Leadership: Evaluations of
candidate performance in the clinical
internship revealed that some candidates
needed more preparation before
implementing projects and activities
Literacy and Second Language Studies:
Assessments did not provide the evidence
the faculty members needed to continuously
review the program
Curriculum and Instruction: the Action
Research Project, which was used as the field
experience, made monitoring the quality,
nature, and supervision of field experiences
difficult
Added explicit field-based practicum to allow for direct
support and guidance for cases.
Developed and implemented a "pre-practicum" portion of
the clinical internship; assessment of candidate
performance during the pre-practicum was used to
formally accept or deny acceptance into the internship
portion of the program
Lesson Plan was replaced with a Weekly Lesson Plan and
Reflective Report assessment. This new assessment is an
improvement the former assessment in three ways.(a)
data gathering and ongoing progress monitoring, and
reflection are included as integral components of the
lesson planning process and they are reflected in the
accompanying assessment rubric; (b) overall lesson
Cohesion i. e. the candidate’s ability to design and
implement integrated, comprehensive, and balanced
approaches, is assessed and (c ) the assessment is based
on candidate performance over time, rather than a single
lesson. In this way, the assessment more accurately
reflects the candidate’s true level of competence.
A new required course designated as a field experience
was designed and will be implemented with the semester
shift
Impact: Advanced Programs
Data
School Psychology: Employer and alumni
program evaluation revealed graduates
were well-prepared, particularly within a
tiered service delivery model
Educational Leadership: Data collection on
the impact on the client and the setting was
inconsistently submitted, as a self-rating,
lacked reliability
Change in response to data
Continue to emphasize tiered delivery model, with
continued review of comprehensive school psychology
practices
The Educational Impact Rubric was re-aligned with the
standards, distributed through the web, and required that
the mentor complete a reliability check on the data
submitted
Literacy and Second Language Studies:
Review of the data indicated to faculty
members that revision of assessments were
needed
Impact on Tutee assessment has been replaced by a Case
Study. The Case Study includes all of the data included in
the former assessment (i.e., pre- and post-assessment
data), with the addition of data on the effectiveness of
specific instructional methods and materials for a student
experiencing difficulty with reading and writing. Further,
the Case Study more authentically represents the depth
and quality of thinking candidates will do routinely as
Reading Specialists.
Data-based changes related to unit operations
Data
Satisfaction Survey revealed that candidates
felt that Education Library and Education
Materials Development Data were
inconvenient
Cohort candidate focus groups with associate
dean
Partnership panels (p-12 principals, HR staff,
assistant superintendents) discussed concern
that mentor teachers were engaged in
excessive paperwork
Partnership panels, candidate evaluations of
programs, and faculty evaluation of programs
expressed concerns re: “drop in” nature of
clinical field experiences, spacing of days, and
length of time in the classroom at one time
Supervisors and candidates identified specific
settings and mentors who were deemed
unwelcoming or practicing inconsistent with
the conceptual framework
Documentation of candidate early field
experiences in foundations courses indicated
candidates were returning to the schools they
attended
Candidate evaluation of field experiences and
university supervisors revealed that specific
supervisors were not visiting required
number of times or were not providing
appropriate and adequate feedback
Change in response to data
Hours were extended for both evenings and weekends
Faculty members assigned to specific courses were
shifted to better align with graduate and undergraduate
expectations and skills
Programs shifted responsibility of documenting
collaborative assessment logs to candidates; disposition
assessment reduced to once per semester; programs
eliminated midterm assessments (except for early
childhood education - mentor teachers expressed the
interest in continuing mid-terms to show candidates
growth)
Handbooks were redesigned to increase specificity of
scheduling time in classroom; programs implemented
embedded field experiences serving as labs related to
courses conducted in the schools
Mentors and sites are eliminated from the pool if
ratings are 2 or less in any single item as rated by both
supervisor and candidate
Intro to education, educational psychology, and
introduction to exceptionalities developed menus of
specific field experience opportunities from which
candidates were required to select
Three adjunct supervisors have been discontinued;
faculty members’ who delivered inadequate or
inappropriate feedback were reassigned
Data-based changes related to p-12 student outcome
Our third program improvement cycle, that related to p-12 student outcomes, was initiated fall 2012.
Data will be aggregated and provided to programs during summer 2012. The following initiatives are in
place to provide the data for analysis and potential data-related changes:
 Analysis of teacher evaluation system data related to UC graduates who are teaching in
Cincinnati Public Schools
 Effect size of changes in student behavior related to behavioral interventions implemented by
candidate
 Meta-analysis of the TPA data related to impact on student learning and analysis o student work
 “Backward tracking” of candidates judged distinguished according to school district evaluation
systems to identify indicators of candidates performance through the preparation program