2.3 Transformation Initiative: Implementing, Sustaining, and Enhancing Progress
We began planning the implementation of our Transformation Initiative during the 2010-2011 academic
year, phasing in change this academic year. As we indicated in 2.1, our TI has pushed us to rethink,
redesign, and re-evaluate several aspects of our assessment system.
The “Third” Improvement Cycle: Collecting, Analyzing, and Using Data Related to p-12 Student
Outcomes
1. Since it was initially designed, our assessment system has included (a) a candidate improvement
cycle tracking the collection, analysis, and use of data to improve candidate performance; and
(b) and programs and unit operation improvement cycle, tracking collection, analysis, and use of
data related to programs, procedures, and operations. With the Transformation Initiative, we
have included a third improvement cycle: collecting, analyzing, and using data related to p-12
student outcomes in order to improve candidate performance, programs, and unit operations.
This is a significant shift and moves us from what we are doing, how we are doing it, and what
our candidates are doing, to looking at the impact of what we are doing in the world beyond the
university.
The Evaluation Mosaic
2. Dean Johnson has proposed the development and institutionalization of a “mosaic of
evaluations” the provide evidence related to our impact on students, schools, and communities.
This mosaic implies that a wide range of assessments and strategies are needed, and that the
focus of assessment is on what happens due to our effort rather than our effort itself. To
encourage this shift in thinking, Dean Johnson released an RFP for seed money for field testing
of these efforts. In addition, a task force meets monthly to discuss ways to increase our
assessment of our impact on students, schools, and communities.
A key aspect of the mosaic is the understanding of the evaluators’ rather than the researcher’s
perspective. As researchers demonstrate their results “beyond a reasonable doubt,” evaluators seek to
find patterns in a preponderance of the data.
The first steps of the Evaluation Mosaic were these:
 Developed a tracking system of our graduates and their performance in the Cincinnati Public
School
 Release an RFP to unleash faculty creativity
 Hold discussion sessions on impact and how to measure our impact on p-12 students and
systems
 Seek out other colleges to replicate a mosaic
 Create a collaborate where we can learn from each other and create a picture of impact
A task force meets monthly In addition, a task force meets monthly to discuss ways to increase
our assessment of our impact on students, schools, and communities. Three proposals that will
begin collecting data during Spring, 2012 include:



Backtracking graduates’ performance at the University of Cincinnati and linking it to AYP
and performance data of those employed in Cincinnati Public Schools
Meta-analysis of candidates’ impact on p-12 students’ behavior change through
generating effect size on single case study designs
Meta-analysis of candidates’ impact on p-12 students’ literacy skills through generating
effect size on single case study designs
Transparency and Treatment Integrity
3. We have moved to a transparent system. Assessments are web-based, and rubrics are
presented to students. Candidate performances on those assessments are shared. The use of
those data to improve programming is shared with candidates, faculty members, and
cooperating teachers.
The Teacher Performance Assessment
4. We have become involved in a national effort in the development and implementation of the
Teacher Performance Assessment. During the 2011-2012 academic year we are engaged in a
full pilot of this assessment. Feedback has consistently been submitted to the assessment
development team at Stanford. In addition, programs have made program changes including
teaching candidates the technology of documenting their performance on video, increasing
opportunities to reflect in response to a prompt, and utilizing embedded formative assessments.
These changes have emerged in response to candidate and faculty discussions related to the
Teacher Performance Assessment.
These efforts are in early stages of implementation. We are committed to sustain them through pilot,
implementation, and analysis phases. At that time, we will make any revisions or enhancements needed
to continue our Transformation Initiative.