Project Evaluation Plan
January 2013
Introduction
After we were awarded the grant, we developed a logic model for the project which was vetted
by the project team, by the external evaluator, Dr. Sarah Woodruff of Miami University, and by
the Internal Advisory Committee at its first meeting in November 2012. That logic model is
presented below.
The LEAF Logic Model
Initiative 1: Pipeline Improvements
Explanation: The pipeline provides a metaphor for how STEM scientists develop an
interest in the field, prepare for it, search for positions, begin professional fields, advance, and
achieve prominence. The NSF ADVANCE program focuses solely on those stages of the
pipeline that begin with the awarding of the doctorate for women STEM scientists who then
begin a career in academe. Continuing with the metaphor, this segment of the pipeline leaks, and
it leaks worse for women than for men.
Impact: The ultimate goal of LEAF’s pipeline improvement initiatives is to ensure that
all STEM units at UC meet or exceed national norms for the inclusion of women STEM
scientists at every career stage.
Long-term objectives (tracked separately for minority women)




Increase the percentage of qualified women candidates for STEM scientist positions
at UC
Increase the percentage of women hired for STEM scientist positions at UC at all
levels
Increase the percentage of tenure-track women who are promoted and tenured
Increase the percentage of women tenured associate professors who are promoted to
full professor
Intermediate objectives





Annual evaluations of deans and heads include consideration of pipeline
improvement initiatives in their units
Each unit has a logic model in place that explains how they will improve their
pipeline
Workload and RPT policies and procedures are aligned with the goal of pipeline
improvements
Recruitment and hiring policies and procedures are aligned with the goal of pipeline
improvements
Best practices and model documents are readily available on the LEAF website



The LEAF website includes a dashboard on pipeline improvements that is populated
with data from Institutional Research
Women STEM scientists have improved RPT dossiers and report greater confidence
about their support systems and chances for professional success
Women STEM scientists submitted more funding proposals
Short-term objectives



Organizational discourse results in increased awareness of and commitment to fixing
pipeline problems
Women STEM scientists participate in LEAF activities and feel empowered to share
their perspectives
All STEM deans and department heads have been trained on pipeline improvement
opportunities
Key Processes









Presidential, provostal, decanal, and head-level discourse explains and promotes goals
of pipeline improvement
LEAF staff, provostal staff, HR staff, and EEO staff coordinate in identifying system
level issues to be remediated
LEAF staff train heads and deans on logic model development
Recruitment, hiring, and RPT workshops are conducted for heads and deans
Best practices seminars bring in ideas from industry, government, and aspirational
benchmark peers
LEAF Grants
Professional development workshops designed to empower junior women STEM
faculty
Professional development workshops designed to engage mid-rank women STEM
faculty in their professions
Professional development workshops designed to engage senior women STEM
faculty in leadership
Initiative 2: Climate Transformation
Explanation: Hiring and retaining STEM scientists is not enough to make them
successful in advancing their careers and achieving professional success. The climate for women
in UC’s STEM disciplines has to improve by becoming more favorable for women and their
interests and concerns. At the same time, as units at UC develop more favorable climates for
women STEM scientists, it will improve their recruitment and retention efforts.
Impact: The ultimate goal of LEAF’s climate transformation initiatives is that women
STEM scientists at UC believe their units are places in which they can prosper and succeed.
Long-term objectives (tracked separately for minority women)





Increase the percentage of women STEM scientists at UC who report feeling engaged
and empowered
Increase the percentage of women STEM scientists at UC who report having good
mentoring
Increase the percentage of women STEM scientists at UC who report feeling that they
are achieving to their full potential
Decrease the percentage of women STEM scientists at UC who report concerns about
UC in terms of work-life balance
Decrease the discrepancy between men and women STEM scientists at UC on all
three of the above measures
Intermediate objectives








Annual evaluations of deans and heads include consideration of climate improvement
initiatives in their units
Each unit has a logic model in place that explains how they will improve their climate
for women STEM scientists
Policy changes at unit, college, and university levels promote work-life balance
Women STEM scientists are better represented in leadership positions and use these
positions to advocate for climate transformation
Best practices and model documents are readily available on the LEAF website
The LEAF website summary data on women and men STEM scientist’s views on
climate issues at UC
Women STEM scientists are empowered to advocate for climate change initiatives
Work-life balance issues relevant to women STEM scientists are prominently
featured in top-down discourse
Short-term objectives




Work-life balance issues relevant to women STEM scientists are prominently
featured in top-down discourse
Women STEM scientists participate in LEAF activities and feel empowered to share
their perspectives
All STEM deans and department heads have been trained on climate transformation
opportunities
Any STEM women scientist who wishes to have learning community or mentoring
experiences will have access to one
Key Processes

Presidential, provostal, and head-level discourse explains and promotes goals of
climate transformation









LEAF staff, provostal staff, HR staff, and EEO staff coordinate in identifying system
level issues to be remediated
LEAF staff train heads and deans on logic model development
Climate and work-life balance workshops are conducted for heads and deans
LEAF allies plan and initiate bottom-up change efforts
LEAF staff create organizational support (e.g., task forces) to address climate issues
that emerge from our research or learning communities as among those most
important address
Best practices seminars bring in ideas from industry, government, and aspirational
benchmark peers
Learning communities are developed that wrap-around the three professional
development workshop series
LEAF facilitates identification of mentors and mentees, and their matching
Visiting scholars to UC help women STEM scientists and the academic leadership
identify new and creative ways of promoting climate transformation
Initiative 3: Mutuality of Interest
Explanation: Our initiative consists in equal parts of top-down and bottom-up
approaches. An example of the former is the expectation that the provost will expect deans and
heads to have unit-level logic models for pipeline and climate improvements. Learning
communities and the expectation that the members will self-organize in advocating for change
illustrates a bottom-up approach. We have proposed that the Accountability and Advocacy
Council (AAC) will manage the interface between top-down and bottom-up approaches by
promoting alignment of the interests of the senior administration and the women STEM
scientists and fostering the kind of mutuality of interest that will ensure sustainability.
Impact: Top-down approaches and bottom-up approaches will in the main be mutually
reinforcing and foster a sense of shared responsibility that will help ensure that LEAF will be
sustained beyond the five-year project period.
Long-term objectives

The long-term objective is sustainability, and perhaps even an end to the idea of the
AAC.
Intermediate objectives


LEAF participants will view the senior administration was supportive and committed
to the project.
Senior administrators and deans will favorably view the contributions of LEAF
participants.
Short-term objectives

The AAC will be a forum for discussing what’s working and what’s not working


The AAC can recommend to the senior administration and to the LEAF participants
about ways to slightly adjust their approaches in order to foster mutuality of interests
The AAC can act like an ombuds to ensure that messages from below get heard at the
top, and that messages from the top are properly interpreted by participants
Key Processes




AAC members are fully briefed by LEAF staff on all data and all activities
AAC members make it known that communications from LEAF staff and LEAF
participants about any problems with the program or in their departments are
welcome, and that the AAC represents a Safe Zone.
AAC and program staff recruit LEAF Allies to serve as our eyes and ears in the units
and colleges and to contribute their special skills on an ad hoc basis.
AAC meet periodically with key members of the senior administration to get
background reports on their concerns and their assessments of what is possible when
Accomplishments to Date
Following is a summary of the evaluation activities we have carried out to date.










We have contracted with our external evaluator, Dr. Sarah Woodruff, of Miami
University.
We have met and talked numerous times by phone to coordinate the work of the internal
and external evaluation teams.
We have obtained 25 years of data on STEM appointments at UC and helped the external
evaluation team merge the files together so that we can study retention.
We have obtained 2012 salary data for the required salary study.
We have worked with the UC architect’s office to gain access to space management
system for the required space study.
We have determined that we can obtain data from the UC provost’s office on
reappointment, promotion and tenure decisions from 2011 forward. These will be used in
creating the NSF-required Indicator Tables for our annual report.
We have educated Internal Advisory Committee about logic models.
We created program logic model (presented above).
We have identified the key informants who will be interviewed in the near future by the
external evaluation team.
Have acquired all job ads from last five years for coding.
Current Efforts
We are in the process of mapping out where the data will be obtained for each of the long-term,
intermediate-term, and short-term objectives.
We are in the process of setting up the LEAF office and getting the staff connected to the people
who can help them acquire and archive key indicators of progress.