Center to Improve Project Performance
Initial Meeting Summary
Date of Meeting: April 30, 2008
Attendees: Pat Gonzalez (OSEP—Contracting Officer’s Representative)
Carol Matamoro Allen (ED Contracts—Contract Specialist)
Zalma Ross (ED Contracts—Contracting Officer)
Tom Fiore (Westat—Project Director)
Elaine Carlson (Westat)
Dave Judkins (Westat)
Susan Berkowitz (Westat)
OSEP staff who are or will be Project Officers for Cohort 1 projects
The meeting proceeded according to the agenda prepared in advance by Drs. Gonzalez and Fiore.
The agenda is appended to this summary.
Clarification of Issues from Westat’s Proposal
The following points were made in discussions regarding the questions Dr. Gonzalez raised related
to Westat’s proposal:
 Martha Wilaby was added to the CIPP staff at the time Westat responded to questions and
prepared the revised final budget. Ms. Wilaby is not a member of any of the CIPP teams. She
will work closely with Dr. Fiore and provide management support, particular related to project
logistics and reporting.
 CIPP staff will craft some additional language for inclusion in the selected projects’ cooperative
agreements regarding the projects’ responsibilities for their summative evaluations and the
expectations for coordination with CIPP. Dr. Gonzalez will send Dr. Fiore a cooperative
agreement template and relevant prospectuses. Among other issues, the cooperative agreements
should address the use of carryover funds. The first cooperative agreement that will require this
language will be for the National Parent Technical Assistance Center, which will be funded by
July.
 Dr. Fiore described the work strategy plan mentioned on p. 19 of the Westat proposal as an
internal document that required no OSEP review. The discussion shifted to the need for a
written work plan, developed by CIPP staff in conjunction with selected project staff, the OSEP
PO, and the COR. This plan will delineate clear expectations for the summative evaluation and
the specific responsibilities that fall to CIPP and to the selected project.
 Dr. Fiore acknowledged that Westat had mistakenly assumed that the selected projects would
have prepared their own summative evaluation designs for their applications. Dr. Gonzalez
pointed out that OSEP did not want the selected projects to prepare those designs knowing that
CIPP was likely to require their redoing. Each applicant, however, will have prepared a logic
1



model, and that logic model should be designed to serve both the formative and the summative
evaluation.
Dr. Fiore clarified how Westat will handle assignments related to requests for general evaluation
TA that come from other OSEP grantees. He explained that final decisions had not been made
but that the preliminary plan was for Compass staff to provide the first level of response. They
would receive toll-free calls and TA emails and, where more in-depth response or an ongoing
relationship is required, they would assign responsibility to individuals with the relevant
knowledge at Compass or Westat.
Dr. Gonzalez indicated that the RFP had not been clear regarding the expectation that CIPP
staff provide an annual workshop on an evaluation issue. Westat interpreted the language in the
RFP as calling for a single workshop, and the proposed budget reflected that interpretation.
Dr. Gonzalez asked about options for arranging for Westat to conduct yearly workshops.
Dr. Fiore responded that Westat would be glad to do so, and he assumed money would be
available within the budget for this general TA function. It was agreed Westat would provide a
workshop this summer at the OSEP Project Directors’ Conference and that Drs. Gonzalez and
Fiore would wait until next year to evaluate the feasibility of subsequent workshops and the cost
implications.
Dr. Gonzalez raised the need for a set of core evaluation questions (approximately four) that
would be responsive to requests from OMB and the Budget Office. She suggested that
questions such as those on page 15 of Westat’s proposal might be appropriate. She also
mentioned that the questions might focus on the areas that are the focus of GPRA: quality, use,
relevance.
Contractual Issues
Zalma Ross and Carol Matamoro-Allen discussed a number of contractual issues:
 Deliverables should be sent to Dr. Gonzalez with a copy to Ms. Matamoro-Allen.
 Westat should notify Ms. Matamoro-Allen and Dr. Gonzalez when 75 percent of the obligated
funds (base funding is $1.7M) has been used.
 The QASP amount is $90K, and it is based on one evaluation plan each year.
 Dr. Fiore will send an email to Ms. Matamoro-Allen to point out a small error under Sec. B-2,
item 18 (Key Personnel Designation) in the CIPP contract where Elaine Carlson was
erroneously named as Associate Project Director.
 Dr. Fiore was reminded to inform Ms. Matamoro-Allen of changes in the rate agreement when
they occur.
 In response to a question from Dr. Fiore, Ms. Ross said that no personnel security clearances
were needed for this low-risk project.
Status of CELL
Jennifer Tschantz, Christy Kavulic, and Gail Houle of OSEP presented information regarding the
Center for Early Literacy Learning (CELL), including the following:
 CELL is part of OSEP’s Technical Assistance and Dissemination (TA&D) Network.
 CELL was funded through a 5-year cooperative agreement and began in October 2006—it is a
new center, not a re-compete of an existing center.
 The grantee is The Orelena Hawks Puckett Institute in Asheville, North Carolina.
2





CELL staff are still doing research/literature review and developing materials—they are referring
to this effort as “practice-based research syntheses” and are currently developing “Practice
Guides.”
CELL will have 10 model-demo sites, called field test sites, where CELL will introduce and test
best practices.
AIR has a subcontract under CELL to provide the TA function for the center.
AIR asked states to apply for TA and received eight positive responses; TA is being provided to
all eight states under a system that groups each into one of three tiers based on state readiness.
Westat should begin its consideration of the CELL summative evaluation plan by focusing on
the existing plan prepared by The Study Group, with consideration of the following information:
o The six core questions are still appropriate.
o CELL staff have reported that they are collecting data for questions one through four.
o CELL staff do not want to address question five, but OSEP staff do.
o CELL staff are OK with question six, although they have not starting collecting data on it.
o CELL staff are developing a fidelity measure for TA and intervention, which will be a
source of data.
o Observations will probably not be an appropriate way to collect evaluation data.
o CELL staff do not believe the evidence-based PD practices and the evidence-based TA
practices in The Study Group’s evaluation plan are specific enough to be relevant to
CELL; OSEP staff do not believe those practices are validated.
CIPP Overview by Westat Staff
Dr. Fiore provided an overview of the CIPP project. Below is a list of some points in his
description:
 CIPP will provide direction to 12 projects, selected by OSEP, as those projects design and
implement their summative evaluations.
 CIPP will also provide TA to other OSEP-funded projects to support improvements in their
evaluation planning, methods, analysis, and reporting.
 CIPP will identify best practices in project design or implementation.
 CIPP will develop evaluation models or methods that will support effective evaluation of future
projects.
 Westat, and its subcontractor Compass Consulting, will organize the support it provides to the
12 selected projects around three teams:
o Design Team—oversee all the work with the selected projects, refine each project’s logic
model, establish the work strategy for each project, prepare specifications for each
project’s summative evaluation, develop draft and final summative evaluation plan for each
project.
o Technical Assistance Team—serve as primary contact with the liaison for each selected
project, work closely with Design Team members on the refining of the logic models and
the development of summative evaluation plans.
o Analytic/Statistical Team—work directly with project staff or indirectly through CIPP
other CIPP staff to support the work of the Design Team and the TA Team through
consultation on sampling, quantitative analysis, and qualitative analysis.
 In addition to developing plans, the CIPP staff will remain involved in each of the 12 project’s
summative evaluations throughout the life of the contract. CIPP staff will provide significant
3


support to projects or actually conduct many summative evaluation activities, including the
following:
o Selecting samples
o Developing draft instruments
o Monitoring data collection and performing reliability checks
o Analyzing evaluation data
o Interpreting findings
o Organizing, reviewing, and editing final project summative evaluation reports
In conjunction with providing general evaluation TA to other OSEP-funded projects, CIPP staff
will create a Web site and will conduct workshops at OSEP-sponsored conferences.
[Dr. Gonzalez said that whether to do a Web site is up to Westat. She also pointed out that we
might need to set up the Web site with mediated access because grantees funded under the
Technology Program and the State Professional Development Grant Program should not have
access.]
CIPP staff will prepare the following reports:
o Brief evaluation summaries, including project-to-date evaluation findings for 3+2s and
quick-turnaround project summaries.
o A contract final report that will present (1) the status of the summative evaluations for each
of the selected projects; (2) models or activities in TA, professional development,
technology, reporting and other methods of disseminating information that show potential
for future use; (3) what has been learned about effective and ineffective summative
evaluation designs and implementation; (4) suggestions for improving the costeffectiveness and efficacy of CIPP.
Presentations on Cohort 1 Projects by OSEP Project Officers
The kickoff meeting concluded with presentations from OSEP staff regarding the upcoming
priorities that will result in the seven projects that, with CELL, will constitute the first cohort of
projects with which CIPP will work. The presumptive OSEP Project Officers for each project
described the expected focus, scope, and funding level.
Dr. Gonzalez noted that the projects will be expected to set aside grant funds specifically for the
summative evaluation work with CIPP. Although the money per year may seem modest, projects
will carry money over from year to year, which means that they will have more money in some years
than their per-year average evaluation dollars. This is expected to allow for adequate levels of data
collection.
4
APPENDIX
Center to Improve Project Performance
Kickoff Meeting
Wednesday, April 30, 2008
Washington, D.C.
Agenda
11:30 – 1:00
Lunch and discussion
o Clarification of issues from Westat proposal
- Work strategy (p. 19)
- Applicants’ summative evaluation designs (p. 23)
- Assignment of providers of general TA (p. 35)
o Resolving annual workshop need
o Possible wording for inclusion in cooperative agreements
o Core evaluation questions for Budget and OMB
o
Deliverables and schedule issues
1:00 – 1:30
Contractual issues (including security requirements)
1:30 – 2:00
Status of CELL
2:00 – 2:30
CIPP overview presentation by Westat
2:30 – 2:45
Break
2:45 – 4:00
Presentations on Cohort 1 projects by OSEP project officers
5