Project Details
Title
Developing and Piloting a Program Review Process Status
REVIEWED
Category 1-Helping Students Learn
Updated
08-29-2010
Timeline
Reviewed
10-15-2010
Planned Project Kickoff
Created
02-17-2010
02-26-2010
Target Completion
Last Modified 10-15-2010
02-26-2011

1: Project Accomplishments and Status

ANSWER: The Program Review Committee was organized in March, 2010 with a total of 8
members representing a cross section of the campus including both administration and
faculty. The Committee has been meeting monthly since April, 2010 with several additional
meetings when it has been determined to be necessary. Arrangements were made with
Consultant Bruce McComb in May to lead the Committee, first in understanding the
construction of an evaluation plan,* and second in the creation of an evaluation plan for our
Program Review process.
*A three stage process is used to develop evaluation plans for many projects at NC State.
In this process, project team members participate in a group process to first develop a logic
model, then an evaluation plan and finally a data template under the guidance of a
consultant/facilitator.

The logic model identifies key inputs, activities (tasks or process), outputs (typically
activity counts and measures of reach), outcomes and finally, assumption(s) upon
which the initiative is based.

The evaluation plan encompasses both formative evaluation and summative
evaluation. Formative evaluation is designed to assess whether the initiative is
working as intended and is likely to produce the intended outcomes. The summative
evaluation focuses on looking at the outcomes achieved -- Were they what was
expected and did they have the intended impact? The summative evaluation results
can also be used to inform long term continuous improvement of the initiative and to
make judgments on the overall value of the initiative. The formative and summative
questions guide the team in determining data or information needed, sources of data,
comparison groups, and methods of data collection and analysis. Based upon this
input, the consultant/facilitator and Institutional Research finalize the evaluation plan.
The data template is a spread sheet that is organized to contain and display the data and information necessary to carry
out the evaluation plan. It serves as a check list to help assure timely collection, tabulation and display of the data. The
team develops the basic structure of the data template. Institutional Research sets up the procedures to populate the
data template.
To put the evaluation plan into action, the data templates and supporting information from
surveys and focus groups are used to answer the evaluation questions.

REVIEW: Over the past few months the institution has made good progress on this project
and has created a solid foundation for the remainder. By working with an experienced
consultant and following a detailed three-stage process to formulate an evaluation plan, the
college has taken many positive steps. Once you begin to implement the evaluation plan,
there will likely to be challenges along the way. However, the three steps described above
appear to be logical, which may help mitigate some complications. In the end, the college
will be much stronger, as this commitment to continuous improvement (AQIP Category 8)
and support of institutional operations (AQIP Category 6) will create an environment that aids
student learning (AQIP Category 1). You can do it!

2: Institution Involvement

ANSWER: Faculty members were recruited for membership by the Faculty Caucus, the VP for
Learning asked the Dean of Technology and Workforce Development to serve as chair and
the other administrators volunteered to serve. The Committee members have remained
engaged throughout the project from what appears to be a genuine desire to create a
useable product that will enhance student learning in an efficient manner.

REVIEW: The openness to reach out to the faculty is an important element of this process, as
it demonstrates valuing their work and opinions (AQIP Category 4 Valuing People). Although
the involvement of the current project team appears to be appropriate, as this process
continues to evolve it will be important to evaluate the possible involvement of additional key
stakeholders. Are the right people working on each task associated with this project, or
could changes improve certain tasks? Are there new employees on campus who could be an
asset to this process? Just as the institution has an eye on continuous quality improvement,
so should the action projects. This can help keep momentum after the initial enthusiasm of
the project begins to wane.

3: Next Steps

ANSWER: The committee will continue to meet monthly with additional meetings as
needed. The evaluation plan is scheduled to be completed November 15, 2010.
A sub-committee will begin drafting sections of a review process which will then be
presented to the full committee for review and reaction with modifications made as
warranted. The Program Review process will be field-tested by January 1, 2011. Refinements
will be made as needed. Full implementation is scheduled for the 2011-12 academic year.

REVIEW: The planned next steps seem ambitious, but do-able. Given the strain that is
sometimes associated with the end of semesters, you may find that sticking to the original
timeline is not feasible. Although it is important to not lose the momentum and enthusiasm,
it is even more vital not to stick to an artificial timeline only to meet the deadline. The most
important element is that the project is successful and completed correctly, even if it takes
more time than was originally planned. If the institution encounters such a situation, be
prepared to review what it is that you hope to accomplish, looking for places where slight
revisions to the timeline can be made to allow for maximum results.

4: Resulting Effective Practices

ANSWER: The process of creating an evaluation plan has proven to be helpful and will be a
tool for use in future planning activities.

REVIEW: An important step in the lifecycle of any project is a careful evaluation of the
process upon its conclusion. In this particular case, you may find many instances where this
evaluation plan can be used that have not been previously considered.

5: Project Challenges

ANSWER: The committee has worked well as a team. There has been good dialog and
interaction within the committee with all members participating. The greatest challenge has
simply been finding time when all members are able to attend.

REVIEW: If time continues to be an issue, it may be necessary to reevaluate the committee’s
work. Can some of the work be divided into subcommittees or small groups where
scheduling may not be as much of an issue? Additionally, if it is a particularly busy period of
the year for some committee members, are there other people on campus who could attend
some meetings or help with the work to avoid overburdening committee members?
As the committee continues its work this year, it will also be important to evaluate where the
project has been and where it is going. It is possible there are additional
challenges/obstacles that exist but may not be readily apparent. Taking a step back to
reassess a project can sometimes provide a new vision or clarity that was not possible
previously.