Programmatic Assessment
within Food Science and
Technology
Thomas H. Shellhammer
Nor’wester Professor of Fermentation Science
Important points we’ve learned since our
start in 2005
• Seek advice
• Start small
• Don’t expect all faculty to engage in the O&A
process
• Accept that the data will not be clean
• Resist collecting too much assessment data
• Focus on closing the loop
• Be flexible and adapt
FST Undergraduate Enrollment
180
160
140
Enrollment
120
100
80
60
40
20
0
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Academic year (beginning in Fall)
Our approach – historical perspective
2000
Notice from IFT to start implementing O&A
Institute of Food Technologists
• International, non-profit professional
organization for the advancement of food
science and technology
• ~22,000 members
• Higher Education Review Board
– Each program must have an assessment plan and
show how they implement that plan
– Each program must document how the results of the
assessment plan are used to continually improve food
science education.
Curriculum Matrix
Our approach – historical perspective
2000
Notice from IFT to start implementing O&A
2004
OSU asks for O&A plan
2005
IFT renewal - full O&A req’d in 5 years
Barb Walvoord visits OSU and FST
Rich Hartel visits FST
Fall retreat FST identifies initial programmatic
outcomes
Seek advice
Expert(s)
Practitioner(s)
Office of Academic Planning and Assessment
Barbara E. Walvoord, Ph.D
• Professor Emerita at the University of Notre Dame,
Indiana.
• Coordinated Notre Dame’s self-study and re-accreditation
visit by the North Central Association’s Higher Learning
Commission, 2004.
• Have consulted or led workshops at more than 350
institutions of higher education throughout the U.S., on
topics of assessment, teaching and learning, and writing
across the curriculum.
• Founding director of faculty-development programs at
Central College in Iowa, Loyola College in Maryland,
University of Cincinnati, University of Notre Dame. Each
program has won national recognition.
Rich Hartel
• Professor, University of Wisconsin, Department
of Food Science
• UW Teaching Academy Executive Committee
2004 – 2007
• IFT HERB member
Start small
First round looked just at two outcomes
Fall 2005 Faculty Retreat
1. We developed student learning outcomes for all of our
undergraduate courses.
2. We learned about how rubrics can help define and
consequently assess high level learning.
3. We identified 4 programmatic outcomes.
a. Core knowledge skills
b. Oral/written communication ability
c. Problem solving skills
d. Student engagement that leads to life-long learning
4. We engaged in thoughtful, meaningful, and holistic
discussions about we want our students to learn
Rubric for Written Communication in the Food Science & Technology Program
Instructor __________________ Course ___________ Term and Yr._________
Student: _________________________ Total Points: __________
Exceeds Expectations
(3 points)
Meets expectations
(2 points)
Topic is well
developed, effectively
supported and
appropriate for the
assignment. Effective
thinking is clearly and
creatively expressed
Topic is evident with
some supporting
details; generally
meets requirements of
assignment.
Organization Writing is clearly
organized with
effective introduction
and conclusion. Each
segment relates to
the others according
to a carefully planned
framework
Reasoning
Substantial, logical, &
concrete
development of ideas.
Assumptions are
made explicit. Details
are germane, original,
and convincingly
interpreted.
Writing demonstrates
some grasp of
organization with a
discernible theme and
supporting details.
Content
Language,
Grammar,
and Usage
Writing is free of
errors in grammar,
punctuation,
capitalization, and
spelling. Paragraphs
are well-focused and
coherent with a
logical connection of
points. Voice and
style are appropriate
for the type of paper
Offers somewhat
obvious support that
may be too broad.
Details are too
general, not
interpreted, irrelevant
to thesis, or
inappropriately
repetitive.
Writing has some
errors but these are
not too distracting.
Paragraphs
occasionally lack focus
or coherence. The
connection of ideas is
sometimes disjointed.
Voice and style don’t
always fit the type of
paper assigned.
Fails to Meet
Expectations (1
point)
Topic is poorly
developed.
Supporting details
absent or vague. Trite
ideas and/or unclear
wording reflect lack
of understanding of
topic and audience.
Writing is rambling
and unfocused, with
main theme and
supporting details
presented in a
disorganized
unrelated way.
Offers simplistic,
undeveloped, or
cryptic support for
the ideas.
Inappropriate or offtopic generalizations,
faulty assumptions,
errors of fact.
Errors are frequent
and distracting, so
that it is hard to
determine meaning.
Paragraphs generally
lack focus or
coherence. There is
not a logical
connection of ideas
or flow of sentences.
Voice and style are
Results from 2005-2006 academic year
1. Assessed written communication
in FST 424, FST 460 and FST 468
2. Assessed technical problem solving skills
in FST 424
3. Reviewed these data at the beginning of the
coming academic year (9/22/06)
Accept that the data will not be clean
Plan for ‘06-’07 year
Written Communication Assessment
1. Begin tracking students over time by assessing writing skills in FST 210
2. Correlate course grade against “reasoning” scores
3. Provide students with the writing skills rubric in their first year via FST 101
4. Assess written communication skills in FST 210, FST 424, FST 461, FST 468
Oral Communication Assessment
1. Assess oral communication skills in FST 407 – Senior seminar
2. Assess oral communication skills in FST 25 – Sensory Science
Core Knowledge
1. Identify venue to administer a core knowledge exam in the senior year
2. Prepare a core knowledge exam (all faculty to contribute questions)
Student Engagement
1. There is continued interest in a Fermentation Science alumni
association…looking for support to initiate this endeavor.
Results from ‘06-’07 year
Written Communication Assessment
88% achieved “meets expectation” or better
Creating new WIC course coincided with improved writing performance (writing
scores moving from 2.14 to 2.41) by Fermentation Science students and
brought enrollment in FST 424 within the guidelines of the Writing Program for
a writing intensive course..
Oral Communication Assessment
95% achieved “meets expectation” or better
Oral communication skills looked good.
Critical Thinking Skills Assessment
Concern over students’ understanding of the foundational knowledge in our
discipline.
Hi Dan,
After long thought, I decided to pull FST 468 from this.
I did this for numerous reasons:
1) I have recurring concerns about how it's objective application.
2) I was extremely busy this past academic year and this was the
easiest thing to let go of
3) FST 468 is a course that will soon be discontinued and therefore the
collection of data on this course seems like a waste of my most limited
resource: time
Sorry,
Not all faculty engage in programmatic
outcomes assessment
Need enough to have a critical mass
Plan for ‘07-’08 year
Written Communication Assessment
Reduce written assessment in favor of spending energy on the Core
Knowledge outcome
Oral Communication Assessment
Reduce oral communication assessment in favor of spending energy on the
Core Knowledge outcome
Critical Thinking Skills Assessment
Postpone further assessment of this area until we have Core Knowledge
assessment data
Student Engagement
Examine alumni survey instrument for use with UG student engagement
assessment
Begin moving FST Alumni Association forward
Plan for ‘07-’08 year
Core Knowledge Assessment
1. Four disciplinary groups formed
Food Chemistry - Penner (leading), Qian, Kennedy, McGorrin, Smith, Park, Ross
Food Microbiology - Osborne (leading), Daeschel, Bakalinsky, Goddik, Su, Sarker, Bruslind
Food Engineering - Shellhammer (leading), Zhao, Torres, Chaplen, Wells
Sensory Science - Lim (leading), Marin, Lederer, Colonna
2. Each group to identify key core knowledge concepts within each discipline that
are required of food scientists
3. Each group to map these concepts to our curriculum
4. Examine upstream course performance and correlate with performance in
foundational food science course - will aid in recruitment and long term student
performance
Be flexible and adapt if necessary
Resist simply collecting assessment data
Try to close the loop
Examples of closing the loop
• Adding writing intensive course improved written
communication scores
• Expanding credit load in two courses led to
improved quantitative scores
– Statistics sessions for Sensory Science course
– Problem solving sessions for Food Processing and
Packaging courses
2010 IFT renewal application
Programmatic Outcomes
• Students graduating with a B.S. in Food Science and Technology
will be able to:
• Effectively express themselves orally, graphically, and in writing.
(Communication Skills ).
• Apply scientific principles to solve problems in Food Science.
(Critical Thinking and Technical Problem Solving Skills ).
• Identify and explain essential foundational principles in Food
Science.
(Core Knowledge).
• Engage in activities that enhance their professional development.
(Professional Engagement).
Assessment Techniques
• Rubric-based assessment of specific learning outcomes
Oral and written communication, core knowledge,
technical problem solving, etc.
• Exit interviews performed by the Department Head with
all graduating seniors.
• Program growth and job placement data.
• Employer survey.
Important points we’ve learned since our
start in 2005
• Seek advice
• Start small
• Don’t expect all faculty to engage in the O&A
process
• Accept that the data will not be clean
• Resist collecting too much assessment data
• Focus on closing the loop
• Be flexible and adapt