From Here, Go Anywhere
"Tidewater Community College's Model
for Assessing Program and Student
Learning Outcomes"
“Effective assessment must begin
with real concerns of the
stakeholders and result in useful
information and recommendations
related to the purpose of
assessment.”
– D.W. Farmer & E.A. Napieralski
Dr. Michael E. Bryan, Coordinator
Office of Student Outcomes Assessment
August 13, 2004
From Here, Go Anywhere
Audience will understand
the Program/Discipline
Review reporting
process, including:
Intended
Outcomes
“Many faculty have been ‘doing
assessment’ for their own sake,
and have not been overwhelmed
with the task, without particular
experience in evaluation
History and driving forces methodology.” – O.J. Nichols (1995)
Supporting resources
Reporting process (e.g.,
deadline, schedule, tasks)
Report format with various
components (e.g.,
intended outcomes,
assessment measures,
analysis)
From Here, Go Anywhere
What is Assessment?
The systematic collection and analysis of
information to improve student learning and
program viability.
“…a learner-centered, teacherdirected approach designed to
improve student learning in the
individual classroom.”
– B.D. Wright (1991)
Why Assess?
From Here, Go Anywhere
“…assessment involves taking a second look at materials generated in the
classroom so that in addition to providing a basis for grading students, these
materials allow faculty to evaluate their teaching.” – B.D. Wright (1991)
Rationale and Driving Forces
“The institution demonstrates that each educational
program for which academic credit is awarded is (a)
approved by the faculty and the administration, and
(b) establishes and evaluates program and learning
outcomes.”
“The institution places primary responsibility for
the content, quality, and effectiveness of its
curriculum with its faculty.”
Principles of Accreditation, Standards for All Educational Programs,
Section III, 1 and 12, August 2003
From Here, Go Anywhere
Reporting
Mandates
“Reports of Institutional Effectiveness (ROIE) is an
annual report required by the State Council of Higher
Education for Virginia (SCHEV) and is intended to
provide meaningful information on the academic
quality and operational efficiency of Virginia’s public
institutions of higher education.”
“Each institutional report is organized into five
sections: institutional mission, college profile
measures, system-wide measures, and institutionspecific measures, and the core competencies.”
ROIE Homepage, http://research.schev.edu/roie/
“One of the most untruthful things possible, you know, is a
collection of facts, because they can be made to appear so many
different ways.” - Karl A. Menninger
From Here, Go Anywhere
Reporting
Mandates
Assessment processes at VCCS colleges have been
mandated for the past eighteen years. Senate Document No.
14, 1986 Virginia General Assembly, provided the legislative
mandate for assessment reporting and SCHEV established
the initial guidelines for colleges and universities to respond.
The Final Report of the Governor’s Blue Ribbon
Commission on Higher Education (February 2000) called for
the creation of a Quality Assurance Plan to:
(1) define core competencies for written communications, oral
communications, quantitative reasoning, scientific reasoning,
critical thinking, and information literacy;
(2) identify measures to assess students’ knowledge and skills;
(3) provide a vehicle to present results publicly.
Report of the VCCS Task Force on Assessing Core Competencies,
July 18, 2002
From Here, Go Anywhere
Local Reporting
Mandates
“Annual reports will be produced to account for what
has been accomplished in enhancing current
programs and implementing new ones, and to identify
emerging developments that call for updates to the
curriculum planning process.”
TCC Comprehensive Five-Year Curriculum Plan (2003-2008), Page 1,
August 2003
“While you are
experimenting, do not remain
content with the surface of
things. Don't become a mere
recorder of facts, but try to
penetrate the mystery of their
origin.” - Ivan Pavlov
From Here, Go Anywhere
Local Reporting
Mandates
Curriculum planning at TCC will occur in accord with the following principles:
The curriculum will be structured and articulated in ways that provide students with
multiple opportunities for connection and advancement as they enter, progress
through, exit, and potentially return to the college.
The curriculum will be responsive to the distinctive needs of business, industry,
government, agencies, and the larger community, continually reinventing itself in
relation to the immediate requests and long-term developments in various
occupational and technical fields.
The curriculum will be developed in consultation with employers and transfer
institutions that are the direct recipients of graduates.
All curricula will be built and maintained on a foundation of excellence. “In the
college transfer area, the college will benchmark itself to the most rigorous
standards of achievement in 4-year baccalaureate degree-granting institutions. In
the fields of occupational-technical study, the college will look to business and
benchmark its performance goals to the standards of the relevant industries
themselves” (Bearings on the Future: The TCC Strategic Plan).
TCC Comprehensive Five-Year Curriculum Plan (2003-2008), Pages 1 & 2,
August 2003
From Here, Go Anywhere
Local Reporting
Mandates
TCC will pursue the following corresponding strategies:
The college will regularly review programming at peer community colleges across
the nation with the goal of identifying current and emerging curricular areas for
possible development and implementation. As resources permit, faculty and
administrative teams will… bring back model programs and practices for adaptation
and implementation at TCC.
Through focus groups, advisory committees, and other means of formal and
informal relationships with employers and the higher education community, the
college will actively solicit input for needed program enhancements and development
as well as communicate the range and availability of existing programs.
The college will actively partner with those who are the primary stakeholders in the
successful outcome of students, looking for ways to engage their support and
sponsorship of students throughout the learning process.
The college will put into place a systematic process for assessing programmatic
and student learning outcomes, the results from which will be used for continuous
improvement of disciplines and programs. Other college activities, processes, and
initiatives (e.g., faculty hiring, adjunct faculty initiative, accreditation, professional
development and renewal, budget development, grant proposals, major gifts
campaign) will be aligned with the pursuit of curricular excellence.
From Here, Go Anywhere
Recap Why We
Assess
External Needs
Requirements for SACS Accreditation
Report of Institutional Effectiveness for
SCHEV
Quality Assurance Plan for VCCS
Internal Needs
Curriculum Planning Process &
Annual Reports for TCC
Intellectual Curiosity
Professionalism
From Here, Go Anywhere
Consultation Services provided by
the Coordinator for the Office of
Student Outcomes Assessment
(OSOA)
Assessment-centered assistance
such as word-smithing intended
outcomes or developing assessment
measures
Related pedagogical assistance
such as instructional strategies,
syllabus design, recordkeeping or
tracking data
OSOA Website
http://www.tcc.edu/welcome/
collegeadmin/OIE/SOA/
index.htm
Supporting
Resources
From Here, Go Anywhere
Recap Support
Resources
Consultation Services
Writing intended outcomes
Selecting or developing assessment
measures
Establishing performance targets
SOA Website
Homepage
SOA Reports
Assessment Clearinghouse
History, documentation, process, timeline, forms, toolkit,
sample reports, articles, best-practices, etc…
Reporting
Process
From Here, Go Anywhere
Tasks
“Lead Deans, in consultation with program/discipline faculty,
complete in its entirety the Program/Discipline Review and
Outcomes Assessment Report for each of their respective
programs and/or disciplines scheduled for review.”
Approved by the Vice President for Academic and Student Affairs on April 9, 2004
http://www.tcc.edu/welcome/collegeadmin/OIE/SOA/review/Process.htm
Provosts screen the reports and offer feedback prior final
submission to the Vice President for Academic and Student
Affairs.
“One important distinction in assessment methods is between
techniques that directly determine whether students have
mastered the content of their academic programs and those that
ask students to reflect on their learning.” – C.A. Palomba & W.
Banta (1999)
From Here, Go Anywhere
Annual Deadlines
April 1st: draft report to Provost.
May 1st: final report to VP.
Scheduled
Program/Disciplines
Cohort #1
4 Programs & 2 Disciplines:
* planning report during 02-03
* implement plan during 03-04
Cohort #2
Additional 15 Programs & 24
Disciplines:
* planning report during 03-04
* implement plan during 04-05.
Cohort #3 - TBD
Reporting
Process
Report Format
From Here, Go Anywhere
Working with the Form
MS Word Template  Online Form
Simply download the form from the OSOA
website to begin working with it
http://www.tcc.edu/welcome/collegeadmin/
OIE/SOA/review/Form.htm
Discuss the sections & elements then
demonstrate functions
Report Format
From Here, Go Anywhere
Sections to the
Plan
Background/
Descriptive Section
Programmatic
Outcomes Section
Student Learning
Outcomes Section
Other Noted Data
Section
From Here, Go Anywhere
Programmatic
Outcomes
Programmatic Outcomes
Top portion of the report form
(between the Background/Descriptive
Section and the Student Learning
Outcomes Section).
Apply same process to the nonstudent portion of the report.
Programmatic outcomes ensure your
program’s viability (e.g., FTEs,
headcounts, modernized equipment).
Report Format
From Here, Go Anywhere
Elements to the Outcomes Sections
Intended Student Outcomes: Knowledge, skills, or values students
should be able to demonstrate after completing your program or
discipline.
Intended Programmatic Outcomes: Non-student aspects that
contribute to your program’s viability.
Assessment Measures: Tool or indicator that yields the most timely
and accurate data used to determine attainment of a stated outcome.
Minimum Competency: A standard that represents a satisfactory,
acceptable-level of performance.
Actual Results: The real results of the data collected for each
assessment measure.
Highlights: Analysis and cited evidence that confirms outcome
attainment.
Enhancements: Analysis and cited evidence that identifies areas for
improvement; strategies planned for the coming year to address
these areas.
Enhancement Update: Describe the impact and level of success of
each enhancement activity (COMING NEXT YEAR).
Step #1:
Outcomes
Writing Intended Student Learning
Outcomes
From Here, Go Anywhere
Outcomes describe specific learning behaviors that
students should exhibit in the context of the
program/discipline/course.
Outcomes are the specific skills, values and attitudes
students should exhibit that reflect the broader goals
(e.g., for students in a freshman writing course, this
might be “students are able to develop a cogent
argument to support a position”).
Often in the assessment literature, “objectives”and
“outcomes” are used interchangeably.
“A student drove himself so hard that he missed the learning curve.”
- Pun of the Day
From Here, Go Anywhere
Step #2:
Measures
Identify & Review Existing
Assessment Measures
What information on student learning/performance do you
currently collect? What assessments are administered?
Direct and Indirect Measures
How do these data sources relate to your newly articulated
outcomes? Does the data provide rich understanding?
Are there gaps between the information you collect and
your outcomes?
What other information do you need to have to understand
whether students are meeting these outcomes?
From Here, Go Anywhere
Step #3:
Results
Establish Minimum Competency
What constitutes exemplary, satisfactory, needs
improvement (e.g., rubrics/scoring guides with
examples of A/B/C/F-level work)?
What are your realistic
expectations (e.g., 90%
will graduate/pass)?
“The great difficulty in
education is to get
experience out of ideas.”
- George Santayana
From Here, Go Anywhere
Step #3:
Results
Collect Data &
Compute Actual Results
Determine when and how to collect the data.
For course-based assessment data  faculty of
selected courses/sections administer
assessments and compile their results.
Lead Deans decide how to collect and enter the
course-based assessment data for analysis.
Compare expectations against actual results.
“You can observe a lot by just watching.”
“We made too many wrong mistakes.”
- Yogi Berra
From Here, Go Anywhere
Step #4: Highlights
& Enhancements
Closing the Loop: Interpreting and Using the Results
Highlights
Analyze the assessment measure data and cite
evidence that confirms the program/discipline/
course is accomplishing its intended outcome.
Enhancements
Cite evidence that identifies areas for
improvement and state enhancement
strategies/activities planned for the coming
year that address these areas.
Implement planned strategies the following
year (Fall 04) and report their impact (Spring
05).
From Here, Go Anywhere
Other Noted Data
Note all significant trends,
developments,
accomplishments, issues,
opportunities, threats, etc..
that were NOT cited
elsewhere in the report.
Make this the “brag book”
portion of the report (e.g.,
faculty accolades,
regional/state/national
awards).
Step #5: Other
Noted Data
Additional
Information
Need Assistance? Clarification?
From Here, Go Anywhere
Visit the Student Outcomes Assessment
(SOA) Website
http://www.tcc.edu/welcome/collegeadmin/OIE/SOA/
index.htm
Call or Email
Dr. Mike Bryan
(757) 822-1073
mbryan@tcc.edu
Questions?
From Here, Go Anywhere