Overview of Institutional Effectiveness
Concepts and Measures
Andreea M. Serban, Ph.D.
Solano Community College
July 19, 2006
1
Definitions
Institutional Effectiveness
“The ability of an institution to match its performance to
established purposes as stated in its mission” (Alfred et al,
p.6)
Expression of institutional values and priorities
“An internal process of planning and evaluation that is
intended to ensure that the college’s performance matches its
purposes. A global process that uses assessment strategies to
demonstrate accountability” (Midlands Technical College,
p.ix).
2
Purposes
Clarification of Mission
Improvement
Identification of Priorities
Working Document
Planning and Decision Making
Enhance Reputation
Historical Record
Accreditation Requirement
Information for Prospective and Current Students
3
Core Concepts
Resources
Stakeholder
Needs
Mission
Results
Costs
4
Institutional Effectiveness and Planning
•Institutional effectiveness measures are part of the annual
evaluation of the strategic plan
•The information is used to determine progress towards
achievement of stated goals in the strategic plan
•Establish a structure to monitor, report and take corrective
actions based on institutional effectiveness measures
•Departmental effectiveness measures are linked to and, as
appropriate, aggregate into institutional effectiveness
measures
•The college needs to have a systematic approach to
communicating the data from the institutional effectiveness
to internal and external constituents on a regular cycle
(annual basis suggested)
5
Modeling and Measuring Effectiveness
Outcomes serve as benchmarks for the college mission by
comparing results with purpose, within the limits of costs and
available resources
When outcomes are compared with an institution’s mission and
goals, the result is a composite picture of an institution’s
effectiveness
An indicator that helps create this composite picture describes a
condition or result that the community college can produce
regularly, report publicly, and use systematically in decision
making
A core indicator is a measure that describes a critical, widely
recognized outcome of mission – one that is clearly responsive to
key constituent groups and is produced regularly.
Source: AACC
6
1999 AACC Model
Mission and Core Indicators
Student Progress
Student Goal Attainment
Persistence (Fall to Fall)
Degree Completion Rate
Workforce Development
Placement Rate in the Workforce
Employer Assessment of Students
Licensure/Certification Pass Rates
Client Assessment of Programs and Services
General Education
Demonstration of Critical Literacy Skills
Demonstration of Citizenship Skills
7
1999 AACC Model
Mission and Core Indicators (continued)
Transfer Preparation
Number and Rate Who Transfer
Performance After Transfer
Developmental Skills
Success in Subsequent, Related Coursework
Outreach
Participation Rate in Service Area
Responsiveness to Community Needs
8
Feasibility of the AACC Model
Core Indicators for which
Data Are Available
Core Indicators for which
Data Are Not Easily Available
Persistence (Fall to Fall)
Student Goal Attainment
Degree Completion Rates
Employer Assessment of Students
Job Placement Rates (data
have limitations)
Client Assessment of Programs and
Services (at a minimum use student
surveys)
Number and Rate Who
Transfer
Success in Subsequent,
Related Courses
Demonstration of Critical Literacy
Skills
Demonstration of Citizenship Skills
Licensure/Pass Rates
Performance After Transfer
Participation Rate in the
Service Area
Responsiveness to Community Needs
9
Expanding on the AACC Model
Examples from Community Colleges
Santa Barbara City College – Criteria for selecting
institutional effectiveness measures
•Related to mission and College Plan
•Reflective of the changes in relevant literature
•Reflective of the ability of the Office of Institutional
Assessment, Research and Planning to obtain the information
from the college student system, other college
databases/sources or external sources
•Quantifiable
•Accepted by the college community as valid
10
Expanding on the AACC Model
Examples from Community Colleges
Santa Barbara City College (list of measures provided in
separate document)
•Measures of institutional effectiveness developed for and
linked to each major area of the college plan
•Discussion of institutional effectiveness trends and
implications for the college included in the annual
evaluation of the college plan
•Annual presentations and discussions with the Board of
Trustees and shared governance groups (College Planning
Council, Academic Senate)
11
Expanding on the AACC Model
Examples from Community Colleges
Long Beach City College (annual institutional effectiveness
report and latest presentation to Board of Trustees provided
in separate documents)
•Uses the AACC outcome dimensions tied to Astin’s
assessment model which differentiates between inputs,
process/environment and outcomes
•Measures of institutional effectiveness tied to the long
range goals specified in the Educational Master Plan
12
Implications for Solano Community College
•The college had previously developed performance indicators
related to the goals included in the Strategic Plan as outlined in
the “Review and Validation of Solano Community College
District’s Strategic Plan” May 2000. Those performance
indicators represent a solid base for creating an institutional
effectiveness framework related to the Strategic Plan goals and
objectives.
•Institutional effectiveness measures, once developed, should be
monitored, reported, discussed and integrated into the annual
evaluation of the Strategic Plan.
13
References
Alfred, R., Ewell, P., Hudgins, J., and McClenney, K. (1999). Core
Indicators of Effectiveness for Community Colleges, Second
Edition. Washington, D C: Community College Press,
American Association for Community Colleges.
Midlands Technical College. (1997). Managing Your Institution’s
Effectiveness. A User Guide. Washington, D C: Community
College Press, American Association for Community Colleges.
14
Resources
Banta, T. W., Borden, V.M.H. (1994). Performance Indicators for Accountability and
Improvement. In V.M.H Borden and T. W. Banta (eds.), Using Performance Indicators to
Guide Strategic Decision Making. New Directions for Institutional Research No. 82: San
Francisco, CA: Jossey Bass.
Burke, J. C., Modarresi, S., Serban, A.M. (1999). Performance: Shouldn’t It Count for
Something in State Budgeting?. Change, November/December, 31(6), 16-23.
Burke, J. C. (1998). Performance Funding Indicators: Concerns, Values, and Models for
State Colleges and Universities. In J. C. Burke and A. M. Serban (eds), Performance
Funding for Public Higher Education: Fad or Trend?. New Directions for Institutional
Research No. 97. San Francisco, CA: Jossey Bass.
Cleary, T. S. (2001). Defining Quality Through the Eyes of Campus Stakeholders.
Community College Journal, August/September, 41-47.
Council for Higher Education Accreditation. (2000). The Common Data Project. CHEA
Occasional Paper, August. Washington: DC: CHEA.
15
Resources
Ewell, P. T. (1992). Outcomes Assessment, Institutional Effectiveness, and Accreditation:
A Conceptual Exploration. Resource Paper for the Council on Postsecondary
Accreditation Task Force on Institutional Effectiveness (ERIC Document No. 343513).
McLeod, M. W., Cotten, D. K. (1998). Essential Decisions in Institutional Effectiveness
Assessment. Visions: The Journal of Applied Research for the Florida Association of
Community Colleges, 2(1), 39-42.
Nichols, J. O. (1995). A Practitioner’s Handbook for Institutional Effectiveness and
Student Outcomes Assessment Implementation. Bronx, NY: Agathon Press.
Ronco, S. L., Brown, S. G.. (2000). Finding the "Start Line" with an Institutional
Effectiveness Inventory. Paper presented at the Annual Forum of the Association for
Institutional Research, Cincinnati, OH, May 21-23, 2000.
Roueche, J. E., Roueche, S. D. E., Eileen E.. (2001). Pursuing Excellence: The Community
College of Denver. Community College Journal of Research and Practice, 25(7), 517-537.
Roueche, J. E., Johnson, L. R., Roueche, S. D., & Associates. (1997). Embracing the Tiger:
The Effectiveness Debate and the Community College. Washington, D.C.: The
Community College Press.
16
Resources
Serban, A.M. (2001). Institutional Effectiveness and Performance Funding for
Community Colleges: Where Do Institutional and State Values and Priorities Converge?
Presented at the Annual Forum of the Association for Institutional Research. Long Beach,
CA, June 4, 2001.
Sorensen, James E. (1998). An Investigative Study on the Systematic Application of
Effectiveness Indicators for Institutional Improvement in Northwest Community
Colleges.
Sullivan, M. M., Wilds, P. C. (2001). Institutional Effectiveness: More Than Measuring
Objectives, More Than Student Assessment. Assessment Update, 13(9), 4-6.
17