Chapter
3
INSTITUTIONAL EFFECTIVENESS
Introduction
This chapter describes the process by which Troy State University ensures quality in its
educational programs, administration, educational support services, and Office of Institutional
Research, Planning, and Effectiveness (OIRPE). This process is known as the Institutional
Effectiveness Cycle (IEC), a process by which (a) reasonable expected results concerning the
achievements of Troy State University students, administration, staff, academic and nonacademic
programs, operations, and procedures are established, (b) assessment instruments are identified
that measure the degree to which the expected results are met, (c) program strengths and
weaknesses are identified through review of assessment results, and (d) plans for improvement
are prepared to strengthen identified weaknesses of the program/unit/service. The IEC was
developed and instituted prior to the Southern Association of Colleges and Schools Commission on
Colleges’ (SACS-COC) visit to Troy State University in 1993. The IEC is a continuous improvement
process implemented in a collaborative effort by all academic and non-academic areas across the
University. The change from the quarter system to the semester system, undertaken beginning in
Academic Year (AY) 1999-2000, allowed the opportunity for program revisions and restructuring.
Since the semester conversion, there have been two complete cycles of the IEC. This chapter
discusses the process of institutional effectiveness since the semester conversion.
3.1
PLANNING AND EVALUATION: EDUCATIONAL PROGRAMS
3.1.1
Educational activities of an institution include teaching, research and public
service. Planning and evaluation for these activities must be systematic,
broad based, interrelated and appropriate to the institution.
The planning process for educational activities which include teaching, research, and public
service is systematic, broad-based, and grounded in the mission of Troy State University. In 1989,
recognizing the importance of institutional effectiveness, the Board of Trustees of Troy State
University issued resolutions that established the OIRPE and the policy of continued, broad-based
institutional planning at the University. The first Committee on Institutional Effectiveness was
composed of 25 members, drawn from every division of the Institution. By the fall of 1990, all
faculty and staff were supplied with the Troy State University Institutional Effectiveness Plan, 1991.
In August 1993, the Manual for Developing Planning Documents was completed to provide
content, process and procedural information in regard to developing the Six Points of Institutional
Effectiveness (SPIE), guidance in conducting the Self-Study, writing Planning Statements (shortrange and long-range), and integrating budget planning for successful implementation of plans. In
3-2
1995, to direct and undergird planning to the end of the century, the Troy State University
Challenge 2000: Strategic Plan to the Twenty-first Century was developed and provided
organizational visions along with strategic directions, strategic goals, guiding principles, external
and internal factors, and planning assumptions.
In 2000, the Chancellor of the Troy State University System charged the Strategic
Planning Council and Vice Chancellor to review strengths and weaknesses of the University and to
look to the external environment for trends and future opportunities for Troy State University. Over
100 faculty and staff on 65 different committees compiled facts and prepared findings and
assumptions that were forwarded to the Chancellor, Vice Chancellor, Provost, and Vice President
for Institutional Research, Planning, and Effectiveness. They, along with a strategic planning
consultant, processed the facts, findings, and assumptions. The committee findings were
organized into nine strategic planning assumptions. This broad-based approach provided the vision
statement, strategic initiatives, goals, guiding principles, external and internal factors, and planning
assumptions through the year 2005. The strategic plans of Troy State University are articulated in
Over the Horizon: Strong Values--Clear Vision, Troy State University Strategic Plan: 2001-2005.
This document replaced the previous five-year strategic plan: Troy State University Challenge
2000: Strategic Plan to the Twenty-first Century. The Strategic Plan is reviewed as a part of the
annual planning process to ensure that the University is meeting the established strategic
initiatives.
Troy State University’s official Mission Statement serves as the unifying theme for the
goals and objectives of each program/unit/service at Troy State University. Planning and evaluation
are systematically accomplished through the Institutional Effectiveness Cycle, which can be found
in the Manual for Annual Planning October 2000 (a revision of the annual planning documents
discussed above). The IEC facilitates the interrelationship of planning and evaluation for the
educational programs and activities, and the faculty assessment instrument ensures that the
elements of teaching, research, and service are incorporated into planning. The IEC also provides
a means for utilizing the assessment results in a systematic, broad-based and continuous
planning/evaluation process and for developing procedures by which corrective measures will be
documented, implemented, and evaluated. The IEC consists of five interrelated components (as
illustrated in Figure 3.1.1.) including the following:
1. Six Points of Institutional Effectiveness--purpose, objectives, expected results,
assessment instruments, assessment procedures, and persons responsible for the
statement of results including plans for improvement);
2. Annual Assessment Reports (known as the Self-Study prior to AY 2000-2001);
3. Annual Plans (contains the Statements of Achievements to evaluate the previous
year’s plans and a description of the plans to be implemented during the next year);
4. Annual Budgets; and
5. Implementation of the Plans.
3-3
ANNUAL PLANNING THROUGH THE INSTITUTIONAL EFFECTIVENESS CYCLE
Step One:
Prepare Six Points of Institutional
Effectiveness
(January)
Step Five:
Implementation
Of Plan
Step Two:
Prepare Annual
Assessment Report
(February)
Step Four:
Prepare Annual Budget
with Annual Plan attached
(May-August)
Step Three:
Prepare Annual Plan
(April)
Figure 3.1.1
The Institutional Effectiveness Cycle begins in January as all academic and non-academic
units develop or update the SPIE for their programs/units or service. The SPIE:
1. Define the purpose of the program, operation or position and how the purpose relates to
the purpose (mission) of the University;
2. Describe the objectives (how that purpose will be achieved);
3. State expected results;
4. Identify assessment instruments or methods;
5. Describe procedures for administering the assessment instruments or methods; and
6. State the person or persons responsible for the statement of results.
The SPIE documents are received, updated and submitted to the Office of Institutional
Research, Planning, and Effectiveness and published on its website. These documents are
updated annually.
During February, the SPIE, assessment data contained in the Compendium of Survey and
Test Results, the annual Fact Book (published on the OIRPE website each fall), grade reports, and
other assessment data are used to prepare an Annual Assessment Report for each academic
program, non-academic operation, and administrative position to the dean’s level. The Annual
Assessment Report states the actual results of assessment for the previous academic year;
3-4
analyzes the assessment results and compares them to the expected results; identifies strengths
and weaknesses of the programs, functions, or operations; and proposes Plans for Improvement.
The Plans for Improvement formulated in the Annual Assessment Report become an
integral part of the Annual Plan. Annual Plans for each department, school, college, division, or unit
are linked to institutional strategic initiatives from the Strategic Plan. During March and April,
Annual Plans are forwarded to the OIRPE and published on its website. The Annual Plan
incorporates Statements of Achievements that reflect the accomplishments of the previous Annual
Plan and a plan for the next year that includes goals/planning statements, implementation
strategies, expected completion dates, and additional costs.
Annual Budget requests are submitted to the Vice President for Financial Affairs each
spring. Annual Plans are attached to annual budgets to ensure that planning requests receive
appropriate financial support. Finally, in the fifth step of the Institutional Effectiveness Cycle, the
Annual Plans are implemented. Collection of assessment results is ongoing for most academic
programs, but analysis of assessment results generally commences in the fall and concludes in
February when the results are reported in the Annual Assessment Report. Table 3.1.1 is a
representative timetable of the Institutional Effectiveness Cycle.
Table 3.1.1
ANNUAL PLANNING CALENDAR
8/1– 9/30
The Strategic Planning Council (Executive Committee) reviews the Strategic
Plan’s mission, objectives, visions, strategic directions, goals, principles, factors
and assumptions--to ascertain accomplishments achieved and modifications
needed in the Strategic Plan.
9/15-10/15
The Compendium of Survey and Test Results is sent to administrators and
published on the OIRPE website. Individual College Survey results by
Departments/Majors are provided to Deans.
10/15
The Office of Institutional Research, Planning, and Effectiveness (OIRPE)
distributes instructions and materials for updating Six Points of Institutional
Effectiveness, completing the Annual Assessment Report, developing Annual
Plans, and preparing Annual Budgets.
10/15-2/1
Departments and Units collect, compile, and analyze assessment outcomes in
preparation for developing the Annual Assessment Report.
11/16-2/1
The Fact Book is sent to administrators and published on the OIRPE website to
aid with Annual Assessment, Annual Plans, and Annual Budgets.
1/1-1/31
The Six Points of Institutional Effectiveness are developed or updated (if
necessary) annually for administrators, academic programs, and non-academic
programs/functions/operations and are forwarded to the OIRPE.
2/1-2/28
Annual Assessment Reports (formerly Self-Studies) for administrators, academic
programs, and non-academic programs, units, functions, or operations are
completed and forwarded to the OIRPE.
3/1-3/31
The Six Points of Institutional Effectiveness and Annual Assessment Reports are
published on the OIRPE website.
3-5
5/1-5/31
Annual Plans are prepared for Departments or Units, Schools, Colleges, and
Divisions, approved by the various levels of supervisory administration, and
forwarded to the OIRPE. Annual Assessment results should be utilized in
developing Plans.
6/1-7/31
This time period is designated for Annual Budget and Plan hearings for deans
and vice presidents to be held with the Vice President of Financial Affairs, the
Vice Chancellor, and the Provost.
8/1-8/16
Approved Annual Plans and Annual Budgets are sent to the appropriate
administrators.
Further evidence of planning and evaluation of the educational activities of teaching,
research, and service may be found in the faculty assessment instruments and processes. Troy
State University recognizes the need for continuous improvement in teaching, research, and
service. To this end, the Comprehensive Faculty Evaluation Plan serves as a tool by which faculty
set professional goals and document progress toward goal attainment. Components of the
Comprehensive Faculty Evaluation plan include a Professional Development Plan, Year-End
Faculty Self-Evaluation, and a Supervisor’s Review. Teaching, research, and service are
assessment components of the Professional Development Plan as outlined in Appendix C of the
Faculty Handbook, 2001 (pp. 86-98). The Comprehensive Faculty Evaluation Plan is a vehicle by
which the department chair or supervisor may assist faculty members in continuous improvement
as part of the promotion and tenure process. Forms used for the Comprehensive Faculty
Evaluation Plan are provided in the Faculty Handbook, 2001 (pp. 86-98) and on-line
(http://spectrum.troyst.edu/~faceval/).
Troy State University emphasizes and encourages quality instruction that may be
enhanced by personal scholarship, research activities, and service to the University and
community. In a continuing effort to improve the quality of educational efforts, both students and
supervisors periodically and systematically review faculty members. Each instructor/course is
evaluated annually by students enrolled in the class by the Student Evaluation Survey of
Course/Instructor. Instructors with last names starting with letters A-M are reviewed each fall term.
Instructors with last names beginning N-Z are reviewed in the spring. Non-tenured faculty holding
tenure-track appointments are reviewed during each term of their first year, and once a year
thereafter. Full-time and adjunct faculty members may be evaluated more frequently as needed.
The evaluation tool for the Supervisor’s Review of Faculty Performance is included in the Faculty
Handbook, 2001 (pp. 90-98). Supervisors review the results of student surveys evaluating the
instructor/course. Supervisors in a confidential and constructive manner share with faculty
members the information gathered through the supervisors’ and students’ assessments.
Suggestions for improvements are then incorporated into the next academic year’s professional
growth and development plan of the faculty member. Through this cycle, the performance and
growth of the Troy State University faculty are continuously reviewed.
3-6
3.1.2
The institution must define its expected educational results and describe its
methods for analyzing the results.
Troy State University defines its expected educational results through a variety of ways.
The primary document that frames educational expectations is the University Mission Statement.
From the Mission Statement all program goals, objectives, and expected results are identified.
In the program or unit’s Six Points of Institutional Effectiveness, each academic program or
unit defines specific educational expectations (purpose, objectives, and expected results)
consistent with the University’s mission and educational objectives. In addition to defining the
specific educational expectations for each program, the Six Points of Institutional Effectiveness
also describe the methods for analyzing the level of achievement of stated results. Each program
or unit’s Six Points of Institutional Effectiveness include a list of assessment instruments used to
measure the extent to which expected results are achieved, the procedures for assessment, and
the person responsible for analyzing the assessment results. Table 3.1.2 lists the person of each
academic program who is responsible for analyzing educational results as stated in the
department’s Six Points of Institutional Effectiveness. All program and unit Six Points of Institutional
Effectiveness may be accessed at http://spectrum.troyst.edu/~oirpe.
The Annual Assessment Report for each program provides an analysis of results based
on the criteria established in the Six Points of Institutional Effectiveness. Because of the variety of
academic programs being assessed, various assessment instruments are used. The most common
tools of assessment are standardized tests, department exit exams, surveys, and internships or
preceptorships. Specialized fields may involve specific assessment methods ranging from a senior
conducting recital for the MS degree to the Air Education Training Command Inspector General
Evaluation in Aerospace Studies. Some professional fields have specialized assessment exams
linked to certification or licensure. The School of Nursing measures the effectiveness of the nursing
curriculum by student achievement on the National League for Nursing (NLN) achievement test
and the COMPASS-RN examination. The Sorrell College of Business assesses program
effectiveness in part by the Educational Testing Service Major Field Test (MFT) in Business.
Additionally, departmental exit exams are required for accounting, finance, information systems,
management, and marketing. Furthermore, the American Institute of Certified Public Accountants
(AICPA) Achievement Test is required for graduating accounting majors. Athletic Training
assesses the achievement of its graduates by administration of an intensive care analysis included
in the student exit exam, and the number of students passing the National Athletic Training
Association (NATA) certification examination. Table 3.1.2 identifies the most commonly used
assessment tools of academic programs.
3-7
√
√
√
√
√
√
√
√
√
√
√
(2)
√
√
√
√
√
(3)
√
√
√
(1)(3)
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
(1) End-of-Course Evaluations (Course/Instructor Evaluation)
(2) Exit Interviews
(3) Case Analysis
(4) Portfolio / Field Experiences
(4)
√
√
√
√
√
√
Other
Alumni / Employer
Surveys
√
√
√
Internship /
Preceptorship
Student Surveys
ACCOUNTING
Florida-Western /Fred Westfall
Phenix City/Dr. Cherie Fretwell
Troy/Dr. Thomas Ratcliffe
ART PROGRAM/EDU.
Troy/Jerry Johnson
ATHLETIC TRAINING
Troy/John Anderson
BIOLOGY
Troy/Dr. Glen Cohen
BUSINESS - GRADUATE
Phenix City/Dr. Cherie Fretwell
Troy/Dr. Earl Ingram (DL courses also)
BUSINESS-UNDERGRADUATE
Atlantic Region/Dr. Christine Burge
Florida-Western/Dr. Fred Westfall
Phenix City/Dr. Cherie Fretwell
Troy/Dr. Tish Matuszek
BUSINESS ADMIN. - GRAD
Atlantic/Dr. Christine Burge
Phenix City/Dr. Cherie Fretwell
Troy/Dr. Earl Ingram
CHEMISTRY
Troy/Dr. Chris King
CLASSICAL & MODERN LANG.
Troy/Dr. Peter Howard
COMMUNICATION ARTS
Troy/Dr. Jim Vickrey
COMPUTER SCIENCE
Troy/Diane Porter
Florida-Western/Duane Barts
Departmental Exit
Exams /
Comprehensive Exams
ACADEMIC AREA AND PERSON
RESPONSIBLE FOR ANALYSIS
Standardized Tests /
National Licensing /
Certifying Exams
Table 3.1.2
MOST COMMON ASSESSMENT TOOLS AND PERSONS RESPONSIBLE FOR
ANALYZING ASSESSMENT RESULTS
(As Reported in Six Points of Institutional Effectiveness and
Annual Assessment Reports, Fall 2001)
(3)
√
(2)
√
√
√
√
√
√
√
√
√
√
(4)
3-8
Other
Internship / Preceptorship
Student Surveys
√
√
√
√
(4)
√
√
√
√
√
√
(3)
(3)
(3)
√
√
√
(5)
(2) (5)
√
√
√
√
√
√
√
√
√
√
√
(1) (9)
(1)
√
√
√
(1) (9)
√
√
√
√
√
√
√
√
√
(1)
(1)
(1)
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
Alumni / Employer
Surveys
COUNSELING / PSYCH. -GRAD
Troy/Dr. Dianne Gossett
Phenix City/Dr. Kathryn Ness
Florida-Western/Dr. Cecil Carlton
CRIMINAL JUSTICE - GRAD
Troy/Dr. Ed Stevens
Florida-Western/Dr. Terry Anderson
Southeast/Mike Estes
CRIMINAL JUSTICE-UNDERGR.
Florida-Western/Tracy Newvine
Troy/Dr. Ed Stevens
Southeast/Mike Estes
EDUCATION UNDERGRADUATE
Phenix City/Dr. Mary Jane Whisnant
Troy/Dr. Donna Jacobs
EDUCATION - GRADUATE
Phenix City/Dr. David Shoemaker
Southeast/Dr. Gene Fober, Dr. James
Franken, & Rebecca Miller
Troy/Dr. Donna Jacobs & Dr. Dianne
Gossett
Pacific/Dr. Bill Shockley
Florida-Western/Dr. Cecil Carlton
Atlantic Region/Dr. Chris Burge
EDUCATION SPECIALIST
Phenix City/Dr. David Shoemaker
ENGLISH
Troy/Dr. Noel Kaylor
ENGLISH LANGUAGE ARTS
Troy/Dr. Noel Kaylor
ENVIRON. ANALYSIS-GRAD.
Troy/Dr. Glenn Cohen
ENVIRON. SCIENCE-UNDERGR
Troy/Dr. Glenn Cohen
Departmental Exit Exams
/Comprehensive Exams
ACADEMIC AREA AND PERSON
RESPONSIBLE FOR ANALYSIS
Standardized Tests /
National Licensing /
Certifying Exams
Table 3.1.2 (Continued)
√
√
√
√
√
√
√
√
√
(1) End-of-Course Evaluations (Course/Instructor Evaluation)
(2) Exit Interviews
(3) Case Analysis
(4) Portfolio / Field Experiences
(5) Senior Seminar
(9) Alabama State Dept. of Education Teacher Preparation Program Performance Profile
(1) (9)
(1) (9)
(4)
√
(2)
3-9
FINANCE
Troy/Dr. Tish Matuszek
GENERAL EDUCATION A.S.
Florida-Western/Dr. Thomas Dunn
HISTORY
Troy/Dr. Allen Dennis
HUMAN RESOURCE MGMT GR.
Atlantic/Dr. Christine Burge
Florida-Western/Dr. Fred Westfall
Southeast/Dr. Aaron Lucas & Dr. Larry Carlton
Troy/Dr. Earl Ingram
HUMAN SERVICES
Troy/Charles R. Whitson
INFORMATION SYSTEMS
Troy/Dr. Tish Matuszek
Phenix City/Dr. Cherie Fretwell
Southeast/Tom Mowery
JOURNALISM Troy/Dr. Steve Padgett
MANAGEMENT - GRADUATE
Atlantic/Dr. Christine Burge
Florida-Western/Dr. Fred Westfall
Pacific/Dr. Bill Shockley
Southeast/Dr. Ron Robinson, Dr. Larry
Carlton, and Holly Green
Troy/Dr. Earl Ingram
MANAGEMENT-UNDERGRADUATE
Florida-Western/Dr. Fred Westfall
Phenix City/Dr. Cherie Fretwell
Southeast (Ft. Benning)/Karen Smola
Troy/Dr. Charles Williams
MARKETING/ECONOMICS
Troy/Dr. Tish Matuszek
MATHEMATICS / PHYSICS
Troy/Diane Porter
MUSIC CONDUCTING-GRADUATE
Troy/Dr. Bill Denison
MUSIC EDUCATION Troy/Dr. Bill Denison
√
√
√
√
(6)
√
√
√
(1)
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
(1) End-of-Course Evaluations (Course/Instructor Evaluation)
(3) Case Analysis
(4) Portfolio/Field Experiences
(6) Academic Profile
(9) Alabama SDE Teacher Prep. Prog. Performance Profile
Other
√
√
√
√
√
Internship /
Preceptorship
√
Alumni / Employer
Surveys
√
Student Surveys
Departmental Exit
Exams
/Comprehensive
Exams
ACADEMIC AREA AND PERSON
RESPONSIBLE FOR ANALYSIS
Standardized Tests /
National Licensing /
Certifying Exams
Table 3.1.2 (Continued)
√
√
√
(4)
√
√
√
√
(4)
(3)
(3)
(3)
√
(1)
(1)
3-10
NURSING
Montgomery-ASN/Dr. Donna Bedsole
Troy-BSN/Dr. Brenda Riley
Montgomery/Phenix City-MSN/Dr.
Brenda Riley
POL. SC. MSIR - GRADUATE
Troy/Dr. Jim Rhinehart
Pacific/Dr. Peter Hefron
POLITICAL SC. – UNDERGRADUATE
Troy/Dr. Jim Rhinehart
PSYCHOLOGY – UNDERGRADUATE
Troy/Dr. Dianne Gossett
PSYCH. COMMUNITY-GRADUATE
Florida/Tracy Newvine
Phenix City/Dr. Kathryn Ness
PUBLIC ADMIN. - GRADUATE
Atlantic/Dr. Christine Burge
Florida-Western/Terry Anderson
Pacific/Dr. Bill Shockley
Southeast/Dr. Thomas Seebode
Troy/Dr. Jim Rhinehart
REHABILITATION
Troy/Charles R. Whitson
RESOURCE MANAGEMENT
Atlantic/Dr. Christine Burge
Florida/Dr. Fred Westfall
Phenix City/Dr. Cherie Fretwell
Troy/Dr. Tish Matuszek
SOCIAL SCIENCES
Florida-Western/Tracy Newvine
Troy/Dr. Ed Stevens
SOCIAL WORK
Troy/Benito Arellano
√
√
√
√
√
√
√
√
√
√
√
√
√
√
Other
Internship /
Preceptorship
Alumni / Employer
Surveys
Student Surveys
Departmental Exit
Exams
/Comprehensive
Exams
ACADEMIC AREA AND PERSON
RESPONSIBLE FOR ANALYSIS
Standardized Tests /
National Licensing /
Certifying Exams
Table 3.1.2 (Continued)
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
(1) End-of-Course Evaluation (Instructor/Course Evaluation)
(2) Academic Profile
(4) Portfolio / Field Experience
(5) Senior Seminar
(7) Quality assurance visits by state and government education agencies or
Military service official's local surveys (if applicable)
(9) Alabama Dept. of Education Teacher Preparation Program Performance Profile
(1) (2)
√
√
√
√
√
√
√
(7)
√
(4)
√
√
√
√
(3)
(5)
3-11
SPEECH & THEATRE
Troy/Dr. Jim Vickrey
SPORT & FITNESS MGMT - GRADUATE
Troy/Dr. Lance Tatum
SPORT & FITNESS MGMT-UNGRADUATE
Troy/Dr. Lance Tatum
√
√
√
Other
Internship / Preceptorship
Alumni / Employer
Surveys
Student Surveys
Departmental
Comprehensive / Exit
Exams
ACADEMIC AREA AND PERSON
RESPONSIBLE FOR ANALYSIS
Standardized Tests /
National Licensing /
Certifying Exams
Table 3.1.2 (Continued)
(8)
√
√
√
√
√
√
(8) Performance Critique
3.1.3
The institution must:
1. Establish a clearly defined purpose appropriate to collegiate education.
2. Formulate goals consistent with the institution’s purpose.
3. Develop and implement procedures to evaluate the extent to which these
educational goals are being achieved.
4. Use the results of these evaluations to improve educational programs,
services, and operations.
Troy State University has a clearly defined purpose that is reflected in the Mission
Statement and Institutional Objectives. The Mission Statement and the nine Objectives are
appropriate to an undergraduate and graduate degree-granting institution. Each academic
department, unit, or service at Troy State University formulates goals consistent with the University
Mission. Goals with alignment to the University Mission are validated in the Six Points of
Institutional Effectiveness and may be verified by reviewing the Six Points of Institutional
Effectiveness for each department, program or unit (located at http://spectrum.troyst.edu/~oirpe/).
Goals and objectives of each program, unit, and service are to be linked not only to the
University Mission, but also to appropriate assessment criteria and expected assessment results.
The Institutional Effectiveness Cycle model allows the academic and non-academic units of the
institution to evaluate the extent to which goals are being achieved by comparing assessment
results with expected assessment outcomes. The continuous assessment IEC model affords the
Institution the ability to develop and implement plans to improve educational programs, services,
and operations.
How Assessment Information has been used to Improve Programs and Services
Numerous examples exist of how assessment information has been used to improve
programs and services across the Institution. A document prepared by the Office of Institutional
Research, Planning and Effectiveness in October 2002 compiles the Statements of Achievements
for the 2001-2002 AY and cites improvements resulting from the Institutional Effectiveness Cycle.
3-12
Each program and unit provides specific information in its Statement of Achievement Report which
evaluates the plans of the previous year, and a summary of these achievements is available in the
Troy State University Statement of Achievements for 2001-2002—Improvements Resulting from
the Institutional Effectiveness Cycle and Annual Planning Statements for 2002-2003, October
2002.. Examples extracted from the annual reports are provided below, and examples of
assessment information are also given.
To assess programs, some departments within the College of Arts and Sciences utilize the
Major Field Tests (MFT) from the Educational Testing Service (ETS). In the History Department,
program effectiveness is in part assessed by student performance on the MFT in history. In 2001,
students at Troy State University scored in the 42.9 percentile on the MFT in history, below the
program goal of the 50th percentile. These results prompted the History Department to restructure
the course requirements for a history major. The new structure affords a better sampling of the
courses offered by the department. Scores from the MFT in future years will determine if the
restructuring has improved the MFT History scores and thus the effectiveness of the History
program. The Major Field Test for biology was used by the Department of Biological Sciences to
build plans for improving their programs. Based on the MFT results, the Biology Department has
implemented a different mix of courses in conjunction with the Mathematics and Chemistry
Departments, and over time there is an expectation that these MFT scores will increase.
The Sorrell College of Business strives to fulfill the institutional objective of “preparing
students with demonstrated competences in their chosen field of study.” The MFT in Business is
used to measure the degree to which this objective is achieved. In 2000, Troy State Business
majors posted a mean score of 139.7. This score was below the national mean of 152.7. Plans to
improve the performance of business majors on the MFT included the development of a procedure
to ensure that students take the exam before being cleared to graduate. The successful completion
of the exam is not a requirement for graduation. As a result, greater emphasis has been placed
upon the business core. After the implementation of these changes during the AY 2000 - 2001, the
Troy State mean score rose significantly from 139.7 in 2000, to 153.6 in 2001, which is consistent
with the national mean of 154.6 in 2001.
The mission of the College of Communication and Fine Arts is “to provide excellence in
instruction in selected communication and fine art disciplines, and provide practical experiences.”
Pass rates (70% or above) on the Basic Music Skills Examination used for assessment in the
Music Education program fell from 76% between 1992-1998, to 50% in 1999 and 55% in 2000.
This assessment prompted the School of Music Education to appoint a faculty theory committee
that recommended changes in theory course syllabi and methods of instruction. The Assessment
Report of 2001-2002 stated that 73% of students passed the Basic Music Skills Exam, a notable
improvement from the previous two years. Also within the College of Communication and Fine Arts,
assessment indicated weaknesses in the research techniques of Broadcast and Print Journalism
students. To strengthen the program, the statistical analysis program Statistical Packages for the
Social Sciences (SPSS) was installed in the Journalism area, and sections on research techniques
and statistical analysis were added to the program.
The College of Education undergraduate K-6 Collaborative Teacher program strives to
“prepare innovative educators grounded in the best practices that equip entry level teachers for
today’s inclusive classrooms.” A weakness noted in the 2000-2001 Assessment Report was that
3-13
only 58% of graduates and 86% of school systems surveyed indicated satisfaction with TSU
students’ ability to work well with exceptional/special-needs students in inclusive settings. In order
to improve this program, specific opportunities for working with special-needs students in inclusive
settings were implemented. The basic survey of exceptional children courses was enhanced to
focus on collaboration in inclusive settings, and Special Education majors were given increased
exposure to teaching core subjects. As a result of these implemented plans, 2001-2002
assessment results reflected an increased level of satisfaction in students’ ability to work with
special-needs students in inclusive settings. Graduating seniors’ satisfaction rating increased from
58% to 76%, and school system satisfaction improved from 86% to 91%.
Within the College of Health and Human Services, the Athletic Training Department strives
to prepare students in the field of athletic training. The achievement of this goal is assessed by
pass rates on the National Athletic Training Association (NATA) National Board Examination. The
development of an informal class designed to prepare students specifically in the format of the
NATA National Board Examination has helped to increase the first-time pass rate from 60% in
1999 to 85% in 2000 and 2001. In 2000, one facet of the NATA National Board Examinations in
which Athletic Training students could still improve was skill in written simulations. Research
indicates that other universities have seen improvements in the NATA National Examination after
incorporating a computer program developed at the University of Southern Mississippi into their
curriculum. Based upon this identified weakness and research, the computer program was
incorporated into the Athletic Training curriculum during AY 2000-2001. Assessment results in
2002 indicated a written simulation score of 85%, exceeding the criterion goal of 80%.
3.1.4
The institution must develop guidelines and procedures to evaluate
educational effectiveness, including the quality of student learning and of
research and service.
The Six Points of Institutional Effectiveness provide the guidelines and procedures for
evaluating educational effectiveness relevant to the department, unit, or service, and assists in
assuring educational effectiveness, quality, customer satisfaction and consistency. The guidelines
and procedures to evaluate educational effectiveness are presented in the Six Points of
Institutional Effectiveness for each educational program, unit, or service and can be found on the
OIRPE website or in hard copy format in the Office. Each academic department, unit, or service
has documented procedures for assessing the quality of student learning, and when appropriate,
the quality of academic research and service. Departments assess the quality of student learning
using many different tools (Refer to Table 3.1.2). Basic tools for assessing student learning are
department exit examinations, standardized tests, qualitative assessments (i.e. student projects
and reports, internships, completing service learning assignments, submitted portfolios, public
recitals, focus groups), and numerous surveys (i.e. graduating student, undergraduate student,
graduate student, alumni and employer surveys). Alumni surveys are sent annually to graduates of
the previous year, and to those graduating five years earlier. Lists of alumni are obtained from the
Alumni Affairs Office. The Alumni Survey measures whether or not the students have acquired the
necessary knowledge to ensure success in the work environment. University-wide results of alumni
satisfaction with the quality of their academic preparation are published annually in the
Compendium of Survey and Test Results. Additionally, all graduating students and samples of
undergraduate and graduate students are surveyed annually. Results of these surveys are also
published in the Compendium of Survey and Test Results. Individual reports by academic major
3-14
are provided to appropriate departments and colleges when the major has a minimum of five
survey respondents. Departments within the Colleges of Health and Human Services, Education,
and Business utilize either state or national standardized examinations associated with certification
or licensure as a method of evaluating educational effectiveness. For example, students take the
National League of Nursing (NLN) achievement tests four times during the Associate Degree
Program in Nursing. These tests allow the faculty to document the effectiveness of student learning
compared to national norms.
In addition to the internally developed guidelines and procedures to evaluate educational
effectiveness, the University has also pursued discipline-specific accreditation through agencies
that have externally developed guidelines and procedures to evaluate educational effectiveness.
These external reviewing agents allow the opportunity to benchmark Troy State University against
nationally adopted standards, best practices, and discipline specific criteria that drive performance
excellence. Through a learning-centered educational environment, designed to pursue
performance excellence coupling internal and external standards, Troy State University is better
able to maximize effectiveness of student learning. Professional accrediting agencies for programs
at Troy State University are listed in the Undergraduate Bulletin 2001-2003 (p. ii), Graduate Bulletin
2001-2003 (p. ii), and the Fact Book, 2002 (p. 100). These include the following:








National Council for the Accreditation of Teacher Education (NCATE)
National Association of Schools of Music (NASM)
National League for Nursing Accrediting Commission (NLNAC)
Council on Social Work Education (CSWE)
Association of Collegiate Business Schools and Programs (ACBSP)
International Assembly for Collegiate Business Education (IACBE)
Commission on Accreditation of Allied Health Education Programs (CAAHEP)
Council for Accreditation of Counseling and Related Educational Programs (CACREP)
Students, faculty, and supervisors assess the quality of course content, information
delivery, and course policies and procedures. Students anonymously complete a Course and
Faculty Member Assessment Survey on the last day of instruction. Evaluations are forwarded to
the Provost for compilation, review, and statistical analysis. Course and Faculty Assessment
Survey summaries are forwarded to the dean, who in turn reviews them with the department
chairs. Department chairs discuss any concerns with the individual faculty member. The guidelines
and procedures for faculty evaluation can be found in the 2001 Faculty Handbook (pp. 90-98).
Teaching, research, and service are the three areas of evaluation used to determine faculty
effectiveness. Through these guidelines and procedures, student learning, research, and service
are assessed.
Student learning in the general studies curriculum is assessed by the Academic Profile
(AP), a general education competency examination administered to juniors and seniors with 60 or
more semester credit hours. Included in the annual Compendium of Surveys and Test Results, the
AP results allow the University to compare the general studies competencies of its junior and
senior level students with national norms. Figure 3.1.4.1 illustrates the comparison of Academic
Profile scores between peer institutions and Troy State University.
3-15
Academic Profile Mean Scores
Comparison of Peer Group with Troy State University
ACADEMIC PROFILE - MEAN SCALE SCORES
Comparison of Peer Group with TSU 1998-1999,1999-2000
and 2000-2001 Groups
450
400
350
300
250
200
150
100
50
0
Total
Humanities
Social
Sciences
Natural
Sciences
Reading
Writing
Critical
Thinking
Mathematics
Group Mean
449
116
115
118
120
117
113
115
TSU MEAN 1998-1999
441
114
114
116
118
115
111
114
TSU MEAN 1999-2000
441
114
113
115
117
115
110
113
TSU Mean 2000-2001
444
114
114
116
118
115
111
114
Figure 3.1.4.1
o
o
o
o
Group Mean (Peer Group) = Means of other
Educational Testing Service (ETS)
TSU Mean 1998-1999 = Mean scale scores for
Examination in 1998-1999
TSU Mean 1999-2000 = Mean scale scores for
examination in 1999-2000
TSU Mean 2000-2001 = Mean scale scores for
examination in 2000-2001
schools comparable to TSU as determined by
all TSU students who took the Academic Profile
all TSU students who took the Academic Profile
all TSU students who took the Academic Profile
Prior to the fall of 1998, student participation for the Academic Profile Examination was
voluntary and performance efforts were minimal. In the fall of 1998, the AP exam became a
requirement for all junior-level students prior to graduation. Although participation was increased,
scores remained at or below national averages. Currently, the Dean of Arts and Science serves as
interim Director of General Studies, following the resignation of the previous Director of General
Studies in the summer of 2002. The Dean of Arts and Sciences is mentoring a member of the
General Studies faculty to take over as Director of General Studies who will direct, monitor, and
evaluate the General Studies program.
In an attempt to determine if the University’s General Studies Program is effectively
improving students’ abilities in the areas tested by the AP exam, a statistical comparison using
analysis of covariance with ACT composite scores as the covariate is employed to compare the AP
3-16
means for a sample of freshmen students with a sample of junior and senior students who are
required to take the AP. Freshmen are recruited from TSU 1101, University Orientation. For
students in 2001-2001, there was a significant difference at the 0.01 Level (F=4.590) between the
mean total score of freshmen students and the mean total score of senior students who took the
exam. Freshman participation is voluntary and sample sizes have been low (i.e. Fall 2001, n=25).
Strict guidelines and procedures are in place for reviewing research proposals. The
University’s Research Review Board (RRB) must approve all research proposals. The purpose of
the RRB is to protect human subjects in research (See Standing Committees of Troy State
University, 2000-2001, pp. 25, 50). The quality of student research can be assessed by the number
of students who have received funding for research projects and/or who have made state or
regional presentations before professional organizations. Several undergraduate and graduate
students in the Department of Biological and Environmental Sciences have conducted research
funded by grants. During AY 2000-2001, departmental faculty members in biology were awarded
more than $550,000 in external funding, including large grants from the National Science
Foundation and the Alabama Department of Public Health. Several smaller grants, which were
funded through state agencies/organizations, have provided direct support for undergraduate
research. These state agencies/organizations include the Alabama Wildflower Society, the
Audubon Society, and the Department of Conservation. Several undergraduate and graduate
students in the Department of Kinesiology and Health Promotion have presented research results
at State/Regional conferences. Presentations have been made at the State Association for Health,
Physical Education, Recreation and Dance Annual Fall Conference and at the Florida State
University Sport and Fitness Management Symposium.
Quality teaching, research, and service are also reinforced by the tenure and promotion
policy and by the criteria for graduate faculty membership at Troy State University. The tenure and
promotion policies are described in detail in the Faculty Handbook, 2001 (pp.45-49). Graduate
faculty membership is required of individuals teaching graduate courses and of those who assist
graduate students as advisors or who serve as members of graduate committees. Membership
requirements are based upon educational level, experience, teaching excellence, professional
involvement, and scholarship. Criteria for graduate faculty membership at the Full and Associate
levels are explained in detail in the Faculty Handbook, 2001 (pp. 35-36) and represent yet another
set of guidelines and procedures to evaluate and ensure educational effectiveness, including the
quality of student learning and of research and service.
3.1.5
The institution’s evaluation procedures must encompass educational goals
at all academic levels and research and service functions of the institution.
The evaluation of academic programs should involve gathering and
analyzing both quantitative and qualitative data that demonstrate student
achievement.
Each program or unit has stated educational goals (i.e. purpose, objectives, and
expected results) that are listed in the Six Points of Institutional Effectiveness for that particular
program or unit. Additionally, as part of the Annual Plan, previous planning goals are evaluated
annually in the Statement of Achievements, and new planning goals are developed for the next
year. The evaluation procedures of Troy State University involve students, faculty, administrators,
community representatives, and external accrediting agencies.
3-17
The evaluation of academic programs involves the gathering and analysis of assessment
results. Troy State University students at both the graduate and undergraduate levels are involved
in the assessment of educational programs. Student learning is evaluated using qualitative and
quantitative techniques of evaluating student projects, research papers, presentations, completion
of service learning assignments, portfolios, public recitals, and focus groups. Assessment is
accomplished through surveys that allow students’ input into the educational programs and
services at Troy State University. Annually, graduating students, undergraduate students, and
graduate students are surveyed by the Office of Institutional Research, Planning, and Effectiveness
(OIRPE), and the survey results are published in the Compendium of Survey and Test Results as
well as in departmental and college reports sorted by major. Quantitative assessment is conducted
through the use of exit exams, comprehensive exams, standardized tests, surveys, and
questionnaires. Tools used to assess student achievements are listed specifically for each program
in the Six Points of Institutional Effectiveness, and assessment results are reported and analyzed
in the Annual Assessment Report for each program annually.
The academic department serves as the fundamental grouping of faculty whose common
professional interests and expertise provide continuity for the instructional programs. Members of a
department work collaboratively with the chair and dean in decisions concerning instructional
programs, departmental operations, and personnel actions. The basic responsibility for maintaining
quality in the program, curricula, and operations of the department rests with the departmental
faculty. Program curricula and individual courses are improved as faculty review survey results
from students, enrollment and employment trends, and technological advancements. Committee
appointments to University-wide committees give the faculty an avenue for qualitative input into the
educational programs, research and service functions of the University (Refer to Standing
Committees of Troy State University, 2000-2001). Quantitatively, faculty set professional goals and
assess their achievement of those goals through the Professional Development Plan and Year-End
Self Evaluation (See Faculty Handbook, 2001, pp. 87-89). Administrative review of faculty
teaching, research and service is accomplished by both quantitative and qualitative supervisor
review (See Faculty Handbook, 2001, pp. 90-98).
Through participation on boards and committees, the community has a voice in the
academic programs, research, and services at Troy State University. For example, community
members, alumni, or contributors are members of the Nursing Advisory Board, Research Review
Board, Homecoming and Alumni Affairs Committee, Athletic Advisory Committee, and others (See
Standing Committees of Troy State University, 2000-2001). Alumni and employers contribute to
educational programming and student learning by information gathered through the Alumni Survey
and Employer Survey. Results of the Alumni Survey and Employer Survey are reported annually in
the Compendium of Surveys and Test Results. Information from the Alumni Survey and Employer
Survey is reviewed and used to improve existing programs or to develop curriculum to anticipate
the needs of the community.
As noted in Section 3.1.4, outside review and assessment of academic programs are
provided by discipline-specific accreditation agencies. Accreditation from those agencies reflects a
defined standard of educational integrity in the programs. Programs, which are accredited by
discipline-specific accreditation agencies, are listed in the 2001-2003 Undergraduate Bulletin,
3-18
2001-2003 Graduate Bulletin, and Fact Book, 2001, as well as in Section 3.1.4 in this Self-Study
Report.
3.1.6
The institution must evaluate its success with respect to student
achievement in relation to purpose, including, as appropriate, consideration
of course completion, state licensing examinations, and job placement rates.
The Institutional Effectiveness Cycle reflects one method of evaluating the success of the
University in attaining its institutional purpose and objectives with respect to student achievement.
The institutional objective of Troy State University “to prepare students to demonstrate
competence in their chosen field(s) at appropriate degree levels” is assessed by retention and
graduation rates, success on national and/or state certification exams, employment and/or
graduate school acceptance in chosen field, raises and promotions of graduates, and students’
opinions as to the effectiveness of their preparation.
Monitoring retention and graduation rates over time can help assess the effectiveness of
both academic and student life programs. Troy State annually publishes the “Non-Returning FirstTime Freshman Student Report” in the Fact Book. The percentage of entering freshmen who return
for a second year has twice exceeded the national average for all institutions since 1993. In
addition, Troy State University has retained more first-time students for a second year than other
public institutions in four out of the last five years. Figure 3.1.6.1 illustrates the percentage of Troy
State University freshman returning for their second year as compared to national averages since
1993 (Source: Fact Book and www.act.org).
Percentage (By Year) of Returning First –Time Students
80
75
70
65
60
55
1993
1995
1997
1999
2001
TSU
National Avg. (All Public)
National Avg. (All Institutions)
Figure 3.1.6.1
In efforts to address low retention, in 1997 a full-time staff member was made responsible for
studying retention patterns and developing programs to address this issue. TSU 1101 was adopted
3-19
as a required General Studies course for all students, with the exception of students who take HON
1101 and transfer students who have completed a college credit orientation program elsewhere.
Efforts to improve academic advising have also been enacted. An advisement handbook was
created and distributed. Prior to the Fall Semester 2002, Dr. John Gardner, was a guest lecturer for
a day-long seminar aimed specifically at increasing freshman retention. The seminar, “Making the
Most of the Undergraduate Experience: A Day with Dr. John Gardner” presented methods by which
faculty can improve their advisement efforts and skills.
Residence Life programming was formalized with presentations on alcohol abuse, time
management, money management, safety, and other such issues. In the fall of 2002, Residence
Life developed Living Learning Communities within Alumni and Gardner Halls. Specialized
programming and intrusive academic advisement were arranged in order to support first-year
student success. Additionally, efforts focused on building a sense of community through limited
self-governance, creative programming, and introduction of faculty mentors. Troy State University
is continually assessing attrition data from non-returning students, looking for demographic trends
and clues to why students do not return, and developing and implementing plans to address
retention needs and issues.
Troy State University annually reports the graduation rates of six-year cohort groups. Troy
State’s cohort graduation rate is summarized in Figure 3.1.6.2 (Source: Fact Book, March 2002).
Graduation Rates
60%
50%
40%
30%
20%
10%
0%
1991-97
1992-98
1993-99
1994-00
1995-01
Figure 3.1.6.2
Troy State students’ performance on national and/or State certification exams has
demonstrated competence in their chosen field. For example, the National Council for Licensing for
State Boards exam (NCLEX) is taken by nursing graduates in the state in which they plan to
practice nursing. Passing of the NCLEX qualifies the graduate to practice as a registered nurse.
Troy State University nursing graduates had a 93% pass-rate on the NCLEX licensing exam in AY
2000-2001. This pass-rate exceeded both the regional average of 83% and the national average of
85% that same year. The Athletic Training department was able to demonstrate the effectiveness
of its program by producing a 60% first-time pass-rate on the National Athletic Training Association
3-20
(NATA) Certification exam in both 1998 and 1999. In 2000, the first-time pass-rate was 85%.
These first-time pass-rates are considerably greater than the national 55% first-time pass-rate in
2000-2001.
The employment rate of Troy State University graduates is another method by which the
achievement of Institutional objectives may be measured. For example, the graduate program in
Environmental Analysis and Management set a goal “to have at least 70% of graduates finding
jobs in the environmental science field.” Assessment results of 2000-2001 reported that 30 of the
32 graduates (94%) were employed in environmental science occupations. In the Sorrell School of
Business, a stated objective of the Accounting Program is to have at least 50% of graduates
employed or in a graduate school program related to their degree one year beyond graduation.
One-year-out Alumni survey results from 2001 indicated 100% of accounting majors were
employed in accounting or a related field, far exceeding the 50% employment goal. Additionally,
the 2001 Five-year-out Alumni Survey indicated that 100% of accounting graduates had received
raises or promotions during their first five years of employment.
These examples illustrate how TSU evaluates its success with respect to student
achievement in relation to purpose, including retention, state licensing examinations, and job
placement.
3.2
PLANNING AND EVALUATION: ADMINISTRATIVE AND EDUCATIONAL SUPPORT
SERVICES
3.2.1
The institution must demonstrate planning and evaluation in its
administrative and educational support services.
The Institutional Effectiveness Cycle is the means by which administrative and educational
support services demonstrate planning and evaluation. IEC procedures are set forth in the Manual
for Annual Planning, October 2000, and provide the means for assessing each administrative and
educational support service of the University. The Manual for Annual Planning, October 2000
includes procedures for the IEC for non-academic and administrative areas as well as for academic
programs. Each unit submits its Six Points of Institutional Effectiveness, an Annual Assessment
Report, an Annual Plan including Statement of Achievements, and an Annual Budget. The Six
Points of Institutional Effectiveness, Annual Assessment Reports, and Annual Plans are published
on the website of the Office of Institutional Research, Planning, and Effectiveness. Each unit has
the responsibility for implementing its Annual Plan. Results from the implementation of the Annual
Plan are documented in the Statement of Achievements. Table 3.2.1 lists the persons responsible
for planning and evaluating the administrative and educational support units along with assessment
tools used at Troy State University.
3-21
Other
Student Assessment
Peer Institution Comparison
Enrollment
Focus Groups
Federal or State Standards
Annual Financial Audit / Financial
Assessments
Student Surveys
Employer / Alumni Survey
(*Position serves UC &
Phenix City)
Reports
ADMIN. & EDUCATION
SUPPORT UNITS/
CONTACT PERSONS
Internal Monitoring /Monitoring Use
& Involvement
ASSESSMENT TOOLS
___________________
Administrators Survey / In-House
Surveys / Questionnaires
Table 3.2.1
Administrative and Educational Support Units
Assessment Tools with Contact Persons
ADMINISTRATIVE UNITS
*Chancellor/Dr. Jack
Hawkins
*Vice Chancellor/ Dr. Doug
Patterson
*V. P. for Financial Affairs/
Clint Carlson
*V. P.Office for Institutional
Research, Planning & Effectiveness /Dr. Angela Roling
*V. P. for Institutional
Advancement/Dr. Jean
Laliberte
Vice President Student
Affairs/ John Schmidt
*V. P. for Technology &
Assistant Vice Chancellor for
Technology/Dr. Bill Flinn
Director of Athletics/ Johnny
Williams
*Provost/Fred Davis
√
(1)
√
√
√
√
√
(2)
√
(2)
√
(2)
√
√
√
√
(2)
√
√
√
√
√
√
(2)
√
√
√
√
√
*Dean of the College of Arts
& Sciences/Dr. Robert Pullen
*Dean of the College of
Communications & Fine
Arts/Dr. Hal Fulmer
*Dean of the College of
Education/Dr. Donna Jacobs
*Dean of the College of
Health and Human
Services/Dr. Don Jeffrey
*Dean of Distance
Learning/Dr. Sallie Johnson
*Dean of Graduate Studies &
Research/Dr. Dianne Barron
(2)
(3)
(2)
√
√
(4)
√
√
(4)
√
√
√
√
√
√
√
√
(4)
√
(4)
√
√
√
√
√
√
√
√
√
√
√
√
*Dean of the Sorrell College
of Business/Dr. Thomas
√
Ratcliffe
Phenix City Vice
President/Dr. Curtis Pitts
√
Phenix City Academic
Dean/Dr. Jerry Williams
Vice President University
√
College/Dr. Susan Aldridge
Academic Dean University
√
College (UC)/Dr. Fred Viohl
UC Atlantic Region
Director/Dr. Christine Burge
√
UC Florida-Western Region
Director/Dr. Fred Meine
√
UC Pacific Region
Director/Dr. Bill Shockley
UC Southeast Region
√
Director/Dr. Aaron Lucas
EDUCATIONAL SUPPORT UNITS
*Accounting Services/Bryan
Helms
√
*Accounts Payable/U. Bryant
Auxiliary Services/T. Burnett
Campus Dining
Services/Chuck Faulkinberry
Center for Bus. & Economic
Services/Janet Kervin
Center for Continuing
Education/Terry Qualls &
Judy Williams
Center for Environ. Research
& Service/Dr. Glenn Cohen
Other
(4)
√
(2)
√
√
(2)
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
Student Assessment
√
√
*Accounts Receivable/Lou
Ann Johnson
Adams Center/ Student
Services/Shea Phillips
*Alumni Affairs/Faith Ward
Peer Institution Comparison
Enrollment
Focus Groups
Federal or State Standards
Annual Financial Audit / Financial
Assessments
Student Surveys
Employer / Alumni Survey
(*Position serves UC &
Phenix City)
Reports
ADMIN. & EDUCATION
SUPPORT UNITS/
CONTACT PERSONS
Internal Monitoring / Monitoring
Use & Involvement
ASSESSMENT TOOLS
___________________
Administrators Survey / In-House
Surveys / Questionnaires
3-22
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
Center for International
Programs/Dr. Curtis Porter
Computer Works/Lottie
Summerville
Department of
Athletics/Johnny Williams
Enrollment Management/
Buddy Starling
Housing & Residence Life/
Conference Services/Herbert
Reeves
*Human Resources/Lee
Vardaman
Instructional Support
Services Administrative
Office/Eleanor Lee
*Inventory Control/Ursula
Bryant
*Library/Dr. Henry Stewart
*Infor.Tech. Services/Vickie
Miles & Info. Tech. Man./
Greg Price (Formerly MIS)
Natural Science Center/
Robert Sheppard
√
√
√
Other
Student Assessment
Peer Institution Comparison
Enrollment
Focus Groups
√
√
v
√
√
(5)
(3)(6)
(7)
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
(2)
√
*Purchasing/Ursula Bryant
Radio & Television/James
Clower
√
√
*Office of Development/
Dawn Railey
*Office of Institutional Research, Planning & Effectiveness/Dr. Angela Roling
Physical Plant/Mark Salmon
Federal or State Standards
√
*Financial Affairs/Clint
Carlson
*Grants & Contracts/Judy
Brighton-Enfinger
Annual Financial Audit / Financial
Assessments
Student Surveys
Employer / Alumni Survey
(*Position serves UC &
Phenix City)
Reports
ADMIN. & EDUCATION
SUPPORT UNITS/
CONTACT PERSONS
Internal Monitoring / Monitoring
Use & Involvement
ASSESSMENT TOOLS
___________________
Administrators Survey / In-House
Surveys / Questionnaires
3-23
√
√
*Student Financial Aid &
Veterans Affairs Office/Carol
Supri
Student Involvement &
Leadership/Barbara
Patterson
*Telecommunications/
Johnny Huffman
√
√
√
√
√
√
University Police
Department/Rod Anderson
√
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
√
Other
Student Assessment
Peer Institution Comparison
Enrollment
Focus Groups
Federal or State Standards
√
Annual Financial Audit / Financial
Assessments
√
√
√
√
√
√
√
√
*University Records/Tamara
Jones
Writing Across the
Curriculum/Joan Word
Writing Center/Elaine
Bassett
√
√
√
*TSU Foundation/Linda
DeRamus
University Orientation
Courses/Eleanor Lee
*University Relations/W.D.
Barron
Upward Bound/Mary Griffin
√
Student Surveys
Student Development &
Counseling/Susan Pierce
Employer / Alumni Survey
(*Position serves UC &
Phenix City)
Reports
ADMIN. & EDUCATION
SUPPORT UNITS/
CONTACT PERSONS
Internal Monitoring / Monitoring
Use & Involvement
ASSESSMENT TOOLS
___________________
Administrators Survey / In-House
Surveys / Questionnaires
3-24
√
√
√
√
√
(2)
√
√
(8)
(9)
√
√
Minutes of Board of Trustees, Executive Committee, Advisory Board
Annual evaluation by Chancellor
NCAA Coaches Certification Test
Annual evaluation by Provost
Computer Application Exercise Test
Physical for athletes
NCAA Eligibility Lists
PLAN/ACT Test
Teacher constructed test
Demonstration of planning and evaluating for the units listed in Table 3.2.1-1 above are
available on the webpage of the Office of Institutional Research, Planning and Effectiveness
(OIRPE), and in the Self-Study Resource Room. During the annual Institutional Effectiveness
Cycle, the Six Points of Institutional Effectiveness, the Annual Assessment Reports, the Statement
3-25
of Achievements, and the Annual Plan documents are due at specific times (see Table 3.1.1
Annual Planning Calendar). The following Table 3.2.1-2 provides the organizational structure of
administrative and educational support services.
Table 3.2.1-2
Organization Structure of Administrative and Educational Support Services



Institutional Advancement
o TSU Foundation
o Institutional Development
o Alumni Affairs
Student Affairs
o University Police
o Enrollment Management
o Student Involvement and Leadership
o Auxiliary Services
o Adams Center/Student Services
o Housing and Residence Life
o Student Development and Counseling
o Upward Bound
o Dining Services
o Financial Aid/Veterans Affairs
o Judicial Affairs
Vice Chancellor’s Office
o
Financial Affairs
 Accounting Services
 Accounts Receivable
 Purchasing
 Inventory Control
 Physical Plant
 Accounts Payable
 Department of Human Resources
o Office of Institutional Research, Planning, and Effectiveness
 Office of Grants & Contracts
 Statewide Transfer/Articulation Reporting System (STARS) Program
 Athletic Program
o Technology
 Radio and TV
 Information Technology Management
 Information Technology Services
 Telecommunications
o Phenix City
 Academic Dean Phenix City
 Director, Distance Learning
o University College
 Academic Dean of Distance Learning
 Academic Dean of University College
 Southeast Region, Director
 Florida/Western Region, Director
 Atlantic Region, Director
 Pacific Region, Director
3-26
Provost
 Library
 Instructional Support Services
 ComputerWorks
 Natural Science Center
 Writing Center
 Writing Across the Curriculum
 TSU 1101
 Registrar
 Records
 Transcripts
o Assistant to the Chancellor
 Creative Services
 University Relations
 Governmental Relations
o
3.2.2
For each administrative and education support service unit, the institution must:
1. Establish a clearly defined purpose, which supports the institution’s purpose
and goals.
2. Formulate goals which support the purpose of each unit.
3. Develop and implement procedures to evaluate the extent to which these
goals are being achieved in each unit.
4. Use the results of the evaluations to improve administrative and educational
support services. Each unit, in its planning and evaluation processes, should
consider internal and external factors and develop evaluation methods,
which will yield information useful to the planning processes of that unit.
Each administrative and educational support unit completes the Institutional Effectiveness
Cycle, which involves a five-component process that is repeated annually. The five components
include the following:
1. Six Points of Institutional Effectiveness
2. Annual Assessment Report provides information for the Annual Plan
3. Annual Plan--(including the Statement of Achievements)--supplies the basis
for the Annual Budget
4. Annual Budget—provides the funds for Implementation of the Plan
5. Implementation of Plan
The Institutional Effectiveness Cycle begins in January as administrators/chairs of
administrative and academic support units develop or update the Six Points of Institutional
Effectiveness. In the Six Points of Institutional Effectiveness:
 Point #1 defines the purpose of that unit and how that purpose supports the purpose
of the University;
 Point #2 describes the objectives (how the purpose will be achieved);
 Point #3 states the expected results;
 Point #4 identifies the assessment instruments;
 Point #5 describes the procedures for administering the assessment instruments; and
 Point #6 states the person or persons responsible for the statement of results.
3-27
By virtue of the completion of Points #2 through #6 of the Six Points of Institutional
Effectiveness and the completion of the Institutional Effectiveness Cycle, Troy State University has
developed procedures to evaluate the extent to which unit goals are achieved. After submission to
the OIRPE, documents are published to the Office’s website, and paper copies are also available.
Administrative and Educational Support Unit’s Purpose Supports Institution’s Purpose:
Most units’ Six Points of Institutional Effectiveness relate the purpose of the unit to the
purpose of the University. For example, the Department of Alumni Affairs’ purpose “to establish
mutually beneficial relationship between Troy State University and its alumni” especially supports
the University’s purpose “to respond to the needs of its students and the larger community through
administrative services, utilization of staff and facilities, teaching, scholarships, creative activities,
research, and public service.” The purpose of the Office of Development is “to seek out and raise
funds through various sources.” Through the Office of Development, the University’s purpose “to
provide an appropriate academic, cultural, and social environment” can be realized. The purpose of
the Police Department relates to the University’s purpose by encouraging “each student to develop
his/her unique potential and become productive members of society” and assists in providing an
“appropriate academic, cultural, and social environment.” The University’s objective “to encourage
and provide essential support services for creative activities and research” is reflected in the
Library’s goal “to facilitate faculty and staff research and curriculum development by providing the
essential resources and support services for creative activities and research.” The Office of
Institutional Research, Planning, and Effectiveness specifically supports the University’s objective
“to provide leadership and planning for future development.” Other examples of purpose
statements can be found in the Six Points of Institutional Effectiveness reports for Instructional
Support Services, the Academic Dean of Phenix City, Grants and Contracts, and Student
Involvement and Leadership.
Administrative and Educational Support Unit Objectives Supporting the Unit’s Purpose:
Clearly formulated purposes, objectives, and expected results (outcomes) supporting the
purpose of each unit are included in each administrative and educational support unit’s Six Points
of Institutional Effectiveness. The following are but two illustrations of how a unit’s purpose is
supported with specific goals.
Student Services has established the following strategies as the unit strives “to increase
awareness among the University community as to the variety of services, programs, and facilities
offered by the Adams Center/Student Services”:
1. Conduct and publicize high visibility, traffic-building “event weeks” organized around a
theme and include event-related activities in all areas of operation.
2. Conduct events and promotions specifically targeted to freshmen to increase
awareness of the Adams Center/Student Services.
3. Develop a series of feature articles for the school and local newspapers, as well as
TV/radio coverage on student activities and recreational programs.
The Office of the Dean of Graduate Studies and Research has the purpose of
“coordinating the development of graduate programs, recruiting efforts of graduate faculty and
3-28
students, acquisition of fiscal assets, and policy related to graduate programs.” To achieve this
purpose, the Dean of Graduate Studies and Research plans to
1. Provide leadership in planning, development, administration, evaluation, and
monitoring the quality and effectiveness of all graduate programs.
2. Provide leadership in increasing graduate enrollment.
3. Provide leadership in the acquisition of fiscal assets to enhance research and
graduate programs.
Evaluation Procedures of Administrative and Educational Support Units:
The SPIE require administrative and educational support units to specify the instruments
that will be used in the assessment of the unit’s successful completion of stated goals. The most
commonly used assessment instruments are the TSU Graduating Students Survey and the TSU
Undergraduate Survey administered by the OIRPE.
The Office of Grants and Contracts Survey Instrument is used to assess the overall
effectiveness of that office. The Office of Alumni Affairs is able to assess its success level by the
number of new alumni chapters chartered each year and by the increase in the number of
members. To further measure its success, the Office of Alumni Affairs assesses alumni
participation in events such as Homecoming and T-Day.
The Center for Business and Economic Services (CBES) conducts applied research and
provides services to business and government agencies throughout Alabama. Activities of the
CBES include applied research, educational activities, management counseling, and publications
such as the Troy State University Business and Economic Review. CBES supports the economic
development of small businesses through two subsidiary programs: the Small Business
Development Center (SBDC) and the Small Business Institute (SBI). The SBDC offers one-to-one
consulting, training, information, and government bidding opportunities to small businesses in a
ten-county area of southeast Alabama. The SBI links business students and faculty together with
businesses to provide research projects for small firms. Students receive practical knowledge and
earn class credit. The quality of services offered by the CBES is evaluated by surveys of CBES
clients and students. Of surveys returned in 2000, 100% of clients ranked the services provided by
the CBES as excellent or good. All of the students returning surveys at the end of the FY 2000
rated the experiences as excellent or good.
The Center of Environmental Research and Service (CERS) delivers environmental
research, education, and service to the TSU community, state, and region. CERS coordinates the
Alabama Non-Point Source Water Quality Education Program and publishes the Alabama Water
Watch and Alabama Non-Point Source Newsletter under contract to the Alabama Department of
Environmental Management (ADEM). CERS has also worked with the Choctawhatchee, Pea, and
Yellow River Watershed Management Authorities to monitor water quality in the watershed and to
train and support volunteer water quality monitors. CERS played a key role in establishing the
Environmental Education Association of Alabama. The quality of services offered by CERS is
monitored by self-assessment of accomplishments.
3-29
STatewide Articulation Reporting System (STARS), a contracted program with the Alabama
Commission on Higher Education (ACHE), is a computerized articulation and transfer planning
system designed to inform students who attend Alabama community colleges about degree
requirements, course evaluations and other transfer information pertaining to specific majors at
each State-funded four-year institution. STARS is an effective way of providing students,
counselors, and educators with accurate information upon which transfer decisions can be made.
STARS is the information link between the State’s public two-year and four-year institutions. The
STARS database is designed to prevent the loss of course credit hours, provide information for the
scheduling of course work, and ease the student’s transition from one institution to another.
Assessment of the services for this ACHE-sponsored program is conducted by outside consultants.
The Department of Radio and Television is responsible for TSU-Television, the TSU Public
Radio Network, support for audio and video materials for distance education courses, promotional
audio and video recordings, technical support for the Southeast Alabama Technology Network, and
taping/airing campus events. The Department of Radio and Television provides facilities,
equipment, materials, and staff to enhance the instructional program for broadcast majors of the
Hall School of Journalism and for other qualified students campus-wide. The Department operates
the Radio Reading Service for the sight-impaired and handicapped. The Department of Radio and
Television uses a Departmental Survey, Television Survey, Radio Listener Survey, and the SEAL
Network Survey to assess its programming and services.
Southeast Alabama Regional In-Service Center, which includes the Center for Continuing
Education (CCE), extends the resources of Troy State University to promote lifelong learning for
adult in-service teachers. Quality program offerings are developed by biannual Needs Assessment
Surveys. To ensure the quality of educational services offered by the CCE Center at Troy State
University, participants of CCE workshops complete a Program Evaluation Form. The feedback
from the Program Evaluation Form has been positive. In the 2001 CCE Annual Assessment report,
more than 95% of participants agreed that the content of programs offered met expectations and
that presentations were clear, useful, and appropriate.
Troy State University offers on-line graduate programs to over 400 students with an overall
better-than-average approval rating. End-of-Course Critiques are administered at the end of each
term. Results indicate positive quality in the educational services offered by Distance Learning.
End-of-Course Critiques in the summer of 2000 showed an excellent or good rating for course
components (92%) and instructor services (75%), which parallels those for courses delivered in a
traditional format.
Following are some other examples of assessment instruments used by units and programs:
 ComputerWorks: Survey of Workshop Attendees and the number of academic
courses using ComputerWorks;
 Natural Science Center: Natural Science Questionnaire and Instructor
Questionnaire;
 Financial Affairs: Annual Financial and Compliance Audit conducted by the State
Examiners of Public Accounts;
 Human Resources: An internal log of supplementary payrolls and benefit
administration exceptions;
3-30



University Police: Customer Service Survey;
Student Involvement and Leadership: Student Government Officer and Senate
Reports; and
Student Development/Counseling: Career Services Career Fair Employer
Satisfaction Survey
Assessment tools specific to the administrative and support units of TSU are identified in Table
3.2.1-1 above.
How the Results of Evaluations Have Been Used to Improve Administrative and Educational
Support Services
The Institutional Effectiveness Cycle involves an analysis of the results from assessment
and identification of unit strengths and weaknesses. Annual Plans are formulated that address
specific strategies to improve any identified shortcomings. For example, the Human Resources
Department found that late time cards and employment paperwork led to supplementary payrolls.
Strategies to eliminate the problem of supplementary payrolls involved reissuing guidelines for the
submission of employee time cards and employment paperwork. Although no obvious weaknesses
were observed through the assessment instruments for the Writing Center, plans to improve the
accessibility of their services were formulated. An on-line tutoring program was developed to assist
students who need help after hours and/or students who work during the day and cannot come in
for face-to-face tutorials. Further examples of improvements made in administrative and
educational support services can be found in the Troy State University Statement of Achievements
for 2001-2002—Improvements Resulting from the Institutional Effectiveness Cycle and Annual
Planning Statements for 2002-2003, October 2002
In AY 2000-2001, in conjunction with the distribution of the then new Manual for Annual
Planning October 2000, the Vice President for OIRPE met with the Executive Committee, the
Deans Council, and department chairs within each college, distributed the new Manual, and
presented a PowerPoint presentation about “Planning through the Institutional Effectiveness
Cycle.” In October 2002, sessions were held to review the procedures for “Planning through the
Institutional Effectiveness Cycle” with vice presidents, deans, department chairs, and directors.
3.3
INSTITUTIONAL RESEARCH
3.3.1
Institutional research must be an integral part of the institution’s planning
and evaluation process.
The Office of Institutional Research, Planning and Effectiveness is integral to the planning
and evaluation processes at Troy State University. The OIRPE coordinates the Institutional
Effectiveness Cycle throughout each University program and support unit. The IEC is completed
using an annual Planning calendar (see Manual for Annual Planning, October 2000, p. 3, or Table
3.1.1). Additionally, five basic activities describe the institutional research portion of the OIRPE:
1. Collecting data about the internal state and performance of the institution.
2. Collecting data about the environment of the institution.
3-31
3. Analyzing and interpreting the collected data.
4. Transforming the data analyses and data interpretation into information that
supports institutional planning, policy making, and decision making.
5. Obtaining assessments of research products from consumers that will enhance
the quality and usefulness of future reports.
Institutional research results are reported and distributed annually by the OIRPE in the
Fact Book, which provides demographic information and The Compendium of Survey and Tests
Results which contains specific assessment results that may be used in a unit’s Annual
Assessment Report. In the Annual Assessment Report, each unit states the actual results of
assessment, analyzes the assessment results, compares the actual assessments results to
expected results, identifies strengths and weakness, and provides plans for improvement. Through
the Institutional Effectiveness Committee, coordinated by the OIRPE, all current programs are
reviewed on a rotating schedule to ensure that appropriate planning and evaluation are conducted
for each program. The schedule for review of existing programs is provided in Table 3.3.2. New
programs are also reviewed by the Institutional Effectiveness Committee to ensure that appropriate
planning and evaluation techniques are identified and included in each new program prior to
approval.
3.3.2
It must be effective in collecting and analyzing data and disseminating
results.
Through the direction of the Office of Institutional Research, Planning and Effectiveness
and the oversight of the Institutional Effectiveness Committee, Troy State University effectively
collects, analyzes, and disseminates research and assessment results. Annually, the
Undergraduate Survey, Graduate Survey, Graduating Student Surveys, Alumni Survey, and
Employer Survey are administered, reported, and disseminated through the OIRPE. Additionally,
the results of the Academic Profile test (administered to students prior to graduation to measure
the effectiveness of the general studies program) and a summary of the American College Testing
scores for enrolled freshmen are reported. Results of these surveys and tests are published
annually in the Compendium of Survey and Test Results. The OIRPE also distributes research
information in the annual Fact Book, which includes basic demographic data of the University.
Copies of these documents are located in each unit of the University and are made available
through the OIRPE web page. Additionally, the OIRPE coordinates the completion of the
Institutional Effectiveness Cycle, which is outlined in detail in the Manual for Annual Planning,
October 2000 (pp. 5-20). SPIE, Annual Assessment Reports, Statements of Achievements, and
Annual Plans for each unit of the University are made available through the OIRPE webpage.
The Institutional Effectiveness Committee reviews all planning and assessment documents of
the IEC on a five-year review schedule (See Table 3.3.2 below). A checklist is used, and any
missing or questionable parts of the SPIE, Annual Assessment Reports, or Annual Plans are noted
and sent to the proper unit along with comments and recommendations. The checklist is published
in the Manual for Annual Planning, October 2000 (p. 34). A more thorough description of the
purpose of the Institutional Effectiveness Committee is available in Standing Committees of Troy
State, 2000-2001 (p. 22).
3-32
Table 3.3.2
SCHEDULE FOR REVIEW OF EXISTING PROGRAMS
Spring and Summer 1999, 2004, and 2009
Athletic Department
Sorrell College of Business and its Centers
General Studies
Programs in these areas in University College
Spring and Summer 2000, 2005, and 2010
College of Arts and Sciences and its Centers
College of Education and its Centers
Honors Program
Programs in these areas in University College
Spring and Summer 2001, 2006, and 2011
College of Health & Human Services
College of Communication and Fine Arts
Distance Learning
Programs in these areas in University College
Spring and Summer 2002, 2007, and 2012
Board of Trustees
All Administrators to the Deans Level
Operations of the Vice Chancellors, the Provost, and Vice Presidents for the following:
Phenix City
Institutional Advancement
Institutional Research, Planning, and Effectiveness
Technology
Financial Affairs
Student Affairs
University College
3.3.3
An institution must regularly evaluate the effectiveness of its research
process and use its findings for the improvement of the process.
Although the Office of Institutional Research, Planning, and Effectiveness has primary
responsibility for conducting institutional research at Troy State University, other offices work
closely with the OIRPE in support of data collection and analysis. The Vice President for OIRPE
communicates with the Chancellor, Vice-Chancellor, Provost, vice-presidents, administrative
deans, and chairs/directors to ensure that the data collected and the reports provided are adequate
for use in evaluation, planning, and assessment. Additionally, a User’s Comments Form is included
in the Fact Book and Compendium of Survey and Test Results as a means of improving the
document format and soliciting comments about the data collected. The OIRPE, like any unit of the
University, regularly evaluates its purpose and relationship to the University’s mission in its Six
Points of Institutional Effectiveness. As part of the Institutional Effectiveness Cycle, the OIRPE
completes Annual Assessment Reports and Annual Plans. The assessment findings of the OIRPE
have been used to improve the research process. Results of the 1998-1999 Self-Study
(Assessment Report) of the OIRPE noted that the Manual for Planning, October 1995 needed
simplification. The OIRPE responded to this finding by simplifying the written documentation
required for the IEC. The OIRPE stated in its 2000-2001 Annual Plans an improvement goal of
placing SPIE, Annual Assessments, and planning documents for all areas of the University on the
OIRPE website. This goal was established in order to reduce the paperwork of the OIRPE and
3-33
increase accessibility to institutional research, planning, and effectiveness information. This goal
was achieved and listed in the Statement of Achievements in the Assessment Report of 20002001. Other examples of evaluation of effectiveness and use of findings to improve the process of
the OIRPE can be noted in the reports on the OIRPE website. The Statement of Achievements on
page 39 of the TSU Statement of Achievements for 2001-2002 (Improvements Resulting from the
Institutional Effectiveness Cycle and Annual Planning Statements for 2002-2003) provides
additional improvements in the OIRPE.
3.3.4
Institutions must assign administrative responsibility for conducting
institutional research, allocate adequate resources, and allow access to
relevant information.
The Troy State University Board of Trustees and the Chancellor have assigned
administrative responsibility for conducting institutional research to the Vice President of
Institutional Research, Planning, and Effectiveness. Each educational program/administrative and
educational support unit is assigned responsibility for the administration of area specific
assessment instruments. The procedures for administering assessment instruments are stated in
each department’s/unit’s SPIE.
Troy State University has allocated adequate resources for institutional research. Troy
State University made the commitment to institutional research in 1989 with the establishment and
funding of the Office of Institutional Research, Planning, and Effectiveness assigned to conducting
and coordinating institutional effectiveness. The OIRPE operates under a budget that includes
salaries for the Vice-President for OIRPE and staff, equipment, in-house printing, supplies, contract
printing, professional travel, postage, and telephone services. The AY 2001-2002 budget of
$228,092.00 allows the University and staff to review systematically and regularly, study, and
improve identified weaknesses.
The Vice President for OIRPE and staff have access to relevant University information,
and the OIRPE has established access to relevant information for all University personnel through
the Fact Book, Compendium of Survey and Test Results, Manual of Annual Planning, and the
OIRPE website. Paper copies of documents are also made available to all administrative offices,
the Library, academic deans’ offices, and all departments.
3-34
RESOURCES

Challenge 2000: Strategic Plan to the Twenty-first Century

Compendium of Survey and Test Results, 1997-1998 through 2000-2001

Fact Book, March 2000

Fact Book, March 2001

Faculty Handbook, 2001

Graduate Bulletin, 2000-2001

Manual for Annual Planning, October 1995

Manual for Annual Planning, October 2000

Manual for Planning, October 2000

OIRPE Operating Budget (AY 2001-2002)

OIRPE web page (http://spectrum.troyst.edu/~oirpe/)

Over the Horizon: Strong Values-Clear Vision, Troy State University Strategic Plan:
2001-2005

Section I Statements of Achievements for 2001-2002 – Improvements Resulting from the
Institutional Effectiveness Cycle

Standing Committees of Troy State University

Student Evaluation Survey of Course/Instructor form

Student Research, Examples of

TSU Annual Planning Statements for 2001-2002

TSU Statement of Achievements for 2000-2001

Undergraduate Bulletin, 2001-2003
3-35
STRENGTHS

Planning is systematic, broad based, interrelated and appropriate to the Institution.

The Institutional Effectiveness cycle is well defined and relates the purpose of each
educational program, administrative and educational support unit to the institutional
purpose, identifies appropriate assessment tools, and establishes measurable objectives
that are assessed and used to improve programs.
AGENDA FOR EXCELLENCE

Suggestion 3.1.1: Although the current Institutional Effectiveness Cycle is broad and well
defined, considering the expansion of University program offerings since its adoption,
technological advances, and the desire to parallel institutional effectiveness methods
across the Troy State University System, the Committee suggests that the current IEC be
modified in a way that effectively measures institutional outcomes without the
cumbersomeness of the current method.

Suggestion 3.1.4: The Committee suggests that Troy State University effectively evaluate
the academic rigor currently in place in the General Studies Program to determine the
appropriateness and effectiveness of the key elements, viz., reading, writing, speaking,
computing, and critical thinking, and if necessary, strengthen course requirements in those
areas. Similarly, the Institution must take the precaution beforehand to make certain that
any curricular revisions and outcomes expectations be based upon reliable data.
o Suggestion 3.1.4 A: The Committee suggests that Troy State University
effectively increase the rigor in the General Studies program.
o Suggestion 3.1.4 B: The Committee suggests that the evaluation methods of the
Academic Profile Exam or similar instrument be embedded, insofar as possible
and practical, within the key components of the General Studies Program.
o Suggestion 3.1.4 C: The Committee suggests that to further measure the
effectiveness of the General Studies programs, more freshmen should be tested
with the Academic Profile (AP) so that there will be a larger group for comparison
in the Analysis of Covariance statistical process as well as in the dependent t-test
comparison scores of students. A more assertive recruitment of freshmen and a
tracking of a group of freshmen through the General Studies program are ways in
which the effectiveness of the Program can be more accurately assessed. Tested
freshmen students should be tracked, and a dependent t-test comparison of the
pre-test (AP at freshmen level) mean and post-test (AP at junior/senior level)
mean for the same students in the General Studies program should be made.
o Suggestion 3.1.4 D: The Committee suggests that a training program on how to
use the results of the Academic Profile or a similar instrument to improve the
General Studies curriculum be provided for those faculty, department chairs, and
deans entrusted with the academic integrity of the General Studies Program.
3-36

Suggestions 3.1 and 3.2: The Committee suggests that Troy State University develop
methods to ensure quality in all reports of the Institutional Effectiveness Cycle.
o Suggestion 3.2.2 A: The Committee suggests that the Office of Institutional
Research, Planning and Effectiveness, along with the upper level administrators
including the Chancellor, the vice chancellors, the Provost, vice presidents, and
the deans, take a more proactive approach in educating departmental chairs and
unit directors in regard to the purpose of the Institutional Effectiveness Cycle.
o Suggestion 3.1.3 and 3.2.2 B: The Committee suggests that although the IEC is
explicit in its instructions to relate assessment results to SPIEs, plans for
improvement to assessment results, and assessment results back to plans, some
programs/units do not effectively parallel these through the cycle. As a result
“closing the loop” examples are obscured in the documentation. Therefore, the
Committee suggests that the Office of Institutional Research, Planning, and
Effectiveness develop a matrix-type form to be completed by each unit. This form
should relate assessment to SPIEs, plans to assessments, and assessments to
implemented plans of the previous year, and the form should be submitted and
returned for resubmission at each step through the IEC.
o Suggestion 3.1.3 and 3.2.2 C: The Committee suggests that deans, the Provost,
and vice presidents review the IEC reports more thoroughly before forwarding
them through channels to the Office of Institutional Research, Planning and
Effectiveness. Providing direct assistance and feedback on a regular basis should
ensure that the IEC functions as a continuous improvement model in which
assessment results are used to formulate plans for improvement of programs,
services, and operations.
o Suggestion 3.2.1: The Committee suggests that all administrative positions, units,
and programs complete the Institutional Effectiveness Cycle by providing the Six
Points of Institutional Effectiveness, the Annual Assessment Reports, the
Statement of Achievements, and the Annual Plan by the designated deadline
dates so these reports may be published on the website of the Office of
Institutional Research, Planning, and Effectiveness in a reasonable timeframe.
o Suggestion 3.2.2 D: Currently, the administrative positions must complete the
IEC under the same stipulations as do academic programs, and support units.
This requirement has caused some confusion, as interpretations have varied in the
method of assessing administrative positions. Therefore, the Committee suggests
that a uniform approach for evaluation of administrative position effectiveness
should be developed.
o Suggestion 3.3.2: The Committee suggests that the Institutional Effectiveness
Committee provide a closer and more frequent review of all Institutional Effective
Cycle reports before the reports are accepted and published to webpage.
3-37
COMPLIANCE CHART
COMPLIANCE
Must
Statement
3.1
3.1.1
Yes
No
EDUCATIONAL
PROGRAMS

3.1.2

3.1.3

3.1.4

3.1.5

NA
Supporting Documentation
Over the Horizon: Strong Values-Clear
Vision, Troy State University Strategic
Plan: 2001-2005
Challenge 2000: Strategic Plan
to the Twenty-first Century
Manual for Planning, October
2000
Compendium of Survey and
Test Results (annually from 1993-1994
through 2000-2001)
Undergraduate Bulletin, 20012003
Fact Book
Faculty Handbook, 2001
Student Evaluation Survey of
Course / Instructor Form 2001
Undergraduate Bulletin, 20012003
Compendium of Survey and
Test Results
OIRPE webpage
Manual for Annual Planning,
October 2000
OIRPE webpage
Manual for Annual Planning,
October 2000
Undergraduate Bulletin,
2001-2003
Section I Statements of Achievements for
2001-2002 – Improvements Resulting
from the Institutional Effectiveness Cycle
Compendium of Survey & Test
Results
2001 Faculty Handbook
Student Evaluation of Course /
Instructor form
2001-2003 Undergraduate
Bulletin
2000-2001 Graduate Bulletin
Manual of Annual Planning, October 2000
Examples of Student Research
Copies of Accreditation Reports
by outside agencies
OIRPE webpage
OIRPE web page
2001-2003 Undergraduate
Bulletin
2000-2001 Graduate Bulletin
Standing Committees of Troy
Location
Self-Study Resource R.
Self-Study Resource
Room
Self-Study Resource
Room
Self-Study Resource
Room
Self-Study Resource
Room
3-38
3.1.6
State University
2001 Faculty Handbook
Compendium of Surveys &
Tests Results
Fact Book, 2001
OIRPE webpage
Copies of Accreditation Reports
by outside agencies
Fact Book, 2001

3.2
Self-Study Resource
Room
ADMINISTRATIVE AND
EDUCATIONAL SUPPORT
SERVICES
3.2.1
3.2.2
3.3


OIRPE webpage
Manual for Annual Planning,
October 2000
OIRPE webpage
Manual for Annual Planning,
October 2000
Self-Study Resource
Room
Self-Study Resource
Room
INSTITUTIONAL RESEARCH
3.3.1

3.3.2

3.3.3

3.3.4

OIRPE webpage
Manual for Annual Planning,
October 2000
Fact Book, 2001
Compendium of Survey & Test
Results, 1999-2000
TSU Annual Planning
Statements for 2001-2002
TSU Statement of Achievements
for 2000-2001
Troy State University Strategic
Plan: 2001-2005
OIRPE webpage
Manual for Annual Planning
Compendium of Surveys &
Tests Results
Fact Book
Standing Committees of Troy
State University, 2000-2001
OIRPE webpage
Manual of Annual Planning,
October 1995
Manual of Annual Planning,
October 2000
Fact Book, March 2000
Compendium of Surveys &
Tests Results, 1999-2000
OIRPE webpage
Manual of Annual Planning,
October 2000
Fact Book, March 2000
Compendium of Surveys &
Tests Results, 1999-2000
OIRPE Operating Budget
Self-Study Resource
Room
Self-Study Resource
Room
Self-Study Resource
Room
Self-Study Resource
Room
Self-Study Resource
Room
OIRPE, 231 Adams Adm.