UNIVERSITY ASSESSMENT PLAN FOR POLYTECHNIC
Goals and Objectives of Assessment
Assessment is, first and foremost, a process. It is not in and of itself a product, although the
process will “produce” a variety of results and documents that are an important part of the process. The
process must lead to continuous improvement in the delivery of service to the University’s many
constituencies.
In the ABET 2000 era, assessment focuses on identifiable outcomes that demonstrate the
achievement of clearly stated missions, goals, and objectives.
Thus, the goals and objectives of the assessment process include:
1.
2.
3.
4.
5.
6.
7.
Documenting the University mission.
Documenting the mission, goals, and objectives of each academic program of the University.
Documenting the mission, goals and objectives of each non-academic unit or function of the
University.
Measuring, on a regular and repetitive basis, the achievement of mission, goals, and/or
objectives by the University as a whole, and by individual academic and non-academic units.
Identifying shortcomings in the achievement of mission, goals, and/or objectives at all levels
of the University.
Development of plans to improve/advance the achievement of mission, goals, and/or
objectives of the University as a whole, and of individual academic and non-academic units.
Incorporating feedback on specific plans and actions into subsequent measurements of
success against mission, goals, and/or objectives.
It is important to recognize that the entire University community must be involved in the process
of assessment and continuous improvement. An Office of Assessment has been established to organize the
process, to conduct the many surveys that will be required, to provide a consistent and complete data base
for informative statistical analyses, to provide a central location for assessment results and record-keeping,
and to assist various units of the University in conducting the assessment process.
The results of assessment, however, must be reviewed and responded to by all affected units of the
University. It is important to note that appropriate responses to assessment results are the responsibility of
virtually every member of the University community, as assessment will touch every academic and nonacademic unit of the Polytechnic.
Organization of the Assessment Process at Polytechnic University
The assessment process at Polytechnic is directed by the Provost as Chair of the Polytechnic
Assessment Team (PAT). The PAT serves as a steering committee overseeing various aspects of the
assessment process throughout the University.
The membership (Nov 2000) of the PAT consists of:
Ivan Frisch
Debbie Cassetta
Dennis Dintino
Haang Fung
Dave Gillette
Ellen Hartigan
Mahnaz Karim
Jeanette Magee
Mike Mainiero
Roger Roess
Provost and Chair
Director of Assessment
Director of Information Services
Director of Special Services
Director of Career Services
Vice-President for Student Affairs
Director of Development Information
Academic Advisor for Computer Science
Director of Institutional Research
Professor
1
Krisoye Smith
Nancy Tooney
George Vradis
Cynthia Wolff
Office of Assessment
Associate Dean
Professor
Library
The membership of this group will change from time to time, but will maintain the same representation as
that indicated in the listing.
The specific conduct of assessment activities is supervised by two individuals, one assigned
responsibility for all academic aspects of assessment, the other assigned responsibility for non-academic
aspects of assessment. These individuals are:
Dr. Roger P. Roess,
Special Assistant to the Dean for Acdemic Assessment
Professor of Transportation Engineering
Mr. Michael Mainiero
Special Assistant to the Provost for Non-Academic Assessment
Director of Institutional Research
The Office of Assessment, established as part of the Office of the Dean of Engineering and
Applied Sciences, is responsible for the day-to-day operations of the assessment process, and for
maintaining a comprehensive data base for assessment. It is led by a Director of Assessment (Ms. Debbie
Cassetta), who reports to the Special Assistant to the Dean for Academic Assessment. It contains a
complement of supporting personnel, including a computer data analyst, secretarial assistance, student
assistance, and a budget including travel, equipment, and substantial M & S support.
There are two working groups that assist. The Working Group for Academic Assessment includes
a representative of every academic department, the library, and special services. It is chaired by Ms.
Cassetta as Director of Assessment. The Working Group for Non-Academic Assessment includes
representatives of various non-academic units, and is chaired by Mr. Mainiero.
Another group, called the Assessment Advisory Board, consists of representatives of student
groups, alumni, and industry. This board performs an advisory role to the Provost, and provides input on
surveys, survey results, assessment processes, and other aspects of the process.
Assessment Baseline(s)
There are four base documents that define the overall context of University missions, goals, and
objectives against which outcomes are to be measured:
1.
Academic Mission and Objectives for Polytechnic University This document includes a
formal statement of the University’s overall mission, and a summary of the mission
statements, goals, and objectives of every academic department and inter-departmental
program.
2.
Strategic Plan The Polytechnic Strategic Plan, formally approved by the Board of Trustees
in Spring 2000, provides an additional set of specific goals and objectives, as well as
strategies to achieve them.
3.
Missions and Objectives for Non-Academic Units of Polytechnic This document includes a
general mission statement for supporting services and departments at the Polytechnic, and
specific missions, goals, and objectives for individual non-academic units.
2
4.
Course Objectives and ABET Competencies
Each academic department maintains a
documented set of course objectives for every course offered. Each undergraduate course
also maintains a list of the ABET-defined core competencies that the course addresses. These
documents are created and maintained by each department, and copies are made available to
the Assessment Office. They are also incorporated into each semester’s course evaluation
surveys.
Each of these documents is reviewed and modified as appropriate each year as part of the assessment
process. While the PAT supervises the effort, all appropriate units and individuals participate in the review
and modification process.
Assessment Approaches
A. Academic Assessment
There are three general approaches employed at Polytechnic in the implementation of academic
assessment:



Traditional review and grading of student material.
Data analysis.
Constituent surveys.
Traditional Grading Approaches
While certainly not sufficient, traditional grading of student material continues to be an important and
necessary part of assessing both the progress of individual students and the success of curricula and their
delivery. Polytechnic has made a substantial effort to insure that a wide variety of assignments are
included over the course of a student’s academic career. These include:






Examinations, including numerical, short answer, and essay style questions.
Homework assignments, including numerical and written work.
Term paper and/or project assignments, both for individuals and groups.
Oral presentations of project/term paper results.
Design projects (group and individual) resulting in a fully documented design report.
Laboratory reports on the conduct and results of laboratory experiments.
Each course employs one or more of these techniques to assess the progress of individual students.
The quality with which such assignments are completed, however, also provides insight into the
effectiveness of the curriculum and its delivery.
Data Analysis
A great deal of insight can be gained by careful analysis of the basic data provided by the University
information systems. For students, the Polytechnic information system contains information on
admissions, financial aid, academic standing and progress, enrollment status, grades, etc.
Typically, new freshman cohorts are identified each year. These become the basis for analysis of
persistence, transfer, and graduation rates. Sub-cohorts based upon gender, ethnicity, H.S. origin, SAT
scores, placement exam results, GPA and others can be identified and separately analyzed to reveal
problems that are unique to various components of the student population.
Correlations among key variables, such as GPA, placement exam results, SAT scores, persistence, and
graduation rates can be explored and examined. Cohorts by major, ethnicity, gender, SAT scores, etc. can
be explored and compared. Such analyses are conducted by the Office of Assessment, and the results are
3
widely disseminated to the PAT and Working Groups, and academic and non-academic departments as
appropriate.
Constituent Surveys
A number of surveys are conducted each year as part of the assessment process. These are listed
below, with a brief annotation of what each consists of and what it hopes to accomplish.
1.
Semester Course Surveys A systematic course and instructor evaluation survey is conducted each
semester for all (undergraduate and graduate) courses at Polytechnic. Each survey has several
distinct parts, including: a) general and specific questions concerning the course material and its
delivery, b) general and specific questions concerning the effectiveness of the instructor and the
various approaches used in the classroom and outside it , c) questions concerning the effectiveness
of the course/instructor in developing the ABET competencies (only those addressed in the course
description and outline are included in the survey), d) questions related to the specific goals and
objectives set for the course, and e) special questions related to aspects of a course that might be
unique. By vote of the Polytechnic Faculty, participation in the course evaluation survey is
required of all undergraduate students in sections of 6 or more. It is optional for all other courses.
Grades are withheld until students required to participate in the survey do so. The course survey is
conducted electronically over Polytechnic’s intra-net.
2.
Senior Exit Surveys Each semester, graduating students (undergraduate and graduate) are asked
to fill out a general exit survey that covers all aspects of their university experience, including
curricula and instruction, facilities, supporting services, social environment and a variety of other
issues. By vote of the Administrative Council, participation is mandatory.
Diplomas are
withheld until an exit survey is completed. The senior exit survey is also conducted electronically
over Polytechnic’s intra-net.
3.
Alumni Surveys Each year, alumni are surveyed at specified intervals after leaving Polytechnic.
Alumni are surveyed one year, five years, ten years, fifteen years, and twenty years after
graduation. These surveys seek to establish the placement and career experiences of alumni, as
well as their opinions on how well Polytechnic prepared them for their careers. This annual
survey is coordinated with the Alumni Office, and is conducted both electronically and with paper
(at the discretion of the respondent). As an experiment, some of these surveys will include a
companion survey for the recipient’s immediate supervisor.
4.
Employer Surveys
Working with the Career Services Office, an annual survey of major
employers of Polytechnic graduates is conducted. This survey seeks to establish views of these
employers on the preparedness of Polytechnic graduates for the working environment. A related
survey of Polytechnic co-op students is also conducted.
5.
Faculty Surveys Every two years, a major survey of Polytechnic faculty is conducted. It is
intended to determine what instructional techniques are being used, gather information on faculty
training needs, assessing the faculty view of the adequacy of teaching facilities and support
services, etc.
6.
Staff Surveys Every two years, a major survey of Polytechnic staff is conducted. It is intended to
gather information on the quality of supporting services throughout the University and to elicit the
views of staff on various relevant issues.
Each year, as the results of these surveys are analyzed and discussed, the survey instruments
themselves are assessed and modified where necessary.
A variety of special surveys may also be conducted on an ad-hoc basis. Some academic units conduct
their own detailed surveys (such as the Library, Special Services, and Career Services) of users. The Office
4
of Assessment and PAT always receive copies of these surveys and their results. In some cases, aspects
or all of a given survey may be built into one of the regularly-occurring surveys noted previously.
B. Non-Academic Assessment
A parallel process to that for academic assessment is also necessary to assess the performance of
“non-academic” offices. Such offices include, for example, the business office, the human resources office,
the purchasing office, the facilities office, etc. These and other offices provide essential services to various
constituencies, including students, faculty, staff, alumni, and industry. A framework for such assessment is
based primarily upon stated goals and objectives in the University’s Strategic Plan, supplemented by annual
Management by Objective (MBO) statements from the various vice-presidents.
Because the Strategic Plan is a broad document, some refinement of the general goals and
objectives is needed to do a realistic outcomes assessment. Using the Strategic Plan and MBO’s a
consolidation of goals and objectives must be performed, and then related to the various non-academic
units responsible for carrying them out. This consolidation takes place annually, and is done by the NonAcademic Working Group and reviewed by the PAT. The process includes:




Review and organize the tactics and strategies defined for non-academic units.
Clarify the outcomes in terms that are conducive to evaluating relative success or failure.
Develop appropriate assessment methods with the objective of minimizing the number of
separate surveys and/or analyses that must be conducted.
Define the process for collection and reporting of results of non-academic assessment.
Because the assessment of non-academic areas is a newer function, and because there are more
complicated issues involved, the process for these assessments must itself be clarified and developed over a
period of several years. It is also clear that the assessment process for non-academic functions must be
carefully reviewed and refined each year.
The process will rely primarily on data analysis, and on constituent surveys.
Constituent Surveys
A good deal of information concerning non-academic functions can be included on surveys that
are already a part of the academic assessment process. For items that impact service to students (like
financial aid, admissions, registrar, facilities, student life, etc), questions may be included on annual senior
exit surveys. The biannual faculty surveys can include questions on services for which they are
constituents (payroll, human resources, purchasing, facilities, student record systems, etc). Alumni surveys
can include questions on the Alumni Office and its services, for example.
In addition to coordinating appropriate questions on regular academic surveys, it may also be
necessary to undertake specific surveys of other constituencies. Other constituencies that are not part of the
academic surveys include staff and parents. Regular surveys of these groups, initially once per year,
perhaps reducing to once every other year in future years, will be developed and implemented.
Data Analysis
Survey results can and will be augmented by hard data analysis from the University’s various
information systems. The success of the Development Office, for example, may be partially accomplished
by comparing actual fund raising amounts with MBO and Strategic Plan targets. Admissions targets can be
compared to actual admission data with appropriate stratifications.
5
Feedback
The results of surveys are summarized by the Office of Assessment, which may also conduct
additional statistical analysis and evaluation of such results. All results are disseminated to appropriate
academic and non-academic units.
Each unit is asked to analyze and evaluate the results of these surveys, and of data analysis results
provided by the Office of Assessment. Units are asked to prepare a formal report each year which
documents a) their interpretation of the overall assessment results, and b) an action plan for the following
year that addresses any shortcomings or improvement needs revealed in assessment results. This is
generally done during the Fall semester, and formal reports are expected by December 1 of the subject year.
Feedback reports, as well as survey results are distributed to all administrators, the PAT, and the
Assessment Working Groups for review. The PAT (assisted by the Assessment Office) prepares an annual
assessment report, summarizing all findings and laying out a general plan of action for the upcoming year.
This summary report is widely distributed to all Polytechnic constituencies, including students.
6
FIGURE 1: SCHEMATIC OF THE POLYTECHNIC ASSESSMENT PROCESS
7
TABLE 1: TYPICAL ANNUAL SCHEDULE
Fall Course Survey
Requests for survey form submittals
Survey forms completed
Surveys conducted
Oct 15
Nov 15
Dec 1 – Dec 31
Spring Course Survey
Requests for survey form submittals
Survey forms completed
Surveys conducted
Mar 1
Apr 1
Apr 15 – May 15
Graduating Student Exit Survey
Review of survey instrument
Preparation of surveys for UG and GR students
Survey of January graduates
Survey of June graduates
Oct
Nov – Dec
Jan
Jun
Alumni Survey
Review of survey instrument
Preparation of surveys for UG and GR alumni
Conduct surveys
Sept
Oct – Nov
Mar – Apr
Faculty and Staff Surveys (Every 2 Years)
Review survey instruments
Prepare survey instruments
Conduct surveys
Jan – Feb
Mar
Apr
Employer/Co-op Surveys
Review survey instruments
Prepare survey instruments
Conduct surveys
Jan – Feb
Mar
Apr
Analysis and Review Schedules
Summarize survey results
Distribute results for departmental and committee
review
Formulation of departmental responses
PAT and other review of results and responses
Preparation of annual report and recommendations
8
As received
Jun – Jul
Aug – Sep
Oct - Nov
Dec