Heavy-duty diesel Technology AAS
Educational Effectiveness
Assessment Plan
Version 1.0
Adopted by
The Heavy-Duty Transportation and Equipment faculty: November 30, 2007
Submitted to
The Dean/Director of the Community and Technical College: November 30, 2007
The Office of Academic Affairs: December 7, 2007
687288425
Submitted 6-16-08
Page 1 of 14
TABLE OF CONTENTS
Mission Statement __________________________________________________________________ 3
Program Introduction _______________________________________________________________ 3
Assessment Process Introduction ______________________________________________________ 3
Program Outcomes _________________________________________________________________ 4
Table 1: Association of Assessment Measures to Program Outcomes _________________________ 5
Assessment Measures _______________________________________________________________ 6
Table 2: Program Outcomes Assessment Measures and Administration_______________________ 6
Assessment Implementation & Analysis for Program Improvement __________________________ 7
General Implementation Strategy _________________________________________________________________ 7
Method of Data Analysis and Formulation of Recommendations for Program Improvement ___________________ 7
Modification of the Assessment Plan ______________________________________________________________ 7
Appendix A: NATEF Exit Exams _____________________________________________________ 8
Measure Description: __________________________________________________________________________ 8
Factors that affect the collected data: ______________________________________________________________ 9
How to interpret the data: _______________________________________________________________________ 9
Appendix B: Employer Interview ____________________________________________________ 10
Measure Description: _________________________________________________________________________ 10
Factors that affect the collected data: _____________________________________________________________ 11
How to interpret the data: ______________________________________________________________________ 11
Appendix C: ASE Exams___________________________________________________________ 12
Measure Description: _________________________________________________________________________ 12
Factors that affect the collected data: _____________________________________________________________ 12
How to interpret the data: ______________________________________________________________________ 12
Appendix D: Practicum Reports _____________________________________________________ 13
Measure Description: _________________________________________________________________________ 13
Factors that affect the collected data: _____________________________________________________________ 14
How to interpret the data: ______________________________________________________________________ 14
687288425
Submitted 6-16-08
Page 2 of 14
MISSION STATEMENT
It is the mission of the Heavy-Duty Transportation and Equipment program at the University of Alaska
Anchorage to prepare students for high paying, high demand careers in the transportation repair and
maintenance field, to assist working heavy-duty diesel technicians in continuing and professional
education, to assist other relevant programs (both secondary and post-secondary throughout Alaska)
and to advance the image of the Heavy-duty diesel and Technology industry.
PROGRAM INTRODUCTION
The Heavy-Duty Transportation and Equipment AAS is designed to prepare students for careers in the
heavy-duty diesel maintenance and repair industry. Curriculum design is based on standards
developed by the National Institute for Automotive Excellence (ASE) and the program is accredited
by ASE through the National Automobile Technicians Education Foundation (NATEF) Classroom
and lab activities are designed to prepare students with specific knowledge and skill sets to assess,
maintain and repair current technology, and to teach strategies required to adapt to new technologies.
The program was successfully reviewed by NATEF and recertified in 2006. The next full
recertification is due by April 2011, with a mid-term review in 2009. The recertification process
requires a self-study by the faculty and local industry partners serving as an advisory committee,
followed by a review by a NATEF certified evaluation team leader working with industry volunteers
from the local area. The complete program standards can be viewed in pdf at
http://www.natef.org/program_standards/pdf/truck_standards.pdf
Because the HDTE AAS and Certificate programs are based on the same core curriculum and the
same national standards, the assessment plans for the 2 programs are nearly identical. It is assumed
that students in the AAS program will perform to higher standards in written and oral communication,
and in the other GER courses.
ASSESSMENT PROCESS INTRODUCTION
This document defines the expected student learning outcomes for the HDTE program and outlines a
plan for assessing the achievement of the stated outcomes. This assessment plan represents a major
change from previous plans in that we have elected to use fewer more general outcomes. We will
continue to assess the specific NATEF technical areas as embedded items within the assessment tools.
The development of the outcomes consisted of a review of NATEF standards by faculty and industry
partners. In addition, the HDTE advisory committee reviews the curriculum at least twice during the 5year certification cycle, and program effectiveness at least one time per year.
One recent addition to the plan is the addition of external national exit tests taken by students in
electrical and heating and air conditioning courses. These tests are provided by NATEF and proctored
by the faculty responsible for the courses. Additionally, student and mentor reports are being
standardized and upgraded for the mandatory practicum. The faculty met and accepted the outcomes
and assessment processes on November 30, 2007.
Because the HDTE AAS and Certificate programs are based on the same core curriculum and the same
national standards, the assessment plans for the 2 programs are nearly identical. The ASE/NATEF
certification of the HDTE program covers both the AAS and certificate programs.
687288425
Submitted 6-16-08
Page 3 of 14
PROGRAM OUTCOMES
At the completion of this program, students are able to:

Demonstrate technical knowledge and skills necessary for success in the heavy-duty diesel
maintenance and repair industry.

Demonstrate academic proficiency necessary to pass national examinations.

Demonstrate proficiency in performing occupationally related tasks in a professional setting.

Integrate knowledge from diverse areas to develop effective diagnostic and repair strategies
involving complex systems.

Demonstrate effective oral and written communication skills necessary for success in the
workplace.
687288425
Submitted 6-16-08
Page 4 of 14
Practicum
Reports
ASE
Exams
Outcomes
Employer
Interview
NATEF
Exit
Exams
TABLE 1: ASSOCIATION OF ASSESSMENT MEASURES TO PROGRAM OUTCOMES
Demonstrate technical knowledge and skills necessary for
success in the heavy-duty diesel maintenance and repair
industry.
1
0
1
1
Demonstrate academic proficiency necessary to pass
national examinations.
1
0
1
0
Demonstrate proficiency in performing occupationally
related tasks in a professional setting.
0
0
0
1
Integrate knowledge from diverse areas to develop
effective diagnostic and repair strategies involving
complex systems.
1
0
1
1
Demonstrate effective oral and written communication
skills necessary for success in the workplace.
0
1
0
1
0 = Measure is not used to measure the associated outcome.
1 = Measure is used to measure the associated outcome.
687288425
Submitted 6-16-08
Page 5 of 14
ASSESSMENT MEASURES
A description of the measures used in the assessment of the program outcomes and their
implementation are summarized in Table 2 below. The measures and their relationships to the
program outcomes are listed in Table 1, above.
There is a separate appendix for each measure that shows the measure itself and describes its use and
the factors that affect the results.
TABLE 2: PROGRAM OUTCOMES ASSESSMENT MEASURES AND ADMINISTRATION
Measure
NATEF Exit
Exams
Employer
Interview
Description
Frequency/
Start Date
Collection
Method
The NATEF exit exams are a series of 8
national exams, corresponding to the 8
automotive areas recognized by the National
Institute for Automotive Service Excellence.
The HDTE program utilizes the Electrical
and HVAC tests from the automotive series.
Tests will be
administered
during the
semester that
the relevant
course was
taken.
Tests are
taken on line.
Faculty or
program
assistant
Fall and
Spring
Semester
Faculty
debriefs
employers and
an evaluation
sheet is
collected.
Faculty
Employer interviews are required in the
preventive maintenance and inspection
course and as part of the practicum process.
Interviews are conducted by faculty, by
students, or by employers brought in from
industry for a career day. Interviews are
conducted and critiqued.
ASE Exams
Practicum
Reports
ASE exams are the recognized national
certification for the industry
Annual
All students in the program are required to
complete a paid practicum. Reports are
gathered from students and supervisors
during the practicum.
Each
semester
687288425
Submitted 6-16-08
ASE Exams
are
administered
at official test
centers. Our
program
submits
information on
graduates and
receives a
report on
those who
successfully
pass the tests.
Faculty
communicate
directly with
students and
mentors
Page 6 of 14
Administered
by
Program
assistant
Faculty
ASSESSMENT IMPLEMENTATION & ANALYSIS FOR PROGRAM IMPROVEMENT
General Implementation Strategy
NATEF exit exams were implemented during the 2006-2007 academic year. Tests were administered
to students in the final semester of the program. Due to constructive feedback from students and
faculty, the faculty elected to have relevant tests administered to students each semester, based on the
courses taken during that semester. (e.g. students will take the exit exam for the electrical area during
the electrical 3 course.) Because NATEF has not yet released tests specific to the Heavy-Duty
industry, HDTE students take electrical and heating ventilation and air conditioning tests from the
automotive technology group. Technology in these areas has sufficient overlap to justify using the
automotive tests.
Employer interviews have been a part of the program for several years but will be included in the
formal assessment process for the first time in Spring 2008. The General Automotive option will pilot
the data collection through in conjunction with the career day.
Method of Data Analysis and Formulation of Recommendations for Program Improvement
The program faculty will meet at least once a year with the NATEF advisory committee to review the
data collected using the assessment measures. This meeting should result in recommendations for
program changes that are designed to enhance performance relative to the program’s outcomes. The
results of the data collection, an interpretation of the results, and the recommended programmatic
changes will be forwarded to the Office of Academic Affairs (in the required format) by June 15 th each
year. A plan for implementing the recommended changes, including of advertising the changes to all
the program’s stakeholders, will also be completed at this meeting.
The proposed programmatic changes may be any action or change in policy that the faculty deems as
being necessary to improve performance relative to program outcomes. Recommended changes
should also consider workload (faculty, staff, and students), budgetary, facilities, and other relevant
constraints. A few examples of changes made by programs at UAA include:
o changes in course content, scheduling, sequencing, prerequisites, delivery methods, etc.
o changes in faculty/staff assignments
o changes in advising methods and requirements
o addition and/or replacement of equipment
o changes to facilities
Modification of the Assessment Plan
The faculty, after reviewing the collected data and the processes used to collect it, may decide to alter
the assessment plan. Changes may be made to any component of the plan, including the outcomes,
assessment measures, or any other aspect of the plan. Faculty will approve any changes to the
program. The modified assessment plan will be forwarded to the dean/director’s office and the Office
of Academic Affairs.
687288425
Submitted 6-16-08
Page 7 of 14
APPENDIX A: NATEF EXIT EXAMS
Measure Description:
The NATEF End of Program test series is designed to measure students' knowledge of course content
and learning outcomes within an automotive training program. The tests include a combination of
theory and vehicle service related questions, and are developed using the NATEF task list.
The End of Program Tests provide instructors and administrators with impartial feedback on how well
students are learning and comprehending the materials presented throughout the curriculum. The
invaluable information provided to students and school personnel makes this test beneficial to any
school. It is available to any entry-level Automobile or Collision Repair and Refinish training program
regardless of ASE/NATEF certification status.
These tests are appropriate for students who have nearly completed an individual automotive training
course or who are preparing to graduate from a technician-training program. Schools choose which
tests in the series to administer to each student. Testing is available in both fall and spring to ensure
maximum usefulness.
AUTOMOBILE


Electrical/Electronic Systems
Heating and Air Conditioning
All students who complete an End of Program Test receive a detailed score report and pass/fail
determination. Students who pass the test areas that correspond to the NATEF core areas for program
certification will receive an ASE Student Recognition document suitable for framing as well as
attractive insignia.
This information was retrieved from the NATEF Website
(http://www.natef.org/end_of_program_test.cfm)
687288425
Submitted 6-16-08
Page 8 of 14
Factors that affect the collected data:
These tests must be administered during a specific time window as scheduled by NATEF.
How to interpret the data:
The tests provide an outside review of the student’s academic proficiency with the body of knowledge
necessary to be successful in the heavy-duty diesel maintenance and repair industry. In addition, a
passing score on the tests indicate the ability to read and interpret data relating to the specific area
relating to the test.
The rate of passing scores on the tests indicates the effectiveness of instruction and the student’s ability
to retain and apply the subject material.
The tests are proprietary in nature and cannot be included in this document. The data will be tracked as
a percentage of students achieving a passing grade for each area tested.
Criteria
Students are well
prepared to take the
standardized NATEF
exit exams.
Satisfactory (3)
90% of students taking the
NATEF exit exams receive
a passing score.
687288425
Needs Improvement (2)
80% of students taking the
NATEF exit exams receive
a passing score.
Submitted 6-16-08
Page 9 of 14
Not Satisfactory (1)
Less than 80% of students
taking the NATEF exit
exams receive a passing
score.
APPENDIX B: EMPLOYER INTERVIEW
Measure Description:
This measure consists of an interview by an either an instructor, a panel of students, or an employer in
the heavy-duty diesel repair industry. Students are interviewed and critiqued. The purpose of this
measure is to ensure that students learn the skills necessary to successfully interview for a position in
the heavy-duty diesel repair industry. A sample assessment form is shown below.
687288425
Submitted 6-16-08
Page 10 of 14
Factors that affect the collected data:
This assessment is made toward the end of the first course in the program. It is expected that the results
may not be as positive as a similar exercise, however the information gathered during this assessment
can be used to prepare students for interviews to achieve a practicum placement.
How to interpret the data:
Criteria
The student
demonstrates clear and
concise written skills in
preparation of resume
Satisfactory (3)
The resume is consistent in
formatting. Format
enhances the reader’s
ability to locate key
information. Grammar and
punctuation are correct.
Needs Improvement (2)
There are few problems in
format, grammar and
layout, but readability is not
dramatically affected.
Not Satisfactory (1)
The format, grammar or
layout interfere with the
reader’s ability to find key
information
The student projects a
professional
appearance,
appropriate for the
position for which
employment is sought.
The student projects a
confident, outgoing,
and positive attitude.
Grooming and hygiene are
excellent. Hair is worn in an
appropriate manner for the
shop environment. The
student is not wearing
excessive jewelry.
The student is on time for
the interview and projects a
level of confidence
consistent with the
applicant’s level of
expertise. The student
expresses a willingness to
learn and a positive attitude
toward employment in the
field and specific to the
employer.
The student clearly and
effectively articulates the
nature of the program in
which she/he is currently
enrolled, goals for
achieving national
certifications, and the
importance of career-long
professional development
to success in the field.
Appropriate clothing is
worn but is wrinkled, soiled,
or excessively worn.
Personal hygiene is
satisfactory.
Clothing is inappropriate for
the work environment. Poor
hygiene or grooming
creates a distraction. .
The student is positive but
does not project
enthusiasm for the career
field or the specific
employer. The interviewer
is required to work at
getting information from the
applicant.
The student projects a
negative attitude toward the
industry, the employer, or
toward work in general.
The student is late for the
interview.
The student is able to
articulate basic information
regarding the AAS program
and expresses a desire to
become ASE certified.
The student is not
knowledgeable about the
program in which he/she is
enrolled.
Responses to questions
indicate good listening
skills. Reponses are clear
and concise. Student
makes eye contact with
interviewer. Communication
indicates enthusiasm for
the position.
Responses are respectful
and on subject but may not
be complete. Listening
skills may not be evident
through applicant’s
answers. Communication
does not indicate
enthusiasm.
The student’s lack of
communication skills is
distracting to the
interviewer.
The student
demonstrates a
knowledge of the
importance of
education and
professional
certification for
success in the heavyduty diesel industry,
The student
communicates in a
clear and concise
manner.
687288425
Submitted 6-16-08
Page 11 of 14
APPENDIX C: ASE EXAMS
Measure Description:
The National Institute for Automotive Service Excellence (ASE) is the national standard for the
automotive industry. All major automobile manufacturers support ASE, and certification through ASE
is a prerequisite for certification for warranty reimbursement qualifications. This measure consists of
an annual query to ASE to determine the number of graduates that pass ASE certification tests.
Factors that affect the collected data:
ASE will not report on the number of our students that take the tests. They will only report on which
graduate have passed the tests. Therefore, no conclusion can be drawn regarding a ratio of our students
that pass or fail the tests. However, since our program supports ASE certification, the number of
graduates achieving certification is a valuable measure.
How to interpret the data:
Criteria
Students are well
prepared to take ASE
certification tests
Satisfactory (3)
70% of graduates are
successful in achieving
ASE certification.
687288425
Needs Improvement (2)
60% of graduates are
successful in achieving
ASE certification.
Submitted 6-16-08
Page 12 of 14
Not Satisfactory (1)
Less than 60% of
graduates are successful
in achieving ASE
certification.
APPENDIX D: PRACTICUM REPORTS
Measure Description:
Students in the Heavy-duty diesel Technology programs are required to complete a minimum of 450
hours of practicum. This measure consists of information gathered by faculty through interviews with
students and with mentors or supervisors.
The practicum mentor or supervisor should complete this form.
This matrix is rated using numbers 1 through 3. 3 = satisfactory, 2 = needs improvement, and
1 = needs improvement.
Question#
TASK
1
The student technician is at work as scheduled.
2
The student technician demonstrates safety in the workplace
3
The student technician demonstrates a positive attitude in the
workplace.
4
The student technician demonstrates skills and knowledge in
technical areas consistent with the level expected.
5
The student technician is able to gather and interpret data to
diagnose complex systems.
6
The student technician demonstrates knowledge of how
various systems and components interact with each other in
diagnosis and repair.
7
The student technician demonstrates an ability and desire to
apply skills to new tasks.
8
The student technician demonstrates proper concern for
quality workmanship.
9
The student technician communicates with supervisor(s) coworkers and customers in a professional manner.
10
The student technician uses clear and concise writing to
record service history.
11
The student technician demonstrates proper preparation for
the practicum experience.
687288425
Submitted 6-16-08
Page 13 of 14
1-3
The practicum student should complete this form.
This matrix is rated using numbers 1 through 3. 3 = satisfactory, 2 = needs
improvement, and 1 = needs improvement.
Question#
1
2
3
4
5
6
7
8
9
10
11
12
TASK
A Journeyman mentor has been assigned to me.
Mentor answers my questions.
Mentor appears genuinely interested in my success.
I am encouraged to ask questions.
I am accepted at the workplace.
Assigned tasks are at my skill level
Quality workmanship is stressed by mentor and management
I have met with m supervisor to set goals for my employment
Workplace safety is stressed by mentor and management
I am treated fairly at this facility
I am prepared for this opportunity
I am proud to be sponsored by this repair facility
1-3
Factors that affect the collected data:
While the interview questions are designed to gather information regarding specific areas of the
practicum experience, some of the data will be inherently subjective. Comparisons between student
and mentor/supervisor interviews will mitigate the subjective nature of the data gathered.
How to interpret the data:
Criteria
Practicum students
are well prepared to
perform in a heavyduty diesel
technology workplace
environment.
Practicum placements
are conducive to
student learning.
Communication
results in agreement
between practicum
students and
supervisors regarding
the mutual value of
the practicum
experience
Satisfactory (3)
Supervisor and mentor
ratings for practicum
student average 2.5 or
higher on the 1-3 scale.
Needs Improvement (2)
Supervisor and mentor
ratings for practicum
student average 2 or
higher on the 1-3 scale
Not Satisfactory (1)
Supervisor and mentor
ratings for practicum
student average less than
2 on the 1-3 scale.
Student ratings for
practicum placement
average 2.5 or higher on
the 1-3 scale
Variation between student
and supervisor/mentor
ratings are less than .5.
Student ratings for
practicum placement
average 2 or higher on the
1-3 scale
Variation between student
and supervisor/mentor
ratings are less than or
equal to1.0.
Student ratings for
practicum placement
average less than 2 on
the 1-3 scale
Variation between student
and supervisor/mentor
ratings are greater than
1.0.
687288425
Submitted 6-16-08
Page 14 of 14