...-1011:
-3
,
---
System wide
Quality Assurance
,11111111r
AvAM
AIMECA
MDalailp
- Continuous Incremental
Improvement
VICTORIA
Quality
Assurance
.
r1
\\\ \\ \\\\\\\\\\\\\\\N\\\N
:S\\
`...,s
.
.
,
ri
COURSE MAITENANCE'PROCESSES
.:. USER'S. GUIDÉ',:,
FOREwOAD
rjt:#
i'
: 's
1 have pleatttire in endorsing the Course Maintenance Processes (CMP) User's
Guide and recommending its use.
+.
'
The User's guide has beep de ëioped as an outcome of research and extensive
:,...
i
consultation ii ithin the Sfalè Training System. It is based upon State Training
ti-,--; Board Currie*.{
.;POtcX,f, `existing Best Practices and the principles of Total r.
...as Quality IVI navel), I-.
.
.
k
J
)
The User 's aide has been "designed to assist those responsible for the `
': developmen implementation and evaluation of curriculum, to achieve one of -,Jo'4.,toe key principles for recognised vocational education and training: "Curriculum
should in'Cle#40 .appropriate evaluation and validation processes to ensure the
,4;-..,.:V
r-
:
..
,
.,..,,
quality ofiriiing."
c
t.8.-,o,,:
Those involved in these processes will find the CMP User's Guide easy to follow 4?:r
in develoin* local policy and in the planning, conducting, analysing and
reporting'Òf.Ourse maintenance processes.
`',.
To assist the iimplementatien of the CMP User's Guide, the Office of Training
i.,:...-t.:arit17: 'Further!: ,fortudaticii' is fundin
t 4:.1
fr
activities dt,4.
,
a number df'
.'pr
g
1963: ' [furthermore
it will monitoii.
i7, -User's Guide:A felleitin
:to document Best Practice
lessional development
effectiveness of the
.
11
;
7 {,
A
TERRY M
ritor:IAERRY
rop; ipiRECTOF4c-7Tiri
_ 0 7FFICE
':r
f-
;
O
a:-;NdD FpRrit:
-ROBox266D_
)
4,
1,-,
;
COURSE MAINTENANCE
.S..SES;;
USER'S GUIDE
ACKNOWLEDGEMENTS
This project was funded by:
The State Training Board
Managed by:
The Melbourne College of Decoration
Project Reference Team:
Ross Gillespie, MCD
and up to July 1992.
Project Manager 1991
Ben Worsteling, MCD /RMIT
-
Project Officer and
Manager July 1992 onwards.
Sonnie Hopkins, Office of Training & Further
Education.
Sue Gaylor, Arts Training Victoria.
Maurie Hughes, Frankston College of TAFE.
Robert
Carmichael, Swinburne University of
Technology, TAFE Division.
This User's Guide was
developed by:
Ben Worsteling, Project Manager
Kathleen Alleaume -Ross, Project Officer
Cover Design:
RMIT Communications Services Unit
Word Processing:
Margaret Lyall, Lisieux Jones
We wish to acknowledge the assistance of the following people in the pilot of
the draft Guidelines.
Angelko Juresko
Graham Cole
Peter Rushbrook
John Kemp
Frank Tindall
Sam McCurdy
Pamela Florence
Judy Johnson
Roger Southern
Susan Dawe
Dianne Baxter
And all other staff members from
associated colleges and industry boards
having contributed to the pilot studies.
The User's Guide was developed and adapted according to
guidelines set by 'Designing Learning Guides for TAFE and
Industry' by Peter Bruhn and Hugh Guthrie.
COURSE MAINTENANCE PROCESSES
. ,
':;: GUIDE;
USER'S
;>:
CONTENTS
Acknowledgements
Page
Introduction
2
2
2
3
3
Purpose
Prerequisite Knowledge and Skills
Background
Resources
Intended Audiences
4
Total Quality Management
Exercise 1 Key Points
How to Use this User's Guide
Options in this Guide
5
7
10
11
Section A - Implementing Course Maintenance Processes
Developing Course Maintenance Policy
Model Course Advisory Committee
Developing Local CMP Policy
State Training Board Policy Framework
14
15
22
24
25
Section B Course Evaluation
Introduction to Course Evaluation
Informal Evaluations
Feedback Diary
Code of Ethics
The Monitoring Process
The Course Moderating Process
30
31
33
34
36
41
Conducting a Formal Evaluation
Course Maintenance Action Plan
Conducting the Evaluation
The Likert Scale
Reporting
Course Maintenance Reporting Structures
Communicating the Course Maintenance Processes
Proformas
Section C - Best Practices
Help us Improve this User's Guide
62
64
66
103
105
Definitions
106
Bibliography
110
44
45
49
50
61
Section: Introduction
Page 1
COURSE
URSE MAINTENANCE ; PROCESSES
USER'S GUIDE
INTRODUCTION
Purpose
The purpose of this guide is to:
1.
Provide an easy to use guide on the processes to be considered for course
maintenance.
2.
Assist various stakeholders in developing course maintenance policy and
implementing these statewide.
3.
Assist those responsible for delivering and maintaining statewide
accredited courses by providing a step by step approach to planning and
implementing course maintenance processes.
These guidelines are recommendatory only.
They propose that Designated Providers undertake a monitoring role in course
maintenance as part of their responsibility for curriculum management. They
further propose that providers accept the role of Responsible College for
particular courses, in which role they will report to the Designated Provider.
Notwithstanding, the use of the guidelines is not limited to such arrangements
being in place. Rather, they may be used by any course provider to assist it in
the process of ongoing course improvement.
Pre - requisite Knowledge and Skills
Policy Development
Knowledge of State Training Board Policies, Goals and Directions, and
Performance Agreements.
Course Ongoing Maintenance
Knowledge of Course Maintenance Policy and course requirements, and general
skills in planning, analysing and reporting.
Section: Introduction
Page 2
COURSE'' MAINTENANCE PROCESSES
USER'S GUIDE
Background
The Melbourne College of Decoration with funding from the Office of Training
and Further Education developed draft statewide course maintenance guidelines
in 1991.
These guidelines were based on "best practices" that included
associated TQM issues. Twenty four TAFE colleges and fourteen Designated
Providers /Network Managers provided information on policies, guidelines and
evaluation proformas. This project also included an extensive literature search,
workshops and further consultation with providers. The project was completed
in December 1991 and was distributed widely.
In 1992 the draft course maintenance guidelines were piloted and this guide
was produced for use by those responsible for the process. A total of 11
projects were submitted for this purpose and included evaluations of statewide
accredited courses, establishment of statewide course advisory committees and
course maintenance policy development for providers. Responses from these
pilots contained feedback essential to the review of the draft guidelines and the
development of this Course Maintenance Processes User's Guide.
Resources
This guide incorporates aspects of policies and course maintenance guidelines
listed in the bibliography relevant to course maintenance.
You will be required to refer to local policy documents such as:
.
.
.
.
Performance Agreements
Board of Studies Policies
Curriculum Development Policies
Course Accreditation Documents
"Getting it right the first time is like
hitting the nail on the head!"
Section: Introduction
Page 3
COURSE> MAINTENANCE PROCESSES
USER'S GUIDE
Intended Audiences
Policy - Development
State Training Board
Industry :Training ¡;:;::::;.
Board
' Designated Providers
: Network Managers
Course Delivery
TAFE Colleges
Private Providers
Course Advisory
Committees
Heads of Department
Advanced Skills Teachers
Course Co- ordinators
Teachers
Trainers
Diagram 1
Section: Introduction
Page 4
COURSE MAINTENANCE PROCESSES;
:: :. :.
USER'S G
GUIDE
U I DE;
.: ... ..
..
.
.
.
Total Quality Management
Evaluation
Implementation
Accreditation Period
A Total Quality Management approach involves many evaluation and change
implementation cycles taking place within the life of one accreditation cycle. At
different stages, different aspects of the course will be evaluated. eg.
A = Achievement
N = Demand
C = Content
D = Delivery
B = Assessment and Reporting
The concept of Total Quality
to course
maintenance. What is proposed in the 'Guidelines' is a systematic approach to
Management
is
fundamental
'small step' improvements rather than the 'big step' changes which have
occurred more often than not in response to reaccreditation requirements.
From the project to establish these guidelines what emerged as 'best practices'
were those course maintenance processes which incorporated aspects of Total
Quality Management. These were:
greater consultation and sharing of responsibility by all stakeholders
continuous feedback and good client relationships
ongoing process of course improvements
a reduction in the duplication of effort by sharing information and resources
greater control over the process rather than the output
a process which is plan driven, not event driven
decision making based on facts and data.
Section: Introduction
Page 5
OURSE MAINTENANCE PROCESSES
USER'S GUIDE
McMahon and Carter, The Great Training Robbery, 1990, provide a very good
model for introducing a quality management approach to training. They
recommend that the entire training process should be underpinned by the
internal organisation, at provider level, which guarantees nothing is left to
chance, and that the match between identified training needs and outcomes be
achieved through the implementation of a quality assurance plan and quality
control tactics.
The key quality improvement questions for any activity are:
Who are my clients?
What are their needs and expectations?
What is my product or service?
Will the product or service meet or exceed the customer's expectations?
What is the process for providing the product or service?
What measure will monitor the performance?
What action will approve the process?
Section: Introduction
Page 6
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Exercise 1
Key Points
In order to begin focusing on the key purpose of this User's Guide, answer the
following questions.
1.
What is /are the course(s) you want to evaluate?
Course
2.
What is the purpose of the evaluation?
3.
Who is going to be involved?
4.
Who will be planning the evaluation?
5.
Who will be conducting the evaluation?
Provider (List all)
Section: Introduction
Page 7
COURSE MAINTENANCE PROCESSES
GUIDE
..USER'S
...............................
6.
Do you have a Course Maintenance Processes Policy?
If you do not have a Course Maintenance Processes Policy, you will need to use
Section A to develop a policy before you proceed.
"We he /p you get it right the first time."
Section: Introduction
Page 8
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
How to Use this Package
IT'S EASY
JUST TURN TO THE NEXT PAGE
AND BEGIN!!!
(It contains all you need)
"We wanted to make it easy for you."
Section: Introduction
Page 9
COURSE MAINTENANCE PROCESSES
USER'S GUIDE >
How to Use this User's Guide
Section A (Developing Course Maintenance Policy)
Those responsible for statewide monitoring and maintenance of courses.
Policy DQvelOf7fn
State Training Board
Industry Training
Board
Designated Providers
Network Managers
<!
Course Delivery
TAFE Colleges
Private Providers
Course Advisory
Committees
Heads of Department
Advanced Skills Teachers
Course Co- ordinators
Teachers
Trainers
Section B (Course Evaluation)
Those responsible for carrying out course
maintenance processes.
Options in This Guide
Diagram 1.1
Section: Introduction
Page 10
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Options in this Guide
This user's guide takes a professional development approach for the user with
learning outcomes which equip the user to undertake course maintenance
processes.
It is divided into three sections:
Section A (Developing Course Maintenance Policy)
Section B (Course Evaluation)
Section C (Best Practices).
On the successful completion of Section A, you will have:
1.
formulated local course maintenance policy;
2.
identified and developed lists of stakeholders;
3.
established best responding times;
4.
developed monitoring, moderating and reporting procedures that are
meaningful to the needs of your institution and
expectations based on principles provided in this guide.
stakeholders'
(Refer Diagram 1.1)
On the successful completion of Section B, you will have:
1.
planned an evaluation to suit your own particular needs;
2.
developed a proforma using guidelines provided in this guide that is
targeted at the selected audience and designed to produce a high return,
as specified in Outcomes;
3.
conducted an evaluation;
4.
analysed and reported on evaluation outcomes given a realistic timeframe
based on principles provided in this guide.
On the successful completion of Section C, you will have:
1.
provided feedback on the usefulness of the User's Guide;
2.
provided (where possible) examples of Best Practices.
Section: Introduction
Page 11
OURSE: MAINTENANCE;. PROCESSES
GUIDE
.USER'S
...............................
In line with competency based training methodology, this guide states as its
learning outcomes that on completion of using this guide, you'll be able to:
1.
Develop a course maintenance policy. (Section A)
2.
Plan a course maintenance process for the accreditation period of a
statewide accredited course. (Section B)
3.
Conduct a formal evaluation
4.
Establish reporting mechanisms to inform students and stakeholders.
.
(Section B)
(Section B)
Section: Introduction
Page 12
z
0
SECTION A
4
Page
Developing Course
Maintenance Policy
15 -21
Model Course
Advisory Committee
22 23
Developing Local CMP Policy
STB Policy Framework
24
25 28
COURSE MAINTENANCE PROCESSES
USER'S, GUIDE,
....
.
.
....
...
.........
.
..
IMPLEMENTING COURSE MAINTENANCE
PROCESSES
SECTION A
On successful completion of Section A, you will have:
1.
formulated local course maintenance policy
2.
identified and developed lists of stakeholders
3.
established best responding times
4.
developed monitoring, moderating and reporting procedures that are
meaningful to the needs of your institution and stakeholders expectations
based on principles provided in this guide.
(Refer Diagram 1.1)
Section A: Implementing Course Maintenance Processes
Page 14
COURSE MAINTENANCE ; PROCESSES
... ... ..
USER'S GUIDE
.
..
.
Developing Course Maintenance Policy
Who should use this section:
* Those responsible for developing
Course Maintenance Policy
according to guidelines as recommended by this User's Guide.
a
Local
The purpose of developing your own policy is to provide you with a framework
tailor -made to meet your organisational needs.
The following guidelines should provide the basis for your policy.
By establishing a course maintenance policy based on these guidelines, and in
accordance with STB policy framework, you will enable course evaluation to
take place without the 'red tape'!
Section A: Implementing Course Maintenance Processes
Page 15
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Components
The Course Maintenance Process should seek for continuous improvements. It
should regularly evaluate curriculum in the areas of:
1.
2.
3.
4.
5.
6.
7.
8.
Content (relevance and quality)
Delivery (quality)
Achievement (progress towards achieving objectives)
Resources (extent and quality)
Assessment and reporting (relevance, accuracy and fairness)
Student and industry demand /support for the course
Suitability of the course for student employment
Enrolment and successful participation rates of all client groups.
Note: See notes on pp. 24 & 26 Curriculum Maintenance Components for
explanation.
Accreditation
The Course Maintenance Process should be specified in the course accreditation
document.
Phases
Course maintenance should be organised in three phases:
1.
Course Maintenance Plan
clarification of what is to be evaluated
identification of persons involved, committees etc.
review of available and current information
design of study, selection of methodology and action plan
2.
Course Maintenance Process
Gathering of facts and data using a combination of:
surveys of students, teachers, industry, and others using the relevant
proformas A -G
coordinated group discussions with students, teachers and industry
representatives
course moderation panels
registers of proposed course changes completed by teachers
Section A: Implementing Course Maintenance Processes
Page 16
COURSE MAINTENANCE :PROCESSES.
USER'S ;GUIDE
student profile data from student records
pre course and post course assessment of students
analysis of statistical records of applications, enrolment, completion
rates, career and employment paths
analysis and sorting of facts and data.
3.
Course Maintenance Reporting and Action
drawing conclusions
preparing recommendations
acting upon the recommendations for maintenance of course quality.
Quality Issues
The Course Maintenance Process should incorporate the principles of Total
Quality Management which would ensure:
greater consultation and sharing of responsibility by all stake holders
ongoing process of course improvements
a reduction in the duplication of effort by sharing information and
resources
continuous feedback and good client relationship
a process which is plan driven, not event driven
decision making based on facts and data gained from the use of evaluation
proformas
greater control over the process rather than the output
implementation of recommendations.
.
Consultation
The Course Maintenance Process should be thoroughly consultative.
Consultation should involve all relevant parties who have a stake in the course
eg. students, teachers, providers, industry, government, unions.
Section A: Implementing Course Maintenance Processes
Page 17
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
The guidelines assume that Designated Providers accept responsibility for
monitoring statewide accredited courses, and in accordance with the National
Framework for the Recognition of Training, report on the effectiveness of
course maintenance implementation to the relevant Industry Training Board(s)
and the Office of Training and Further Education.
The process for consultation will vary according to the nature of the course, the
industry (location and size) and should include a range of appropriate monitoring
methods. Regular scheduled meetings of:
course advisory committees
industry working groups
teacher networks
On -going industry visits and liaison.
Scheduled industry /provider moderation of students' work through:
display and /or exhibition of student work
special forums, seminars and conferences
visits
interviews
work review panels
verification processes
Various survey methods including:
questionnaires
observation
interviews
Related Policies
The Course Maintenance Process should be responsive and be consistent with
other government policy and practices re:
Accreditation
Curriculum
Code of Research Practice
Industrial Democracy
Social Justice
Recognition of Prior Learning
Articulation
Section A: Implementing Course Maintenance Processes
Page 18
..
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
The following components should be selected for evaluation and course
maintenance as a minimum:
Content
The "content" of curriculum will mean appropriateness and quality of:
all curriculum documentation (syllabus)
the course design
aims and objectives
Delivery
The "delivery" of a program will mean:
the mode of delivery - eg. self paced, distance learning, problem centred,
inquiry, practical /theory
timing of delivery - eg. part -time, full -time, evening, day, sequencing of
components, block release and fleximode
the effectiveness of teaching support material including technology
employed
the effectiveness of teaching and learning strategies used
Achievement
Student achievement will refer specifically to:
gains in acquiring the skills, knowledge and attitudes as stated in the
learning outcomes of the course;
the capacity to demonstrate application of the acquired skills, knowledge
and attitudes on the job.
Note 1:
Provision ought to be made for recognition of achievements
arising from the course of study which were not pre - planned.
Note 2:
The measurement of "distance travelled" or specific learning
attributable to the course of study can best be measured
after assessment of skills, knowledge and attitudes prior to
course commencement.
Provision for students to gain
credits and exemptions through recognition of prior learning
is a measure of the quality of course maintenance.
Section A: Implementing Course Maintenance Processes
Page 19
::::;COURSE MAINTENANCE: PROCESSES
USER'S .GUIDE
Resources
The resourcing of a course will relate to the appropriateness, extent and quality
of:
teaching staff
physical facilities (rooms, workshops, studios, libraries, laboratories, live
work areas)
mechanical equipment, technology, tools and consumable materials.
Assessment and Reporting
Assessment is the appraisal of student performance on a continuous basis,
measuring achievements in skills, knowledge and attitudes and making
judgements about these. Reporting follows assessment and informs students,
employers, and others of the progress and specific achievements and
competences gained by students. Both assessment and reporting can occur on
a regular informal basis (formative) or can occur more formally summarising
final achievement at the end of a course of study (summative).
Evaluation of the assessment and reporting methods, . processes and outcomes
should be undertaken in relation to their:
relevance
accuracy
fairness
Student and Industry Demand /Support for Course
Demand /support will relate to :
recording and monitoring of unmet demand
enrolments in continuing courses
evidence from advisory committee meetings, surveys, interviews and
industry visits
training needs analysis
Section A: Implementing Course Maintenance Processes
Page 20
DURSE MAINTENANCE PROCESSES:
USER'S GUIDE:
Suitability of the Course for Student Employment
Course suitability to equip students for employment will relate to:
post graduate surveys
advisory committee advice
responsiveness of the course to shifts in skills demand.
Enrolment and Successful Participation Rates of all Client Groups
Enrolment and successful participation by client groups relates to :
recognition of prior learning implementation
actual enrolment records
measures of successful participation as measured by completion and /or
student, employer assessment and judgement.
Section A: Implementing Course Maintenance Processes
Page 21
COURSE: M AINTENANCE` PROCESSES
.
.
USER'S GUIDE
MODEL COURSE ADVISORY COMMITTEE
Membership
Members' representation shall be for the contribution they can make because of
their knowledge and expertise in the area.
Committee size is determined by the scope of its stakeholders but there should
be a minimum of 12 persons.
Wherever possible the committee should aim for a gender balance.
At least 60% of the membership should be directly from the relevant industry.
To ensure a strong link with the Industry Training Board, at least one member
should be a member or nominated person of the Board where there is a relevant
ITB.
The chairperson should be from industry.
An executive officer /secretary should be from one of the providers.
Wherever possible decision making should be by consensus.
An Advisory Committee may co -opt additional expertise to assist with
a
particular task, usually via ad hoc working parties.
General
Course Advisory Committees:
Are established to provide advice to the Responsible College and
Designated Provider on course development and maintenance.
Provide a forum for discussion by members of industry, providers and
other groups where appropriate i.e. education sectors, community and
government bodies.
May establish ad hoc working groups for a specific time to undertake
particular tasks as required.
Section A: Implementing Course Maintenance Processes
Page 22
'COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Functions
To receive, consider and make recommendations to the Responsible College and
Designated Provider based on information from provider evaluation reports.
(Evaluation Proformas F & G).
To ensure that course content keeps abreast of changes in industry.
To provide advice on equipment and materials currently used in industry.
To assist with course moderation to ensure a consistency of learning outcomes
across all providers.
To provide advice on and, if
possible, assist
with student work place
experience.
Role of Designated Provider and Responsible College
1.
The role of the Designated Provider should include the facilitation of
accreditation /reaccreditation and course maintenance.
2.
In the case of course evaluation, the Designated Provider should forward
to the Responsible College the instructions /information on the evaluation
In turn, the Responsible College should return the
response /action to the Designated Provider. The Designated Provider
should present this information in its annual report to the State Training
to be conducted.
Board.
3.
In instances of minor change to an accredited course, the Responsible
College should seek the endorsement of the Designated Provider.
4.
Resource issues may be referred by the Designated Provider to the Office
of Training and Further Education.
5.
The Designated Provider should manage the course maintenance process.
6.
Adequate reporting and feedback should take place between the
Responsible Colleges and Designated Providers.
Section A: Implementing Course Maintenance Processes
Page 23
COURSE MAINTENANCE: PROCESSES
USER'S GUIDE
Develop Local CMP Policy
Your Tasks:
Develop policy and procedures which reflect
State Training Board and College organisational
statements.
Develop course maintenance procedures that are
realistic and achievable.
Conduct information sessions to raise awareness
amongst staff of the importance of course
maintenance processes.
Section A: Implementing Course Maintenance Processes
Page 24
COURSE MAINTENANCE PROCESSES:
USER'S GUIDE
STATE TRAINING BOARD POLICY FRAMEWORK
The following recommendations are extracts from the Curriculum Policy for
Vocational Education and Training in the State Training System (July 1992) and
the Interim Guidelines for Accreditation.
The Interim Guidelines for Accreditation (August 1991) produced by the State
Training Board state that:
Principle 9, Ongoing Monitoring and Evaluation
"Vocational education and training courses submitted for accreditation must
identify monitoring and evaluation processes to be used."
A process for the monitoring and evaluation of courses, including delivery and
assessment modes, will identify how a course fulfils its stated objectives.
. ccor /ng y; : accreditation
submissions should:'
Outline the :established:
mechanism which .:.........;;
will monitor
:.;
student
student
t. performance, the
..:...: :::......: ..
appropriateness of the course,
and identify any required
amendments.::
....:.
Describe the manner in which
staff and -course participants
clpants:::.:
have::input:::into;:;the:;:course:::
standards and delivery.
;.
Your Tasks:
Outline your course maintenance plan for the
course in the accreditation submission.
Describe how you are going to involve staff
and participants in the course maintenance
process.
Section A: Implementing Course Maintenance Processes
Page 25
..
COURSE MAINTENANCE. PROCESSES
.
..
.
...
.
;..
USER'S:
`
I
Please note the following relevant extracts from the Curriculum Policy for
Vocational Education and Training in the State Training System:
National Context
"A curriculum policy for the State Training System is a critical document. It is
the framework for ensuring that vocational education and training is appropriate
for and responsive to:
the rapidly changing economic environment and the need to improve our
competitiveness in world markets;
the needs of industry, learners and the broader community;
consistency with developing national training structures and standards."
Role of Curriculum Policy
"The curriculum policy for the State Training System is a policy umbrella within
which:
key reforms can be implemented and emerging training issues resolved
resources needed for the research and development, delivery and
evaluation of recognised vocational education and training can be
identified and allocated
key stakeholders in the State Training System can develop, deliver,
evaluate and recognise vocational education and training."
Curriculum guidelines have been set "in order to assist implementation of the
policy. Best curriculum practices within the State Training System will be
documented. These best practices will be provided as a set of guidelines for
implementing the policy."
The development and delivery of recognised vocational education and training
encompasses five stages including:
"evaluation and
effectiveness"
validation
processes
to
assess
and
improve
training
Section A: Implementing Course Maintenance Processes
Page 26
COURSE MAINTENANCE
PROCESSES;
;.
USER'S
.
:
In regards to evaluation, the Curriculum Policy, states that:
"Curriculum should include appropriate evaluation and validation processes to
ensure the quality of training."
In regards to criteria for development and delivery, it states that curriculum
should be developed:
"with evaluation and validation mechanisms which ensure on -going relevance to
industry and the market place."
As a Criterion for Delivery, Curriculum should be delivered so as to:
"provide for a progressive review of the appropriateness of the curriculum."
In establishing the responsibilities of key stakeholders, Designated Providers and
Network Managers should:
"manage and develop curriculum for an industry or group of industries
in
accordance with the development criteria detailed in this Policy;"
"convene curriculum advisory groups including nominees from the relevant
Industry Training Board."
State Training System Providers including TAFE Colleges and Registered Private
Providers should "develop and deliver recognised training in accordance with the
delivery criteria detailed in this Policy."
Further, it states that curriculum based on the principles and criteria detailed in
this Policy will lead to recognised vocational education and training which:
"is redeveloped after thorough formal and /or informal evaluation by key
stakeholders."
Section A: Implementing Course Maintenance Processes
Page 27
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Your Tasks:
Establish advisory groups involving industry
representatives and including Industry Training
Board representatives to review the
appropriateness of the curriculum.
Maintain a record of 'best practices' in the
development and maintenance of curriculum for
your review.
Section A: Implementing Course Maintenance Processes
Page 28
COURSE MAINTENANCE PROCESSES
USER'S 'GUIDE
Section B: Course Evaluation
Page 29
e?¡
SECTION B
Page
Course Evaluation
30
Introduction to Course Evaluation
31 35
Informal Evaluations
Feedback Diary
Code of Ethics
aP
The Monitoring Process
36 43
The Moderating Process
Conducting a Formal Evaluation
Planning
Conducting (includes
Evaluation Proformas)
Analysing
Reporting
44
-
102
z
0
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
COURSE EVALUATION
SECTION B
On the successful completion of Section B, you will have:
1.
planned an evaluation to suit your own particular needs
2.
developed a proforma using guidelines provided in this guide that is
targeted at the selected audience and designed to produce a high return,
as specified in Outcomes
3.
conducted an evaluation
4.
analysed and reported on evaluation outcomes given a realistic timeframe
based on principles provided in this guide.
Section B: Course Evaluation
Page 30
COURSE: MAINTENANCE= PROCESS
USER'S GUIDE
Informal Evaluations
Informal evaluations take place in an unstructured, and unrecorded manner and
are almost invariably verbal.
Although not 'officially' conducted, informal evaluations provide an important
source of information from course participants and staff.
Whether or not informal evaluation takes place depends largely on how
receptive the teacher, Head Of Department and /or Board Of Studies is to
learning students' /stakeholders' points of view.
A cluster of students holding back a teacher late after class to complain about
their workload is 'informal evaluation'. This small group could be representative
of the whole class.
Holding regular feedback /review sessions is particularly useful to both teacher
and students.
Informal evaluations are rarely invited, and when they do they often place the
recipient of this information in a quandary; do you risk popularity with students
by doing nothing, or possibly inconvenience other colleagues by doing
something?
The solution to this problem is to approach informal evaluation from the point of
view of Total Quality Management, which encourages student feedback and a
positive attitude towards complaints.
Complaints are essentially student feedback that enables us to provide them
with more effective learning.
Informal evaluations have a dual purpose:(1)
They spontaneously bring to the teacher body's /provider's attention
course participant feedback that may have not been evident before the
complaint (there will be the occasional positive reinforcement).
(2)
They provide you with an opportunity to sound out teaching
effectiveness or student satisfaction by asking questions informally.
This section provides you with a feedback diary on page 33. Record all informal
evaluations which you feel will be of value to yourself or colleagues at a later
date.
Section B: Course Evaluation
Page 31
COURSE MAINTENANCE PROCESSES
USER'S
_...... GUIDE
...
.
Your Tasks:
Note informal feedback received and consider
conducting a formal evaluation at a later stage of
some of the variables mentioned.
Occasionally consult your students to seek their
comments - make notes on these too.
Section B: Course Evaluation,
Page 32
COURSE;' MAINTENANCE PROCESSES
USER'S GUIDE: >:
FEEDBACK DIARY
Date:
/
/93
Feedback Initiated by:-
Feedback:
Comments:
Recommendations:
Results:
Section B: Course Evaluation
Page 33
COURSE MAINTENANCE PROCESSES
USER'S
GUIDE
................
...............................
CODE OF ETHICS
In planning, conducting, analysing and reporting your evaluation, you will need
to take the following ethical guidelines, as defined by "A Code of Research
Practice" for TAFE into consideration.
Your evaluation should be conducted in a manner that is fair, accurate
and relevant. Take care to reduce bias in the planning, conducting and
reporting stages of your evaluation.
The welfare of participants must take precedence over everything.
When the participant (student /industry representative /staff member) has
been guaranteed anonymity, that person's identity should not be
associated with the information given, which should be held as
confidential.
If a participant chooses to withdraw at any point in the evaluation, his /her
rights must be respected.
Ascertain that the true nature of the evaluation being undertaken is
revealed to the participants.
The reporting process must be based on the data collected during the
evaluation.
recorded.
Both the positive and the negative findings should be
Where funding has been provided full accountability must be provided.
Dissemination according to the terms of the contract is carried out to the
specified audience.
It is essential that mechanisms
recommendations are implemented.
are put into place to ensure that
Section B: Course Evaluation
Page 34
COURSE° MAINTENANCE PROCESSES,:
USER'S GUIDE :
Your Tasks:
If developing CMP Policy
Include the content of this section in your
policy and promote its adoption.
If conducting an evaluation
Frequently refer to this section as a guide
when planning, conducting or reporting on
an evaluation, to ensure you are acting
accordingly.
Section B: Course Evaluation
Page 35
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
THE MONITORING PROCESS
The State Training Board's "Curriculum Policy for Vocational Education and
Training in the State Training System" defines monitoring or verification as:
"...the process of quality assurance involving internal, local and /or external
validation of the integrity of the training system ".
It goes on to add:
"...it should not be confused with assessment ".
Monitoring differs from an evaluation in so much as it does not seek out new
information, but measures data
effectiveness /student performance.
already
available
to
evaluate
course
Evidence from the existing policies and guidelines indicates that most course
maintenance processes have been contained and confined to 'Course
Maintenance Procedures' as stated in course accreditation documents and
comprising continuous monitoring of the course involving students, teachers
and industry personnel. It is proposed in this document that a mechanism be
established for the continuous monitoring of the course.
COURSE MONITORING
This section helps you set course monitoring in action. In order to monitor the
effectiveness of the course and the current level of student satisfaction:
1.
Performance Indicators are set.
2.
Information enabling us to measure the effectiveness of the indicators as
identified is determined.
Section B: Course Evaluation
Page 36
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
The first column of the table below lists relevant Performance Indicators, as obtained from a client follow -up survey designed by National Centre for
Vocational Education and Research. The second column lists sources of information that may enable you to measure the effectiveness of the relevant
indicator. The third is for your personal use. Where the suggested resources are deemed inadequate, you should list your preferred approaches in column
3.
PERFORMANCE INDICATOR
SUGGESTED SOURCES FOR DATA
Teacher Performance (level of
satisfaction with teacher performance)
Informal feedback
Student surveys
Teacher Effectiveness (perception of
teacher /trainer knowledge and skills)
Informal feedback
Student surveys
High Quality Curriculum and Training
Materials (level of satisfaction with
curriculum and training materials)
Curriculum and training materials
Recognition of Prior Education,
Training and Experience (level of
satisfaction with skill recognition,
CBT, credit granted and articulation
provisions)
Board of Studies
Senior RPL Consultant
Outcomes of Training (employment
opportunities for exit students)
Previous surveys
Relevance of Training (level of
satisfaction with relevance of training
to employment and industry needs)
Work experience feedback
Industry feedback and donors
Outcomes of Training (opportunities
for disadvantaged and specifically targeted students)
EEO Officer
HOW CAN I FIND OUT?
Section B: Course Evaluation
Page 37
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
PERFORMANCE INDICATOR
SUGGESTED SOURCES FOR DATA
Student Support Services (level of
satisfaction with student support and
advisory services, particularly for
disadvantaged students and for course
and career information)
Student Records /Union
Student Financial Support (adequacy
of availability and level of income
support and fee relief)
Financial Services
Pathways to and through Vocational
Education and Training
Student surveys
Articulation arrangements
Implementation of Competency -based
Training
Accreditation document
Effect of Flexible /Self -paced Learning
Strategies in Vocational Education and
Training
List of available resources
Provision of Accredited Courses by
Non - government Providers (Private
Providers)
Information Fairs /Yellow pages
Registry of Private Providers
Profiles of Completing
Students/Trainees
Industry Associations /Boards
HOW CAN I FIND OUT?
Informed Choice
Unmet Demand
Course Applications
Section B: Course Evaluation
Page 38
COURSE;:: MAINTENANCE .PROCESSES
USER'S GUIDE
You will need to refer to this table again.
The decision to conduct a formal evaluation will need to be made as a result of:
a 'team' evaluation of the information already available
aspects of the course that need to be evaluated
whether or not the information already available is sufficient.
Section B: Course Evaluation
Page 39
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Your Tasks:
Complete the table in this section to begin
accessing relevant information.
As you compare and evaluate this information,
make notes and report those results to the
appropriate bodies.
If the information you have acquired is either
non- existent, or is insufficient in order to
measure the relevant performance indicators,
meet with your course evaluation team to
discuss a formal evaluation process.
Maintain a record of best practices in course
monitoring for each course monitored.
Section B: Course Evaluation
Page 40
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
THE COURSE MODERATING PROCESS
Course moderation ensures that courses offered system -wide are of a similar
standard. Moderating of the course must involve students, staff and industry
personnel.
Depending on the field of study and type of course, the moderating process
may vary.
Example 1
The Child Care Advisory and Moderating Committee has established "Guidelines
for Moderating Procedures ". These guidelines recommend that the Assistant
Executive Officer of the Child Care Advisory and Moderating Committee
(CCAMC) develops a schedule regarding the subjects to be moderated. The
Moderation Schedule is designed to:
"a.
ensure that all subjects within any given course will be moderated at
regular intervals;
b.
have related subjects in different child care courses moderated at the
same time."
* Guidelines for Moderation Procedures Child Care Courses
All colleges offering these courses are advised to submit information to
teachers' meetings.
Meetings are held every second month. Each meeting elects a Minute Secretary
and prepares a summary for the Advisory Committee. A Representative is
elected to collate relevant information and attend the subsequent meeting. A
summary of recommendations and comments are circulated to relevant
HODs /Course Managers by the Advisory Committee.
Example 2
The Finished Art Course is offered by a number of colleges statewide. Effective
delivery and maintenance of this course is shared by relevant Industry advisory
bodies and working groups. The levels of participation and operation of these
groups contributing to the success of the course make this course moderation a
good example of best practice in TAFE training.
Section B: Course Evaluation
Page 41
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
The aim of the Working Group is to have a constant and meaningful dialogue
between the education provider and the Graphic Arts Industry.
The infra structure of the course extends to four different levels, including an
Industry Advisory Committee (Level 1 - defined TAFE Advisory Committee role).
This group provides support and endorsement of the syllabus consistent with
the job profile for the Finished Artist.
Level 2 which consists of the Finished Art Industry Working Group acts as a
sub - committee to the Level 1 Working Group and is responsible for the
maintenance and operation of the course.
The Finished Art Handbook, 1991, states the following:
The responsibility for the ongoing evaluation and
validation lies with the teachers in conjunction with
the finished art working parties and the Industry
Consultative Group.
will participate in regular moderation
meetings each year to discuss and record changes
-providers
needed in course structure, content and teaching
methodologies and information obtained from surveys
on course participants and graduates.
- providers will conduct an annual exhibition of student
work which will be judged by industry representatives
who will make awards in recognition of student
achievement.
This exhibition provides the main basis for moderation
between colleges of student work.
From Curriculum Document
Associate Diploma Finished Art
December 1986
The infrastructure includes also a third level (Co- ordinators /Heads of Finished
Art, acting as executive for all teachers of finished art in those colleges offering
the program), and a fourth level, Finished Art, Teaching and Teaching Practice
and the implementation of the course to meet effective industry requirements
for training. This level includes all current students and graduate students of
the course since its inception in 1975.
Section B: Course Evaluation
Page 42
OURSE MAINTENANCE PROCESSES
USER'S GUIDE
Your Tasks:
Establish or maintain a course moderating
committee to ensure courses are moderated at
regular intervals.
Establish an appropriate procedure for course
moderation depending on the nature /delivery of
the course.
Establish your own committee, representatives,
schedule and reporting structure to maintain
participation.
Maintain a record of 'best practices' in the
procedures you have developed to moderate
each particular course.
Section B: Course Evaluation
Page 43
COURSE; MAINTENANCE PROCESSES
USER'S GUIDE
CONDUCTING A FORMAL EVALUATION
Planning
Course maintenance should be plan driven, not event driven.
Unless you have your course maintenance strategy planned every step of the
way, you are likely to begin reacting according to problems arising in each
stage, and will most likely lose your way.
"Plan ahead!"
To get somewhere safely, it is best to set your goal and keep it firmly in mind.
Your course maintenance plan should address four areas:
1.
clarification of what is to be evaluated (set your own Performance
Indicators)
2.
identification of persons involved
3.
review of available and current information (monitoring)
4.
design of study, selection of methodology action plan.
Your Action Plan must include:
Course Monitoring
Course Moderation
Informal Evaluation
Formal Evaluation
Section B: Course Evaluation
Page 44
COURSE MAINTENANCE PROCESSES
ER'S GUIDE
COURSE MAINTENANCE ACTION PLAN
You will need to refer to the following two documents:
1.
Your local Course Maintenance Process Policy.
2.
Exercise 1 'Key Points'.
3.
Exercise 2 Course Monitoring (Please complete that section before
proceeding).
Now go on to the following questions:
1.
What does this evaluation need to measure? How?
Performance Indicator
How?
(Include both our suggestions
and your own indicators)
(What source of information
will you use? Will you need a
formal evaluation ?)
Section B: Course Evaluation
Page 45
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
2.
Have you identified those who will be working on the evaluation?
To help you focus on distribution of tasks, fill out the following table:
Will be performed by:
TASKS:
1. Plan the Evaluation
1.
Monitor
2.
3.
4.
Select the Performance Indicators to be
evaluated
Gather the information available
Design a Formal Evaluation.
2. Conduct the Evaluation
If questionnaires are selected in methodology:
1.
2.
3.
4.
5.
6.
Design the proforma
Test the proforma
Administer the proforma
Collate the results
Analyse the results
Report.
Who will co- ordinate the functions
and train new team members on aims
and objectives of evaluation?
If interviews are selected in methodology:
1.
2.
3.
4.
5.
Design of interviews
Interviewing
Collating and coding results
Analysing results
Reporting results.
3. Analyse the Results of the Evaluation
Collate the results, all the different methods
used and draw some conclusions.
Who will provide you with word
processing and desktop publishing
support?
4. Report the Results of the Evaluation and suggest
recommendations to the appropriate bodies.
Section B: Course Evaluation
Page 46
COURSE MAINTENANCE PROCESSES:.
USER'S GUIDE
OTHER TASKS:
Certain tasks may emerge which fall outside of staff's allocated /usual
duties. It will help you to plan for these now. Identify who will perform
these tasks.
These tasks commonly include:
Will be performed by:
Briefing of interviewers
Designing Coding Lists
Analysing quantitative results
Coding quantitative responses
-
Clerical /support tasks
Printing of questionnaires
Mailing questionnaires
Receiving and collating questionnaires
Section B: Course Evaluation
Page 47
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Your Tasks:
Complete the Action Plan in this section.
Complete this User's Guide section on Course
Monitoring.
Meet with your evaluation team or assemble an
evaluation team to analyse the results of the
course monitoring process.
Do you wish to proceed to a formal evaluation?
Section B: Course Evaluation
Page 48
MAINTENANCE PROCESSES
USER'S GUID
CONDUCTING THE EVALUATION
The following learning activities will guide you through the steps necessary to
construct a Course Maintenance Evaluation Proforma that is targeted to your
selected audience, and is designed to produce a high return of responses.
Please follow the instructions in bold italics on the example proformas and use
suggestions made in standard type to help you select the most appropriate
quality course indicator.
Developing your own Proforma
The following proformas are a revised version of the proformas provided in the
guidelines issued in the Course Maintenance Process Report December 1991.
These proformas are not designed to meet your specific needs, so you must use
them only as a basis for developing your own.
Please follow the instructions in bold italics and use suggestions made in
standard type to help you select the most appropriate quality course indicator.
The proformas are available on computer diskette formatted in WordPerfect 5.1.
For further details contact:
Ben Worsteling
School of Design
RMIT
PO Box 2476
MELBOURNE VIC 300
Two points to remember when developing a proforma:
(1)
Avoid asking for any information (such as results, level of attendance)
here that can be obtained somewhere else. (Refer to exercise completed
in the section "The Monitoring Process ").
.
(2)
Do not ask a question from a respondent group whose answer is not
going to be relevant or appropriate.
Section B: Course Evaluation
Page 49
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
THE LIKERT SCALE
The Likert Scale is a five point attitude scale, that is recommended in this user's
guide for the following reasons.
1.
It is easy to use.
2.
It produces consistent results.
3.
It is simple to score.
4.
It makes the results of the evaluation easy to report.
5.
It is statistically reliable.
Variations on the five point scale are possible, for example, a four point scale
can be used instead of Likert, if you have reservations about the possibility of a
neutral score. Thought should be given to the fact that students will invariably
feel neutral about some aspects of the course. A four point scale does promote
a more definite choice and may be more attractive to course designer, but may
be in fact less effective in determining the exact feelings of the respondent.
Neutral, or additional alternatives can be added to the Likert Scale.
For
example:
Example 1
What is your overall rating of the training program?
Poor
Fair
Adequate
Very Good
Excellent
Section B: Course Evaluation
Page 50
MAINTENANCE PROCESSES.::
USER'S GUIDE>
Example 2
Please give me a rating for my performance in the following areas - circle one
number only.
Excellent
Adequate
Poor
1.
Knowledge of the subject
1
2
3
4
5
2.
Speaking clearly and intelligibly
1
2
3
4
5
Example 3
Please rate the following aspects of the course.
Presentation of Course
Organisation of Course
Learning Resources
Use the following key:
1
Very Good
2
Good
4
Poor
5
Very Poor
3
Borderline
If not relevant
to course
Additional response
option signifies lack of
relevance
8
The limitation of the Likert Scale in regards to Course Maintenance Processes is
that it fails to explain the reason behind the attitude. Whilst a student may
score a statement as 'fairly good', if there is no indication of why it may not
help you improve the course. It is suggested that an open ended question be
added wherever clarification is required by teaching staff.
Section B: Course Evaluation
Page 51
COURSE MAINTENANCE PROCESSE
USER'S; GUIDE::
Produce your own Likert Scale Options
It is not vital to the validity of the scale that you use the responses given in the
proformas for the Likert Scale.
If you do not feel satisfied with the responses we suggested, follow the
following guidelines to produce your own.
(1)
(2)
Create, select or compose a pool of statements. Make sure that your
statements are not neutral (they need something to agree or disagree
with), or at an end of the continuum.
If you have created your own scale, you may wish to test the scale on
approximately 100 respondents to see if the responses suggest agreement,
disagreement, strong agreement /disagreement or neutral feelings.
j
%
"It is statistically reliable and will not seek to destroy you.
Section B: Course Evaluation
Page 52
COURSE ;MAINTENANCE; PROCESSES
USER'S GUIDE,
Administering the Proforma
First of all, it is a good idea to number your questionnaires before sending them
out.
If the proforma is not administered under carefully thought out conditions and
timelines, it could impair your results and ruin the project.
example, be sure to make it as easy as possible for the
students /respondents to return the questionnaire, without extra cost to them for
For
example, include the cost of a self- stamped envelope when budgeting for
evaluation.
Distribute the surveys when it is likely that they will be able to answer the
questions. For example, there is no point in asking them questions about the
resources used in some sessions or program, if no such sessions have yet taken
place, or before they have had a chance to form an opinion and be assessed.
Make sure the survey is not distributed too late. The last day of the term, for
example, is not advisable as few students would want to be burdened with the
responsibility of returning a questionnaire.
It is not a good idea to hand deliver a survey to a student yourself, if the
questions or part of the questions pertain to your own teaching style. Students
will feel inhibited in their responses, or may think that their answers to the
survey could influence your assessment of their performance.
Surveys should be administered
module /unit /program being evaluated.
by
a
teacher
not
teaching
the
If you have distributed the survey to students at the college during the school
year /term, provide a box for respondents to deliver their responses.
Section B: Course Evaluation
Page 53
COURSE; MAINTENANCE PROCESSES
USER'S GUIDE
The next exercise helps you plan your survey distribution. We provide you with
a guideline to avoid the above pitfalls. You might want to consult your work
team if you are working with one.
1.
Distribution date:
(Guideline: Students - 2 weeks before end of term)
2.
Method:
(Guideline: Students - to complete at the end of the lesson)
Industry: Post - return self- addressed envelope
3.
Method of Recovery:
Students:
Teachers:
Industry:
(Guidelines:
Return through post, or post box on college premises)
Your Tasks:
Plan carefully as to when you are going to send
out your questionnaires to students, teachers and
industry.
Plan how students will return the questionnaire
(eg. post box, return envelope).
Section B: Course Evaluation
Page 54
COURSE: MAINTENANCE PROCESSES
USER'S GUIDE
Suggestions for Focusing Questions
Ask yourself...
1.
What is the purpose of this question in the evaluation (ie. what will happen
after it's done ?)
2.
Who will be interested in or use the results?
(eg. teachers, industry
contracts, BOS)
3.
What specific decisions will be made as a result?
4.
What will we know after the evaluation that we do not know now?
5.
What can we do after the evaluation that we cannot do now because of
lack of information?
(Taken from: A Manual for Training Facilitators, Curriculum Department, Box Hill
College of TAFE)
(-
Section B: Course Evaluation
Page 55
COURSE MAINTENANCE ;PROCE SSES
USER'S GUIDE.;
Face to Face Interviewing
A face to face interview should be planned similarly to a questionnaire.
The questions must be set out and read to the respondent exactly as they are
written, in the same order for each respondent.
When writing a face to face interview questionnaire, ensure that the questions
will make sense being read out, by testing them out loud with a colleague or
team member.
The questions can be designed in the same way as a questionnaire proforma
with the response coded in the same manner.
There are two ways you can conduct and code face to face interviews:
1.
2.
Write down the interviewee responses to each question word for word,
and check with him /her that responses have been accurately understood
and noted. Code later according to coding list.
or
Tape each interview with care as you go along. You might care to play the
tape back to the interviewee. Code the responses later according to your
coding list.
Select interviewers that have little to do with the course being evaluated.
The interviews must be conducted in as professional and uniform a manner
as possible if the results are to be reliable.
Your Tasks:
Design interview questions like you do a
questionnaire /proforma. Keep in mind that they will need
to be read aloud.
Decide whether it is better to write down the answers as
you go along or to tape and code later.
If you use standard questions using scales like the Likert
Scale, you will need a card showing the scale to your
interviewees.
Conduct face to face interviews in an objective
professional manner.
Section B: Course Evaluation
Page 56
COURSE MAINTENANCE PROCESSES
USER'S .GUIDE
Analysing
Now that you have received questionnaires containing these answers which you
are keen to make sense of, you will need to gather together a team of
information 'coders' and 'processors' to extract the information contained.
The results of the Likert Scale can be collated by counting the number of times
a response to a question was selected.
The example shown over the page was prepared by the Outer Eastern College
of TAFE.
Ensure that the total number of responses is always the same for each question
and equal to the number of respondents.
Once the results have been collated, percentages should be calculated and it is
recommended that the Course Advisory Committee analyse and act upon results
as they see fit.
Data collected and its significance would be relative to the
college /provider's own quality standards.
It is suggested that the results obtained by the Likert Scale, would act as a
'barometer' for client satisfaction, whereas the open ended questions would be
more pertinent to the needs of staff such as: evaluation of teaching,
effectiveness, module sequence, presentation.
What to do with open ended questions?
(1)
Upon receipt of each questionnaire carefully check all questions have been
answered and indicate receipt on list.
(2)
In order to code questionnaires you will need to develop classifications and
or coding list for each question by:
(a)
Selecting a sample (of approximately 20 surveys) of responses.
The aim is to access a set of examples of each type of respondent.
(b)
Typing out each responses on a separate page so that they are
easy to sort.
(c)
(d)
Start sorting them into appropriate groups.
If necessary take another sample, but continue to do so until a
stable set of categories emerges.
Section B: Course Evaluation
Page 57
COURSE MAINTENANCE PROCESSE
USER'S
............. ..... ... GUIDE
.....
.<.:
.
..
.
.
Table 1: A summary of responses from participants completing the module 133 (spreadsheet
and business graphics).
Module Content
(a)
To what extent did each of the following objectives in the module meet your needs?
Student Responses
3
4
5 Total
11
7
7
26
4
6 12
3
26
0
0
7 11
8
26
3
0
5
9
6
2
26
5
0
4
5
6
6
26
0
1
2
0
0
1
0
1
1.3 use all of the spreadsheet graph commands to convert
spreadsheet data into graphic screen and hardcopy displays
0
1.4 combine macro and graphical commands to execute
slideshow displays of data in graphical format
1.5 install and configure a spreadsheet package onto a
microcomputer
1.1 set up macro commands for execution within spreadsheets
1.2 design and implement a customised spreadsheet to meet a
business application need
.
Comments (What would you suggest to make the content more effective ?)
.
.
(b)
so far everything I have learnt has been excellent
don't cram so much into a single term
Program Presentation
Student Responses
0
1
2
3
4
5
Total
0
1
2 11
9
2
26
0
0
2
8
7
8
26
2.3 to what extent was the use of training aids effective
1
2
1
12
5
5
26
2.4 to what extent were you able to contribute during sessions
1
1
3
9
9
4
26
2.5 to what extent does the module reflect effective
preparation
0
1
6
7
7
4
26
2.6 to what extent was the material presentation effective
0
1
2 10
7
6
26
2.7 to what extent were examples and illustrations effectively
0
1
2
9 10
4
26
2.1 to what extent were the module objectives made clear at
the outset
2.2 where practicable, to what extent were the sessions
designed for active participation
used
Section B: Course Evaluation
Page 58
COURSE MAINTENANCE<:
PROCESSES
..:
.....
USER'S GUIDE:>
.
.
.
Categories must be mutually exclusive, only one category should be possible for
any give response.
Categories should be logical. Create a classification with a step wise structure,
starting with a few broad categories and progressively breaking them down into
finer, more detailed categories.
In some instances you may need to anticipate responses and their categories.
For example, to the question:
"Was the venue unsatisfactory in any way ?"
It is not unreasonable to anticipate that responses will pertain to the following:
-
Not enough light
Inadequate furniture
No parking
Lack of refreshment
Unsatisfactory equipment
Too cold /hot.
It is clear that the responses will only be in the negative
-
your reporting will
need to stress the different aspects of the venue.
Constructing the Coding Lists
The results that will make up a Coding List spell out the evaluator's frame of
reference. Thus, the Coding List needs to be constructed by someone very
familiar with the aims of the survey. It will be almost impossible to judge on
how meaningful a respondent's answer is unless you know what it is the
evaluation is trying to find out.
Once classifications have been established (using exercise outlined above) a
coding frame can then be constructed, see working example below.
Section B: Course Evaluation
Page 59
OURSE MAINTENANCE PROCESSES
USER'S GUIDE
Coding List
Question:
CLASSIFICATION
FREQUENCY COUNT
Analysing Data Collected from Open Ended Questions
Classification and their frequency should be converted to percentages and again
can only be interpreted by the Course Advisory Committee or staff acquainted
with the course.
Section B: Course Evaluation
Page 60
COURSE:
M.A. AIRITENANCE;
USER'S
PROCESSES
<;
REPORTING
Course maintenance reporting and action involves:
drawing conclusions
preparing recommendations
acting upon the recommendations for maintenance of course quality.
Conclusions drawn from data collected both quantitatively and qualitatively
should be reported in an unbiased way, supplying both positive and negative
findings.
Depending on whether the course is provided statewide, or solely by one
provider, conclusions and results need to be reported to at least one of the
following bodies:
Course Maintenance Committee (Provider Level)
Board Of Studies
Course Advisory Committee (if the course is taught by several providers)
Responsible College (conducting evaluation on behalf of Course Advisory
Committee).
The Designated Provider will need to be informed of the results of the
evaluation and will make appropriate recommendations.
A major change, as defined below, almost invariably requires reaccreditation and
will need to be reported to the Designated Provider.
A major change is defined as one which results in:
a change in the level of the award
a change to course aims or overall objectives
a provision for an additional occupational group or major stream of study
a substantial change in the course duration
a change which necessitates a change in course title
a change which takes the course outside VETAB's policy and guidelines
where a series of minor changes have resulted in what might reasonable be
judged to be a major change.
It is the responsibility of the Designated Provider to advise the ITAB and VETAB
of changes necessary to accreditation of the course.
Section B: Course Evaluation
Page 61
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
COURSE MAINTENANCE REPORTING STRUCTURES
COURSE ADVISORY
COMMITTEE
Monitors courses
RESPONSIBLE COLLEGE
PROVIDERS
Heads of Department
<=>
Collates data from
Providers
<=>
Reviews course
dY
INDUSTRY
TRAINING BOARD
Provides overall
advice on training
needs
MANAGEMENT
CONSORTIUM
DESIGNATED PROVIDER
<=>
Authorises minor
changes and notifies
providers
<=>
Provides advice
u
OTFE
Deals with issues as
appropriate
NETWORK
MANAGER
Authorises minor
change and notifies
providers
ITAB
Accredits courses at
Industry Level and
updates course
database
VETAB
(Policy Body)
Accredits courses
for which there is no
ITAB
Diagram 2
Section B: Course Evaluation
Page 62
URSE MAINTENANCE PROCESSE
USER'S GUIDE
NOTES:
1.
Responsible College is a member
nominated to manage the project.
of the Management Consortium
2.
Management Consortium supports the Designated Provider in its role and
responsibilities. It therefore makes recommendations to the Designated
Provider.
Your Tasks:
Based on the example given in
Diagram 2 develop a similar
reporting structure adapted to
your organisation.
Report results of evaluation in
an unbiased way to the
'appropriate stakeholders /body.
Inform the Designated Provider
of minor changes.
Report major changes to the
Designated Provider.
Maintain a record of practices in
the reporting of course
evaluation results for each
course evaluated.
Section B: Course Evaluation
Page 63
COURSE MAINTENANCE PROCESSES
USER 'S GUIDE.
COMMUNICATING THE COURSE MAINTENANCE PROCESS
Before finalising your proformas, you will need to write an Introduction.
The Introduction should:
1.
2.
3.
explain the purpose of the evaluation
request proposed respondent participation
emphasise the importance of proposed respondent contribution.
The purpose of an introduction is to explain the relevance of the information
gathering to the respondent and to enlist their co- operation in a meaningful way.
Never 'entice' the respondent into responding against their will, or make false
promises to encourage their participation. (See section on Code of Ethics).
Remember that course maintenance incorporates the principles of Total Quality
Management. We therefore need to ensure that:
there is considerable consultation and sharing of responsibility by all
stakeholders
there is continuous feedback and good client relationship
decisions are made about the course based on facts and the data gained
from the use of evaluation surveys rather than guesses
the survey also includes requests for suggestions and comments as to how
the program can be improved
the recommendations made from the surveys are implemented as soon as
possible and related back to survey participants.
The course maintenance process should be, therefore, thoroughly consultative.
You will note that all of the above require communication.
It is important that we encourage course participants, teachers and stakeholders
to take their share of responsibility in the course maintenance process.
Section B: Course Evaluation
Page 64
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
Your Tasks:Tell those completing the course evaluation
proformas why they are doing it, for whom,
and what will be done as a consequence of it.
Maintain a good client relationship with your
students by ensuring there is continuous
feedback.
Make decisions about the courses based on
facets communicated by students or
stakeholders themselves.
Report the results of the surveys, and
changes made consequently back to survey
participants immediately.
Inform the industry stakeholders involved in
course maintenance as to what changes have
been made and why.
"Tell absolutely everybody,
and do it systematically."
Section B: Course Evaluation
Page 65
BUILDING
PROFORMA
SESSION EVALUATION - STUDENTS
Building A Student Course Evaluation Proforma
A
Identify
Course /Unit /Module
Insert: Introduction
Participant's Name:
Optional but your responses will be held confidential.
Please rate the sessions using a Likert rating scale from 1 to 5 for all the
dimensions as described below.
You may wish to include /exclude questions that better reflect your own
organisational quality standards.
(a)
The learning outcomes
were not clearly stated
at the beginning
(b)
Failed to meet my
expectations of the
1
2
3
4
5
The learning
outcomes were
clearly stated at
the beginning
1
2
3
4
5
Met my
expectations
1
2
3
4
5
Informative, I
learnt a lot about
the topic
1
2
3
4
5
Provided ample
session
(c)
Uninformative about
topic /skills being learnt
(d)
Provided little or no
interaction
(e)
Had activities
inappropriate to learning
outcome
(f)
opportunity for
interaction
1
2
3
4
5
Had appropriate
activities
Disorganised
1
2
3
4
5
Well organised
(g)
Wasted much time
1
2
3
4
5
Made efficient
use of time
(h)
Not effective in helping
me achieve the industry
specified competency
1
2
3
4
5
Very effective, I
feel clear about
the industry
specified
competency to
be achieved
_.
Page 66
SESSION TITLE
a
b
Ramsden Dodds' A Guide to
c
d
e
f
Evaluation (1989)
g
h
COMMENTS
suggests that you always
include a question regarding teaching style at the end of the questionnaire, such
as "What piece of advice could you give me /the lecturer that might improve
his /my /her lecturing /delivery ?"
Page 67
BUILDING
PROFORMA
PARTICIPANTS COMPLETING A
PROGRAM /MODULE /UNIT
B
College /Institution:
Program /Module /Unit:
INSTRUCTIONS
Does your organisation still refer to performance objectives?
Insert: Introduction
This questionnaire uses the Likert rating scale. Responses range from 1
all well, to 5 - extremely well.
not at
Consider each response carefully and please place a circle around the number
which best reflects your response to each question.
Likert Rating Scale
1.
1
2
3
4
5
Not At
All Well
Not Very
Well
Fairly
Well
Very
Well
Extremely
Well
COURSE CONTENT
To what extent did each of the following units and learning outcomes /elements of
competence in the program meet your needs.
Unit of Competence:
Elements: (List)
1
2
3
4
5
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
1.1
1.2
1.3
1.4
1.5
1.6
1
1
1
1
1
What can you suggest that would make the content of the course more effective in you
acquiring the right skills?
Page 68
To each of the following items, make deletions /alterations to suit your own program /course.
2.
PROGRAM PRESENTATION
2.1
To what extent were the
2.2
Where practicable, to what extent were the
sessions designed for active participation?
To what extent was use of resources
effective?
To what extent were you able to contribute
during sessions?
To what extent did the program reflect
effective preparation on the part of the
2.3
2.4
2.5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
performance /learning objectives made clear
at the outset?
teachers?
2.6
2.7
To what extent was the way in which the
whole course was presented effective?
To what extent were examples and
illustrations effectively used?
1.
What modifications, if any, would you suggest to make the presentation more
2.
Teacher's performance
3.
Time course was held
4.
Course venue
effective in the areas of representation of course materials?
If you are physically disabled or requiring special facilities, did you find the venue satisfactory?
3.
PROGRAM ORGANISATION
3.1
To what extent did the physical aspects of
the venue help with the effectiveness of the
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
course.
3.2
To what extent did the venue surroundings
meet your needs eg. partitions,
refreshments /commodities?
3.3
3.4
To what extent did the catering
arrangements meet your needs.
To what extent was the training program
well organised?
What improvements, if any, would you suggest to improve the actual organisation of
the program?
Page 69
4.
PROGRAM EFFECTIVENESS
4.1
To what extent have you acquired the
performance /learning objectives intended in
the course?
4.2
To what extent are you confident about
applying the competence acquired from the
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
program on- the -job?
4.3
To what extent has the program met your
needs?
4.4
What were your needs?
Please
list up to three recommendations you would make to improve the overall
effectiveness of the program.
5.
PROGRAM RATING
What is your overall rating of the training program?
Poor
6.
Fair
n
Adequate
Very Good
Excellent
STUDENT SUPPORT SERVICES
To what extent were you satisfied with the student
support services provided (example: canteens,
1
2
3
4
5
Health Service)?
Page 70
BUILDING
PROFORMA
GRADUATES
C
College /Institution:
Course /Program:
DATE:
Insert Introduction
1.
(a)
Current Employer:
(b)
Current Position:
(e.g. level of seniority)
(c)
Year of Entry to Course (Certificate):
(d)
Employer at Time of Entry:
(e)
Position at Time of Entry:
(Your response will be held confidential)
2. From the units listed below and conducted during 19 , indicate the modules attempted and
for each unit rate according to:
(i)
(ii)
its relevance to your current employment
its interest level
3. Were you clear about what training the course is designed to provide?
Yes
No
Unsure
Explain
4. Did the course provide you the level of training that you expected?
Yes
No
Unsure
Comments:
Page 71
Tick one for each
unit attempted
Unit Attempted
Tick one for each
unit attempted
Relevance
High
Moderate
Interest
Low
High
Moderate
Low
Page 72
5. We attached a list of the units offered at the tin a you were studying for your course. Were
the units that you remember here of low interest? Indicate the unit and give a brief review
for your response.
(Use reverse side if space insufficient)
6.
(a)
In your opinion, does your employer give 'adequate' recognition to the course?
Yes
(b)
No
If yes, this is demonstrated by:
(tick one or more)
(i)
Time release
(ii)
Salary increase on completion of course
(iii)
Refund of course /unit fees on satisfactory completion or progress
(iv)
Inclusion of course in career /promotion structure
(v)
Other
E
E
E
Page 73
(c) If no, list what recognition (if any) is given.
(d) Why did you undertake the course?
7.
Would you be interested in receiving information on further advanced study and articulation
arrangements?
Yes
8.
No
n
Please rank (5 highest - 1 lowest)
Strengths of course are:
(i)
Social interaction with other students
(ii)
Introduction to new technologies
and new information
(iii) Personal /professional development
CI
Page 74
(iv) As a stepping stone to higher studies
(v)
E
Emphasis on practical skills
(vi) Other
Areas which need improving (please specify).
9.
10. Do you think that the work experience component of the course was useful?
Yes
No
Unsure
Comments
11. Do you think that the time spent in work experience as enough?
Too long
Just right
Too short
Comments
12. Were you clear about the assessment requirements in the course?
Yes
No
Comments
I
Page 75
13. Do you think that the methods of assessment used in the course were appropriate?
Yes
No
Comments
14. Were you satisfied with the teaching resources used in the course?
Yes
No
Not applicable
LI
Class notes
Practical equipment used
n
Videos
Films
n n
Excursions
n
n
Comments
15. Was enough time devoted to:
Group discussions
Yes
No
Practical class activities
Yes
No
n
16. Do you think that the overall presentation of the course was satisfactory?
Yes
No
Comments
Page 76
17. What were the best aspects of the course?
Comments
18. What aspects of the course, do you think should be improved?
Comments
19. Do you think that what you learnt in this course will improve your chances of obtaining
employment?
Yes
No
Comments
20. Did the course help you to identify other possible career paths?
Yes
No
Comments
21. Do you think that what you learnt in this course will assist you with other employment
opportunities?
Yes
No
Comments
Page 77
22. Do you have any other comments about the course which you would like to make?
Yes
No
Comments
Thanks for your co- operation.
Page 78
BUILDING
PROFORMA
STUDENTS WHO HAVE LEFT COURSE
D
College /Institution /Provider:
Program:
Insert Introduction
DATE:
1. Current Employer:
2. Current Position:
(e.g. level of seniority)
3. Do you plan to complete the course?
Yes
No
(Your response will be held confidential)
If yes, anticipated year and semester of return
Year
19
Semester
If no:
4. What were your reasons for enrolling in the course and which of these aims were met?
(a) Why did you leave the course?
(i)
To enrol in another course
(ii)
Unable to continue course due to work
commitments
(iii)
Please tick
(one or more)
E
E
Failed some /all of my units /modules and did not
wish to continue course
(iv)
Passed most of my units /modules but did not
wish to continue course
Page 79
(v)
Only interested in doing part of the course
(vi)
Other (please specify)
Answer either (b) OR (c)
(b)
If you were prevented from continuing the course, indicate the reason /s:
Please tick
(one or more)
(i)
Financial problems
(ii)
Employer would not give me sufficient time off
to attend the course
(iii)
Difficult to travel from work to the college
to study
(iv)
Lacked sufficient time for private study
(v)
Unsatisfactory study facilities at home
(vi)
Personal reasons (eg. accident, illness,
o
o
o
pregnancy, marriage)
(vii) Transfer to other district /town /city
(viii) Pressure from other studies
(ix)
Pressure from outside interests such as
hobbies or sporting commitments
(x)
Timetable inappropriate
(xi)
Other (please specify)
Page 80
(c)
If you chose not to continue the course, indicate your major reason(s):
Please tick
(one or more)
(i)
Course was not meeting my requirements
o
(ii)
Course standard too low
E
(iii)
Course standard too high
(iv)
Too much work expected
(v)
Did not like teaching staff
(vi)
Did not like course structure
(vii)
Did not like course content
o
o
o
(viii) Did not like fellow students
(ix)
Timetable inappropriate
(x)
Other (please specify)
o
o
Page 81
STAFF RESPONSE
BUILDING
PROFORMA
E
Provider:
Course:
Staff Number:
Insert Introduction
SUBJECT
Subject Name:
1.
Is the name of the unit /module appropriate?
Yes
I
1
No
If no, suggest an alternative.
2.
Is the order of the topics appropriate?
Yes
No
If no, suggest modifications.
3.
Are there any topics which should be included /deleted?
Yes
I
I
No
If so, please specify.
Page 82
4.
Is the order of topics appropriate to:
(i)
Student's skill development
Yes
No
(ii)
Other units /modules being studied
Yes
No
If no to either (i) or (ii), write down suggested changes.
PREREQUISITES
5.
List the prerequisites for this unit /module:
6.
Are the entry level competencies stated in the course document appropriate?
Yes
No
If no, indicate change(s) required.
7.
List the elements (other courses /units /modules) for which (this unit /module) is stated
prerequisite.
8.
Is (this unit /module) a suitable prerequisite for the elements you have listed in (7)?
Yes
No
If no, suggest modifications.
Page 83
MODE OF DELIVERY
9.
Indicate the mode of delivery for this unit /model.
Day (weekday on campus)
Evening (weekday on campus)
Block (weekday on campus)
E
E
Fleximode
10.
In your opinion is the mode of delivery appropriate?
Yes
No
I
l
If no, please explain.
11.
Indicate which other modes of delivery you think would be appropriate.
Day (weekday on campus)
Evening (weekday on campus)
Block (weekday on campus)
Fleximode
12.
E
Indicate which modes you would consider to be inappropriate.
Day (weekday on campus)
Evening (weekday on campus)
Block (weekday on campus)
Fleximode
I
n
Page 84
Please explain your response for any modes which have been marked as inappropriate.
RELEVANCE
13.
Is the balance between theory and practical work appropriate?
Yes
No
Comments
14.
(i)
Indicate whether this unit is a core or "elective" unit (for this question, units
within streams are "elective" units).
Core
(ii)
Elective
Is this classification appropriate?
Yes
No
If no, explain your answer.
15.
(i)
If the unit is an "elective" unit, is it a compulsory unit within a stream?
Yes
(ii)
I
No
Is this classification appropriate?
Yes
No
If no, explain your answer.
Page 85
OBJECTIVES
16.
Are there any performance /learning objectives which should be reworded (ie. where
they are clumsily written or unclear)?
Yes
No
If yes, please specify the element of competency and suggested rewording.
17.
Are modifications required for the "College Based Instruction "?
Yes l
l
No
If yes, please specify (attach separate page(s) if necessary).
RESOURCES AND EQUIPMENT
18.
Are the items of equipment as listed in the syllabus document consistent with:
(i)
the College Based Instruction?
Yes
No
If no, please specify.
(ii)
New Technologies?
Yes
No
If no, please advise of latest (appropriate) developments.
Page 86
19.
Are the items of equipment readily
available?
Yes
No
(ii) affordable?
Yes
No
(iii) accessible?
Yes
No
(i)
f
I
I
If no to any of (i) - (iii), please specify.
20.
Is the equipment list adequate for this subject?
Yes
[
I
No
If no, specify additional equipment either necessary (n), or desirable (d) (Attach separate
sheet if required).
21.
Are the references listed in the course document readily
a)
available?
Yes
No
b)
affordable?
Yes
No
c)
accessible?
Yes
No
If no, please specify.
22.
Are the references appropriate?
Yes
i
No
If no, suggest references which you believe are more appropriate.
Page 87
23.
Specify any references, videos, etc. which should be added to the resource list (attach a
separate page if necessary).
ASSESSMENT /PRACTICAL WORK
24.
Are the assessment guidelines recommended in the syllabus document appropriate for
the corresponding objectives?
Yes
No
If no, specify which performance /learning objectives require an assessment change and
recommend the nature of change (attach a separate page if necessary).
25.
Is the recommended practical work appropriate to the assessment guidelines?
Yes
No
If no, specify changes required (attach a separate page if necessary).
26.
Should a practical manual or skills /tasks manual be developed for this unit /module?
Yes
No
If yes, would you be prepared to contribute to the development of same?
Yes
27.
1
No
Does the syllabus document require rewriting - either partially or totally?
Yes
No
If yes, would you be prepared to contribute your time to this process?
Yes
No
Page 88
28.
Please attach the following to this survey:
(i)
A copy of all tests and assignments given to students.
(ii)
Class notes and materials given to students.
(iii)
Samples of students' work (with permission) in the following categories:
very good
average
poor.
Page 89
PROFORMA
PROVIDER
F
COURSE EVALUATION
REPORT
COURSE NAME:
COURSE CODE:
CONTACT NAME:
Please return completed Evaluation Report to:
Page 90
DATE:
1.
COLLEGE COURSE INDUSTRY ADVISORY COMMITTEE
Is there an active College Course Industry Advisory Committee for this course?
Yes
No
If there is an active Committee, do you maintain contact with it?
Yes
No
Who is the convenor for this Committee?
NAME:
ORGANISATION:
ADDRESS:
PHONE NO.:
2.
COURSE AIMS
Are there any particular aims which the College Course Industry Advisory Committee
consider are inappropriate for the course? Please comment.
3.
COURSE STRUCTURE
3.1
LENGTH OF COURSE (Overall) - in terms of being able to achieve its stated
aims. Please circle appropriate description.
TOO SHORT
SATISFACTORY
TOO LONG
Comments
Page 91
3.2
LENGTH OF UNITS /MODULES: Are there any units of study /modules that you
wish to comment on regarding duration?
LENGTH
MODULE NAME
(NO. OF
HOURS)
TOO
SHORT
TOO
LONG
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(i)
(k)
(I)
(m)
(n)
(o)
(p)
(q)
(r)
(s)
(t)
(u)
(y)
(w)
(x)
Page 92
3.3
In relation to the units of study /modules you listed in 3.2, are there any
comments regarding the appropriateness of aims and objectives of same?
Please use alphabetic identification from 3.2.
3.4
Are any of the units of study /modules no longer relevant? Please comment.
3.5
Are there any additional units of study /modules which you believe should be
included in the course?
4.
ASSESSMENT PROCEDURES
Are there any aspects of course assessment on which you would like to comment?
Page 93
5.
RESOURCES
5.1
STAFFING:
Does the delivery of this course have any Staff Development
requirements on which you would like to comment?
5.2
FACILITIES AND EQUIPMENT: Are there any comments regarding
facilities /equipment required to conduct the course?
5.3
SERVICES AND SUPPORT: Are there any comments regarding services and
support required to conduct the course?
Page 94
6.
PLANNING
6.1
SELECTION OF PARTICIPANTS: Briefly summarise procedures followed when
selecting participants for the course. Do you have any comments about the
relevance of these procedures?
6.2
Are there any comments regarding the co- ordination of the course between
Colleges?
6.3
Do you feel there is a need for Staff Development in the implementation of
Course Evaluation and Validation procedures? Please list suggestions:
7.
ENROLMENTS
7.1
Number of FT students:
Year 1:
7.2
Number of PT students:
Year 1:
0
0
Year 2:
L1
Year 2: f
Page 95
I
PROFORMA
TO BE COMPLETED BY RESPONSIBLE COLLEGE
G
COURSE EVALUATION SUMMARY
COURSE NAME:
CODE:
CONTACT NAME:
1.
PHONE NO.:
STATEWIDE COURSE ADVISORY COMMITTEE
Convenor:
Phone No.:
Number of meetings during year:
Average number of industry representatives attending meetings:
2.
RESULTS OF EVALUATION
Has the evaluation shown that any changes should be made to the course?
Yes
No
If yes, has the Statewide Course Industry Advisory Committee endorsed the proposed
changes?
Yes
No
Please attach the list of proposed changes.
3.
DATA COLLECTION
Apart from College Course Evaluation Reports, what data has been used in evaluating
the course and deciding upon the changes needed?
Page 96
4.
ENROLMENT DETAILS
Melbourne Metropolitan Colleges:
Number of colleges running course:
Number of Evaluation Reports received:
Number of FT students:
Year 1:
Year 2:
Number of PT students:
Year 1:
Year 2:
Country Colleges:
Number of colleges running course:
Number of Evaluation Reports received:
Number of FT students:
Year 1:
Year 2:
Number of PT students:
Year 1:
Year 2:
PLEASE RETURN TO:
Page 97
EVALUATION OF THE
QUESTIONNAIRE FOR WORK EXPERIENCE EMPLOYERS
Introduction:
An evaluation is being conducted of the course, to determine whether it meets
industry training needs, as an entry level course. As an employer who provided
a student /s with relevant work experience during the course, your opinions on
the relative merits of the course in preparing students for the workplace would
be most useful.
Your responses will be treated in the strictest confidence and will be added to
those of other employers to form an overall appreciation of the suitability of the
course. In this way, the course can be altered to meet the industry needs.
Please complete one questionnaire for each student, engaged by your business
for the Work Experience component of the course.
Please answer all questions by ticking the appropriate box, and /or adding
comments in the spaces provided.
Business Name:
Address:
Type of Business:
Section A:
1.
At which TAFE College did your student do the course?
(if relevant)
2.
Please indicate the gender of your Work Experience student.
Male
3.
Female
Please indicate the approximate age of your Work Experience student.
Under 18
18 -21 Yrs
31 -40 Yrs
Over 40 Yrs
22 -30 Yrs
Page 98
4.
Please indicate the extent of the Work Experience which you provided.
1 day
5.
2 days
1 wk
3 days
2 wks
How did the College initiate arrangements for the Work Experience
component with you?
Telephone
Face -to -face
6.
Letter
Other
Were you fully briefed on the objectives of the Work Experience
component of the course?
r-7
Yes
Uncertain
No
Comments:
7.
During the Work Experience component, was the student's progress
monitored by the College in any way?
Yes
Uncertain
No
Comments:
8.
Were you required to provide the College with a final report on the
student's
performance
component?
Yes
No
on
completion
of
the
Work
Experience
Uncertain
Comments:
9.
Do you believe Work Experience is a worthwhile component of the course
for the student?
Yes
No
Uncertain
Comments:
Page 99
Section B:
10.
How satisfactory was the student in each of the following areas:
Please tick only one box per item
Unsatisfactory
Response to Instruction
E
E
E
Attitude
El
Enthusiasm /Interest
Punctuality
Acceptable
Very
Satisfactory
Customer Relations
Reliability
n
Personal Presentation
Willingness to Learn
Ability to Perform
El
given Tasks
11.
Please indicate the level of supervision that your Work Experience
students required as compared to other new employees.
More
12.
Same
Less
Uncertain
Do you think the student demonstrated a suitability for the industry
overall?
Yes
No
Uncertain
Comments:
Page 100
Section C:
13a.
Please indicate how useful you believe each of the subjects offered in the
course is to someone entering the industry.
Please tick only one box per subject
Not
Useful
Possibly
Useful
Useful
Very
Useful
Communication
Problem Solving
n
Technology Studies
Career Planning /Work Educ.
n
Retail Working Environment
Industrial Relations
n
Product Knowledge
Selling Techniques
Point of Sales Systems
Retail Calculations
n
Stock Control
Loss Prevention
n
n
CI
13b. Are there any additional subjects you think should be included?
T1
Yes
No
If yes, please list:
14.
Would you be prepared to assist in the Work Experience arrangement
again in the future?
Yes
No
Uncertain
Comments:
Page 101
15.
Do you have any other comments about the Work Experience component
that you would like to make?
Yes
No
Comments:
Thank you for your co- operation.
Page 102
SECTIO NC
Page
Best Practices
103
Help Us Improve this
105
User's Guide
Definitions
Bibliography
106
110 -111
_
0
6
COURSE MAINTENANCE PROCESSES
USER'S GUIDE
BEST PRACTICES
SECTION C
On successful completion of Section C, you will have:
(
1.
provided feedback on the usefulness of the User's Guide
2.
provided (where possible) examples of Best Practices.
)
Section C: Best Practices
Page 103
COURSE MAINTENANCE PROCESSES.
.....................
........................
USER'S
GUIDE
................. ..............:...:::...:::.....::::..;
Recognition of Prior
Learning
Determination, on an individual basis, of the
competencies obtained by a learner through previous
training, work experience and /or life experience and
the advanced standing the learner is entitled to as a
result of these competencies.
Responsible College
A term used informally to denote a Provider that has
been given responsibility for statewide curriculum
management of an accredited course by a Designated
Provider.
Skill
A skill encompasses the holistic application of the full
range of cognitive and psychomotor functions
required, together with appropriate knowledge and
attitudes to perform a work function.
Standards
A level against which to measure acceptable
performance. A precise statement of the level of
performance required to be regarded as competent in
performing a particular task.
Structured Training
A training course or program which is delivered in the
workplace and /or off - the -job. The competencies to be
acquired are clearly stated before the commencement
of training.
Training (recognised)
Refers to accredited courses, training programs and
skill acquisition, formal or informal, that are related to
the attainment of competencies leading to a credential
or statement of attainment.
Training Program
(recognised)
Refers to a sequence of training which contains a
component(s) of an accredited course and which leads
to a statement of attainment.
Section C: Best Practices
Page 108
COURSE MAINTENANCE PROCESSES
USER'S; GUIDE:
Acronyms and Abbreviations
ACTRAC Australian Committee for Training Curriculum
ASCO
Australian Standard Classification of Occupations
ASF
Australian Standards Framework
ASIC
Australian Standard Industrial Classification
CBT
Competency -Based Training
CMP
Course Maintenance Processes
CSB
Competency Standard Body
DACUM
Developing a Curriculum
DEET
Department of Employment, Education and Training
ITAB
Industry Training Advisory Body also: Industry Training Accreditation
Board
MOVEET Ministers of Vocational Education Employment and Training ` +'
NCVER
National Centre for Vocational Education and Research
NFROT
National Framework for the Recognition of Training
NTB
National Training Board
RATE
Register of Australian Tertiary Education
ROT
Recognition of Training
RPL
Recognition of Prior Learning
TAFE
Technical and Further Education
VEETAC
Vocational Education, Employment and Training Advisory Committee
VET
Vocational Education and Training
-
Section C: Best Practices
Page 109
OURSE: MAINTENANCE
USER'S ;GUIDE,:
BIBLIOGRAPHY AND RESOURCE PUBLICATIONS
1.
A Code of Research Practice, Victorian TAFE Board, Box Hill, 1987.
2.
An Approach to the Evaluation of TAFE Institutions, TAFE National Centre
for Research and Development Ltd., Adelaide, 1984.
3.
Bell, Malcolm. Guidelines for the Evaluation of TAFE Programs, TAFE
Services, Victoria, 1982.
4.
Carmichael. Australian Vocational Certificate Training System, Australian
Government Publishing Service, July 1992.
5.
Coughlin, Peter. Designing and Conducting On -going Course Evaluation,
State Training Board, Victoria, 1989.
6.
Crooks,
7.
Duignan, Patrick A. Evaluating Programs and Projects in TAFE, Centre for
Administrative and Higher Education Studies, University of New England,
1986.
8.
Gardney G.
Social Surveys for Social Planners, Holt, Rinehart and
Winston, Sydney, 1978.
9.
Gillespie, Batt, Verity. Reporting Student Performance in TAFE, Victorian
Technical and Further Education Board, Melbourne, 1987.
Terry. Assessing
Kensington, NSW, 1988.
Student
Performance,
HERDSA
No.
8,
10. Hayton, Neylon, Inglis, Hayes. Integrating Quality Concepts into TAFE
Courses, TAFE National Centre for Research and Development Ltd.,
Adelaide, 1989.
11. Making Changes. Evaluation and Validation of TAFE Programs, collected
documents, TAFE National Centre for Research and Development Ltd.,
Adelaide, 1988.
12. McMahon, F. & E. Carter. The Great Training Robbery: A Guide to the
Purchase of Quality Training, The Falmer Press, London, 1990.
13. NIES TQM 'How To' Approach, Guide to Concepts, Principles and
Imperatives, Dept. of Industry, Technology and Commerce, ACT, 1990.
14. Oppenheim, A.N.
Questionnaire
Heinemann, London, 1973.
Design
and
Attitude
Measurement,
Section C: Best Practices
Page 110
.
COURSE MAINTENANCE, PROCESSES
USER'S GUIDE
15. Ramsden, Paul and Dodds, Anges, Improving Teaching and Courses: A
Guide to Evaluation, Centre for the Study of Higher Education, The
University of Melbourne, Parkville, 1988.
16. Sommer, R. & B.B. Sommer. A Practical Guide to Behavioural Research
(Tools and Techniques), 2nd Edition, Oxford University Press, New York,
1986.
17. Silvey, J.
Deciphering Data: The Analysis of Social Surveys, Longman,
New York, 1975.
18. Standards for Evaluations of Educational Programs, Projects and Materials,
developed by the Joint Committee on Standards for Educational
Evaluation, McGraw -Hill, New York, 1981.
19. Survey for the Evaluation of the Training of Hairdressing Apprentices
1990. Curriculum Dept., Box Hill College of TAFE, 1990.
,
,r.',
;.,r. :
<
l..
3
.
fw r/ t
t.
.
9i,.
.
,
.
FS
r1 .
f;
!
Section C: Best Practices
Page 111