Topic Content
Developing Direct Observation Tools
Developing Direct Observation Tools
 Aligning Assessment Tools to identified Assessment Methods
 Developing Direct Observation Tools
 Reviewing and Trialing Direct Observation Tools
Aligning Assessment Tools to identified Assessment Methods
After you have
 unpacked the Unit of Competency to identify the evidence requirements
 selected the most appropriate assessment methods and
 identified the assessment tools that align with each of the assessment methods
the next step is to develop the identified assessment tool containing both the instrument
and the procedures for gathering and interpreting the evidence, and then to review and
trial the tool.
Direct Observation
Real work/real time activities in the workplace or
work activities in a simulated workplace
Direct Observation tool
Observation Checklist + questions to
accompany checklist + sample responses +
instructions to learner and assessor
(including advice on interpreting evidence)
Direct observation is a method which enables you to:
 observe directly what learners can do
 question them to determine if they understand why they are doing it
Evidence comes from real work in real time under real or simulated workplace
conditions (replicating the demands, interruptions, problems and pressures of the
workplace) and may be collected by the assessor or a third party.
This assessment method allows learners to demonstrate their vocational and
employability skills and their application of workplace procedures (including occupational
health and safety) either in the workplace, or for logistical reasons (such as safety,
expense or time), at another time or location. It is unlikely for example, that learners will
be able to demonstrate machine shutdown procedures during normal work hours. So
the assessor may ask the learner to demonstrate this skill out of working hours or at
another site, (such as a Registered Training Organisation (RTO)), using similar
machinery.
When planning and conducting an observation:
 check that the method complies with the requirements of the Training Package and
the Unit of Competency (including the Range Statement, Elements and Performance
Criteria, and Evidence Guide. Note that some Units of Competency will specify
appropriate and inappropriate assessment methods.)
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 1 of 13
Topic Content
Developing Direct Observation Tools









ensure materials, equipment, procedures and conditions reflect current industry
practice
focus on the whole workplace task rather than separate components
set realistic expectations (eg time limits and quality specifications)
consider work cycles and situations so that the observation produces reliable
evidence and does not interrupt normal routines
supplement observation with questioning or an interview, to draw out the way in
which underpinning knowledge, employability skills and the dimensions of
competency are being applied in practice
consider other people in the work environment, such as customers and supervisors
who can confirm that observed performance is consistent (not just produced for the
assessment situation)
share the criteria upon which the assessment is based with the learner and relevant
others
remember that the learner might feel nervous while being observed
use a checklist and predetermined questions to:
o help focus on key aspects of the work activity or skill
o record that the learner has performed all required tasks
There are situations where it might not be possible or it may be inappropriate to use
observation. These could include:




work activities that occur irregularly, take place over an extended period of time or
are difficult to observe (eg dealing with an emergency situation, planning a process
or managing a team task)
work activities that involve issues of privacy and confidentiality (eg counseling,
providing feedback to team members on work performance, or dealing with
confidential or private client information)
situations where it may be culturally inappropriate to observe the activity
situations where the presence of an observer may compromise workplace safety
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 2 of 13
Topic Content
Developing Direct Observation Tools
Developing Direct Observation Tools
Assessment Tool(s)
When you decide to use Direct Observation, your
assessment tool will need to include:
 An observation checklist
 Questions to accompany the checklist
 Supporting documentation:
o Sample responses
o Instructions to learner
o Instructions to the assessor including
advice on interpreting the evidence
The instrument/s and
procedures used to gather and
interpret evidence of
competence:
(a) Instrument – the specific
questions or activity
used to assess
competence by the
assessment method
selected. An
assessment instrument
may becomponent
supported of
bythe
a tool development and the RTO’s continuous
An important
profile of acceptable
improvement process is to:
performance and the
 review
the tool
with other
and trial the tool with sample learners prior to
decision
making
rules assessors
or
useguidelines to be used by
assessors.
 evaluate
the effectiveness of the tool following use
(b) Procedures – the
information or
instructions
given to the tools are provided on the following pages. Please note
Excerpts
of Direct Observation
candidate
and
these are provided asthe
samples used by assessors NOT as “best practice” exemplars.
assessor about how the
assessment isChecklist
to be
The Observation
conducted and recorded.
An observation
(AQTF) checklist:
 enables focussed observation
 is based on workplace tasks and written in workplace language (it is not a re-iteration
of the elements and performance criteria from the Unit of Competency)
 incorporates room for structured notes that can be referred to when making the
assessment decision or giving feedback to learners
 provides a prompt for the assessor and a guide for the learner.
The complexity of the checklist will mirror the complexity of the task/s being observed.
Questions to accompany the Checklist
Observation checklists are supported by a list of performance questions which allow the
assessor to gather evidence in relation to the learner’s understanding of the task
which the assessor is observing.
In addition to the task skills which are observed during the activity, these questions
assist the assessor to gather evidence in relation to the dimensions of competency:
 contingency management skills (what would you do if…?)
 job role / environment skills (what are the procedures and policies for…?)
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 3 of 13
Topic Content
Developing Direct Observation Tools

task management skills (how did you prepare for this task, how do you manage
when …?)
Questions can also focus on the learner’s confidence to transfer skills to other
situations.
The questions may be incorporated into the observation checklist or be provided as an
additional document.
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 4 of 13
Topic Content
Developing Direct Observation Tools
Transport and Distribution Cluster
Learner Name:
Assessor Name:
Unit/s of competency






Workplace Task
Assist team member shift materials using mobile crane
TDTD197A Shift materials safely
TDTE897A Process workplace documents
TDTF197A Follow occupational health and safety procedures
TDTF297A Conduct housekeeping activities
TDTG197A Work effectively with others
TDTI297 Apply customer service skills
Date of Assessment
Location
Procedure to be observed: Observation of the learner assisting in mobile crane operations, including the
setting up and dismantling of the crane, stowing and maintenance of gear, housekeeping of crane and lifting
site and assistance in lifting operations.
PERFORMANCE
During the activity did the learner satisfactorily:


follow OHS procedures related to the operation of a
mobile crane under close supervision?
assist with housekeeping procedures on both the
vehicle and the lifting site?
assist with the setting up and dismantling of the
mobile crane?
assist with the transport and operation of the mobile
crane?
work effectively within the mobile crane team?

carry out the required calculations accurately



Yes
No
Comments

apply customer service procedures and policies
when dealing with clients?
RESPONSES TO QUESTIONING
(Satisfactory response – Yes  No )
Q1: What are the procedures and policies for housekeeping in a mobile crane vehicle depot and
lifting site?
Response:
Q2: What are your responsibilities in applying them?
Response:
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 5 of 13
Topic Content
Developing Direct Observation Tools
Q3: What are your functions when you assist with the setting up and dismantling of mobile cranes?
Response:
Q4: What would you do if you started to feel tired while you were assisting in the setting up and
dismantling of a mobile crane?
Response:
Q5. What would you do if you found that the crane or lifting gear was defective or faulty?
Response:
Q6. What would you do if a customer complained to you about an aspect of a lifting job on which
you were working?
Response:
OVERALL THE LEARNER’S PERFORMANCE WAS:
Satisfactory
Not Satisfactory
Feedback to learner:
Learner’s Signature:
Assessor/Observer’s Signature:
Adapted from Designing assessment tools for quality outcomes in VET WA DET 2008
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 6 of 13
Topic Content
Developing Direct Observation Tools
Practical Assessment - AURT210170A
(The assessor chose to develop questions to determine underpinning knowledge as a separate document
– they are not included in this example)
Learner Name:
Unit of Competency Code:
AURT210170A
Unit of Competency Name:
Inspect and Service Braking Systems
Unit Descriptor
This unit identifies the competence required to carry out the service of
hydraulic, mechanical, and power assisted (if applicable) braking
systems and their associated components
Workplace or RTO:
RTO workshop
Observation:
During the demonstration of skills, did the learner satisfactorily complete the
following tasks:
Not
Satisfactory
Satisfactory
Instructions for demonstration: The apprentice will demonstrate the correct technique and method for
Inspecting and servicing braking systems. The teacher will assess the demonstration and provide feedback to
the student.
Use manufacturers’ workshop manual/publication/database:
 to access, interpret and apply information/specifications
 safely use hand and power tools and equipment, observe safety procedures
Inspect hydraulic brake system for correct operation and serviceability
 eg leaks, spongy lever, brake drag
 smooth lever operation and correct freeplay
 fluid condition and selection of replacement fluid depending on application
 stop light operation and adjustment if applicable
 brake discs/rotors for correct run-out and thickness
 brake pads for serviceability
 calliper operation
Inspect mechanical brake system for correct operation and serviceability
 correct cable free play adjustment
 cable condition and smooth operation
 shoe actuating camshaft in backing plate for smooth operation
 determine by adjustment and operational action if further action is required
 eg shoe,drum or cable replacement
 correct stop light operation and adjustment
 correct twin leading shoe adjustment (if applicable)
Note: A basic knowledge of hydraulic principles, mechanical lever principles, and the possible causes
of vapour lock, brake fade spongy or pulsating lever and brake drag is essential.
Comments
The student’s demonstration was:
Satisfactory
Not satisfactory
Assessor name: _____________________
Date:
/
/
Assessor signature: _________________________
Excerpt– retrieved from Resource Generator http://www.resourcegenerator.gov.au/
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 7 of 13
Topic Content
Developing Direct Observation Tools
Supporting Documentation
In addition to the Observation Checklist and questions to accompany the Checklist, the
assessor must include in the tool:
 Sample responses
 Instructions to learners
 Instructions to assessors and/or observers and advice on interpreting the evidence
Sample responses
The sample responses (sometimes called marking guides or benchmarks or ghost
markers) identify the performance indicators for the task activity and requirements of
answers to the knowledge questions.
The excerpts below provide examples of how advice to assessors (or other observers)
on the required responses can be provided.
THCMAN05A: Install LP Gas Systems in Recreational Vehicles
Plan and organise activities effectively:
 Obtain and interpret plans / specifications
 Determine gas load / design requirements from design drawing or qualified source
 Calculate storage capacity using recognised formulae or tables acceptable to the relevant State
Authority
 Select capacity which is adequate for the load details given in the job instruction
 Ensure capacity selected for the application does not exceed requirements given in AG 601
 Estimate quantity and type of materials from design drawing or on site dimensions
 Select materials and components to comply with AG 601
 Ensure material quantities ordered are sufficient to complete the installation
 Ensure materials, equipment, appliance/s and cylinder/s delivered correspond with order
form/delivery docket
 Check materials, equipment, appliance/s and cylinder/s for acceptable condition or report safety
hazards encountered while installing an LP gas system.
Integrate health, safety and security procedures:
 Identify and understand staff responsibilities with regard to safe working practices with petroleum
 Understand and describe characteristics of petroleum products
 Identify and understand the nature and characteristics of LPG
 Follow safe application and handling procedures of LPG
 Follow emergency procedures in the case of an LPG leak explosion or fire
 Identify application for portable extinguishing equipment correctly
 Identify HAZCHEM, HAZMAT signage
 Use personal protective equipment in accordance with OH&S requirements
 Use manual lifting and handling equipment/techniques in accordance with OH&S requirements
…….(continues)
Excerpt from Suggested Assessor Checklist – retrieved from Resource Generator http://www.resourcegenerator.gov.au/
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 8 of 13
Topic Content
Developing Direct Observation Tools
SIRXCLM001A Organise and Maintain Work Area
Marking guide and comments
The marking guide provides a range of acceptable and expected responses. This guide is not exclusive as
individual work environments may give the learner the opportunity to provide other acceptable responses.
As the assessor you will need to make a professional judgement on these responses as there may be
occasions when not all responses are appropriate and this will require you to have an understanding of the
store’s policies and procedures.
Learner name
Workplace
Assessor name
Ob=Observation (Observed directly by assessor) ; Per=Performance (May be scenario, role play, etc)
Did the Student apply store policies to:
(Tick Ob column if able to demonstrate)
Maintaining safe, uncluttered, clean and
tidy work areas, including where
applicable:
counters
fixtures
Ob
Marking guide and Comments
Dust, clean, wipe, spray, put pens etc. away,
maintain supplies of rolls etc., maintain impulse or
counter stock, clear returned stock, check
equipment, check cords are safe, remove rubbish,
remove hazards
Comments
Face up, recovery, remove damaged stock, remove
out of date stock, rotate stock, clean fixtures, check
condition of fixtures, remove waste, report and/or
remove hazards
Comments
sinks
Clean, spray and wipe, tidy up, put away, empty
bins, remove food scraps, report and/or remove
hazards
Comments
storage areas
Remove waste, report or remove hazards, check
condition of stock, check condition of fixtures, keep
walkways clear
Comments
…….(continues)
Excerpt from Suggested Assessor Checklist – retrieved from Resource Generator
http://www.resourcegenerator.gov.au/
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 9 of 13
Topic Content
Developing Direct Observation Tools
Instructions to learners
As part of the assessment tool the assessor must provide clear instructions to learners so that they know exactly what is expected of them. These instructions may be written
as part of the checklist or included as a separate document (sometimes called an
Assessment Cover Sheet or Assessment Criteria Sheet)
Good assessment instructions:
 identify the tasks which are to be performed,
 are comprehensive covering the who, what, when, where and how of the
assessment situation (eg materials required, time frames, assistance available,
specific location)


are aligned to the correct AQF level
are written in plain English using standard industry terminology
Example:
Instructions to learners:
The assessment for this unit requires you to answer a series of questions to test your knowledge and to
be observed by your assessor in the workplace. The skills and knowledge to be assessed are described in
the assessment Matrix in your Learner Guide, and in the Learner Log or Supervisor Checklist in your
Training Record Book for this unit. Your workplace supervisor is required to complete and sign the
Workplace Supervisor’s Third Party Report. These assessment activities are detailed below:
1. Observation
You will be observed by your assessor as you apply knowledge and skills relating to using tools,
chemicals and equipment for the safe and efficient cleaning and maintenance of work areas.
The Observation / Performance Checklist will be used.
2. Verbal or Written Questions as appropriate to the assessment context or learner needs
You will be asked a series of questions by your assessor to assess your underpinning knowledge.
The Questioning Checklist (written or verbal) will be used.
3. Workplace Supervisor verification
Verification by a workplace supervisor that the tasks have been completed satisfactorily.
The Workplace Supervisor Checklist will be used.
…….(continues)
Retrieved from Resource Generator http://www.resourcegenerator.gov.au/
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 10 of 13
Topic Content
Developing Direct Observation Tools
Instructions to assessors
Assessors need advice on how to use the observation checklist and how to interpret the
evidence provided when making a decision of satisfactory / not satisfactory.
Reliability is enhanced when clear instructions to assessors /observers are provided so
that they know
 what resources are needed to be prepared for the assessment activity
 exactly what they are looking for in relation to learner responses
 any other issues that need to be taken into account
The excerpts below provide advice to assessors in two different vocational areas:
Example:
Manufactured mineral products Training Package
Introduction
This advice is based on the following design assumptions:
 assessors will meet all the requirements as defined in the assessment guidelines
 assessors will have access to, and use the Training Package as part of their assessment process
 assessors will contextualise these templates to suit the particular assessment situation
 while these templates necessarily are for individual units of competency, assessors are
encouraged to develop combined assessment tools which will enable integrated assessment of
more than one unit of competency concurrently
 the candidate should be able to describe/explain their actions as they demonstrate a competency –
ie answer the question ‘what are you doing/why are you doing it?
Reasonable Adjustments and Accommodations in Assessment
(In accordance with current Disability Discrimination legislation and standards)
Reasonable adjustments for assessment may include, but are not limited to the provision of:
 specialized equipment and resources eg adaptive technology, assistant devices and equipment,
including applications of emerging technologies, pictorial procedure manuals, symbols, signage
and graphical representations
 support provided by appropriately trained support persons eg. Job Support Officers, notetakers,
adaptive communication means which might include services of a communication facilitator or
interpreter
 in certain instances, competency may be demonstrated through consultation with and/or under
direction of a nominated person/supervisor such as a workplace mentor.
…….(continues)
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 11 of 13
Topic Content
Developing Direct Observation Tools
Example:
CUETGE2A Assist with staging


This unit applies to assisting with staging that involves a variety of floors and operable pieces.
This unit should be assessed at an entertainment event or venue on one or more occasions.
Observation
Observation is a popular assessment method, because it is direct and authentic. However, it will often
need to be supplemented by other forms of assessment to obtain sufficient evidence of all aspects of
competence.
The assessee will be observed:
 assisting with preparation for staging
 assisting with pre-production requirements and technical/dress rehearsals
 safely operating standard hand and power tools
 safely using a tallescope, hydraulic lifter and an A-frame ladder
 tying a variety of knots
 using swaging tools
 handling ropes correctly
 plotting cues in accordance with the sequencing, equipment requirements, timing and speed
requirements of a given show
 carrying out basic set assembly
In this unit simulation can be used to assess a wide range of competencies including those listed under
“Observation”.
…….(continues)
Excerpt retrieved from Resource Generator http://www.resourcegenerator.gov.au/
Reviewing and Trialing Direct Observation Tools
Before you use your direct observation assessment tools it is important that you review
the tools with other assessors and trial them with relevant stakeholders to confirm
validity and reliability and to obtain advice on fairness and flexibility.
Focus particularly on whether the tools satisfy the needs of learners and the
requirements of the unit of competency, if the instructions are clear and if other
assessors can reliably use the tools you have developed.
Stakeholders could include:
 trainers
 other assessors
 a sample group of learners
 supervisors / industry representatives
 language, literacy and numeracy advisors
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 12 of 13
Topic Content
Developing Direct Observation Tools
You can use information provided by the reviewers to target potential problems with the
tools and highlight areas for improvement. It is up to you to modify the tools based on
their responses and depending on the magnitude of the revisions, you may feel it is
necessary to go back to the reviewers to seek further feedback.
To demonstrate evidence of continuous improvement it is important to keep a record of
the trial/s including:
 the stakeholders involved
 their recommendations
 the revisions you made
Don’t forget to maintain version control on your tools to ensure you (and other
assessors) use the most current version.
Developing Direct Observation Tools
 Aligning Assessment Tools to identified Assessment Methods
 Developing Direct Observation Tools
 Reviewing and Trialing Direct Observation Tools
106743030
 The State of Queensland (Department of Education and Training), 2009
Page 13 of 13