On Demand Testing Fact Sheet
About the assessment tool
Brief description
The On Demand Testing Program is an online tool for teachers to use when, where and how
they choose. The system provides adaptive and linear tests.
English and Mathematics general ability tests and dimension‐specific tests are provided and are
linked to the AusVELS.
The tests measure:
English – Literacy-Reading, Language-Spelling, Language-Writing
Mathematics – Measurement & Geometry, Number & Algebra, Statistics & Probability
It is a time‐saving tool that can be administered to a single student and/or a whole class.
Students sit the tests online. The tests are scored by the system and teachers receive instant
feedback via a suite of reports.
Uses of the Tests Include:
• Pre‐ and post‐testing of students
• Assessing the ability levels of new intake or late arrival students
• Identifying strengths and weaknesses of individual students
• Corroborating teacher judgements
• Assisting in forward planning of teaching programs
What does the
assessment tell you
about the child?
Adaptive tests generate a Standard Score calculated using the difficulty value of each item
presented to a student. The Standard Score is an indication of the AusVELS level at which the
student is working for Mathematics and English. These tests provide an indication of a student’s
general ability level and can be used for pre and post‐testing to determine student growth over
time.
All on Demand tests provide diagnostic reports about the student’s strengths and weaknesses
in the assessed areas.
Longitudinal reporting is provided via the Student Tracking Report. This report collates
students’ accumulated results for all tests sat over time.
What are the
components of the
tools?
Two Types of tests are available:
1.
Computer Adaptive Tests deliver sets of questions to students that vary according to student
ability. Depending on the responses given in previous questions, the system presents
progressively easier or more difficult questions to the student.
In the Mathematics domain, the system currently contains general adaptive tests (60
questions) and single dimension tests (30 questions) in Number & Algebra,
and Measurement & Geometry. In the English domain, the system currently contains
general adaptive tests (60 questions) and single dimension tests (30 questions) in LiteracyReading, Language-Spelling and language-Writing.
2.
Linear tests provide a fixed set of questions. All students are presented with the same
questions in the same order during the test. Students can navigate backwards and
forwards through linear test questions. Student responses are saved and stored by the
computer and teachers are able to view and analyse the results at a student, class or
question level.
Both types of tests present these types of questions:

multiple choice

short answer

hot spot

drag and drop

mathematical calculator.
Assessment takes between 30 and 45 minutes to complete on average.
What is the style of
assessment?
All assessments are completed independently by the student. The assessments can be
completed by an individual student, a class or whole cohort of students.
All assessments are marked by the system. On completion of each test, results at the student
and class level are available to the teacher immediately via a suite of reports.
What are the
limitations of the tool?
This program requires a secure network and secure log‐in to access tests.
Students do not get direct feedback; only teachers can access the results.
Are there FAQs about
the assessment?
See FAQ section 5 in Teacher Training Guide at
http://www.vcaa.vic.edu.au/Documents/ondemand/ondteachertrainingguide.pdf. These are
mostly technical troubleshooting.
How was the
assessment tool
developed?
The tool began as VSAM in the 1990s. It was used from 2002 – 2006 as an online alternative to
the pencil‐and‐paper version of the Year 7 Achievement Improvement Monitor (AIM) tests. In
2005, AIM Online trials were conducted for Years 3 and 5. In 2006, VCE exams in two subjects
were successfully trialled using the program.
On Demand items are developed through a collaborative process with expert practitioners. The
VCAA conducts regular monitoring and validation of its item banks.
How long has the tool
been used?
Computer adaptive tests have been available since 2002.
Relevant research
underpinning tool
Based on the work of Weiss & Kingsbury (1984), the construction of the VCAA’s Computer
adaptive tests comprises five technical components. These being:
1. Calibrated item pool
2. Starting point or entry level
3. Item selection algorithm
4. Scoring procedure
5. Termination criterion
Validation process and
data for the tool
Content was initially purchased from a range of commercial sources and over the years has
been complemented with test items produced by the VCAA. The item writing and review
processes are ongoing.
All tests in the On Demand Assessment program are calibrated and scored using the Rasch
model. To ensure the accurate calibration of test items, psychometric analysis is undertaken
regularly by the VCAA’s Measurement Unit. Similarly, to ensure compliance with AusVELS,
items are reviewed by curriculum experts.
Link to the assessment
tool
User guide
http://www.aimonline.vic.edu.au/
Training support
materials
Help desk
http://www.vcaa.vic.edu.au/prep10/ondemand/docodownload.html
Background to the assessment tool
Further support
http://www.vcaa.vic.edu.au/prep10/ondemand/index.html
Freecall 1800 827 721
vcaa.ondemand.support@edumail.vic.gov.au
Supporting Information for the assessment tool