The 11th 334 Symposium
Advancing Teaching and Learning: Evidence, Outcomes and E-learning
Lingnan University
4 March 2011
Application of an On-Line
Diagnostic English Language Test
Overview of presentation
 Background
 The DELTA system
 Applications of DELTA
Background
DELTA Context
 Opportunity provided by planning for 4-year
curriculum for English language centres to review
their curricula and provision of learning, teaching
and assessment
 Tertiary education’s focus on outcomes-based
learning, teaching and assessment
 Increased emphasis on learner autonomy and the
independent learner
 Bachman report to UGC on language
enhancement
The Bachman Report (1)
While there may be a number of specific, research and
development (R&D) projects involving interinstitutional collaboration among language center staff
at different universities, one that is of particular
importance, in my view, is the collaborative R&D
project involving HKIEd, PolyU, and LU, on the
development of a system for assessing English language
proficiency ... there is a need for much greater
collaboration in this area.
Language Enhancement in Hong Kong Universities: Some
Observations and Recommendations (Annex A, p.4)
Lyle F. Bachman, July 2010
The Bachman Report (2)
2. An overall plan for the development of language assessments could include
the following:
 Different tests developed for complementary purposes, some could be CR,
objectives-based, and some could be NR, and independent of any
particular learning objectives.
 Tests with some components to be delivered face-to-face and some online.
 Integration with a language [learning] management system that could
include E-portfolios for tracking students’ LE activities and progress in
language proficiency.
 A delivery platform that is flexible enough to be used for formative and
summative course-based assessments by all eight institutions. This
flexibility would make it usable for use in general foundational language
courses, language for academic purpose courses, and discipline-specific
language courses, as well as for certification. (Annex A, p.17)
Relevance and Significance of DELTA
The DELTA project is relevant to the partner institutions and
this symposium for several reasons:
 It is a genuinely collaborative, inter-university project in which
language centres share knowledge, experience and resources.
 It provides additional data about students’ language proficiency.
 Partner universities will be able to use DELTA results for various
purposes.
 It will feed into the teaching and learning process with forward
perspective to 2012.
 It will create the opportunity to engage in collaborative research.
PolyU assessment outline for 4-year curriculum
Entry
Exit
What is DELTA?
An English language proficiency assessment system
designed specifically for the Hong Kong academic context
which can




diagnose students’ strengths and weaknesses
track and report on students’ language gains
help students to plan their language learning
inform curriculum development
Assessment Format

Four components - Listening, Reading, Grammar and
Vocabulary

Academic & general text-based

Multiple-choice items

Items calibrated using Item Response Theory
(Winsteps) analysis

Assessment targeted to each student’s proficiency level
DELTA Track
DELTA Track and Component Skills Profile
COMPONENT SKILLS PROFILE
Component Diagnostic Report
Grammar
Difficulty
Subskills




















phrasal verb
gerund
part of speech
pronoun
present perfect
comparative
conjunction
possessive
gerund
passive
simple present
word form
infinitive
preposition
part of speech
modal
pronoun
reflexive pronoun
past continuous tense
infinitive
Overall proficiency level = DELTA 88
The report for Grammar above indicates subskills to work on. Click on the relevant subskill to find
resources to help you to improve.
The DELTA system
The DELTA System
ENROLMENT /
REGISTRATION
student-specific test selection
tracking
TEST DELIVERY
automated scoring & test analysis
REPORTING
report generation & transmission
Learning Management System (Moodle) and/or e-Portfolio
DELTA system (admin)
Customizable set up
 Test setup (venue, sessions, enrolment period, etc.)
 Item database (active / inactive items)
 Test difficulty level for groups of students
 Student report content
Management
 Enrolment and attendance
 Item database
 Results (Export to Winsteps for analysis)
New student (1st attempt of DELTA)
Completes
a generic
test
Receives
email
reminder to
register
Receives an email that
results are available
Sets DELTA
target (with the
advice of a
language
teacher)
2nd year student (2nd attempt of DELTA)
DELTA TRACK
200
Completes
a targeted
test
(targeted to
proficiency
level)
Receives an email that
results are available
180
160
D
E 140
L
T 120
A
(Re)sets
DELTA target
(with the advice
of a language
teacher)
M 100
E
A
80
S
U
R 60
E
40
Receives
email
reminder to
register
20
Seeks language support to
achieve target
0
0
1
2
3
ATTEMPT
4
Applications of DELTA
@ CITYU
Relevance to English Language
Teaching and Learning at CityU
Students
-Language support (individualised)
-Tracking information
Providers of
language courses
-Placement (students with no
HKDSE/equivalent)
-Formative information
Parent
departments &
senior
management
-Needs report
-Tracking information
-Overall
-Per college/school
DELTA & English support for CityU
students from 2012
Year
DSE Eng Level
4/+ at entry
DSE Eng Level
3 at entry
DELTA
4
DELTA
3
DELTA
2
English
English
DELTA
1
English
English
English
Predicted number of test takers
2011
approx 1300 (selected Year 1 students)
2012
approx 5200 (all Year 1 students) +
approx 1300 (from 2011)
2013
approx 2600 (all Year 1 students) +
approx 5200 (from 2012) +
approx 1300 (from 2011)
http://elws2.cityu.edu.hk/Indep
endentLearning/writing2.html
DELTA
report
Independent
learning
guidance
Language
learning plan
http://elws2.cityu.edu.hk/Indep
endentLearning/writing2.html
Independent
learning
guidance
Language
learning
plan
Meeting
with a
Language
Learning
Advisor
Applications of DELTA
@Lingnan
Application of DELTA at Lingnan
The DELTA system at Lingnan University is informing and
interacting with the teaching-learning process by:
 Providing additional information to help in curriculum
planning and decision making
 Promoting learner autonomy and guiding students in
independent learning
 Providing valuable input to both students and faculty for
academic advising
Curriculum Planning and Decision Making
Planning and developing the 2012 curriculum
All new courses in 2012 will contain significant Independent
Learning (IL) component in which students will plan and
execute their individualised learning pathways with help
from DELTA student reports
CEAL will track language proficiency development of all
students by making the DELTA test mandatory once per year
CEAL currently anticipating proficiency targets and how to
work with parent departments
Promoting Learner Autonomy and Independent Learning
DELTA student reports enable students and CEAL staff to identify:
specific language subskills for IL work
resources that need to be developed
COMPONENT SKILLS PROFILE
Input to provide information and help in Academic
Advising
 Lingnan currently using Degree Works as a tool for
advisors and students in academic advising
 Students and staff able to discuss English language
proficiency, development and targets
 CEAL hopes to work with academic advisors to enable
better understanding and more effective use of DELTA
student reports
Sample student report in DegreeWorks
The DELTA Team
Centre for Language in Education (HKIEd)
Michelle Raquel, Gwendoline Guan
Centre for English and Additional Languages (LU)
Christine Shirley, Jerry Gray
English Language Centre (PolyU)
Alan Urmston, Carrie Tsang, Felicia Fang, Cherry Ng
English Language Centre (CityU)
Jane Lockwood, Eva Lui
External Consultant
Professor Trevor Bond, James Cook University, Australia