Diagnostic Testing : only the
beginning
A component in a longer process
CDC Steele
University of Manchester
University of Manchester
• Dates back as far as 1824 by several
different routes but took its present form in
2004.
•
•
•
•
•
35 000 students
29 000 Home/EU
6 000 Overseas
26 000 UG
9 000 PG
University of Manchester
• 11 500 staff
• 4 000 academic
• 2 000 research
• 341 Buildings
University of Manchester
• Four Faculties
– Engineering and Physical Sciences
– Life Sciences
– Medical Sciences
– Humanities
University of Manchester
• School of Mathematics
– 90 academic staff
– 400-500 mathematics students per year
Service courses to ~ 1000 students
in various parts of the University.
– Aerospace
Engineering
– Civil Engineering
– Mechanical
Engineering
– Chemical
Engineering
– Electrical
Engineering
– Chemistry
– Optometry
– Computer Science
– Materials Science
– Foundation Year
Entry Profiles of Maths Students
• A-levels
– C1, C2, C3, C4
– 2 more modules from Mechanics (M1, M2 etc,)
Statistics (S1, S2 etc), Decision (D1, D2 etc)
– Many students students AS- or A-level in
Further Maths
– Generally require A+ in maths A-level
Entry Profiles of Service Students
• A-levels
– C1, C2, C3, C4
– 2 more modules from Mechanics (M1, M2 etc,)
Statistics (S1, S2 etc), Decision (D1, D2 etc)
– Some students AS- or A-level in Further Maths
Entry Profiles of Students
• A-levels. Most Engineering schools ask for
Grade B in maths (part of ABB etc)
• Other UK Qualifications
• International or European Qualifications
• University of Manchester Foundation Year
Degree Structure
• Most Students : 3 year degree. (BSc, BEng
etc)
• Some able students : 4 year undergraduate
masters, M Math, M Eng etc
• Each year : 2 semesters : Sept – Jan and Feb
– June. 12 weeks teaching plus exams at end
of semester.
The streamed system for
Engineers : 1996-2009
• Engineers in several
schools ‘streamed’
into P-stream, Q/Mstream, R-stream.
• Streaming follows
diagnostic test but
not only outcome of
diagnostic test
P – stream.
‘Elite’
Q/M – stream.
‘Standard’
R – stream. ‘At
Risk’
Typical P-stream student
•
•
•
•
A at A-level or equivalent
Strong on sections C1, C2, C3
May be weak on some C4 material
May also be weak on a little earlier
material.
Typical Q-stream student
•
•
•
•
•
B,C at A-level or equivalent
Reasonable on sections C1, C2,
May be weak on some C3 material
Not terribly good on C4 material
May also be weak on a little earlier
material.
Typical R-stream student
• May not have officially satisfied entry offer
• May have large areas of maths which are
causing difficulty.
The diagnostic and followup
programmes
• A means to measure weaknesses and
concentrate efforts of students in the early
weeks towards areas where they may
have underachieved in the diagnostic test
– Rusty
– Not time for teaching
– Genuine difficulty with topic.
– May not have been taught certain material
Pre-diagnostic mailshot
• Mock diagnostic posted to prospective
students mid-August including
– Mock test paper
– Answers
– Other possible questions
– Links to paper and web-based resources
describing the topics.
– Computerised Practice Questions
Diagnostic Test
• A test in Freshers’ week designed to
measure student profiles across various
mathematical topics.
Diagnostic Testing at
UMIST/Manchester
• Carried out since 1996
• 1997-2001 : 24 questions (6 sections) in 40
minutes
• 2002-2009 : 48 questions (12 sections) in
80 minutes
• Change following curriculum 2000.
• Total mark less important than series of
‘mini-marks’ for various sections.
Diagnostic Testing at
UMIST/Manchester
•
•
•
•
Carried out since 1996
1996-2004 Short answers
2005-2008 Multiple-choice (optical reader)
2009 Multiple-choice (manual entry)
• Carried out in Freshers’ week (week 0).
Different times for different schools etc.
Questions and Answer box 19962004
Questions 2005-2009
Diagnostic Test : Sections
• A : Arithmetic and
Algebra
• B : Geometry and Trig
• C : Sequences
• D : Functions and
Algebra
• E : Polynomials
• F : Logarithms and
Exponentials
• G : Simple
Differentiation
• H : Simple Integration
• I : Rational Functions
• J : Further
Differentiation
• K : Further Integration
• L : Vectors
A-H : C1, C2, C3 Material : I-L : C4 material
Diagnostic Followup Programme
• For students on P- and Q- streams.
• R-stream could itself be considered as a
followup
• Allocates up to 2 sections from A-L for Pstream, up to 2 sections from A-H for Qstream.
Diagnostic Followup Programme
• Allocates up to two sections based on
marks of 0/4, 1/4, or 2/4.
• Sections A (Arithmetic and Algebra) and D
(Functions and Algebra) allocated
preferentially, then in alphabetical order.
• Students told of relevant sections via website towards end of week 1 of semester.
Diagnostic Followup Programme
• Students informed of
– Relevant section(s)
– References to relevant materials
– Includes HELM Resources
– Pro-active and re-active sessions in
Mathematics Resource Centre.
– Practice ‘tests’ on computer.
– Leading up to
Diagnostic Followup Programme
• Computerised test in week 4 of semester
on relevant topic(s)
• Similar to practice tests.
• Exercise counts 20% towards coursework
for relevant course unit (itself worth 20% of
unit assessment).
Diagnostic Followup Programme
• HELM Resources
• HEFCE FDTL4 project 2002-2005
• Workbooks on various mathematical
topics.
• Workbooks on
mathematical topics
Example
Engineering Application
Diagnostic Followup Programme
• Manchester Mathematics Resource
Centre.
– Support Centre for students to drop into with
Mathematical Enquiries.
– Also runs pro-active sessions centring on
material on Diagnostic Followup.
– Centre is within an Engineering
Building but open to
ALL students.
Diagnostic Followup Programme
• Computer-assessment
• Use of STACK, developed by Dr Chris
Sangwin, University of Birmingham
• Accepts mathematical input
• Gives specific feedback
• Varies coefficients sensibly
Diagnostic Followup Programme
Diagnostic Followup Programme
Results 2007
A
Stud Diag Follo
ents Test wup
75
83
90
G
Stud Diag Follo
ents Test wup
3
89
53
B
114
67
84
H
12
69
83
C
184
38
90
I
18
49
88
D
163
65
89
J
24
49
87
E
41
70
93
K
29
87
F
24
60
95
L
69
92
Results 2008
A
Stud Diag Follo
ents Test wup
58
89
78
G
Stud Diag Follo
ents Test wup
62
76
88
B
166
70
84
H
47
70
74
C
179
65
87
I
117
46
67
D
181
82
89
J
84
43
68
E
66
78
80
K
67
32
53
F
117
64
86
L
45
62
88
Diagnostic Results 2002-09
Longer Consequences of
Diagnostic Test
• Practice Tests, HELM Materials, Resource
Centre all there for further use etc.
Purposes of Diagnostic Test
• Used to assign students to lecture course
groupings (until now)
• Used to assign students to tutorial groups
• Used to inform staff of entry profiles of
students
• Used to assign each student an
individualised programme of work
Diagnostic Followup for Maths
Students
• Same procedure up to diagnostic test i.e.
mailshot, practice questions,
• Followup made available to maths
students but not feeding into coursework.
• Students told of weaker sections and
encouraged to take part in followup on
these.
• Can be used as ‘indicator’ of students who
may or may not be engaging with the
course.
Other Diagnostic Tests
University of Manchester
Foundation Year
• Students from countries with an
educational system which stops slightly
short of level for entry to 1st year
• Students desiring a change of direction
• Students not quite reaching the level for
entry to 1st year
University of Manchester
Foundation Year
• Test B
• 40 minute test.
• Three sections from Test A plus 3 sections
on more basic work
• Practice questions provided in advance
plus opportunities to try questions
electronically.
• Used to inform staff, to allocate students to
lecture courses and tutorial groups etc.
Materials Science
• Cannot ask for Mathematics A-level or
equivalent. Major variations in arrival
mathematics profile
Materials Science
• Test D : 50 minutes. 8 sections i.e. six
from Test B test plus two more from test A.
• Used to decide which students need to go
in group for more intensive sessions.
Late arrivals
• Can be a problem with compulsory
followup.
• Test session a week or two later etc.
• ‘Envelope’-based test system.
Conclusions
• Students arrive with potential weaknesses
in some areas.
• Take Diagnostic test and weak areas are
identified
• Diagnostic Followup addresses areas of
weakness
• Students can continue to access materials
after exercise officially finishes.