Project: Enabling Visible and Effective Learning in Engineering
Tom Baldock (project lead), David Callaghan, Mehmet Kizl, David Knight, Liza O’Moore, Jan Meyer
Project website: www.civil.uq.edu.au/teachingandlearning
The Meyer et al (2012) AAEE conference paper reports on the first stage of this two-year project; in particular on
modelling patterns of variation in student learning across the first three years of the engineering degree, the initial
successful trialling of an established mechanism1 (see www.RoLIsps.com) to develop students’ metalearning capacity
(framed against an engineering education literature review), with some immediate evidence of consequent selfinitiated qualitative change in students’ learning behaviour. The theoretical model of student learning underpinning
this work is represented by the constructs embedded in the domain of the Reflections on Learning Inventory (RoLI)
which was administered in four different courses across the first three years of study2.
Current work. A triangulation methodology has been used to identify threshold concepts in courses in which we
engage academic staff, students, and analysis of assessment data, the purpose being to adjust curricula, pedagogy,
and assessments accordingly. Work to date has focussed on the critical features of the threshold concept of critical
flow in a third-year course on fluid mechanics. As Baldock puts it: ...critical flow is the Threshold Concept, leading to
understanding of hydraulic controls being the threshold capability, the consequence of which leads to what we
actually need, which is student capability to analyse gradual varied flow (gvf) in any circumstances.
Contributing to the triangulation methodology were (a) Academic Staff. To assist the analysis of the threshold
concept the two teachers involved independently created embedment concept maps (using C-map) for comparison
and discussion3. (b) Students. In written accounts there was wide variation in the manner in which students
approached learning the threshold concept. And, consistent with theoretical expectations, students who used
deeper forms of learning (e.g., linking back to previously understood concepts) expected to receive higher marks
than those who relied on repetition and memorisation. Students also indicated that knowing how, and when, to
apply different concepts in problem solving was troublesome. In responding to this observation a repertoire of
metacognitive activities related to assessment has been developed, one of which has been productively trialled at
third-year level. (c) Assessments. Using concept-grouped subsections of assessments to predict total exam scores is
one empirical method of identifying candidate threshold concepts. In addition, linking in-semester assessments to
performance on various concepts on the exam (e.g., linking a practical on gradually varied flow to a set of exam
questions on gradually varied flow) has provided useful data on how to adjust within-semester assignments. If an
assignment has little predictive power, better alignment is required between the activity, anticipated learning, and
the measure of student performance (i.e., actual learning). Open-ended and multiple choice-style assessments have
been analysed to provide feedback directly to students with respect to their conceptual understanding as well as
providing data-driven justification to teachers to adjust curricula, pedagogy, and future assessments4.
1
This online mechanism enables individual students to construct, for reflective purposes, a visual representation of their own
learning behaviour in a given learning context. Linking associated metalearning activity to assessment proved effective in
boosting voluntary participation by students. For further details on the modelling and metalearning aspects of the Project
contact jan.meyer@uq.edu.au
2
Conceptually contrasting patterns of variation were exhibited in the aggregate (group-level) analysis of individual learning
behaviour data. Evident across the year groups, in particular, were persistent patterns of (variation in) reproductive and/or
pathological forms of ‘learning’ that signalled an opportunity, indeed a need, to promote learning shifts towards deep-level,
integrative, forms of learning linked to the conceptual understanding of material. Evaluative experiences were solicited in one
third-year group in which 84% of students found the metalearning activities useful, just under 60% indicated they would rate the
course more favourably because of the focus on learning, and 0% would rate the course more negatively. In general, students
indicated that the metalearning activities showed them how important it is to understand topics as opposed to relying on
memorisation without understanding, and many students noted that they changed how they listened in class, took notes, or
studied following such activities. Following the recommendations of student reflections and focus groups we intend in 2013 to
further embed metalearning activities in a semester 1 course soon after students begin at UQ, and again in a Year 2 course.
3
For further details contact t.baldock@uq.edu.au or dave.callaghan@uq.edu.au
4
For further details contact db.knight@uq.edu.au