So how are you going?
Exploring Evaluation and Monitoring
A/Professor Deborah West
Office of Learning and Teaching
WHY?
• Continual improvement for student outcomes
• To become better teachers
• Help us to understand if we are doing the right
things to improve retention
• Can also link to teaching and learning citations,
awards, promotion process
What is SoTL?
The scholarship of learning and teaching refers to a
systematic approach to reflecting on teaching practice to
improve outcomes for students. It has a number of key
features:
• Focus on improving student learning
• Systematic approach
• Focus on action/impact – doing something with the
knowledge (not just research for research sake)
• Dissemination/sharing/peer review
Exploring Evaluation and Monitoring| 1 May 2015| Slide 3
Key steps in SoTL
1. Using a theory, model or framework to ground the
initiative and provide the justification of action
2. Identifying an intervention
3. Formulating an investigative question
4. Conducting an investigation
5. Produce a result in the form of a public artefact
6. Inviting peer review/dissemination
Exploring Evaluation and Monitoring| 1 May 2015| Slide 4
Activity
• Brainstorm on your table (and write them down on the
paper provided) the sources of data/evidence that you
might be able to use to reflect on your unit, the curriculum
and teaching and learning more broadly (any aspect)
Exploring Evaluation and Monitoring| 1 May 2015| Slide 5
How?
• Learnline tools
– Unit reports
– Performance dashboard
– Survey tool
– Statistics tracking
– Item analysis on tests
• My View
• Grade Reports
• Student Focus groups
• Peer review
• Moderation
• Benchmarking/Calibration
Exploring Evaluation and Monitoring| 1 May 2015| Slide 6
Sally Kift’s Transition Pedagogy
http://transitionpedagogy.com/
Exploring Evaluation and Monitoring| 1 May 2015| Slide 7
Transition
http://transitionpedagogy.com
The first year curriculum explicitly assists transition
academically and socially into learning in higher
education and the new discipline.
• Good first year curriculum design aids transition from
a student’s previous educational experience to the
nature of learning in higher education and their new
discipline as part of their life long learning journey.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 8
Diversity
http://transitionpedagogy.com
The first year curriculum embraces and supports the diversity and
reality of students’ backgrounds, previous experiences and
preparedness for university.
• Good first year curriculum design acknowledges student diversity
because diversity may exacerbate transition issues and, in the
current massified sector, few assumptions can be made about
students’ entering knowledge, skills and attitudes. Diversity in this
context is very broad and includes, for example, membership of atrisk or equity groups, non-traditional cohorts, and students’ various
patterns and timing of engagement with the first year curriculum
(e.g., mid-year entry).
Exploring Evaluation and Monitoring| 1 May 2015| Slide 9
Design
http://transitionpedagogy.com
The first year curriculum is designed intentionally
for commencing students, based on evidence from
practice and research.
• Good first year curriculum design is student-focused,
explicit and relevant and provides the foundation and
scaffolding for learning success. The first year
curriculum objectives should be articulated and,
desirably, the first year curriculum should form a
coherent, integrated whole.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 10
Engagement
http://transitionpedagogy.com
The first year curriculum incorporates pedagogies, teaching approaches
and materials that engage students in their learning and facilitates
interactions with peers and staff.
Good first year curriculum design enacts an engaging and involving pedagogy.
AUSSE (ACER, 2008) now provides us with very clear guidance around the
“activities and conditions likely to generate high quality learning” (ACER, 2008,
vi) and evidences that “all aspects of engagement have a strong positive
relationship with a range of general, specific, social, personal, ethical and
interpersonal capabilities” (ACER, 2008, ix). Pascarella & Terenzini (2005, 646)
record that –
With striking consistency, studies show that innovative, active,
collaborative, and constructivist instructional approaches shape learning
more powerfully, in some forms by substantial margins, than do
conventional lecture-discussion and text-based approaches.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 11
Assessment
http://transitionpedagogy.com
The first year curriculum aids transition to higher
education assessment and provides early feedback.
• Good first year curriculum design aids students’ transition to
higher education assessment, introduces a range of
appropriate assessment practices and provides early
feedback on student progress to students and staff.
Assessment increases in complexity from first to later years.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 12
Evaluation and Monitoring
http://transitionpedagogy.com
The first year curriculum should itself be evaluated and
should monitor for student engagement.
• Good first year curriculum design is itself evidence-based
and evaluated, and desirably includes mechanisms to
monitor for student engagement and performance, the latter
allowing then for timely intervention in aid of students who
are at risk of not being successful.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 13
Activity
Where do our data sources fit with Sally’s model?
On the handout put in the various sources of data that
might help you with the key elements.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 14
Activity: Asking the right questions
Individually think about some questions that you could add to
the My View survey in two groups:
• That fit with the model in various domains
• Other questions that you would like to ask student to help
you improve your unit and/or your teaching.
Write your questions on yellow sticky notes
Discuss your questions with others on the table considering
question structure, appropriateness etc.
Exploring Evaluation and Monitoring| 1 May 2015| Slide 15
What Next?
•
•
•
•
Think about it
Prepare
Plan
Think about the approach and the
questions
• Be consistent
Exploring Evaluation and Monitoring| 1 May 2015| Slide 16