Carmel McNaught
Why the Scholarship of
T&L Matters in the Sciences
Creative commons license
1
Plan of the presentation
1.
2.
3.
4.
5.
6.
7.
8.
Pressure and complexity
Scholarship of teaching and learning
Evaluation, research, & evaluation research
Paradigms & mixed methods
LEPO framework
An example of a good learning design
Implications for institutional policy & processes
Summary of principles
http://www.redstarresume.com.au/uploads/images/thumbs/thumbs/thumbs/The_Plan1.jpg
2
1. Pressure and
complexity
3
http://tell.fll.purdue.edu/JapanProj/FLClipart/Adjectives/busy.gif
http://tinyurl.com/5svpmtk
4
http://www.activeedgept.com.au/application/uploads/assets/cache/active-reward-img.png
A moment of reflection
1. What are the key
priorities in teaching?
2. What are the
moments that bring a
smile of satisfaction
to our faces?
3. How can Science
teachers remain true
to their role in the
face of external
pressures?
D. Kennedy
5
Do we need a genuine
paradigm shift?
http://ircamera.as.arizona.edu/NatSci102/NatSci102/images/paradigm-shift.gif
6
2. Scholarship of
teaching and
learning
7
Student
learning
needs
Aims/
desired
learning
outcomes
Learning
activities
Feedback
for
evaluation
Content/
fundamental
concepts
Assessment
Actual
learning
outcomes
8
Student
learning
needs
Aims/
desired
learning
outcomes
Overall
area of
research
Learning
activities
Specific
research
questions
Feedback
for
evaluation
Content/
fundamental
concepts
Assessment
Actual
learning
outcomes
Research
instruments
& methods
Research
data
Disciplinary
research
conventions
Research
analysis
Findings &
discussion
9
Scholarship of Teaching and
Learning (SoTL)
https://my.vanderbilt.edu/sotl/files/2013/08/circles.png
10
Boyer scholarships
 The scholarship of discovery – traditional
disciplined-based research
 The scholarship of integration – connections
across disciplines and contexts
 The scholarship of application – professional
and community-oriented research
 The scholarship of teaching – where the
principles of scholarly inquiry are applied to
planning and implementing teaching.
Boyer (1990)
11
A problem!
Traditional value structure in
universities
An equitable value structure for
universities
Scholarship of discovery
more important
than
Scholarship
of discovery
Scholarship
of integration
Scholarship
of application
Scholarship
of teaching
Scholarship of Integration
more important
than
Scholarship of application
more important
than
Scholarship of teaching
McNaught (2014)
12
3. Evaluation,
research, &
evaluation
research
13
Argument for evaluation research
 Engagement of teachers in
reflection and change requires …
 Evidence which requires …
 Evaluation data.
 This process is scholarly inquiry
and hence the term evaluation
research.
 Also, the way we teach is a model
for how we want our students to
learn.
http://www.cosmas.in/images/inquiry.jpg
14
15
Phillips, McNaught, & Kennedy (2011)
‘Good’ evaluation research
The ‘SOLO’ Taxonomy
Structure of Observed
Learning Outcomes
State
Recognise
Recall
Tell
Enumerate
Describe
List
Clarify
Do algorithms
Compare/
contrast
Explain causes
Analyse
Relate
Apply
Predict
Theorise
Generalise
Hypothesise
Reflect
Create
Design
Misses point
Missing the point
Single point
Multiple
unrelated points
Quantitative phase
Logically related
answer
Unanticipated
extension
Qualitative phase
After (Biggs & Collis, 1982; Biggs, 2003; McNaught, Cheng, & Lam, 2006)
A framework for academic writing
Writer's
block
Stuck at Postit Notes Stage
State
Recognise
Recall
Tell
Google
anyone!
Enumerate
Describe
List
Clarify
Do algorithms
An OK
conference/
B journal paper
Compare/
contrast
Explain causes
Analyse
Relate
Apply
Predict
The paper you
want to write!
Theorise
Generalise
Hypothesise
Reflect
Create
Design
Misses point
Missing the point
Single point
Multiple
unrelated points
Logically related
answer
Unanticipated
extension
17
An example of how the SOLO Taxonomy can
be used to assess students’ writing
Explanation of SOLO
categories (Biggs & Collis,
1982)
Simplified
classification
categories
Type of writing
Misses the point
Non-substantive
•
Social
Substantive
•
•
Adding new points
Enhancement and clarification
of points
•
Making clear contrary
statements
Developing complex
arguments
Single point
Multiple unrelated points
Logically related answer
Elaborated substantive
Unanticipated extension
•
18
http://carmodyconsulting.net/db4/00314/carmodyconsulting.net/_uimages/jigsawpuzzle.jpg
19
Phillips, McNaught, & Kennedy (2011)
4. Paradigms &
mixed methods
20
Some heavy terms …
Paradigm
Equivalent terms
Positivist
(Postpositivism)
scientific, objective, empirical, quantitative, analytic,
hypothetical deductive, functionalist
Interpretivist
relativist, constructivist, naturalistic, humanistic,
hermeneutic, systemic, qualitative
Critical
theory
neomarxist, postmodern, inductive, praxis, feminist
Pragmatic
critical realist, eclectic mixed methods, disciplined
eclectic
21
Phillips, McNaught, & Kennedy (2011)
Design-based research
Predictive research
Hypotheses based
upon observations
and/or existing
theories
Experiments
designed to test
hypotheses
Theory refinement
based on test
results
Application of
theory by
practitioners
Specification of new hypotheses
Design-based research
Analysis of practical
problems by researchers &
practitioners in
collaboration
Development of solutions
informed by existing
design principles &
technological innovations
Iterative cycles of
testing & refinement
of solutions in
practice
Reflection to produce
'design principles' &
enhance solution
implementation
Refinement of problems, solutions, methods and design principles
After Reeves (2006)
22
5. LEPO framework
23
LEPO framework
Phillips, McNaught, & Kennedy (2011)
24
Forum
transcripts
X
X
X
• How do learners use the
online environment in
order to learn?
X
X
Log data
Learner
reflective
reports
• What sort of team work
occurs?
Specific evaluationresearch questions
Staff interview
Learner group
interview
Example of a simple evaluationresearch matrix
X
X
25
Document
Review
Interviews
Surveys
E-r matrix for effectiveness research of
learning outcomes
•
•
•
• To what extent have learners achieved the expected learning
•
outcomes after engaging with the e-learning environment?
•
•
Data sources
Questions
• What knowledge, skills or conceptual understanding have
learners developed?
• Are improvements apparent in those assessment(s) that are
•
directly based on the desired learning outcomes?
• What unintended learning outcomes have occurred?
• Are any observed benefits widespread or limited to some
learners?
•
•
•
26
6. An example of a
good learning
design
27
One example that our students like
 Support comes from peers
 The following example has been
used successfully in HK for many
years & is still in use.
Mohan & Lam (2005)
http://growthmindseteaz.org/sitebuilder/images/giveup-249x450.jpg
28
Key principle 1
We are all busy!
http://tell.fll.purdue.edu/JapanProj/FLClipart/Adjectives/busy.gif
So, it’s time to ...
http://julianfranklin.com/SimpleMachinesLogoSM.jpg
And ensure that students, NOT
teachers, do the work …
29
Overview of L4U Model
case 1
STAGE 1:
Warming Up
Identify
key
issues,
create
hypothesis …
STAGE 2:
Digging In
case 2
Peer
evaluation &
assessments …
Minders’
Feedback
Find appropriate
resources …
STAGE 4:
Evaluation &
Assessment
Solving
the
case …
STAGE 3:
Working Out
30
7. Implications for
institutional
policy & processes
31
Key principle 2
Rewards: Annual reviews, promotion
policies & processes
 Organizational
change is a tricky
business!
 But it can be done …
 Compromises,
balancing competing
views …
 E.g. how does one
‘count’ research?
http://tinyurl.com/comm2resp
32
Polarity theory: How does one ‘count’
research?
Research quality improving conceptual
understanding
Research innovation new strategies to explore
understanding
Quality in research
productivity
Research ethics strengthen human
values in research
AND rather
than OR
Research innovation seeking new funding
models
Johnson (1992)
Research ethics curbing plagiarism
Quantity in research
productivity
Research quality journal metrics
33
8. Summary of
principles
34
http://tinyurl.com/princsfuture
 Keep your core principles alive and in your conscious
mind.
 Conduct theoretically-based evaluation research.
 Design-based research seems like a good option.
 Articulate clear goals & evaluation-research questions.
 Adopt methodologies & methods of data collection which
are appropriate to the goals of the evaluation research.
 Conduct evaluation research as an integral part of
teaching.
 Use evaluation-research evidence in institutional and other
professional settings.
35
Patience is a virtue
 Effective change takes
time.
 Monitoring over time is
needed for evidence to
be convincing.
http://i197.photobucket.com/albums/aa259/odea_photos/626time.jpg
Thank You
37
References 1
 Biggs, J. B. (2003). Teaching for quality learning at university (2nd ed.).
Buckingham: Society for Research into Higher Education & Open University
Press.
 Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: the
SOLO taxonomy (structure of the observed learning outcome). New York:
Academic Press.
 Boyer, E. L. (1990). Scholarship reconsidered. Priorities of the professoriate.
Princeton, New Jersey: The Carnegie Foundation for the Advancement of
Teaching.
 Johnson, B. (1992). Polarity management: Identifying and managing
unsolvable problems. Amherst Mass.: HRD Press.
 McNaught, C. (2014). Inducting postgraduate students into a global
academic community. Invited essay. The Journal of KAUIR&WL, Linguistic
and intercultural communication series. 3–4(17–18) 4–13.
 McNaught, C., Cheng, K. F., & Lam, P. (2006). Developing evidence-based
criteria for the design and use of online forums in higher education in Hong
Kong. In N. Lambropoulos & P. Zaphiris (Eds.). User-centered design of
online learning communities, pp. 161–184. Hershey, PA: Information
Science Publishing.
38
References 2
 Mohan, J., & Lam, P. (2005). Learning for Understanding: A web-based
model for inquisitive peer-review learning activities. In G. Richards & P.
Kommers (Eds.), ED-MEDIA 2005 (pp. 2083–2090), Proceedings of the
17th annual World Conference on Educational Multimedia, Hypermedia &
Telecommunications, Montreal, Canada, 27 June–2 July. Norfolk VA:
Association for the Advancement of Computers in Education.
 Phillips, R. A., McNaught, C., & Kennedy, G. E. (2011). Evaluating elearning: Guiding research and practice. New York & London: Routledge.
 Reeves, T. C. (2006). Design research from a technology perspective. In J.
van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.),
Educational design research (pp. 52–66). London: Routledge.
39