Workplace Learning
The Evidence for Teaching,
Learning and Assessing in the
Workplace
Karen V. Mann PhD
1
Overview
1.
2.
3.
4.
5.
What are the assumptions of WL?
What are the advantages and
limitations?
What is the role of the teacher in WL?
What is the role of the learner in WL?
Future directions
2
Workplace learning

Broad term, covers a range of topics

Young and old

Evolving our understanding of apprenticeship
3
Workplace Learning

Deliberate learning

Physicians need to “go beyond the mere
accumulation of experiences” (van de Wiel et al.,
2011, p. 83)



Often overlooked as a valuable tool
Essential for the improvement of
trainees’ performance
Life long practice
4
Assumptions
1.
2.
Experience
Supervision
5
Experience

Empowering and encouraging environment


Trainee must feel supported to engage in experiential
opportunities (van der Zwet et al., 2010)
Opportunities to participate (e.g., clerkships) in
environments that challenge the trainee and
provide quality encounters
(Eraut, 2011; Pearson & Lucas, 2011; van der Zwet et al., 2010)

Participation is motivating and encourages
feeling like a doctor (Dornan et al., 2007)
6
Motivation in the Learning
Environment: An Example
Experience-based learning in undergraduate students
•
Participation was central to students’ learning
•
Supported participation
–
–
–
•
Affective
Pedagogic
Organizational
Participation at all levels:
–
–
–
–
Boosted confidence
Increased motivation
Enhanced professional identity
Increased confidence
(Dornan, et al, 2007)
7
Experience
’We have definitely learned a lot from it
that will be useful in our final exams
and beyond. The downside of being
here...it would be nice to have a little
bit more clinical experience on our
own, rather than just sitting in with
the GPs.’ (Fifth-year medical student)
(Pearson & Lucas, 2011)
8
Experience


Learning from the problems
encountered (van de Wiel et al., 2011)
Clinical learning occurs through
engagement with many professional
groups and levels of experience
through a respectful manner (Pearson & Lucas,
2011)
9
Experience
Several factors influence experience in workplace learning
 Enthusiasm and commitment of teacher
 Presence of clear objectives and guidance
 Patient cases / clinical work



Immediate problems vs. gaps in skills/knowledge
Cooperation of colleagues, supervisors and other
specialists
Recognition and respect when contributing to the
team
The quality of supervision (r = 0.73, p < .01),
patient mix (r = 0.65, p < .01), and independence
(r = 0.48, p < .01) impact instructional quality
(van der Zwet et al., 2010)
10
Adequate supervision
a.
Assessment
-Supervisory assessment and self-assessment
b.
Observation
-Mini-clinical evaluation and direction observation
of skills
c.
Discussion
-Clinical cases, case based discussion
d.
Feedback
-Peers, coworkers & patients; multi-source
feedback; objective tools- mini-PAT, TAB &
PSQ
11
Assessment




Evaluation of skills and practices undertaken
in day-to-day work
Assessment supports development of
competencies
Competency does not always predict
performance
Trainees’ concern about meeting expectations
of teachers (Eraut, 2011)
12
Assessment

How does assessment change learning?


Credibility and Relevancy
Multiple sources of feedback for trainees to
reflect upon
‘Certainly the more respect I had for them the more
negative feedback and the more straightforward feedback I
could tolerate from them.’ (P6)
’…after I told the patient the bad news, then the family told
me, “You seem cold or kind of not compassionate.” I took
that very seriously’ (P3)
(Watling et al., 2012)
13
Observation




Direct observation provides the data for
feedback (Ryan et al., 2010)
Faculty, Student and System issues can
influence the ability to observe or be
observed (Kogan et al, 2011)
Faculty need support to develop skill in
observation and feedback (Fromme et al., 2009)
Observation and feedback requires planning
and intention
14
Discussion

Consulting with colleagues/peers on a
case




Incidental learning
Who to approach?
Coming to a common ground on
differences of opinions
Opportunities to ‘learn to talk’, and
‘learn from talk’ (Lave and Wenger,1991)
15
Discussion

Difficult cases discussed in daily or
special review meetings help solve
medical problems and provides
alternative solutions for trainees
’I see many routine cases. Occasionally you have to look
things up. Occasionally it’s good to test your own opinion
against those of others. That’s what the Friday afternoon
patient review meetings are for,…our strength is that we’re a
team; by talking to each other, we improve our level.’
(van de Wiel et al., 2011)
16
Feedback
Providing Feedback

Critical in guiding/adjusting the trainee to
desired outcomes and positive impact on
doctors’ learning & performance
(Eraut, 2011; Miller, 2010; Norcini & Burch, 2007; van de Wiel
et al., 2011)

Consistent feedback from a credible source
can change clinical performance
(Veloski et al., 2006; Watling et al., 2010)
17
Feedback
Receiving feedback


Multisource feedback is the most evaluated form of
feedback, but there is conflicting evidence on its
effectiveness (Miller, 2010)
Effectiveness of feedback for the learner depends on
credibility of advice and frequency of negative feedback
(Eraut, 2011)

Integrating/evaluating is dependent on who is
providing the feedback (e.g. Patient vs. Peer)

Feedback that is incongruent with self-assessment may
be rejected and evoke strong emotional response
(Sargeant et al, 2009)
Patients rated doctors significantly higher on performance
than their colleagues (4.34 vs. 3.69, t(df, 66)=7.7, p < 0.001)
(Archer & McAvoy, 2011)
18
Feedback
Seeking & incorporating feedback

Learning and change can occur when
feedback indicates a need for change


Positive interpretation of feedback and
believing change is possible improves
learning and growth
Most often sought when immediate care
needs to be provided (van de Wiel et al., 2011)
19
Tensions in seeking and using
feedback
•
•
•
Between people
• Wishing to learn and improve vs. wishing
to appear knowledgeable and confident
among peers and superiors
Tensions in learning environment
• Providing genuine assessment feedback
vs. “playing the evaluation game”
Tensions within self
• Wanting feedback yet fearing
disconfirming information
(Mann et al, 2011)
20
Advantages of Workplace Learning



Contributes to professional and personal
growth, and increases confidence in
ability (Lester & Costley, 2010)
Benefits the organization
Allows for learning at both the individual
and the collective level (Billett 2004)
21
Limitations of Work-base learning



Reciprocal relationship between
environment and learner
Often no definitive guide for facilitating
WBL
Assessment and feedback may be
influenced by interpersonal relationships
(Norcini & Burch, 2007; Watling et al., 2010)

Faculty development, and learner
support are key to success
22
Role of the Teacher

Offer







comfortable and safe working and learning
environments
multiple opportunities to engage in practice
meaningful and emotional encounters with
patients
observation
purposeful and relevant feedback
trust and value in trainees’ competencies
and skills
care, commitment and enthusiasm for
23
teaching
Role of the teacher
’[The GPs] will say you need to know that, go
away and read that and it is nice because
you do not often get that. People just go
‘you should know this’. Well there is a lot of
things that I should know but to actually be
given a title and certain key things to go
away and learn is extraordinarily helpful.’
(Fifth-year medical student, Pearson &
Lucas, 2011)
Role of the Teacher

Encourage:





critical thinking
questioning and seeking answers
knowledge sharing
engagement in cases and learning
opportunities
checking/discussing mutual
understandings (e.g., coming to a
common solution)
25
Motivation in the Learning
Environment: An Example
How residents learn



Feedback is essential to motivation
Learning and performance goal
orientation
Instrumental and supportive
leadership
(Teunissen et al., 2009)
26
Preceptors’ Influence on Motivation
Instrumental Leadership
Supportive Leadership
Clear Goals
Structured Work
Guidelines
Friendly, Approachable
Considerate
More Feedback Seeking
Increased Feedback
Provision and Value
More Feedback Seeking
Increased Feedback
Provision and Value
(Teunissen, et al., 2009)
27
Informal Learning and the Hidden
Curriculum
•
•
•
•
•
•
Loss of idealism
Adoption of a “ritualized” professional identity
Emotional neutralization
Change of ethical integrity
Acceptance of hierarchy
Learning of less formal aspects of “good
doctoring”
(Lempp & Seale, 2004)
28
The Hidden Curriculum
•
Power and hierarchy
•
Patient
•
Emerging
accountability
dehumanization
•
Balance and sacrifice
•
Hidden assessments
•
Faking it
•
Emotional
•
Human connection
suppression
•
Limits of medicine
(Gaufberg et al., 2010)
29
How is Caring Learned?

Professional and pedagogical caring

A vehicle for integrating scientific and human
aspects of professional practice

Interactions between teachers and students

Long term effects of caring
(Cavanaugh, 2002; Haidet et al., 2006)
30
Faculty’s Role in Teaching
Humanistic Behaviour

Faculty selected by residents as role models &
teachers of humanistic care

Observations revealed non-verbal behaviours:
•
•
•
•
Demonstration of respect, overtly and frequently
Building a personal connection
Eliciting patients’ emotional responses to illness
Demonstration of self-awareness and reflection
(Weissman et al., 2006)
31
The Student-Supervisor Relationship
The student-supervisor relationship functions to:
•
•
•
•
•
Facilitate direct transmission of patient-centred
knowledge, skills and attitudes
Provide social support for the student’s patient-centred
behaviour
Provide support of the student as person
Mirror patient-centered behaviour by being learnercentred
Address supervisor vulnerability
(Bombeke et al. , 2010)
32
Role of the learner


Engage in WL to learn and to improve
patient care
 Unsure of solution and/or gap in skills
 Practical experience triggers reflections
and problem solving
Self-assessment (Eraut, 2011; van de
Wiel et al, 2011)
 Seeking feedback is related to trainees’
ability to adequately assess their own
performance
33
Role of the learner

Provide support to peers and
encourage discussion and
exchange of information

Informal peer learning was strong and
important in practice
‘…you get to compare your level of knowledge with
your peers and also.. .you know, you can teach them
and they can teach you as well...’ (Pearson & Lucas,
2011,)
34
Where Is the Learner’s Voice?


Residents in Family Medicine viewed receiving
specific, timely, and frequent feedback as key
to making the CBAS a worthwhile investment
for them.
Feedback encouraged their own self-assessment
and motivation to fill gaps.

Faculty development is critical

Learner development and orientation are critical
(Ross et al, 2012)
35
Future Direction

Little work has been done to empirically assess the
effectiveness of work-based learning on doctors’
performance and learning

Need to understand what environments facilitate and
support WBL

Future work should look at incorporating mixed
methodologies and focus on the quality of the data

Literature is dominated by theoretical studies (van de

What they ‘should’ do vs. what they ‘actually’ do
Wiel et al., 2011)
36
Summary
1.
2.
3.
4.
5.
What are the assumptions of WL?
What are the advantages and
limitations?
What is the role of the teacher in WL?
What is the role of the learner in WL?
Future directions
37
Thank you!
Your comments and questions?
References









Archer, J. (2010). State of the science in health professional education: effective feedback. Med Teach,
29: 855-71.
Archer, J. & McAvoy, P. (2011). Factors that might undermine the validity of patient and multi-source
feedback. Med Edu, 45: 856-93.
Billett S. (2004). Workplace participatory practices: conceptualising workplaces as learning
environments. J Workplace Lear, 16:312–24.
Bombeke K, Symons L, Debaene L, et al. (2010). Help: I’m losing patient centredness! Experiences of
medical students and their teachers. Medical Education, 44: 662-673.
Cavanaugh SH. (2002). Professional caring in the curriculum. In Norman G, van der Vleuten CPM &
Newble D (eds.) International Handbook of Research in Medical Education. Dordecht: Kluwer, pp. 981996.
Dornan T, Boshuizen H, King N, Scherpier A (2007). Experience-based learning: a model linking the
processess and outcomes of medical students’ wokplace learning. Med Educ 41:84-91
Eraut, M. (2011). Informal learning in the workplace: evidence on the real value of work-based learning
(WBL). Development and learning in Organizations, 25(5): 8 -12.
Fromme, H. B., Karani, R., & Downing, S. (2009). Direct observation in medical education: Review of
the literature and evidence for validity. Mount Sinai School of Medicine, 76: 365 0 371.
Lave J, Wenger E (1991). Situated Learning. Legitimate Peripheral Participation. Cambridge, UK:
Cambridge University Press.
39
References









Gaufberg L, Batalden M, Sands R & Bell SK. (2010). The hidden curriculum: what can we learn from
student narrative reflections? Academic Medicine, 85: 1709-1716.
Haidet P, Kelly PA, Bentley S, et al. (2006). Not the same everywhere: patient-centered learning
environment of nine medical schools. Journal of General Internal Medicine, 21: 405-409.
Lester, S. & Costley, C. (2010). Work-base learning at higher education level: value, practice and
critique. Studies in Higher Education
Lempp H, Seale C. (2004). The hidden curriculum in undergraduate medical education: qualitative
study of medical students’ perception of teaching. British Medical Journal , 329: 770-773.
Kogan, J., Conforti, L., Bernabeo, E., Iobst, W & Holmboe, E. (2011). Opening the black box of clinical
skills assessment via observation: a conceptual model. Med Edu, 45L 1048-1060.
Mann K, van der Vleuten C, Eva K, Armson H, Chesluk B et al. (2011). Tensions in informed selfassessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med
86:1120-1127.
Miller, A. (2010). Impact of workplace based assessment on doctors’ education and performance: a
systematic review. BMJ, 341(c5064).
Norcini, J., & Burch, V. (2007). Workplace-based assessment as an educational tool: AMEE Guide No
31. Med Teach, 29: 855-71.
Pearson, D., & Lucas, B. (2011). Engagement and opportunity in clinical learning: Findings from a case
study in primary care. Medical Teacher, 33: e670-77.
40
References









Ross, S., Poth, C., Donoff, M., Papile, C., Humphries, P. et al. (2012). Involving users in the refinement of
the competency-based achievement system: An innovative approach to competency-based assessment.
Med Teach, 34(2): e143-147.
Ryan, J., Barlas, D., & Sharma, M. (2010). Direct observation evaluations by emergency medicine faculty
do not provide data that enhance resident assessment when compared to summative quarterly
evaluations. Academic Emergency Medicine, 17: S72 - 77.
Teunissen P, Stapel D, van der Vleuten C, Scherpbier A, Boor K, et al. (2009). Who wants feedback? An
investigation of the variables influencing residents’ feedback seeking behaviour in relation to night shifts.
Acad Med 84:910-917.
van der Vleuten, C (1996). The assessment of professional competence: development, research and
practical implications. Adv Health Sci Educ, 1: 41-67.
van de Wiel, M., van den Bossche, P., Janssen, S. & Jossberger, H. (2011). Exploring deliberate practice
in medicine: how do physicians learn in the workplace? Adv in Heal Sci Educ, 16: 81-95.
van der Zwet, J., Zwietering, P., Teunissen, P., van der Vleuten, C. & Scherpbier, A. (2010). Workplace
learning in general practice: Supervision, patient mix and independence emerge from the black box once
again. Med Teach, 32: e294-99.
Veloski, J., Boex, J., Grasberger, M., Evans, A., & Wolfson, D. (2006). Systematic review of the literature
on assessment, feedback and physicians’ clinical performance: BEME Guide No 7. Med Teach, 28: 117-28.
Watling, C., Driesen, E., van der Vleuten, C., & Lingard, L. (2012). Learning from clinical work: the roles
of learning cues and credibility judgments. Medical education, 46: 192-200.
Weissman P, Branch W, et al. (2006). Role modeling humanistic behavior: leaning bedside manner from
41
the experts. Academic Medicine 81: 661-667.