Career learning for the 21st
century
CPD series for Community Learning Champions
Module 5b:
Introduction to evaluating and measuring impact in career development 2 –
extension activities
1
Contents
Introduction to the group work workshop
3
Tutor notes
Workshop outline
Workshop aim and objectives
Checklist of resources required
LSIS workshop – trainer’s feedback
LSIS workshop – attendee’s feedback
3
3
4
4
14
16
[Table of contents will be created automatically by designer – you can
ignore]
Download
This CPD module is available to download in PDF format from the Excellence Gateway Career
Development section.
Publisher
Published by the Learning and Skills Improvement Service (LSIS).
© LSIS [month year]
Publication reference: LSIS[xxx]
2
Introduction to evaluating and measuring impact in
career development workshop 2
Tutor notes
Workshop outline
This session is an extension activity to the evaluation and measuring impact in career development
workshop. It is aimed at CLCs who want to actively develop their understanding of evaluation and impact
measurement. This session will give CLC schemes the opportunity to involve their CLCs in developing a
framework for measuring impact and continual improvement that meets the needs of their scheme.
Participants will actively work on identifying appropriate measures of impact and can work with scheme
managers/coordinators to agree improvements to current systems. Participants should be asked to bring
along the handouts from the group work workshop, in particular the copies of the PowerPoint slides.
This module can be delivered by anyone who is an experienced or qualified training professional if they
take the time to familiarise themselves with the materials including the web links, handouts and
references. It would, however, be an advantage if the trainer had experience and/or training in career
development and knowledge of the CLC programme, particularly in relation to those CLCs attending the
workshop. This will enable wider and deeper exploration of the topics covered in relation to the CLC roles
of participants.
The materials have been designed to be delivered as they stand, contextualised or modified to suit the
time available and the needs of the participants, scheme or organisation. The PowerPoint slides and
handouts are available as separate files.
This workshop is an extension to the fifth in a series of CPD workshops produced by LSIS and available
to download from the LSIS Excellence Gateway. There are six others in the series:

An introduction to career development

Introduction to Interviewing

Introduction to values and ethics in career development

Introduction to delivering career development through group work

Introduction to reaching potential by raising aspirations

Introduction to the Blueprint for Careers
LSIS has also produced a series of 10 workshops for people within the FE and Skills sector who support
learners with their career development which are also available to download from the LSIS Excellence
Gateway. These may be relevant for CLC schemes specifically working in this area or for those CLCs
who wish to do more in depth work around a particular topic. These are available to download from the
LSIS Excellence Gateway.
LSIS have also developed an online resource, “Career learning for all,” available free of charge on the
LSIS virtual Learning Environment just go to www.leadershiplearning.org.uk and create your own login.
This has been designed for those involved in supporting learners with their career development, both
specialists and non-specialists.
3
Workshop aim and objectives
The aim of the workshop is to develop a framework for measuring impact and continual
improvement that meets the needs of their scheme.
The objectives of the workshop are to enable participants to:



Relate evaluation and impact to their own context
Identify areas of their scheme that can be measured to demonstrate impact
Contribute to the development of an effective framework for evaluation and measuring impact that
meets the needs of their scheme
Example workshop timetable
Time
Session/s
13.30
Welcome, Domestics, Introduction, aim and objectives,
13.40
Impact of career development and introductions
13.50
Career development audit
14.05
How can we improve
14.25
Measuring impact
14.35
Break
14.50
Ways to measure impact
15.05
Developing a framework
15.45
Conclusion
15.55
End
Checklist of resources required
Fire evacuation procedures.
PowerPoint™ slides 1 to 9.
Flip chart and marker pens, or whiteboard.
Screen.
LCD projector.
Laptop.
Copies of handouts 1 and 2.
Evaluation form: LSIS workshop – trainer’s feedback (see page 14).
Evaluation form: LSIS workshop – attendee’s feedback (see page 16).
4
Introduction to evaluating and measuring impact in
career development
Presented by –
Date –
Just to identify strengths and areas to improve are no longer enough. An
organisation needs to be self knowing and have a robust continuous
improvement plan - a plan that is reflected in all areas of the organisation.
Ofstead: September 2009 framework
Slide 1
Customise the visual to include your own details and date of the workshop
1.
Present the domestic arrangements and fire safety for the venue
2.
Introduce yourself and the session – this is a follow up session to the introduction to measuring
impact
3.
Remind the group that we looked at this quote in the introductory session. Ask group what they
remember about continuous improvement – what does it mean?
Resources: Fire evacuation procedures and copies of PowerPoint slides
Timing: 5 minutes
Transition statement: Let’s look at the aim and objectives of this workshop.
Your notes:
5
Aim and objectives
Aim:
To develop a framework for measuring impact and
continual improvement that meets the needs of your
schemes.
Objectives
By the end of the session learners will be able to:
•Relate evaluation and impact to your own context
•Identify areas of your scheme that can be measured to
demonstrate impact
•Contribute to the development of an effective framework
for evaluation and measuring impact that meets the needs
of your scheme
Slide 2
Introduce the overall aim of the workshop.
Relate this aim back to the work we did in the introductory session.
Go through the objectives and explain that we will be looking at starting the process of developing
effective and robust systems to monitor, evaluate and measure the impact of the work they do as CLCs.
Ask if there are any questions.
Resources:
Timing: 5 minutes
Transition statement: We are going to start by revisiting some of the things we looked at in the
introductory session.
Your notes:
6
Impact of career development
Work in pairs, preferably with someone you do not know well.
Introduce yourselves: name, job role, experience in evaluating
and measuring impact of the work you do as a Community
Learning Champion.
Agree between you the outcomes and impact that should result
from delivering a consistent and high quality service to the
people and communities we work with and support.
You have 5 minutes for this activity
Slide 3
Ask the group to split into pairs and try to ensure they work with people they do not know well if this is
possible. Talk through the slide ensuring everyone is aware of what is required. It may be helpful to
encourage participants to think out ‘outcomes and impact’ as the difference their work should/could make
to both individuals and communities.
When the 5 minutes are up, display the list they came up with during the introductory session. Since that
session they will have been thinking more about evaluation and impact. Have they come up with anything
new? Do they now want to change anything on the original list? Anything they want to remove?
Amend the original list in light of discussions and agree as a group which ones benefit the individuals,
which benefit the CLCs as volunteers and which benefit the organisation/scheme as a whole. This has
been a good reminder of the potential impact of the career development work we do and how it could be
measured.
Resources: Original list of suggestions from introductory session
Timing: 10 minutes. 5 minutes for the activity and 5 minutes feedback and discussion
Transition statement: Let’s remind ourselves of how we think our schemes are performing.
Your notes:
7
Completing the career
development audit
Activity
Individually go through the audit tool and make judgments
relating to career development processes and practices within
your scheme/organisation
In fours, compare your assessments, noting similarities and
differences
Slide 4
Before showing slide:
Give out blank audit tool and remind participants that we did some work on this at end of last workshop
looking at how you thought your scheme was performing against the statements on the audit.
We are going to revisit that exercise and then take it further to begin to develop a system for measuring
the impact of the work you do as CLCs.
Activity
Ask the group to work individually and then in fours. Ask group to do this without reference to the audit
sheet they completed in the introductory workshop. The group should have been thinking about
measuring their impact in the interim period so there may be some very different responses!


Individually review each of the areas included in the audit tool for their organisation
As a group, compare and contrast their assessments 5 minutes
5 minutes
Take feedback from the group, drawing out on a flip chart the areas where schemes are effectively
evaluating career development provision and areas where there are gaps.
Resources: Copies of handout 1 - ‘Audit tool’, flipchart and markers
Timing: 15 minutes - 5 minutes set up and individual work, 5 minutes discussion in fours, 5 minutes for
feedback and discussion.
Transition statement: We will now look at how we might improve some of the areas where we have
identified gaps.
Your Notes:
8
How can we improve?
Capacity to Improve
• The ability of an organisation to continue improving standards
and progress, based on what it has accomplished so far or to
maintain exceptionally high standards.
Activity
• Following from your audit, select one area that your
scheme/organisation needs to improve and then identify a
range of interventions that could be considered in order to
make this improvement
Slide 5
Ask the participants to read the definition given and ask them on why they think continuous improvement
is important. Possible answers include:





An ongoing and relevant view of strengths and development needs
Evidence of monitoring and review
Allows for development of clear detailed plans for improvement for the future based on lessons from
the past
Provides evidence of improvement
Ensures a quality service is being delivered and maintained
Activity
Ask the groups of four to split into pairs and to take one area from each of their audits that they have
identified as an area of improvement.
NB: You may wish to allocate areas to each group to ensure all areas of weakness are covered.
Ask them to consider 2 or 3 interventions that could be implemented in order to bring about improvement.
Ask them to put these on a few flipcharts using the headings below. Each pair should pass on the
flipchart to the next to add their findings.
Area/practice that needs improving
Interventions that could be considered
Organise the flipcharts to be displayed in the room.
Ask each pair to comment on why they have chosen their interventions. If a large group, ask the group to
read the flipcharts and comment on patterns, themes and learning from the exercise.
Resources: Flipchart paper, markers
Timing: 20 minutes - 10 minutes setup and discussion in pairs and 10 minutes feedback
Transition statement: We need to be able to measure impact both qualitatively and quantitatively.
Your Notes:
9
Measuring impact
•
The impact that effective CLC career development activity has
will reflect in the outcomes for clients. For example in:
•
•
•
•
•
•
•
•
Taking up learning (informal or formal)
Skills development
Increased wellbeing
Improved confidence and self esteem
Improved employability
Achievement
Progression
Gaining accreditation
Measuring impact is about setting clear measures in each of
these areas and being able to track this to the effectiveness of the
career development activity. Please remember that measures can
be both quantitative and qualitative
Slide 6
Before showing slide 6 ask the group for their ideas about what client outcomes would help to
demonstrate the impact of their scheme’s career development activities. Write these up on flip chart as
suggested by the group.
Show slide 6 and say that this shows some examples of how CLC career development activity could have
significant impact on clients and consequently on the others involved in heir scheme, stakeholders,
communities and their organisations/ institutions. All schemes will be different. How do the examples on
the slide differ from the list the group came up with? Are there any here not on your list that might be
relevant to your scheme?
Go through and discuss each outcome identified as relevant and ensure that individuals understand the
final statement.
Resources:
Timing: 10 minutes
Transition statement: Now we’ve identified what outcomes would help demonstrate impact let’s look at
how we might be able to measure these.
Your Notes:
10
Ways to measure impact
There are many ways to measure impact and it’s important to
chose methods that suit what you do, how you do it and your
capacity/resources.
•
•
•
•
•
•
•
•
Clients forms ; application, client data, equailty monitoring, etc
Evaluation forms
Discussion groups
Postal surveys
Email surveys
Online surveys such as ‘Survey Monkey’
Telephone surveys
Longitudinal surveys
To be effective we need to select the right methods and talk to the
right people; including clients, stakeholders, partners, staff and
volunteers.
Slide 7
Before showing slide 7 ask the group for their ideas about different ways we could gather information that
would help us measure how effective we’ve been in achieving the outcomes for clients that we’ve
identified. Who should we be asking? Write these up on flip chart as suggested by the group.
Show slide 7 and say that this shows some examples of different ways we can gather the information we
need. How do the examples on the slide differ from the list the group came up with?
Give out handout 2 - Ways to measure
Discuss the pros and cons of each method and how effective each might be within their scheme.
Resources: Copies of handout 2 – ‘Ways to measure’.
Timing: 5 minutes introduction and initial discussion. 10 minutes for discussing pros and cons of different
methods.
Transition statement: Now we have a clear idea of what we need to measure and have identified how
we might best do this within out scheme, let’s have a go at coming up with some measurement tools for
your scheme.
Your Notes:
11
Measuring the impact of your
scheme’s career development
activities
Activity
In groups of four, choose one of the client outcomes that could
be measured quantiviely.
Selct an appropriate way to measure this and develop a tool that
could be used in your scheme. For example: a survey
questionnaire, terms of reference for a focus group, list of
questions for a partner telephone survey, a new evaluation form.
A member of each group will feedback at the end of the activity:
What you chose and why, brief outline of your ideas.
Slide 8
Before showing slide 8 tell the group that we are going to put this into practice and give them the
opportunity to develop some impact measurement tools that could be used in their scheme.
Show the slide and make sure everyone understands the task. Tell the group that it’s very likely they will
not get a final ‘version’ completed today but that draft/outline ideas will be developed (perhaps within a
CLC working group).
Resources: pens and paper.
Timing: 5 minutes introduction and initial discussion. 25 minutes for the activity 10 minutes feedback.
Transition statement: Let’s look at what we’ve covered today
Your notes:
12
Summary
•
•
•
What impact could effective career development activity have
on clients?
What did we identify as suitable methods for gathering the
information we need within our schemes?
Who do we need to talk to/ask to get the information we
need?
Thank you for attending, please complete an evaluation form
Slide 9
Go through the questions and if necessary revisit slides if any areas of difficulty arise. Ask for any
outstanding questions.
Give out the evaluations and ask for these to be completed.
Thank the group and close the session.
Resources: Evaluation forms
Timing: Five minutes for review and five minutes for evaluation and close.
Transition statement: Thanks for attending and for your input in helping us to begin to develop robust
systems for evaluation. All ideas from the session will be typed up and circuited.
Note: If schemes are considering a CLC task group to focus on this area then invite participants to
express interest in being part of this to take this work forward.
Your Notes:
13
LSIS workshop – trainer’s feedback
Please use this form to feed back how useful you found the training materials provided. This is
not to measure the success or otherwise of your workshop, but to help us improve the quality of
the materials provided.
Name of workshop: [insert name]
Date of workshop:
Using a rating of 1 = very good to 7 = very poor, please answer the following questions:
Rating
Overall how happy were you with the materials?
How well did the course meets its aim and objectives?
How would you rate the handouts (if applicable)?
How would you rate the PowerPoint™ presentation?
How would you rate the exercises?
How would you rate the trainer’s background information?
Which parts did you find most useful?
Which parts did you find the least useful?
What do you think could be done to improve the materials?
Please comment on the suitability of the timings provided.
Any other comments?
14
15
LSIS workshop – attendee’s feedback
Name of workshop: [insert name]
Date of workshop:
Using a rating of 1 = very good to 7 = very poor, please answer the following questions:
Overall how useful did you find the course?
How would you rate the materials provided?
How would you rate the trainer?
How useful were the exercises?
How would you rate the pace of the course?
How would you rate the length of the course?
Which parts did you find most useful?
Which parts did you find the least useful?
What do you think could be done to improve the workshop?
Any other comments?
16