A guide to
tracking
learner
destinations
A guide to tracking
learner destinations
© NIACE 2014
Published by the National Institute of Adult Continuing Education
(England and Wales)
21 De Montfort Street
Leicester LE1 7GE
Company registration no. 2603322
Charity registration no. 1002775
NIACE is the National Institute of Adult Continuing Education, the national voice for
lifelong learning. We are an internationally respected development organisation and
think-tank, working on issues central to the economic renewal of the UK, particularly in
the political economy, education and learning, public policy and regeneration fields.
www.niace.org.uk
Follow NIACE on Twitter:
@NIACEhq
@NIACECymru (Wales)
@NIACEbooks (Publications)
All rights reserved. No reproduction, copy or transmission of this publication may be
made without the written permission of the publishers, save in accordance with the
provisions of the Copyright, Designs and Patents Act 1988, or under the terms of any
licence permitting limited copying issued by the Copyright Licensing Agency.
Contents
Foreword
5
Introduction
7
Why track and why track now?
7
Benefits to providers from tracking learner destinations
9
The development and purpose of this guide
The challenges of tracking
10
12
How the challenge depends on the outcome criteria
12
How the challenge depends on the level of evidence and learner
verification required
14
How the challenge depends on whether third party corroboration is required
14
The implications for resourcing tracking
16
Preparing to track
18
Data systems
18
Staffing used within tracking
20
Securing learner buy-in
21
Personalising the process
23
Choosing when to track
23
Choosing the method(s) and the order in which they are used
25
Can other organisations help?
28
Tracking by telephone
29
The effectiveness of tracking by telephone
29
The process of telephone tracking
30
The timing of the telephone call
31
Using text messaging
32
Issues and solutions when undertaking telephone tracking
33
Tracking through e-approaches
34
The process of tracking by e-approaches
34
The design of the e-message
35
The effectiveness of tracking by e-approaches
36
Issues and solutions when using e-approaches
37
A guide to tracking learner destinations
Tracking by post
38
The process of tracking by post
38
The design of the postal message
39
The effectiveness of tracking by post
40
Issues and solutions relating when tracking by post
41
Tracking through face-to-face contact
42
The process of tracking through face-to-face contact
42
The effectiveness of tracking through face-to-face contact
44
Issues and solutions when undertaking tracking through face-to-face contact
44
Conclusions and recommendations
45
Annex 1 A ten-point strategy for successful tracking
47
Annex 2 The survey methodology and questions
48
Annex 3 The trial methodology
54
Annex 4 Tracking tools (similar to those used in the trial)
57
Annex 5 Acknowledgements
60
Foreword
In the past providers have not been required to collect destination data, but recently
more and more providers have put in place measures to track and record the
destinations of their learners. This is primarily because of the introduction of ‘job
outcome payments’, brought in to compensate providers who lose funding when
learners leave their course for employment. Additionally, funding from the European
Social Fund, City Deal and Department for Work and Pensions is already often based on
job outcomes.
Policy makers and providers are beginning to recognise that while qualifications are an
important recognition of achievement, they may not reflect the full range of policy
outcomes, particularly those that are employment related. This piece of work was
commissioned by the Department for Business, Innovation & Skills to explore how
providers are already tracking learners’ destinations, and to try and understand the
issues facing providers. Following this, 15 providers took part in a trial which involved
tracking learners by telephone, post and electronic methods.
The result of the work is this guide to tracking learners’ destinations, which will be of
interest to providers who want to develop or extend learner destination tracking. It will
also be of interest to policy makers, funders and other stakeholders wishing to know
more about the process of destination tracking. NIACE believes that tracking the
destination of learners can be challenging, but with imaginative, contextualised
approaches implemented in a systematic manner, it will be a valuable tool. We
recommend a mixed-method approach through which a provider successively uses
several of the methods described in this guide.
Knowing where our learners are going when they leave provision is necessary, not only
for funding purposes but also to ensure that we are aware of where, how and why
students leave courses.
Carol Taylor OBE
Director for Development and Research, NIACE
5
Introduction
NIACE is the National Institute of Adult Continuing Education, the national voice for
lifelong learning. We are a nationally respected development organisation and thinktank, working on issues central to the economic renewal of the UK, particularly in the
political economy, education and learning, public policy and regeneration fields.
Most people learn for a purpose. Outcomes from learning are important to learners and
should be important to learning providers. Although tracking outcomes have not always
been required in the past, providers are increasingly expected to know the destinations
of their learners. With the recent government announcement that it is considering how
funding can be more strongly linked to outcomes in the future, it is important that
providers now establish the means to track destinations.
As well as meeting the needs of learners and funders there are benefits to providers as
destination data can inform the evolution of provision, support equalities analyses and
provide robust quantitative evidence which proves the value of learning when lobbying
for funding. However, we recognise that tracking learner destinations is challenging. It
requires maintaining links with learners after they have left provision, securing their
cooperation in revealing their destinations, systems for processing and storing
destinations data and sometimes corroborating this data with third parties such as
employers. The Department for Business, Innovation & Skills (BIS) therefore
commissioned this guide to support providers, which we hope will be useful.
Destination tracking is continuously evolving. The guide provides a snapshot of current
approaches. These will evolve further; automated digital approaches and greater data
sharing between organisations working in partnership could considerably improve
results and make tracking more affordable. But providers must not wait for these
developments. They must endeavour to start tracking all their learners from now on.
Why track and why track now?
Skills and qualification outputs from learning are, for most learners, just a means to an
end. Learners have different ends in mind but they commonly wish to enter
employment, progress in work, become self-employed or know how to better support
their family and community.
Funding through the Adult Skills Budget has been to date primarily based on learning
participation and qualification achievement. Therefore this has not required providers to
track and record destination outcomes. However, many providers have begun to track
outcomes for at least some of their learners because:
7
A guide to tracking learner destinations
•
in 2012/13, the Skills Funding Agency introduced job outcome payments to
compensate providers for their loss of qualification achievement funding when
learners abandon learning to start employment; and
European Social Fund, City Deal and Department for Work and Pensions (DWP)
funding is already often based on job outcomes.
•
In its latest Skills Funding Statement1 the government announced that it is also keen to
explore how funding through the Adult Skills Budget might be based on outcomes by
stating that:
❝Qualifications are an important recognition of achievement. However, we
recognise that they do not reflect the full range of policy outcomes we are
looking for. We have put in place measures to capture learner destinations
on the Individualised Learner Record (ILR) as we consider how funding can
be more strongly linked to outcomes in the future. We expect providers to
place a greater emphasis in 2014/15 on capturing destination
information.
❞
Key message for providers
The government have announced that it is keen to explore how funding through the
Adult Skills Budget might be based on outcomes.
As well as capturing destination information to possibly support outcome funding, the
government intends to develop a set of outcome-focused measures that will enable
learners and employers to make more informed choices about which provider they use
and which qualification they take. The Skills Funding Statement,1 BIS, 2014, announced
that the government has been working on three core measures: destination (into further
learning or into/within employment), progression (through learning), and earnings
changes (following completion of learning).
❝All provision for those aged 19 and over funded by the Skills Funding Agency
❞
(through the ASB, Offenders Learning and Skills Services, or Loans) is in
scope including Apprenticeships and Traineeships.
In Getting the Job Done: The Government’s Reform Plan for Vocational Qualifications,2
BIS announced that it intends to publish experimental data at provider level in July
2014. The Reform Plan3 also announced that BIS is exploring how matched data,
derived from linking the skills, benefits and tax data systems, can be presented at
2 Skills Funding Agency (2014) Skills Funding Statement 2013–2016.
3 BIS (2014) Getting the Job Done: The Government’s Reform Plan for Vocational Qualifications. At:
https://www.gov.uk/government/publications/vocational-qualification-reform-plan
8
A guide to tracking learner destinations
qualification level. However, there are legal and technical constraints on how this
matched data can be used, in particular at an individual level. Therefore it cannot fully
replace the direct value and utility of providers tracking their learners’ destinations.
Consequently, in March 2014, the Skills Funding Agency published a second version of
its guide to data collection requirements for individual learner records (ILR) for
2014/15,4 which outlines new criteria for the collection and recording of learner
destination and progression data:
❝A new entity has been added to the ILR for 2014 to 2015 to record
destination and progression outcomes for a learner, such as gaining
employment or going onto further study. These outcomes will usually be
reported after a learner has completed a programme of learning.
❞
Ofsted5 is also keen to have recourse to destinations data, saying:
❝The ability to judge the true effectiveness of provision will depend, among
other things, on the availability of robust data on learners’ destinations.❞
Benefits to providers from tracking learner destinations
Aside from meeting government policy objectives, enabling learners and employers to
make more informed choices and making inspections more robust, learner destination
tracking is beneficial for providers as:
•
•
•
•
•
robust quantitative evidence of positive outcomes will enable providers to prove the
value of learning when lobbying for funding;
destination data can inform the evolution of provision;
destination data enables more effective equalities analyses;
tracking outcomes facilitates re-engagement of past learners for further learning; and
tracking outcomes facilitates employer engagement for future work-based learning.
Key message to providers
There are considerable benefits to providers from tracking learner outcomes.
4 Ibid.
5 Skills Funding Agency (2014) Specification of the Individualised Learner Record for 2014 to 2015,
Version 2. At: www.gov.uk/government/publications/ilr-specification-validation-rules-and-appendices2014-to-2015
9
A guide to tracking learner destinations
The development and purpose of this guide
The policy changes and benefits described above mean the time has come for all
learning providers to develop the means to routinely track the destinations of all their
learners. This is less daunting for some providers than others. Providers already funded
through the European Social Fund, City Deal and the DWP may simply be able to extend
their destination tracking arrangements for these outcome-funded forms of provision, to
other forms of provision. Some providers delivering the Unit Offer for the Unemployed
may be able to extend job outcome tracking systems established through the 2.5 per
cent of their Adult Skills Budget that they were required to set aside in 2011/12 to build
capacity to deliver skills provision for unemployed learners. However, providers who have
not tracked learner destinations before can find it challenging and, without support, it
can take them a long time to achieve reasonable response rates. Therefore, to equip
themselves rapidly and effectively, it is vital that inexperienced providers learn from the
experience of others. This guide helps inexperienced providers to do that.
Although some providers have been tracking learner destinations for some time, no
research had ever been done on what and how they were tracking. Therefore, to inform
this guide, BIS commissioned NIACE to carry out a provider survey to identify:
•
•
•
•
•
•
•
which learners are being tracked and why;
what destinations are being tracked and why;
what approaches are being taken to track destinations;
the cost of these approaches;
the benefits of tracking these destinations;
how the intelligence providers had acquired through tracking destinations had
influenced their provision; and
how providers developed their capacity to track destinations.
In additon, the survey gave providers an opportunity to identify challenges they faced in
tracking learner destinations and how they might evolve their approaches to improve
their effectiveness. The survey methodology and questions are available in Annex 1.
After the survey, NIACE invited providers to take part in a trial of different approaches
to tracking learner destinations. Fifteen providers took part in the trial, which involved
tracking learners by telephone, post and digital approaches. The trial methodology is
described in Annex 2. Although we intended the trial to comprise providers with varying
experience of tracking destinations, we found that the majority of providers applying to
take part had relatively little experience. However, this meant we learnt more about the
issues facing inexperienced providers as well as more about the methods used in the
trial.
10
A guide to tracking learner destinations
This guide draws upon the results of the survey and the experience and insights of
providers taking part in the trial to provide a description of:
•
•
•
the challenge of tracking learner destinations;
the necessary preparations for tracking; and
methods through which learners can be tracked.
The guide ends with a conclusion in which we make a series of recommendations arising
from our research. The guide will be primarily of interest to providers wishing to begin or
extend learner destination tracking. However, it may also be of interest to policy makers,
funders and other stakeholders wishing to know more about the process of destination
tracking. It is also important to note that at the time of going to press it was likely that
a consultation on funding traineeships, including an outcome element, was imminent.
11
The challenges of tracking
This section looks at the origin of the challenges providers face when tracking
unemployed learner destinations. Comments and quotes taken from providers who
participated in the survey and trial are used throughout this section to illustrate how the
challenge of tracking depends upon:
•
•
•
the outcome criteria requirements;
the evidence and verification requirements; and
whether third party corroboration is required.
The section ends by considering the impact of these challenges on the level of resources
required for tracking.
How the challenge depends on the outcome criteria
When providers are tracking destinations for their own purposes they can set the
outcome criteria for themselves. However, when funding is outcome dependent, learning
providers have to adopt the outcome criteria set by the funder. A large proportion of
providers responding to our survey (63 out of 84) used criteria set by funders when
tracking their learner destinations.
Different funders often stipulate different criteria for the same type of destination, for
instance:
•
•
•
For an Adult Skills Budget job outcome payment, compensating for the loss of
qualification achievement income due to a learner starting work, the job must last 16
hours or more per week for at least four weeks in a row.
For a Work Programme job outcome payment, a participant must have been in
employment and off benefits for a minimum of three or six months, depending on
how far they were deemed to be from the labour market.
For a co-financed ESF 2007–13 job outcome payment the learner must be in
employment that lasts for eight hours or more per week for a minimum of four weeks,
within 13 weeks of leaving the ESF project.
As shown above, job outcome criteria can vary by the number of hours the job lasts, how
many weeks the job has been sustained and how soon the job started after the
provision ended.
Providers have to take account of the outcome criteria when choosing their tracking
method(s). For instance, to track Work Programme participants to record if their job lasts
16 hours or more for a minimum of three or six months and then to track them for a
12
A guide to tracking learner destinations
further 18 months to claim four weekly sustainment payments, many Work Programme
providers have established long-term, ongoing support through which they have the
opportunity to track the participant. This is costly but the only reliable way of helping
participants to sustain their jobs and ensure they stay in touch so they can be tracked.
Key message to providers
Outcome criteria stipulated by funders vary considerably. As a provider’s choice of
tracking approach is influenced by the outcome criteria they are working to,
providers may have to adopt a different tracking approach for each of their different
types of outcome-funded provision.
Some learners have to be tracked against more than one of these criteria for the same
type of outcome; for instance, a learning provider that is also a Work Programme subprime provider may claim an Adult Skills Budget job outcome payment and a Work
Programme job outcome payment if the participant prematurely leaves ASB-funded
skills provision to start a job, as stated in the Funding Rules 2014 to 2015:6
❝If the learner takes part in the Work Programme as well as learning funded
❞
by us, claiming a job outcome payment is not affected by the Work
Programme. That is, this is not considered to be a duplication of funding.
Our survey indicated that providers were not tracking for just one type of outcome. The
majority of providers tracked employment (77 out of 84), self-employment (62 out of
84), progression to further learning (71 out of 84) and volunteering (60 out of 84).
However, less than half the survey respondents tracked progression to job search (29 out
of 84) and inactivity (40 out of 84). The criteria used by providers when tracking
outcomes other than employment was much less demanding, many providers only
requiring learners to have simply started. This may reflect the lower interest of funders
in these non-employment outcomes and that the provider’s interest in tracking the
destination was also satisfied by just knowing the learner had started. The exception
was that many of the providers that tracked participation in job search activities used
quite demanding criteria, i.e. learners taking part for a minimum number of hours per
week and for a minimum number of weeks or sessions. This could be because the often
mandatory nature of participation in job search activity means attendance registers are
often in place, upon which providers can draw. Therefore, providers may be setting
themselves more demanding criteria if the means to acquire the required data to meet
this criteria is relatively straightforward. To track multiple outcomes for each learner
against different criteria requires providers to have recourse to a variety of robust
tracking methods and very versatile management information systems (MIS). The
complexity has obvious resource implications.
6 Ofsted (2013) Ofsted Annual Report 2012/13: Further Education and Skills. At:
www.ofsted.gov.uk/resources/ofsted-annual-report-201213-further-education-and-skills
13
A guide to tracking learner destinations
How the challenge depends on the level of evidence and
learner verification required
As well as setting different criteria for the same types of outcomes, funders require
different levels of evidence and verification for these outcomes. Evidence and
verification requirements vary from a simple verbal learner declaration to a signed
learner statement. For all types of outcome, the majority of providers responding to our
survey only required a simple verbal declaration that destination criteria were met.
When higher levels of evidence are required, this has implications for the tracking
approach, for example telephone tracking can capture a verbal learner declaration but
not a signed statement. There are also implications for the approach providers take to
record, as keeping written statements requires storage.
Key message to providers
Outcome verification and evidence requirements stipulated by funders vary
considerably. As a provider’s choice of tracking approach is influenced by the
verification and evidence requirements they are working to, providers may have to
adopt a different tracking approach for each of their different types of outcomefunded provision.
How the challenge depends on whether third party
corroboration is required
Unless corroborative evidence is sought, destination tracking relies on the honesty of the
learner, either when making a verbal declaration or in signing a written statement.
Presumably because their funders found this unacceptable, 20 out of the 60 providers
surveyed that tracked employment destinations and 26 out of the 49 providers surveyed
that tracked self-employment destinations, had to collect corroborative evidence. This
corroborative evidence took the form of an employer declaration, wage slips, or
employment contact or letter for employment outcomes and HMRC registration or proof
of trading from bank statements and invoices for self-employment outcomes.
Very few providers required corroborative evidence of being off benefits for employment
destinations (two out of 60) or self-employment (three out of 49). This may be due to
data sharing restrictions.
❝...some JCP advisors will confirm verbally that learners have signed off from
❞
benefits and entered work but do not provide anything in writing to confirm for
our records.
Further education college
14
A guide to tracking learner destinations
Providers told us it was relatively straightforward for them to acquire corroborative
evidence of progression to further learning when this was in-house. However, it proved
much more difficult to acquire corroborative evidence of progression to further learning
delivered through other learning providers. For this reason, if acceptable to their funder,
they tended to rely on verbal learner declarations of progression to further learning,
even though they had ample corroborative evidence on their MIS for some of these
learners.
Documentary evidence or signed statements from third parties such as employers or
other training providers clearly provide ideal forms of corroboration but they are much
harder and therefore more expensive to acquire than learner statements.
❝ESF funding is based on evidencing job data through third parties. Not being
❞
able to get this puts the contracts at risk and this is extremely frustrating
when we know learners have jobs but can’t get the evidence.
Further education college provider
Evidencing and validation arrangements for DWP-funded provision do not require
providers to collect forms of third-party corroboration. However, the DWP itself validates
job outcomes and sustained job outcomes by conducting a series of pre-and postpayment checks:
❝The checks include a pre-payment off benefit check on 100% of all job
outcome claims submitted, and a post-payment check for a sample of claims
to ensure that the individual has been in work for the required period of time
and it is in line with the outcome definition. 7
❞
In order for these checks to be feasible the DWP requires providers to submit
considerable contact details:
❝DWP does not prescribe the way in which you should track individuals or
record information about their employment. There is however specific
information that you are required to input to the Provider Referral and
Payment System (PRaP) when you submit the claim. This is:
• whether the individual’s job is in employment or self-employment;
• employer contact details including the address, employer contact name,
full • business telephone number and e-mail address;
• the individual’s contact details;
• job title(s);
7 Skills Funding Agency (2014) Funding Rules 2014 to 2015, Version 2. At:
www.gov.uk/government/publications/sfa-funding-rules
15
A guide to tracking learner destinations
• job start date(s) (Date 1 in PRaP);
• the date the Job Outcome qualified for a claim (Date 2 in PRaP);
• the number of hours worked each week;
❞
• the working/shift pattern;
• an employee identifier such as works or payroll number.
8
The implications for resourcing tracking
Some destination tracking methods such as telephone tracking can be resource
intensive.
❝The experience of trialling these approaches has highlighted the fact that
❞
we will need a dedicated resource to carry out destination tracking in the
future as it took up a lot of staff time.
Adult community learning provider
However other methods can be automated, for instance an MIS will often be able to
automatically send an email to every learner that it identifies as having left between
three months and one month ago. Investing to ensure systems have the necessary
versatility can reduce costs over the long term, as taking an ad hoc approach can be
unduly expensive.
❝Just manually mining the datasets for learner contact details and ensuring
each learner fitted the criteria was a challenge.❞
Adult community learning provider
The costs of tracking generally increase as outcome criteria and evidence requirements
become more stringent. Therefore the affordability of tracking hinges on the balance
between the level of funding and the stringency of the outcome criteria and evidence
requirements. Several providers explained that ESF funding, despite having relatively
stringent criteria for job outcomes, was generous enough to cover the costs the tracking
required.
❝The funding received from ESF delivery allowed the college to implement a
❞
dedicated recruitment and tracking team to assist learners in gaining
employment post their course.
Further education college
8 DWP (2014) Generic Provider Guidance v2, chapter 5. T:
www.gov.uk/government/publications/framework-generic-guidance-provider-guidance
16
A guide to tracking learner destinations
Key message to providers
The affordability of tracking hinges on the balance between the level of funding and
the stringency of the outcome criteria and evidence requirements. Investing to
ensure MISs have the necessary versatility can reduce costs over the long term,
thereby influencing this balance.
17
Preparing to track
This section considers:
•
•
the infrastructure that needs to be in place; and
the preparations that are necessary prior tracking.
In view of the government’s stated policy to require providers to track the destinations
of all their learners, many providers may wish to use 2014/15 to develop a wholeorganisation approach to tracking learner destinations. Ideally, all the necessary internal
infrastructure will be up and running before tracking begins in order to ensure
consistency and efficiency.
Data systems
When asked in our survey, 49 out of 84 providers said they used a learning management
system (LMS) or customer relationship management (CRM) system to store learner
contact details prior to tracking and store learner destinations data following tracking.
Following participation in our trial, some providers were considering redesigning their
MISs to optimise them for tracking the destinations of future learners. Tracking past
learners without MIS modifications had been difficult as learner contact details that
had been acquired for other purposes were often in an unhelpful format and had gaps –
sifting through to retrieve contact details had been time consuming and costly.
Providers hoped that once they made a bigger commitment to tracking learner
destinations, the quality of their contact data would improve.
❝The information on our databases (register) would be much more accurate
❞
and detailed if we routinely tracked all our unemployed learner
destinations.
Adult community learning provider
Providers intended to make a variety of MIS modifications. Some intended to
incorporate a flagging system to alert staff to contact learners at the appropriate time:
❝We would implement a tracking spreadsheet for all learners with a flagging
❞
system to alert us to contact learners after 4 weeks of finishing their course
in order to find out how they are doing.
Adult community learning provider
18
A guide to tracking learner destinations
Others intended or had already established a means through which learners could be
contacted automatically at the appropriate time:
❝We currently automate the destination tracking process through MIS so
❞
that, for example, after a 3 month period an email is automatically sent out
to learners requesting information.
Independent training provider
Another provider has begun to investigate ways in which learners could update their
progression themselves on to a data system:
❝We are looking to enable learners to update their progression information
❞
onto our learner management system... We are looking at more efficient
solutions with regards to this.
Independent training provider
The learners’ responses could also be managed automatically as some CRM systems
can be set up to automatically tag and record emails that are sent and received under a
destination tracking code, thereby providing a means to store outcome evidence in the
form of emailed learner statements.
Many providers used Microsoft® Excel® spreadsheets to coordinate their tracking
activities and record and analyse progress. These spreadsheets could also host
information such as out-of-use telephone numbers and email addresses that led to
email bounce-backs. In some instances, providers linked these Excel® spreadsheets to
MIS/CRM databases as a way of automatically updating learner contact details.
❝The spreadsheet allows us to run a weekly report to share information as
❞
part of a wider approach to our CIS (College Information System) and
databases.
Adult community learning provider
Providers found it helpful to prepare these spreadsheets in advance to ensure they were
compatible with their data systems so that information could be uploaded/imported
easily when the need arose.
19
A guide to tracking learner destinations
Key message to providers
MIS system modifications including those that facilitated automated tracking
approaches are key to the efficiency and affordability of tracking.
Staffing used within tracking
Some methods of destination tracking can require a lot of staff resources. The chart
below shows the different staff types used for tracking by our survey respondents.
Figure 1. Staff used in destination tracking by providers that responded to our
survey
70 60 50 4030 20 No. of providers
De
di
ca
te
d
tra
ck
in
g
of
Ad
fic
m
er
LM
in
ist
Sa
dm rato
r
in
ist
S
ra
Le
ub
to
ct
je
r
ur
c
tl
er
ea
/tu
de
to
r
r/t
Tr
ea
ai
ch
ne
er
r
/
as
Te
se
ac
ss
he
or
ra
Ca ssis
ta
re
er
nt
sa
Tr
ai
dv
ne
ise
d
r
vo
lu
Su
nt
b
co eer
nt
ra
ct
or
0-
Identifying enough staff resource to undertake tracking can prove particularly
challenging for smaller providers as they are more likely to have limited staff capacity.
Several small providers that responded to our survey said they had to resort to using
tutors to track past learners.
While using tutors can lead to a greater response rate as they are known to the learner,
the loss of teaching capacity makes this a rather costly way of securing learner buy-in.
As part of their whole-organisation approach to destination tracking, some providers
established an internal call centre to manage the number of phone calls required.
Depending on the provider context, another solution to destination tracking might be to
consider outsourcing.
20
A guide to tracking learner destinations
❝As a whole organisation approach we use subcontractors (to carry out the
tracking of learners to ensure that no one is missed.❞
Independent training provider
Securing learner buy-in
Because destination tracking requires the cooperation of learners, it is important to
secure their buy-in to the process. This begins with just forewarning them that tracking
will take place after they have left.
❝We inform learners whilst they are still on the course that we will be
contacting them at some point post course.❞
Adult community learning provider
Many learners will respond positively if they know the future of the college or training
provider depends on being able to record the destinations of their learners. Providers
should not be coy about explaining this.
❝It helps to provide the right information and explain why you need the data,
❞
for example how their input will help to secure ESF funding in the future to
run courses.
Adult community learning provider
Providing an explanation also ensures that learners do not jump to the wrong
conclusion:
❝Unemployed learners can be reluctant to give an email address because of
❞
their relationship with JCP. They may feel we are part and parcel of the
same system.
Further education college
Explaining the reason for tracking is also helpful because learners may suffer from cold
call fatigue. As a result, they may routinely not respond to telephone calls from unknown
numbers and may have multiple email addresses set up for spam and junk mail
purposes. If the provider secures their learners’ buy-in, learners are more likely to give
the provider their bona fide email addresses and may save the provider’s telephone
number in their phone’s memory so that they recognise and pick up the tracking call
when it comes.
21
A guide to tracking learner destinations
Raising awareness could take place at strategic points while a learner is attending a
course, for example at the stage of induction. One provider suggested that they would
implement changes to the learner agreement by building in destination tracking as part
of an ‘early intervention’ initiative in order to encourage a greater response from
learners both during the course and on exiting the course.
It also helps to get to know the circumstances of learners as this can help a provider
identify the most effective way to make contact once a learner has left.
❝Plan carefully, and when you are still delivering to the learners that you wish
to subsequently track. Get to know the learners better so that you have more
contact details and can anticipate difficulties in reaching past learners. Pay
attention to convincing people of the value to the community of helping
with our tracking activity.
❞
Adult community learning provider
Learner contact details need to be systematically gathered in anticipation of future
tracking and kept up to date on MIS databases. The probability of reaching a learner
through these contact details deteriorates over time as learners change their email
address and mobile number. Learners’ postal addresses can also change. Ironically, this
is more likely to happen as a result of them finding employment, the destination
providers are most interested in. Refreshing contact details on the day the learner leaves
their provision and gathering multiple forms of contact details will increase the
probability of reaching the learner at a later date.
From our survey it was apparent that providers use a variety of staff to collect learner
contact details and secure learner buy-in. However, because they are trusted, tutors tend
to be best placed to do this. A wide variety of approaches can be used:
❝We would ensure more consultation with learners in order to find out how
they would like to be contacted – perhaps set up a learner focus group for
learners who have left in order to provide the learner voice and to improve
destination tracking methods and approaches, and speak to learners to
discern a variety of approaches. We are currently trialling a learner
newsletter which could include information about destination tracking in
order to raise awareness.
❞
Adult community learning provider
Key message for providers
The success of tracking hinges on learner buy-in.
22
A guide to tracking learner destinations
Personalising the process
Personalising destination tracking methods helps learners recognise the tracking request
when it’s made:
❝We also personalised the form ... we provided additional information about
❞
the learner’s course and their course tutor so that they could identify with
the information in the survey/email in order to encourage a response.
Adult community learning provider
Messages can be personalised by:
•
•
•
•
•
•
•
adding a logo and other visuals to emails;
making reference to the learner’s course;
making reference to any plans the learner described;
mentioning the tutor during the phone call or signing off using the signature of the
learner’s tutor;
using an appropriate tone and language;
using an email address or telephone number that the learner will recognise; and
maintaining a learner focus by asking if they have any questions or further needs.
Providers told us that they had used incentives with varying degrees of success to
encourage learners to share details of their destinations. Incentives were generally small,
often entering learners into a prize draw:
❝We used a draw for a ten pound Tesco voucher before but to get a better
❞
result we might have to offer more like an iPad which would be too
expensive for us.
Adult community learning provider
Key message for providers
Personalising messages to learners is an obvious and inexpensive means of securing
their buy-in.
Choosing when to track
Timings will be determined primarily by the purpose of tracking. For example, when
funded by destination outputs the funder’s timing criteria is always going to determine
the timings of when destination tracking takes place, whereas if a provider is required to
demonstrate the impact of destination tracking for an annual report, the publication
23
A guide to tracking learner destinations
timetable might be the influencing factor. Timings can also be determined by feasibility,
i.e. when capacity is available to track, and the time after which the response rate
becomes too low to make tracking feasible.
When taking a whole-organisation approach to tracking, identifying a shared timeframe
will make it easier to involve other departments and additional cohorts of learners. If the
choice is theirs, many providers prefer to track soon after a course has finished. If left
too long, unless learners are well primed and providers are well prepared, there may be
issues around changes to learner contact details, or learners may question why they are
being contacted after a prolonged period of time which could impact on response rates.
❝...if it has been more than a year since their course, a learner might well ask
why they’re being contacted after all this time in order to provide
information ... We would contact learners much earlier; we like the idea of a
phased approach so that there is more than just one available window of
when we contact learners – for example following learners’ progress over a
set period of time after a course has finished...
❞
Adult community learning provider
Tracking early on may not elicit positive results straight away, it might be too soon.
However, building in a phased approach over a pre-determined timeframe which is
communicated to the learner in advance (part of raising awareness and priming the
learner) will hopefully ensure a higher response rate over time.
❝...when a learner leaves we would contact them after three months, then
again at six months, nine months, and again at 12 months in order to get a
timeline of progression and ascertain how their circumstances may have
changed during that period.
❞
Independent training provider
❝...a longer timeframe guarantees that people will have more to say as they
are more likely to have found a job.❞
Independent training provider
Determining the timings may be based on other factors such as learner and provider
contexts, and the time of year. For example, some providers’ local labour market may be
greatly affected by seasonal variation, for instance people may be more likely to report
a job outcome at peak season in seaside resorts. Personalising the timing of when
destination tracking takes place will help to optimise the success rate.
24
A guide to tracking learner destinations
The importance of timings does not only apply to the period of time of when a provider
implements destination tracking, but also when during the week and during the day
tracking should be attempted.
❝...the time of day was an issue – the team tried to call in the morning and
❞
late afternoon during working hours which might have made it difficult to
reach learners who were in work.
Adult community learning provider
The timings can be helpfully informed by asking learners in advance about their plans
and finding out when might be the best time during the week and day to contact them.
Key message for providers
The timing of the tracking attempt influences the response rate. Therefore, if the
timing is within the provider’s control rather than being imposed by a funder, the
provider should trial when they get the best response rate to tracking.
Choosing the method(s) and the order in which they are used
For each cohort of learners, when preparing to track providers need to work through the
following questions:
•
•
•
Why are you tracking this cohort of learners? For example, to claim outcome-based
funding, to gather data to use in marketing materials, to reassure local stakeholders.
By when do you need the data? For example, by the next ILR return date, within a
month of the end of a project, four weeks before marketing materials need to be
published.
What data do you need? For example, any job outcome that meets a funder’s
criteria and evidence requirements, job outcomes in a particular sector to showcase
to stakeholders, learning progression outcomes to inform course planning.
Key message for providers
Providers are only in a position to choose their tracking method(s) once they know:
•
why they are tracking a particular cohort of learners;
•
by when they need the data; and
•
what data they need.
25
A guide to tracking learner destinations
The tracking method(s) should be determined only once the answers to these questions
are known as different methods meet different needs. For instance, if time constraints
allow for nothing else and the outcome evidence requirements are satisfied by a simple
verbal declaration, a provider may choose to only track learners by telephone.
Alternatively, if the evidence required to claim funding is a signed statement of a
sustained long-term outcome, then, despite the additional cost, the provider might
choose to establish a form of ongoing support that may provide the opportunity for
face-to-face tracking alongside other tracking methods.
Many providers that responded to our survey said they used a staged, mixed-method
approach to ensure they captured outcomes from learners that could not all be reached
through a single method.
❝By trial and error, we have developed a three stage approach: Email and
text, telephone, letter.❞
FE college provider
Undertaking each method successively in a mixed-method approach saves resources by
eliminating the learners whose destinations have been revealed by each previous
method used. Although the method descriptions in the next section consider each
method on its own, we anticipate that most providers will use a combination of these,
either running simultaneously or in a successive manner as in our trial methodology
described in Annex 2.
Ensuring all learners find the method of tracking accessible is important. Some learners
may not have access to IT, or may have poor IT literacy skills and for others language
barriers and different cultural perspectives.
❝There were literacy and language issues for learners (adult literacy learners)
– it is important to think about the tone of voice, use of language, and use of
plain English, and in order to encourage a full response it helps if a tutor who
is known to the learner makes the call. Think about using a script to ensure
clarity over the phone.
❞
Adult community learning provider
Obviously it is important that the providers themselves do not impede their own tracking
process. Inadvertently some providers may have policies in place that prevent contact
from being maintained through a particular approach:
❝We could not send emails as it is council policy not to do so (the council is
concerned about e-security). The council prefers formal letters to be sent.❞
Adult community learning provider
26
A guide to tracking learner destinations
Other providers found that, given an opportunity, a significant number of learners ticked
the box on their enrolment form to say that they did not want to be contacted:
❝We had to omit approximately 50% from the trial because of data
❞
protection as they had specified that they did not want to be contacted.
Adult community learning provider
How successful a particular destination tracking method is will not always guarantee
that it is the primary method used to contact learners. This is because providers will
most likely opt for the most cost-effective approach as their primary method, mainly
because the destinations of at least some learners can be identified in this way,
meaning a lower volume of learners need only be contacted using a second, more costly,
method.
For example many providers found that email was the least successful of all the trial
methods (email, telephone, and post) either because of email bounce-backs or a lack of
response from the majority of learners contacted, but it was still their primary method
for the following reasons:
•
•
•
•
Cost effective – takes up a minimal amount of staff time and resources
Quick to implement
Email can be automated and synchronised via data systems
Easy to personalise
❝Email would be the most cost effective way of tracking learners however it is
not a very effective approach.❞
Adult community learning provider
❝...going forward we would use the email approach and spend time to ensure
that learners’ contact details are up to date. This would help to save time in
the long-run, and by using our database more effectively this would help to
save on costs.
❞
Adult community learning provider
Determining the order of methods may rely on combining different approaches that
naturally complement one another; for example, a provider might find that telephone as
a primary method can be further enhanced by implementing secondary methods such
as pre-call text notifications or an automated telephone service. Combining different
approaches in this way may help to optimise the primary method by widening the
reach, making the process more efficient and cost effective.
27
A guide to tracking learner destinations
Key message for providers
It is rare for just one tracking method to result in an adequate response rate.
Therefore, in most instances, providers opt to use several methods successively.
Can other organisations help?
Learners often receive help from other organisations after they have finished their
course. These organisations will often know about the learners’ destinations following
their participation in learning, and if this data is shared it can at the very least alert
providers that a learner may have reached a destination or sometimes altogether
negate the necessity for providers to track learners.
❝In some locations, Jobcentre Plus (JCP) is happy to share whether someone
❞
has signed off so long as the individual has signed to the effect that they
are happy for JCP to share information with us.
Independent training provider
However, there can be many barriers to acquiring destinations data from other
organisations including:
•
•
•
•
•
data protection limitations;
unreliable partnership working arrangements;
data not being available in the tracking timescale;
unreliable and inaccurate data recording; and
data being in the wrong format for tracking purposes.
Given the extent of the destination tracking challenge, many providers were anxious for
these barriers to be addressed so that data captured by other organisations could be
made accessible to them.
28
Tracking by telephone
The effectiveness of tracking by telephone
Our survey responses indicated that telephone tracking was the most popular single
tracking method used by providers (of the 107 providers who responded to the survey,
71 used telephone tracking). This was primarily because response rates tended to be
higher for telephone tracking in comparison with any other approach.
Table 1. The popularity of tracking methods amongst survey respondents and the
average response rate for each tracking method
Telephone
calls
Email
Text
messaging
Apps
Postal
survey
Social
media
The number of
71
providers that said they
used this method
54
40
0
32
17
Average response rate
36%
34%
n/a
19%
12%
56%
Telephone tracking was also popular because it was perceived by providers to:
•
•
•
•
•
•
•
•
be relatively straightforward to undertake;
provide an instantaneous response;
be relatively inexpensive;
allow greater personalisation;
allow greater explanation as to why the information is required;
give the learner an opportunity to provide additional information, for example how
their course helped them to get a job;
provide an opportunity for the provider to market further learning opportunities; and
provide an opportunity to signpost learners to learner guidance services if they would
like to undertake further learning.
A major drawback of the telephone approach was its inability to generate a written
learner statement. To try to mitigate this, during our trial of approaches we asked
providers to request that learners completed written confirmation form that was sent
out to them after they had disclosed their destination over the phone. However
providers found very few learners sent back the form.
❝...all the people we spoke to by phone were happy to share information
about their destination however many must have been cautious about
putting the same information down in writing on the written confirmation
form...
❞
Independent training provider
29
A guide to tracking learner destinations
Providers speculated that this reluctance to confirm destinations in writing stemmed
from caution relating to the application of benefit rules:
❝Learners who come over to us from JCP can be highly suspicious when it
comes to paperwork and signing forms – many learners are wary of signing
forms as they are afraid that this might lead to their benefits being cut. This
is the biggest barrier when it comes to destination tracking.
❞
Independent training provider
The process of telephone tracking
Providers used a variety of telephone tracking approaches to contact learners, for
example:
•
•
•
•
•
administrators calling them from a centralised telephone system;
tutors calling them from a staff landline or mobile telephone (this is more
personalised as the tutor is known to the learner);
externally contracted call centres;
automated text messaging services that request a learner to text back details of their
destinations; and
automated telephone services requesting learners to select keypad options
representing their destinations.
On the whole, providers found that learners reached by phone were happy to provide
the information as requested.
❝The telephone approach was the most effective. Once we had got through to
❞
a learner and explained why we were calling they were more than happy to
share information with us.
Adult community learning provider
However, providers also said that response rates were better if the staff were well
trained; a friendly and approachable telephone manner had a significant impact on
response rates.
❝We adopted a friendly, positive tone over the phone and ensured that we
listened to the learner. The learner is crucial to all of this therefore we are not
just imparting information explaining why we are contacting learners, but we
are also there to listen.
❞
Adult community learning provider
30
A guide to tracking learner destinations
Providers said that it was important to think about the tone of voice, use of language,
and use of plain English in order to encourage a full response. They also often used a
pre-prepared script that they continuously improved based on its effectiveness. It is
helpful to personalise the telephone script based on the type of learner and the course
they attended to ensure that the message is accessible and most likely to elicit a
response.
❝We mentioned the trainer’s name and asked if they had received their
certificate if they seemed hesitant over the phone.❞
Independent training provider
It is important to show sensitivity if learners disclose that they have not achieved an
outcome. Again, working out a way to respond in advance can help the staff to respond
appropriately.
The timing of the telephone call
If a learner is at work they may be too busy to respond, or unable to respond to a
tracking call. It may be possible to work out the best time to call a learner by considering
in which sector/occupational role they are most likely to have found work, and making
the call at a time when people in that role are less likely to be at work.
❝Going forward, we would look at many sectors rather than just focusing on
one, e.g. retail or construction. This will have an impact on when telephone
calls are made. We will personalise the time of the call appropriately to suit
the working hours of each sector.
❞
Independent training provider
If it is not possible to anticipate the most appropriate time to make the call, make the
first call during office hours and if this is unsuccessful, make a second call at a time
outside office hours, for example in the early evening or during the weekend. Providers
told us that persistence did pay off but it required making multiple calls.
❝...we made three attempts per learner if there was no initial response.❞
Adult community learning provider
Ideally a provider will have recorded all of a learner’s possible phone numbers to ensure
that they have the best chance of reaching them.
31
A guide to tracking learner destinations
❝..we record both mobile and landline numbers, however we use mobile
numbers as the first point of contact...❞
Adult community learning provider
Using text messaging
Text messaging was deemed to be a cost-effective way of contacting learners as it could
be used to reach a large number of learners in one go; providers suggested that some
learners might respond better to texts, particularly younger learners; and that text
messaging is also an effective way of reaching hard-to-reach learners.
❝...it (the trial) made us realise that there should have been a fourth approach
using text messaging as an early notification system...We do know that
some learners prefer to answer/ respond to text messages, mainly pre-18
learners...
❞
Adult community learning provider
Text messaging can be personalised, for example text messages could be sent via a
known number, signed by a named individual and refer to the specific course
undertaken by the learner.
Many providers are already using text messaging for other purposes so using it for
destination tracking would be a simple process and relatively cost-free.
❝We already use a text messaging service as a way of contacting learners and
❞
disseminating information, and we are looking to embed this system into
our destination tracking methodology.
Independent training provider
32
A guide to tracking learner destinations
Issues and solutions when undertaking telephone tracking
Issue
Solution
A centralised telephone system will often use an
unrecognised or private number which may put
people off from responding.
Ask learners to store the number so that they
recognise it as a ‘known number’.
The cost of contacting mobile phone numbers can
be high.
Collect landline numbers as well or use another
method first such as text messaging to reduce the
number of calls required.
A lot of time can be lost if several calls have to be
made because learners are not picking up.
Secure learner buy-in whilst they participate in
learning.
It can be difficult for departmental staff to make
themselves available to make calls outside of
office hours.
Share responsibility, establish a new centrally
based post/team or outsource.
The later the tracking is done the more likely the
telephone number is to have changed, therefore
frequent updating of contact numbers is
necessary.
On the last day of a course, ask learners to update
their contact details, and provide a back-up
landline number (if they have one).
Learners tend not to respond to voice messages
because they might incur costs or because they’re
too busy when picking up the message.
Try text messages instead.
Learners ask for additional support over the phone
that cannot be provided by the caller.
Have a list of organisations and departments to
which learners can be signposted.
33
Tracking through e-approaches
E-approaches to tracking are straightforward to implement and very cost effective in
comparison with other methods. As the chart shows below, email was the most popular
e-approach used to track learners.
Figure 2. Provider survey respondents’ rates of use in different e-approaches for
tracking
60 50 40 30 -
No. of providers
20 10 0-
Email
Social media
Online survey
The process of tracking by e-approaches
E-approaches are used in a variety of ways:
•
•
•
•
a blanket email to all learners regardless of the course they attended;
a personalised email to each learner;
ongoing contact through social media providing the opportunity for learners to
update the provider on their progress and destination(s); and
online surveys through which learners can update the provider.
Providers often used email as a first method within a mixed method approach because
it was an inexpensive way of reaching large numbers of learners. Because they are easily
automated and cheap to send, emails are ideal for a phased approach through which
learners are contacted regularly to request an update on their destination(s).
❝Our trainers send a blanket email every 4 to 6 weeks for 6 to 8 months
asking if they need any support and what they are now doing.❞
Independent training provider
34
A guide to tracking learner destinations
Providers that used social media to track learner destinations only did this if they had
already established a social media presence for other purposes. They found that their
social media presence provided an opportunity to remind learners why it was important
to share details of their destination(s). It also afforded an opportunity for learners to
update their contact details. Feedback from learners in online forums, chat rooms and
comment boxes helped to promote destination tracking to other learners.
Providers that intended to track using digital approaches often supported learners to set
up their own email/social media accounts as part of their provision.
❝We generally have email addresses for all our learners as part of their
❞
provision involves establishing an email account so that they can use the
universal job-match system.
Independent training provider
The design of the e-message
The success of a digital approach will depend on the design of the message. Higher
response rates are achieved through messages that:
•
•
•
•
are short;
use plain English;
are attractive; and
include logos and other recognisable content.
A core digital template that is shared by all departments undertaking tracking can
ensure that data is collected consistently. Individual departments can include additional
content if they find that helpful.
❝...we adjusted it (template) a little so that recipients had the option to
provide feedback on how their course helped them.❞
Adult community learning provider
Templates can be linked to data systems such as a CRM system if the template and the
database share the same fields, allowing data to be imported.
Digital messages lend themselves well to personalisation. Providers said they did this by:
•
•
•
•
using an email address that would be recognised by the learner;
inserting a logo and other visuals that the learner would recognise;
referring to the learners’ former course title; and
including the name and e-signature of the tutor.
35
A guide to tracking learner destinations
When reflecting on their performance in the trial, some providers that had used a very
basic email text recognised that personalisation might have led to a greater response
rate.
❝...if we had personalised the email template that would have made a
❞
difference...Use a recognised email address, for example somebody that the
learners know from the Careers Advice Service.
Adult community learning provider
❝We will be looking to personalise our approach; we will broaden the email
questions in order to get a wider picture of a learner’s progress...❞
Independent training provider
The effectiveness of tracking by e-approaches
With an average response rate of 36 per cent, tracking by email was less effective than
telephone tracking at 56 per cent, but considerably better than using a postal survey at
19 per cent. Using social media was relatively ineffective at just 12 per cent, so this
could not be used as a sole method although it may usefully supplement other methods.
Although the average response rate was 36 per cent, there was a wide variation of the
response rate to email tracking reported by individual providers. Some providers found
that email was particularly time sensitive:
❝The email approach works well if the learner has been in the system
❞
recently, however the impact of this approach deteriorates over time, so this
approach is not as effective if a long period of time has elapsed.
Independent training provider
For some providers, the email approach was the least effective method:
❝Although email = minimal cost, it also elicited the poorest response which
was slightly surprising.❞
Adult community learning provider
The very low response rates within some providers may be due to:
•
•
•
•
a time lapse during which email addresses changed;
suspicion over spam emails;
caution about describing destinations in writing; and
cultural and age differences in the use of email amongst the learners tracked.
36
A guide to tracking learner destinations
Learners may find digital approaches to be too impersonal, particularly if emails or esurveys were sent out ‘cold’ without any prior notification or explanation. People can be
very unforgiving if an email does not convince them or capture their interest.
❝There is a lesson here – providing enough information, being as clear as
possible, and explaining the reasons why you need this information is so
important as one small element of confusion could be enough to put
someone off from responding.
❞
Adult community learning provider
Email may be particularly unsuitable for some ESOL learners due to their language
needs. However, many ESOL learners use social media for keeping in touch with family
members in their country of origin so there may be scope to make the most of this.
Issues and solutions when using e-approaches
Issue
Solution
Learners are cautious about sharing their email
address and may even give a provider an email
address they specifically use for spam.
Raise awareness amongst learners about the
importance of destination tracking.
Emails may be viewed as spam and
ignored/deleted.
Personalise messages as much as possible.
The response rate is very dependent on the
wording of the message.
Involving learners in the design of the message
and trialling it with small learner cohorts provides
an opportunity to refine it.
Online surveys can be ignored because of
‘evaluation fatigue’.
Raise awareness beforehand to learners so they
expect, and buy into, the survey.
Email addresses can quickly become defunct.
Provide opportunities for learners to update the
contact details that you hold for them, possibly
through social media.
Emails are sometimes viewed as a low priority and
can elicit a minimum or delayed response.
Raise awareness so that learners prioritise the
tracking email.
Learners may not use their email account
frequently enough to respond in the necessary
timeframe.
Asking learners about their use of email can
forewarn you if they are unlikely to pick up a
tracking email.
Learners may have limited IT skills.
Offer ICT provision for any learner with IT skill
needs.
Learners may have limited access to IT equipment. Provide IT facilities through work clubs, internet
cafes or introduce learners to IT facilities within
their community.
37
Tracking by post
The process of tracking by post
Sending letters is one of the more traditional ways of keeping in touch with learners. As
such, a postal approach may be more successfully used when tracking specific cohorts of
learners that respond well to letters, for example people aged 50+. Among other learner
groups, letters may be viewed as an outdated mode of communication or, at worst, junk
mail.
However, bulk mailing may be best suited to a whole-organisation approach rather than
for tracking the destinations of smaller cohorts of learners.
Organisations undertake postal approaches in a number of different ways, for example:
•
•
•
a blanket approach using a mail merge;
a personalised approach that is specific to a learner’s course, with a letter signed by
their course tutor; or
the use of a postcard as a less formal approach, with a tear-off section to complete
and return.
Unfortunately, tracking by post requires a lot of planning and can require a significant
amount of human and physical resource in order to cover:
•
•
•
the printing of forms, flyers, letters, surveys, and postcards;
the staff time required to set up mail merge templates and stuff envelopes; and
the cost of two stamps per letter as it is advisable to include stamped-addressed
envelopes for return.
Other providers, however, found the postal method much cheaper as they were able to
use a ‘mailsort’, or other automated process, for the automatic direction of mail. Even
where a mailsort is not available it may be possible to offset some of the costs. Many
providers routinely send out information in bulk in the form of flyers, leaflets and
brochures, as well as evaluation forms and letters by post to their learners. Therefore for
some providers it can be cost effective to piggyback on this by also including a
destination data request.
Where all postal correspondence is sent out as part of a whole-organisation approach,
the communications plan is likely to pre-ordain a set schedule when mail-outs take
place. It would therefore be necessary to ensure the next mail-out corresponds with the
timeframe available for tracking.
38
A guide to tracking learner destinations
Other key questions to ask are:
•
•
•
•
Can the destination tracking questions be incorporated into other evaluation
forms/paperwork being sent out to learners?
Will there be too much paperwork such that learners disregard much of it or get
distracted from completing the request for destination details?
Is there an automated schedule in place through an MIS or CRM system?
Is there a communications protocol that has to be adhered to when mailing learners?
The design of the postal message
Tracking by post can use a variety of hard copy formats including:
•
•
•
letter-headed paper;
a standard evaluation/survey design; and
postcards and leaflets with pre-paid, tear-off response slips.
Suitable templates may already exist that can be adapted for destination tracking
purposes. There may be a protocol in place concerning the different types of template
that can be used and with regard to branding and the use of logos.
Providers may wish to trial with a small cohort of learners which hard copy format elicits
the best response. Some providers might find that a combined approach is helpful; for
instance, learners could be sent a letter through the post requesting them to text
message back the details of their destination.
As with all other methods, personalisation of the postal message leads to an increased
response rate. The approach to personalisation is similar to that used to personalise
email messages, for example by:
•
•
•
•
using a postal address that would be recognised by the learner;
inserting a logo and other visuals that the learner would recognise;
referring to the learners’ former course title; and
including the name and signature of the tutor.
❝...we provided additional information about the learner’s course and their
❞
course tutor so that they could identify with the information in the survey in
order to encourage a response.
Adult community learning provider
It is also important to use plain English and take into consideration accessibility options
with regards to font size and style, colours and contrast.
39
A guide to tracking learner destinations
Tracking by post allows providers to ask more questions than any other approach:
❝With the postal approach we would provide an option on the form for
❞
learners to provide more information about their situation if not in work in
order to encourage a response.
Adult community learning provider
In some instances, providers were able to make referrals to other provision because of
the additional information acquired through a response to the postal method:
❝...we were able to refer someone onto a functional skills course because of
the information that they provided in the survey. Destination tracking as a
whole has provided the opportunity for us to promote the provision that we
offer simply because of the conversations had (by phone) or the information
received by email and post.
❞
Adult community learning provider
It is also important to include a ‘return by’ date to ensure that learners know by when
they should respond.
The effectiveness of tracking by post
With an average response rate from our provider survey respondents of just 19 per cent,
using a postal approach was much less successful than using a digital message
approach with an average response rate of 36 per cent, or using a telephone approach
with an average response rate of 56 per cent.
Some providers found the postal method useful within a mixed-method approach; many
providers believing that a postal approach might secure a response from people unlikely
to be engaged through a digital approach. An interesting example of an unintentional
combination of methods involved a provider whose ESOL learners chose to visit the
provider with their letters so that they could be translated. This led to a response rate
that was far more successful than the telephone or digital approaches with these
learners.
One provider found that the postal method was the most successful response to
destination tracking simply because the cohort of learners they targeted within our trial
were aged 50+:
❝...learner circumstances (age related) might have had more of an impact on
the postal approach simply because that might be an approach that they are
40
A guide to tracking learner destinations
❞
more familiar with compared with the younger generation.
Adult community learning provider
Despite this, many providers had reservations about using the postal method at all due
to its expense.
❝The postal approach was the second most effective (method) and providing
❞
a stamped-addressed envelope certainly helped ... however we would not be
able to adopt the postal approach because of costs
Adult community learning provider
Issues and solutions relating when tracking by post
Issue
Solution
Learners may be reluctant to put things in writing.
Raise awareness, reassure and secure learner buyin while they are still in provision.
Learners may not get around to responding even
though they intend to, or they might respond very
late.
Include a prominently displayed ‘please respond
by…’ notice.
Some learners might be disinclined to respond to
letters as they are perceived as very formal.
Use a stamped-addressed postcard instead of a
letter.
Forms sent in the post are often regarded as junk
mail.
Raise awareness that a form/letter will come while
learners are in provision and show them in
advance what it will look like so they recognise it
when it arrives.
Learners are survey fatigued.
Avoid a survey format.
Addresses may be out of date.
Provide opportunities for learners to update their
contact details and use mixed methods.
The postal approach might not reach the intended
recipient.
Include a return to sender option if the intended
recipient does not receive it.
41
Tracking through face-to-face
contact
The process of tracking through face-to-face contact
Tracking through face-to-face contact is made possible if and when providers encounter
learners after their course.
Community based providers encounter learners routinely by virtue of being based in the
neighbourhood in which their learners live. This has always afforded opportunities for
learners to inform provider staff of their destination(s). Arguably before more formal
tracking arrangements were established by other providers, community based providers
were better informed than any other type of provider about their learners’ destinations.
Opportunities for informal encounters abound.
❝I carry outcome forms with me wherever I go. Learners have filled them in
on the bus, in the supermarket and on virtually every street corner!❞
Community-based third sector provider
To some extent, the success of informal tracking through face-to-face contact relies on
the strength of the relationships formed between learners and community based staff.
Short interventions are unlikely to engender strong relationships. But community based
provision is so often first steps and intensive that strong relationships between staff and
tutors frequently develop.
❝I have no trouble in getting to know how things turn out for the learners.
❞
They keep coming back to tell me how they are getting on even though the
project finished some time ago.
Community based adult community learning provider
In densely populated areas, community based providers are uniquely placed to go ‘door
knocking’ to acquire destination details. This is because community based providers
may well have provided learning to several people in each street of their neighbourhood.
Community based providers that are also community organisations also encounter
learners that come to community events and celebrations. These events provide an ideal
opportunity to gather details of learner destinations.
42
A guide to tracking learner destinations
Opportunities for other providers to encounter learners are fewer but they still exist.
Work clubs and other forms of ongoing support are ideal for maintaining contact with
learners and thereby capturing destination details soon after an outcome is secured.
Gathering details through ongoing support via regular face-to-face meetings and
telephone calls is the primary means through which Work Programme providers gather
details to meet the Work Programme’s stringent requirements for job outcome
payments (paid only once the participant has been in a job for three or six months,
depending on the participant’s benefit payment group) and sustainment payments
(paid on a monthly basis thereafter for up to two years). Investment is required to
provide these forms of ongoing support, so providers will only offer it if the stringency of
the criteria and evidence requirements demand it and their funding is generous enough
to allow it.
Post-provision information, advice and guidance (IAG) support similarly affords the
opportunity to ask about the outcome of recent learning.
Awards ceremonies attract some learners more than others but when thoughtfully
marketed through social media they can be very effective at engaging close-knit groups
of disadvantaged learners who might not respond well to other tracking methods.
Some providers routinely capture destination details when learners visit to collect their
certificates. Certificates could be sent in the post but asking learners to collect them in
person provides the opportunity for destination details to be recorded. It also ensures
the certificates do not get lost or damaged in the post.
❝We ask learners to come into college to collect their CSCS (Construction
❞
Skills Certification Scheme) card. This gives us an opportunity to ask if they
have found work.
Further education college provider
It is important for the provider to make coming in as easy as possible for the learner.
❝At times we pay travel costs for the learners to come in to the centre for this
purpose.❞
Further education college provider
43
A guide to tracking learner destinations
The effectiveness of tracking through face-to-face contact
Tracking through face-to-face contact can be highly effective. Concerted efforts by
community based groups to capture destinations through every possible opportunity
can secure response rates of 80 per cent and more. For other providers the success of
face-to-face tracking depends on the availability of a suitable encounter. Where
providers have these encounters success rates are high but it is unlikely that all learners
are encountered to the same degree, so it tends to be an approach that is adopted on a
cohort-by-cohort basis.
A learner is much more likely to be willing to provide a written destination statement
through face-to-face contact than through post or email. This is because:
•
•
•
the learner is present to write the statement and hand it to the provider
representative there and then, eliminating any risk that it will be put off to another
day;
it is as personalised as any method could be; and
there is ample scope to reassure the learner as to the purpose and necessity of
collecting the data.
Issues and solutions when undertaking tracking through faceto-face contact
Issue
Solution
For learners to be able to drop in to collect
certificates there needs to be someone to greet
them.
Establish a reception function (possibly staffed by
volunteers or current learners wanting work
experience) or designate an individual to deal with
ex-learners when they arrive.
Learners dropping in unannounced to see tutors
can cause time management difficulties for staff.
Arrange for all enquiries for tutors to go through a
reception or individual responsible for front of
house and leave outcome forms with them in case
the tutor is unavailable when a learner comes in.
Informal encounters cannot be planned and may
not generate the required response rate in the
timeframe available.
Consider arranging a more formal encounter
within the timeframe and a mixed-method
approach as a failsafe.
A wide variety of staff may informally encounter
learners.
Provide training and regular reminders that
recording destination data is important for the
viability of the organisation. Consider incentivising
staff to relay back the destination data they
gather.
Learners do not turn up in enough numbers to any
events that are put on.
Consult learners on what would attract them as
the current incentive may not be adequate.
44
Conclusions and
recommendations
Although tracking learners is challenging, it is feasible if providers develop their
approach imaginatively and then undertake tracking systematically and persistently.
Generally, one method alone will not reach enough learners to secure an adequate
response rate. Therefore it is advisable to adopt a mixed-method approach through
which a provider successively uses several of the methods described in this guide. The
most inexperienced providers that took part in the trial found that their first attempt at
tracking generated disappointing response rates; success lies in optimising each method.
The advice given in the methods sections of this guide will help providers to do this.
Given that government policy is now to require providers to track the destinations of all
their learners, many providers will choose to adopt a whole-organisation approach to
tracking which they are likely to find is more efficient and cost-effective. Even through a
whole-organisation approach, there is no denying that destination tracking incurs costs.
These costs increase as the stringency of outcome criteria and evidence requirements
increase. Therefore, funders must carefully consider what they can reasonably expect
from providers in terms of outcome criteria and evidence requirements if they do not
wish to make additional funding available to cover the costs of tracking.
If funders are more than just interested in outcomes and decide to link funding to them,
it is vital that, through the outcome payment mechanism, providers can attain enough
income to cover the costs of their delivery. This will require outcome payments to be
based on the returns that can be achieved from a realistic response rate.
Providers consulted through research undertaken by CFE and NIACE and published in
Research to Assess the Impact of FE Funding Changes Relating to Incentives for Training
Unemployed Learners,9 BIS Research Report 96 (2013), called for job outcome payments
to be differentiated according to the labour market conditions within the locality in
which each provider operates. Currently, Work Programme job outcome payments do
not reflect overall labour market conditions, let alone local labour market conditions, as
payments for different benefit payment groups are the same whether the overall and/or
local economy is buoyant or sluggish. Consequently, Work Programme providers have
struggled to earn enough income to meet their costs from the job outcome payments
attainable in what has turned out to be a very sustained economic downturn.
A common criticism of outcome funding is that it drives providers to give greater levels
of support to those learners more likely to secure the funded outcomes; for example, if
paid on job outcomes, providers would be driven to give greater help to learners who are
most likely to find work. This has been termed ‘creaming and parking’ in other publically
45
A guide to tracking learner destinations
funded services where this phenomenon has been known to occur. Creaming and
parking is a risk where providers are paid a unit price for each outcome achieved
regardless of the learner’s likelihood of achieving that outcome. To guard against this
risk, a differential payment arrangement is used within the Work Programme that pays
higher job outcome payments for participants within benefit payment groups that are
deemed to be further from the labour market. Even then, Can the Work Programme Work
for All User Groups10 reported that there was growing evidence that the Work
Programme’s differential payment structure based on benefit payment groups did not
cover the real costs of delivery, which risked people needing greater support being left
unattended or ‘parked’ on the programme. Therefore, in response to the DWP
consultation on its commissioning strategy in 2013, NIACE recommended that the
Programme’s job outcome pricing differentiation should be based on robust data
relating to the actual costs of helping different groups of participants into employment.
The same applies to the adult skills system.
Based on our research that underpins this guide and the considerations above, we
recommend the following to government:
•
•
•
•
•
Providers should be given enough time to develop robust approaches to tracking
before Adult Skills Budget funding is linked to outcomes.
Research should be undertaken to assess the affordability of securing stringent
criteria, evidence and corroboration requirements.
Research should be undertaken to determine whether additional funding for tracking
should be made available to small providers that do not benefit from economies of
scale.
Any funding linked to outcomes should be based on a sophisticated payment
structure that takes into account reasonable response rates, national and local labour
market conditions and robust data relating to the actual costs of helping different
groups of participants secure different outcomes.
Steps should be taken to remove the barriers that prevent other publicly funded
services from sharing their knowledge of learner destinations.
10 House of Commons Work and Pensions Committee (2013) Can the Work Programme Work for All User
Groups? At: http://www.publications.parliament.uk/pa/cm201314/cmselect/cmworpen/162/162.pdf
46
Annex 1. A ten-point strategy for successful tracking
1 To secure their cooperation, explain to learners why you are going to track their
destinations.
2 So they recognise it, describe the timing and nature of the tracking process.
3 Your success in tracking will be heavily dependent on the quality of your learner
contact detail data. Take multiple forms of contact details and seek to refresh them
on the day the learner leaves.
4 Adopt a mixed-method approach because a single method will never be able to
reach every learner.
5 Select methods for your mixed methods approach based on why you are tracking,
when you need to track, and what data you need from the tracking process.
6 Used methods successively within this mixed-method approach so that you can
eliminate learners as their destinations become known and later methods need to be
applied to a smaller number of learners.
7 Unless you have time constraints, use the cheapest method first.
8 To ensure that you do not waste a lot of resources using inefficient methods,
optimise methods with a small cohort of learners that are representative of your
wider learner cohort.
9 At the same time that you optimise methods, re-design management information
systems so that they offer the greatest automation and efficiency possible.
10 Personalise the tracking message that you send to learners.
47
A guide to tracking learner destinations
Annex 2. The survey methodology and questions
The following survey was launched as an e-survey on the Cvent system and remained
open to respondents for six weeks in October and December 2013. The survey was
brought to providers’ attention through broadcast emails sent across NIACE’s extensive
provider networks. In addition, notices describing the survey which included a hyperlink
to the survey were placed in several newsletters and bulletins. The survey received a total
of 108 responses from providers, 84 of whom said they were tracking the destinations of
learners.
Survey of different approaches to unemployed learner
destination tracking
The National Institute of Adult Continuing Education (NIACE) has been commissioned
by the Department for Business, Innovation & Skills (BIS) to research different
approaches to unemployed learner destination tracking.
Through this call for information we would like to gather data from providers that are
already routinely tracking unemployed learner destinations.
The findings of this survey will be used to inform policy and will be shared with survey
participants through a published report. NIACE will also be looking to work with
providers to trial approaches evolved from those described by survey participants. If you
feel your approach is helpful, please complete the survey to bring it to our attention.
All responses will be reported anonymously. The NIACE Privacy Policy is available at
www.niace.org.uk/privacy-policy.
•
•
•
•
•
This survey has been designed to be user-friendly and should take up to 20 minutes
to complete.
This survey needs to be completed in one sitting.
Questions marked with * are critical to us and will need to be completed in order to
progress through the survey.
In order to move through the survey, please use the ‘Next’ and ‘Previous’ buttons at
the bottom of the screen. Please do not use the ‘Back’ and ‘Forward’ buttons on your
internet browser window.
This survey will close on Friday 15th November 2013.
If you have any queries or would like more information please email:
alistair.lockhart@niace.org.uk, or call 0116 2047076.
48
A guide to tracking learner destinations
Questions on which learners and destinations you track and why you track
them
*1
Does your organisation track learner destinations?
• Yes (move on to question 2)
• No (survey ends)
*2
Do you track destinations of:
• Unemployed learners
• All learners
• Learners who are not unemployed (survey ends)
*3
For which unemployed learners do you track destinations? Please select all that
apply.
• Adult Skills Budget funded provision QCF full Level 2 (learners aged 19+) and
Level 3 (learners aged 19 to 23)
• Adult Skills Budget funded provision of QCF vocational skills Award, unit and
other small qualifications
• Adult Skills Budget funded provision of training within sector based work
academies
• Adult Skills Budget funded provision of English and maths
• Adult Skills Budget funded provision of ESOL
• Adult Skills Budget funded provision of training within the Prince’s Trust Team
Programme
• |Adult Skills Budget Funded provision of personalised learning programmes for
learners with learning difficulties or disabilities
• 24+ Advanced Learning Loan funded provision of full time QCF Level 3 (learners
aged 24+)
• Work Programme participants
• European Social Fund (ESF) Programme participants
• Education Funding Agency (EFA) funded provision
• Community Learning funded provision
• Other charitable and non-government funded projects
• Other, please specify:
*4
What percentage of your unemployed learners do you try to track?
5
For these unemployed learners, do you track: (Please select all that apply)
• Employment destinations [yes, no]
• Self employment destinations [yes, no]
• Progression into further learning destinations [yes, no]
• Progression into job search provision [yes, no]
• Volunteering destinations [yes, no]
49
A guide to tracking learner destinations
• Inactivity [yes, no]
• Other, please specify:
6
If you collect data on more than one destination but there is a destination that
you are particularly interested in, what is it?
• Employment [yes, no]
• Self employment [yes, no]
• Progression into further learning [yes, no]
• Progression into job search provision [yes, no]
• Volunteering destinations [yes, no]
• Inactivity
• Other, please specify:
7
Why do you track learner destinations? Please select all that apply.
• To meet funding output evidence requirements
• To provide statistics for marketing to engage learners and inform their choices
• To demonstrate impact for funders, partners and community stakeholders
• To demonstrate impact to referral agencies
• To inform curriculum development
• To better understand learner destinations and progression routes
• To monitor equality and diversity outcomes
• To re-engage past learners
• To engage new learners
• To inform Information, Advice, and Guidance (IAG)
• To motivate current learners
• To engage new employers
• To evaluate different destination tracking approaches
• Other, please specify:
Questions on the approach you take to tracking and its associated costs (these
questions will help us ascertain the cost effectiveness of different approaches
to tracking)
8
50
Please define the criteria that need to be met for you to record:
• Employment destinations e.g. learners have to remain in employment for 16 or
more hours per week for more than four continuous weeks.
• For self employment destinations e.g. learners have evidence of trading for three
months.
• For progression into further learning destinations e.g. learners have begun study
towards a new learning aim.
• For volunteering destinations e.g. learners have volunteered for 3 or more hours
per week for more than four continuous weeks.
• Progression into job search provision e.g. attending job search support activity
for one day a week for more than three weeks.
A guide to tracking learner destinations
• Inactivity e.g. not in employment, learning volunteering or job search for more
than 13 weeks.
• Other, please specify:
9
What evidence do you require of the particular destinations you track?
• For employment destinations e.g. a learner declaration, a signed employer
statement etc.
• For self employment destinations e.g. proof of trading and a business plan.
• For progression into further learning destinations e.g. completed enrolment form
specifying a new learning aim.
• For volunteering destinations e.g. a signed declaration from the learner.
• Progression into job search provision.
• Inactivity.
• Other, please specify:
10
Does your organisation undertake destination tracking itself or do you use a subcontractor to undertake destination tracking on your behalf?
11
Does your organisation, or your sub-contractor, use any of the following human
resources to track destinations? Please select all that apply.
• Dedicated Tracking Officer(s) [insert approx. annual cost]
• Administrator(s) who undertakes tracking amongst other duties [insert approx.
annual cost
• LMS (Learning Management System) administrator(s) [insert approx. cost]
• Curriculum Manager(s) [insert approx. annual cost]
• Subject Leader(s) [insert approx. annual cost]
• Lecturer/Tutor/Teacher(s) [insert approx. annual cost]
• Trainer/Assessor(s) [insert approx. annual cost]
• Teacher Assistant(s) [insert approx. annual cost]
• Careers adviser(s) [insert approx. annual cost]
• Other, please specify and give the approx. annual cost:
12
Does your organisation or sub-contractor use any of the following types of physical
resources to track destinations? Please select all that apply.
• New or upgraded Learning Management Systems (LMS) [insert approx. annual
cost]
• Learner destination monitoring forms [insert approx. annual cost]
• Telephone calls [insert approx. annual cost]
• Email messaging [insert approx. annual cost]
• Text messaging [insert approx. annual cost]
• Smartphone apps [insert approx. annual cost]
• Postal surveys [insert approx. annual cost]
• Social media [insert approx. annual cost]
• Other, please specify and give the approx. annual cost:
51
A guide to tracking learner destinations
13
Could you easily adapt your approach to meet future changes in funder
requirements?
• Yes
• No [ – please state why]
14
What, if any, would be the cost implications of having to track these destinations
for more than one year (please explain your answer)?
Questions on the performance of your approach to tracking
15
Please estimate the percentage response rate (not the outcome rate) do you get if
you track:
• For all destinations
• For employment destinations only
• For self employment destinations only
• For progression into further learning destinations only
• For progression into job search provision only
• For volunteering destinations only
• For inactivity only
• Other, please specify
16
Approximately what percentage response rate do you get if you use the following
approaches to track destinations? If you do not use any of these approaches then
please leave the box blank:
• Telephone calls
• Email messaging
• Text messaging
• Smartphone apps
• Postal surveys
• Social media (please specify)
• Other (please specify)
17
How has your capacity to track destinations evolved over time?
18
What challenges, if any, are you currently facing in tracking learner destinations?
19
What, if anything, could you do to further improve the effectiveness of your
approach to destination tracking?
20
What could be done by others such as funders, policy members, partners, or other
stakeholders to make it easier for you to track destinations? Please state up to
three key actions.
52
A guide to tracking learner destinations
21
Have your destination tracking outcomes influenced your curricula or provision
arrangements?
• Yes [how?]
• No
22
Is there anything else that you would like to tell us?
Questions about you
23
Are you a:
• Senior manager
• Operational manager
• Practitioner
• Other, please specify:
24
Which of the following best describes your organisation? Please select one.
• A community learning provider
• A general further education and/or tertiary college
• An independent training provider
• An independent specialist college
• A third sector provider
• A Work Programme provider
• Any other type of learning provider? Please specify:
25
Would you be interested in taking part in NIACE’s forthcoming trial of approaches
to tracking learner destinations? [If yes, go to question 26 and then question 29, if
no, go to question 27]
26
Please provide your contact details below.
Name
Organisation
Email
Telephone
27
May we contact you if we want to find out more about one or more of your
responses? [if yes, go to question 28, if no, survey ends]
28
Please provide your contact details below.
Name
Organisation
Email
Telephone
29
May we contact you if we want to find out more about one or more of your
responses? [Yes, No]
53
A guide to tracking learner destinations
Annex 3. The trial methodology
The following description and diagram of the trial process were sent to trial providers:
Description of the trial process
Learner type to be trialled: sector based work academy learners or Full Level 2 learners
or English learners or ESOL learners.
The trial process
a)
b)
c)
Please identify 45 learners that completed their provision between May 1st 2013
and August 31st 2013 and another 45 learners who completed their provision
between September 1st and December 31st 2013. Please see these two groups on
the process diagram (enclosed with the email).
Split each group of 45 learners into three subgroups of 15 learners each. Contact
each subgroup using a different method i.e. telephone, email and postal survey to
enquire whether they have found employment following their learning. Please
record the employment outcome data you gather for each learner and the
response rate for each of the different methods.
For the two subgroups of fifteen learners who you attempt to reach by telephone:
a. If the learner is reached by phone and gives a verbal declaration that they have
secured a job outcome, please send the learner a confirmation form to complete
and send back in a stamp addressed envelope. Please record the response rate of
learners sending back a completed written confirmation.
b) If you do not reach a learner by phone, please try to contact them by email and
if that is also not successful, please send them a postal survey two weeks later.
Please record the response rate to each method used.
The types of destinations to be tracked
We only ask you to track and record job outcomes. This is to enable us to incorporate
providers in the trial that are not already tracking other destinations. However, when
conducting the trial you may choose to enquire about additional destinations if this is
helpful to you. If you wish to ask about other destinations, this will require you to amend
the tracking email text, postal survey and confirmation form that we will supply. Please
feel free to use your own versions or brand/ amend the tracking email text, postal survey
and confirmation form to fit your needs. We would be grateful for any feedback you
wish to give on the forms we supply and will ask you to share your own versions if you
wish to use them.
54
A guide to tracking learner destinations
The definition of a job outcome to be used in the trial
For the purpose of the trial, we define a job outcome as employment that is a minimum
of 16 hours per week sustained for a minimum of four weeks. All jobs that meet this
definition count as a job outcome regardless of how long ago the learning took place.
Written corroboration required for each job outcome
Through the trial we will secure written corroboration through written confirmations of
verbal declarations made over the telephone, through emails and through completed
postal surveys. We do not need employer corroboration of these written learner
declarations or any form of evidence of an outcome being achieved.
Data retention and sharing
Please retain the learner outcome data you gather as a result of participating in the
trial. We will ask you to complete a questionnaire on this data and on your experience of
conducting the process. To supplement the questionnaire, we will also ask you to take
part in a one hour telephone or face-to-face interview with one of our research staff.
What we will learn from the trial
Your involvement in the trial will enable us to:
•
•
•
•
test which single tracking method i.e. telephone, email or postal survey works best for
learners;
test how feasible it is to gain written confirmation of learners’ verbal declarations
made over the telephone;
identify if a combined approach of telephone then email then postal survey results in
a much greater response rate in comparison to just using the telephone; and
enable us to compare the effectiveness these methods for tracking learners shortly
after their learning and at least six months after completing their learning.
Because the aim of the trial is to learn about the effectiveness of different approaches
to destination tracking, the response rate for each method, rather than the job outcome
rate, is of greatest importance to us. We therefore do not need to see the data
pertaining to each individual learner. However, to help us assess if the subgroup was
broadly representative of the learner type, we will ask you to supply the average (mean)
job outcome rate for each subgroup. All the data you supply will be reported
anonymously. The NIACE Privacy Policy is available at www.niace.org.uk/privacy-policy.
Although our precautions will ensure that the data any specific provider supplies cannot
be attributed to them, we will fully acknowledge the support of each provider
participating in the trial within our report to BIS.
55
A guide to tracking learner destinations
Process diagram for provider
Telephone single method
Learners
completing
between May 1st
and 31st August
2013
Email single method
Postal survey single
method
Provider chooses
which learner
cohort to track i.e.
either SBWA
learners, ESOL
learners, English
learners or Full
Level 2 learners
If reached by phone,
written confirmation
form sent
If not reached by
phone, then email
and if still not
reached, then send
postal survey
If reached by phone,
written confirmation
form sent
Telephone single method
Learners
completing
between 1st Sept
2013 and Dec
31st 2013
56
Email single method
Postal survey single
method
If not reached by
phone, then email
and if still not
reached, then send
postal survey
A guide to tracking learner destinations
Annex 4. Tracking tools (similar to those used in the trial)
Sample email text
Email subject title: How are you getting on?
Dear [learner name]
To help us know if your course on [learner’s course title] was helpful to you, please tell us
about any work have found since.
To reply, please click reply, copy and paste the table below into your reply, complete the
table and then click send.
Have you found work since your course?
Yes / no
(please delete as appropriate)
If you have not found work, would you like us to
send you details of other courses in case they are
helpful to you?
Yes / no
(please delete as appropriate)
If you have found work, please tell us your job title?
If you have found work, please tell us your
employer’s company name (this will be kept
confidential and we will not contact your employer)
If you have found work, has this job involved
working more than 16 hours per week?
If you have found work, have you been done this
job for more than four weeks?
Please contact us on [your IAG advice line phone number] if we can be of further help to
you.
Many thanks
[sign off with the name of a tutor or other practitioner that the student will recognise]
Any information that you send us will be treated as confidential within our organisation
and will only be used anonymously to help us develop new courses and show the impact
of our courses to our funders.
57
A guide to tracking learner destinations
Postal survey text
Dear [Learner name]
To help us know if our course was helpful, please tell us about any work you have found
since finishing your course.
Please answer the questions below and return this form in the stamp addressed
envelope provided.
a) Have you found work since your course?
Yes
No
b) If you have not found work, would you like us to send you details of other courses in
case they are helpful to you?
c) If you have found work, please tell us your job title.
d) If you have found work, please tell us your employer’s company name (this will be
kept confidential and we will not contact your employer).
e) If you have found work, has this job involved working more than 16 hours per week?
Yes
No
f) If you have found work, have you done this job for more than four weeks?
Yes
No
Please contact us on [your IAG advice line phone number] if we can be of further help to
you.
Many thanks
[the name of the learner’s tutor or another practitioner that the student will recognise]
Any information that you send us will be treated as confidential within our organisation
and will only be used anonymously to help us develop new courses and show the impact
of our courses to our funders.
58
A guide to tracking learner destinations
Written confirmation form
Dear [Learner name]
Many thanks for letting us know when we phoned you that you have found work since
finishing your course on [learner’s course]. Our funders sometimes ask us to keep written
confirmation from learners that they have found work to prove that our courses have
been helpful.
To provide this written confirmation, please answer the questions below and return this
form in the stamp addressed envelope provided.
a) Please tell us your job title.
b) Please tell us your employer’s company name (this will be kept confidential and we
will not contact your employer).
c) Has this job involved working more than 16 hours per week?
Yes
No
d) Have you done this job for more than four weeks?
Yes
No
Please contact us on [your IAG advice line phone number] if we can be of further help to
you.
Many thanks
[the name of the learner’s tutor or another practitioner that the student will recognise]
Any information that you send us will be treated as confidential within our organisation
and will only be used anonymously to help us develop new courses and show the impact
of our courses to our funders.
59
A guide to tracking learner destinations
Annex 5. Acknowledgements
We would like to thank the providers that responded to our survey and the following
providers for their participation in our trial of approaches to destination tracking:
The Adult College Barking & Dagenham
Blackpool Borough Council Adult, Family & Community Learning
Bury Council Adult Learning Service
Derby City Council Adult Learning Service
Employment World @ Derby College
HIT Training Ltd / Work Ready People
learndirect
Manchester City Council Adult Education Service
Norfolk County Council Adult Education Service
Workforce @ Preston’s College
Shropshire Council Learning, Employment & Training Services
Swindon College
Wirral Borough Council Lifelong & Family Learning Service
Wirral Metropolitan College
Woodspeen Training
For all enquiries, please contact Alistair Lockhart-Smith at
alistair.lockhart@niace.org.uk or Rob Gray at robert.gray@niace.org.uk.
60
© NIACE 2014
Published by the National Institute of Adult Continuing Education
(England and Wales)
21 De Montfort Street
Leicester LE1 7GE
Company registration no. 2603322
Charity registration no. 1002775
NIACE is the National Institute of Adult Continuing Education, the national voice for
lifelong learning. We are an internationally respected development organisation and
think-tank, working on issues central to the economic renewal of the UK, particularly
in the political economy, education and learning, public policy and regeneration
fields.
www.niace.org.uk
To download a copy of this publication and for a full catalogue of all NIACE publications, visit http://shop.niace.org.uk
Twitter: @NIACEhq
@NIACECymru (Wales)
@NIACEbooks (Publications)
All rights reserved. No reproduction, copy or transmission of this publication may be
made without the written permission of the publishers, save in accordance with the
provisions of the Copyright, Designs and Patents Act 1988, or under the terms of any
licence permitting limited copying issued by the Copyright Licensing Agency.