Outputs, Outcomes and Impact
Andrew Harris – Derwen College
Fiona Voysey – National Star College
Natspec and QSR
• A National ISC PRD group (National Star, RNCB, Treloars,
Henshaws & Derwen)
• The data would to used to benchmark across the sector,
including GFE LLDD. In addition the data may be included in
ILR and FfE.
• For a short period the group was joined by representatives
from AoC, NASS & Ofsted.
Outline
• To explore the concept of outcomes and how
they relate to ISCs politically and in practice
• To consider the current Natspec QSR project
and outcomes for learners with complex
needs
How do you know your
college is good?
• Ofsted / CSCI inspection
• Internal QA
• The progress young people make with us
But…
• No national framework for outcomes
• What providers value as outcomes are not
always the priorities of others
• Confusion and some disagreement over what
we should be measuring and how
Why measure outcomes?
• To show the effect a placement at your college
has on a young person.
• To show potential placers that you provide a
high quality service –
But- can you meet both functions with the same
set of data?
The Project
• The challenge:
To produce quantitative data on personal/individual success that
allows comparison across providers.
• The Proposal:
To produce annual data on the achievement of predicted Every
Citizen Matters (ECM) outcomes for individual learners,
identifying the numbers and percentages of learners who are
‘ahead/over’, ‘in line with/on’ or ‘behind/under’ the learning
needed to meet their goal, amalgamated for the provider as a
whole and against each ECM theme.
The Benefits
• Allows personalisation within a nationally recognised
framework for consistency (RARPA plus ECM)
• Measures success in outcomes which are valuable to learners
and which are controlled by the provider
• Supports self-assessment and evidences ‘distance
travelled’/value added
• Does not prescribe or constrain curriculum offer, programme
or provider type, enables links to FL
• Links to local authority outcomes and Ofsted inspection
• Measures success in outcomes which are valued by
stakeholders and commissioners
Steps…
• Pilots to test process, establish guidance and criteria for levels
of performance
• Guidance on process including what might be included under
each ECM outcome, including PI’s
• Guidance on how to best contextualise the data including use
of evaluative criteria based upon CIF.
• Parameters for small numbers of learners
• Clarify definitions of complex needs and learners for whom
this approach is appropriate
• Validation and quality assurance (requires robust RARPA
processes and self-assessment with validation through peer
review and external tests through Ofsted).
• Establish links to ILR
Other recommendations
• The use of destinations against predictions could be a useful
indicator but should not be used as a measure of success as
there are too many issues outside the control of the provider.
• Students who die or whose health deteriorates such that
continued attendance is impossible should be removed from
success rates and retention data
Conclusions – June 2010
• Learners individual learning goals (ILGs) and ECM
outcomes?
• Are your RARPA processes robust? How do you achieve
this?
• The concept of ‘ahead/over target’, ‘in line with/on target’,
behind/under target.
• Consider the use of percentages in data collection.