VALUE-ADDED QUESTIONNAIRE ON NON-ACCREDITED LEARNING
Introduction/Preamble
As you are probably aware, NIACE DC has been commissioned by DELLS to research and
develop a Value Added Toolkit consisting of guidance and evaluation techniques to be
used to measure the value added benefits for learners of learning which falls beyond the
scope of qualification or credit-based outcomes (non-accredited learning)
The Value Added Toolkit should have considerable benefits for ACL providers across
Wales, providing approaches, techniques and support that will:





Clarify concepts and approaches associated with ‘value added’. (Value added
involves comparison between the attainments and characteristics of learners at entry
and progress made and achievements at exit. ‘Distance travelled’ is an alternative
phrase for ‘value added’
Identify value added practice across ACL providers in Wales
Help practitioners plan, deliver and evaluate non accredited provision more
effectively.
Encourage practitioners, managers and partners involved in ACL provision to
adopt and develop RARPA (Recognition and Recording of Progress and
Achievement) principles in line with Estyn’s Common Inspection Framework
Include additional examples of good value added practice as appropriate.
The following questionnaire will be the key research tool for identifying current best
practice in Wales with a view to developing a toolkit from which all ACL providers can
benefit. It is anticipated that there will be a variety of perceptions and practices relating to
‘non-accredited’ provision across the Principality. The research is intended to gain as wide
a view as possible. A prototype toolkit will be trialled with a representative sample of ACL
providers in Wales The project schedule can be viewed via this link.
QUESTIONNAIRE
GENERAL QUESTIONS
Please note that these questions refer exclusively to non-accredited learning.
1. Are you a tutor, coordinator or manager?
Manager
2. Is there a specific document which summarises, for staff and Estyn, the Quality
Assurance arrangements for non accredited learning in your organisation at
course/programme/institutional level? Our quality assurance document encompasses all
courses at the moment, no document specifically for non- acc provision
3. Would a Value-added toolkit for non-accredited learning be of use to you?
What would you hope to gain from it? How might you contribute to it? Yes, a VAT would
be invaluable to demonstrate both gains in ‘soft’ outcomes combined with an all
Wales framework for recognising achievements. We could contribute through trials/
1
pilots for a simple method of recognising achievement and ways and recording and
quantifying value added outcomes.
4. How would you rate the effectiveness of your current system for recognising and
recording non-accredited learning? Good system for quantifying VA outcomes.
Framework for recognising and recording achievement needs more development and
embedding.
5. What improvements do you think could be made to your current system?
Improvements to methods of recording achievement, initial and final self assessment
and integrating with VA outcomes into a coherent framework.
QUESTIONS ON THE RARPA STAGED PROCESS
You are invited to rate your own performance on a scale of 1-5 (Estyn grading)
1 AIMS
1-5
Do your non-accredited (NA) programmes/learning activities have:
 Clear, simple course outlines/descriptors?
 Easily understood intended learning outcomes?
 Information on resources needed?
 Information on any pre-requisites for participation in the programme?
Can you cite examples of good practice?)
2
3
2
2
2
1-5
INITIAL ASSESSMENT
Do you or your learners (please state which) record :
 Starting points?
 Previous knowledge/skills/experience?
 Individual Learning Plans
 Limiting factors or inhibitors?
 Support needs (practical or intellectual)?
 Views on why they are attending?
 Concerns?
Can you cite examples of good practice?
2
2
4
2
3
2
3
3
1-5
INDIVIDUAL CHALLENGING GOALS
Do you ensure there are clearly stated and suitably challenging objectives for:

all programmes? (Can you provide evidence for this?)
3
2

individuals wherever feasible? (Evidence?)
3
Examples of good practice?
Do you identify additional ‘soft’ outcomes for individual learners to do with
confidence, social skills, personal goals etc…? How do you do this? End of course
evaluation fed into MIS system for analysis. Includes learner assessment on how
confidence, communication skills and motivation have increased (if at all). To
course, centre, curriculum area levels
Examples of good practice?
4
FORMATIVE ASSESSMENT & FEEDBACK
1-5
Are learners made aware of how their progress and achievements will be
assessed? When, how and by whom? At start, by tutor and verbally.
Are learners encouraged to reflect on and assess their own progress and
achievement? When, how and by whom? Varies according to subject at the
moment.
Do practitioners give learners regular feedback? If so, which of the following
formats are used. (Tick as appropriate)
3
4.1 Methods used
Tick
√
Tick
√
Tick
√
Informal verbal feedback during learning sessions
Informal verbal feedback at the end of learning sessions
Planned one-to-one meetings with individual learners
Formal written feedback on submitted work/activities—some-----
3
2
Tick
√
Formal summative written feedback at the end of a programme
Other methods:
4
FORMATIVE ASSESSMENT & FEEDBACK
Are learners encouraged to share their progress/achievements with peers and
discuss progress/achievements in small groups? When and how?
Can you give examples of best practice in relation to the recognition and
recording of progress and achievement during learning programmes?
Which of the following forms of evidence are used to record learning during a
learning programme? (Tick as appropriate)
1-5
3
3
3
4.ii
Methods used: Note method varies according to subject
Learners’ files
Tutor records of assessed activities
Journals/diaries/logs
On line logs/diaries (‘blogs’)
Portfolios
Artwork/photographs/videos/other visual materials
Performances
Group &peer assessment
Exhibitions/displays/showcases
Individual or group learner testimony
Tick
√
Tick
√
Tick
√
Tick
√
Tick
√
Tick
√
Tick
√
Artefacts
Digital images to show improving performance
Other forms of evidence:
‘Learners Walls’ containing learners’ testimonies.
Examples of good practice?
5
SUMMATIVE ASSESSMENT
Tick
√
Which of the following forms of summative assessment methods are used?
Again can vary according to subject at the moment
Learner self assessment
Tutor assessment
Peer review
Group review
Joint review
Learner satisfaction review
Tick
√
Tick
√
Tick
√
Tick
√
Any of the methods relating to
formative assessment (see Q4.ii)
Examples of good practice?
4
QUALITY ASSURANCE
Tick
√
Which of the following QA features are in place for monitoring the quality of
non-accredited provision? (Tick as appropriate)
Recording and evaluating learners’ outcomes- in some cases
Learner perception surveys on quality of provision
Team meetings on learners’ progress???
End of programme/activity review
Tutor involvement in organisational Self Assessment and planning for Estyn
inspection- to some extent
Quality Improvement action planning
Tick
√
Tick
√
Tick
√
Mapping of RARPA principles to the CIF. Not yet
Any other features:
5
The RARPA Principles
Element
Evidence
1. Aim(s) appropriate to an
individual learner or groups of
learners (CIF KQ’s1, 2, 3 and
5)
Clearly stated aim(s) for all programmes
2. Initial assessment to
establish the learner’s starting
point (CIF KQ’s1, 2 and 4)
Record of outcomes of process of establishing
learners’ starting points
[Could include aims which do not specifically
mention a learning aspiration, for example, in some
informal and community based non-accredited
learning]
[Process and level of detail will vary according to
the nature and duration of the learning programme.
Records may include learners’ self-assessment of
prior learning and/or learning and support needs]
3. Identification of
appropriately challenging
learning objectives: initial,
renegotiated and revised (CIF
KQ’s1 2, 3, 4
Clearly stated suitably challenging objectives for all
programmes and, wherever feasible, for each
learner
4. Recognition and recording
of progress and achievement
during programme
(formative assessment): tutor
feedback to learners, learner
reflection, progress reviews
(CIF KQ’s1, 2 and 4)
Records of learner self-assessment, group and peer
assessment; tutor records of assessment activities
and individual/group progress and achievement.
Learners’ files, journals, diaries, portfolios,
artwork; videos, audiotapes, performances,
exhibitions and displays; individual or group
learner testimony; artefacts, photographs and other
forms of evidence
[The level of challenge which is appropriate will
vary according to initial assessment of learners’
needs, aspirations and starting points. Learning
objectives may be amended during the learning
programme, for example, as a result of formative
assessment]
[Research indicates that learners prefer the term
‘feedback’ and that learners’ capacity for reflection
and informed self-assessment would be enhanced
by more dialogue with tutors and the sharing of
criteria and norms used to evaluate progress and
achievement]
6
Element
Evidence
5. End of programme learner
self-assessment; tutor
summative assessment;
review of overall progress
and achievement. This will be
in relation to appropriately
challenging learning objectives
identified at the
beginning/during the
programme. It may include
recognition of learning
outcomes not specified during
the programme
(KQI ‘s, 2 and 4)
Records of learner self-assessment, group and peer
assessment; tutor records of assessment activities
and individual/group progress and achievement.
Learners’ files, journals, diaries, portfolios,
artwork; videos, audiotapes, performances,
exhibitions and displays; individual or group
learner testimony; artefacts, photographs and other
forms of evidence
[Evidence is likely to comprise qualitative and
quantitative information and to demonstrate
planned learning outcomes and learning gains
identified subsequently]
7
RARPA Principles mapped to the Common Inspection Framework
RARPA Principles
Aims appropriate to an
individual learner or groups of
learners (clearly stated learning
aims)
Initial assessment to establish
the learner’s starting point
Identification of appropriately
challenging learning objectives:
initial, renegotiated and revised
Recognition and recording of
progress and achievement during
programme (formative
assessment): tutor feedback to
learners, learner reflection,
progress reviews
End of programme learner selfassessment; tutor summative
assessment; review of overall
progress and achievement
Key Questions
KQ1 How well do learners achieve?
KQ2 How effective are teaching and assessment
KQ3 How well do the learning experiences meet
the needs and interests of learners and the wider
community?
KQ5 How effective are leadership and strategic
management?
KQ1 How well do learners achieve?
KQ2 How effective are teaching, training and
assessment?
KQ4 How well are learners cared for, guided and
supported?
KQ1 How well do learners achieve?
KQ2 How effective are teaching, training and
assessment?
KQ3 How well do the learning experiences meet
the needs and interests of learners and the wider
community?
KQ4 How well are learners cared for, guided
and supported?
KQ1 How well do learners achieve?
KQ2 How effective are teaching, training and
assessment?
KQ4 How well are learners cared for, guided and
supported
KQ1 How well do learners achieve?
KQ2 How effective are teaching, training and
assessment?
KQ4 How well are learners cared for, guided and
supported?
8