CfE Management Board – 15 January 2014 – Paper 5.1
A Formative Evaluation Strategy for 3-18 Learning in Scotland
At the last CfE Management Board meeting, a strategy on the evaluation of CfE was
discussed and Curriculum for Excellence Management Board confirmed that it
wished to take ownership of this strategy, and oversee its continuing development.
This paper sets out in more detail the purpose of the evaluation strategy and asks
Management Board:

To consider the draft strategy

To discuss the role of the CfE Management Board in the overall
governance of the strategy and in relation to the CLTA Forums

To consider timing of the OECD review
Management Board Secretariat
January 2014
CfE Management Board – 15 January 2014 – Paper 5.1
A Formative Evaluation Strategy for 3-18 Learning in Scotland
Strategy
The purpose of this strategy is to ensure that Scotland’s 3-18 education system is
able to learn and adapt over time, and is equipped to keep up with and respond to
change and feedback from the system. The strategy will capture and evaluate the
full range of activity and evidence available, and will have particular reference to
Curriculum for Excellence, in terms of:
 the distinctive features which underpin CfE
 the approach to learning and teaching being implemented in Scotland’s
schools
 the national curriculum framework
 national activity in support of implementation.
A number of strands will contribute to this:
 the planned OECD review
 the establishment and operation of Curriculum, Learning, Teaching and
Assessment (CLTA) National Forums, facilitated by Education Scotland
 engagement with the wider education and academic community on
implementation of CfE, and the approach being taken to evaluation
 Analysis of existing evidence
 Independent research, both as part of the National Forums process, and more
broadly
Policy focus
The key policy questions are:
Outcomes
 Is Curriculum for Excellence achieving what it set out to achieve?
 Have learners experienced positive change in the way in which they learn, and
has there been an improvement in terms of learners outcomes, and reducing
inequality?
 Are learners better able to make a smooth progression throughout their learner
journey, and in particular into further learning or employment?
CfE framework and support
 Are current developments within the system having a positive impact for local
authorities, schools, teachers, employers, Further and Higher Education,
learners, and parents?
 Have practitioners been able to use the CfE policy framework to improve learning
and teaching, and are they receiving appropriate support at the local and national
levels?
 Does the system have the capacity to continue to learn and adapt?
CfE Management Board – 15 January 2014 – Paper 5.1
Outcomes of evaluation




Those with a stake in the system are able to draw on a strong evidence base to
enable them to make improvements in their area of influence. We have a strong
qualitative independent evaluation of learning under Curriculum for Excellence,
which will provide a basis for ongoing review. This evidence will cover a range of
parties engaged in education system, including learners, practitioners, and
parents. This evaluation will offer international comparison, to enable us to
continue to learn from other approaches.
An established route for a range of stakeholders to maintain the relevance of the
curriculum and approaches to learning, teaching and assessment in a systematic
and on-going way
A role for inspection evidence in informing approaches to learning and teaching,
both at the system-level and within individual establishments.
Impact
assessment will also be built into Education Scotland’s planned interventions in
their support role.
A clearer understanding of gaps in our evidence base will develop as we
progress, and a recognition that a range of sources are needed. Engagement
with the wider community will be important here, to help us reflect on our
coverage of the main aims and principles of CfE.
Governance
At their September meeting, the Curriculum for Excellence Management Board
confirmed that it wished to take ownership of this strategy, and oversee its continuing
development. Further discussion on the Board’s role would be welcome, including in
relation to the key dates when input would be needed and the connections with wider
consultation which has been taking place on strands of the strategy i.e. CLTA forums
and OECD review
Role of Royal Society of Edinburgh and the wider academic community
The Scottish Government is working with the RSE Education Committee to explore
how it might best support this process, by contributing to an independent research
perspective. More specifically, it is proposed that the RSE Education Committee
 Contributes strategically to the design and progress of this strategy
 Helps to identify key research questions and methodologies to be addressed
through this strategy
 Assists in the development of a specification for the OECD review – early 2014
 Contributes to and comments on the background summary report being
submitted to the OECD – expected September 2014
 Input to the initial OECD scoping visit – possibly December 2014
 Meet with the OECD during field visits phase to offer an independent perspective
on the implementation of CfE – expected Spring/Summer 2015
 Contribute to consultation on the specification of the CLTA National Forums –
during January 2013
 Assesses and advises on the research dimension to the overall strategy
CfE Management Board – 15 January 2014 – Paper 5.1
This strategic role of the RSE Education Committee will sit alongside engagement
with the wider academic community, for example through the CLTA National
Forums.
Annexes
Annex A
Annex B
Annex C
Annex D
Summary of existing evidence relating to CfE produced by Scottish
Government and its agencies
OECD review plans
Summary of plans for the Curriculum, Learning, Teaching and
Assessment Forums
Distinctive features of CfE – paper from the RSE Education Committee
CfE Management Board – 15 January 2014 – Paper 5.1
ANNEX A
Data gathered by the Scottish Government
Introduction
The Scottish Government Education Analytical Services Division oversees a range
of analytical work generating data on the education system. Below, we have listed
our key activities. The purpose of listing our main information is to illustrate the
breadth and depth of information that can be used to give a picture of the education
system. Within these datasets there are a number of headline indicators.
The information is of two types. One is administrative data which reports both key
inputs into the system (buildings, teachers, spending) and outcomes (attainment,
destinations). Secondly, survey research which fills the gaps in routine data,
including information on literacy and numeracy at intermediate points in schooling,
international comparisons. These programmes could be supplemented by specific
pieces of work filling in the evidence base, for example, qualitative research with
stakeholders. The forthcoming OECD review will offer one such supplement to this
base.
There are also additional sources of data which will assist on considering the
outcomes of the system, focusing on issues beyond just school-aged education.
These include Growing Up in Scotland (GUS), the Scottish Schools Adolescent
Lifestyle and Substance Use Survey (SALSUS), and the Scottish Employer Skills
Survey. Education Scotland’s intelligence, inspection reports and beyond, will also
offer a rich source of evidence.
In considering these sources of evidence, we will need to ensure the views of all
those with a stake in the system are explored.

Data on outcomes

Survey of Literacy and Numeracy. Annual sample survey looking alternately
at literacy and numeracy in P4, P7 and S2

PISA. The OECD survey of member states and other partner countries on
achievements in reading, maths and science, with questionnaires designed to
be internationally comparable. Produced every three years. The 2012 data on
maths attainment will be published at the end of 2013. PISA 2015 will focus
on science.

Routine statistical data on sustained leaver destinations (a National
Indicator), and post appeals attainment (from the Scottish Qualifications
Authority)
Data on activity

Routine statistical data school meals and PE, exclusions and attendance.
CfE Management Board – 15 January 2014 – Paper 5.1
Data on processes

PISA also contains data on school management practices and pupil
perception. In 2012 school heads and students will be surveyed. In 2015,
parents will be surveyed in addition

SSLN also surveys teachers and pupils on experiences.
Data on inputs

Education spending by local authorities

Routine statistical data on pupils, teachers and estates (Sept-Dec 2013)
Education Analytical Services
October 2013
CfE Management Board – 15 January 2014 – Paper 5.1
ANNEX B
OECD Review
Purpose
The Scottish Government have agreed that the OECD will carry out a review of
progress on Curriculum for Excellence, and the quality of the associated learning
and teaching being provided in the schools-based education system. This is to
assist and inform the system in reflecting on current strengths and weaknesses, and
to provide a basis for continuing to develop priorities and practice.
The OECD’s ownership of this process will ensure independence, and will provide
opportunities to offer comparative evidence on wider international approaches. Their
completion of the 2007 review on Quality and Equity of Schooling in Scotland will
help offer a basis for this new work, provide continuity and also bring a close
connection with original aims of Curriculum for Excellence and our Improving
Attainment agenda.
Scope
The OECD work will focus on the school-based education system, but will also
consider transitions to and from this. The review will be formative – implementation
of Curriculum for Excellence is on-going, and this piece of work should not view the
programme as complete. The Scottish Government will provide a range of policy
questions to the OECD to help inform their work.
Consideration is being given to a two-phased approach, which will allow for future
follow-up evaluation work once the full roll-out of the new CfE qualifications is
complete and the first cohorts of learners have completed their Senior Phase and
made their transition beyond school.
We expect the OECD’s work to involve two key stages: desk-based research and a
field visit to Scotland to visit schools and gather evidence from key stakeholders.
We will provide the researchers with current evidence we, and our agencies hold, to
help inform this first stage. The second stage, the field research, will involve direct
interaction with a range of interests, including learners, local authority officials,
teachers, parents, Government and its agencies, and wider partners.
CfE Management Board – 15 January 2014 – Paper 5.1
Provisional timelines
In respect of the programme for CfE implementation, the following two alternative
timelines for review are currently being considered. Dates are approximations,
pending further planning discussions with the OECD, including on possible follow-up
in 2017/18:
Evaluation
of
learning
during
education/transition to senior phase
Date
Activity
the
Sep 2014
SG background report
Dec 2014
OECD pre-visit
Spring/Summer 2015
OECD Field Visits
broad
general
October/November 2015 OECD Report
Evaluation including the first cohort of learners completing
the new Higher Qualifications
Date
Activity
Feb 2015
SG Background Report
May 2015
OECD Pre-visit
Autumn 2015
OECD Field Visits
Feb 2016
OECD Report
CfE Management Board – 15 January 2014 – Paper 5.1
ANNEX C
Curriculum, learning, teaching, assessment (CLTA) National Forums
Background
During the period October to December 2013, Education Scotland engaged with SG,
CfE Management Board, Implementation Group and, jointly with SG, a range of
partners and stakeholders.
This exercise explored views on the purpose,
composition, working arrangements, role of participants, and the process for setting
up and running the Forums in 2014. These discussions led to the proposals outlined
in this paper.
1. Response to the engagement
Overall, the proposal to set up 3-18 CLTA National Forums received a positive and
encouraging response and indicated that there is a wide authorising environment to
progress the proposal. All groups agreed that CfE has the potential to ensure the
curriculum remains relevant to learners’ needs. There was optimism that this might
reduce the need for major curriculum overhaul every few years. All groups indicated
their wish/willingness to be involved in the Forums in some way or other.
1.1
Overall purpose
 The engagement helped to clarify the purpose. As a result, the following will
serve as the initial draft purpose for the first tranche of implementation.
The overall purpose of the CLTA National Forums is to secure, consolidate and
embed Curriculum for Excellence, maintain its relevance for all learners and ensure
it is capable of delivering high-quality learner outcomes.
1.2
Timescale
The engagement provided clear advice that the Forums should be phased in, with
the first tranche serving as pathfinders. As a result, the Forums will be introduced in
the following sequence and timescale.
January – March 2014
 CLTA4. 3-18 National Digital Learning Forum
 CLTA6. 3-18 National Expressive Arts Forum: Expressive Arts
 CLTA2. 8-16 National Middle Years Forum
April – June 2014
 Second tranche – three more Forums (to be decided) will be established.
August – December 2014
 Third tranche – remaining Forums to be established.
CfE Management Board – 15 January 2014 – Paper 5.1
1.3
The overall structure
The proposed initial structure of Forums and how they connect is shown here.
CLTA1
CLTA2
CLTA3
CLTA4
CLTA5
National
National
National
National
National
Early
Middle
Upper
Digital
Gaelic
Years
Years
Years
Learning
Forum
Forum
Forum
Forum
Forum
CLTA6
Expressive
Arts 3-18
CLTA7
Health and
Wellbeing
3-18
CLTA8
Languages
Literacy
3-18
CLTA9
Mathematics
Numeracy
3-18
CLTA10
Religious
and Moral
3-18
CLTA11
Sciences
3-18
CLTA12
Social
Subjects
3-18
CLTA13
Technologies
3-18
Particular features of the proposed Forum structure stemming from the engagement
include the following.

 Broad agreement of the need to prevent proliferation of Forums for crosscutting aspects of the curriculum.
CfE Management Board – 15 January 2014 – Paper 5.1
 Broad agreement that Forums should be established for each of the 3-18
curriculum areas in line with the structure in CfE, with separate Forums for
Gaelic and for Glow/Digital Learning.
 The proposal for 3 x stage Forums rather than conventional sector Forums
received strong support. These Forums will need to ensure that full attention
is given to cross-cutting aspects of the curriculum, specific aspects of delivery,
and wider contexts of provision. The structure is also intended to help ensure
transitions across sector interfaces are improved. Examples of these crosscutting aspects include literacy, numeracy, and health and wellbeing.
 Request that Forums be both physical and virtual communities.
 3-18 joined-up curriculum areas to promote the learner journey in a seamless
way
 Preference for ‘hard-wiring’ of key aspects of provision into the main Forum
structure, including for example parental involvement, CLD, ASN/special, as
facilitated by the design of CfE, and overlapping learner journey Forums to
reduce the risk of compartmentalized working
1.4
Membership
Through discussions on membership, agreement emerged around the following.
 Broad agreement that a ‘person specification’ should be used to ensure
membership of Forums meets the requirements of purpose.
 Broad agreement that a membership pool should be created to ensure all
stakeholders and partners can be appropriately represented within the totality
of the Forums, whilst not necessarily in each one.
 General recognition that Forums might draw from nominees from a range of
partner/stakeholder groups, alongside invited specialists as appropriate, and
with expressions of interest being opened up to the wide education
community.
 Agreement that recruitment to the Forums should be on the basis of interview
for suitability against the person specification.
 General recognition that the Forums will require a small standing group
representing key agencies.
 General agreement that membership for each individual Forum should be
around 15, and based on a core group and a phased, rolling membership
structure.
 Forums should have key partners at the core, including SG policy, ES, SQA
and ADES
2.
Proposals for January – June 2014
Proposed actions for the first half of 2014 are outlined in this section. Detailed
milestones will be built in early 2014 from these proposed actions.
2.1
Further engagement
 Discussions with further groups of partners and stakeholders will take place.
These will continue to be formative and are likely to have influence on the
present proposals, their implementation, and on the authorising environment.
The first of these will be on 13th January with the RSE Education Committee.
CfE Management Board – 15 January 2014 – Paper 5.1
 The possibility of a small symposium to consider membership of Forums and
to help draw up the person specification is being explored.
2.2
First meetings
 Establishment and meeting of a steering group in early January to consider
the process outlined in this paper and to make arrangements to oversee the
process thereafter, reporting to CfE Implementation Group and Management
Board. The precise composition of the steering group is to be determined.
 Inaugural meetings of the core standing groups for the first tranche of 3
Forums will be held in January. These meetings will be called by the lead
Education Scotland Senior Education Officers for the curriculum areas. The
initial core group, facilitated by an Assistant Director, will design the
implementation pathway for the Forum.
These inaugural meetings will make recommendations to the steering group
as follows.
 The person specification
 The preferred blend of membership from the three sources described
above, including any specific requirements of the particular Forum
 Timescales for recruitment and next steps towards recruitment
 Topical features of national implementation in the context of the Forum
 Preparation of a structure for a draft remit, and commissioning of the
draft remit
 Risks
 Recruitment plan
 Next steps and milestones.
2.3
Recruitment
A draft person specification will be prepared in January 2014 and will be used for an
initial call for nominations from key stakeholder and partner groups, for a national call
for expressions of interest, and to underpin invitations to identified specialists. A
programme of recruitment will be carried out during February. The first meetings of
the 3 Forums will be held in March 2014.
3.
Opening agenda themes
As well as exploring operational aspects of the new Forum, initial meetings of
Forums will be asked to consider the following themes, and thereafter to consider
how the Forum proposes to take them forward. These questions will be refined and
firmed up during the early part of 2014 through further engagements with key
stakeholders and partners.
 Progress towards securing a joined-up 3-18 curriculum
 Progress towards improved learning, teaching and assessment based on
learners’ individual and collective needs
 Progress towards securing better outcomes for all learners and reducing
inequity in outcomes for disadvantaged and vulnerable groups
 The impact of innovation, opportunities, challenges and perceived constraints
CfE Management Board – 15 January 2014 – Paper 5.1
 The suitability of the evidence base on which the Forum can base its work,
and what needs to be done to ensure robust breadth and triangulation of
evidence
 Progress towards building confidence in Scottish education through CfE
 The usefulness of advice and materials provided locally and nationally
 Key strengths and areas for development
CfE Management Board – 15 January 2014 – Paper 5.1
ANNEX D
Consideration of distinctive and innovative features – paper from the Royal
Society of Edinburgh
In framing the evaluation, the original aims and intentions of Curriculum for
Excellence need to be borne in mind. This will assist with considerations on whether
the intended outcomes of CfE are being achieved. In line with the agreed role of the
Royal Society of Edinburgh’s Education Committee to assist with designing the
evaluation strategy, we have been discussing with them the key aspects on CfE
which should be explored. The following RSE contribution offers thoughts on the
elements of CfE which should be included specifications for evaluation work,
including for the OECD review.
___________________________________________________________________
Any external evaluation of Curriculum for Excellence clearly has to focus on trying to
establish whether CfE has served to raise standards in Scottish schools and whether
it has helped to ‘narrow the gap’. In other words, it is bound to pay considerable
attention to analysing trends in attainment and achievement and seeking to
determine whether these have been affected by changes that form part of the
implementation of the new curriculum.
The evaluation must consider some of the more important components of CfE and
seek to determine what role, if any, they have played in bringing about the changes
that have been observed.
However, it is evidently of greatest interest to look at the contribution of those
elements of CfE that are innovative and distinctive. This will not merely be important
to those in other countries that are watching CfE with a view to perhaps adopting
some aspects of it but, more significant from a Scottish perspective, it will allow
some estimation of how successfully innovative features have been introduced and
whether or not these have added value to the enterprise.
CfE comprises a significant number of elements that might be considered as falling
into this category. The following could be considered for inclusion:
1. Curriculum






Adoption of a limited number of long-term strategic objectives (the so-called
four capacities)
The attempt to define the curriculum in terms of learning experiences and
expected outcomes for the pupil (the Es and Os)
The use of curriculum principles, especially, personalization and choice,
depth, challenge and enjoyment
An increased focus on the development of skills, especially advanced
cognitive skills
A balance between subject-based and interdisciplinary activity
An enhanced value placed on learning outwith school (with implications for
partnership working)
CfE Management Board – 15 January 2014 – Paper 5.1
2. The nature of learning



Constructivist pedagogy or, in the terms of CfE, ‘active learning’
Improved continuity across transitions (pre-five to primary, primary to
secondary and secondary to various positive destinations) with consequent
requirements for improved partnership working
Learner engagement
Although significant, the following aspects are not especially distinctive or innovative.
Rather, they reflect more familiar educational intentions. Given the need to keep the
exercise manageable, it would seem appropriate to focus on the items under 1 and
2, above, but accepting the need to remain vigilant on the matters below.
3. Learning pathways




Phases of education (broad general and senior), within the context of a
continuum of learning from 3-18
Establishment of pupil entitlements
The development of new national qualifications
Improvements in assessment approaches
Keir Bloomer, 10.11.13