Dissemination of PSA Level 3 Evaluation
December 2010
Providing Inspection Services for
Department of Education
Department for Employment and Learning
Department of Culture, Arts and Leisure
Agenda
• Terms of Reference
• Considerations
• Evidence Base
• Main Findings
• Recommendations
• Next Steps
• Conclusion and Questions
Considerations ….
 Outcomes- summary evaluation report plus four





specialist reports
Reports issued to each college last term- followup inspection work in four colleges
Same team of specialist inspectors in each
college
Evaluation Reports take due account of key
policy drivers
Evaluation Reports well- populated with
examples of good practice
Dissemination activities ….
Terms of Reference 1…
Scope of Evaluation- level 3 FE
provision in-
•
•
•
•
ICT and computing
construction and built environment
electrical and electronic engineering
mechanical and manufacturing
engineering
Terms of Reference 2… to inspect and report on
 the quality of the curriculum for each priority skill






area
the colleges’ strategic planning for the provision
the effectiveness of employer engagement and
links with key stakeholders
the quality of the provision for learning
the standards of students’ work
Also to
identify best practice
inform Department on impact of policies
Evidence Base
 320 lessons
 Small number of

lessons to address
weaknesses in fulltime students’
literacy, numeracy
and ICT skills
Sample of mandatory
and optional units to
improve technical
literacy and
numeracy skills
 College senior


management teams
curricular team
leaders and course
teams students
Sample of employers
Meetings with key
stakeholders
including DEL- Skills
Strategy and Sectoral
Bodies
Main Findings… overall effectiveness
Overall effectiveness for each college it is very good one college
 good in three
 satisfactory in two
Within each priority skill area:
 Electrical and electronic engineering is good or
better in 5 colleges- one is outstanding
 ICT and computing it is good or better in 4
colleges-one is outstanding
 Construction and mechanical and manufacturing
engineering only good or better in half the
colleges-the provision in mechanical and
manufacturing engineering in one college is
inadequate
Provision to meet needs of learners and local economy
 Full-time provision fine in most colleges but…..
 2 year part-time craft and technician courses in


nearly all of the priority skills areas is inadequate
1 year or short part-time up-skilling level 3
courses including bespoke courses for industry
varies across the priority skills areas and across
the colleges
Significant gender gap with few female students
recruited to full-time and part-time courses
Leadership and Management…1



STEM Expertise at a High
Level…a critical success
factor
Fragmented curriculum
planning across these
areas results in gaps in
provision and the
underutilisation of
resources
Strategic Project
Management
Methodologies can/do
make a difference



Much more needs to be
done to foster
collaboration within and
across colleges
Middle managers have
significant responsibilities
Challenges of deploying
best staff to meet different
priorities
Leadership and Management…2
 Quality Assurance arrangements are


implemented systematically but are not
sufficiently honed to identify and respond to
weaknesses
The quality and management of specialist
physical resources varies significantly across the
sector
Management of specialist resources…a few
examples of ineffective management of these
resources which has an adverse impact on the
students’ learning experiences particularly in
construction and mechanical/manufacturing
engineering
Leadership and Management..3 Economic
Engagement
 2 colleges have well -developed strategies to


support economic engagement across these
areas- they are well-informed and linked effectively
to the changes in the wider business environment
including sustainable technologies
A common thread in both was the further
development of extensive links with industry that
had been gained over a lengthy period of time in
one or more legacy colleges backed up with action
to up-skill the expertise of the lecturing staff
In the other colleges there are examples of good
practice in at least one priority skill area that are
not shared effectively in the other areas
Leadership and Management..3 Economic
Engagement
•
•
•
•
Nearly all of the colleges have well- developed links with
local and regional employers and sectoral bodiesparticularly at senior management level or through wholecollege business support units
These links have a more positive impact when senior
managers draw on the skills and expertise of those STEM
middle managers and course teams who have welldeveloped links with industry
Some examples of good practice in those colleges who
have been leading or have participated on projects funded
under the Innovation Fund: Employer Support Programme
WDFs.. there were few examples of changes in their level
3 provision across these areas to meet identified skill
needs. Examples of good practice ….
Quality of the Curriculum
 In 4 colleges students have good opportunities to




develop relevant technical and practical skills
In 2 colleges these opportunities are either limited
or vary significantly across the priority skills areasthe curriculum is unduly theory-based and
assessment led resulting in lower standards and
high levels of student dissatisfaction
Most courses have a good blend of mandatory
and optional units to develop the students’
technical literacy and numeracy skills
Additional qualifications can help but..
College admissions criteria vary significantly and
are not applied sufficiently rigorously
Quality of Teaching and Learning
•
•
The quality of
teaching and learning
in the majority (69%)
of the lessons is
good or better and a
significant minority
(30%) are very good
or outstanding
A minority (27%) of
the lessons are just
satisfactory
•
•
Most lecturers make
use of ILT in their
lessons but
insufficient use is
made of the more
interactive features
of educational
technology to
engage and
motivate students
One example of
best practice in
enhancing lecturers’
skills in making
good use of ILT
Assessment
•
•
•
More work needed to enable course teams to
work collaboratively across campuses in the
design and moderation of coursework
assignments
Some examples of good practice to reduce the
assessment burden on students- integrated
assignments
For most areas assessment planning is discrete
and unit-based which results in an excessive
assignment workload for students
CEIAG
•
•
Although students have good access to
specialist careers information advice and
guidance services in nearly all of the colleges
more needs to be done to integrate this into the
students’ main vocational units and through their
timetabled tutorial programmes
The use of work-related learning to enhance the
students’ knowledge and understanding of the
world of work is inadequate in nearly all of the
colleges
Student Support
•
•
•
•
The student tutorial programmes in nearly all of the colleges
are not sufficiently well-organised to support students
effectively in their learning particularly in reviewing their
progress and setting targets for improvement
With the exception of 2 colleges electronic or paper-based
student learner agreements provide little added value in
helping students plan their work effectively
In some cases, course teams made more widespread use of
other online reporting tools that they had developed to track
the progress of their students
In the best practice(1 college) students have access to a
well-designed tutorial system which gives good coverage to
personal and social development-the college through its
student support service has excellent arrangements in place
to support students at risk of withdrawing from their courses
Standards and Outcomes..1
•
•
•
•
The standards of the students’ work across the
priority skills areas are mostly good in 4 colleges
In the other 2 colleges while there are pockets of
good practice the standards of work in most of
the priority skills areas are just satisfactory or
inadequate
Most of the students achieve good standards of
work in literacy and numeracy within their
professional and technical units
Students can significantly enhance their literacy/
numeracy skills in their professional and
technical units
Standards and Outcomes..2
•
•
•
Well-planned project work relevant to the needs
of industry can significantly enhance the
standards of students’ work
Achievement rates vary significantly across the
colleges and across the different priority skills
area
Achievement rates for part-time courses are
mostly higher than on full-time courses, where
high levels of student drop out are a key
weakness in a significant minority of courses
Actions required to bring about Improvement 1: Improve
the quality of the curriculum across the Professional and
Technical Areas
Links to other Evaluation
Aims
•
•
Ensure that the students
enrolled on full-time
courses have the relevant
academic qualifications
and interests to meet the
challenges of the courses
Promote these courses
more effectively to female
students, where
enrolment levels are
insignificant
•
•
•
•
to improve the quality of
learning
to improve standards
to improve the strategic
planning
to improve the quality of
learning
Actions required to bring about Improvement 2:
Improve the strategic planning for the provision
Links to other Evaluation
Aims
•
Foster more collaborative
approaches to identify and
exploit key connections
across these PSAs to
meet gaps in provision
and make more effective
use of specialist physical
resources and to respond
effectively to the needs of
industry


to improve the quality of
the curriculum
develop/ enhance
employer engagement
Actions required to bring about Improvement 2:
Improve the strategic planning for the provision
Links to other Evaluation
Aims
•
Take action to address
provision that is
satisfactory or inadequate
particularly in construction
and mechanical/
manufacturing
engineering where only
half of the colleges have
good or better quality
provision
•
•
•
to improve the quality of
the curriculum
to improve the quality of
learning
to improve standards
Actions required to bring about Improvement 2:
Improve the strategic planning for the provision
•
•
Links to other Evaluation
Aims
Ensure that human resource
strategies are aligned more 
closely to curriculum
planning particularly where 
course teams work in
industry and in post-primary
schools
Develop more collaborative
planning across the sector •
and within the colleges to
widen student access to high •
quality specialist physical
resources particularly
•
expensive advanced
manufacturing equipment
•
to improve the quality of
learning
to improve standards
to improve the quality of
the curriculum
develop/ enhance
employer engagement
to improve the quality of
learning
to improve standards
Actions required to bring about Improvement 3: Develop
and enhance employer engagement and links with key
stakeholders

•
Links to other Evaluation
Aims
Make more effective use of
the existing expertise of
those lecturers and
curriculum leaders with a
strong background in STEM
in curriculum planning, and
in developing courses that
meet the needs of industry
and the NI workforce
Develop more innovative
approaches to promote and
deliver part-time courses to
meet the needs of the
workforce to enhance their
skill levels to meet the needs
of industry
•
•
•
•
•
to improve the quality of
the curriculum
to improve the strategic
planning
to improve the quality of
learning
to improve the quality of
the curriculum
to improve the strategic
planning
Actions required to bring about Improvement 4: Improve
the quality of the provision for learning
Links to other Evaluation
Aims
 Provide all full-time students
with good opportunities to
develop their technical and
practical skills and to
undertake work-related
learning tasks to enhance
their employability skills
•
•
•
•
to improve the quality of
the curriculum
to improve the strategic
planning
develop/ enhance
employer engagement
to improve standards
Actions required to bring about Improvement 5: Improve
the standards of students’ work
Links to other Evaluation
Aims
 Implement coherent
strategies to improve
student retention rates
on those full-time
courses with low levels
of achievement


to improve the strategic
planning
to improve the quality of
learning
Next Steps …
Publication of Reports
Follow-up Inspections
Conclusion/ Questions….