STUDENT TEAM MEMBER
HANDBOOK
September 2014
CONTENTS
A.
General conduct of the external review
1
B.
Outline of external review activities
4
C.
Framework for external review:
indicative prompts for STMs
6
D.
Finding the learners and their
information
11
E.
Your report and discussions of
findings with the college
12
Appendix 1 – Evidence form
13
Appendix 2 – Student Team Member
report pro-forma
15
Appendix 3 – Checklist for reports
17
Appendix 4 – Glossary
18
Appendix 5 – PRAISE framework
20
A.
GENERAL CONDUCT OF THE EXTERNAL REVIEW
(Please refer also to PowerPoint presentation notes)
As you read this handbook, you will need to refer to the document, External Quality
Arrangements for Scotland’s Colleges: updated August 20131 and to the
Guidance for External Reviewers: A handbook for members of Education
Scotland external review teams, September 2014 edition. You will also require to
be familiar with Education Scotland’s Code of Good Practice for Safeguarding.
1.
The quality arrangements document provides you with details of external
review within the context of Education Scotland activities. The reviewers’
handbook provides additional guidance for all reviewers, including advice on
safeguarding, equality and diversity and sustainability. You should also refer
to the PRAISE framework (Appendix 2) which defines the principles
underpinning best practice on review activities.
2.
The Managing Inspector (MI) manages the review. He or she will ensure that
the review is rigorous and fair and that it has regard to the individual college
circumstances. You must consult the MI promptly if you need advice or meet
any difficulties. Within the team (between five and ten members, plus one or
two Student Team Members (STM), depending on the size of the college),
there will be a number of lead writers, each with responsibility for coordinating
evaluations in relation to specific quality indicators (QI).
3.
Where there are two STMs on a review team, one of you will have been
designated as STM1. Where you are STM1, you have responsibility for
coordinating the overall STM activities and reporting back to the MI, the
team and the college. As STM2, you must collaborate with STM1 as
appropriate and ensure that you contribute to the writing of the STM
report.
4.
Remember at all times that you are an advocate for learners and potential
learners whilst working with Education Scotland. You do not represent your
college, your Students’ Association (SA) or NUS and you must behave
appropriately. You must exercise discretion, confidentiality, a professional
attitude and enthusiasm in the discharge of your duties. You will receive
further information on these issues during your training sessions.
5.
Reviews put considerable pressure on staff and learners and you should try to
minimise this as far as you can. Explain what you are doing and why, and
make it clear that you will respond helpfully to questions about the review
process. Remain flexible and be prepared to adjust your draft plans to
accommodate learners wherever possible. Give learners opportunities to talk
about key issues that are important to them.
1
http://www.educationscotland.gov.uk/Images/ExternalQualityArrangementsColleges2013_tcm4813723.pdf
1
6.
You should remind yourself to be objective at all times and be open to the
views of others. For example, don't divert an interview to tell learners how
you do it in your college or your student association. Do not align yourself
with complaints, for example on funding or members of staff.
7.
Keep in mind the three key principles of high quality learning, learner
engagement and quality culture and focus your review on aspects of learner
engagement within the college, as well as the quality of the learner
experience as expressed through the Quality Indicators (QIs)2.
8.
Use the attached indicative questions as a guide to aid discussion and not as
checklists. They will not suit every circumstance because of variations in
mission, catchment area, college size, structure and the nature of the student
groups you interview. Adapt the questions as appropriate. Remember, your
role in the team is to listen to and report back on the learner voice first
hand.
9.
There is no need to go through all the indicative questions. There may be
circumstances where you ask none of them at all. It is important that you
listen to and record what learners say. It is advisable to start with a general,
wider question, such as “What is it like to be a learner in this college?”. You
can then direct the discussion, if you wish, using the indicative questions.
Over the week, try to have some information for most of the reference QIs. If
you have findings relating to any additional QIs these aspects may be
recorded as other significant quality indicators.
10. Ask open questions such as: “How would you describe the advice you
received from the college when you applied to your programme?”. This is
preferable to: “Did you receive good advice from the admissions section when
you applied for your programme?”. A closed question invites a yes or no
answer, whereas you require a more detailed response to inform your report.
11. Sometimes you may find one dominant group member provides you with most
of the information. These individuals may have strong views. It is imperative
that you check these views out with other learners, particularly others within
the group, to establish whether or not they share these opinions.
12. Note the views of learners from each of your meetings on the Evidence Form
(EF) provided (Appendix 1) and use these to inform your report.
13. In larger colleges, there will be two STMs. Use this additional resource well to
sample learners appropriately. You must also coordinate your findings
throughout the week and cooperate in writing the STM report.
14. Try to cover a representative range of learners from a breadth of programmes
and modes of attendance. Be careful to include learners from different ethnic
backgrounds and age groups, and remember to include learners with
2
http://www.educationscotland.gov.uk/Images/QFAug2012_tcm4-727419.pdf
2
disabilities (including sensory or mobility), as well as a balance of males and
females. For example, try to include learners from:
o
o
o
o
o
o
mainstream full-time and part-time FE and HE3 programmes across the
college;
access or pre-entry programmes;
distance/flexible/blended learning programmes;
evening programmes;
programmes where they are receiving additional learning support; and
the Students’ Association.
You should also try to include international learners and school/college
learners. Not all colleges will have all of these categories. However, it is
important that you speak with as wide a range of learners as possible.
15. Pay attention to the social and cultural equality and diversity and the college
response to it in all your activities, taking account of the nine ‘protected
characteristics’ of - age, disability, gender reassignment, marriage and civil
partnership, pregnancy and maternity, race, religion or belief, sex, and sexual
orientation. (See Guidance for External Reviewers: A handbook for members
of Education Scotland external review teams, September 2014 edition).
16. Be aware of issues relating to health and wellbeing within the college. The
college may have healthy eating initiatives; offer free gym membership;
provide extensive career advice or have initiatives to support the mental
health of learners. How conscious are learners of these?
17. You should also pay attention to the college response to sustainability and
record this. You may observe evidence of the college commitment as you go
round campuses, such as positioning of recycling bins and posters. Learners
may be involved in various sustainability activities within their programmes.
They may also be engaged in activities within the wider college environment
and through the Student Association. Examples include encouraging the use
of public transport, recycling, fair trade products in refectories, and
awareness-raising campaigns.
18. If you have any problems or uncertainties at all consult the MI straight away.
3
UHI colleges excepted.
3
B.
OUTLINE OF EXTERNAL REVIEW ACTIVITIES
1. Pre-review activity4

Depending on practical considerations such as distance and travel, you will be
invited by the MI to the Planning Day at the college which normally takes place
approximately eight weeks prior to the review. You will be asked to come to the
college to meet with the Student President and other learner representatives.
This provides an opportunity for you to meet key college contacts and see at
least part of the college. You may be asked to make a short presentation to
learner representatives. Be guided by the MI.

At this meeting, the MI will issue the learner engagement questionnaire to the
student president of the college. The MI will ask the student president to return
the completed questionnaire four weeks prior to the review. The way in which the
responses to the questionnaire are presented is the decision of the Students’
Association and not Education Scotland. However, the Student President will be
advised that you can be contacted for advice and is given your contact details. If
you have not attended the planning day, the MI will ask you to contact the student
president, prior to the questionnaire return date, to introduce yourself.

You should use these opportunities to ensure that he/she understands what is
required and to check on progress.

You will also be asked to attend the Team Briefing Day, held approximately two
weeks prior to the external review. At this meeting the MI will brief the team on
various aspects of the review. The MI will introduce the learner engagement
questionnaire and the issues it raises. You should be prepared to discuss these
and to identify issues that you may wish to raise with learners during the review.
You must inform the MI as soon as possible of any difficulties in attending this
event. If you are unable to attend the team briefing day, it is unlikely you will be
able to join the review team.

At the team briefing day, there will be an opportunity to plan your review
activities and how you will ensure you speak to learners from all campuses. This
is particularly important in large, multi-site colleges and where there is more than
one STM.

You are expected to read any correspondence/documents sent to you by
Education Scotland.
2. External review week

You will be present throughout the review. In 2014-15 reviews, the review team
will be in college Monday, Tuesday, Wednesday and Thursday. The team will
4
The timescales given in this section are for guidance. Time scales may alter, depending on the size
of the college, the point in the year that Reviews take place and the impact of local holidays.
4
be together, discussing and writing, on Friday. STM1 will meet with the HMIs
and either a Lead Officer or Assistant Director on the following Monday for a
moderation meeting. At this meeting, the team will agree the conclusions that will
be presented to the college.

You should confirm with the MI on Monday your schedule for the week and
conduct meetings as appropriate.

You will take part in the team meeting each afternoon or as otherwise scheduled
by the MI. As STM1, you will be required to share your findings with the team.
So it is important that STM1 and STM2 arrange to speak with each other at some
point each afternoon, by phone if necessary.

In 2014-15, the discussion of findings with the college will take place on Tuesday
of the week following the review. STM1 will attend alongside the MI and one
other team member.
At the end of the review, you will discuss your performance on review with the MI.
This discussion will help shape the training and support you are offered, as well
as inform future deployment. You will also be asked for your feedback on the
process of the review and this helps inform self-evaluation within the team and
Education Scotland. You should use these opportunities to develop your
participation, the role of the Student Team Member, and the effectiveness of the
review process.

Remember to record the number of school learners, International learners
(non-EU only), and government funded learners with whom you meet. The MI
will let you know if the college has credit rated its own provision using the SCQF
framework.

Finally, ensure that on review week, you pay attention to your own health and
wellbeing. You will be required to work hard and the review is an intense,
although very rewarding, experience. It is important that:
o
o
o
o
o
o
o
If appropriate, you have appropriate personal protection equipment (PPE)
and Education Scotland will supply this, if needed;
You read carefully, understand and follow the Education Scotland Code of
Good Practice for Safeguarding;
You eat and sleep regularly, and if you take unwell on review, you advise
the MI immediately;
You organise your travel arrangements in advance, allowing plenty time to
travel between locations. Particularly, you should identify when you can
car-share or use public transport;
The MI knows where you are throughout the review. Plan ahead and notify
any changes;
You have the telephone numbers of team members and the MI has your
mobile number; and
If you experience any difficulties, you inform the MI immediately.
5
C.
FRAMEWORK FOR EXTERNAL REVIEW: INDICATIVE
PROMPTS FOR STMs
High quality learning
How well are learners progressing and achieving relevant, high
quality outcomes?
2.
Impact on learners and other users of college services
2.3
How well do learners make progress, attain qualifications and achieve
more widely?










How do you know that you are making satisfactory progress in your
subjects?
What kind of feedback do you receive from teaching staff?
Do you use a personal learning plan (PLP)? How helpful has this been?
What do you know about core skills, skills for employability and
citizenship? How is your progress in these skills monitored and recorded?
Was there anything in your programme that you had already achieved and
completed before you started and if so, what was done about that?
In what ways have your personal learning skills improved/developed since
commencing your programme?
How has your programme prepared you for employment?
Have any special arrangements been made to accommodate other
priorities in your life?
What do you know about the college’s actions regarding sustainability?
How has being at college helped you improve your health and wellbeing?
In what ways?
6
How effective are the college’s learning and teaching processes?
5.
Education, training and lifelong learning
5.1
How well does the college develop and deliver programmes and
services to meet the needs of learners from all backgrounds and
circumstances?





5.2
How well do learners learn?




5.3
What measures does the college take to meet the needs of specific groups
of learners?
Tell me about the timing and location of your programme. Are teaching
staff flexible?
How does the college promote equality and diversity (for example different
races, cultures, disabilities, genders, ages etc.) to learners? How mixed
are learners on your programme (male/female; ethnic origin; disability)?
Has your programme turned out as you thought it would? In what ways?
Do curriculum materials and resources on your programme acknowledge
and reflect social and cultural diversity?
Are you motivated in your learning? In your lessons, how do teaching staff
motivate learners/include everyone? Is everyone part of the class?
Do people enjoy learning on the programme? Why?
What kinds of resources do you use in your learning? How available are
these?
In what ways are you encouraged to learn independently?
How well do planning, teaching and the use of resources ensure
effective learning?
This indicator will mainly be reviewed by reviewers as part of the generic
review of learning and teaching. However, any learner feedback that you feel
will contribute to this aspect of the review will be welcomed.




5.4
How would you describe the relationship between lecturers and learners?
Do staff set appropriate standards in the classroom/workspace?
Are learning activities well planned? How are learners involved in
planning their lessons?
What’s the learning environment like? For example,
o Classroom resources
o Access to ICT
o Room size for the number of learners
o Lighting, heating
How well is assessment used to promote effective learning?

How are assessments arranged? Are they spaced well throughout the
year?
7



5. 5
How well are potential and current learners provided with information,
advice and support?





5. 6
Are you effectively prepared for your assessment? Are they well timed?
How do teaching staff provide you with feedback after an assessment?
Do they encourage you and enable you to reflect on your progress? How
does this help your learning?
Are you aware of any learners who have special arrangements for
assessments on account of additional support needs?
What kind of guidance and information did you receive before you applied/
when you were applying to the programme? Was it accurate and useful in
helping you make decisions?
What was induction like? Does everyone have induction? What about
second year learners?
How approachable are guidance and support staff? Are they helping and
supporting you effectively?
What types of support are available (internal and external)? How is this
publicised to learners? What access is there to specialist services?
What have you been told about progression and articulation
arrangements?
How well does the college sustain continuous enhancement of learning
and teaching through self-evaluation and internal review activities?






In what ways have learners been engaged in providing feedback on
programmes and services?
What kinds of opportunities do you have to take part in or be represented
in Programme Review Team meetings?
How do you contribute to the evaluation of the college’s provision and
services?
Is there a class representative system?
Is there evidence of improvement in any aspect of the college since you
have been here?
Give an example of improvements that have resulted from your feedback.
8
Learner engagement and Quality culture
How well are learners engaged in enhancing their own learning
and the work and life of the college?
How well is the college led and how well is it enhancing the quality
of its services for learners and other stakeholders?
How well are learners engaged in enhancing their own learning and the work
and life of the college? is an over-arching confidence statement. This means that
the review team will look for evidence of meaningful learner engagement, as
appropriate, in each element of the framework.
Therefore, the STM should liaise with learners in order to establish the extent to
which learners are genuinely engaged in enhancing their own learning and in other
processes within the college. Examples that you find (or don’t find) for this
confidence statement may also inform the statement relating to quality culture: How
well is the college led and how well is it enhancing the quality of its services
for learners and other stakeholders?
Below are some general prompts5 that you may wish to consider when speaking with
learners about learner engagement. Remember that you shouldn’t try to ask every
learner every question!
1. How does the college encourage learner engagement, generally i.e.
 in enhancing and personalising your own learning;
 in improving college programmes; and
 in enhancing the work and life of the college?
2. How do college staff take account of your views in class/in the college generally?
Do they encourage engagement with you? Do they encourage you to make
suggestions and express your opinions? In what ways?
3. How are you informed about improvements and changes that have been made to
programmes/services as a result of learners’ feedback and suggestions?
4. Is there a class representative system here? How does the system work? Is it
effective?
5. Do you know who the student president is? How useful/effective is the Student
Association here? Do they do things for you and other learners?
6. How do learners contribute to decision-making within the college as a whole?
7. On which college committees are there learner representatives? Do you receive
feedback from them? Do you know what is discussed on college committees?
5
It is also important to be aware of terminology used within the college, for example, is it a Student
Association or Student Union?
9
8. Do you attend programme team meetings (or equivalent)? What kinds of
improvements occur as a result of your inputs?
9. Have learners had any input from external agencies, such as Sparqs and NUS?
Have learners had any other training to help them engage with the college
effectively? Does the college have other training arrangements in place for class
reps?
10. Do learners get involved in planning and contributing to college events and
activities?
11. Are learners encouraged to use their skills to benefit the wider community?
How?
10
D.
FINDING THE LEARNERS AND THEIR INFORMATION
1. Every college is different and therefore every external review will be different.
2. The MI will speak with you at the beginning of the week to explain how your
meetings should be scheduled and ensure that you are covering all the relevant
campuses and outreach bases.
3. It is important that we do not disrupt learners and learning during the external
review week.
4. You should try to speak with learners in non-scheduled meetings, in informal
areas of the college, outwith your programmed meetings. You will arrange this
with the MI and you should visit areas such as refectories/cafes, learning
centres, common rooms and social areas (including Students’ Association
facilities). Remember to ensure that you cover all campuses at some point
during the review. You must politely ask learners if they are aware that the
external review is happening this week and would they mind giving you a few
minutes to answer some questions. You must assure them that their responses
are non-attributable and that they help you to construct a picture of various
aspects of the learner experience in the college. The MI will advise you how to
go about this.
5. You will have a meeting with the student president and/or his or her
representative(s). This meeting is an indicator of the learners’ view of the
college. You should use the response to the learner engagement questionnaire
as an initial basis for your discussions. However, there may be occasions
where the president’s view and the learners’ views do not agree. If this occurs,
seek advice from the MI.
11
E.
YOUR REPORT AND DISCUSSION OF FINDINGS WITH THE
COLLEGE
Your report text
1. Prior to the review, you will be provided with templates - Evidence Forms (EFs)
and Record of External Review Findings (RERF) - on which you should record
your findings. The MI will ensure that you know how to use these. Only issues
that have been triangulated by your own evidence, or evidence from other team
members, can be recorded.
2. You must be mindful of Freedom of Information (FOI) and equalities issues at all
times when recording your findings.
3. Observations from your week’s activity will normally be discussed each afternoon
at team meetings and more extensively at the team meeting on Friday.
4. As well as completing a RERF, you will also prepare a written report that forms
the basis of your oral report to the college on the Tuesday. This will be sent to the
college, following the moderation event. The MI will advise on the wording and
structure of the report (see Appendix 2). The report should not exceed 1,500
words and should be written in plain English, under the following headings.
o
o
o
o
The college context
High quality learning
Learning engagement
Quality culture
5. The MI will check your report with you. It will also be subject to a quality
assurance check on Monday at the moderation meeting.
6. As with the reports of other team members, your report findings will be
incorporated into the final report by the MI and lead writers.
The Tuesday discussion of findings
1. As STM, you will be present throughout the Tuesday discussions.
2. Your findings will have been incorporated into the final report presented to the
college at this meeting.
3. As STM1, you will deliver your report orally at the discussion of findings and the
MI will ask you to provide the principal, senior managers and student
representative(s) with contextual detail which will be beneficial to the college. In
addition, it is likely you will be asked for additional detail on aspects of your
findings.
12
Appendix 1
Evidence form
College:
Reviewer:
Date:
Reference no:
Participants:
Activity: discussion, visit, documentation:
Record of Findings
Reference QI/s
13
Record of Findings
Reference QI/s
14
Appendix 2
Student Team Member report pro-forma
The headings below should be those you use in the report. The bullet points are
there to help you structure your report sand are not intended to be mandatory or
exclusive.
Name of college
Review w/c enter date
STM Report
Name of STM
The college context

First impressions, feel of the college/campuses

Number of learners, campuses visited, subject areas

Evidence gathered through class representative meetings, Student
Association meetings or any scheduled review meetings
 Any special event, activity, notable campus speciality
ENTER TEXT
High quality learning

Learning and teaching methods, learning and teaching environment, feedback
on progress, independent learning

Learner experience of pre-course, on-course and pre-exit advice, guidance
and support

Learner experience of assessment, student services and other college
services to support learning

Personal development for learners, preparation to progress to employment or
further study, acquisition of life skills.
ENTER TEXT
15
Learner engagement

Culture of learner engagement, role of Student Association and cross campus
arrangements.

Class representative arrangements and training, representation of learners on
college, team or programme committees and/or Board of Management

Relationship between Student Association, class representatives and college
staff/managers for engagement with learners
ENTER TEXT
Quality culture

Learners’ knowledge of Student Association, class representative
arrangements, quality processes for learning and teaching and other college
services.

Arrangements for learners to raise concerns/comments with college staff
about their experience and how well the college responds to feedback.

Examples of actions raised and addressed to improve the learner experience.
ENTER TEXT
16
Appendix 3
Checklist for reports
Quality check your report against the following list:

Have your told a clear and coherent story of the learner experience?

Is there unnecessary repetition?

Are there any key messages that apparently contradict each other?

Are the key strengths and areas for improvement clearly flagged up in the
report? Do they accord with those in the main report?

Have you laboured some points unnecessarily (e.g. the Student Association)?
Avoid –















6
Passive voice
Long sentences
Too many ideas in the one sentence
Descriptive rather than evaluative text
Jargon
Nonsense
Punctuation errors (especially apostrophe’s6)
Grammar errors
Use of acronyms
Content in the wrong place
Lack of coherence in paragraphs
Over-use of some words e.g. effectively
Hanging ‘this’ at the end of a sentence
Tautology e.g. ‘good relationships have been well developed’
Repetition – of words and of key points.
This is an example of an error!
17
Appendix 4
Glossary of terms
AA
Associate Assessor Practitioners, generally senior managers, who are
recruited from the sector and participate in external review activities
AEV
Annual Engagement Visit Formal annual visits to colleges by the college
HMI to consider certain aspects of quality improvement.
EF
Evidence form Recording tool for all reviewers, to record meetings,
documentation, observations etc.
HMI
Her Majesty’s Inspector
MI
Managing Inspector The Managing Inspector is an HMI who has the
responsibility of leading the particular review activity.
PI
Performance indicator These are the quantifiable indicators used by
colleges, Education Scotland and the Scottish Funding Council in order to
signal learner success. The measure used are generally – enrolment, early
withdrawal, further withdrawal, partial success (completed the programme
but did not achieve the award), and successful completion.
QI
Quality indicator The Quality Indicators are the 32 separate elements that
make up the quality framework used by Education Scotland. There are 13
mandatory QIs that are used on all reviews. The others may be used on
reviews, depending on issues within the sector and within the college that
may signal excellence of risk.
RERF
Record of external review findings Documentation used on external
reviews to capture evaluations emerging from meetings, observations and
documentations and recorded on EFs.
STM
Student Team Member YOU!
TM1
Team member one Generally external review teams have a designated
TM1 who has certain responsibilities in the organisation and management
of the review. This person will generally accompany you and the MI to the
sharing of findings.
18
Quantification terms:
Education Scotland uses the following terms to describe numbers and proportions:
Almost all
Most
Majority
More than a few
Few
over 90%
75-90%
50-74%
15-49% However, this term is rarely used and generally terms
such as ‘almost half’, ‘a third’, ‘25%’ and so on are used.
up to 15%
19
Appendix 5
The PRAISE framework has been developed as a best practice guide to inspection
and review approaches, particularly in relation to professional dialogue with college
staff and dialogue with learners. The principles are set out below:
Best Practice Framework
for inspections and reviews
Purpose – being clear about the overall purpose of the inspection/review and
retaining this throughout. Creating a shared agenda with staff in the organisation
and amongst members of the inspection/review team.
Relationships – building and maintaining constructive relationships
throughout the process as the basis of a high quality inspection/review.
Awareness – maintaining a high level of awareness of the context in which
staff are operating, of their feelings and reactions to the process and of the
inspector’s own approach and its impact.
Information gathering – careful inquiry to gather and analyse evidence.
Retaining an objective stance, testing assumptions and assimilating data before
evaluating.
Sharing information – communicating thoroughly throughout the process to
prepare and inform staff. Encouraging staff to be open in providing their
perspective and sharing findings as the inspection/review progresses.
Enabling – treating people with respect, engaging them in professional
dialogue, recognising their efforts and sharing findings in a constructive way to
encourage ownership and learning to take place.
20