ATTITUDINAL SURVEYS
AND SELF EVALUATION
JUNE 2011
London Conference
Geoff Davies
What are schools about?
•
•
•
•
•
•
Effective Learning
Opening windows on a world
Choices
Realising potential
Achievement and Standards
Qualifications
BUT DO YOU RECOGNISE THIS???
You arrive at school one morning and you receive the following information
•Two pupils have misbehaved on the bus upsetting a third child
•A parent has phoned saying her child is being bullied in school and refuses to
come to school
•A pupil has told her LSA that her mother has slapped her hard across the face
the previous evening and there are finger marks on her face
•Two girls have been fighting in the canteen before school
•A member of staff is very upset after a disagreement with a colleague
•The gym roof is leaking and you have examinations scheduled there for the day
•You receive a serious complaint about a teacher’s conduct
•You are due to attend a meeting in an hour with the Chair of Governors
What are schools about?
• Preventive Detention
• Social function
•
•
Nobody else may take on this function
•
•
•
•
•
•
Effective Learning
Opening windows on a world
Choices
Realising potential
Achievement and Standards
Qualifications
For some pupils just keeping them off
the streets is a major achievement
• Providing security and a safe
environment
• Pupils not at ease will not learn
Schools are complex organisations
Far too complex to make decisions by hunch!
Can you really know your school without detailed research and surveys?
Think about
SELF EVALUATION
What do you do already?
Why do you do it?
What might you do to develop your
practice?
Can the CEM centre help?
Best practice in self-evaluation
A survey of schools, colleges and local authorities
OFSTED REPORT JULY 2006
Best practice
Leaders gave priority to and led self-evaluation personally.
They had a clear overview of their institutions, based on an accurate understanding of
strengths and weaknesses
Self-evaluation was integral to the culture of the organisations. People at all levels were
committed to it and fully involved. It was a continuous process, governed by the needs of
the institution rather than the requirements of external bodies. It was clearly built into
management systems.
External inspection supported but did not replace internal review. It provided, however, the
main external source of validation. An increasingly sophisticated use of a widening
range of performance indicators enhanced the quality of self-evaluation.
Rigorous analysis of strengths and weaknesses, particularly of teaching and learning,
led to the clear identification of priorities and strategies for improvement.
Sharply focused monitoring, based on clear indicators, helped institutions to measure the
extent to which their work improved outcomes for pupils and young people.
The views of those who received services, particularly learners but also parents and
carers, were actively sought and influenced decision making.
Schools put the process of self evaluation at the heart of their culture
WIDE RANGING PUPIL ANALYSIS
PROMOTING ACTION RESEARCH
EFFECTIVE QA SYSTEMS
PERCEPTION SURVEYS
SCHOOL SELF EVALUATION
DEVELOPING LEADERSHIP AND MANAGEMENT SKILLS
EFFECTIVELY TARGETING AND MEASURING SUCCESS
COLLABORATIVE REVIEWS
BENCHMARKING TO HELP JUDGEMENTS
It is better to identify a problem yourself than have others do it for you
25 YEARS OF CEM CENTRE DATA
Carol Fitz-Gibbon 2001 British Psychological Society
It gradually dawned on me that providing the data to
schools was the most important outcome of the effort, far
more important than writing research papers…..
The provision of data to practitioners meant that they
participated in the research. Indeed they were the only
ones who knew the surrounding circumstances for their
classrooms, their department, each pupil, each family,
etc. They were the major players: the ones who could
interpret and learn from the detailed data.
DOING IT ALL!
PRIORITISING
Can school leaders find the time?
Important but not urgent
Is it more like? 65%-80% or 15%
Urgent but not important
Is it more like?
I keep six honest serving men
Who taught me all I know
Their names are What and Why and When
And How and Where and Who
Rudyard Kipling The Elephant’s Child
15% or 60%
WHAT information do I want?
WHY do I want it?
WHEN do I need it?
HOW do I collect it?
WHERE can I find it?
From WHO do I get it?
The CEM centre ticks many of these boxes for schools
Paper based surveys:
•Alis and Yellis have their ‘Extended Questionnaires’
•Yellis probes attitudes to school and student
perceptions of the school and also their subjects
• Alis focuses extensively on their subjects
•A wealth of historical data
On-line attitudinal surveys
• MidYIS, SOSCA, INSIGHT Yellis and Alis
• Developed to meet new needs
• Can be more easily adapted to meet new
developments
• The Parental Questionnaire
• The Event mapper
Why use our surveys when you
can write your own?
• Questions re-analysed for reliability and
validity
• No data to process
• ‘background’ of other schools enables you
to see differences ,if they are significant
and monitors attitudes over time
Some examples
Yellis Attitudinal Feedback
Attitudinal Comparison Graphs
The Yellis Baseline questionnaire looks briefly at students' attitudes
towards education and their aspirations for the near future. Feedback from
this questionnaire takes the form of five graphs that look at students' job
aspirations, cultural capital, likelihood of staying in education and feelings
about their current school.
Basic Comparison Graphs2010.pdf
.
Basic Comparison by Gender Graphs.pdf
Attitudinal Trend Graphs
These 35 graphs illustrate how your student's attitudes change year-byyear - based on the responses given during the Yellis Baseline Test
session.
Year 10 Basic Comparison Trends.pdf
Further Attitudinal Comparison Graphs
Feedback from this questionnaire takes the form of
23 graphs that illustrate how the responses of your
students to the Attitudinal Questionaire differ when
compared to all Yellis students in the same cohort.
Year
. 11 Extended Comparison Graphs.pdf
Extended Comparison Graphs
The following list gives the titles of Comparison Graphs
currently provided by the Extended Yellis questionnaire:
•Alienation Indicators
•Attitudes to Design & Technology
•Attitudes to English
•Attitudes to a Foreign Language
•Attitudes to Mathematics
•Attitudes to Science
•Career-Relevant Activities Experienced
•Sources of Careers Information Found Useful
•Preferences for Kinds of Work
•Items in the Freedom from Free Scale
•Design and Technology Homework
•English Homework
•Foreign Language Homework
•Mathematics Homework
•Science Homework
•Career Choice Motivators
•Parental Involvement: Mothers
Your data is above the Yellis average
•Parental Involvement: Fathers
Your data is below the Yellis average
•Influences on Staying-On
•Traumatic Events Experienced by Pupils
Your data is about the same as the Yellis average
•Places Where Pupils Feel Unsafe
•Work Place Preferences
Your data is about the same as the Yellis average
•Cigarettes, Alcohol and Drugs
Further attitudinal trends
These 197 graphs illustrate how your student's
attitudes change year-by-year - based on the
responses given on the Yellis Attitudinal
Questionnaire
Online Attitudinal Questionnaires
The Extended MidYIS Attitudinal Questionnaires will help you to answer these questions and meet
the relevant requirements set out in the Framework for inspecting schools.
Topics Covered
Three questionnaires are included in the registration fee; the General, Induction and Parental
questionnaires.
General Questionnaire
The General questionnaire includes a series of questions relating specifically to the completion of
your Self Evaluation Form (SEF) ie bullying, racism, sectarianism, healthy lifestyles.
Other sections include:
Attitudes to school and study: do your students enjoy and feel happy in school?
Attitudes and motivation toward subjects: what do your students feel about the classes they
take? How does their motivation vary with subject?
Bullying in school: do your students ever feel threatened or bullied? If so, where?
Extra-curricular participation: do your students participate in additional clubs, sports or school
trips?
Drug use: how prevalent are cigarettes, alcohol and illegal drugs amongst your students?
Career aspirations: what expectations do your students have for their future after school?
Peer pressure: do your students feel pressure from their friends to behave in anti-social or antischool ways?
Socio-economic status: what are the educational levels and occupational status of the parents of
your students?
Cultural capital: what support for learning (books, visits to museums, libraries) is there in your
students' homes?
We recommend that the general questionnaire is taken in term 2 or 3.
The general questionnaire takes about 45 minutes to complete.
Enhance your feedback by providing your teacher groups! - If you want to enhance your
subject data please provide us with your pupil teacher groups
Healthy Lifestyles :
I take part in PE and Games lessons.
I take part regularly in a sporting or outdoor activity provided by the school either at lunchtime,
after school or at weekends.
I take part in playground games at lunchtimes or breaktimes.
I make use of the sporting facilities at school.
The school encourages me to be physically healthy.
The school encourages everyone to take part in sports no matter how good or how bad they are.
The sporting and recreational facilities at school are excellent.
I am unlikely to smoke because the school has told me about the dangers of smoking.
How many cigarettes per day can someone safely smoke ?
I eat less sweets and crisps now because the school has told me about the health risks involved.
I eat more healthily because the school has warned me not to eat too much fried food.
I am more likely to drink responsibly because the school has told me about the dangers of drinking
alcohol.
How many measures of alcohol (glasses of wine, half pints of beer) can someone drink before it
affects their driving ?
How often do you see bullying in the corridors at the change of lessons ?
How often do you see bullying in the toilets ?
How often do you see bullying in outdoor lessons ?
How often do you see bullying at lunchtimes ?
How often is there bullying via e-mail in school ?
How often is there bullying by text messaging in school ?
I understand the school’s policy on bullying.
I know who to tell if I am bullied.
The school would be able to stop me being bullied if it happened.
I would tell someone in school if I saw someone else being bullied.
There is a lot of bullying in school.
ALIS
The Extended ALIS Questionnaire, generally takes
about an hour to complete, covers many areas including attitudes to school (or college), courses,
Teaching and Learning and career plans.
Students are also asked to comment on their
responses to some open-ended questions.
The questionnaire can either be taken on paper (in
Term 2 of Year 2) or online (in Term 2 of Year 1).
How has this been used?
PEDAGOGY….ALIS surveys
SUBJECT A
Why the
improvement?
2008
SUBJECT A 2005
SUBJECT A 2007
SUBJECT A 2008
DO NOT GET TOO EXCITED!
We have made a comparison of perceived teaching
methods as analysed by ALIS in 2004-5 with those in
2007-8. Some subject areas have appeared to change
their methods radically. Others have not. Though the
samples are small it is an interesting exercise to try to
correlate it with the departments statistical process
charts over that period. One would like to say that
changes in the variety of teaching methods result in
improvement but the evidence is a little tenuous so far.
e
E x
x a
e m
r p
c a
i p
s e
e r
s s
SUBJECT
e
s
s
a
y
s
R
e
a
d
i
n
g
d
i
s
c
u
s
s
i
o
n
s
n
o
t
e
s
d
i
c
t
a
t
e
d
o
w
n
n
o
t
e
s
h
a
n
d
o
u
t
s
P
r
a
c
t
i
c
a
l
w
o
r
k
U
s
i
n
g
A
V
I
T
R
e
s
e
a
r
c
h
i
n
g
P
r
e
s
e
n
t
i
n
g
h
e
l
p
i
n
g
s
t
u
d
e
n
t
s
o
r
i
g
i
n
a
l
w
o
r
k
2005 to 2008
ART
BIOLOGY
BUSINESS
CHEMISTRY
COMPUTING
DRAMA
FOOD
PRODUCT DESIGN
ENGLISH
FRENCH
GEOGRAPHY
HISTORY
ICT
MATHS
MUSIC
PE
PHYSICS
RE
WELSH
YES
NO
MORE (M)
LESS (L)
SIGNIFICANT
CHANGE
YES
NO
YES
NO
YES
NO
YES
YES
NO
YES
YES
NO
YES
NO
YES
YES
YES
NO
YES
11
7
54
29
L
L
M
L
L
L
M
L
M
L
L
L
L L
M
M
M
L
L
L
M
L
L
M
M
L
M M
M
L
M
L
L
M
M
M
M
L
M M M
M
M M M
M
M M M
M
M
L M
M M
M
M
M
M M
M
4
3
M
M
M M
L
L
L
5
3
L
L
1
2
L
3
6
4
3
5
1
6
2
5
1
M M M
M
M
M M
M M
L
M
2
2
3
1
2
0
3
0
2
2
6
3
3
0
A USEFUL STAFF
DEVELOPMENT
EXERCISE
SOSCA Attitudes to Science Survey
The Science Assessment includes a survey of pupils' attitudes to
science. The feedback from the survey shows you each pupil's
average score on each sub-section, on a scale of 1 to 5 from
negative to positive attitudes, and their overall score.
How your school's pupils scored on each sub-section is also
shown against a background of all the pupils in the SOSCA
sample.
Lastly, pupils are asked to reply to three questions about
science and their free-text comments are reported anonymously
and in random order for your information.
Perception of Science
This area covers all aspects of science outside the classroom and in the wider world. A typical
question is:
Science causes many environmental problems.
Relevance of Science to Self
This section explores pupils' perceptions of how science relates to them, in particular when
studying science at school. A typical question is:
I need to do well in science to get the job I want
Enjoyment of Science
This area explores pupils' enjoyment of the science they are taught in school. A typical question
is:
I look forward to doing experiments
Environmental Empathy
This section includes pertinent questions about waste and the environment to assess what the
pupils have done recently with regard to actions generally considered good or bad for the
environment. A typical question is:
My empty cans always get recycled.
.
Whole School Summary
FINALLY
THE MOST IMPORTANT WORD IN
EDUCATION???
CULTURE
Changing attitudes and the
culture of the school is a long
game so you need reliable data
over a period of time
CHALLENGE
Take action, with meaningful consequences for success or failure.
ATTITUDINAL SURVEYS
AND SELF EVALUATION
JUNE 2011
London Conference
Geoff Davies