FFT Data Analysis Project
© Fischer Family Trust, 2009
Using FFT Live 3.0
How FFT data can help your school improve
Primary Schools (KS2)
Fischer Family Trust
June 2009
www.fischertrust.org
FFT Data Analysis Project
© Fischer Family Trust, 2009
KEY AIMS OF THIS TRAINING SESSION
To gain an overview and use some of the key
features AND reports available on FFT Live
To interpret and use some of the main reports
available on FFT Live
To gain an understanding of how FFT data, alongside
other data, can be used in key situations (SEF, self
evaluation, inspection, target setting) by a variety of
staff to help your school and LA improve
Allow time for discussions on FFT Live, FFT reports,
‘best practice’ and potential ‘misuse’ of data
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Aims and Ethos

Background to the Fischer Family Trust (FFT)
 Independent charity, founded by Mike Fischer, co-founder of Research Machines
(RM). FFT supports projects in health and education
 Education Projects include the data analysis project managed by Mike
Treadaway, using national data sets provided by DCSF and Welsh Assembly
Government
 Aim is to help schools make effective use of value-added test and TA data to
raise individual pupil attainment and improve progress

Analyses are provided to support effective school self-evaluation:
 Data provides questions not answers
 Data analysis should be used by schools to promote discussion, evaluation and
planning
 Analyses for different groups of pupils, and a range of indicators, to help
identify strengths or areas for development/intervention
 Use the past to inform the future
FFT Data Analysis Project
© Fischer Family Trust, 2009
Background to using data
FFT Data Analysis Project
© Fischer Family Trust, 2009
A five-stage cycle for school improvement
Where can FFT
data
support school
improvement?
Source: DCSF
FFT Data Analysis Project
© Fischer Family Trust, 2009
Before we start – the concept of triangulation
RAISEonline/
other data models
FFT analysis
Professional Judgement
Basis for action
Investigate Further
Check Accuracy
Challenge Assumptions
FFT Data Analysis Project
© Fischer Family Trust, 2009
Introduction to FFT Live
FFT Data Analysis Project
© Fischer Family Trust, 2009
How can I access FFT data?
From your LA
Paper reports
Electronic reports (pdf)
Excel files
School FFT database
Direct via FFT Live 3.0
www.fftlive.org
Logins are available
From your LA FFT contact
or School FFT contact
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0
 What is FFT Live 3.0?
 FFT Live 3.0 is FFT’s updated online system giving
schools and LAs online access to a range of FFT data
and reports to support target setting and self evaluation
 The reports are based on similar methodologies and
models used in the standard database reports
 FFT Live 3.0 gives users access to a greater range of
reports than the previous version of FFT Live and also
includes a number of reports which are only available
online
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0
 Who is FFT Live 3.0 aimed at?
 FFT Live includes a broad range of reports to support target
setting and self evaluation and can be used by a wide variety of
staff in schools
 Teachers, Department Heads and Heads of Year will find the new
pupil/subject estimates and value added information particularly
useful in helping to review progress and set challenging targets.
 SLTs can now access a greater range of strategic data to support
whole school and group evaluation and target setting
 School governors can also access school level data to support
SLTs in evaluation and planning.
 LA staff including advisers, SIPs, information teams and senior
managers
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0
 Why use FFT Live?
 Allows quicker access to a wide range of FFT reports and data for school
and LA staff
 Allows access to pupil, group and school data to support target setting
and self evaluation
 Some reports (e.g. pupil estimates showing a range of probabilities) are
only available on FFT Live
 Greater level of flexibility allowing users to choose their own options (e.g.
model types and ranks)
 Allows users to export data directly from FFT Live
 Gives access to Self Evaluation booklets
 Improved look and feel
 Online help and guidance available for individual reports
FFT Data Analysis Project
© Fischer Family Trust, 2009
Logging in to FFT Live 3.0
 How do I log-in to FFT Live 3.0 (www.fftlive.org)?
 Schools can now access FFT Live in a 4 ways:
1. SCHOOL ADMIN account supplied by the LA (e.g. 9992000Admin) or
created by a school itself (e.g. ASmit9999) – max 2 per school
2. SCHOOL USER account (e.g. ASmit9999). An FFT Live school admin
account holder can create as many user accounts as needed for staff in
schools (e.g. teachers).
3. GENERIC SCHOOL account (your school number – e.g. 9992000). Using
this account you can access school and group level data. The account is
supplied by your LA.
4. GENERIC PUPIL account (your school number plus ‘P’ – e.g. 9992000P).
Using this account you can access school, group AND PUPIL level data.
The account is supplied by your LEA.
FFT Data Analysis Project
© Fischer Family Trust, 2009
Logging in to FFT Live 3.0
 How do I log-in to FFT Live 3.0 (www.fftlive.org)?
 If you don’t have an account of your own, you can also access FFT
Live using an ANONYMOUS account:
1. ANONYMOUS SCHOOL ACCOUNT
Access school reports
Username: 9992004
Password: ANON (case sensitive)
2. ANONYMOUS PUPIL ACCOUUNT
Access school AND PUPIL reports
Username: 9992004X
Password: ANON (case sensitive)
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0 Main Menu
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0 Help Facilities
You can also access specific help
on each report by choosing the
‘Help with this report’ option.
You can access a range of help
topics from any screen using the
Help menu
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0 Report Screen Layout
(3) View Report button
(1) Menu Bar
(4) Page options
(2) Drop-down options
(5) Resize screen options
Press this button whenever
you make any changes
(6) Export and Print options
Export options include
PDF, Excel, CSV, and XML
FFT Data Analysis Project
© Fischer Family Trust, 2009
FFT Live 3.0 Report Screen Layout
(1) Report title and details
(3) Main report
(2) Description of report
(4) Key
(2) School and LA name
FFT Data Analysis Project
© Fischer Family Trust, 2009
What’s on FFT Live 3.0?
Value Added Significant Areas 3 Year Summary Report (KS1-2)
Value Added Significant Areas 3 Year Detail Report (KS 1-2)
FFT Data Analysis Project
© Fischer Family Trust, 2009
What’s on FFT Live 3.0?
Pupil VA Summary Report (KS1-2)
FFT Data Analysis Project
© Fischer Family Trust, 2009
What’s on FFT Live 3.0?
Pupil Estimates Summary Report (KS2)
Pupil Estimates Detail Report (KS2)*
*Only available on FFT Live3.0
FFT Data Analysis Project
© Fischer Family Trust, 2009
What’s on FFT Live 3.0?
School Estimates Report (All KS)
FFT Data Analysis Project
© Fischer Family Trust, 2009
What’s on FFT Live 3.0?
Analyses to Support Self Evaluation Booklet**
**Available through school logins only
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for self evaluation
and target setting using FFT
Live
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
(1) VA Sig Areas Grid
How well are our learners doing (pupil/group/school)?
What about in-school variation?
What interventions have/haven’t worked?
What are the big issues for our school?
What other information can we use to ‘triangulate’
our evaluation?
(2) VA Sig Areas Detail
(3) Pupil VA Summary Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
What more should we aim to achieve?
Which areas require intervention?
What must we do to make it happen?
How are we going to track pupils?
How do we measure success?
(4) Pupil Estimates Detail Report
(5) School Estimates Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
Improving self evaluation in schools: Using FFT estimates
PART 1
Using FFT Live for school self evaluation
FFT Data Analysis Project
© Fischer Family Trust, 2009
A description of value added
 What is Value Added?
 Basic Value Added (VA) measures the PROGRESS made by
an individual pupil, or group of pupils, between Key Stage 1
(the starting point) and Key Stage 2 (the end point).
 Basic VA compares the progress made by each pupil with
the average progress made by ‘similar pupils’, defined by
FFT as pupils with similar prior attainment, gender and
month of birth.
 Contextual Value Added, or CVA, is a version of VA which
also takes into account school context factors and
individual pupil context factors
FFT Data Analysis Project
© Fischer Family Trust, 2009
A description of value added
 What is Value Added?
 VA or CVA can be calculated for individual pupils,
groups of pupils or for a whole school.
 Group or school level VA scores are simply based on
the average of individual pupil VA scores.
FFT Data Analysis Project
© Fischer Family Trust, 2009
Value Added Models - Overview
2 basic models are used in FFT Live:
 Model PA (Prior Attainment) - Value Added
Prior
Attainment
Gender
Month of Birth
 Model SX (School EXtended) – Contextual Value Added
Prior
Attainment
Gender
Month of Birth
School
Context
Pupil
Context
FFT Data Analysis Project
© Fischer Family Trust, 2009
Factors included in FFT VA Models
Pupil Factors
PA
SX
Mean Test Level (where available)


Mean TA Level


Subject Variations


Gender


Month of Birth


EAL

FSM

SEN Stage, Statemented

Ethnicity

Mobility (joined late / time in school)

School Factors
PA
SX
Mean Intake Test Level

Spread of Intake Test Level

FSM Entitlement (Percentile Rank)

Geodemographic Data (Percentile Rank)

FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
(1) VA Sig Areas Grid
How well are our learners doing (pupil/group/school)?
What about in-school variation?
What interventions have/haven’t worked?
What are the big issues for our school?
What other information can we use to ‘triangulate’
our evaluation?
(2) VA Sig Areas Detail
(3) Pupil VA Summary Report
Guidance:
KS1 to
KS2 -Value
Added (Significant
Step 3: ‘Drilling
down’
KS5 Subject
Estimates Areas Grid)
VA Basis: Select from 2 different
value added models – PA (Prior
attainment) and SX (School
Extended)
PA (Prior attainment) is a simple
value added model where estimates
are based on the progress of similar
pupils nationally taking into account
pupil prior attainment, gender and
month of birth.
Matched Pupils: Those pupils who’s
prior attainment results can be matched
with the most recent results. When using
this report be careful if there are large
differences between the total number of
pupils in a group and the number of
matched pupils used in the report.
SX (School Extended) is a full
contextual value added model
where estimates are based on the
progress of similar pupils nationally
(PA model factors plus SEN status,
FSM status, ethnicity, mobility and
EAL) in similar schools (based on
free school meals, deprivation,
cohort ability).
Pupil Group: Select
All pupils (default) or
‘multi-select’ any
combination of pupil
groups (girls, boys,
FSM etc.)
Key Stage: Select
KS1-2, KS2-3, KS3-4
and KS2-4 VA reports
where available.
Group Size: Only show Pupil Groups with a minimum
of 5,10, 15 or 20 pupils. Default setting is 5.
What does the report show?
This report highlights potential strengths,
weaknesses and trends across a range of
indicators, subjects and pupil groups in the
school. The reports covers a 3 year
period.
In this example Science appears to be a
potential issue with a large number of
areas highlighted in blue (showing that
actual attainment was significantly below
the estimated attainment over the last 3
years.
However, the upward arrows also show
that there was a significant improvement in
value added in one year (e.g. the % of girls
achieving Level 5+ Science).
In contrast however, a number of other
areas are highlighted in green (e.g. % of
girls attaining L5+ in Maths) showing that
performance in this these areas was
significantly better than estimated.
Value added scores are calculated
by comparing the attainment of each
individual student with that of
‘similar’ pupils nationally. Where
attainment is better than expected
then value added is positive and
vice versa.
The blank areas show where actual
attainment was broadly in line with the
estimated performance (i.e. no statistically
significant difference between estimated
and actual performance).
Year: Select report for a 3 year
period (e.g. 2004/05 – 2006/07).
Pupil Group: Groups are split by
gender, prior attainment, FSM, SEN,
ethnicity and LAC. Girls/Boys Lower,
Middle and Upper Groups are based
on an average of Test and TA prior
attainment results. Pupils in the Lower
group, for example, are in the lowest
third nationally based on prior
attainment. Pupils in the Middle group
are in the middle third and those in the
Upper group are in the top third
nationally based on average test/TA
prior attainment results.
What do the colours and arrows indicate?
The colours show where actual attainment is statistically significantly above or below estimated performance over a 3 year period (based
on the progress of similar pupils nationally).
A green highlight shows where actual attainment is significantly higher than estimated attainment over a 3 year period. A blue
highlight shows where actual attainment is significantly lower than estimated attainment. If there is no highlighting this shows that there
is no statistically significant difference between the actual and estimated performance.
Arrows show where there is a statistically significant change in VA performance over a 3 year period compared to similar pupils
nationally. Arrows pointing upwards show there has been a significant improvement in one year (↑) or both years (↑ ↑). Arrows pointing
downwards show where there has been a significant decline in performance in one year (↓) or both years (↓↓). A significant improvement
in one year followed by a significant decline the following year (or vice versa) is represented by a ↑ ↓ symbol.
Example:
KS1 todown’
KS2 Value
(3Estimates
Year Significant Areas Grid)
Step 3: ‘Drilling
- KS5Added
Subject
FFT Data Analysis Project
© Fischer Family Trust, 2009
How is statistical significance shown in reports?
Differences between estimated and actual results are shown
using colours:
Where the Actual result is:
Significantly above the estimate (in statistical terms)
above the estimate / below the estimate / same as estimate
Significantly below the estimate (in statistical terms)
FFT Data Analysis Project
© Fischer Family Trust, 2009
How is statistical significance over time shown in reports?
Significant changes over time (trends) are shown using arrows:
Where the Actual result is:
Significantly better in one year compared to another
Significantly better two years running
Significantly worse in one year compared to another
Significantly worse two years running
Volatility – significant improvement and significant decline over 3 year
period
FFT Data Analysis Project
© Fischer Family Trust, 2009
Significant Changes
The arrows show significant changes NOT a change in significance
Y
r
1
Example 1
Value-added
significantly
below
Example 2
Y
r
1
Value-added
significantly
below
Y
r
2
Significant Improvement
Value-added broadly inline with other schools
Significant Improvement
Y
r
2
Y
r
3
Value-added
significantly
above
Significant Improvement Y
Value-added broadly inline with other schools
r
3
Value-added
significantly
above
FFT Data Analysis Project
© Fischer Family Trust, 2009
Significant Changes
The arrows show significant changes NOT a change in significance
Significant decline
Y
r
3
Example 3
Value-added
significantly
below
Y
r
2
Value-added broadly inline with other schools
Y
r
1
Value-added
significantly
above
Yr1 – Yr 2: Small change in value added rank but
significant change in actual value added as it is at the
extreme. This occurs when VA is very high or very low
Significant decline
Example 4
Y
r
3
Value-added
significantly
below
Y
r
1
Significant Improvement
Value-added broadly inline with other schools
Y
r
2
Value-added
significantly
above
FFT Data Analysis Project
© Fischer Family Trust, 2009
Statistical significance
 In statistics, a result is called statistically significant if it
is unlikely to have occurred by chance.
 "A statistically significant difference" simply means there
is statistical evidence that there is a difference; it does
not mean the difference is necessarily large, important or
significant in the common meaning of the word.
FFT Data Analysis Project
© Fischer Family Trust, 2009
When is something likely to be “statistically significant”?
It is difficult to say without actually calculating statistical
significance as each individual situation will vary.
However, in general, the likelihood of something being
statistically significant will increase:
 as the difference between the estimated and actual result
for a group of pupils become larger
 as the number of pupils in a cohort or group increases
 As the ‘spread’ of results for a group of pupils
reduces
FFT Data Analysis Project
© Fischer Family Trust, 2009
Activity 1: Using the Significant Areas Grid Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
(1) VA Sig Areas Grid
How well are our learners doing (pupil/group/school)?
What about in-school variation?
What interventions have/haven’t worked?
What are the big issues for our school?
What other information can we use to ‘triangulate’
our evaluation?
(2) VA Sig Areas Detail
(3) Pupil VA Summary Report
Guidance:
KS1 to
KS2 -Value
Added (Significant
Step 3: ‘Drilling
down’
KS5 Subject
Estimates Areas Detail)
VA Basis: Select from 2 different
value added models – PA (Prior
attainment) and SX (School
Extended)
PA (Prior attainment) is a simple
value added model where estimates
are based on the progress of similar
pupils nationally taking into account
pupil prior attainment, gender and
month of birth.
Matched Pupils (3yrs): Those pupils
who’s prior attainment results can be
matched with the most recent results.
When using this report be careful if there
are large differences between the total
number of pupils in a group and the
number of matched pupils used in the
report.
Group Size: Only show Pupil
Groups with a minimum of
5,10, 15 or 20 pupils.
Default setting is 5.
Key Stage: Select
KS1-2, KS2-3, KS3-4
and KS2-4 VA reports
where available.
SX (School Extended) is a full
contextual value added model
where estimates are based on the
progress of similar pupils nationally
(PA model factors plus SEN status,
FSM status, ethnicity, mobility and
EAL) in similar schools (based on
free school meals, deprivation,
cohort ability).
In this example, Maths Level 5+ is
highlighted for Girls. This is because for
the 3 year period 2004/05 – 2006/07, the
combined girls achievement was
significantly above estimated
achievement (based on the progress of
similar pupils nationally.
It is important to understand that for an
area to be significantly better than
expected, this does not mean that every
single year has to be green. In this case,
the estimate/actual difference is only
significant in one year (05/06). The other
years show actual results above estimates
(positive numbers) but they are not
significant. However, the combined effect
over 3 years is significant and thus Maths
Level 5+ is shown as ‘Significantly above’
for Girls in the Detail report and on the
Significant Areas Grid.
Value added scores are calculated
by comparing the attainment of each
individual student with that of
‘similar’ pupils nationally. Where
attainment is better than expected
then value added is positive and
vice versa.
Pupil Group: Groups are split by
gender, prior attainment, FSM, SEN,
ethnicity and LAC. Girls/Boys Lower,
Middle and Upper Groups are based
on an average of Test and TA prior
attainment results. Pupils in the Lower
group, for example, are in the lowest
third nationally based on prior
attainment. Pupils in the Middle group
are in the middle third and those in the
Upper group are in the top third
nationally based on average test/TA
prior attainment results.
What does the report show?
This report should be used alongside the
significant areas grid report to highlight
potential strengths, weaknesses and
trends in more detail. Data is only shown
for those indicators which are statistically
significant (i.e. where actual attainment is
significantly above or below estimated
attainment or where there has been a
significant change in performance over a 3
year period).
Actual: Actual performance of
MATCHED pupils
What do the colours indicate?
The colours show where actual attainment is statistically significantly above or below estimated performance
over a 3 year period (based on the progress of similar pupils nationally).
A green highlight shows where actual attainment is significantly higher than estimated attainment over a 3
year period. A blue highlight shows where actual attainment is significantly lower than estimated attainment.
If there is no highlighting this shows that there is no statistically significant difference between the actual and
estimated performance.
Example:
KS1 todown’
KS2 Value
(Significant
Step 3: ‘Drilling
- KS5Added
Subject
Estimates Areas Detail)
FFT Data Analysis Project
© Fischer Family Trust, 2009
Activity 2: Using the Significant Areas Detail Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
(1) VA Sig Areas Grid
How well are our learners doing (pupil/group/school)?
What about in-school variation?
What interventions have/haven’t worked?
What are the big issues for our school?
What other information can we use to ‘triangulate’
our evaluation?
(2) VA Sig Areas Detail
(3) Pupil VA Summary Report
Guidance:
KS1-2down’
Pupil-Value
Added Summary
Step 3: ‘Drilling
KS5 Subject
Estimates
VA Basis: Select from 3 different
value added models – PA (Prior
attainment), SE (Socio
Economic) and SX (School
Extended)
PA (Prior attainment) is a
simple value added model where
estimates are based on the
progress of similar pupils
nationally taking into account
pupil prior attainment, gender
and month of birth.
KS1 Results: Includes Reading, Writing and Maths
Test Levels and English, Maths and Science TA levels
where available.
Overall VA Score
An overall estimate and actual points score is
shown for each pupil. The points score is an
average across all KS2 subjects and uses the
QCA points scoring system (decimalised level x
6). For example a mid level 4.5 is 27 points (4.5
x 6). Level 5.0 = 30 points and Level 5.5 = 33
points etc.
KS2 Value Added: This section
shows the estimated and actual level
achieved for each subject along with
the probability of achieving Level
4+/5+. The Levels are shown in
decimal form (based on marks). As
an example, a level 4.0 to 4.33 is
equivalent to a 4C whilst a level 4.67
to a 4.99 is equivalent to a 4A.
A pupil’s level is highlighted where a
pupil’s actual performance is half a
level or more above (green) or below
(blue) the estimated level (based on
the progress of similar pupils
nationally). This is not the same as
statistical significance. It is simply a
highlight to draw the readers
attention to potentially high/low value
added.
SE (Socio Economic) is a
contextual value added model
where estimates are based on
the progress of similar pupils
nationally (as in the PA model) in
similar schools (based on free
school meals, deprivation, cohort
ability).
SX (School Extended) is a full
contextual value added model
where estimates are based on
the progress of similar pupils
nationally (PA model factors plus
SEN status, FSM status,
ethnicity, mobility and EAL) in
similar schools (based on free
school meals, deprivation, cohort
ability)
Value added scores are
calculated by comparing the
attainment of each individual
student with that of ‘similar’
pupils nationally. Where
attainment is better than
expected then value added is
positive and vice versa.
What does the report show?
In the example report , Keira Neon
looks to have made excellent
progress in Maths compared to
similar pupils nationally. Her actual
performance (5.4) was more than
half a level above her estimated
performance (4.3) with her actual
level highlighted in green.
Prior attainment Group: L (Lower),
M (Middle) and U (Upper) Groups
are based on an average of Test and
TA prior attainment results. Pupils in
the Lower group, for example, are in
the lowest third nationally based on
prior attainment. Pupils in the
Middle group are in the middle third
and those in the Upper group are in
the top third nationally based on
average test/TA prior attainment
results.
KS1 NCSS – National Curriculum Standardised Score
Based on an average of KS1 test results and month of birth (age
standardised). Other things being equal, the higher the KS1 average the
higher the standardised score. Where 2 pupils have the same KS1
average, the younger pupil (according to the month of birth) will have a
higher age standardised score.
The standardisation of results was carried out in 1998 using a a mean of
100. Since then the national mean has increased above 100 as test/TA
outcomes have improved. The national mean for the Year Group is
shown at the bottom of the report.
By scanning horizontally, you can
also see if this pattern is consistent
across subjects for individual pupils.
In Keira’s case, she looks to have
done well in both English and
Science. However, although it in
both cases her actual level is above
her estimated level (based on the
progress of similar pupils nationally),
the difference is less than half a level
and so neither is highlighted in
green.
It is important to remember that the
half a level or more difference is not
a significance test and is simply an
arbitrary level.
Example:
KS1-2 down’
Pupil Value
Step 3: ‘Drilling
- KS5 Added
SubjectSummary
Estimates
FFT Data Analysis Project
© Fischer Family Trust, 2009
Activity 3: Using the KS1-2 Pupil VA Summary Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
Keeping a Balance
A school might wish to emphasise CVA:
 As the closest approach to eliminating factors outside of
the school’s control
 The fairest way to evaluate school effectiveness
A pupil will be more concerned with ‘What did I attainment?’ than
with ‘Did I make good progress?’
Which will matter most to a potential employer?
Effectiveness
(CVA)
Progress (VA)
Raw Scores
AND
not
OR
FFT Data Analysis Project
© Fischer Family Trust, 2009
Improving target setting in schools: Using FFT estimates
PART 2
Estimates, Predictions and Target Setting
FFT Data Analysis Project
© Fischer Family Trust, 2009
Estimates, targets, predictions
 Deciphering the vocabulary:
Predictions
Estimates
Targets
FFT Data Analysis Project
© Fischer Family Trust, 2009
Estimates, targets, predictions
ESTIMATE + PROFESSIONAL KNOWLEDGE
FFT data will only ever give you an estimate.
It is not a replacement for targets or target setting!
PREDICTION
PREDICTION + CHALLENGE
TARGET!
FFT Data Analysis Project
© Fischer Family Trust, 2009
Why do FFT include these factors when producing pupil estimates?
 For pupils, we know that in general:
 KS2 attainment is highly dependent on attainment at KS1
 Girls make different progress than boys
 Autumn born pupils have higher attainment than Summer born
pupils
 Pupils’ prior-attainment in English often has a greater impact on
subsequent progress
 but we also know that School Context has an impact on
pupil progress:
 Pupils from disadvantaged backgrounds tend to make less
progress (geodemographic data)
 Prior attainment for the cohort can have an impact on future
achievement
FFT Data Analysis Project
© Fischer Family Trust, 2009
What does FFT include in their estimates for pupils and schools?
Pupil Factors
Test Levels (Fine levels)

Teacher Assessment Levels (where available)

Variation between subjects

Gender

Month of Birth

School Factors
Average Test level for whole cohort

Spread of Test Levels for whole cohort

FSM Entitlement (Percentile Rank)

Geodemographic Data (Percentile Rank)

FFT Data Analysis Project
© Fischer Family Trust, 2009
Estimate Models - Overview
There are 2 basic types of estimates:
 PA - Type A (estimate based on similar pupils nationally)
Prior
Attainment
Gender
Month of Birth
 SE - Types B (50th percentile) and D (25th percentile)
(estimates based on similar pupils AND school context)
Prior
Attainment
Gender
Month of Birth
School
Context
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
What more should we aim to achieve?
Which areas require intervention?
What must we do to make it happen?
How are we going to track pupils?
How do we measure success?
(4) Pupil Estimates Detail Report
(5) School Estimates Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
Attainment
If only life were this simple!
Estimate
Year 2
Year 6
FFT Data Analysis Project
© Fischer Family Trust, 2009
Attainment
It’s more like this:
Estimate
Year 2
Year 6
Guidance: KS2 Pupil Estimates (Subject) Report
Subject: Select individual
subjects (English, Reading,
Writing, Maths, Science) or ALL
to see all subjects on one report.
Highlight Top: Select options for
colour coding grades within the
top 5-25%
Est Basis: There are 2 Basic estimates – PA (based on Prior
Attainment, gender and month of birth) and SE (School
Extended) based on PA plus school context factors such as FSM,
deprivation and cohort ability. Select All to view both estimates
at the same time.
Estimate Rank: This option is
used in conjunction with the
Estimate Basis (PA/SE) and allows
users to select a rank from 50 to 5.
Coloured boxes (Linked to
‘Highlight Top option’
above):
Selecting PA (Prior attainment)
and 50, for example, will produce
estimates based on the progress
made by similar pupils (prior
attainment, gender and month of
birth) in schools at the 50th
percentile for value added. This is
how Type A estimates are
produced.
Orange highlight – shows
the Level with the highest
probability (orange).
Green highlight - shows the
Levels achieved by pupils
within the TOP
5%,10%,15%,20% or 25%
(user option)
Selecting an est basis of SE
(School Extended) with a rank of
50 will produce Type B estimates
and changing the rank to 25 will
produce Type D estimates
(progress of pupils in schools at
the 25th percentile for value
added). The higher the rank (e.g.
5), the more challenging the
estimates will be.
KS2 Estimates: The report shows estimates for each individual pupil and subject level from 2 to
5+. In this example, Toby
Tina Argon
Chromium
has ahas
69%
a 72%
chance
chance
of achieving
of achieving
a Level
a Level
4 and4 aand
25%
a 20%
chance of
achieving
chance
of aachieving
Level 5 in
a English.
Level 5 inThe
English.
probabilities
The probabilities
are based are
on the
based
progress
on theofprogress
similar pupils
of similar
nationally
pupils
nationally
in the previous
in the previous
year (i.e.
year
25%
(i.e.of72%
pupils
of pupils
similarsimilar
to Tinatoachieved
Toby achieved
a Levela5Level
last year).
4 last
Level 5Levels
year).
is highlighted
5+ is highlighted
in Green as
in Green
it is within
as itthe
is within
top 25%
the of
toplevels
25% achieved
of levels achieved
achieved achieved
by similar
pupils
by
similar
lastpupils
year Level
last year
4 is (this
highlighted
can beinchanged
orange using
indicating
the Highlight
that it is the
Top‘most
menulikely’
above).
or highest
Level 4 is
probability level
highlighted
in orange
(69%).
indicating that it is the ‘most likely’ or highest probability level (72%).
Showing estimates as probabilities rather than a single estimated level allows users to
see the whole range of levels that similar pupils have achieved in the past. The reports
can then be used to set both challenging and realistic targets. Probabilities
Probabilitiescan
canalso
alsobe
be
more empowering for students and teachers. A 20% probability of achieving Level 4+
reminds us that last year, 2 in 10 similar pupils did actually achieve this
thislevel!
level! What
support will pupils like this need to attain Level 4 or higher?
Level 4+ Summary: Calculated by adding the Level
4 and 5+ probabilities together. If the figures do not
match exactly this will be due to:
a) rounding, or
b) minimum and maximum probabilities are set at
1% and 99%. This is to avoid the situation whereby
pupils appear to have no chance of achieving a
particular grade or are 100% certainties to achieve a
particular Level.
Example: KS2 Pupil Estimates (Subject) Report
FFT Data Analysis Project
© Fischer Family Trust, 2009
Activity 4: Using the KS2 Pupil Estimates Report
Using Estimate reports with pupils (KS2 example)
Using reports appropriately and effectively
Probabilities are included for all levels from 2 to 5+.
The colour coding shows the most likely grades within the top 5,10,15, 20
or 25% (as selected by the user) in green and the grade with the highest
probability in orange.
When looking at potential progress for individual pupils (or a group of
pupils) the estimates and colour coding can be used to support the
process of target setting but should not take the place of target setting.
Take the example of Nina Caesium to the left. Think for a moment about
what the information is telling you. Whilst her highest probability level in
English is a 4 (73% and highlighted in orange), the estimates are telling
you that last year, 19% of similar pupils (nearly 1 in 5) attained level 5
(shaded green as these grades are within the top 25%). With support,
could Nina be one of those 5 or could she actually attain a level 5 quite
easily? It’s rarely as simple as just using the ‘highest probability’ or ‘most
likely’ grade!
The data shows that there’s a chance that Nina may be able to attain far
more but what other questions would you need to consider when setting a
target for Nina?
• What do you think Nina could achieve?
• What have similar pupils in your school achieved in the past?
• What are your aspirations? What are Nina’s aspirations?
• What other data is available to help you set an ambitious but
appropriate target for Nina?
• What additional work would Nina need to do to achieve a level 5?
• Which areas of the curriculum is Nina strong/weak in?
• What additional resources would be required to ensure that Nina (and
similar pupils) have a chance of achieving a level 5 in English?
Remember, use the reports ALONGSIDE other data, your own
professional judgement and aspirations AND THE ASPIRATIONS
AND MOTIVATIONS OF CHILDREN THEMSELVES!
FFT Data Analysis Project
© Fischer Family Trust, 2009
A framework for Self-Evaluation and target setting using FFT Live
What more should we aim to achieve?
Which areas require intervention?
What must we do to make it happen?
How are we going to track pupils?
How do we measure success?
(4) Pupil Estimates Detail Report
(5) School Estimates Report
Example: KS2 School Estimates
Key Stage: Select KS2,3 or 4
estimate reports where available
Group Size: Only show Pupil Groups with a
minimum of 1, 2 5 or 10 pupils. Default setting
is 5.
Pupil Group: Options to show pupil
groups based on FFT criteria (default) or
DSCF Pupil Groups used for LA SALTS
Target setting.
Estimates Overview
Estimates are now shown as a range from
lowest (on the left hand side of the report)
to highest (on the far right hand side of the
report). However, the order of the estimate
types themselves may vary for individual
indicators and pupil groups.
Estimates based on national progress
(Type A, B and D estimates)
Estimates based on the historical progress
of similar pupils (Type A estimates) in
similar schools (Type B and D estimates)
nationally last year.
Estimates based on your LA’s/school’s
own Value added (Green box)
Pupil Group: Groups are split
by gender, prior attainment,
FSM, SEN, ethnicity and LAC.
Girls/Boys Lower, Middle and
Upper Groups are based on an
average of Test and TA prior
attainment results. Pupils in
the Lower group, for example,
are in the lowest third nationally
based on prior attainment.
Pupils in the Middle group are
in the middle third and those in
the Upper group are in the top
third nationally based on
average test/TA prior
attainment results.
This estimate is a good starting point for
target setting discussions at school or LA
level. The estimate itself is based on how
similar pupils (based on prior attainment)
have performed in YOUR school on
average over the past 3 years. The
estimate is shown in a green box and its
place on the report is dependent upon
where the estimate sits in comparison to
other estimates.
High LA/school value added in the past
would be reflected by a higher estimate
which would appear at or towards the right
hand side of the report.
Group Size is based on ‘matched’ pupils only (i.e. pupil’s where prior attainment results can be matched with the
most recent results). When using this report be careful if there are large differences between the total number of
pupils in a group and the number of matched pupils used in the report
Low value added in the past would be
represented by a lower estimate which
would appear at or towards the left hand
side of the report.
In this example, the data suggests that the
progress of lower prior attaining boys and
girls has been lower than the progress of
similar pupils nationally (with green box
estimates of 32% and 30% appearing at
the left hand side of the report.
Example: KS2 School Estimates
FFT Data Analysis Project
© Fischer Family Trust, 2009
Activity 5: Using the KS2 Summary Estimate Report
FFT Data Analysis Project
FFT Live Administration Tools
FFT Data Analysis Project
Logging in to FFT Live 3.0
 How do I log-in to FFT Live 3.0 (www.fftlive.org)?
 Schools can now access FFT Live in a 4 ways:
1. SCHOOL ADMIN account supplied by the LA (e.g. 9992000Admin) or
created by a school itself (e.g. ASmit9999) – max 2 per school
2. SCHOOL USER account (e.g. ASmit9999). An FFT Live school admin
account holder can create as many user accounts as needed for staff in
schools (e.g. teachers).
3. GENERIC SCHOOL account (your school number – e.g. 9992000). Using
this account you can access school and group level data. The account is
supplied by your LA.
4. GENERIC PUPIL account (your school number plus ‘P’ – e.g. 9992000P).
Using this account you can access school, group AND PUPIL level data.
The account is supplied by your LEA.
FFT Data Analysis Project
FFT Live 3.0 Administration Utilities
Schools can now create and administer their own FFT Live
accounts using the school administration facility
What modules are available to School FFT Live Administrators?
All Administration tools are available from the Admin Tools menu under Manage
Users.
Manage Users
- Add User
- View/Modify All Accounts
- View Account Statistics
FFT Data Analysis Project
FFT Live 3.0 Administration Utilities
Adding a User
The Add User module allows School FFT Live Administrators to create new school
user accounts or an additional Administration account (maximum 1 additional
administration account per school). The process is carried out using a simple 2
step wizard. Once you’ve created a new account, FFT Live also gives you the
ability to modify it using the View/Modify All Accounts module.
FFT Data Analysis Project
FFT Live 3.0 Administration Utilities
Viewing and Modifying All Accounts
The View/Modify All Accounts module allows School FFT Live
Administrators to view details for all accounts on a single sheet.
Administrators can also change account settings, modify accounts and
export account details – useful when securely providing usernames and
passwords for new users.
FFT Data Analysis Project
FFT Live 3.0 Administration Utilities
Viewing Account Statistics
The View Account Statistics module allows School FFT Live
Administrators to view login statistics for all available accounts. This can
be useful for analysing use by school staff, to find users having problems
logging in (due to incorrect passwords) or to check security (where a user
is concerned that someone else may be using their account for example).
FFT Data Analysis Project
FFT Live 3.0
 Using FFT Live accounts in school – First 6 Steps
1. Your Headteacher should receive details of your FFT Live
School Admin Account (username and password) from your
LA.
2. The Headteacher should nominate 1 member of staff to be
an FFT Live Admin account holder and then provide them
with the username and password. No other members of
staff should have access to this account.
3. When the nominated Admin user first logs in to the account
they should add their personal details (name, job type, email
address) using the MODIFY USER feature in the
VIEW/MODIFY ALL ACCOUNTS module
FFT Data Analysis Project
FFT Live 3.0
 Using FFT Live accounts in school – First 6 Steps
4. A school can create one extra FFT Live Admin accounts
itself. We recommend that the Headteacher should also
nominate a 2nd member of staff to be an additional FFT Live
Admin Account holder (the existing FFT Live Admin account
holder can create one other Admin account in a school)
5. The Headteacher should then consider which members of
staff require access to FFT Live and ask the FFT Live Admin
account holder to create these accounts
6. All account details should be passed SECURELY to
individuals (e.g. face to face, through SECURE email system,
via 2 letters – one with username, the other with password)