A
A
W
P
1:
FFT
T
Summary This working paper is one of set of working papers which explore issues related to assessment in art. This seeks to describe in quite simple terms the basis of FFT reports so that teachers can better understand how to use this information. Dan China
Appendix 1 is an anonymised copy of an actual FFT report. Appendix 2 is a copy of guidance notes published by FFT.
There is a widely held mythology about Fischer Family Trust (FFT) data. At the heart of the misconception is the idea that FFT provides targets or predictions for art based upon prior performance in English and
Maths. However, it is not true. FFT is not some sort of obscure psycho-metric analysis, which uses performance in English and Maths to mystically convert qualities of literacy and numeracy into an assessment of a student’s aptitude and ability in art (or any other foundation subject for that matter).
However, FFT does present relevant statistical data which is useful to schools and teachers. Like most statistics it proposes questions not answers.
FFT Reports :
In essence FFT uses the SAT scores of a student taking art in order to identify all those students, with identical SAT score, who had taken art in the previous year. The actual performance, in art, last year, of this national sample group, is then used to estimate the statistical likelihood of this student achieving a particular grade. In essence FFT says: ‘Last year, of all the students who took art, with this same KS2 SAT score, 60% achieved a 'C'. Therefore, if this student has the same SAT score then, statistically speaking, there
is a 60% chance that they too will get a 'C' – discuss’. It is not a prediction, just a statistical best fit. And, of course, it could be that this particular student is actually more like some of the 40% who did not get a 'C': hence the need to discuss.
The FFT report for each student does not just show just the one grade that is the most probable
(statistically speaking) but shows the estimated probability of achieving each grade. So, let us assume that last year for a particular group with the same SAT score:
10% actually achieved a ‘B’;
40% actually achieved a ‘C’;
30% actually achieved a ‘D’;
10% actually achieved an ‘E’;
10% were ungraded.
This information is used to estimate the probability that a student this year with exactly the same starting point will achieve particular grades. In the case above, the FFT report would indicate that the student with
1

this starting point has a 10% estimated probability of getting a ‘B’; a 40% estimated probability of getting a
‘C’; a 30% estimated probability of getting a ‘D’ and so on.
The FFT report does indicate the grade which has the highest probability of being achieved (see the example of an FFT report below). This is the one achieved by the highest percentage of the students in the sample group: which in this example is a 'C' (achieved by 40% of the group). The response from the school and the teacher should be to consider whether the student is, in fact, most like one of the 40% who achieved a 'C', or whether the student is more like one of the other 30% who achieved a ‘D’ or one of the
10% who achieved a ‘B’. Thus, in this example professional judgement together with FFT data could lead to a perfectly proper target which is could be a ‘B’, ‘C’ or ‘D’ grade.
So in FFT reports there is no single 'prediction' or 'target': the report simply provides the school with statistical information, which suggests that, when taking account of the thousands of students in the previous year, with the same starting point, this student has a 40% estimated probability of getting a ‘C’ and that this is the most likely, but not the only, probability.
Confusion often arises because information from the FFT report is frequently condensed and only the one
'highest probability' grade is actually given to teachers: usually as a non-negotiable ‘FFT target' or
'prediction'. This misrepresents the report. This is unhelpful and can bring into disrepute the process of target setting because it will obviously be the case that some students will not match the most likely probability. For some students and their teachers such an imposed FFT ‘target’ will simply be wrong and will fail to inform improvement. This can discredit the whole process of target setting in the eyes of students and teachers.
In this paper it has been necessary to simplify some aspects of the FFT analysis in order to illustrate the underlying principle of using past performance to produce statistical ‘estimates’ of what might happen if pupils make progress that is in line with that of similar pupils in previous years.
Predictions or Targets or Estimates:
One source of the confusion may be in the widespread use of the word 'prediction' which implies a causal linkage, rather than a statistical likelihood ie measuring rainfall can predict flooding . FFT reports are much closer to the statement ... If a school's teaching statistics for last term suggests that 80% of teaching was satisfactory or better, an Ofsted inspector this term is more likely to see a satisfactory lesson than an
unsatisfactory lesson. It may be likely but it is not a certainty. FFT always uses the word 'estimate' and never uses words 'prediction' or 'target'.
In fact it is only the student’s teachers who can produce ‘predictions’. They know the student, their past performance and the demands of the course. Teacher predictions must be based on informed and accurate assessment. Assessments will need to identify performance across the range of assessment objectives and use different types of evidence. Assessments should be accurate and moderated if they are to be helpful in identifying ‘predictions’ of future performance and negotiating challenging ‘targets’. The point is FFT provides statistical ‘estimates’, teachers provide informed ‘predictions’ and schools then set challenging
‘targets’.
The proper understanding of FFT is important to ensure targets are appropriate and have integrity. In some schools target setting tends to reflect 'scenario one' below. This is can be an alienating process which can cause teachers to dismiss the process as deeply flawed and irrelevant. However, in 'scenario two' teachers can recognise the FFT data for what it is and will also have their own professional judgement recognised.
This is more likely to lead to challenging but sensible targets being set and achieved.
2

Scenario one: The head of department receives an email with the targets pre-specified for their students.
Some reference is made to these as being ‘FFT targets’ and that they are non-negotiable.
Scenario two: The head of department has a meeting with a senior leader who shares the full FFT report for the subject and explains the basis of the 'estimated probability' of each student achieving the various grades. The head of department is invited to review and discuss these in the light of the department's
(moderated) end of KS3 level assessments and predictions. An agreement is reached as to the most appropriately challenging target for each student based upon FFT probabilities, internal level assessment records, and the professional judgement (predictions) of the teachers.
In scenario one the targets are shrouded in mystery. FFT is invoked, inappropriately, as the source of the target. Where students do not match the most likely profile, targets may be inappropriate and consequently dismissed and disparaged by the teachers. Teachers have no ownership of these targets and are not fully committed to achieving them. In scenario two the head of department is fully aware of the nature of the statistical data and can use it appropriately to negotiate and agree a reasoned professional judgement as to the appropriate targets. Senior leaders are able to discuss, understand and negotiate what actions need to be taken by the department to ensure students reach agreed targets. As a consequence they are better able to monitor progress. Teachers have ownership of the targets set for themselves and their students and are fully committed to achieving them.
Recommendations:
1.
Teachers should ask to see the full FFT probability estimates for their students when involved in establishing targets.
2.
Subject leaders should always refer to their own end of KS3 level assessments as well as FFT reports when establishing and considering their predictions.
3.
Subject leaders should recognise that their professional predictions and subsequent target getting is predicated upon good assessment which takes account of the profile performance across the range of assessment objectives/criteria.
4.
Teachers should remember that FFT reports are just statistics based on the performance of previous cohorts of similar children. They are not psychometric tests which identify the aptitudes in art of any individual students.
5.
Teachers should remember that FFT does not give either targets or predictions - only schools do this.
6.
Senior Leadership Teams should support teachers in understanding the nature and relevance of FFT estimates and reports to ensure the credibility of targets and target setting.
3

Appendix 1
In the example of an FFT report below:
 Candidate C has only three percentage points between the estimated probability of achieving three different grades. The FFT report simply does not differentiate adequately between these three grades. Accurate moderated KS3 level assessment will help throw light on the appropriate target.
 Several candidates with a 40+% estimated probability of achieving a 'C', nevertheless, also have a one in five (20%) estimated probability of achieving a B. If their target is simply established, according to the highest probability score, as a 'C' they may never be challenged to achieve a 'B'.
Accurate moderated KS3 level assessment will help throw light on the appropriate target.
 Candidate E has a 38% estimated probability of achieving a 'C', however, they also have a 14% estimated probability of achieving an 'A'. If their target is simply established, according to the highest probability score, as a 'C' they may never be challenged to achieve an 'A'. Accurate moderated KS3 level assessment will help throw light on the appropriate target.
4

Appendix 2
Guidance notes published by the Fischer Family Trust 2007
MAKING BEST USE OF FFT ESTIMATES
The FFT Data Analysis project produces ESTIMATES of likely attainment. The estimates are calculated for each pupil From these statistics school and LA estimates are calculated. They are called estimates – not predictions or targets – because they provide an estimate of what might happen if your pupils make progress that is line with that of similar pupils in previous years. Be aware that the accuracy of FFT estimates varies. They are, for the most part, based upon prior attainment in core subjects. Teachers should never use FFT estimates to lower targets set by the school. If they ‘disagree’ with the estimate, and think the pupil can do better, then they should be confident to set a higher target.
Using FFT estimates
As the basis for a discussion about expectations.
FFT estimates should be used alongside other information, particularly recent teacher assessment and recent performance. They can be used to enable realistic and challenging targets to be set. To inform planning, teaching and to engage pupils. Estimates can provide an indication of the ‘most likely’ level or grade. They are only ‘indicators’ and should not be seen as rigid predictions or targets. They can be used, alongside other information, in discussions in pupil progress meetings and in subject or school review meetings.
To inform discussion about targets.
FFT provide a range of estimates to enable comparisons to be made. Types A and B are based upon the progress made by all pupils (nationally) in the previous year. Type A (PA) is based upon pupils’ prior attainment, type B (SE) takes into account prior attainment and the socio-economic context of the school.
They provide an estimate of what might happen if pupils make the same progress as happened, on average, last year.
Two further estimates provide an indication of what might happen if we project that pupils will make better progress than in previous years. Type C is based upon what you would need to do to play your part in achieving either national or LA targets. Type D (TQ) is based upon the progress made by schools in the top
25% percentile of value-added scores nationally.
To be aspirational for pupils in the light of current circumstances.
If an estimate indicates a 10% chance of achieving a particular level, think “is this pupil going to make much better progress than ‘expected’?. Is it possible that they may be the 1 in10 to achieve the level?” Equally, if the chances are 90%, think “might they need some support to ensure that they are not one of the 10% who don’t reach the level”.
To use past to inform planning for the future
Ensure your approach is the one which you feel is appropriately challenging whichever estimates you use.
Think about past performance and aim high for the future.
Schools where valued-added has been very high are likely to find that the ‘type D’ estimates are too low. If a school has previously been in the top 10 percent nationally then aiming to be 25th in the percentile ranking is (in value-added terms) aiming for lower pupil progress than has been achieved before.
For other schools, aiming to achieve what the ‘best’ schools achieve is a worthwhile aim. Some schools where value-added has previously been very low – perhaps those in the lowest 25% nationally – have felt
5

able to set targets based around ‘type D’ estimates. Other schools in this situation feel that a two stage process - aiming for ‘type B’ in the short term and type D in the medium term – is more appropriate.
Using FFT estimates with other information
Review pupil progress against estimates by regular assessment.
Engage the pupils and staff in discussions about curriculum targets, planning and teaching. Ensure interventions and resources are targeted appropriately, particularly for those that are not making
‘expected‘ progress. Monitor and evaluate the impact of intervention programmes.
Take into account individual pupils’ expertise and interest in the subject.
This can be especially important for pupils with relatively low prior-attainment – they will have relatively low estimates but may well have a particular aptitude or interest in your subject. Also, pupils with EAL may have ‘apparently’ low prior-attainment but they make fast progress as their English develops.
Remember that some variation is the norm.
Most pupils make broadly average progress but some achieve much higher or lower than the estimate would indicate.
A pupil might have an estimated KS level of 3.9 – which means that, on average, pupils with similar priorattainment have, in the past, achieved just below level 4. They might also have, on the FFT report, a 10% probability of achieving level 5. This means that, in the past, 1 out of every 10 pupils like this did attain level
5.
Having an appropriate view of the accuracy of estimates
Overall (school-level) estimates are the average of the individual pupil estimates and, because of this, have a high level of accuracy. For a range of reasons, including those earlier, some pupils to will exceed their estimates and some may not reach them. If, a school wants to aim to be better than ‘average’, and most do, it must ensure that as many pupils as possible achieve ‘added value’ results.
Fischer Family Trust
May 2007
6