Using FFT Live Primary Schools Mike Treadaway Director of Research Fischer Family Trust Contents Data Landscape • Change and Continuity? FFT Live • Self-Evaluation • Estimates - Particularly KS1 • Data Literacy – Applying Key Principles Other Reports • Student Explorer • Exports Data Landscape FFT DfE 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Value-Added Contextual Value-Added 3 Year Summaries Estimates Also: • Floor Standards • Expected Progress Whether or not to Contextualise? Type of Analysis Advantages Disadvantages Raw Outcomes e.g. %L4+ Provides a view as to whether pupils are attaining the knowledge and skills needed for their next stage of education? When used to compare schools, depends strongly upon attainment of the school’s intake. Value-Added Enables a focus upon progress irrespective of context e.g. How well are our FSM pupils doing compared to our nonFSM pupils? Comparing value-added outcomes for schools with different contexts and makeup of intakes can be misleading. Contextual Value-Added Aims to take account of factors which are outside the school’s control and which, from analysing national data, can be shown to influence progress. If used to look ahead (estimates) then danger of ‘self-fulfilling prophecy’ for pupils in groups who currently under-perform nationally. Differences – VA and CVA • Compare PA (Value-added) and SX (Contextual Value-Added) outcomes for KS2->4 in 2009/10. – Use KS2 Average Points Score as outcome measure. – Look at how many change significant state. – Analyses by grouping schools in different socio-economic contexts. School VA compared to CVA School Change in VA Score Context Better Worse Context Avg Min Max Q1 14% 4% Q1 +0.30 -1.6 +1.6 Q2 9% 8% Q2 +0.01 -1.6 +3.0 Q3 7% 11% Q3 -0.08 -1.6 +1.9 Q4 11% 9% Q4 +0.05 -1.8 +2.2 To get some idea of the likely change for your school, compare PA and SX value-added outcomes on FFT Live or in FFT Self-Evaluation reports. In Lambeth, most schools have VA better than CVA – this is mainly related to the progress of ethnic minority pupils. Recommendations For Self-Evaluation: • VA (FFT PA model) and CVA (FFT SX model) are both important • Contextual Value-Added is probably still the best starting point • For schools with high % of ethnic minority pupils (other than Black Caribbean) CVA is likely to be LOWER than VA • If CVA is good, particularly in a school serving a socio-economically disadvantaged area, VA provide a view as to whether pupils are making good progress irrespective of context For Estimates: • Thinking about range is probably the most important principle • PA model probably the best starting point, but need to take account of: • What the school has achieved in previous years • Impact of pupil characteristics and school context • Other factors, including parental support, school interventions ..... And, in the end, what matters most to each pupil is the knowledge and skills they take with them to the next stage of their education! Expected Progress • DfE Approach – Whole Levels, Subject Specific Input – So, for example, 84% L2 in Mathematics at KS1 attained L4+ at KS2 in 2011 • FFT Approach – Differentiate within levels where possible (2ABC) – Take into account prior attainment in English and Mathematics KS1 English L4+ KS2 Maths KS1 Maths KS2 L4+ Maths 1 80% 2C 58% 2C 82% 2B 86% 2B 86% 2A 97% 2A 89% 3 92% FFT Live Includes: • Latest updates • Future releases • Help Files SETS • Key Changes for 2011 – Default changed to PA (Prior Attainment) – SX (Contextual VA) continues to be available – Option to SORT available on Pupil Estimates report Self-Evaluation Significant Areas Grid • Range of pupil groups and indicators • KS1 Contextual and KS1->2 VA / CVA • 3 Year summary, Overall + Trends Significant Areas Detail • Shows actual attainment and VA/CVA scores for each year • Can be accessed by clicking on Significant Areas grid Pupil Value-Added • KS1->KS2, VA or CVA • Highlights where progress is mode than half a NC level above or below national average Estimates School Summary Estimates • Range of Indicators and Pupil Groups • A – PA 50th Percentile; B – SE 50th Percentile, • D – SE 25th Percentile; S – School Previous Pupil Summary Estimates • Types A, B, D • Range of indicators (overall level, L4+, L5+) • Reading and Writing not included School Subject Estimates • Core Subjects including Reading and Writing and Expected Progress for English and Mathematics • Outcomes for all levels (not just L4+ and L5+) • Some pupil groupings available Pupil Subject Estimates • Can be accessed directly or from School Subject Estimates • Show likelihood of attaining all levels (L2 to L5) • Also provides option to show ‘most likely level’ Using Estimates Two basic approaches • Start with pupil data • Take into account other factors • Add up and compare with overall estimates • Revise as necessary to arrive at overall targets • Use overall estimates and evaluation of past performance to draft overall targets • Use ‘RANK’ option in FFT School Subject Estimates to set overall figure at your overall targets • Click on link to pupil estimates • Estimate -> Prediction -> Target • Consider option of range - realistic and ambitious • Don’t add up ‘estimated levels’ Adding up estimated levels – why not? • Consider a group of 20 ‘identical’ pupils • They all have the same estimates: L2 L3 L4 L5+ GM HGM GA 5% 20% 50% 25% 4 25% 5 If we add up the ‘GM’ grades If we average the ‘chances’ for each level and multiply by number of pupils L2 L3 L4 L5 0 0 20 0 L2 L3 L4 L5 5% 20% 50% 25% 1 4 10 5 Note : Adding up predicted or target grades is fine – assuming that they are not simply just using unmoderated estimates! KS1 Estimates • In development section • Introduced in summer 2011 following a year of trialling and feedback – which was very positive! • Examples of relationship between FSP and KS1 attainment Remember that these are averages! CLL -> Reading Chart below shows KS1 Maths Levels for FSP MAT scores between 18 and 22 EYFSP MAT score between 18 and 22 MAT -> Maths 40% 35% 30% 25% 20% 15% 10% 5% 0% 1 2C 2B KS1 Mathematics Level 2A 3 FSP to KS1 - Factors Factor Overall EYFSP Score CLL Score MAT Score PSE Score Gender Age Socio-Economic Context Approximate Weightings CLL MAT PSE Gender Age PA Model SE Model KS1 Reading 58% 26% 8% 5% 2% KS1 Writing 58% 24% 5% 11% 1% KS1 Mathematics 44% 43% 3% 9% 1% Other Reports • Export Section • Provides additional data (e.g. UPN, FFT_ID) which is not available on SETS reports • Removes formatting and colours, so easier to import into other systems • • • • Student Explorer Select students by range of characteristics Look at individual pupil history Developments: • FSM as pre-set group (can currently select using drop-down menus) • Pilot estimate report which takes into account factors such as EAL and when arrived in UK Questions We have introduced KS1 estimates • Should we also include KS1 Value-Added reports • School-Level? • Pupil-Level? Student Explorer • Ability to select by FSM (current or ever) will soon be included • Are there any other important pupil characteristics which would be useful to have easily available on the main menu? Any other suggestions for development?