The link - e
advertisement
e-asTTle linking study Introduction The purpose of the linking study was to provide a link between the current and previous versions of the e-asTTle writing scale. It presents a way of converting e-asTTle writing scores measured on the previous version of e-asTTle to those measured on the current version. This document: ο· outlines how this conversion was determined ο· provides a brief description of the utility of the marking rubric for the current version of e-asTTle writing. The link This section describes the link between the previous and current e-asTTle writing scales. Converting previous e-asTTle scores to current e-asTTle ππΎπ scores In order to convert a previous e-asTTle scale score above 1250 to a current e-asTTle scale score, apply the following formula: πππ ≈ 0.95 ∗ π + 95 Where: ο πππ denotes the current e-asTTle writing score ο π denotes the previous e-asTTle writing score This is summarised in tabular form in Table 1. Scores will be less precise after they have been converted. An estimate of error has been calculated by combining the standard error associated with linking with the measurement error in the previous and current versions of e-asTTle. The standard error on a score converted from the previous version of e-asTTle is around 70 aWs points. A previous e-asTTle score of 1250 or lower should not be converted. Page 1 of 7 Converting current e-asTTle πππ¬ scores to previous e-asTTle scores In order to convert a current e-asTTle scale score above 1280 to a previous e-asTTle scale score, apply the following formula: π ≈ 1.05 ∗ πππ + 100 Where: ο πππ denotes the current e-asTTle writing score ο π denotes the previous e-asTTle writing score Scores will be less precise after they have been converted. An estimate of error has been calculated by combining the standard error associated with linking with the measurement error in the previous and current versions of e-asTTle. The standard error on a score converted from the current version of e-asTTle is around 73 previous e-asTTle score points. A current e-asTTle score of 1280 or lower should not be converted. Table 1 Table of scale conversions Previous e-asTTle scale score (π) Previous e-asTTle scale score (πππ ) 1250 1283±70 1275 1306±70 1300 1330±70 1325 1354±70 1350 1378±70 1375 1401±70 1400 1425±70 1425 1449±70 1450 1473±70 1475 1496±70 1500 1520±70 1525 1544±70 1550 1568±70 1575 1591±70 1600 1615±70 1625 1639±70 1650 1663±70 1675 1686±70 1700 1710±70 1725 1734±70 1750 1758±70 1775 1781±70 1800 1805±70 Page 2 of 7 Methodology The linking was determined by the data obtained from two different approaches to creating a link: equipercentile equating and linking determined by double marking previous e-asTTle scripts. These two approaches were implemented separately and are described separately below. The data they generated was pooled and then subjected to a linear regression analysis. This is also described separately. Equipercentile linking The data enabling seven equipercentile links between the previous and current versions of easTTle writing was created – one link for each of the Year 4, Year 5, Year 6, Year 7, Year 8, Year 9 and Year 10 student populations. This data was drawn from the normative information provided by the current and previous e-asTTle tools. Specifically, the points below which 0.1%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% and 99.9% of the student populations for a given year-level scored on each version of e-asTTle writing were generated. These are depicted in Figure 1. Figure 1 The equipercentile link data 2000 1900 1800 Current e-asTTle score 1700 1600 1500 1400 1300 1200 1100 1000 1000 Year 4 1200 Year 5 1400 Year 6 1600 1800 Previous e-asTTle score Year 7 Year 8 Year 9 2000 Year 10 Page 3 of 7 This graphic suggests a strong relationship – the data is obviously linear and sits very close to a line of best fit. This linear behaviour is not surprising given that both scales are transformations of logit-based writing measurement scales. For example, at Year 6 both scales are constructed so that the mean score is 1500 and the standard deviation is 100 score points. Linking based on double-marking Sixty-nine student scripts (responses to e-asTTle prompts) – already marked using the previous e-asTTle writing rubric – were re-marked using the current e-asTTle rubric. This was implemented independently by two of the developers of the current e-asTTle writing rubric. These two developers were asked to justify or re-mark any outliers. Three student scripts with previous e-asTTle scores below around 1250 were removed from the data. This was because the relationship between the previous and current e-asTTle scores for these scripts was markedly different from that for the remaining scripts. The reason for this difference may be due to the current e-asTTle rubric being able to distinguish responses at lower levels of writing proficiency than the previous e-asTTle rubric. The resulting data is depicted in Figure 2. For comparative purposes, the data including the removed data points is depicted in Figure 3. Figure 2 The student script data 2000 1900 1800 Current e-asTTle score 1700 1600 1500 1400 1300 1200 1100 1000 1000 1100 1200 1300 1400 1500 Previous e-asTTLe score Marker 1 1600 1700 1800 Marker 2 Page 4 of 7 Again, the linear relationship in this data seems fairly clear. Some of the data sits further from a line of best fit than that depicted in Figure 1 but this is to be expected given that variance between markers has been introduced. Figure 3 The student script data including the removed points 2000 1900 1800 Current e-asTTle score 1700 1600 1500 1400 1300 1200 1100 1000 1000 1100 1200 1300 1400 1500 1600 1700 1800 Previous e-asTTLe score Marker 1 Marker 2 It is clear that the data points in Figure 3 which have previous e-asTTle scores less than around 1250 are demonstrating a relationship different from the remaining data points. Linking using the pooled data The equipercentile links and the script data were pooled and a line of best fit was overlaid on the pooled data. The line of best fit from the pooled data was used to determine the link between the previous and current e-asTTle writing scales. This is depicted in Figure 4. Page 5 of 7 Figure 4 The pooled linking data 2000 1900 y = 0.9534x + 94.542 R² = 0.8246 1800 Current e-asTTle score 1700 1600 1500 1400 1300 1200 1100 1000 1000 1100 1200 1300 1400 1500 1600 1700 Previous e-asTTle score Markers and norms 1800 1900 2000 Linear (Markers and norms) The pooled data agrees closely with its linear model as is indicated by the high R-squared value. An estimate of the standard error associated with linking – around 50 πππ – was estimated by using the standard error of the intercept parameter in the linear regression performed to find the link formula. Utility of the marking rubric The following observations were made by the two developers of the current e-asTTle writing rubric who marked the student scripts used in this linking study. They focus on the utility of the current e-asTTle marking rubric. Page 6 of 7 ο The current marking rubric has been informed by close observation, marking and moderation of student writing from Year 1 to Year 10. It therefore reflects a broader range of writing proficiency than the previous e-asTTle writing rubrics. In particular, it can recognise those students who are just emerging as independent writers, allowing the achievement of all independent writers to be assessed. ο The current rubric is based on the principle of recognising student achievement in seven elements of writing. It contains detailed, precise descriptions of skill development in each element. Because the writing used to develop them was produced by students from Years 1 to 10, the descriptions capture the early stages of development in each element. The rubric can therefore provide meaningful diagnostic information for any student who is able to communicate at least one or two simple ideas independently. Information at this level was not available in the previous version of e-asTTle writing. ο Because the current rubric involves assigning ‘Rubric Scores’ in each element rather than curriculum levels, there is the possibility of greater accuracy in scoring decisions. Rather than first placing a student’s writing within a broad curriculum level then considering finer distinctions within this (ie, the BPA system of the previous version of e-asTTle), the current rubric allows for a consideration of observable skill levels within each element. This consideration is independent of the curriculum. It is the student’s overall performance that is compared with the curriculum. ο The current e-asTTle marking rubric is quite robust. That is, it can be applied effectively to writing produced in response to previous e-asTTle writing prompts, as long as the purpose for the writing is to describe, narrate, recount, explain or persuade. Similarly, it can be applied effectively to any writing prompts comparable to the current or previous e-asTTle writing prompts. This is because the elements of the current rubric focus on observable indicators of writing proficiency. The scores from prompts that are not part of e-asTTle should not be entered into the tool itself. ο Although some of the elements of the current e-asTTle marking rubric do not translate directly into the language of the previous marking rubric, the link described in this document is statistically sound. This indicates that the overall writing scores of the previous and current versions of e-asTTle are readily able to be translated and underscores the robustness of the current e-asTTle marking rubric. Page 7 of 7
