Relative gain as evidence of student learning
Contents
Explaining relative gain
Importance of relative gain
Relative gain peer group
Interpreting Relative Gain displays
Shaded regions
Bars, markers, and lines
Supplementary Information
What can relative gain tell a teacher about their students?
What can relative gain tell a teacher about their curriculum?
Concluding comments
1
1
1
2
2
2
3
3
4
Graph 1
Graph 2
Table 1
Table 2
5
6
7
8
Explaining relative gain
One way teachers can collect evidence of student performance to inform planning and
pedagogy is through relative gain between NAPLAN tests.
Relative gain is a means of comparing how much a student has gained in terms of NAPLAN
performance from Test 1 to Test 2 relative to his or her peers with the same starting score.
In this way, we are comparing like with like.
Importance of relative gain
A common method of measuring student gain is by subtracting a student’s Test 2 score (e.g.,
Year 5, 2011 NAPLAN Reading scale score) from their Test 1 score (e.g., Year 3, 2009
NAPLAN Reading scale score). This method refers to raw gain or the difference between a
student’s Test 1 and Test 2 scores.
The problem with raw gain is that it is influenced by a student’s starting score. That is,
students with lower (Test 1) starting scores tend to gain more than students with higher (Test
2) starting scores. This bias in raw gain means that students with lower starting scores will
be over-represented as having greater gains than their peers with higher starting scores.
Relative gain attempts to overcome the bias in raw gain by grouping students into “peer
groups” based on their score at Test 1. This way, relative gain provides a more specific and
appropriate benchmark for assessing student gain.
Relative gain peer group
If a Year 5 student (Tom) achieved a NAPLAN Reading scale score of 370 in Year 3 2009
(Test 1), Tom would be grouped with all Year 5 Queensland state school students who
achieved the same starting score as Tom in Year 3, 2009. Tom’s relative gain peer group for
NAPLAN Reading would therefore comprise Year 5 Queensland state school students with:
1) NAPLAN Reading scale scores in both Year 3 2009 and Year 5 2011
2) a NAPLAN Reading scale score of 370 in Year 3, 2009 (Test 1).
1
Interpreting Relative Gain displays
Graphs 1 and 2 below provide examples of how student relative gain is displayed in
OneSchool for a hypothetical Year 5 class (5x) looking specifically at students present at the
time of test – i.e., students who sat both tests at the same school.
There are three components to look at when interpreting the Relative Gain displays:
1. Shaded regions
 The shaded regions of the graph represent a student’s relative gain – where a student
stands (in terms of gain from Test 1 to Test 2) compared to his or her relative gain
peer group (i.e., peers with the same starting score).
 There are three shaded regions in the graph:
- The light blue region at the bottom section of the graph captures Low Gainers –
students who gain less than their relative gain peer group.
- The grey region in the middle section of the graph captures Similar Gainers –
students whose gain is similar to their relative gain peer group.
- The dark blue region at the top section of the graph captures High Gainers –
students who gain more than their relative gain peer group.
 Student relative gain is determined by the shaded region of the graph within which a
student’s Test 2 scale score (diamond marker) falls. For instance, if a student’s Test 2
score falls in the dark blue region, then that student is a High Gainer or a student who
has gained more than his or her relative gain peer group.
2. Bars, markers, and lines
 Bars
- Each bar in the relative gain display represents a student’s raw gain.
- The length of the bar represents how much a student has gained from Test 1 to
Test 2 — the longer the bar, the greater the student has moved from Test 1 to Test
2.
- The colour of the bars represents the direction of a student’s gain. Green bars
represent improvements in performance from Test 1 to Test 2 while red bars
represent declines in performance (i.e., negative gain).
 Markers
- Markers situated at the top and bottom of the bars represent a student’s Test 1 and
Test 2 scale scores. Scale scores are plotted on the vertical y-axis of the graph.
- The square marker represents a student’s Test 1 scale score and the diamond
marker represents a student’s Test 2 scale score.
- Students (plotted on the horizontal x-axis of the graph) are ordered based on their
Test 1 score from lowest to highest. Student attributes (including gender,
2
indigenous status and language background) are also included alongside the
student’s name to provide context around a student’s performance.
 Lines
- The horizontal lines across the graph represent the national average scale score for
a particular NAPLAN strand and year level.
- The dotted line represents the National Test 1 average and the solid line represents
the National Test 2 average.
3. Supplementary Information
 Each relative gain display is accompanied by a table of supplementary information
about the graph (Tables 1 and 2).
 The supplementary table gives information about the class (‘Group’ column) and the
school. The first section of the table focuses on relative gain — the count and
proportion of Low, Similar and High Gainers are presented for the selected class
(group) and school.
 The second section of the table focuses on information depicted by the bars, markers
and lines (as described above).
 Contextual information elicited through the Test 1–Test 2 student matching process is
also provided. This information includes: 1) students who have a scale score at both
tests; 2) students who did not sit Test 1/Test 2 at the school; and 3) students who sat
both tests at the school.
 Supplementary information about student matching (e.g., the proportion of students
who sat both tests at the selected school) is important to keep in mind when
interpreting the relative gain displays, particularly when associating relative gain
outcomes with specific initiatives implemented in the selected school or class.
What can relative gain tell a teacher about their students?
1. Identifying exceptional students
- Relative gain displays can help a teacher identify which students in the class are
gaining in performance at a rate that is higher (or lower) than other state school
students who started at the same point.
- In Example State School, there are four students in class 5x who are High Gainers
in Reading (Students KRU, RIT, RAP and BAR, See Graph 1). These students
have gained at a greater rate in NAPLAN Reading from Year 3 2009 to Year 5 2011
compared to their relative gain peer group.
- In contrast, 2 of the 20 students in class 5x are Low Gainers in Reading (Students
WOO and MAC, See Graph 1). These students have exhibited lower gains than
students in their relative gain peer group and may require additional support in
reading.
2. Similar raw gain but different relative gain
3
- A student may experience similar raw gain to another student in the class but the
two students may have gained at different rates compared to their relative gain peer
group.
- For instance, in Graph 1, Student PET and BAR appear to have similar raw gain –
the difference in their performance from Test 1 to Test 2 appears to be similar.
- However, Student BAR’s relative gain places this student above his or her relative
gain peer group while Student PET has gained at a rate that is similar to his or her
relative gain peer group.
- Both students have gained at a similar magnitude but only one student has gained
higher than his or her peers with the same starting score.
3. Relative gain across strands
- Relative gain displays are also useful in helping teachers identify students whose
gain is not consistent across test or subject areas.
- For instance, in Graph 1, Student KRU has exhibited exceptionally higher gains in
Reading compared to his or her relative gain peer group. However, in Numeracy,
Student KRU’s gain places this student below his or her relative gain peer group.
- Therefore, Student KRU is excelling in Reading but seems to need additional
support in Numeracy.
What can relative gain tell a teacher about their curriculum?
- Just as relative gain is useful in helping teachers identify students whose gain is not
consistent across subject areas, it is also useful in helping teachers identify curriculum
areas that the class (as a whole) excels at or requires support in.
- As can be seen in Table 1, 19% of students in class 5x are High Gainers in Reading
while 9% are Low Gainers in Reading with only one student experiencing Negative
Gain.
- In contrast, as can be seen in Table 2, 0% of students in class 5x are High Gainers in
Numeracy, 47% are Low Gainers in Numeracy, and four of the 21 students have
experienced Negative Gain.
- This pattern of results suggests that class 5x is experiencing reasonable gains in
Reading but lower gains in Numeracy compared to their relative gain peer groups and
to the school as a whole.
- Students in class 5x may therefore require additional support in numeracy.
Concluding comments:
1. Relative gain is a means of comparing how much a student has gained from Test 1 to
Test 2 relative to his or her peers with the same starting score.
2. Relative gain is important because it gives teachers a more specific and appropriate
benchmark for measuring how much a student has gained from one time point to
another.
4
3. In interpreting relative gain displays, it is useful to understand and integrate three
pieces of information in the displays: 1) shaded regions of the graph; 2) bars, markers,
and lines; and 3) supplementary information presented in accompanying tables.
4. Relative gain can tell a teacher important information about their students or their
curriculum that they can subsequently use to enhance their teaching practice.
5. As always, NAPLAN results must be taken in context. These are a single point in time
test using a limited set of questions. Whilst there is greater validity associated with
interpreting results from large numbers of students one should be careful not to overinterpret the results of small schools or even individual students.
5
Graph 1: NAPLAN Student – Relative Gain – Students at time of Test – Reading
6
Table 1: NAPLAN Student – Relative Gain – Students at time of Test – Reading supplementary information
7
Graph 2: NAPLAN Student – Relative Gain – Students at time of Test – Numeracy
8
Table 2: NAPLAN Student – Relative Gain – Students at time of Test – Numeracy supplementary information
9