DEPARTMENT OF EDUCATION 2014 NAPLAN Reporting and Analysing Achievement Data User Guide National Assessment Program Literacy and Numeracy 1 Assessment & Reporting 2014 CONTENTS Introduction 3 Interrogating the data 4 RAAD data sets 5 Getting started 6 Front Page 7 Students 8 Assessment Data 9 Student Distribution 12 Performance Profile 13 Individual Profile 14 Comparison Graphs 15 Filter Students 17 Printing 19 Exporting data to Excel 21 Copy data to clipboard Word 23 Appendices A-D 25-28 CONTACTS: Your regional office or Assessment and Reporting Gabrielle West Consultant National Assessment Numeracy p: 8944 9246 e: gabrielle.west@nt.gov.au Leanne Linton Senior Advisor Assessment and Reporting ESL p:89449248 e: Leanne.linton@nt.gov.au Salima Spring Consultant National Assessment Literacy p:89449245 Felicity Green Consultant National Assessment & Reporting p: 8944 9209 e: felicity.green@nt.gov.au Sarah Belsham Senior Program Manager National Assessment p: 8944 9239 e: sarah.belsham@nt.gov.au 2 Assessment & Reporting 2014 e: salima.spring@nt.gov.au Introduction An assessment culture that involves the transformation of quality assessment data into knowledge and ultimately effective teaching is critical to advancing students’ literacy and numeracy achievement. This user guide outlines the features and report facilities available in the Reporting and Analysing Achievement Data (RAAD) programme. This tool enables educators to aggregate, evaluate and disaggregate student performance results for each of the NAPLAN tests and provides schools with useful diagnostic data. It is designed to support school leaders and classroom teachers to interrogate student performance data for the purpose of school improvement. Suggested reflection questions useful for developing a plan to improve student achievement, have been provided. The booklet works hand-in-hand with the Data Analysis Guide: using assessment data to inform planning, question analysis documents and copies of the NAPLAN test papers. They are part of a larger suite of documents aimed at supporting teachers to make the most effective use of the NAPLAN achievement data. These can be found on the NT Department of Education website: http://www.education.nt.gov.au/teachers-educators/assessment-reporting/nap Data Analysis Guide This guide provides an easy-to-use framework for analysing the data to inform instructional decision making. Reading, Writing, Language Conventions and Numeracy Analysis Booklets and PowerPoints help teachers to analyse the NAPLAN questions and results. Support documents 2014 These booklets support teachers to develop good quality teaching and assessment strategies for everyday classroom use across all year levels. They also contain specific activities that will prepare students for the NAPLAN tests. NAPLAN Past Tests PDF copies 2008-2014 tests are available on Learning Links as well as analysis documents, writing marking guides and answer spread sheets. These tests can be used in all early, primary and middle school classes in a variety of ways. http://ed.ntschools.net/ll/assess/naplan/Pages/default.aspx 3 Assessment & Reporting 2014 Interrogating the data NAPLAN data provides evidence, rather than hunches or perceptions, on which to base future teaching, learning and assessment decisions. Using an Assessment of, as and for Learning approach will achieve optimum use of the NAPLAN data and make a significant contribution to improving student outcomes. The data analysis process incorporates a review of test questions to understand what concept/s are being assessed, focus on the ‘language’ used in the test questions and analyse error responses. Teachers can select specific areas where students, individually, in small groups or as a class group, should be actively involved in the analysis process. Throughout this guide, reflective questions will be included at the beginning of each new RAAD data set screen. Assessment Purpose Key Questions RAAD Data Sets Assessment of learning assists school leaders and teachers to use evidence of learning to assess student achievement against goals and standards. It provides summative information on a student’s achievement at a point in time for reporting purposes. Does the school mean in each of the test domains (for the whole test and by question) differ significantly from the national mean? How does the school distribution of student achievement compare to the national range (highest 20%, middle 60% & lowest 20%)? Which students are below, at and above the National Minimum Standard? What patterns of distribution can be observed e.g. even spread or clustering? Are there performance patterns for subgroups of students e.g. gender distribution, ESL students, and class groups? Can patterns to student responses be observed? At a year or school level? What data could provide information on the impact of specific programs? Does this data validate teachers’ assessment of the students? Assessment Data & Student Distribution Individual Profile, Performance Profile, Assessment Data Individual Profile Assessment Data, Comparison Graphs & test papers Assessment Data, Comparison Graphs, past test papers Assessment for learning enables teachers to use information about student progress to inform their teaching. Assessment for learning focuses on the learning needs of students. Assessment as learning enables students to reflect on and monitor their own progress to inform their future learning goals. Assessment as learning actively involves students in their own learning. 4 Which concepts and skills emerge as strengths or weaknesses for an individual student? What learning needs to be targeted? What patterns of response have implications for classroom learning programs? What information about student misconceptions can we gather, when looking at incorrect responses? Student Distribution Student Distribution Student Distribution Student Distribution Assessment Data & Comparison Graphs Student Distribution Where is the student placed in relation to other students in that year level? Performance Profile or Student Distribution How can the student be involved in reflecting on their performance? What are the personal learning goals of each student? Individual Profile & test papers Individual Profile Assessment & Reporting 2014 RAAD data sets The following types of information or data sets can be found in the RAAD. These can be analysed for the year level, class, sub-group or individual student. Student Details Student name, date of birth, gender, Indigeneity and LBOTE (Language Background Other Than English), class or form group Students’ participation status for each test – score, absent, exempt, withdrawn Program codes which can be added to group students for data filtering and sorting Assessment Data Question descriptors, writing criteria descriptors or spelling words Questions organised by category e.g. ‘Space’ or ‘Opinion’ National, school and expected % correct for each question Students’ details, raw and scaled NAPLAN scores and National NAPLAN Bands Students’ incorrect multiple choice responses for each question Student Distribution Australian and NT mean (average) and school or subgroup means Location of students (their initials) on the National Achievement Band scale Students’ NAPLAN scaled scores Students’ performance against the NAPLAN Bands and comparison to other students Location of student/s against the achievement of all Australian students in that year group – top 20%, middle 60%, lowest 20% Performance Profile Alignment of descriptors for each question against the National NAPLAN Bands (easiest to most difficult) Pattern or list of correct and incorrect question responses and descriptors Individual Profile Individual student’s performance, their NAPLAN score location against the National NAPLAN Bands and questions, and the National Minimum Standard Pattern or list of correct and incorrect question responses and descriptors Comparison Graphs 5 Coloured stacked graphs visually showing the group’s responses (correct and incorrect) for each item The questions are ordered by the Australian percentage correct – easiest to most difficult. Assessment & Reporting 2014 Getting started The RAAD application, the student data file and RAAD User Guide are placed on every government school server in Term 3, immediately following the release of the National NAPLAN Summary Report. Nongovernment school files will be sent out separately. The RAAD application and user guide can also be found on the NT DoE website: http://www.education.nt.gov.au/teachers-educators/assessment-reporting/nap On the new school server: Go to Drive U: then to Teacher Public On an old school server: Go to Drive N: Locate and open the 2014_RAAD_V2 file It contains RAAD application Student data file RAAD User Guide Double click the blue and red RAAD.exe application Teachers can copy and save this folder to their laptops for easy analysis and use of the data. Click the Run button The RAAD.exe application will ‘pick up’ all the student data and present it in a useable form. School name (2014) RAAD Non-government schools can set up a similar folder. The student data file cannot be opened directly, if you double click it, it will not open. Note: It is advisable for schools to set up a dedicated NAPLAN_RAAD folder and create subfolders for each year 2008-2014. This allows a review of past results by teachers. Analysis documents created for past tests should also be saved in these folders. 6 Assessment & Reporting 2014 Front Page Begin looking at the student information and NAPLAN data Select a Year Group and Assessment Area on the front page. 1. File has a drop-down menu to save, print, export and copy. 6. Tick boxes to hide or Show Student Initials and include or Exclude exempt students. 2. Year Group shows the year levels in your school. This must be selected to view different tests. 7. Student: shows All or individual student names. 3. Assessment Area selects the test type. This must be selected to view display buttons. 4. Display buttons present a variety of reports: Students Assessment Data 8. Front Page button takes you back to this home screen from any page or report. Student Distribution 9. Help and Information lists and describes the different displays and reports. Performance Profile Individual Profile Comparison Graphs 5. Filter Students by subgroup to view reports: Reset Filters defaults all filters back to All. 7 Order by: selects the way to view data Score Ascending Score Descending Surname (A-Z) Surname Descending (Z-A). 10. Manual and Information: Links to the NT DET web page to access RAAD application and User Guide. 11. Directions for getting started on this page and viewing reports. Assessment & Reporting 2014 Students Questions What are the student background details e.g. gender, date of birth? Who participated in the tests? Which students have missing data? What subgroups will I set up? What comments could I include? Process Select Year Group e.g. Year 3 Select Assessment Area e.g. Reading Display buttons are now activated Students Assessment Data Student Distribution Performance Profile Individual Profile Comparative Graphs Select Students button The Students screen below shows student demographic details and NAPLAN scaled score for the Year 3 Reading test First Name Last Name Date of Birth Gender Aboriginal (Indigenous status) LBOTE (Language background other than English) Class or form group Score (NAPLAN scaled score) Program Code* Program fields 1 to 3* Comments* (* Schools can add codes and annotations against student names) Student names Student details can be adjusted or changed. Progam Codes and Comments can be added, see Setting up new Student Subgroups on page 21. 8 Assessment & Reporting 2014 Assessment Data Questions What patterns of performance can be identified by - comparing the year level % correct to the Australian % correct? - analysing students’ incorrect responses (distractors – a, b, c or d) to test questions? Note: Have copies of the test papers on hand for teachers to refer to individual questions. Process Select Year Group e.g. Year 5, and the Assessment Area e.g. Numeracy Select Assessment Data The Assessment Data screen shows the correct and incorrect responses for each student. 1. Student names, DOB, programs, class groups etc. 2. The following information is provided for each question: Correct Answer – The correct multiple choice response i.e. a, b, c or d Maximum Score – Achievable for each question All Australian Schools % – The percentage of students across Australia who correctly answered the question (approximately 250 000 students per year level) Your School % – The percentage of students in the school or subgroup who correctly answered the question Expected % – The probable percentage of students in this school group that would be expected to answer the question correctly. Note: This is a statistical value that is calculated using a probability model that takes into account your students’ responses to each question and the degree of difficulty of each question in this test. 3. Total Score is the raw score for this student e.g. 38 (out of a possible 40). The Score (NAPLAN) is a vertically scaled score calculated across Years 3-9 on a scale of 0 to 1000, see Appendix C. 4. The NATIONAL NAPLAN Band is the National Achievement Band for this student (Band 1 to Band 10 depending on the year level). Minimum Standard column describes the student’s achievement below, at or above the National Minimum Standard for this year level. 5. Questions are grouped in a specific way for each test: Reading: Questions are grouped by text type – Information, Procedural, Narrative, etc. Numeracy: Questions are ordered by mathematical strand – Space, Number, etc. Grammar & Punctuation: Questions are grouped into Punctuation and then Grammar. 9 Assessment & Reporting 2014 Assessment Data (continued) 5 Detailed information about each question and students’ responses 1. Hide will expand the data screen to the left and cover most of the menu bar. Show will appear if you have selected Hide and will reverse this to show the full screen again. 2. Question numbers sorted by category, strand or test e.g. NC03 = Non-Calculator question 3. 3. Correct answers to each question either a, b, c, d or the multiple choice pair of answers e.g. c, d. 4. Horizontal rows of the students’ responses to each question. Student responses for each question are recorded in the row alongside the student’s name: 1 indicates a correct response for either a multiple choice or a short answer question 0 indicates an incorrect short answer response a, b, c, d, e indicates the incorrect multiple choice response given by student blank cell indicates that the student did not answer that question. Note: When the students’ names are not in view, hover the cursor over any of the responses, 0, 1, a, b, c, d, e to display the name of the student who recorded that response. 5. Short answers to some questions e.g. 35 in the Numeracy question 6 shown above, or 3, 4, 1, 2 in sequencing questions in the Reading test. 6. Question description or the skill assessed in each question is displayed when the cursor hovers over the question number e.g. ‘Locates a fraction on a number line’ in the Numeracy test. It is recommended that copies of the real test papers be available when viewing this data. 7. Expected % correct will be flagged green when the School % correct is significantly higher than expected. This alerts you to possible ‘strengths’ in teaching programs and good understandings in students’ learning. See Comparison graphs for a visual representation of this. 8. Expected % correct will be flagged red if the School % correct is significantly lower than expected. This alerts you to possible ‘gaps’ in teaching programs and misconceptions in students’ learning. Click on File in the top left corner Select Export Data to export data to an Excel spreadsheet (p 22) Select Copy to Clipboard to take a screenshot that can be pasted into a Word document (p 23) Select from the menu in Order by, in the bottom left hand corner of the screen, to sort the data according to your requirements 10 Assessment & Reporting 2014 Assessment Data (continued) Questions How did the school’s mean compare to the Australian mean for each criteria? Were there one or two critieria where the students performed well or poorly compared to the Australian mean? What does this mean for the way we teach writing in our school? Process Select Year Group e.g. Year 5 and Assessment Area e.g. Writing The Maximum Score for each of the 10 criteria is shown in the top row - Audience = 6 - Text Structure = 4 - Ideas = 5 - Persuasive Devices = 4 - Vocabulary = 5 - Cohesion = 4 - Paragraphing = 3 - Sentence Structure = 6 - Punctuation = 5 - Spelling = 6 The Australian Mean is shown in the row below this The School Mean is shown in the third row for each of the criteria. This allows easy comparison Individual student’s Total Score, Score (NAPLAN) which is the scaled score, NAPLAN band and the scores for each of the writing criteria are shown across each row Note: In the NAPLAN data for 2011- 2014 the writing text type is Persuasive – the 4th criterion is Persuasive Devices; Paragraphing has a total score of 3. In 2008-2010 the writing text type is Narrative – the 4th criterion is Character and Setting; Paragraphing has a total score of 2. The writing data can be - saved to an Excel file and graphed in many different ways (p 22) - viewed in the Comparison Graphs display with separate bar graphs for each Criteria (p 16) compared across year levels as, unlike the other tests, the writing test stimulus is the same for all year levels 3 to 9. A common marking rubric is used – see page 3 for the Writing Marking Guide. The total score is out of 48. 11 Assessment & Reporting 2014 Student Distribution Questions What is the distribution or spread of students across the National NAPLAN Bands? Are students clumping in a certain area and how can we ensure that students will make progress? What % of students fall in the top 20%, middle 60% and lower 20% of Australian achievement? Process Select Year Group e.g. Year 7 and Assessment Area e.g. Reading Select Student Distribution This screen shows the location of a group of students (their initials) against the National NAPLAN Bands and the overall Australian % achievement. 1. Australian Schools Mean, All NT Schools Mean and the School (or sub-group) Mean 2. The School % compared to the top 20% of all Australian School achievement, middle 60% and lower 20% 3. National Achievement Bands aligned to the NAPLAN scaled score ranges e.g. Band 4 - 399 to Band 9 - 674 for this Year 7 group 4. Students Above the national minimum standard 5. Students At the national minimum standard (shown between the two blue lines) 6. Students Below the national minimum standard Go to File and select Print to print the screen Select Print Setup then change setting to A3 paper to see more detail 12 Hover over the initials of any student to see their full name. Note: Students below the national minimum standard are ‘at risk’ of not progressing well at school. Students at the national minimum standard or just above it should be closely monitored. Tick the Exclude exempt students box to include or exclude exempt students. Percentages in the columns to the right will be recalculated while mean scores will not change as exempt students scores are not included in the mean. Assessment & Reporting 2014 Performance Profile Questions What patterns of performance are evident when looking at individual student responses to the test questions? Does my classroom assessment agree with this data? Can my students use this report to set learning goals? Process Select Year Group e.g. Year 7 and Assessment Area e.g. Reading Select Performance Profile to show all question descriptors or criteria levels Click on student’s initials and they will be highlighted by a red box e.g. EB This student’s correct question responses are shown with a tick and in green text The incorrect responses are highlighted in red text Click on a new student and view their responses Note: The questions are ordered according to the degree of difficulty – from the easiest at the bottom of the screen, to the most difficult at the top of the page. The question number is in brackets at the end of the descriptor. This report will help the teacher to pinpoint the students’ areas of strength and weakness. Once the initials of a student are highlighted in the red box, the teacher can then click on the Individual Profile button which will show an individual report for each student. Details are on the next page. Click on a student’s initials Un-tick the Show Student Initials box The selected student’s initials will be blocked out in red while all other student initials will be blocked out in black Click on another set of student initials and it will now become the red selected student box Note: This feature can be used to show students and parents where the student is located in relation to others in the group. 13 Assessment & Reporting 2014 Individual Profile Questions Where does the individual student sit in relation to the national minimum standard? What areas of strength and weakness are shown for this individual? Does my classroom assessment agree with this data? Process Select Year Group e.g. Year 9 and Assessment Area e.g. Numeracy Select Performance Profile first and click on the student’s initials as shown on the previous page Then select Individual Profile button The screen becomes customised for an individual student. It shows: - the student’s name at the top of the page and above the bold black line that represents their NAPLAN Scaled Score - their correct question responses in green text with a tick beside it - the incorrectly answered questions highlighted in red text - the question number in brackets. Return to Performance Profile to select a different student then view their Individual Profile. Note: The questions are ordered according to degree of difficulty – from the easiest at the bottom of the page to the most difficult at the top of the page. The question numbers are in brackets at the end of the each descriptor. Students can easily see the questions and associated areas of the curriculum where they experienced difficulty. Go to File and select Print to print the Individual Student report. Select Print Setup then change the setting to A3 paper to see more detail. Performance and Individual Profile screens can be shared with students and/or parents. Teachers, parents and students are able to: - locate the National NAPLAN Band and the NAPLAN scaled score for each student determine the location of the student within the year level or group cohort see the pattern of correct and incorrect responses for each student identify areas of strength and/or weakness as each question’s skill descriptor is stated use this information to help develop specific learning goals for improvement. Access to the actual test paper is useful when analysing the student’s error responses, see Appendix B. 14 Assessment & Reporting 2014 Comparison Graphs – Reading, Spelling, Grammar & Punctuation and Numeracy Questions What test items did the group have difficulty with compared to the national performance? Was there a common incorrect distractor (a, b, c, d or e) that many of the students chose for a specific question? What can this error analysis show both in terms of student conceptual development and teaching pedagogy? Process Select Year Group e.g. Year 9 and Assessment Area e.g. Numeracy Select Comparison Graphs The stacked bar graph shows the students’ responses to each question. The questions are ordered from left to right by the degree of difficulty calculated from all of the Australian results – easiest to most difficult. Go to File, select Print or Select Copy to Clipboard and paste the graph into a Word document which can be saved The green dotted line below shows an approximation of the national profile of performance. The red arrows indicate questions where the students performed lower than the national level and conversely, the green arrows indicate questions where the students’ achieved a better result than the national profile. The green part of the bar shows the number of students who answered correctly in this group; other coloured parts of the bar show the numbers of incorrect a, b, c, d or e distractors chosen by the students. 15 Assessment & Reporting 2014 Comparison Graphs – Writing Questions Were there one or two critieria where the students performed poorly or well? Do other classes show similar results in these criteria? Which score do we want to aim for in each criteria for our grade level (the main spike)? What does this mean for the way we teach writing in our school? Is a whole school approach needed across all grade levels? Process Select Year Group e.g. Year 5 and Assessment Area e.g. Writing Select Comparison Graphs The Comparison Graphs for writing differ from the other tests. Each criterion has its own separate bar graph showing how many students scored a particular score e.g. in the graph shown here for Audience: 8 students scored 2 19 students scored 3 5 students scored 4. The most common score for each group is very obvious as it is the highest spike. Comparison Graphs can be printed out directly or copied and pasted into a Word document so that it can be saved. This graph can be used when asking reflective questions and when comparing groups in the same year level and between year levels. Note: In the NAPLAN data for 2011- 2014, the writing text type is Persuasive; the 4th criterion is Persuasive Devices; and Paragraphing has a total score of 3. In 2008-2010 data, the writing text type is Narrative; Character and Setting is the 4th criterion; and Paragraphing has a total score of 2. The Narrative and Persuasive Writing Marking Guides are available with the past tests, see page 3 and the Australian Curriculum Assessment and Reporting Authority website. http://www.acara.edu.au/default.asp 16 Assessment & Reporting 2014 Filter Students Questions What patterns of performance are evident in the Assessment Data, Student Distribution and Comparison Graphs displays for specific subgroups of students? What changes in pedagogy and resource allocation can improve the achievement of these subgroups? Process Select Year Group e.g. Year 7 and an Assessment Area e.g. Reading School Name Reading Year 7 Go to Filter Students section Select one or several of the available filters from the drop down menus e.g. Gender – Male Student names Assessment Data now shows only males in this Year 7 cohort. New Subgroup % and Expected % have been calculated for each question as well as the associated red and green highlights. Select other displays e.g. Student Distribution Reading Year 7 The new Subgroup Mean is displayed and the spread of the subgroup results New Subgroup % for each section is displayed and can be compared to the Australian percentages (in the top 20%, middle 60%, and lowest 20%) New filter combinations can be chosen from the drop down menus Select the Reset Filters box to return to All students All reports and data sets shown thus far can be filtered in a variety of ways by selecting one or more combinations of demographic filters from the Filter Students drop down menus. Teachers can filter in a myriad of ways to create data for any subgroup of students. For example filter by: - individual classes to locate specific areas of strength and weakness - male/female or indigenous/non-indigenous responses to questions that would indicate varied abilities in certain areas - other specific groups to help teachers analyse results and use this information to inform decisions that change classroom practice or reallocate resources etc. 17 Assessment & Reporting 2014 Filter Students - Setting up new student subgroups PROFILE box Questions What are the specific subgroups that can be found in our school? What subgroups in our school receive specific programs to improve performance and how can we track this? What patterns of improved performance are evident in the Assessment Data, Student distribution and Comparison Graphs displays for specific subgroups of students? What changes in pedagogy and resource allocation may be informed by the data? Process Select Year Group e.g. Year 5 and Assessment Area e.g. Reading Select Students display button Student names Double click on one student’s name and a Student Properties box will appear Change the student’s personal details Add or change Program Codes using a 2-digit code e.g. EA for EAL/D students or QS for Quick Smart students etc. Click OK to save Create a new group by using the Shift or Control keys to highlight a group of students Click the Open button at the bottom of the screen. A Student Properties box will appear Student names Add or change the Class group and/or Program Codes for the selected group Click OK and these Class or Program Codes details will appear in the Program Codes in the Filter Students drop down menus In the Students display screen, individual and class group details can be changed at any time. The demographics, Class and Program Codes can be adjusted for students so that any new subgroup can be created and filtered to obtain specific data sets for analysis. 18 Assessment & Reporting 2014 Printing Questions What display screens or groups do I need to view, save and print for easy data analysis? What is the best way to present the data for a specific group or individual in the different test domains? Process Select Year Group e.g. Year 3 and Assessment Area e.g. Numeracy Select the Assessment Data display screen Choose the student group that you need by using the Filter Students categories If required, re-order the data results by selecting from the drop down menu for Order by Go to File select Print and follow the usual print preview, print set up process that you would do with Word documents The printed report from the Assessment Data screen is a summary that lists: Numeracy Year 3 - Students’ names, class, DOB and other details - Total Score, the raw test score e.g. 11 (out of 32) - Score (NAPLAN) which is the scaled score Student names - NAPLAN band (National Achievement Band) - Minimum Standard level descriptions: below, at or above. This report will not include the individual question responses or the writing criteria scores. Copying the data to Clipboard or exporting the data to Excel will provide this information (p 21 - 22). 19 Assessment & Reporting 2014 Printing (continued) To print the Student Distribution or Individual Profile screens Student Distribution Set up the required group and order the data as required Print reports to highlight the spread of student achievement e.g. Student Distribution Print reports to show areas of strength and weakness for individual students e.g. Individual Student Profile Print reports to show the most common incorrect answers chosen by the students and compare results with Australian achievement e.g. Comparison Graph (as shown below) Print on A3 paper to make the reports easy to read 2014 Numeracy Year 3 20 School Name Assessment & Reporting 2014 Individual Student Profile Export data to Excel Questions In what ways do I need to view, manipulate and save data sets as part of our school’s analysis? What is the best way to present the data for a specific group or individual in the different test domains? Process Select Year Group e.g. Year 3 Assessment Area e.g. Spelling Go to File Click on Export Data Save As box will appear, direct it to an appropriate folder e.g. 2014_RAAD Type in a precise file name e.g. 201_Spelling_Yr3 Click Save The file will be saved as a CSV file which is a Comma Separated Values file. This is a simple type of Excel document with limited functionality, used in applications like RAAD. Minimise or close the RAAD screen and open the newly saved CSV file. Go to File Select Save As and save this file as an Excel Workbook in the same folder and give it the same name as the previous CSV file. The CSV file can now be deleted to save confusion. 21 Assessment & Reporting 2014 Export data to Excel (continued) The Excel file will enable student data and group statistics to be presented in a variety of ways Changes can be made e.g. - Adjust column and margin size - Delete unnecessary columns Student names - Set up page layout to landscape - Use the Sort and Filter tools in Excel to rearrange the data Create innovative graphs to compare and sort information for use when analysing the data Use the Excel Chart Wizard to create graphs for presentation of results and analysis of trends over time e.g. bar graphs to compare means over a number of years or pie graphs to show one year’s breakup of results. Create graphs that compare your school’s results with the Australian results. Look for areas of strength and weakness. 2014 Year 9 Writing These visual representations of the data can act as springboards for staff discussion on the efficacy of programs currently being used and new ways of teaching that may need to be trialed. (See Data Rules in the Appendices.) 22 Assessment & Reporting 2014 Copy data reports to Clipboard using Word Questions How can I present the data to teachers for analysis and discussion? Process Select Year Group e.g. Year 3 Assessment Area e.g. Reading Filter and order the group as required Click on File Select Copy to Clipboard This will take a ‘picture’ of the data screen to paste into a Word document. Go to Start button Open a new Microsoft Office Word document Go to Page Layout (at the top of the Word document) Change the Orientation to Landscape Use Narrow Margins to enable the clipboard picture to fit on the page. Select Paste. This will place the clipboard picture of the data report into a new word document Click on this ‘picture’ of the data and go to Format to adjust and crop the report Print onto A3 size paper to see more detail if required. Student names Teachers can use this hard copy to analyse student responses, discuss results and compare percentages. 23 Assessment & Reporting 2014 Copy data reports to Clipboard using Word (continued) Examples of clipboard reports This Numeracy clipboard report shows the student information and test question responses grouped by Numeracy strand. This is excellent for data analysis of student responses. Teachers can - note the questions that have been highlighted as green or red (p10) - look for areas where most of the class performed well or had difficulty (strength/ weakness) - look at individual scores - identify the criterion where a section of or the whole class have scored below the mean - compare the class mean with the Australian mean. It is recommended that teachers refer to the Writing Marking Guides, see page 3. This data report in combination with the Comparison Graph can point to areas of strength and weakness in the students’ writing. 24 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 A3 Li l y Moni el m a Br i tti e Vi ci a Pett Bi l l i s Deanet Fel i o Char t Pett Kenn Pegi ndr a Jer t Howar y Bober y Char t Mi r i na Gl or ol ar a Deni ffana Er i s ti n Howar y Vi ca Fel i o Bober y Er i s ti e Br edr i s co Wal vadl ey Er i s ti n Jer t Kather tl e Ser to Kenn 30/1/2001 F 27/4/2001 F 10/4/2001 F Dunney 7/4/2000 M Pi er s on 10/10/2000F Col l es 2/7/2000 F Gonzal m es 4/10/2000 F Huncer 27/5/2001 M Bel ds 17/10/2000M Hal er 7/7/2000 M Hal er 4/8/2000 F Jor d 22/11/2000F Pi er s on 27/1/2001 F Bel ds 3/5/2001 M Jor d 25/5/2001 F Al er 15/7/2000 F Hal er 4/2/2001 F Edwar ds on 29/7/2001 F Al er 30/9/2000 F Huncer 26/2/2001 F Rhol t 27/10/2000F Pi el s 16/9/2000 F Rhol t 16/10/2000M Al er 11/6/2000 M Taynol d 16/3/2001 F Dunter 6/11/2000 F Bur nol s 2/7/2000 F Fr ay 4/5/2001 M Jor d 18/10/2000M Phi ns ti n 2/1/2001 M Fr ay 14/1/2001 M Dunney 22/11/2000F Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 27 26 25 23 22 22 21 20 20 19 18 18 18 17 16 15 14 14 13 11 10 10 9 8 8 4 1 0 0 0 0 0 538 524 509 480 464 464 449 434 434 418 402 402 402 387 371 355 339 339 324 295 282 282 269 256 256 203 140 89 89 89 89 89 Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band Band 6 6 6 6 5 5 5 5 5 4 4 4 4 4 3 3 3 3 3 2 2 2 1 1 1 1 1 1 1 1 1 1 Above Above Above Above Above Above Above Above Above Above Above Above Above Above Above Above Above Above Above At At At Bel ow Bel ow Bel ow Bel ow Bel ow Bel ow Bel ow Bel ow Bel ow Bel ow Assessment & Reporting 2014 6 4 5 4 5 4 2 6 5 6 2.271.712.121.712.021.830.192.031.792.39 1.661.251.531.161.411.380.281.631.381.75 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 2 3 2 2 2 2 2 2 2 2 2 2 1 2 1 1 1 2 1 1 1 1 1 1 1 0 0 3 3 3 2 3 2 2 2 2 2 3 2 2 3 2 2 1 1 2 1 1 1 1 1 1 1 0 2 2 2 2 2 1 2 2 2 2 2 1 1 2 1 2 1 1 1 1 1 1 1 1 1 0 0 3 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 2 1 1 1 1 1 1 1 0 2 3 2 3 2 3 2 2 2 2 2 2 2 2 1 1 2 2 1 1 1 1 1 1 1 0 0 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 0 1 0 0 0 0 0 0 0 0 3 4 3 3 2 3 2 3 2 2 2 2 3 2 2 2 2 2 1 2 1 1 1 1 1 0 0 4 3 3 3 3 2 3 2 2 2 2 2 2 1 2 1 2 1 1 0 1 1 1 0 0 0 0 4 3 3 3 3 3 3 3 3 3 1 2 2 1 2 2 2 2 1 2 2 2 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 Comment Spel l i ng Punct uat i on Sent ence St ruct ure Paragraphi ng Cohesion Nar r ati ve Vocabul ary Charact er and Set t i ng Ideas T ext St ruct ure M i ni m um St andard NAPLAN band Maxi m um Scor e Al l Aus tr al i an School s Mean/Pr op Cl as s Mean/Pr op Pi er s on Student Gi l er Names Taynol ds Score (NAPLAN) Gender Abori gi nal LBO T E Absent T ot al Score Last Nam e Program Code Program Code 1 Program Code 2 Program Code 3 F irst Nam e Class The teacher can Dat e O f Bi rt h 9999 - Fic tional Sc hool This Writing clipboard 2009 W riting Year 3 report shows all the student information and the scores for each of the 10 writing criteria for one class group in Year 3. Audi ence - analyse common error responses e.g. most of the students responded incorrectly with a ‘b’. Appendix A RAAD The RAAD application and the User Guide are available on the Department of Education website: www.education.nt.gov.au/teachers-educators/assessment-reporting/nap NAPLAN Test Papers NAPLAN test papers, answers, alignment documents and marking guides are available on the Learning Links portal which can be accessed through the NT Department of Education staff website. http://ed.ntschools.net/ll/assess/naplan/Pages/default.aspx 25 Assessment & Reporting 2014 Appendix B National scale of achievement For each assessed area a NAPLAN scale of achievement has been developed. The performance of Years 3, 5, 7 and 9 students are measured on a single common achievement scale. It is divided into 10 achievement bands representing increasing demonstrations of proficiency in the skills and understandings of the assessment. This also enables individual achievement to be compared to the national average. Every raw score for a test is converted into a NAPLAN scaled score out of 1000. For example, an average Year 3 student’s score is around 400, while an average Year 9 score would be around 580. This is a vertical scale and a student’s score in 2014 can be compared to their 2010 and 2012 scores to gauge progress. Student results are reported against the NAPLAN achievement bands. There are 5 separate scales: one each for reading, writing, spelling, grammar and punctuation (Language Conventions) and numeracy. A score in one assessed area is not comparable with a score in another test domain. The achievement scale’s 10 bands are represented across the four year groups, with each year group being reported across 6 bands. This allows for tracking and monitoring of student progress over time. Refer to the graphical representation of the achievement scale below. The second bottom band for each year level represents the national minimum standard (orange). This standard represents the minimum range of acceptable performance for the given year level as demonstrated on the assessment. Students whose achievement places them in the band below are identified as below the national minimum standard (red). For a more detailed description of student reports and the National Minimum Standards go to: www.nap.edu.au/results-and-reports/student-reports.html Students who are below the national minimum standard should receive focussed intervention and support to help them achieve required literacy and numeracy skills. Students who are at the national minimum standard are at risk of slipping behind and need close monitoring and additional support. 26 Assessment & Reporting 2014 Appendix C Means summary sheet The RAAD Student Distribution screen shows the school and Australian means. This template can be used to collate and compare all the means. This top level analysis may provide a starting point for closer analysis and action in certain areas of the curriculum for each year level. In what areas did our school mean vary greatly from the Australian mean? 2014 Year 3 Year 5 NAPLAN Mean Aust School Aust School Reading Writing Spelling Punctuation and Grammar Numeracy 2014 Year 7 Year 9 NAPLAN Mean Aust School Reading Writing Spelling Punctuation and Grammar Numeracy 27 Assessment & Reporting 2014 Aust School 28 Assessment & Reporting 2014