VCE School Leaders Briefing 2015 Running Times 9:30 Administrative Procedures 9.50 Special Provision update 10.00 Curriculum update 10:40 School-based Assessment Audit 10.50 Break 11.10 Exam Conduct and Supervision 11:20 Statistical Moderation and Study Score Administrative Procedures School and VCAA responsibilities • Schools • Provision and oversight of a suitable program • Collection of student details and enrolment data • Entry of student data on VASS • VCAA • QA data • Processing student enrolment and result data • Reporting to students, schools and other stakeholders • VCE and VCAL Administrative Handbook 2015 Key school tasks • Run eligibility • Identify students who are not eligible and adjust program • Print Student Full Details / Student Personal Details Forms • Distribute for students to confirm all details • Print class lists for teachers • Final checks before due dates Examples of SRRU identified data errors Examples of SRRU identified data errors VASS Critical dates ENR Enrolment 1 on 23 February • venues for oral and performance exams • setting of end-of-year examination timetable (to minimise clashes and 3 in one day) Enrolment 2 on 27 April • stationery for the GAT • DET will base VET funding on enrolment as of this date. Enrolling students into a Second Language (EAL) • • • • VASS will disable VASS/VCE coordinators’ ability to grant EAL approval for students whose country of origin is Singapore or India. Approval for these countries can only be granted by the VCAA. Schools are required to forward the applications to the VCAA for a thorough assessment to be made. Applications need to include supportive school documentation and passport stamp and/or International Movement Records to determine if the student is eligible for EAL. If approval is granted, it will be entered on VASS and the student will be able to be enrolled. Student transfer and credit from interstate studies • • • • Students must apply to the VCAA through the Victorian school at which they are enrolled using the Application for Credit towards the VCE/VCAL form. At Units 1 and 2 level, schools will recommend the amount of credit to be granted on receipt of an interstate school or authority transcript and after consulting the equivalent qualification guidelines. Credit recommended by the school must be entered on the application form (Part E). Previous school reports and certificates should be attached. Decisions on the amount of credit at Units 3 and 4 level will be made by the VCAA. Privacy Considerations • Both the VCAA and schools have obligations under privacy legislation to protect the personal information of students. • The VCAA is aware that some schools engage a Third Party to analyse end of year data. • The VCAA discussed this use with the Victorian Privacy Commissioner. • We were advised that schools can best meet their privacy obligations by providing de-identifiable data to third party analysts. Contact Details Student Records and Results Kerry Veal, Data Management Team Leader Phone : 9032 1754 veal.kerry.a@edumail.vic.gov.au Special Provision What’s New in Special Provision In 2014 the VCAA commenced a review of its Special Provision policy and procedures. The review process consists of 4 key stages : • an online survey (March – April 2014) • focus group consultation – using feedback from online survey (September 2014) • Development of high level recommendations (October 2014) • Policy development and staged implementation (ongoing). Special Provision Review - continued The aim of the review is to ensure that: • VCAA policy and procedure reflects current relevant State and Federal legislation • Key stakeholder concerns are considered and addressed • There is greater consistency in the approval of special provisions across School-based Assessment (school) and VCE examinations (VCAA) • Stakeholders have more certainty about the appropriateness of special provisions for School-based Assessment and VCE examinations • School and VCAA application processes are as efficient as possible. Special Provision – 2015 and beyond • 2015 Special Examination Arrangement applications will arrive in schools from February 23 (currently on VASS as download) • Administrative advice re 2015 Special Provision policy and procedures will be available on the VCAA website by the end of February www.vcaa.vic.edu.au/Pages/vce/exams/specialprovision/special provision.aspx • Stakeholders will be kept informed on the progress of the Special Provision review via VCAA notices and/or VCAA website www.vcaa.vic.edu.au/Pages/vce/exams/specialprovision/special provisionreview.aspx Curriculum Studies in Review 2015 • Art • Business Management • Economics • Food and Technology • Global Politics • Music • Physical Education • Religion and Society • Studio Arts • Texts and Traditions Once accredited, for implementation in 2017 Curriculum development and review Input into the Review and Accreditation Cycle Curriculum Managers Teacher and stakeholder consultation Subject Associations Review process Evidence-based review by panel comprising teachers, university, TAFE and industry representatives. • International and national benchmarking • Analysis of data - Enrolment trends - School-based and examination results - Examiners’ reports - State reviewer reports from coursework audits - Online surveys - Teacher focus groups Consultation on draft study designs Consultation register now open Notice to Schools when individual study consultation is open: • Draft study design • Questionnaire • Summary of proposed changes Teacher involvement • • • • • Study review panels Text list advisory panels Exam assessors Exam panels State reviewers Excellent professional learning opportunities. Call for applications in VCAA Bulletin; apply through SSMS. • All activities recognised by VIT. New VCE studies for implementation in 2015 • Algorithmics (Higher Education Scored Study – HESS) Units 3 and 4 only (see February Bulletin) • Chin Hakha Units 1 and 2 – 2015 Units 3 and 4 - 2016 Publication of study designs online only. Advice for teachers separate companion document on the study page Revised and new VCE studies – implementation in 2016 English/EAL, Literature, English Language, Bridging EAL (new, units 1-2 only) Mathematics – all, inc. Specialist Maths Units 1 and 2 Ancient History (new), Revolutions, Australian, Global Empires (new), and 20th Century Biology, Chemistry, Environmental Science, Physics, Psychology Geography Computing All studies published online by end February/early March Implementation timeline • 2015: statewide professional learning program to prepare for each revised or new study • 2016: accreditation period from 1 January 2016.Implementation of newly accredited studies EXCEPT English/EAL, Literature: Units 1 and 2 - 2016 Units 3 and 4 – 1 January 2017 Biology, Chemistry, Environmental Science, Physics, Psychology : Units 1 and 2 - 2016 Units 3 and 4 – 1 January 2017 Timeline for English/EAL and Literature Study Status Implementation Professional learning Advice for teachers First year of examination English/EAL Now available on website Units 1-2 – 2016 Term 2 – 2015 Units 3-4 – 2017 State-wide Term 3 – 2015 2017*EAL will Units 1-2 – 2016 Term 2 – 2015 Units 3-4 – 2017 State-wide Term 3 – 2015 2017 Now available on website 2016 – Units 1-4 Term 2 – 2015 Term 3 – 2015 2016 To be published by early March 2016 – Units 1-2 Term 2 – 2015 Limited face-toonly Term 3 – 2015 NA TBC TBC NA include a listening task as part of schoolassessed coursework and examination 2017 program Literature Now available on website program English Language Web conferences (minor review) Bridging EAL face and polycom (new) Foundation English Information gathering TBC Foundation English monitoring activity • The VCAA is currently gathering information about Foundation English to inform a review in 2016. • The VCAA is keen to hear from schools and providers delivering Foundation English to participate in focus groups during Term 1 and Term 2. Contact Sean Box, Curriculum Manager, English: box.sean.m@edumail.vic.gov.au or (03) 9032 1691. Selecting texts for the English classroom • The VCAA, in partnership with the Stella Prize Schools Program and the Victorian Association for the Teaching of English (VATE), will be holding a panel discussion about texts, text selection and why it matters. • Facilitator Bec Kavanagh will discuss issues facing text selection in the context of the secondary English classroom with panellists academic Dr Larissa McLean-Davies, author Cate Kennedy and teacher Louise Bourke. • Find out more about the event from the VCAA website: http://www.vcaa.vic.edu.au/Pages/vce/pd/selectingtext.aspx Contact Sean Box, Curriculum Manager, English: box.sean.m@edumail.vic.gov.au or (03) 9032 1691. Computing - 2016 Change of name • Units 1 and 2 Computing • Units 3 and 4 Informatics • Units 3 and 4 Software Development • introduction of SATs in both 3-4 sequences • incorporates key concepts and terminology within Australian Curriculum Digital Technologies • no longer approved programming list School based assessment resources Assessment Handbooks Published for all VCE studies • SAC performance descriptors are advisory (except for Extended Investigation where their use is mandated) • SAT criteria and descriptors of level of performance are mandatory • Apply to Units 3-4, can be used to inform Units 1and 2 • Published online only on the individual VCE study pages Companion document Procedures for Assessment in VCE Studies Advice for Teachers – new format • Merger of Advice for Teachers and Assessment Handbook into one online publication • For studies published in 2015 onwards Bulletin article November 2014 School-assessed Tasks 8 studies with SATs • Mandated assessment criteria and performance level descriptors on a 10 point marking scale • Published each year in February on individual study pages • Authentication record for each study • Full day professional learning program conducted annually in February/March Industry Pathways • New in 2015 (two year program first award in 2016) • Students who complete the Industry Pathway will have this title included on their VCE or Senior VCAL certificate. Available in four industries: • Building and Construction • Community Services and Health • Manufacturing and Engineering • Sport and Recreation Industry Pathways • Must include VET studies, English, Maths (Literacy and Numeracy in VCAL), further studies related to the industry theme 80 hours of workplace learning. • Contextualisation to the industry theme is recommended • More information on Industry Pathways available on the VCAA website: http://www.vcaa.vic.edu.au/Pages/vet/Industry_pathways_VCE_VCAL. aspx Contact: Mark Frankland Special Projects Manager frankland.mark.s@edumail.vic.gov.au (03) 9032 1732 VCE (Baccalaureate) Introduced in 2014. 3,828 students awarded the VCE (Baccalaureate) in 2014 (approx. 8% of VCE completions) To be eligible to receive the VCE (Baccalaureate) the student must satisfactorily complete the VCE and receive a study score for each prescribed study component: • a Units 3 and 4 sequence in English or Literature or English Language with a study score of 30 or above; or a Units 3 and 4 sequence in EAL with a study score of 33 or above • a Units 3 and 4 sequence in either Mathematics Methods (CAS) or Specialist Mathematics • a Units 3 and 4 sequence in a VCE Language • at least two other Units 3 and 4 sequences FAQs on VCAA website: http://www.vcaa.vic.edu.au/Pages/vce/studies/index.aspx Higher Education Studies • Contribute to satisfactory completion of the VCE as a 3-4 sequence • No more than one HE study can contribute • Contribute to the ATAR as 5th or 6th unscored study (incremental scale) • List of approved 1st year university studies on VCAA website: www.vcaa.vic.edu.au/pages/vce/studies/studiesextension Northern Hemisphere Assessment Timetable • • • • • VCE program on a different timetable: Sept – June Commences with Unit 1 and 2 enrolments in 2015 Unit 3-4 from September 2016; exams June 2017 Starting with 12 studies. Same curriculum and assessment requirements and processes • Emerged out of current VCE international program • Available to Victorian Schools and students in the future • EOI and stakeholder reference group in coming months. VCAA sponsored Awards • VCE Season of Excellence • VCE Leadership Awards (formerly VCE Achiever Awards) • VCAA Plain English Speaking Award • Margaret Schofield Memorial Scholarship (music performance) • Premier's VCE Awards • John Button School Prize • VCAL Achievement Awards School-based Assessment Audit Why does the VCAA audit? Provides opportunities to: • Validate the quality of the school-based assessment program • Identify instances for school improvement • Set up professional conversations about teaching and learning Selection of Schools All VCE providers are audited for at least one study. Random selection except for: • Schools that are teaching the study for the first time (new providers) or, for particular studies, after a break of three years • Schools where problems were identified the year before • Schools where there was low correlation between the internal and external scores following statistical moderation in the previous year (2014) Audit Process • All studies are subject to the audit process. This includes the school-assessed task component of any study. • The school-based assessment audit process is undertaken in four stages. • Stages 1 and 3 relate to the completion and submission of an online questionnaire. • Stages 2 and 4 related to the submission of further evidence (if required). • Schools notified of audit requirements via email and VASS (new) for Stage 1 on 2 March and Stage 3 on 22 June. • Completion of the online questionnaire for Unit 3 (Stage 1) due by 20 March and Unit 4 (Stage 3) 17 July. Audit Process • Notification of the outcome of the audit and any further evidence needed for Stages 2 and 4 sent to school via email and VASS. • Further evidence may be in the form of ─ Coursework tasks ─ Marking schemes/assessment criteria ─ Timelines ─ Authentication records • Further evidence needs to be submitted to the VCAA electronically with the accompanying coversheet as a single PDF for each study by ─ Stage 2: 22 May ─ Stage 4: 11 September Audit Process • Audit panel checks tasks against the requirements of the Study Design • School-based assessment must allow the students the opportunity to meet the outcomes • The school is notified immediately if a serious anomaly is detected (e.g. incorrect text being taught) • Feedback is often included in email notification to school. Audit Outcome • “Submission meets requirements” - Meeting minimum requirements of study design. Suggestions for improvement can be provided. • “Further evidence required” – to ensure the requirements are being met. Feedback provided as to why. • “Immediate follow up with Curriculum Manager required” – The submission indicates possible error or poor practice. Resources General administrative information for school-based assessment - www.vcaa.vic.edu.au/pages/vce/index/VCE General Advice and Policy VCE and VCAL Administrative Handbook 2015 • Report on Lost/Stolen/Damaged School-assessed Tasks Individual study pages for SAT information • Authentication Record for School-assessed Tasks • Teacher additional comment sheet for Technology studies School-based Assessment Unit Results Schools report students’ results for completion of VCE Units, and VCE VET units of competence/modules VCE Unit Results are reported as: S = Satisfactory Completion N = Not Satisfactory J is to be used where the student: • is no longer attending class • has not submitted work for assessment VCE VET unit of competence/module results are reported as: S = Satisfactory Completion N = Not Yet Completed Satisfactory Completion Satisfactory VCE Unit Result The student receives S for a unit when all the outcomes are achieved A student must: • Produce work that demonstrates achievement of the outcome/s • Observe school and VCAA rules If a teacher judges that all outcomes are achieved, the student satisfactorily completes the unit. The decision to award an ‘S’ for the unit is distinct from the assessment of levels of achievement. Importance of Internal Moderation • Consistency of teacher decisions • Procedures for making school-based assessments • For Statistical Moderation purposes, each school’s assessment in a study is treated as a single group not as separate unique teaching classes Indicative Grades Indicative Grades: • primary purpose is to be used as part of the quality assurance procedures for marking examinations, however they are also used in the calculation of the DES. • are the prediction of a student’s actual level of achievement on the examination • the rank order and level of spread of the indicative grades for the school cohort in the study is of prime importance The published previous years grade range can be used to inform this process. Indicative Grades Suggestions for producing Indicative Grades: • Indicative Grades should be based on a similar task to the exam or even a past exam. Not necessarily the same as coursework performance. • The previous years grade range can be used to inform this process http://www.vcaa.vic.edu.au/Pages/vce/statistics/subjectstats.aspx • Not a traditional notion of what a grade’s value may be. In one Study the A+ exam grade range may be 98-100, yet may be 74-100 in another. Contact Details Student Records and Results Nick Dobroff Phone : 9032 1743 dobroff.nick.n@edumail.vic.gov.au Exam Conduct and Supervision VCE Examination Conduct and Administration Support Resources • • • • • VCE Examination Manual VCE Examination Document Register VCE Exams Navigator GAT Information brochure Training/Briefing Sessions for chief supervisors and relevant school personnel - Training Sessions for new Chief Supervisors and/or new school personnel - MAY - Briefing Sessions for ongoing Chief Supervisors and school personnel – AUG/SEPT • VCAA Bulletins and Notices • VCAA website - vcaa.vic.edu.au/vce/exams VCE External Assessment Agreement 2015 • All Principals must read and sign this agreement to certify that they fully understand all VCAA requirements for administering and conducting VCE external assessments. • The Principal may delegate the management of their VCE external assessments to appropriate school personnel. 2014 Examination Conduct and Supervision issues – focus for 2015 • Need for school personnel to work with and support supervisors, particularly when only one supervisor • Ensure students are adequately prepared • School personnel to carry out materials checks prior to entering examination room • Mobile phones/electronic devices – final check before reading time commences – confiscate then return • Understanding and administering Special Examination Arrangements 2014 Examination Conduct and Supervision issues – focus for 2015 • Additional response materials cannot be brought in by students. Spare general answer books should be provided. Lined paper as a last option, must include student number, question number etc. Fill-in incident report & journal when lined paper supplied. • Supervisors/school personnel to be aware of emergency evacuation plan • Observation visits during VCE examinations and the GAT will continue in 2015 VCAA Key Contacts • VCE Examination Planning and Logistics: • Maria Fragale • Phone: 03 9225 2255 • Email: fragale.maria.m@edumail.vic.gov.au • Susan Meadows • Phone 03 9225 2237 • Email: meadows.susan.s@edumail.vic.gov.au Statistical Moderation and Study Score Calculation What and Why? Statistical Moderation • A process that moderates the School-based Assessment scores • Uses the exam scores because it is the only task that is exactly the same for each student in the study What is Statistical Moderation? Statistical Moderation Process to ensure comparability of results Aims to eliminate marking differences between schools, not within Aligns external and internal assessment scales Maintains rank order given by the school The Common Scale Statistical Moderation USA $299 Singapore $370 Australia $350 Which is the cheapest ? The smartphone from Singapore Why is it Needed? Statistical Moderation Assessment marking varies between schools Measurements are only comparable when they are expressed on the same scale School-based assessment can only be included in the ATAR (ENTER) if it is moderated How is it Done? Statistical Moderation Identify the moderation groups in each study Calculate an external score for each student Align the distributions of school-based assessment and external scores Calculate moderation formula and then apply to the coursework score for each student Using the Common Scale Raw School Assessment Statistical Moderation External Score = Moderated School Assessment Highest Score in School Highest Score in external Upper Quartile in School Upper Quartile in external Median in School Lower Quartile in School Zero score in School Median in external Lower Quartile in external Zero score in external Typical Group Statistical Moderation Raw Raw Score Score (0-100) (0-100) Examination (0-120) External Score (0-100) Moderated Score 100 95 89 89 99 92 87 88 92 86 84 85 86 83 82 79 85 77 78 78 83 72 75 75 83 69 74 75 81 69 74 73 68 59 68 67 65 56 66 66 62 52 63 60 61 49 61 57 60 43 58 55 59 35 53 54 57 32 51 51 47 31 51 48 40 30 50 45 35 29 49 41 MAXIMUM UPPER QT. MEDIAN LOWER QT Capable Student in a Lower Performing Class Statistical Moderation RS RS RS (0-100) (0-100) (0-100) EXAM EXAM EXAM (0-120) (0-120) (0-120) EXT EXT EXT (0-100) (0-100) (0-100) MOD MOD MOD (0-100) (0-100) (0-100) 93 93 111 111 98 98 98 78 78 88 88 84 84 80 72 72 72 72 69 69 69 72 72 70 70 68 68 69 70 70 68 68 66 66 66 61 61 64 64 55 55 55 59 63 54 54 56 62 52 52 53 60 50 50 MAXIMUM MAXIMUM MAXIMUM MEDIAN MEDIAN MEDIAN Tie at the Top Statistical Moderation RS CRS 100 EXAM 120 EXT 100 MOD 100 100 119 100 100 100 119 100 100 CRS EXAM EXT MOD 98 120 100 100 98 119 100 100 98 119 100 100 CRS EXAM EXT MOD 100 119 100 88 100 81 82 88 100 81 82 88 Ranked Scores Report Statistical Moderation Schools with multiple classes undertaking the same VCE Unit 3/4 sequence, now have an additional resource available via a VASS report. This may be accessed via the following menu path: Results Admin VCE Reports School Scores By School (Ranked) The report will allow the teachers to compare the GA scores in a ranked format across the school. Partnerships and Statistical Moderation Statistical Moderation • Statistical Moderation works most effectively with groups larger than 5 • The VCAA requires that schools form partnerships • Schools who find it difficult to find a partner for any reason; lack of time / lack of access / lack of willingness can be reassured that the statistical moderation process is run as usual with the addition of precautionary checks VET and Statistical Moderation Statistical Moderation • The moderation group for VCE VET coursework scores is the RTO that has delivered the program for a group of students. • The group may be all the schools auspiced by the RTO Important Points to Remember about Stat Mod Statistical Moderation • Required so that school assessment can be used fairly as part of VCE assessment • Any adjustment to a student’s school-based assessment is made solely on the basis of the relationship between the groups schoolbased assessment and external scores for the group. • Does not change the rank order of students as determined by the school • Not influenced by students with negative anomalous results as they are excluded from the moderation process before moderation formulas are calculated • Students whose scores are excluded have their scores moderated using the formulas calculated for the rest of the group Statistical Moderation Report Statistical Moderation Statistical Moderation Report Statistical Moderation Statistical Moderation Report Statistical Moderation Statistical Moderation Report Statistical Moderation Statistical Moderation Report Statistical Moderation Report Statistical Moderation Study Score Calculation Stat Mod and Study Score • To be eligible for a study score a student must achieve ‘S’ results for units 3&4 in the one year, unless granted Interrupted Studies status. • Achieve two or more GA results in the study • A study score is calculated using the students’ final numeric score in each GA with each GA contributing a specified percentage • Number between 0-50 with mean 30 and standard deviation 7 • Study scores are distributed according to the properties of a normal distribution curve Study Score Calculation Stat Mod and Study Score Study Score Calculation • Calculate • Subtract • Divide • Multiply • Add Stat Mod and Study Score Study Score Example 1 Stat Mod and Study Score MAX. SCORE WEIGHTING MEAN STANDARD DEVIATION A+ GRADE SCALE GA1 100 25% 69.21 15.52 84 – 100 GA2 100 25% 70.78 14.95 85 – 100 GA3 120 50% 66.32 18.61 94 – 120 STUDENT A SCORE GRADE STANDARDISED SCORE WEIGHTED SCORE GA1 100 A+ 1.984716 0.496179 GA2 100 A+ 1.953861 0.488465 GA3 106 A+ 2.132657 1.066328 STUDENT B SCORE GRADE STANDARDISED SCORE WEIGHTED SCORE GA1 84 A+ 0.953468 0.238367 GA2 90 A+ 1.285180 0.321295 GA3 95 A+ 1.541442 0.770721 2.050972 = 47 1.330383 = 40 Study Score Example 2 Stat Mod and Study Score MAX. SCORE WEIGHTING MEAN STANDARD DEVIATION A+ GRADE SCALE GA1 100 25% 69.21 15.52 84 - 100 GA2 100 25% 70.78 14.95 85 – 100 GA3 120 50% 66.32 18.61 94 - 120 STUDENT C SCORE GRADE STANDARDISED SCORE WEIGHTED SCORE GA1 96 A+ 1.726904 0.431726 GA2 98 A+ 1.820124 0.455031 GA3 92 A 1.380202 0.690101 STUDENT D SCORE GRADE STANDARDISED SCORE WEIGHTED SCORE GA1 84 A+ 0.953468 0.238367 GA2 90 A+ 1.285180 0.321295 GA3 95 A+ 1.541442 0.770721 1.576858 = 42 1.330383 = 40 Distribution of Study Scores as a % Approximate proportion of students who achieve a study score on or above the stated values. STUDY SCORE ON/ABOVE ENGLISH 42,000 students THEATRE STUDIES 1,000 students 50 0.3% 126 3 45 2% 840 20 40 9% 3790 90 35 26% 10, 950 260 30 53% 22, 320 530 25 78% 32, 850 780 20 93% 39, 170 930 Things to Remember Stat Mod and Study Score • Indicates how the student performed against all other students who did the study in that year. • Calculated using the numeric results for the GA not the grade. • Moderated coursework scores are used when calculating study scores to ensure fairness. • A strong performance on one GA and a weak performance in another GA will not necessarily result in a high study score • Studies where there are less than 250 students are reviewed each year by a Study Score Review Panel. Contact Details Student Records and Results Nick Dobroff Phone : 9032 1743 dobroff.nick.n@edumail.vic.gov.au