Presentation Outline (a) Introduction (b) Enrolment Statistics (c) Enhanced Monitoring (d) Target Success Indicators (e) Progress - Examination Cycle (f) Progress Chart (g) Conclusion 2 BRIEFING ON EXAM AND ANA READINESS Introduction 3 Introduction • Public Examinations in South Africa is considered to be internationally comparable. • Significant progress made over the last nineteen years. • National Assessment is a fledgling system that is beginning to take root. • The DBE continuously strives to improve and enhance examination and assessment functionality. • Examinations and Assessment system is responsive to challenges and changes. 4 Change Factors (a) The need to adjust the examinations and assessment landscape to the environmental changes: - the need to improve learning outcomes - understanding the different purposes of assessment. - need to develop 21st Century Skills (b) The constant search for appropriate standards. (c) The need to strengthen initiatives leading to the development of a national examination and assessment system. (d) The need to identify the threats to examinations and assessment credibility and establish how best these risks can be mitigated. (e) The need to institutionalize Annual National Assessment so that regular and meaningful feedback is provided to the system. 5 State of Examinations and Assessment in the Republic Examinations (a) Well established and rigorous business exam processes. (b) National question papers of a high standard have contributed to improved teaching and learning. (c) Efficient administration of examinations. (d) Highly committed staff. (f) Significant strides made towards the establishment of a single national examination system. (g) SBA system that is showing signs of improvement. (h) Marking system that is steadily improving. 6 State of Examinations and Assessment in the Republic National Assessment (a) Significant progress in the administration of Annual National Assessment. Still a developing system. (b) Improved provincial capacity to manage both NSC exams and ANA. (c) Improved quality of data (d) Printing outsourced and done in-house. (e) Dichotomy of test design being explored and will be piloted. (f) Data capture at 85% level. (g) Data processing and release completed in record time. (h) Diagnostics have added a new dimension to teaching and learning 7 The Challenges (a) Question Papers: attaining international standards. (b) Marking : inter-marker reliability. (c) School based assessment: quality assessment at all levels. (d) Standards: Grade 10 and 11 (e) Assessment purpose: diagnostic versus systemic. (f) Feedback: effective data utilisation. (g) Trend Analysis: National versus International and Year to Year comparisons. 8 BRIEFING ON EXAM READINESS NATIONAL SENIOR CERTIFICATE 9 BRIEFING ON EXAM READINESS enrolment 10 Scope and Size of 2014 NSC Exams • • • • • • • • • • Total Candidates: Full Time Candidates: Part Time Candidates : Question Papers: Printed Question Papers: Scripts: Examination Centres: Invigilators: Markers: Marking Centres: 689 625 551 588 138 037 258 7.2 million 7.0 million 6 740 65 000 41 564 118 11 NSC Full-Time Enrolments per Province 2014 Province Name Total Entered 69 860 26 899 101 340 148 925 73 546 46 187 26 382 9 556 48 893 EASTERN CAPE FREE STATE GAUTENG KWAZULU-NATAL LIMPOPO MPUMALANGA NORTH WEST NORTHERN CAPE WESTERN CAPE 551 588 NATIONAL 12 12 13 13 NSC Full-Time Enrolments per Province 2010 to 2014 Province Entered 2010 Entered 2011 Entered 2012 Entered 2013 Entered 2014 Eastern Cape 68 294 68 174 69 199 74 998 69 860 Free State 28 448 26 510 24 760 28 019 26 899 Gauteng 94 642 87 631 91 475 99 504 101 340 130 302 127 053 132 272 150 154 148 925 Limpopo 95 869 74 669 78 182 83 594 73 546 Mpumalanga 54 542 49 592 48 989 51 206 46 187 North West 29 601 25 931 27 575 29 539 26 382 Northern Cape 10 406 10 426 9 281 10 693 9 556 Western Cape 47 062 559 166 41 273 45 602 511 259 14 527 335 48 783 576 490 48 893 KwaZulu-Natal National 551 588 14 15 15 NSC Enrolments - 2013 and 2014 comparison 2013 Province Full Time Part time Difference in FT 2014 Total Full Time Part time Total 2014-2013 Eastern Cape 74 998 20 536 95 534 69 860 21 359 91 219 -5 257 Free State 28 019 3 208 31 227 26 899 3 018 29 917 -558 Gauteng 99 504 41 408 140 912 101 340 42 355 143 695 1 860 KwaZulu-Natal 150 154 24 888 175 042 148 925 26 192 175 117 -1 029 Limpopo 83 594 16 993 100 587 73 546 19 673 93 219 -10 015 Mpumalanga 51 206 7 391 58 597 46 187 7 761 53 948 -4 968 North West 29 539 3 604 33 143 26 382 3 799 30 181 -3 152 Northern Cape 10 693 1 760 12 453 9 556 2 110 11 666 -1 014 Western Cape 48 783 10 858 59 641 48 893 11 770 60 663 213 Total 576 490 130 646 707 136 551 588 138 037 689 625 -23 920 16 17 NSC Full-Time Subject Enrolments 2014 Subject Accounting Afrikaans First Additional Language Agricultural Sciences Business Studies Economics English First Additional Language Geography History Life Sciences Mathematical Literacy Mathematics Physical Sciences Entered 129 352 83 817 80 476 212 630 141 339 444 689 241 921 118 758 291 317 319 191 232 432 172 259 18 NSC Full-Time Subject Enrolments 2010 to 2014 Entered 2010 Entered 2011 Entered 2012 165 522 140 849 137 527 Afrikaans First Additional Language 78 502 69 287 76 797 88 727 83 817 Agricultural Sciences 88 075 79 680 79 828 85 098 80 476 Business Studies 206 625 191 850 199 377 223 467 212 630 Economics 151 911 136 652 137 524 153 644 141 339 English First Additional Language 462 959 424 346 430 521 464 997 444 689 Geography 215 815 203 805 217 875 244 457 241 921 90 595 88 290 96 481 111 568 118 758 Life Sciences 292 865 270 540 283 611 307 303 291 317 Mathematical Literacy 288 370 281 613 297 074 330 789 319 191 Mathematics 270 598 229 371 230 194 245 663 232 432 Physical Sciences 210 168 184 052 182 083 19 259 187 170 172 Subject Accounting History Entered 2013 Entered 2014 148 374 129 352 Mathematics Full-Time Enrolments 280 000 270 598 270 000 260 000 250 000 245 663 240 000 230 000 229 371 230 194 2011 2012 232 432 220 000 210 000 200 000 2010 2013 2014 20 21 22 Male % 31 129 12 502 45 499 68 621 33 926 20 937 12 217 4 332 21 493 250 656 38 731 14 397 55 841 80 304 39 620 25 250 14 165 5 224 27 400 300 932 44.6 46.5 44.9 46.1 46.1 45.3 46.3 45.3 44.0 45.4 Female % Female EASTERN CAPE FREE STATE GAUTENG KWAZULU-NATAL LIMPOPO MPUMALANGA NORTH WEST NORTHERN CAPE WESTERN CAPE NATIONAL Male Province Enrolments NSC 2014 in terms of Gender 55.4 53.5 55.1 53.9 53.9 54.7 53.7 54.7 56.0 54.6 23 BRIEFING ON EXAM AND ANA READINESS enhanced monitoring approach Monitoring Approach (a) Key focus: building a credible, national examination and assessment system (b) Five pillars of the enhanced monitoring approach: - Development of Norms and Standards - Mediation of Norms and Standards - Coordination of examination processes - Monitoring and support of PEDs - Evaluation and feedback Monitoring Approach (c) Monitoring and support entailed the following: • identifying the risks. • collaborative review. • four-tiered approach. • part-time monitors. • state of readiness (d) Final State of Readiness visit in September 2014. Target Success Indicators a) High quality question papers are set and moderated. b) Examination centre and candidates are accurately registered on the examination computer system. c) Question papers are accurately printed. d) Question papers are distributed on time to all schools and candidates and strict security measures adhered to in the distribution process Target Success Indicators e) SBA is completed in accordance with policy prescripts and moderated. f) All examination and SBA marks are accurately captured. g) All examination irregularities are appropriately managed. h) All candidates are accurately and timeously resulted. i) Effective feedback provided to schools. j) All eligible candidates receive certificates. Target Success Indicators k) All candidates including learners with special needs are accommodated in the examination. l) Examinations are conducted under controlled conditions across all examination centres.. m) Answer scripts accurately marked and moderated. n) All examination marks are accurately captured on the examination computer system. BRIEFING ON EXAM READINESS registration Registration of Candidates/Centres a) All candidates have been registered on the Examination Computer System. b) Most PEDs registered candidates in Grade 10 and data is rolled over each year. c) Two schedule of entries have been sent to all schools to verify accuracy of registration data. d) Subject changes, immigrant candidates and special concessions have been appropriately managed. e) DBE to conducting an audit of the provincial registration data. Registration of Candidates/Centres f) All enhancements on the examination computer system will be completed by 30 September 2014. g) A complete dry run on the system will be commence by 30 September 2014. Dry run done in conjunction with Umalusi h) All independent centres have been evaluated to ensure that they satisfy the criteria for registration. i) In cases where there has been some doubt about the integrity of the centre, the examination will be administered by the PED, or will be closely monitored. Registration of Candidates/Centres k) Concessions granted to candidates with special needs and include: • Additional time • Amanuensis • Scribe l) Special extension on the SBA validity period granted by the Minister to repeat candidates, where the validity period had expired. BRIEFING ON EXAM READINESS question papers Question Papers a) All 258 question papers for November 2014 and March 2015 examinations have been set and moderated by the DBE to ensure a national standard b) Question papers have been approved by Umalusi, edited, quality assured and handed over to PEDS based on their printing plans. c) Adaptation of the question papers for sighted and deaf candidates is complete d) Brailing of the adapted papers for the blind is in progress blind, partially 36 Question Papers Changes based on CAPS: • Mathematics – two papers. • Economics – two papers. • All Home Language papers – one short text has been replaced with a long text. • History – one additional essay. • Consumer Studies – one additional question. Preparation for CAPS: • Exemplars developed for subjects with significant changes. • Examination Guidelines reviewed for all subjects. 37 Question Papers Other Changes: • Separate panels for Paper 1 and Paper 2 in Maths and Physical Science. • Fairness review of question papers. • Additional tier of editing 38 BRIEFING ON EXAM READINESS printing, packing and distribution Printing, Packing and Distribution a) Printing and packing progressing according to plan. For security reasons question papers made available to PEDS only a week before printing commences. b) Earliest that printing could commence was 1 August 2014. c) Storage facilities across all PEDs have been inspected and security has been improved at distribution points d) Question papers will be distributed to examination centres on the morning of the examination, except in the Western Cape. 40 Printing, Packing and Distribution e) In the Northern Cape, only in schools that are far from the districts, will question papers be stored at schools. f) Western Cape will deliver to schools in weekly consignments. 41 BRIEFING ON EXAM READINESS writing Writing of the examination a) Examination commences on 27 October 2014 and concludes on 28 November 2014. b) Invigilators who supervise the writing of exams are in the process of being trained across all PEDs. c) A common national Manual on Invigilation is used across all examination centres. d) To inculcate a sense of commitment to complying with the examination code of conduct, all learners will sign a Pledge on 17 October 2014, at a Pledge Signing Ceremony. 43 Writing of the examination e) The writing of the examinations will be monitored by DBE, PEDs and Umalusi. DBE has appointed 30 independent monitors who will be deployed to the provinces. f) Schools with previous irregularities will be closely monitored. 44 BRIEFING ON EXAM READINESS script collection Collection of Answer Scripts a) All PEDs have a clear control process for the management of Answer books. b) All scripts to be returned to the district office the same day. c) Scripts to enjoy the same attention and security as the question paper. d) Four PEDs will be using bar coded labels in the 2014 NSC examination. This will enable PEDs to scan scripts and ensure better script control. e) All scripts must be correctly labelled with centre number, subject and must be sealed at the school level. 46 Collection of Answer Scripts f) All scripts to be controlled and checked at each stage of the process. g) All PEDs will follow specific norm times for the return of scripts between the different collection points. BRIEFING ON EXAM READINESS marking Marking a) Markers have been appointed based on the PAM criteria and most PEDs have added learner performance as an additional criteria. b) National Marking Guideline discussions will be hosted in Pretoria for all subjects, to ensure standardisation of marking across all PEDs. c) Internal moderators to be appointed for each subject at each marking centre. d) DBE will appoint external moderators to evaluate the marking at each centre. 49 Tolerance Range • Tolerance range (TR) introduced for the first time to control differences in moderation. • Tolerance range (TR) is the agreed degree of deviation between the marked and moderated mark. • Tolerance range should not exceed 2 – 3% - at the total mark level and at the question level • TR exceeded: - in more than 50% of the scripts – remark batch. - in 50% of the scripts – moderate additional scripts. - less that 50% of the scripts - accept Authorisation of CMs and IMs • Chief markers (CM) and Internal moderators (IM) during the national training session will be evaluated on their compliance to the TR. • After multiple opportunities if the TR is not reached, then the CM or the IM will not be authorised. • Given that the CMs and IMs have already been appointed, their services cannot be terminated. • Therefore, DBE will arrange for support for these officials at the marking centre. Authorisation of Markers • Prior to the commencement of marking, each marker will be given a batch of 10 scripts to mark. • If 50% of the scripts are within the TR, marker is authorised to mark. • If less than 50% of the scripts are within the TR, the marker is given a second batch of scripts. • If the marker is still outside the TR, then he is called to a consultative meeting where his future involvement in the marking process is determined. • Need for a reserve pool of markers. Centralised Marking • Centralization of the small enrolment subjects, where the pooling of scripts from across the nine provinces and allocating the marking of the scripts from all nine provinces to one province, will be piloted in 2014. • The marking of these small subjects will serve as a springboard to experiment with other subjects with larger enrollments at a later stage. • A province that is allocated a specific subject/subjects will take full responsibility for the marking of the subject/s. This responsibility would entail: – – – – – – Appointment of markers, senior markers and chief marker and internal moderator Training of the markers Venue for marking Accommodation of the markers Management of the marking process Control and security of the scripts Centralised Marking - 2014 • Bilateral arrangement between PEDS where the number of scripts is less than 1000. • The DBE will coordinate and monitor the process nationally to ensure that there are no glitches. • No PED will be allowed to mark subjects where all the QA levels cannot be complied. • The DBE will mark centrally - Dance Studies and Agricultural Technology. • Centralised Marking of Deaf and Braille scripts by WCED and GDE BRIEFING ON EXAM READINESS school based assessment School Based Assessment (SBA) a) DBE conducted Audit of provincial SBA moderation systems focused moderation of assessment tasks in 18 districts across all PEDS, in July 2014. b) Second national moderation will be conducted in October 2014 and will evaluate the assessment tasks and the learner evidence. c) Provincial moderation to be completed by 16 October 2014. d) All SBA marks will be captured by 15 November 2014. 56 School Based Assessment (SBA) f) Common Assessment Task developed by the DBE was administered across PEDs, in Life Orientation, on 5 September 2014. g) National Marking Guideline discussions will be hosted in Pretoria for Life Orientation, to ensure standardisation of marking across all PEDs 57 BRIEFING ON EXAM READINESS processing, release and appeals Processing, Release and Appeals a) Marking will be completed by 14 December across all PEDs. b) Mark Capture will be completed by 18 December 2014. c) Proposals in respect of adjustments for the pre-standardization meeting – 20 December 2014. d) Pre-standardization meetings will be held on 21 & 22 December. e) Umalusi standardisation meeting to be held on 23 December 2014. f) Results will be thoroughly checked by DBE, PEDs and Umalusi from 23 – 30 December. g) Umalusi approval meeting to be held on 30 December 2014. 59 Processing, Release and Appeals h) Ministerial announcement on the 2014 NSC Results: 5 January 2015. i) Release of results to candidates by schools : 6 January 2014 j) Remark and re-check will take place early in February 2014 k) Candidates given 14 days after the release of the results, to apply for a re-mark or re-check. l) If candidates are still not satisfied, with the outcome, candidates will be allowed to view his/her script. m) Based on specific criteria, candidates are allowed to write the supplementary examination. 60 BRIEFING ON EXAM READINESS analysis and feedback Analysis and Feedback a) Detailed analysis of results per school, per district, per province, per subject will be made available. b) Qualitative analytical reports from chief markers/internal moderators will be consolidated for distribution to schools. c) Outcomes of the Umalusi standardisation meetings will be made available to curriculum specialists and examining panels. d) Workshops will be conducted with teachers and subject advisors. e) Under-performing schools will be brought to account and improvement plans will be designed. f) Curriculum specialists to develop subject improvement plans. BRIEFING ON EXAM READINESS certification Certification a) The issue of the certificate signifies the culmination of the examination cycle. b) The certificates for candidates that wrote the examination in 2013 was distributed to candidates at the end of June 2013. c) The 2014 certificates are scheduled to be released to candidates by end May 2015. BRIEFING ON EXAM READINESS irregularities Irregularities The serious irregularities experienced in 2013 and the action taken: (a) Dramatic Arts – question on rape. (b) Marker appointments in KZN. (c) Lost scripts (d) Copying (reduced) BRIEFING ON EXAM READINESS progress chart Progress Chart TARGET INDICATORS PROGRESS 1. High quality question papers are set and moderated 2. Examination centre and candidates are accurately registered on the examination computer system. 3. Question papers are accurately printed. In progress 4. Question papers are distributed on time to all schools and candidates and strict security measures adhered to in the distribution process All preparations in place 5. All candidates including learners with special needs are accommodated in the examination. 6. Examinations are conducted under controlled conditions across all examination centres. All preparations in place Progress Chart TARGET INDICATORS PROGRESS 7. Answer scripts accurately marked and moderated. Marker appointments being finalised; marking centres established. 8. All examination marks are accurately captured on the examination computer system. Data capturers being appointed; capture centres established 9. SBA is completed in accordance with policy prescripts and moderated. In progress 10. All examination and SBA marks are accurately captured. Data capturers being appointed; capture centres established. 11. All examination irregularities are appropriately managed. Structures and training in process 12. All candidates are accurately and timeously resulted. System enhancements being finalised 13. Effective feedback provided to schools. Diagnostic analysis 14. All eligible candidates receive certificates. Systems in place BRIEFING ON EXAM READINESS ANNUAL NATIONAL ASSESSMENT Introduction (a) DBE, working together with PEDs, will administer ANA tests on all learners in grades 1-6 & 9, plus a pilot study on representative samples of schools and learners at grade 7&8 levels from 16 to 19 September 2014 (b) Time table for the assessment was sent to PEDs and schools in the first quarter of the 2014 school year (c) Key systems for continuous monitoring of preparations and processes of ANA include bi-monthly meetings with Heads of Exams from PEDs and at least one monitoring visit to individual PEDs by the DBE before the tests are written (d) DBE 2014 visited all PEDs between May & June and the focus was on assessing the state of readiness for ANA 2014. Key elements of the ANA process 1. Test development & quality assurance 2. Registration of learners on the GET system. 3. Printing, packing and distribution of tests to schools 4. Administration of tests in schools 5. Marking and moderation of learner responses 6. Capturing, cleaning and analysis of data 7. Reporting 8. Monitoring of the processes - ongoing 9. Independent verification if the processes and outcomes of the assessment Test development & quality assurance • Language and mathematics tests for all the affected grades were developed by teams of selected educators and subject advisors • For quality assurance, separate teams moderated and edited the tests • Tests for grades 3, 6 & 9 were piloted in selected schools across the provinces in February 2014. • In addition, the tests for grades 3, 6 & 9 were submitted to experts who serve on the ANA Advisory Committee for comments and all received inputs were used to finalise the tests Test development & quality assurance • For the Foundation Phase all tests are versioned into the eleven official languages. • All tests were print-ready by end of May 2014 and adaptations for learners with special learning needs have been completed which include Brailling for learners who are blind, large font size for those who experience sight challenges and videos for learners who are deaf Registration of learners (a) Registration of learners includes capturing essential information that will be used in:• Determining the numbers of test booklets to print, pack and distribute per grade, subject and language • Preparing reporting templates so that results can be reported at national, provincial, district, school and individual learner levels (b) To date around 7 million learners have been registered on the GET platform Registered learners for ANA 2014 Prov. Total Number of learners Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 9 EC 1,043,043 183,372 167,624 156,199 147,452 133,595 126,375 128,426 FS 372,779 57,856 57,075 53,842 52,536 45,904 44,517 61,049 GT 1,270,418 197,264 211,414 195,193 186,866 169,975 159,142 150,564 KZN 1,583,300 272,753 245,297 230,632 220,965 198,261 195,754 219,638 LP 910,142 143,930 137,420 128,029 122,333 106,622 106,625 165,183 NC 172,308 28,909 26,277 25,005 25,241 23,313 22,087 21,476 NW 351,299 69,290 63,688 60,324 58,642 51,369 47,986 0 MP 557,963 96,866 87,994 81,888 78,535 70,032 68,131 74,517 WC 583,814 100,078 91,999 84,032 85,187 76,296 71,605 74,617 SA 6,845,066 1,150,318 1,088,788 1,015,144 977,757 875,367 842,222 895,470 Printing, packing & distribution • Eastern Cape, Mpumalanga, Northern Cape and Western Cape opted to print their tests using their existing facilities. Three of the PEDs have completed the printing and packing. Only Northern Cape is still busy with the printing. • Tests for Free State, Limpopo and North West are under print by CTP in Cape Town. KwaZulu-Natal & Gauteng test materials are printed by Lebone Litho Printers in Gauteng. Both service providers must complete distribution to Nodal Points by 12 September Administration of tests in schools • Tests will be written in all schools from 16 to 19 September • Principals will allocate teachers to invigilate classes that they do not teach • Independent schools that participate in ANA for purposes of being evaluated for subsidies will be invigilated by a district official • DBE will employ independent monitors to reinforce monitoring that will be done by DBE, PED & District officials Marking & moderation of scripts • Marking will be preceded by memo discussions at a national and district level • Teachers will mark scripts of their learners after writing • SMTs will moderate the marking before schools issue reports to parents • Samples of grade 3, 6 & 9 scripts from all schools will be moderated by appointed markers at centres managed by PEDs • Marking centres will provide feedback to schools on the findings of moderation Marking & moderation of scripts • All participating independent schools will be marked by appointed markers at the centres that have been selected and will be managed by the PEDs Capturing, cleaning & analysis of data • All marks and relevant assessment data per learner will be captured on existing IT systems such as SASAMS, CEMIS and others but will eventually be uploaded onto the national GET system. • Trained Systems Administrators will clean the data and prepare it for analysis Reporting • DBE professionals will compile reports – a national report that reflects provincial and district results & a diagnostic report that identifies the skills and knowledge that learners were able or not able to demonstrate in the assessment • The Minister will release the national report in December • The diagnostic report will be disseminated to schools in January so that teachers can plan how they will address the identified learning deficiencies in the new year Verification of ANA • For independent verification of all the processes as well as the outputs of ANA the DBE has appointed a service provider to conduct the verification. • Service provider will monitor the administration of the tests in a random sample of 125 schools per province during the week of 16 to 19 September • They will independently mark, process and report performance from the sampled schools at grades 3, 6 & 9 Conclusion • DBE, together with PEDs, have monitored the preparations for ANA 2014 through bi-monthly meetings. All reports from PEDs indicate that PEDS are ready for ANA 2014 • Provincial visits that were conducted in May/June by the DBE verified the state of readiness of each province • All schools have received the time table for ANA 2014 • Based on an evaluation of the key components of ANA, the system is ready for ANA 2014 Conclusion • The DBE and the PEDs are ready to administer the 2014 NSC examination and ANA 2014 • The DBE will vigorously pursue its agenda of building a national examination and assessment system, that is internationally comparable and credible. • DBE is harnessing the untapped potential of learner assessment. • Data analysis and data utilisation to enhance teaching and learning is the focus.