National Senior Certificate Examinations - 2012 Presentation to Portfolio Committee 12 February 2013 Presentation Outline 1. Introduction 2. Overall systemic improvements 3. Quality improvements 4. Lessons from Standardisation 5. Diagnostic subject analysis 6. Interventions 7. Qualification challenges 8. Umalusi Concerns 9. Conclusion 2 Introduction Introduction • The education enterprise is a complex activity which is multifaceted and dependent on a host of intervening variables. • The outcome of the NSC examination is the most important indicator of the performance of the system. • The NSC as a qualification has been implemented for the last five years • Therefore it is appropriate to reflect on the NSC: - as a qualification - learner performance Introduction • The national curriculum has established itself over the past five years: Teachers have adjusted to the national curriculum standards Remediation programmes have yielded steady improvements Improved allocation of resources to schools Ongoing provision of professional support to educators and administrators Established processes, policies and practices in the national examination and assessment system • Teachers, subject advisors and the examining panels have come to grips with the national curriculum and its associated assessment The Class of 2012 • Entered Grade 1 in 2001. • Born in the year when the new democratic dispensation was ushered in. • Education systems and processes were fairly stable. • Exposed to implementation of new curriculum Introduction Action Plan 2014, Towards Schooling 2025, sets specific targets: – Increase the number of learners eligible for bachelors study. – Increase the number of Grade 12 learners who pass Mathematics – Increase the number of learners who pass Physical Science Overall Systemic Improvements NSC performance: 2008 to 2012 9 % Achieved Difference % Achieved in 2011 Ranking % Achieved Total Achieved Total Wrote Province EASTERN CAPE 63 989 39 443 61.6 9 58.1 3.5 FREE STATE 24 265 19 676 81.1 3 75.7 5.4 GAUTENG 89 627 75 214 83.9 1 81.1 2.8 127 253 93 003 73.1 6 68.1 5.0 LIMPOPO 77 360 51 745 66.9 8 63.9 3.0 MPUMALANGA 47 889 33 504 70.0 7 64.8 5.2 NORTH WEST 27 174 21 609 79.5 4 77.8 1.7 NORTHERN CAPE 8 925 6 661 74.6 5 68.8 5.9 WESTERN CAPE 44 670 36 974 82.8 2 82.9 -0.1 511 152 377 829 73.9 70.2 3.7 KWAZULU-NATAL NATIONAL 2012 NSC passes by type of qualification Province Total Wrote Achieved % achieved % Achieved % Achieved % Bachelor achieved Diploma achieved Higher achieved NSC achieved Bachelor Diploma Certificate Higher NSC Certificate EC 63 989 11 246 17.6 16 148 25.2 11 998 18.8 51 0.1 FS 24 265 6 937 28.6 8 553 35.2 4 181 17.2 5 0.0 GP 89 627 32 449 36.2 30 422 33.9 12 335 13.8 8 0.0 KZN 127 253 34 779 27.3 36 841 29.0 21 274 16.7 109 0.1 LP 77 360 15 324 19.8 20 103 26.0 16 301 21.1 17 0.0 MP 47 889 9 495 19.8 14 277 29.8 9 633 20.1 99 0.2 NW 27 174 7 445 27.4 9 151 33.7 5 010 18.4 3 0.0 NC 8 925 2 055 23.0 2 787 31.2 1 819 20.4 0 0.0 WC 44 670 16 317 36.5 14 599 32.7 6 053 13.6 5 0.0 26.6 152 881 29.9 88 604 17.3 297 0.1 National 511 152 136 047 Admission to Bachelor Studies (target of 32%) 12 Proportion of 22-25 year-olds with matric since 2002 50% 45% 40% 35% 30% 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: STATS SA General Household Surveys, 2002- 2011 Achievement - Bachelor Studies • Sector on track to achieve target for 32% of Grade 12s able to be admitted for Bachelor degree studies (currently at 26,6% in 2012 from 24.3% in 2011 and 19.9% in 2008) – Over 15, 000 increase year on year – 2011 was low for historical reasons so growth in improvements of the order of 10% anticipated Achievement Status per Quintile Achievement Status Quintile Quintile Quintile Quintile Quintile Quintile Quintile 1 2 3 4 5 99 None Totals Achieved Bachelor 16 296 19 752 25 252 18 497 46 795 9 253 202 136 047 Achieved Diploma 26 885 30 251 34 476 24 179 29 851 7 113 126 152 881 Achieved H-Cert 21 208 22 504 22 083 11 660 8 052 3 067 30 88 604 167 72 39 8 0 11 64 556 Quintiles:72 579 81 850 54 344 84 698 19 444 Achieved NSC Total Achieved 1 to 5 99 None 0 297 358 377 829 Lowest to highest ranked Independent/Special/Private Schools No Quintile not available 15 Number of candidates in schools per percentage interval per quintile % Interval (Schools) Exactly 0% Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Total 37 9 56 0 0 102 No with 0 to 19.9% 1 671 2 142 920 67 168 4 968 No with 20 to 39.9% 9 731 7 688 6 285 1 302 774 25 780 No with 40 to 59.9% 23 828 23 497 22 565 7 389 2 268 79 547 No with 60 to 79.9% 35 937 44 762 46 929 28 662 12 215 168 505 No with 80 to 100% 26 671 29 879 39 065 32 903 77 971 206 489 1 227 1 036 1 953 1 208 99 102 109 013 117 773 71 531 Exactly 100% Total 14 635 20 059 108 031 505 450 16 Candidates’ performance in selected subjects 2008 – 2012 (at 30% level) 17 Number of Districts per Percentage Interval – 2011 / 2012 Province Total number of Districts Below 50% 50% to 59.9% 60% to 69.9% 70% to 79.9% 80% and above 2011 2012 2011 2012 2011 2012 2011 2012 2011 2012 23 5 3 6 7 8 8 4 5 0 0 5 0 0 0 0 0 0 4 0 1 5 Gauteng 15 0 0 0 0 1 0 4 3 10 12 KwaZulu-Natal 12 0 0 1 0 7 3 4 9 0 0 Limpopo 5 0 0 1 0 3 4 1 1 0 0 Mpumalanga 4 0 0 1 0 2 2 1 2 0 0 North West 4 0 0 0 0 0 0 2 2 2 2 Northern Cape 5 0 0 1 0 1 1 2 2 1 2 Western Cape 8 0 0 0 0 0 0 1 1 7 7 81 5 3 10 7 22 18 23 25 21 28 Eastern Cape Free State Total 18 Quality Improvements Quality Improvements Comparing the achievements of 2012 to that of 2011, the following is a confirmation of the improvement in quality: (a) 15 280 more bachelor passes (b) Nine of the gateway subjects improved at 30% achievement level. (c) Eight of the gateway subjects improved at the 40% achievement level. (d) 17 937 more candidates passed Mathematics (at 30%) (e) 13 175 more candidates passed Mathematics (at 40%) (f) 13 477 more candidates passed Physical Science (at 30%) (g) 8 967 more candidates passed Physical Science (at 40%) 20 Quality Improvements Comparing the achievements of 2012 to that of 2011, the following is a confirmation of the improvement in quality: (h) Increase in the distinction rate across key subjects. (i) Of the 81 districts, only three performed below 50% (5 in 2011) (j) Number of districts performing above 80% increased from 21 to 28. (k) Gini coefficient decreased from 0.192 to 0.169. 21 Lessons from Standardisation Standardisation of Results • Ensures that the cohort of learners is not advantaged or disadvantaged by fluctuations of results. • Mitigates fluctuations of learner performance as a result of factors other than learner performance. • Standardisation is based on comparing the performance of the current cohort with previous cohorts. Therefore, standardisation may reveal problems relating to: o the standard of the question paper. o the standard of the marking. o poor subject performance. Standardisation of Results Year Subjects Raw marks Adjusted Adjusted downwards upwards 2010 58 39 10 9 2011 56 45 8 3 2012 61 44 12 5 Lessons • • Acceptance of the majority of the marks based on raw marks, points to the setting of question papers of the appropriate standard and confirms that the marking standards are in keeping with previous years. Downward adjustments based on the premise that: • the question paper was possibly less challenging than previous years; or the improvement in the performance of candidates is justified, but too high for one year. Upward adjustments based on the premise that: the paper could have been more challenging than in previous years. learner performance of this cohort has deviated from previous years due to poor teaching and learning. Lessons Downward Adjustments (12 subjects): • Accounting, Physical Sciences, Geography, Tourism, Design, Dramatic Arts, Civil technology, Electrical Technology, English First Additional Language, Xitsonga Home Language, Tshivenda Home language and Sesotho First Additional Language. • These subjects will be closely scrutinised by the examiners to ensure that the appropriate standard is set in 2013. • In terms of NSLA, this is a confirmation that the NSLA strategy is beginning to bear fruit in these subjects, and PEDs should be encouraged to continue with the interventions. Lessons • DBE will explore and develop a method of establishing the impact of these interventions so that the required evidence can be provided to Umalusi to inform the standardisation process and avoid the downward adjustment. Lessons Upward Adjustments (5 subjects) • Life Sciences, English Home language, Afrikaans Home language, Isizulu Home Language and Mathematical Literacy. • The examining panels will conduct a review of these papers to confirm if there were indeed of a higher standard • These 5 subjects must be the focus of the NSLA in the current year, so as to improve performance in these subjects Diagnostic subject Analysis Purpose of Subject Analysis • Provides feedback to learning and teaching and assists in school based diagnostic assessment and intervention • Communicate to teachers and subject advisors the common areas of weakness in the subject and ensures that these areas are constructively addressed in the teaching programme planned for 2013 30 Scope and Approach • The report covers 11 of the subjects with high enrolments namely: Accounting, Mathematics, Physical Sciences, Agricultural Science, Business Studies, Economics, English First Additional Language, Geography, History, Life Sciences and Mathematical Literacy. • Subject analysis based on: • input provided by markers; • analysis of a selected sample of scripts 31 Generic Improvements • Improvement in the quality of answers provided, which confirms that improvement strategies are making a difference. • Gradual consistent improvement in the pass rate in each of the subjects analysed both at the 30% and 40% achievement levels. • Fewer candidates scoring between 0-29% in most of the subjects. • Improved performance in Accounting and Physical Science was observed. Possibly due to an increase in the number of candidates who offered these subjects in combination with Mathematics Generic Shortcomings • Inadequacies relating to foundational knowledge and skills; and basic concepts and principles which negatively impact on advanced learning and understanding. • Inadequate numerical skills required to do simple calculations in Mathematics as well as other subjects where numerical skills were assessed. • Lack of linguistic skills: lack of appropriate language skills which are required to interpret questions and write proper paragraphs. Text based questions - poorly answered. • Failure to apply knowledge and skills. Questions requiring application, higher order thinking skills like problem solving, analysis and evaluation registered the worst performance across all the subjects analysed. • Inadequate preparedness of candidates for the examination, which manifested as a lack of mastery of content 33 Mathematics There was a decrease in the percentage of candidates who performed at 0-29% in 2012 compared to 2011; An improvement in the number of candidates who performed at each of the categories from 40% and above. 25.0 20.0 Percentage 15.0 10.0 5.0 0.0 0 TO 9 10 TO 19 20 TO 29 30 TO 39 40 TO 49 50 TO 59 60 TO 69 70 TO 79 80 TO 89 90 TO 100 2011 9.5 23.1 20.0 17.2 11.6 7.6 5.1 3.4 1.9 0.6 2012 8.3 18.8 18.9 18.3 13.0 9.2 6.4 4.1 2.2 0.7 35 Maths P1: Average marks per question expressed as a percentage Question Content assessed Q1 Algebra, equations and inequalities Q2 Patterns and sequences Q3 Patterns and sequences Q4 Functions and graphs Q5 Functions and graphs Q6 Functions and graphs Q7 Annuities and finance Q8 Calculus Q9 Calculus Q10 Calculus Q11 Linear Programming 36 Maths P2: Average marks per question expressed as a percentage Question Content assessed Q1 Data handling Q2 Data handling Q3 Data handling Q4 Data handling Q5 Analytical geometry Q6 Analytical geometry Q7 Transformation geometry Q8 Trigonometry Q9 Trigonometry Q10 Trigonometry functions Q11 Trigonometry Q12 Trigonometry Q13 Transformation geometry 37 Key areas of Concern Proposed Interventions Poor understanding of the basics and foundational competencies taught in earlier grades such as Algebraic manipulations, factorisation, solutions of equations and inequalities. Focus on knowledge, skill and concepts taught in earlier grades. Over dependence and focus on previous papers without proper grounding and understanding of the concepts. Candidates therefore regurgitate answers to similar questions from previous papers. Q 10 on calculus, and Q 5 on functions and graphs in P1; Q 10 on Trigonometric functions in P2 registered the worst performance of 24% average nationally . Learners should be first taught the concepts and question papers should be utilized for practice. Expose learners to non-routine questions more regularly. It is not possible to predict the nature of problem solving questions that will be assessed year on year, therefore regular exposure to non routine questions will assist in developing learners problem solving thinking skills. Learners should be exposed to a variety of questions, not just ‘stimulus-response’ questions, and not limited to those appearing in past question papers The following topics: Analytical geometry, trigonometric functions, calculus, functions 38 and graphs need focused attention Accounting In 2012, there was a decrease in the percentage of candidates who performed at 0-29% compared to 2011. An improvement in the number of candidates who performed at each of the categories from 40% and above. 30.0 25.0 Percentage 20.0 15.0 10.0 5.0 0.0 0 TO 9 10 TO 19 20 TO 29 30 TO 39 40 TO 49 50 TO 59 60 TO 69 70 TO 79 80 TO 89 90 TO 100 2011 0.2 10.5 26.1 27.3 16.0 8.4 4.7 3.0 2.3 1.5 2012 0.6 12.1 21.8 22.9 16.5 10.5 6.6 4.2 3.1 1.8 40 Average marks per question expressed as a percentage Question Content assessed Q1 Inventories and VAT Q2 Bank reconciliation and debtors’ age analysis Q3 Manufacturing Q4 Company financial statements and fixed assets Q5 Interpretation of company information Q6 Projected income statement and internal control 41 Key areas of Concern Proposed Interventions Lack of proper understanding of basic accounting concepts, basic formats for cash flow and balance sheets Knowledge and understanding of the basic formats used in Accounting forms the basis of all Accounting and should be focused on by teachers. Basic theory and basic formats possibly not covered or not well taught, as candidates responses lacked mastery Curriculum coverage needs to be closely monitored Question 2 on Bank Reconciliation and debtors analysis registered the least average performance (this topic is taught in grade 11 ) Revision of relevant Grades 10 and 11 content required for examinations in grade 12 Q6 on projected income statement and internal control also noted poor performance. Preparation of cash budgets, projected income statements, and interpretation thereof needs focus Physical Sciences General improvement in performance in Physical Science in 2012 compared to 2011 . 25.0 20.0 Percentage 15.0 10.0 5.0 0.0 0 TO 9 10 TO 19 20 TO 29 30 TO 39 40 TO 49 50 TO 59 60 TO 69 70 TO 79 80 TO 89 90 TO 100 2011 9.5 23.1 20.0 17.2 11.6 7.6 5.1 3.4 1.9 0.6 2012 8.3 18.8 18.9 18.3 13.0 9.2 6.4 4.1 2.2 0.7 44 Physical Sc. P1: Average marks in each question expressed as a percentage Question Content assessed Q1 One word items Q2 Multiple choice questions Q3 Vertical projectile motion Q4 Momentum and relative velocity Q5 Work and energy Q6 Doppler Effect Q7 2D and 3D wave fronts Q8 Parallel plate capacitors and electric fields Q9 Electric Circuits Q10 AC circuits Q11 Photo-electric effect 45 Physical Sc . P2: Average marks per question expressed as a percentage Question Content assessed Q1 One-Word Items Q2 Multiple Choice Q3 Nonmenclature of organic compounds Q4 Physical properties of organic compounds Q5 Reactions of organic compounds Q6 Rate of reaction Q7 Chemical equilibrium Q8 Galvanic cells Q9 Electrolyctic cells Q10 Batteries Q11 Fertilisers 46 Key areas of Concern Proposed Interventions Some centres performed really well and others displayed a serious lack of knowledge which suggests that candidates may not have been taught certain portions of the content Knowledge and understanding of the concepts in the subject Lack of knowledge of basic definitions. Lack of exposure to practical work was evident in the nature of responses provided by a number of candidates on questions based on practical work. Rate of reaction (P2), and diffraction and interference (2D and 3 D wave fronts ) in P1 need greater attention in 2013. Syllabus completion must be monitored closely Poor Mathematical skills relating to reading and interpretation of graphs, handling of calculators. The worst performance in P1 was in Q7 based on 2D and 3D wave fronts (40.1% average score) and in Q6 in paper 2 on rates of reactions (33.2%) 47 Life Sciences Life Sc registered a drop in the pass rate. There was an increase in the percentage of candidates who performed at 0 - 29% in 2012, compared to 2011, a drop in the number of candidates between 30% to 59% level. 30.0 25.0 Percentage 20.0 15.0 10.0 5.0 0.0 0 TO 9 10 TO 19 20 TO 29 30 TO 39 40 TO 49 50 TO 59 60 TO 69 70 TO 79 80 TO 89 90 TO 100 2011 0.0 5.0 20.8 28.0 21.2 12.5 6.6 3.7 1.9 0.3 2012 0.3 8.1 22.1 26.2 18.5 11.7 7.0 4.0 2.0 0.3 49 Average performance per question in Life Sciences Paper 1 Question Content assessed Q1 Multiple choice, matching, terminology and short questions Q2 Meiosis, protein synthesis and monohybrid cross Q3 Pedigree diagram, DNA and genetic modification Q4 Human evolution, phylogenetic tree, natural and artificial selection 50 Average marks achieved per question as expressed as a percentage: P2 Question Content Assessed Q1 Multiple choice, matching, terminology and short questions Q2 Brain, Eye, Reproduction, Temperature regulation Q3 Population growth form, human population growth and age-gender pyramids Q4 Culling, competition, adrenalin and reflex action 51 Key areas of Concern Proposed Interventions Limited knowledge of basic terminology and concepts. Emphasis on learning of terminology. Glossary of biological terminology can be provided Lack of associated Mathematics skills required to draw graphs e.g. pie charts. Teachers need to pay attention to skills relating to drawing graphs, interpreting graphs, making conclusions from data (both P1 and P2) Inappropriate interpretation of questions Syllabus completion needs to be ensured. No aspect of the syllabus should be omitted since all aspects are assessed. 52 Interventions Generic Interventions • Districts and schools will continue to guide learners in making the right subject choices and combinations • Subject advisors will convene workshops with teachers, where these reports could be thoroughly discussed and mediated, and problem areas addressed with teachers. • Monitoring of the utilisation of the Diagnostic Reports. • Monitoring of syllabus completion • Focused interventions targeting areas of weaknesses highlighted in the report combined with general systemic interventions Assessment Interventions • Improving the Quality of School Based Assessment • Development of quality assessment items (bank of items for SBA) • Publication of exemplars of learner responses to questions in selected subjects (Geography, History, Eng. FAL P3, Economics) • Preparation for NSC 2014 based on assessment as outlined in CAPs Advocacy Development of exemplar question papers Development of examination syllabi/guidelines in all subjects (Gd. 12) Teacher development • Teacher support will be provided in the following ways: – Develop study guides to assist teachers /learners in areas needed – Develop sample lessons on sections where teacher content knowledge has been found to be deficient – Focused training of teachers on specific content – Monitor and support training and implementation of difficult content • Learner support material – Development and distribution of Mind the gap series in certain key subjects on specific content areas with examination practice questions Learner support • Development and distribution of Mind the gap series in certain key subjects on specific content areas with examination practice questions • Provision of self study guides and past question papers for revision Curriculum Coverage • Monitoring tool to track curriculum coverage at school level has been developed. • Monitoring tool will be administered by managers at the various levels in the system. • Reporting on a quarterly basis to the PED and DBE. • DBE to conduct sample verification visits to authenticate the reports Qualification Challenges Pass Requirement – 30% • Public perception that the pass requirements for the NSC are pegged at a low level • This is based on mis-information on the full package relating to pass requirements. • NSC serves as a school leaving certificate, as well as a certificate for admission to higher education – hence the different categories of NSC passes Mathematics/ Maths Literacy • • • Increasing percentage of learners are migrating to Mathematics Literacy. Maths Literacy is selected as a soft option but not recognised for admission to University studies in certain faculties. Issues: - is the Maths/Maths Literacy divide useful in promoting Mathematics in the country. - what is the purpose of Maths Lit and is this being achieved. - what control measures need to be instituted to curb gate-keeping by school principals. Trends in Matric Mathematics since 2008 Numbers wrote maths Number passed maths Proportion Maths pass taking rate maths Proportion passing maths 2008 298821 136503 45.70% 56.10% 25.60% 2009 290407 133505 46.00% 52.60% 24.20% 2010 263034 124749 47.40% 48.80% 23.20% 2011 224635 104033 46.30% 45.30% 21.00% 2012 225874 121970 54.00% 44.19% 23.86% Life Orientation • • • • • Life Orientation focuses on the development of moral, civic, social and physical skills. Assessment is school based (100%) Given the inflated performance of learners, LO is not recognised for University admission. DBE piloted a Common Assessment Task (CAT) developed nationally, in 2012. Issues: - Should the assessment of LO be changed to incorporate an external examination? - Does LO lend itself to external examinations? Life Orientation • Issues: - Should LO be retained as a NSC subject? - What should be done to improve the status of LO. School Based Assessment • SBA still forms 25 % of final mark • Quality of SBA tasks remains an area of concern • How do we improve capacity of teachers to set good quality SBA tasks? • Utilisation of SBA as a means of driving performance Ministerial Committee • Ministerial Committee will be constituted and will commence with their research by April 2013. • Committee will comprise of a Technical Research Team and a broader Reference Group. • Preliminary report will presented to the Minister by July 2013. Umalusi Concerns Question Papers (a) Non adherence to time frames and the 18 month cycle • Eighteen (18) month cycle to be progressively implemented (2013 – 2014). • In 2013, Umalusi prescribed timeline of first moderation April 2013 and final moderation by July 2013, will be adhered. • The panel for 2014 has been appointed and will commence with the setting process in April 2013 (well ahead of schedule) • Separate panel has been appointed for each paper, per subject to reduce time required for setting and to improve quality (b) Competency of examiners/capacity of panels • Rigorous recruitment process has been concluded. Panels have been reviewed for 2013 / 2014 • Examiners subjected to a competency test based on subject knowledge and assessment skills. • Performance of examiners is evaluated on an annual basis. Appointment extended based on performance status. • Intensive training session scheduled for March 2013 – focus on question taxonomies. SBA and Life Orientation: (a) Poor quality and standard of school based assessment tasks • Exemplar assessment tasks will be developed and distributed to schools (for both SBA and LO). • Common Assessment Task (CAT) for Life Orientation to be administered in September 2013. • DBE will conduct a national moderation of the June and Preparatory examinations in seven (7) key subjects. • Moderation systems at school, district and provincial levels will be rigorously monitored by the DBE. SBA and Life Orientation • The national panel of moderators will be deployed to selected districts to conduct moderation of SBA and LO • Exemplars of learner evidence (responses) from the 2012 NSC examination will be published with the marking comments and awarded scores. Marking (a) Marker competency: • Policy and Regulations on the implementation of competency tests for markers will be approved by the Minister and published by March 2013. • Will be implemented in seven (7) key subjects and will form part of the criteria utilized for the appointment of markers. • Early administration will allow for a training programme to be implemented prior to the marking process. Marking • Report on the outcome of the re-mark results to be shared with each of the PEDs. • Assist in identifying subjects with significant variations. • Markers will be brought to account where there are major changes. • DBE to conduct random audit of markers appointed by PEDs. • More intensive moderation of marking by PEDs and DBE. Administration of Examinations • Examination processes across all PEDs will be standardised through the use of standard operating procedures, outlined in Examination Process Manuals . • Security across the whole exam system will be enhanced through the declaration of examinations as a national priority by NATJOC. • Printing quality and accuracy will be given attention through intensive monitoring and ensuring compliance to control processes. Conclusion • The data emanating from NSC 2012 and prior, has been analysed closely to ensure that important trends are identified and acted upon. • The Ministerial Committee which is in the process of being constituted by the Minister will address the curriculum challenges that are emanating from the five year review. • The Diagnostic Report which focuses on the key subjects must be discussed with all subject advisors and teachers so that these shortcomings are decisively addressed in 2013. • 75