An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams by Bernard Rivers Published on 26 February 2015 at www.BernardRivers.com/exam-analysis Contents A: Executive Summary ......................................................................................................................... 2 B: Introduction ...................................................................................................................................... 3 C: What this analysis does not look into – performance by college ....................................................... 3 D: Main findings: Increases over time, and significant variations among Triposes, in percentages of students awarded Firsts and Upper-Seconds ................................................................................... 4 E: Norm referencing ............................................................................................................................. 9 F: Comparison of Tripos Classes with ‘A Level’ performance ............................................................. 10 G: Comparison of Cambridge with other Russell Group universities ................................................... 11 H: The General Board’s Education Committee ................................................................................... 12 I: Issues arising ................................................................................................................................. 12 J: Recommendations ......................................................................................................................... 13 K: Link to Tripos-specific memos and data ......................................................................................... 14 L: Open discussion ............................................................................................................................ 14 M: Future plans ................................................................................................................................... 14 N: Data and sources ........................................................................................................................... 14 O: Postscript regarding work done in 1969 ......................................................................................... 15 Tripos-specific index.............................................................................................................................. 16 Bernard Rivers is a retired British economist and a Cambridge University graduate. During 2013-2014 he was a visiting fellow at Cambridge. From 2002 to 2012 he was Executive Director of Aidspan, an international non-governmental organisation he founded in New York and later moved to Nairobi. Previously, he founded and ran a software company in New York. He divides his time between Kenya and the USA. Bernard Rivers (Bernard.Rivers@gmail.com and www.BernardRivers.com) PO Box 5295-80401, Diani Beach, Kenya. +254-736-280702 PO Box 54, Cornwallville, NY 12418, USA. +1-518-239-4598 An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 1 A: Executive Summary During the period from 1960 to 2014, the percentage of Cambridge University undergraduates awarded a First Class Honours degree increased from 10% to 24%, and the percentage awarded a First or an Upper-Second increased from 34% to 78%. It is not clear to what extent this increase is a manifestation of “grade inflation” (a term that implies a lowering of standards) and to what extent it is caused by a steady increase in the calibre and diligence of the students, and/or in the quality of the teaching. An analysis that I have conducted also reveals that the percentages of students awarded Firsts and Upper-Seconds vary significantly among the different Triposes (i.e. subject areas). My findings, which are based entirely on public-domain data, include the following: Between 2000 and 2014 these growths were most marked among Arts Triposes. During that period, the percentage of Part II (i.e. final-year) Arts students awarded a First increased from 17% to 30%, and the percentage of such students awarded a First or an Upper-Second increased from 84% to 94%. No significant growth took place in the average across the Sciences Triposes, though it did within one or two individual Sciences Triposes. For Part II students between 2000 and 2014, examiners for the Law Tripos awarded a First to an average of just 17% of their students; examiners for the Mathematics Tripos awarded a First to an average of 32% of their students; and examiners for the Education Tripos steadily increased the percentage of students to whom they awarded a First from 11% to 47%. Meanwhile, examiners for Part II students in the Chemical Engineering Tripos awarded a First or an UpperSecond to an average of just 69% of their students; examiners for the History Tripos awarded a First or an Upper-Second to an average of 97% of their students; and examiners for the English Tripos steadily increased the percentage of students to whom they awarded a First or an UpperSecond from 75% to 97%. In 2014, not one of the 50 Part II students in the Land Economy Tripos and only one of the 149 Part II students in the Modern and Medieval Languages (MML) Tripos was awarded a Lower-Second or a Third. Yet that same year, a Lower-Second or a Third was awarded to 26% of the Part II Mathematics and Engineering students. And of the 875 students who took Part II Theological and Religious Studies between 2000 and 2014, not one was awarded a Third. These findings are not explainable solely by variations between Triposes in the entry qualifications of students. For example, the percentage of Mathematics students who arrived at Cambridge in 2011-2013 with a Universities and Colleges Admission Service (UCAS) Tariff score of at least 600 points was 83% while the percentage of History students who arrived with such a score was only 39% – but the percentage of final-year students awarded a First over the years 2012-2014 was higher for History than for Mathematics. Between 2002 and 2012, Cambridge University awarded Firsts and Upper-Seconds to somewhat lower percentages of final-year students than did Oxford University, but to somewhat higher percentages of final-year students than did the other 22 Russell Group universities collectively. These findings raise questions about what is the true meaning, or value, of a First or an Upper-Second. It appears that nobody has previously conducted a full analysis of the data. It also appears that Cambridge University’s central administration has never pushed for a consistent approach among Triposes to the assignment of classes. Accordingly, I recommend that the university’s faculties and departments discuss this issue with each other and with the central administration, and then make appropriate changes in policy and practice. I recommend that these changes primarily involve norm referencing (i.e. pre-determining the percentages of Firsts and Upper-Seconds that will be awarded), a practice currently followed only with a small number of the Triposes. More specifically, I recommend that for each Tripos, whatever the recent percentages of students awarded Firsts or Upper-Seconds in that Tripos, the percentage of students awarded Firsts and UpperSeconds should not exceed the university-wide averages that have pertained over the past ten years. This means that (i) for first- and second-year students, no more than 20% of those taking each Tripos should be awarded a First, and no more than 75% should be awarded a First or Upper-Second; and (ii) for final-year students, no more than 25% of those taking each Tripos should be awarded a First, and no more than 85% should be awarded a First or Upper-Second. An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 2 B: Introduction During 2013-14, after an absence of over 40 years, I spent a year at Queens’ College, Cambridge, this time not as an undergraduate but as the equivalent of a visiting fellow. When I arrived I was struck by how many Firsts and Upper-Seconds and how few Lower-Seconds and Thirds were being awarded across the university as compared with when I was an undergraduate. This got me wondering about how extensive and persistent such changes have been and what might be the reasons. One reason I’m interested in this topic is that in 1969 I served on an Economics faculty staff-student committee that studied the extent to which two independent markers assign different grades to the same exam scripts, and that reviewed whether students should have the option to submit dissertations in place of one or two exams. When some of our recommendations were, after minor modifications, accepted by the faculty and students but not by the university, there was a major student protest and a consequent inquiry by Lord Devlin. (See Postscript, below, for more information.) The official research project I conducted during my recent time at Queens’ was on a completely different topic, and I extend my sincere thanks to the college for hosting me while I worked on that project. But because of my interest in the data and politics of exams, and because it appeared that nobody had previously studied the subject in depth, I decided that in my spare time I would build a database of Cambridge University exam results (making use only of public-domain data) and would then conduct an analysis. Most of that work was conducted while I was in residence at Queens’, but I did not seek endorsement from Queens’, from the individual faculties, or from the university, and they bear no responsibility whatever for the analysis. The same applies to HESA and Unistats, discussed below. Background note for those unfamiliar with the Cambridge system: Each Cambridge undergraduate normally spends three years studying for a “Tripos” (i.e. a subject area) and attending lectures provided by one or more faculties or departments. At the end of each year, the student takes several three-hour examinations. Based upon his/her performance in those exams, possibly supplemented by his/her performance in some practical tests and/or a short dissertation, nearly every student is placed in “Class I” (the best), “Class II Division 1”, “Class II Division 2”, or “Class III”. Colloquially, this is known as earning a “First” (i.e. First Class Honours), an “Upper-Second”, a “Lower-Second”, or a “Third”. A very small number of students are classified in some other way, including being deemed to have failed. The class awarded each student takes no account of his/her performance in classwork and in weekly supervisions through the year. The student is classed each year; no aggregate class is provided based on performance over all three years. During the student’s first year, a few Triposes provide “unclassified” exams, in which students simply pass or fail, and a few Triposes do not divide the Second Class degree into an Upper-Second and Lower-Second. C: What this analysis does not look into – performance by college Two Cambridge graduates, Martin Baxter and Peter Tompkins, have for many years independently produced annual tabulations of Cambridge exam results. Baxter’s tables are not published; they are privately circulated on a restricted basis within the university. Tompkins’ tables are drawn upon primarily by the Independent newspaper for occasional articles on the topic. Both Baxter and Tompkins seek to quantitatively compare the exam performance of students in different colleges. The methodology used by both Baxter and Tompkins is to assign five points for a First, three for an Upper-Second, two for a Lower-Second, and one for a Third, and then to report the average “score” of students within each college, subject by subject and across all subjects. Baxter and Tompkins each then use further tweaks to that methodology. While the approach of Baxter and Tompkins focuses on colleges, my approach focuses on faculties, departments and Triposes, because these are the controlling entities regarding the assignment of exam classes. A particular college might select for admission or for financial support students who it believes may perform particularly well in the exams, or it might give its students particularly effective academic support; but it cannot actually determine or control the classes given to individual students. An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 3 D: Main findings: Increases over time, and significant variations among Triposes, in percentages of students awarded Firsts and Upper-Seconds Finding 1: There have been significant increases over time in the percentage of Cambridge students awarded Firsts and Upper-Seconds. Between 1960 and 2014, as shown below, the percentage of all students across the university awarded a First in their first-, second- or final-year exams increased from 10% to 24%. And the percentage across the university awarded a First or an Upper-Second increased from 34% to 78%. (The term Percentage Increases is used to describe this phenomenon.) Chart 1: Cambridge Percentage Increases, 1960-2014, for first-, secondand final-year students The remainder of this analysis deals only with the years 2000-2014. Finding 2: Within each calendar year, there have been extensive variations among Triposes in the percentages of students awarded Firsts and Upper-Seconds. (The term Percentage Variations is used for this.) This is illustrated in the following chart, which provides blue lines for all Part II students in each of the 22 larger Triposes. The black lines shows university-wide totals. Chart 2: Percentage Variations among Triposes An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 4 If we repeat Chart 2, but use red lines for Sciences Triposes and green lines for Arts Triposes, we find as follows. (See Section M for which Triposes are defined as Arts and which are defined as Sciences.) Finding 3: Until a few years ago, Part II examiners for Arts Triposes (shown in green) generally awarded Firsts to a lower percentage of students than did those for Sciences Triposes (red). But Part II examiners for nearly all Arts Triposes have consistently awarded Firsts or Upper-Seconds to a higher percentage of students than have those for Sciences Triposes. Chart 3: Differing patterns of behaviour by examiners for Sciences and Arts Triposes Part II students Finding 4: From 2000-2014, the average percentage of Part II Arts students awarded a First increased from 17% to 30%, and the average percentage of such students awarded a First or an Upper-Second increased from 84% to 94%. For Part II Sciences students, average percentages were essentially flat. Chart 4: Growth in percentages among Arts Triposes vs. Sciences Triposes for Part II students An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 5 Finding 5: For Part II students between 2000 and 2014, as shown in Chart 5A, examiners for the Law Tripos awarded a First to an average of just 17% of their students, whereas examiners for the Mathematics Tripos awarded a First to an average of 32% of their students. And examiners for the Education Tripos steadily and significantly increased the percentage of students to whom they awarded a First from 11% to 47%. Meanwhile, over the same years, as shown in Chart 5B below, examiners for Part II students in the Chemical Engineering Tripos awarded a First or an Upper-Second to an average of just 69% of their students, whereas those for the History Tripos awarded a First or an Upper-Second to an average of 97% of their students. And examiners for the English Tripos steadily and significantly increased the percentage of students to whom they awarded a First or an Upper-Second from 75% to 97%. In 2014, the Triposes with the highest percentages of students awarded Firsts were Education (47%), MML (38%) and History (35%); those with the lowest percentages were Law (24%) and Music (22%). These and others are shown in Chart 9. Chart 5A Chart 5B Finding 6: In 2014, not one of the 50 Part II students in the Land Economy Tripos, and only one of the 149 Part II students in the MML Tripos, and only three of the 200 Part II students in the History Tripos, were awarded a Lower-Second or a Third. Yet that same year, a Lower-Second or a Third was awarded to 55 out of 214 Part II Maths students (26%) and 63 out of 242 Part II Engineering students (26%). Finding 7: Considerable variations can also be found among the different subject options within the Natural Sciences Part II Tripos. In 2014, for instance, in subjects that had at least 25 examinees, percentages of students awarded Firsts ranged from 7% (Biological and Biomedical Sciences) to 44% (Chemistry), and percentages of students awarded a First or Upper-Second ranged from 76% (Biological and Biomedical Sciences) to 100% (Pharmacology). Some of these are shown below. (It should be noted, however, that admission to study some of these Part II subject options is selective.) Chart 7A Chart 7B An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 6 Finding 8: The behaviour of Part II examiners for four Arts Triposes is of particular note. In 2012, 2013 and 2014, the percentage of Part II students in each of the History, MML, Classics and English Triposes who were awarded a First or an Upper-Second was higher than the percentage across all Part II Sciences students who were awarded a First or an Upper-Second or a Lower-Second. Chart 8: Part II for four Arts Triposes vs Part II for all Sciences Triposes combined, on average during 2012-2014 Finding 9: The same issue is also illustrated in the next chart, which shows, for 2014 only, the percentage of students assigned to each class (blue for First, brown for Upper-Second, orange for Lower-Second and purple for Third) for each Part II Tripos. The chart shows that all students assigned a Lower-Second within four Science Triposes (Chemical Engineering, Mathematics, Engineering and Computer Science – jointly circled in black on the left) would, if they had scored similarly in eight Arts Triposes (Music, Theological and Religious Studies, Classics, History of Art, English, History, MML and Land Economy – jointly circled in black on the right), have been assigned an Upper-Second. Chart 9: For 2014, comparison in approaches to the Lower-Second among Part II Triposes An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 7 Finding 10: On average over the years 2000–2014, Part II examiners in seven of the 22 main Triposes (the Arts Triposes Theological and Religious Studies, History of Art, Geography, MML, History, Classics and Asian and Middle Eastern Studies) have assigned a Third to less than 0.5% of their students. Over this same time period, Part II examiners in four other Triposes (the Science Triposes Mathematics, Engineering, Computer Science and Chemical Engineering) have assigned a Third to over 4.5% of their students. It’s worth noting that of the 875 students who took Part II Theological and Religious Studies between 2000 and 2014, not one was awarded a Third. See the chart below. Chart 10: For 2000-2014, comparison in approaches to the Third in Part II among Triposes Finding 11: As shown in Chart 11, the percentages of students awarded Firsts and Upper-Seconds are higher in Part II (i.e. final-year) exams than in first- and second-year exams. More specific percentages are as in the following table, which will be drawn upon in Section J (Recommendations) below. Firsts Firsts + Upper-Seconds Year 1 + Year 2 Final year Year 1 + Year 2 Final year 2000 18.4% 21.9% 67.2% 81.2% 2014 21.4% 29.4% 78.6% 87.9% Average over last 10 years 20.3% 25.2% 75.1% 85.3% University-wide totals Chart 11: University-wide first-year exams (dots), second-year exams (dashes) and Part II (i.e. final-year) exams (solid) An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 8 E: Norm referencing When examiners “normalise” or “norm reference” their grades, or “mark to the curve”, they specify in advance how many of their students will be awarded each class. Thus, if they specify that 20% of their students will be awarded a First, they mark all students and then assign a First to the 20% with the best marks. The target “normalisation percentages” within a Tripos might, or might not, vary from year to year. And the target might, or might not, vary between first-year, second-year and final-year students. Finding 12: A recent Chair of the Mathematics Faculty wrote to me that “For as long as I can remember, the Faculty of Mathematics has norm referenced its Tripos results. E.g. in the 70s there were 25% firsts, but this was increased to about 30% firsts in the mid-80s.” This is illustrated in Chart 12A. Most other Triposes appear not to normalise – see for instance the Economics Tripos, in Chart 12B. Chart 12A: Illustration (using the Mathematics Tripos) of norm referencing. First-year exams (dots), second-year exams (dashes) and finalyear exams (solid). Chart 12B: Illustration (using the Economics Tripos) of apparent lack of norm referencing. First-year exams (dots), second-year exams (dashes) and final-year exams (solid). An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 9 F: Comparison of Tripos Classes with ‘A Level’ performance One question that naturally arises is whether the Triposes in which the highest percentages of students have been awarded a First are the same Triposes that were most often chosen by students who arrived at Cambridge with extremely high GCE A Level scores. This is examined as follows. Finding 13: The percentages of students who were awarded a First in Part II during the years 20122014 ranged from 38% (Education Tripos) down to 17% (Music Tripos); these are shown in the brown columns below. The blue columns show the percentages of students admitted in 2011-2013 to study each such Tripos who had a UCAS (Universities and Colleges Admissions Service) Tariff score of at least 600 points. (This score is even higher than the 560 points that would be earned by a student who had four “A Star” passes at A Level.) The Tripos with the highest percentage of students with UCAS Tariff scores of at least 600 points was Mathematics, at 83%; the Tripos with the lowest such percentage was Theological and Religious Studies, at 20%. The chart suggests that no obvious relationship exists between the brown columns and the blue ones. For instance, the percentage of Mathematics students who arrived at Cambridge during 2011-2013 with a UCAS Tariff score of at least 600 points was very high (83%), and the percentage of History students who arrived with such a score was much lower (39%); but the percentage of Part II students awarded a First over the years 2012-2014 was higher for History than for Mathematics. In other words, there is no obvious evidence that “the best students” (based on GCE A Level scores) choose to study a few particular Triposes and that students from these Triposes are then awarded the most Firsts. Chart 13: Brown columns: Percentages of students awarded a First in Part II during 2012-2014. Blue columns: Percentages of students admitted in 20112013 with a UCAS Tariff score of at least 600 points. An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 10 G: Comparison of Cambridge with other Russell Group universities Finding 14: For final-year Sciences subjects collectively, as shown in Chart 14A, Cambridge University awards both Firsts and Upper-Seconds to a somewhat lower percentage of students than Oxford University does, but to a somewhat higher percentage of students than the other 22 Russell Group universities do collectively. For final-year Arts subjects collectively, as shown in Chart 14B, Cambridge awards Firsts to about the same percentage of students as Oxford does, and awards Upper-Seconds to a slightly lower percentage of students than Oxford does. Both universities award final-year Firsts and Upper-Seconds to a significantly higher percentage of students than the other 22 Russell Group universities do collectively. (See Note 3 on Russell Group universities in Section M below on data.) It is the case at Cambridge (as already discussed in Finding 4), and at Oxford, and at the other Russell Group universities collectively, that from 2002 through 2012: (a) final-year Firsts were awarded to a higher percentage of Sciences students than Arts students, but (b) final-year Firsts plus Upper-Seconds were awarded to a higher percentage of Arts than Sciences students. See Charts 14A and 14B. Charts 14C and 14D are as per Charts 14A and 14B except the Russell Group universities are shown individually rather than collectively. As shown, in 2012, Russell Group universities other than Cambridge and Oxford awarded Firsts to between 21% and 38% of their final-year Sciences students, and awarded Firsts or Upper-Seconds to between 61% and 86% of their final-year Sciences students. The figures for Arts are 12% to 24% for Firsts, and 74% to 89% for Firsts or Upper-Seconds. Chart 14A (Sciences, final year) Chart 14B (Arts, final year) Cambridge (red and green), Oxford (purple and blue), and other Russell Group universities collectively (black) Chart 14C (Sciences, final year) Chart 14D (Arts, final year) Cambridge (red and green), Oxford (purple and blue), and other Russell Group universities individually (black) An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 11 H: The General Board’s Education Committee I sent an earlier version of this analysis to Alice Benton, the university’s Head of Educational and Student Policy. She shared it with Professor John Rallison, the University’s then Pro-Vice-Chancellor for Education, who placed the matter on the agenda of a meeting of the Cambridge University General Board’s Education Committee, which he chaired. Ms. Benton subsequently wrote to tell me that the Committee, in discussing the matter, noted that responsibility for classing criteria lies with Faculty Boards. She said that the Committee also noted that Faculty Boards might wish to consider the data along with other factors “in reviewing the core competencies and classing criteria for their courses.” An earlier version of this analysis was also sent to the Chair and various examiners at most of the Cambridge University faculties. All were given the opportunity to send comments or corrections prior to publication of this final version. No corrections were received. The comments received have, when relevant, been taken account of in this version. I: Issues arising The above findings raise a number of issues, as follows: Issue 1: Do the steadily increasing percentages of Cambridge students being awarded Firsts and UpperSeconds, as illustrated in Chart 1, represent “grade inflation”? (This term is generally used to indicate that exams have been made easier or that the exam score needed in order to be awarded a First or an Upper-Second has been made lower.) Or do the increasing percentages result instead from “better students” – i.e. from a steady increase in the average calibre and diligence of the students – and/or from “better teachers”? Opinion: This can’t be answered from the available data. But my own view, based on multiple conversations with Cambridge faculty, is that across the university as a whole, “better students” is at least a factor in the increases in the percentages of students awarded Firsts and Upper-Seconds. (However, for an opinion at the Tripos level, see the next point.) Issue 2: Why is it that the percentage of Part II students being awarded Firsts in the Education Tripos increased from 11% in 2000 to 47% in 2014 (see Chart 5A), and that the percentage of Part II students being awarded Firsts or Upper-Seconds in the English Tripos increase from 75% in 2000 to 97% in 2014 (see Chart 5B)? Have those Triposes really attracted dramatically improving students, year by year? Or is grade inflation taking place? Opinion: These increases cannot be fully explained by improvements in student quality. It is clear that some grade inflation is taking place in these Triposes. Conversely, for certain Triposes such as Mathematics (see Chart 12A), it is clear that there has been no grade inflation, because percentages of students being awarded Firsts and Upper-Seconds have remained static since at least 2000. If, over the years, there has been any improvement at all within those Triposes in the calibre of the students or in the quality of the teaching, this means that what has been going on there actually represents “grade deflation”, because it is becoming steadily harder for students of a particular level of ability to earn Firsts or Upper-Seconds in those Triposes. Issue 3: What are we to conclude from Chart 5A, which shows that the percentage of students awarded Firsts in the Mathematics Tripos has averaged 32% since 2000, whereas for Law the average has been just 17%? Likewise, what are we to conclude from Chart 5B, which shows that the percentage of students awarded Firsts or Upper-Seconds in the History Tripos since 2000 has averaged 97%, whereas for Chemical Engineering the average has been 69%? Is this because far more of “the very best students” study Mathematics or History than study Law or Chemical Engineering? Opinion: The analysis in Finding 13 provides no evidence that the Triposes that award the most Firsts and Upper-Seconds are able to do so because they have attracted far more of the very best students. As a result, there appears to be no logical and equitable reason why some Triposes award Firsts and Upper-Seconds to far higher percentages of students than do others. An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 12 (Note: A member of the History faculty responded to a draft version of this analysis by saying “the History Faculty criterion for matriculation is the high probability that the candidate, if accepted, will be capable of achieving an upper-second degree or higher.” At first sight, this might explain why, in 2014, 98.5% of History Part II students were awarded a First or an Upper-Second. But given that the selection of students for entry to Cambridge is carried out by the colleges, not by the faculties, this faculty member is implying that the colleges universally seek to honour the History Faculty’s criterion for matriculation. If so, why is it that the colleges also admit students among whom, in some other subjects, much lower percentages end up being awarded Firsts or Upper-Seconds? Do the colleges say to themselves, “We must only admit the super-capable applicants for reading History; but for other subjects we can set the bar somewhat lower”? A more likely explanation is that it is the History faculty itself that lowers the bar, come exam time, ensuring that just about everybody gets a First or an Upper-Second.) Issue 4: Why is it that in 2014, per Finding 6, not a single one of the 50 Part II students in the Land Economy Tripos, and only one of the 149 Part II students in the MML Tripos, and only three of the 200 Part II students in the History Tripos, were awarded a Lower-Second or a Third – while a Lower-Second or a Third was awarded that year to 55 out of 214 Part II Mathematics students (26%) and 63 out of 242 Part II Engineering students (26%)? And why is it that, per Finding 10, of the 875 students who have taken Part II Theological and Religious Studies since 2000, not one has been awarded a Third? Opinion: I can find no logical or equitable justification for these findings. Issue 5: Why is it that Cambridge’s central administration has apparently never pushed for a consistent approach among Triposes to the assignment of classes? Opinion: I don’t know, but I would argue that the differences in the policies of the examiners of the different Triposes have led to there being less true meaning, or value, in a First or an Upper-Second than there could and should be. J: Recommendations I recommend that the university’s faculties and departments discuss this issue with each other and with the central administration, and then make appropriate changes in policy and practice. I recommend also that these changes primarily involve norm referencing, i.e. pre-determining the percentages of Firsts and Upper-Seconds that will be awarded. More specifically, I recommend: (a) That the Cambridge University General Board’s Education Committee propose maximum percentages of students taking each Tripos who will be awarded Firsts and Upper-Seconds. (b) That for each Tripos, whatever the recent percentages of students awarded Firsts or UpperSeconds in that Tripos, the percentage of students awarded Firsts and Upper-Seconds should not exceed the university-wide averages that have pertained over the past ten years. Drawing upon data in Finding 11, this means that (i) for first- and second-year students, no more than 20% of those taking each Tripos should be awarded a First, and no more than 75% should be awarded a First or Upper-Second; and (ii) for final-year students, no more than 25% of those taking each Tripos should be awarded a First, and no more than 85% should be awarded a First or Upper-Second. (c) That after the publication of class lists each year, the General Board’s Education Committee publish a report showing what the actual percentages were for each Tripos/student-year combination. And that this report include, for each Tripos/student-year combination for which actual percentages did not comply with the maxima recommended in the previous point, an explanation for this by the examiners. An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 13 K: Link to Tripos-specific memos and data At www.BernardRivers.com/exam-analysis, links are provided to (a) a downloadable version of this analysis; (b) one downloadable memo per Tripos, each memo containing detailed charts comparing the awarding of Firsts and Upper-Seconds in that Tripos with the awarding of such classes in other Triposes; and (c) one downloadable spreadsheet file per Tripos, providing all relevant data for that Tripos. L: Open discussion At www.BernardRivers.com/exam-analysis, an opportunity is provided for readers of this analysis to comment on its findings and recommendations, and to share information on any relevant non-confidential discussions and developments that have arisen within the individual faculties and departments. I believe that use of this facility will lead to a more informed debate. M: Future plans I plan, in late 2015 or early 2016, to publish an updated version of this analysis that incorporates 2015 exam results and that reviews any notable changes that the 2015 results reflect. N: Data and sources All the Cambridge-related data drawn upon for this analysis are in the public domain. This analysis deals with those Part II (i.e. final-year) Cambridge University undergraduate examinations and related forms of assessment for which the successful candidates were awarded First Class Honours, Upper-Second Class Honours, Lower-Second Class Honours or Third Class Honours. The lines for specific Triposes shown in the charts in this analysis cover the 22 Cambridge University Triposes that since at least 2000 have provided exams for undergraduates in at least their second and third years, and for which the Part II exams have been taken by at least 25 students every year from at least 2008 through 2014. With the colour red used for Triposes defined by the university as Sciences and the colour green used for those defined as Arts, the 22 Triposes are: Archaeology and Anthropology; Architecture; Asian and Middle Eastern Studies (previously, Oriental Studies); Chemical Engineering; Classics; Computer Science; Economics; Education (including Education Studies); Engineering; English; Geography; History; History of Art; Land Economy; Law; Mathematics; Modern and Medieval Languages; Music; Natural Sciences; Philosophy; Politics, Psychology, and Sociology (previously, Social and Political Sciences); and Theological and Religious Studies. Lines are not shown for Triposes that do not meet all the above criteria – namely Anglo-Saxon, Norse, and Celtic; Human, Social and Political Sciences; Linguistics; Management Studies; Manufacturing Engineering; Medical and Veterinary Sciences; Psychological and Behavioural Sciences; and Theology for Ministry. However, these Triposes are included as appropriate when Part II totals/averages are computed across Sciences/Arts/all Triposes. The data used for Finding 1 regarding 1960-2000 were obtained from an issue of the University Reporter at www.admin.cam.ac.uk/reporter/1999-2000/special/21/1.pdf, and regarding 2001-2010 were obtained from an issue of the University Reporter at www.admin.cam.ac.uk/reporter/201011/special/04/studentnumbers0910.pdf. The data used for Finding 13 that relate to UCAS Tariff scores were obtained from http://unistats.direct.gov.uk. The data used for those parts of Finding 14 that relate to universities other than Cambridge were drawn from a spreadsheet kindly provided by the UK’s Higher Education Statistics Agency (HESA). The HESA data only relate to years ending 2002 through 2012. All other data related to Cambridge University for 2000 through 2010 were taken from Tables 21 through 26 in the university’s “Student Numbers” publications at www.admin.cam.ac.uk/offices/planning/sso/reporter/index.html. All data related to Cambridge for 2011 through 2014 were taken from Tables EXM_01 and EXM_02 in the university’s “Student Statistics” publications at www.admin.cam.ac.uk/offices/planning/sso/studentnumbers/index.html, and (for An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 14 Preliminary and Theology for Ministry results) from university class lists available at the University Library and elsewhere. Note 1: For first-year students, the following Triposes do not divide the Second Class degree into an Upper-Second and Lower-Second: Classics, Medical and Veterinary Sciences (from 2001 to 2012), and Natural Sciences. Care has been taken, when computing the percentage of students within (say) first-year Arts who were awarded a “First or Upper-Second”, to include in the denominator only those students who took exams for which a divided Second was offered. However, this was not possible in Chart 1, because the source data do not provide the required level of detail. Thus, in Chart 1 only, the percentages of students shown as being awarded an Upper-Second or a Lower-Second are expressed as a percentage of all students taking exams that year rather than as a percentage of all students taking exams for which a divided Second was offered. Note 2: Some Sciences students at Cambridge are provided with the opportunity to do an extra year of study during which they take Mathematics Part III, Engineering Part IIB, Chemical Engineering Part IIB, Manufacturing Engineering Part II, Medical and Veterinary Sciences Part II, or Natural Sciences Part III. The taking of these exams is not required in order to be granted a first degree. Accordingly, results from these exams are excluded from this analysis. Likewise, LL.M. results are not included. Note 3: The 24 Russell Group universities discussed in Finding 14 are Cambridge, Oxford, The Queen’s University of Belfast, Birmingham, Bristol, Cardiff, Durham, Edinburgh, Exeter, Glasgow, Imperial College , King’s College London, Leeds, Liverpool, London School of Economics (LSE), Manchester, Newcastle-upon-Tyne, Nottingham, Queen Mary and Westfield College, Sheffield, Southampton, University College London, Warwick and York. In Chart 14C (Sciences), LSE is excluded because it has very few science students, and in chart 14D (Arts), Imperial College is excluded because it has very few arts students. O: Postscript regarding work done in 1969 In 1969 the Cambridge University economics faculty board set up a staff-student committee to review and make recommendations regarding forms of assessment. Professor Nicholas Kaldor served as Co-Chair representing the Economics Faculty, and I served as Co-Chair representing Economics students. In those days the only form of assessment used with Economics students was a set of three-hour examinations carried out at the end of each academic year; there was no dissertation option. All the exam scripts were first marked by a set of “first markers”. Approximately half the students (namely, those who appeared to be destined for a First or a fail, those who were close to the borderline between one class and the next, and those whose marks were all over the place) then had each paper marked by a set of “second markers”. Unlike with the economics faculty’s present system, the second markers did not know what marks the first markers had assigned. The committee started by commissioning me to conduct a statistical analysis of the consistency between first markers’ marks and second markers’ marks. This concluded that 24% of those who were marked by two sets of markers ended up with a different class than they would have received if they had been marked only by the first set of markers. The committee made various recommendations to the faculty board, one of which was that students should have the option of replacing two of their six exam papers with a dissertation. The economics faculty board and students debated these recommendations over the following two years until agreement was reached on a modestly revised set of recommendations. The faculty board then submitted these to the university’s General Board for approval. After a 10-minute discussion, the General Board accepted two of the faculty board’s recommendation (one in revised form), and rejected the other two. Shortly afterwards, on 3-5 February 1972, approximately 2,000 students conducted a “sit-in” at the university’s main administrative building to protest the rejections by the General Board. This was the largest protest that has ever taken place at the An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 15 university. The university then commissioned Lord Devlin to conduct an official inquiry into what had happened, and why, and what the lessons were. Devlin received submissions from nearly 100 interested parties. His 80-page report took nearly a year to produce; it made many criticisms of the university, and proposed many reforms. The reports and submissions are accessible in the University Library archives. Tripos-specific index Note: In the following, “memo at website” refers to a memo specific to each Tripos that can be accessed at www.BernardRivers.com/exam-analysis. Anglo-Saxon, Norse, and Celtic Archaeology and Anthropology Architecture Asian and Middle Eastern Studies Chemical Engineering Classics Computer Science Economics Education Engineering English Geography History History of Art Human, Social and Political Sciences Land Economy Law Linguistics Management Studies Manufacturing Engineering Mathematics Medical and Veterinary Sciences Modern and Medieval Languages (MML) Music Natural Sciences Philosophy Politics, Psychology, and Sociology Psychological and Behavioural Sciences Theological and Religious Studies Theology for Ministry. p 14 + memo at website pp 7, 8, 14 + memo at website pp 7, 8, 10, 14 + memo at website pp 7, 8, 10, 14 + memo at website pp 2, 6, 7, 8, 10, 12, 14 + memo at website pp 7, 8, 10, 14 + memo at website pp 7, 8, 10, 14 + memo at website pp 3, 8, 9, 10, 14, 15 + memo at website pp 2, 6, 7, 8, 10, 12, 14 + memo at website pp 2, 6, 7, 8, 10, 13, 14 + memo at website pp 2, 6, 7, 8, 10, 12, 14 + memo at website pp 7, 8, 10, 14 + memo at website pp 2, 6, 7, 8, 10, 12, 13, 14 + memo at website pp 7, 8, 10, 14 + memo at website p 14 + memo at website pp 2, 6, 7, 8, 10, 13, 14 + memo at website pp 2, 6, 7, 8, 10, 12, 14 + memo at website p 14 + memo at website p 14 + memo at website p 14 + memo at website pp 2, 6, 7, 8, 9, 10, 12, 13, 14 + memo at website p 14 + memo at website pp 2, 6, 7, 8, 10, 13, 14 + memo at website pp 6, 7, 8, 10, 14 + memo at website pp 2, 6, 7, 8, 10, 14 + memo at website pp 7, 8, 10, 14 + memo at website pp 7, 8, 14 + memo at website p 14 + memo at website pp 2, 7, 8, 10, 13, 14 + memo at website p 14 + memo at website An analysis of how often “Firsts” and other classes are assigned in Cambridge University exams Page 16