Analysing the effects of the introduction of the new Project
Maths syllabus on beginning undergraduates’ performance
of basic mathematical skills in Ireland
Páraic Treacy1 & Fiona Faulkner2
1
College of Education, University of Derby & 2Dublin Institute of Technology
P.Treacy@derby.ac.uk & Fiona.Faulkner@dit.ie
Abstract
The mathematical under-preparedness of students entering universities in
Ireland, UK, Australia and the US has been widely reported with beginning
undergraduates displaying a lack of basic mathematical skills, as well as
fragmented understanding, inadequate concept knowledge, and inability to
successfully solve mathematical problems (O’Donoghue 2004, Gill et al. 2010,
Lawson 2003). This issue has been of particular concern in Ireland recently,
with beginning undergraduates’ performance of basic mathematical skills
showing signs of significant decline over the last 10 years (Treacy & Faulkner
2015). New mathematics syllabi, commonly referred to as ‘Project Maths’,
were gradually introduced into Senior Cycle and Junior Cycle of the Irish
secondary school education system from 2010 onwards to address some of
the criticisms of the approach to teaching mathematics that existed in Ireland.
These new syllabi aim to place greater emphasis on student understanding of
mathematical concepts, enabling students to relate mathematics to everyday
scenarios with increased use of contexts and applications. This study
analyses, through mathematics diagnostic test data gathered between 2008
and 2014, the effects that the introduction of the new Senior Cycle syllabus
has had on beginning undergraduates’ basic mathematical skills.
Introduction
The mathematical under-preparedness of students entering universities in
Ireland has been reported recently, with significant declines in the execution of
basic mathematical skills observed (Treacy & Faulkner 2015).
This
phenomenon in which students are under prepared for the mathematical
demands of their undergraduate courses is regularly referred to as the ‘Maths
Problem’ (Lawson & Croft 2012). The ‘Maths Problem’, characterised by
student deficiencies such as a lack of basic mathematical skills, fragmented
understanding, inadequate concept knowledge, and inability to successfully
solve mathematical problems, has been an issue of concern in the Ireland, UK,
Australia, and the US (O’Donoghue 2004, Gill et al. 2010, Lawson 2003).
Allied to this documented decline in basic mathematical skills among
beginning undergraduates, concerns arise from the finding that over four-fifths
of school leavers in Ireland reported significant differences in approaches to
teaching and learning between secondary level and further/higher education
(McCoy et al. 2014). Low retention rates in higher education STEM courses
have been attributed to students’ shortcomings in mathematics (Bourn 2007)
which would lead one to consider that these reported issues in Ireland will
have a similarly negative effect in the near future.
In Ireland, steps have been taken to address deficiencies in secondary level
students’ mathematics capabilities. Junior Cycle (years 1-3 of secondary
level) and Senior Cycle (years 5-6 of secondary level) secondary mathematics
curricula have been overhauled with the introduction of the ‘Project Maths’
syllabi. The aim of these new syllabi is to place greater emphasis on student
understanding of mathematical concepts with increased use of contexts and
applications of mathematics in real world scenarios. The new Project Maths
syllabi intend to encourage a greater focus on problem solving skills while
aligning assessment with these revised classroom practices (Project Maths
Team 2010, DES 2010).
The introduction of new mathematics curricula at secondary level is not the
only focus of change in Irish secondary mathematics education. Focus has
also been placed on the qualifications of practicing secondary mathematics
teachers in the country also. In a 2009 study, 48% of teachers in Ireland
assigned to teach secondary level mathematics were found to be ‘out-of-field’
i.e. teaching a subject which does not match their training or education (Ní
Ríordáin & Hannigan 2009). These findings prompted the introduction of the
Professional Diploma in Mathematics for Teaching (Part-time) so that these
teachers could upskill and ensure they became fully qualified to teach
secondary mathematics. This 2-year part-time blended learning programme
was implemented nationwide in 2012. The first cohort of approximately 300
teachers graduated in January 2015 while two further cohorts are working
towards this goal at present with another cohort due to commence studies in
September 2015.
As mathematics is the strongest predictor of successful student progression
through tertiary level education in Ireland (Mooney et al. 2010) and has been
proven to play an important role in a country’s economic success (EGFSN
2008), this focus on enhancing the quality of teaching and learning within the
subject is not surprising. The effects of these efforts on student performance
in mathematics should prove to be of great national interest over the coming
years. This study aims to offer early indications of some elements of these
effects, particularly the effects on student performance of basic mathematical
skills and algorithms.
The findings outlined in this paper will track changes in beginning
undergraduate students’ mathematics competencies in specific topics between
2008 and 2014. The means by which these students’ performance of basic
mathematical skills was determined will be outlined along with a general profile
of the students’ mathematical backgrounds. Furthermore, a detailed analysis
of the trends related to student performance on the UL mathematics diagnostic
test will be charted.
Mathematical Background of Irish Students
The vast majority (>99%) of students who leave secondary level education to
enter tertiary level education in Ireland have studied mathematics throughout
primary and secondary school. Mathematics is one of three subjects that
students must complete in the Leaving Certificate examinations which act as
their final examinations at secondary level. Typically at age 17 or 18, students
sit the Leaving Certificate mathematics examination at one of three levels –
higher, ordinary, or foundation. Changes in how these students have been
assessed in mathematics at Leaving Certificate have occurred since 2011 with
the introduction of the Project Maths syllabus. This syllabus was phased in to
the Senior Cycle – the final two years of secondary education – in all Irish
secondary schools, starting in September 2010 and eventually fully rolled out
by September 2013 (Project Maths Team 2010). Elements of the Project
Maths syllabus were thus incorporated into the Leaving Certificate
mathematics examinations in June 2011, with gradually more added in June
2012, and June 2013. The first examination to be completely based on the
new syllabus in all Irish secondary schools took place in June 2014.
This timeline was important when determining the data to explore through this
research. The years 2008 to 2014 were selected so that a pattern could be
observed in the years before the phased introduction of the new syllabus, as
well as the years in which the new syllabus was gradually introduced, and the
first year (2014) in which the new syllabus was fully implemented within the
Leaving Certificate examinations. In November 2013, an early assessment of
the impact of the Project Maths syllabus was published which concluded that
there did not appear to be any link between the new syllabus and any overall
decline or progression in students’ achievement in mathematics (Jeffes et al.
2013). The analysis outlined in this paper will examine these claims and
provide further evidence of the effects this transition has had on student
performance in mathematics in specific topics.
The importance of the effects of this change in curriculum can be linked,
among other factors, to the fact that results in the Leaving Certificate
examinations determine whether or not a student transitions directly from
secondary level education to their chosen course of study at tertiary level in
Ireland. Points are awarded to students based on the grades achieved in the
Leaving Certificate examinations. The number of points achieved by a student
determines the tertiary level courses to which he/she may gain entry. To boost
the numbers of students opting to study mathematics at Higher Level for the
Leaving Certificate, 25 bonus points were awarded to students who achieved a
grade D3 or above in the Higher Level Leaving Certificate mathematics
examination. This measure, introduced in 2012, has had a noticeable effect
on the proportion of students opting to study Higher Level mathematics at
Leaving Certificate. This phenomenon will be discussed later.
The Diagnostic Test and Methodology
Students entering first year service mathematics modules for science based
and technology based undergraduate degree courses at UL have been given a
40 question mathematics diagnostic test. This test aims to assess the
application of basic mathematical skills and understanding in selected
mathematical topics. These topics are arithmetic (13 questions), algebra (8
questions), geometry (4 questions), trigonometry (3 questions), co-ordinate
geometry (4 questions), complex numbers (2 questions), differentiation (3
questions), integration (2 questions), and modelling (1 question).
Of the forty questions, thirty four are set at a Leaving Certificate Ordinary Level
standard or below. The remaining six questions are set at a Leaving
Certificate Higher Level standard. Students who score 18/40 or less are
considered to be at risk of failing their service mathematics module thus they
are deemed to be ‘at risk’. Research by Faulkner (2012) objectively proved
that setting this as the cut-off point for terming students to be ‘at risk’ would be
an accurate prediction of their likely performance based on retrospective data.
Students who are termed ‘at risk’ are encouraged to take advantage of
mathematics support services available at the University of Limerick in a timely
and regular manner to address any gaps in their mathematical knowledge and
skills.
The UL diagnostic test was designed specifically to suit the mathematical level
of the students in each of the targeted service mathematics modules. The
design of the UL diagnostic test was carried out through analysis and
adjustment of an initial list of 70 questions by a team of experienced service
mathematics lecturers as they reduced this to the final 40 question version
used from 1998 to the present day. The UL diagnostic test was piloted in Irish
secondary schools and compared with the European Society for Engineering
Education (SEFI) Core Level Zero syllabus for engineers, the Irish Junior
Certificate mathematics syllabus, the Irish Leaving Certificate mathematics
syllabus, and further diagnostic tests to ensure validity of this research tool
(Gill et al. 2010).
At this point it is worth noting that the documented change to the Leaving
Certificate syllabus may place some limitations on the relevance of the UL
diagnostic test in assessing the basic mathematical skills of beginning
undergraduates. The new Leaving Certificate mathematics syllabus aims to
place greater focus on problem solving and the application of mathematics in
varying contexts in contrast to the old syllabus which placed greater emphasis
on the execution of procedure and algorithms (DES 2010, Project Maths Team
2010).
Having stated that, the basic mathematical skills required to
successfully answer the forty questions of the UL diagnostic test are still part
of the new syllabus thus students would need to be proficient in these skills in
order to apply their understanding in different contexts.
Data on the cohort of beginning undergraduate students in science based and
technology based degree courses who sat this diagnostic test have been
collected each year between 1998 and 2014. The same diagnostic test was
used in each of these years to ensure reliability. The UL database, which
contains data from these diagnostic tests, currently holds information on over
10,700 students. This paper will focus on data from 2008 to 2014 so that clear
trends in the period before, during, and just after the phased introduction of the
Project Maths syllabus to the Leaving Certificate mathematics examinations
can be observed and analysed.
Analysis of this data was carried out in an attempt to aid the authors in
answering the following research questions:
 What level of performance are beginning undergraduates in UL service
mathematics modules displaying with regard to basic mathematical skills
in each of the topics examined by the UL diagnostic test?
 What are the trends in performance of basic mathematical skills of
students entering first year service mathematics modules at UL as the
Leaving Certificate commenced and completed the phased introduction
of the Project Maths syllabi?
 Is there any evidence of grade dilution in Leaving Certificate
mathematics in the period 2008-2014?
Background of students
Analysis of the achievements in Leaving Certificate mathematics will give an
insight into the typical profile of students entering service mathematics
modules at UL. The majority of students who completed the diagnostic test
from 2008 to 2013 had studied mathematics at ordinary level in the Leaving
Certificate with the (between 50.4% and 60.8%). However, from 2012
onwards, the proportion of students who had studied at Ordinary Level began
to fall while the proportion of students who had studied mathematics at Higher
Level was increasing. By 2014, the proportion of students with a Higher Level
Leaving Certificate mathematics background (51.2%) was greater than the
proportion of students with an Ordinary Level Leaving Certificate background
(43%).
This significant change may be explained in part by the offer of 25 bonus
points in the Leaving Certificate for students who obtain a grade D3 or better in
the Higher Level mathematics examination. This initiative commenced in 2012
and was continued in 2013 and 2014. Since the introduction of these bonus
points, there has been a noticeable increase in the proportion of students
opting for Higher Level mathematics at Leaving Certificate in Ireland,
improving from 16% in 2011 to 22% in 2012, 25.6% in 2013 and 27% in 2014
(State Examination Commission 2014). This change has been reflected in the
data recorded in the UL database.
Students that completed their secondary level studies outside Ireland, did not
sit their Leaving Certificate in the previous 10 years, or did not sit a Leaving
Certificate mathematics examination at all were termed as ‘Other/Missing’ in
the database. These students typically accounted for a small percentage of
each cohort i.e. between 2.4% and 12.4%, with no pattern of note to this
variation.
45
40
35
2008
30
2009
25
2010
20
2011
2012
15
2013
10
2014
5
0
HA
HB
HC
HD
NS
OA
OB
OC
OD
Figure 1. Percentage of students entering UL service mathematics modules
that achieved specific Leaving Certificate mathematics grades.
Analysis of students’ performance in mathematics at Leaving Certificate (see
Figure 1) would indicate that students have achieved similar grades in Leaving
Certificate examinations in the timeframe explored (2008-2014). There are,
however, some differences in the years 2012, 2013, and 2014 when compared
to the years previous as the proportion of students achieving Ordinary Level A
grades (OA) declined while the proportion of those achieving Higher Level C
grades (HC) and Higher Level D grades (HD) increased in each of the years
from 2012 to 2014. This change could be explained by the introduction of
bonus points during that period – those that typically would have been
expected to achieve Ordinary Level A grades in 2012, 2013, and 2014 may
have opted to study Leaving Certificate mathematics at Higher Level to
attempt to take advantage of the bonus points on offer and achieved Higher
Level C and D grades.
Treacy & Faulkner (2015) have previously reported on the proportion of
students assessed through the UL diagnostic test who were deemed to be ‘at
risk’ of failing their service mathematics final exam for the period 2003-2013.
This figure had increased significantly in that period: from 30.6% in 2003 and
25.6% in 2004 to 53.7% in 2012 and 48.7% in 2013. Similar figures of
students ‘at risk’ have been observed in the 2014 cohort, with 48.6% at risk.
As such, an increase in the proportion of students studying Higher Level
mathematics at Leaving Certificate has not coincided with an increase in
overall performance by students in the UL diagnostic test.
Results
Changes in performance by students transitioning from secondary to tertiary
level education were analysed in the nine mathematical topics examined
through the UL diagnostic test. When mean scores in arithmetic, algebra,
geometry, trigonometry, complex numbers, and differentiation are compared
for the years 2008 and 2014, a statically significant decline (p < 0.05) is
observed. Conversely, mean scores for coordinate geometry and integration
indicated a statistically significant increase when data from 2008 and 2014 are
compared. These findings are interesting as they provide insight into the
effects the introduction of Project Maths has had on performance of basic
mathematical skills in each of these topics. However, as observed previously,
the grade profile of these students is noticeably different when students from
2008 are compared to students from 2014. So, to ensure that a balanced
analysis of the data is performed, comparisons will be drawn between students
who achieved similar Leaving Certificate mathematics grades so that
conclusions can be confidently drawn concerning the level of improvement or
decline in performance of basic mathematical skills by students during the
period 2008 to 2014.
80
70
60
Arithmetic
Algebra
50
Geometry
Trigonometry
40
CoordinateGeometry
ComplexNos
30
Differentiation
Integration
20
Modelling
10
0
2008
2009
2010
2011
2012
2013
2014
Figure 2: Overall diagnostic test performance by topic 2008-2014.
The grades which will be considered will be higher level B, C, and D grades.
The number of students in the database with higher level A grades was not
sufficient enough to carry out reliable statistical analysis thus it will not be
outlined here. Also, the additional volume of data that would need to be
displayed to properly discuss the trends observed amongst students from an
Ordinary Level mathematics Leaving Certificate background was such that
these students could not be included in the analysis carried out for this paper.
Students who achieved Leaving Certificate Higher Level B grades in the 2008
cohort performed similarly in most topics to those who achieved Leaving
Certificate Higher Level B grades in the 2014 cohort. Statistically significant
changes (p < 0.05) were observed in some topics with improvements occurring
in coordinate geometry and modelling, and a decline in performance in
differentiation. Comparison of 2009 and 2014 produced similar results with
statistically significant (p < 0.05) improvement in geometry and declines in
algebra, geometry, and differentiation.
Comparison of students who achieved higher level C grades in Leaving
Certificate mathematics in 2008 and 2014 produces a greater range of
discrepancies. Statistically significant (p < 0.05) declines in performance are
evident in six of the nine topics: arithmetic, algebra, geometry, trigonometry,
complex numbers, and differentiation.
The latter three of these topics
displayed particularly pronounced declines (p = 0.000) in mean performance
by students in 2014 compared to 2008 at this grade level. One topic,
coordinate geometry, showed statistically significant improvement. Comparing
performance in 2009 to 2014 also produces statistically significant (p < 0.05)
declines in the same six topics and improvement once again in coordinate
geometry.
When mean performance in the specific categories of the diagnostic test of
students who achieved higher level D grades in the Leaving Certificate are
compared for 2008 and 2014, there are statistically significant (p < 0.05)
declines in seven of the nine topics. These topics are arithmetic, algebra,
geometry, trigonometry, complex numbers, differentiation, and integration.
The first four topics stated here had clearly significant declines (p = 0.000).
There is no topic in which performance has increased significantly.
Table 1: Summary of statistically significant changes in beginning
undergraduates’ mean performance of basic mathematical skills by topics and
by Leaving Certificate grades achieved.
Topic
Higher Level B
Higher Level C
Higher Level D
Arithmetic
–
–
Algebra
–
–
Geometry
–
–
Trigonometry
–
–
Co-ordinate
Geom.
Complex
Numbers
Differentiation
+
–
–
–
–
–
–
Integration
Modelling
+
+
+ = Statistically significant increase in performance when data from 2008 and
2014 are compared.
– = Statistically significant decline in performance when data from 2008 and
2014 are compared.
Discussion
The comparisons drawn above between mean scores at each individual
mathematical topic by students who achieved Higher Level Leaving Certificate
grades B, C, or D highlight declines in basic mathematical skills of students
who achieve those grades.
Statistically significant declines in mean
performance for arithmetic algebra, geometry, trigonometry, complex numbers
and differentiation at both higher level C and D grades indicate that the basic
skills inherent in these topics are in decline amongst students entering
university with these Leaving Certificate mathematics grades.
As 2014 was also the first year that the fully implemented ‘Project Maths’
curriculum was examined through the Leaving Certificate, this also raises
questions concerning how this new syllabus affects the performance of basic
mathematical skills by students studying this syllabus. The shift in focus from
emphasis on the execution of procedure and algorithms in the previous Senior
Cycle mathematics syllabus to applications of mathematics and problem
solving in the new Project Maths syllabus would, according to this data, appear
to be negatively affecting students’ execution of basic mathematical skills.
Such a change may be expected and does not necessarily indicate a decline
in overall mathematical ability of these students. It has been documented that,
in some instances, secondary students appear to be successfully drawing
together their knowledge across different mathematical topics thus indicating
that they are beginning to acquire a deeper understanding of mathematics and
how it can be applied (Jeffes et al., 2013). However, the observed declines
outlined in this paper is an indicator that execution of the basic mathematical
skills may be an emerging issue that needs to be strongly considered so that
deficiencies in this element of students’ mathematical abilities can be
addressed.
On a positive note, it is interesting to note that performance in the modelling
question of students who achieved higher level B grades in the Leaving
Certificate in the diagnostic test show significant improvement between 2008
and 2014 (see Table 1). The aforementioned shift in focus to application of
mathematics in different contexts would appear to be aiding the modelling
capabilities of the students who experienced the ‘Project Maths’ syllabus at
secondary level.
Analysis of student performance when Leaving Certificate grades are
controlled (see Table 1) would indicate that there are a greater number of
statistically significant declines in mean performance of students within the
individual mathematical topics the lower the grade achieved at Higher Level by
the student. This could be an indicator that the basic mathematical skills
required to achieve a grade C or a grade D in higher level Leaving Certificate
mathematics in 2014 are below that which was required in 2008. Grade
dilution must be considered in light of these findings as it is clear that the basic
mathematical skills required to achieve Higher Level C and D grades in
mathematics at Leaving Certificate in 2014 are below that which were required
in 2008. Grade dilution can only be definitively determined through further
research specific to this issue on a national scale but the indicators present in
the data analysed suggest such a study is warranted.
Conclusion
The phased introduction of the new Project Maths Senior Cycle syllabus in
Ireland aims to place greater emphasis on student understanding of
mathematical concepts with increased use of contexts and applications of
mathematics in real world scenarios (DES 2010, Project Maths Team 2010).
Allied to that, the introduction of bonus points awarded to students who pass
mathematics at Higher Level in the Leaving Certificate has been put in place to
encourage study of the subject at this level in secondary education. The
bonus points measure has been successful as there has been a marked
increase in the proportions of students studying Higher Level mathematics at
Leaving Certificate. However, it would appear that this measure and the new
syllabus have not had a positive effect on performance of basic mathematical
skills among beginning undergraduate students entering UL from Irish
secondary education.
Significant declines in beginning undergraduates’ performance of basic
mathematical skills are evident, particularly among students who have
achieved Higher Level C grades and Higher Level D grades. These declines
are especially noticeable in topics such as arithmetic, algebra, geometry,
trigonometry, complex numbers and differentiation. This fall off in performance
does not bode well for successful transition to study of service mathematics
modules at tertiary level and, given that mathematics is the strongest predictor
of successful student progression through tertiary level education in Ireland
(Mooney et al. 2010), this is an issue which needs to be addressed sooner
rather than later.
References
Bourn, J. (2007). Staying the course: The retention of students in higher education.
London: The Stationery Office.
DES (Department of Education and Skills) (2010). Report of the Project Maths
Implementation Support Group, Dublin: Department of Education and Skills.
EGFSN (Expert Group on Future Skills Needs) (2008). Statement on Raising
National Mathematical Achievement: To the Tánaiste & Minister for Enterprise,
Trade and Employment and the Minister for Education and Science,
November 2008.
Faulkner, F. (2012). An analysis of performance in mathematics for technology
undergraduates and an investigation of teaching interventions for these
students.
Gill, O., O’Donoghue, J., Faulkner, F., & Hannigan, A. (2010). Trends in performance
of science and technology students (1997–2008) in Ireland. International
Journal of Mathematical Education in Science and Technology, 41(3), 323339.
Jeffes, J., Jones, E., Wilson, M., Lamont, E., Straw, S., Wheater, R. and Dawson, A.
(2013). Research into the impact of Project Maths on student achievement,
learning and motivation: final report. Slough: NFER.
Lawson, D. (2003). Changes in student entry competencies 1991–2001. Teaching
mathematics and its applications, 22(4), 171-175.
Lawson, D., Croft, T., & Waller, D. (2012). Mathematics support past, present and
future. Innovation, Practice and Research in Engineering Education. (See
http://www.
academia.
edu/2715773/Mathematics_support_
past_present_and_future) [accessed 21 January 2015].
McCoy, S., Smyth, E., Watson, D., & Darmody, M. (2014). Leaving school in Ireland:
a longitudinal study of post-school transitions, in ESRI Research Series. ESRI:
Dublin 2014.
Mooney, O., Patterson, V., O'Connor, M., & Chantler, A. (2010). A study of
progression in Irish higher education.
Ní Ríordáin, M., & Hannigan, A. (2009). Out-of-field teaching in post-primary
mathematics education: An analysis of the Irish context. A NCE-MSTL
Research Report. Limerick: NCE-MSTL.
O’Donoghue, J. (2004). An Irish perspective on the “Mathematics Problem”. In
Plenary lecture at Irish Symposium for Undergraduate Mathematics Education
(Vol. 2).
Project Maths Development Team (2010). Overview of Project Maths. [cited 9th
February 2015]; Available from: http://www.projectmaths.ie/overview/.
State Examination Commission (2014). State Examination Statistics: Athlone.
Treacy, P., & Faulkner, F. (2015). Trends in Basic Mathematical Competencies of
Beginning Undergraduates in Ireland, 2003-2013. International Journal of
Mathematical Education in Science and Technology, yet to be published.