CPA PROGRAM
EXAM REPORT
2014
LEADING THE WORLD IN
ACCOUNTING EDUCATION
The CPA Program is a leading professional education program in accounting, finance and business.
The success of the CPA Program is reflected in its continual growth for over 25 years. The Program is
portable, international and is based on self-directed and independent learning. It is a complete
education and practical experience program, carefully structured to provide maximum benefit to
candidates now and as they progress in their careers.
CPA Australia undertakes continuous improvement to enhance the CPA Program’s global relevance
and currency of content, and provide more ways for people to develop a career built on professional
accounting skills. CPA Australia provides entry pathways designed to offer flexible entry modes for
those who have not previously completed an accounting degree.
Recognising the complex business environment in which organisations operate, CPA Australia has a
broad education program that is designed to produce high-quality graduates with well-rounded
business and employability skills together with sound technical competencies.
The CPA Program comprises six postgraduate level subjects plus three years of mentored experience
in an accounting, finance or business role.
The syllabus for the CPA Program focuses on creating strategic business leaders with the technical
skills to meet the challenges of contemporary global business. The content of the Program is globally
relevant, with a focus on providing flexibility of learning and delivery modes. The CPA Program is
designed to meet the requirements of the International Education Standards issued by
the International Federation of Accountants. Importantly, the CPA Program continues to address the
needs of employers and leads to a highly valued and globally recognised designation. CPA Australia
continues to apply the same rigorous standards of competence as has always been required to
achieve the CPA Australia designation.
The flexible learning focus ensures that those who successfully complete the CPA Program have
much more than a solid working knowledge of accounting, finance and international business. CPAs
demonstrate initiative and strong organisation and time management skills, and are positioned to be
future leaders of the profession. They are regarded as committed and motivated self-starters who are
work-ready.
CPAs are valued as offering a unique combination of skills that drives employer demand for CPAs.
They are seen to drive business success through their technical expertise and ability to see the
bigger picture and take a strategic view of business. Graduate and employer surveys continue to
show extremely high levels of satisfaction with the skills developed through the CPA Program.
The Professional Qualifications Advisory Committee meets four times a year to provide management
with expert advice on the educational and training framework for members of CPA Australia so that
management may develop and monitor educational and training strategy relating to the professional
programs. This continual review ensures that the CPA Program remains at the forefront of
international accounting education programs.
All CPA Program learning materials are updated annually. A Learning Management System with
interactive online learning support provides candidates with additional support in their studies. In
addition, a range of additional learning resources and tools are available to complement the distance
learning materials.
CPA Program exams are a comprehensive assessment of the learning objectives for each subject
area. High standards are maintained by processes that include expert monitoring at all stages, from the
development of the exam questions to the full statistical analysis of every question and every exam
paper. The CPA Program is supported by processes certified externally under ISO 9001:2008
as meeting international quality management standards.
The CPA Program provides a challenging, rewarding and world-class program that ensures the high
standards of the profession are maintained. It is a benchmark for professional accountants and the
foundation for lifelong professional learning.
Contents
The CPA Program
Introduction
The Professional Qualifications Advisory
Committee
The objective of the CPA Program
Structure of the 2014 CPA Program
Foundation exams
CPA Program
Practical experience requirement
1
1
1
2
2
3
3
4
2014 CPA Program Foundation exams
Introduction
Retake policy
Exam enrolments
Exam venues
Exam preparation process
Exam security process
Setting the passing standard
Standard Setting using the Modified Bookmark method
Definition of a ‘minimally competent candidate’
Number and criteria of SMEs
Duration of standard setting exercise
Statistical equating methodology
Scaled scoring ranges
Equating of exam forms
5
5
6
6
7
7
8
8
8
9
9
9
9
9
10
2014 CPA Program exams
Introduction
Exam enrolments
Exam venues
Exam preparation process
Exam quality process
Statistical equating methodology
Equating of exam forms
Scaled scoring ranges
Setting the passing standard
Psychometric evaluation
Level of difficulty
Discrimination power
Distractor functioning
Reliability
Exam Policy Advisory Committee
11
11
12
12
12
13
14
14
14
15
15
15
16
16
16
17
Candidate performance
Foundation exams
CPA Program
Reporting of results
18
18
19
20
2014 committees and staff
Professional Qualifications Advisory Committee Members
Exam Policy Advisory Committee Members
Special Consideration Committee Members
CPA Program Staff
21
21
21
21
22
1
THE CPA PROGRAM
INTRODUCTION
The CPA Program commenced in 1986, and is CPA Australia’s professional education
program. Completion of the CPA Program is required for candidates to advance to CPA
status. The CPA Program is designed to provide graduates with a solid base of technical
accounting knowledge, together with the broad business knowledge and skills required
to prepare them to become the strategic business leaders of the future.
CPA Australia has strict quality assurance processes in place to ensure the CPA
Program continues to be recognised as a leading professional accounting education
program.
The Professional Qualifications Advisory Committee
The CPA Program has been developed by CPA Australia in consultation with senior
academics and industry experts to ensure that it provides a practical, up-to-date,
postgraduate approach to business studies.
To ensure the CPA Program remains at the forefront of professional business education,
the Professional Qualifications Advisory Committee advises on the educational
framework for members of CPA Australia. This ensures relevance and quality, so that
the program continues to meet the needs of members and business.
The Professional Qualifications Advisory Committee is made up of eight eminent
individuals from academia, industry, large firms and the public and not-for-profit sectors.
The Professional Qualifications Advisory Committee is chaired by Professor Kim
Langfield-Smith, FCPA, Vice-Provost (Academic Performance) Monash University. A full
list of members is provided on page 21.
Key responsibilities of the Professional Qualifications Advisory Committee are to:
• provide strategic oversight and quality assurance of the CPA Program, including
ongoing development and maintenance;
• provide advice, direction and oversight to the design, curriculum, conduct,
assessment and quality assurance of the CPA Program;
• provide advice on the development of the CPA Program, including its design,
curriculum, assessment and quality assurance so that it is positioned as the
highest-quality professional education program by professional accounting
2
•
•
•
•
bodies and the business community;
review and advise on all educational components of the CPA Program;
provide advice on the use of leading-edge technology to enhance and
complement the delivery and assessment of the CPA Program;
provide advice and recommendations in respect of the strategic position of the
CPA Program with undergraduate and postgraduate university education, the
education continuum and within the professional work environment; and
provide advice on the development of the practical experience requirement of
the CPA Program.
The objective of the CPA Program
The CPA Program provides graduates with a world-class, internationally recognised
professional qualification. The program is designed to enhance a CPA’s competitive
advantage for leadership positions in finance, accounting and business in the private
and public sectors. Distinctively professional in orientation, the CPA Program is
strategically integrated with other tertiary education in a continuum of learning for career
success. The CPA Program offers an exceptionally high standard of quality and is
recognised by ISO 9001 certification. It is a rigorous, integrated education and
experience program. The CPA Program integrates postgraduate education with practical
experience to provide opportunities to develop technical and strategic knowledge and
skills, and implement these in authentic situations.
As well as ensuring that graduate CPAs are well informed about the latest technical
issues and developments in the profession, the program is positioned within the broader
context of contemporary business. The professional level compulsory segments include
emphasis on:
• corporate governance, ethics and integrity;
• development and implementation of business strategy; and
• decision-making and leadership.
The CPA Program equips graduates with strong analytical and problem-solving
capabilities designed to add real value.
In 2014 there were over 26 000 candidates enrolled in the CPA Program, with over 52
000 individual subject enrolments per annum. The CPA Program is the largest
postgraduate program in Australia and possibly the largest postgraduate accounting
program in the world. The CPA Program is based on the core knowledge that all new
members have completed through their studies in accounting, commerce or business.
All CPA Program subjects are prepared by authors drawn from industry, commerce,
the public sector and tertiary education. Subject outlines, learning objectives and all
related materials are reviewed annually by CPA Australia technical staff and
independent panels of subject matter experts. The CPA Program utilises high-quality
open-learning educational resources including printed study material and online learning
tasks.
STRUCTURE OF THE 2014 CPA PROGRAM
The CPA Program comprises post-graduate education units together with a practical
experience requirement. Candidates are required to complete a total of six subjects,
including four compulsory and two electives.
Candidates who have not completed a recognised accounting degree are able to satisfy
core knowledge requirements by undertaking Foundation exams.
3
Foundation exams
The Foundation exams are designed to test candidates against a series of learning
objectives. The learning objectives are designed to incorporate focus on core knowledge
required for successful entrance into the CPA Program.
The average Foundation exam candidate is required to complete three exams.
The Foundation exams offered in 2014 were:
• Foundations of Accounting
• Accounting Concepts and Principles
• Financial Accounting and Reporting
• Fundamentals of Business Law
• Economics and Markets
• IT and Business Processes
• Business Finance
• Management Accounting
Candidates must complete all their required Foundation exams and advance to Associate
status within 10 years of their first Foundation exam enrolment.
CPA Program
Candidates enrolling in the CPA Program are required to pass six subjects and complete
the Practical Experience Requirement to complete the CPA Program. Each subject is one
semester in duration with a recommended 150 hours of self-study. The total study time
recommended for six subjects is approximately 900 hours.
Candidates must complete three compulsory, one capstone, and two elective subjects:
Three compulsory subjects:
• Ethics and Governance
• Strategic Management Accounting
• Financial Reporting
Two elective subjects selected from the following:
• Advanced Taxation, Singapore Taxation or Malaysia Taxation
• Financial Risk Management
• Advanced Audit and Assurance
• Contemporary Business Issues
Capstone (compulsory):
• Global Strategy and Leadership
Associate members joining CPA Australia from 1 July 2007 must complete the CPA
Program and advance to CPA status within six years of joining CPA Australia. Associates
who joined from 2004 to 30 June 2007 must complete the CPA Program within its
required timeframe and advance to CPA status within eight years of joining CPA
Australia.
Further information on the CPA Program subject offering is available on the CPA
Australia website.
4
Practical experience requirement
All candidates are required to complete three years of supervised practical experience in
a professional accounting, finance or relevant business role. The practical experience
requirement is designed to complement the theory in the professional level segments.
Candidates are required to demonstrate competence in the technical, personal
effectiveness, business and leadership skill areas. A full member of an IFAC body will
need to verify that a candidate has successfully demonstrated the required skills.
There were over 18 000 Associate members enrolled in the practical experience
requirement through 2014. These members, on average, show consistently stronger
results in the distance education units than those who are not enrolled, with more than a
70 per cent higher success rate in exams.
5
2014 CPA PROGRAM FOUNDATION
EXAMS
INTRODUCTION
Applicants for Associate membership of CPA Australia are required to undertake a
Member Entry Pathways Assessment. Each applicant is assessed against the core
knowledge requirements to commence the CPA Program. Applicants who do not meet
all the core knowledge requirements are advised of the Foundation exams that they
must successfully complete to progress to the professional level of the CPA Program.
Candidates are not obliged to complete the exams with CPA Australia. They can
complete their required core knowledge at a higher education provider and apply for a
reassessment of their qualification.
All Foundation exams were offered continuously from 28 April through until 19
December 2014. This extended exam period allows greater flexibility for candidates,
who can schedule their exams at times to suit their work and lifestyle.
exams are closed book. Each exam is of 3 hours and 15 minutes duration.
All Foundation exams consist of 100 multiple choice questions and are offered using
computer-based testing. All exams comprise a combination of scored and unscored
items, thus providing an opportunity for CPA Australia to confirm the statistical validity of
an exam item before it is scored. Questions are delivered randomly to ensure a level of
uniqueness for each exam and results are available to candidates on the day of their
exam.
Foundation exams:
• Foundations of Accounting
• Accounting Concepts and Principles
• Financial Accounting and Reporting
• Fundamentals of Business Law
• Economics and Markets
• IT and Business Processes
• Business Finance
• Management Accounting
6
Each Foundation exam has a set of learning objectives. Candidates are examined
against each of these learning objectives. The Foundation exams do not comprise a
distance education program, nor a program of study in the same mode as the CPA
Program, but are a series of exams to test candidates’ knowledge and skills in the
required core knowledge areas.
Success in the Foundation exams ensures candidates have met all the prerequisites to
enable them to enter the CPA Program with the same chance of success as those who
have come through with an accounting degree. The CPA Program builds on the
knowledge examined in the Foundation exams.
Retake policy
In 2012 CPA Australia introduced a retake policy for those candidates undertaking
Foundation exams. The retake policy limits the maximum number of times a candidate
can retake the same exam, and also the timeframes within which exam resits are
permitted. This policy was introduced for the benefit of those candidates for whom the
Foundation exams may not be the most suitable study method.
All candidates have a maximum of four attempts at any one exam, with a requirement
for a minimum of 30 days between exam sits. The required waiting period has been
implemented to ensure candidates allow adequate study time between exams.
Candidates who are not successful in their Foundation exams are provided advice on
available learning support and possible alternative arrangements to assist them in
meeting the core knowledge requirements to be able to commence the CPA Program.
Exam enrolments
Total Foundation exam enrolments are shown in Table 1.
Table 1 Foundation exam enrolments
Exam
Total 2014
enrolments
Foundations of Accounting
371
Accounting Concepts and Principles
1094
Financial Accounting and Reporting
1202
Fundamentals of Business Law
1476
Economics and Markets
223
IT and Business Processes
375
Business Finance
376
Management Accounting
630
Total:
5747
7
Exam venues
Foundation exams are conducted by Pearson Vue on behalf of CPA Australia. All exams
are conducted at computer testing centres operated by Pearson Vue or its authorised
representatives around the world. In 2014 there were a total of 5747 enrolments in
Foundation exams. The exams were offered in over 50 countries, at 165 exam locations.
Candidates are required to schedule their own exam date within the relevant exam
window, which provides flexibility for them to plan their study around work and
other commitments. Figure 1 shows the exam locations for 2014.
Figure 1 Exams delivered by country in 2014
Singapore
6%
New Zealand
2%
Vietnam
7%
Other
3%
Malaysia
4%
Indonesia
2%
India
6%
Australia
61%
Hong Kong
6%
China
3%
Exam preparation process
CPA Program Foundation exam questions are written by experienced practitioners and
academics who are specialists in each subject area. Writers are contracted by Pearson
Vue and undergo a training program designed to ensure skills in the structure, design
and analysis of multiple-choice testing items. CPA Australia is involved in the design and
management of the training sessions, together with educational experts from Pearson
Vue. Training is conducted using a variety of delivery methods including face-to-face
and webinars, thus ensuring ease of accessibility for writers all over the world.
All items are reviewed by educational experts from Pearson Vue and CPA Australia to
ensure items meet the requirements for testing learning objectives, and testing the
required levels of knowledge. Detailed ongoing analysis is undertaken to ensure all
items are performing as required during the testing process. Each exam consists of a
prescribed distribution of new and pre-tested items. All new exam items are pre-tested
to evaluate statistical reliability and performance, and these items are not included for
the purpose of a candidate’s score in the exam.
All exams are prepared in accordance with a prescribed weighting for each learning
objective.
8
To protect the integrity of exam content, great care is taken to ensure that only those
that need to see exam content have access to it. File access is restricted, printing is kept
to a minimum and the distribution of all information is tracked.
Security audit documentation is maintained to keep track of the flow of documents
between Pearson VUE, CPA Australia and subject matter experts throughout each
phase of exam development.
Exam security process
Candidates are required to present photographic and signatory identification before
taking their exams. An electronic signature and a digital photograph is captured and
compared with the identification previously provided. This biometric data is also used
when the candidate first takes a seat in the exam room. The candidate’s digital
photograph appears on the screen and the test administrator confirms the right
candidate is sitting the right exam.
Setting the passing standard
Standard Setting using the Modified Bookmark method
1
The CPA Program Foundation level uses the Modified Bookmark method for setting the
passing standard for all exams. The passing standard is set on a triennial basis, with
additional standard setting undertaken as learning objectives change for each exam.
Following the initial standard setting process, all exams are pre-equated using statistical
information related to the level of competence required and level of item difficulty.
The Modified Bookmark method is a widely used item-centered method for setting a
passing score. In this method, the standard setting process is done after the results
have been analysed. The following steps are generally carried out during this method:
1 A group of subject matter experts (SMEs) for each exam is convened, forming the
Standard Setting Committee (the Committee).
2 The items are ordered from easiest to hardest based on their location (difficulty) on
the underlying measurement scale.
3 SMEs review items, beginning with the easiest item, and decide if a minimally
competent candidate has a 0.50 probability of answering the question correctly. The
assumption is that the easy items at the beginning of the scale will be marked as
‘Yes’ and, as the judge progresses to the harder items, there will be a transition
point to the items being marked as ‘No’.
4 The SMEs are asked to place a bookmark at a transition point where the majority of
responses change from ‘Yes’ to ‘No’.
5 There are usually multiple rounds of placing the bookmark, with discussions
between the SMEs between rounds.
6 The pass score is determined by averaging the item difficulties corresponding to the
bookmark locations.
7 Once the pass score has been determined, the percentage of candidates passing
the score will be reviewed to determine the preliminary impact of the pass-score
setting exercise.
8 The result is then submitted to the Committee. If the result is not acceptable, the
Committee should nominate an acceptable pass percentage (informed by the
knowledge of the exercise) and then set a new preferred pass score.
9 The SMEs would then be asked to examine the items in the vicinity of this new pass
score to see whether or not they agree with the modified image of the minimally
competent candidate. If so, the new pass score is accepted for recommendation to
1
Additional information on this method can be found in Cizek, G. J. (ed.)
(2001), Setting Performance Standards: Concepts, Methods, and Perspectives,
Lawrence Erlbaum, Mahwah, New Jersey.
9
the Committee; if not, there would have to be a further round of discussions among
the SMEs to reach consensus on the refined image of the minimally competent
candidate.
Definition of a ‘minimally competent candidate’
The description of the minimally competent candidate is developed during the standard
setting meeting using the course Learning Objectives and test specifications. For the
purposes of this standard setting exercise, a ‘minimally competent candidate’ is defined
as the least capable candidate able to pass (at a minimum level) the CPA Program.
Number and criteria of SMEs
Each Foundation exam standard setting process requires the attendance of between six
and eight SMEs. SMEs are selected on the basis of meeting the following criteria:
1 Expertise in the subject matter.
2 Familiarity of undergraduate studies subject matter.
3 Knowledge of CPA Program professional level content.
Duration of standard setting exercise
The standard setting process for each Foundation exam is conducted over a full day.
Sessions are carried out in a face-to-face and virtual environment using WebEx and
teleconference technology.
Statistical equating methodology
As new items are seeded in exams, they are calibrated onto the existing measurement
scale according to the Rasch Unidimensional Model. Once this has been done, it is
possible to use any of the calibrated items in the bank to generate an exam and the
resulting measure will locate a candidate along the scale. The cut-score is also located
on the scale so it is possible to compare the location of the candidate to the cut-score. If
the candidate is located at or above the cut-score then they have passed; if they are
located below the cut-score, they will fail.
Scaled scoring ranges
Foundation candidates receive a pass or fail mark in addition to a candidate feedback
report, which is a graphic representation to indicate a candidate’s relative strength or
weakness against each learning objective.
In Semester 1, 2011 CPA Australia introduced the use of scaled scoring for each exam.
In addition to receiving a pass or fail result, candidates receive their score and
information disclosing the score required to pass.
Table 2 demonstrates the score scales (the lowest possible score to the highest
possible score) and the passing score required for each Foundation exam. A different
scale range is used to easily identify each exam.
10
Table 2 Exam scale ranges
Exam scale ranges
Lowest score
Required
passing score
Highest score
Foundations of Accounting
250
320
375
Accounting Concepts and
Principles
875
945
1000
Financial Accounting and
Reporting
625
685
750
Fundamentals of Business Law
375
430
500
Economics and Markets
125
190
250
0
65
125
Business Finance
500
570
625
Management Accounting
750
820
875
Exam
IT and Business Processes
The scaled score allows candidates to judge how their score relates to the pass mark.
The scale does not compare their score to other candidates, it is a unique view of their
overall performance in the exam.
Equating of exam forms
Item response theory is used to calibrate scores from two or more exam forms on the
same scale. Exam forms are drawn from the calibrated item bank and no item appears
on an exam before it has been trialled and equated to the benchmark scale. Through
equating, passing standards are set so that an equivalent level of proficiency is required
to pass different forms of the exam. Each candidate who is administered a set of exam
items receives a statistically equivalent exam: one that is neither statistically easier nor
harder than any other candidate.
All Foundation exams have been pre-equated since Semester 1, 2011. The use of
pre-equating allows candidates to receive their results on the day of their exam.
11
2014 CPA PROGRAM EXAMS
INTRODUCTION
Exams were offered for each CPA Program subject at the end of each semester. The
exam periods were between 29 April and 2 May 2014 for Semester 1, and between 18
to 24 October 2014 for Semester 2. Each exam was of 3 hours and 15 minutes duration.
The exams for most elective subjects consist of 100 per cent multiple-choice questions.
The exams for compulsory subjects in 2014 comprised a multiple-choice component
(worth approximately 70–80 per cent) and a constructed-response component (worth
approximately 20–30 per cent). The exam for Singapore Taxation consisted of
constructed-response questions only. The exam for Malaysia Taxation consisted of a
multiple-choice component (worth approximately 20 per cent) and a constructedresponse component (worth approximately 80 per cent).
The exam for the capstone subject Global Strategy and Leadership consisted of a
constructed-response component (worth 80 per cent) and a multiple-choice component
(worth 20 per cent). The constructed-response component was based on case studies
which had been made available to candidates before the exam. All constructedresponse questions in the exam are based on the case studies.
All CPA Program exams were open book and non-disclosed. Open book exams are
exams where candidates are generally permitted to bring notes and books of their choice
into the exam. As candidates have access to such material, open book exams
emphasise application and analysis-type exam questions. Exams are open book
because this more closely reflects the conditions candidates are likely to experience in
their working environment. The exam for Singapore Taxation restricts the materials which
may be brought into the exam to relevant legislation only. In non-disclosed exams,
questions and answers are confidential and they are not published after the exam results
are made available.
12
Exam enrolments
The total number of exam enrolments for the CPA Program in 2014 was 52,864. Exam
enrolments by country are shown in Figure 2.
Figure 2 Exam enrolments by country in 2014
SINGAPORE
6%
NEW ZEALAND
1%
VIETNAM
1%
OTHER
2.5%
MALAYSIA
9%
HONG KONG
6%
FIJI
0.5%
CHINA
6%
AUSTRALIA
68%
Exam venues
CPA Program exams are conducted by Pearson Vue on behalf of CPA Australia. In
2014 exams were offered in both paper-based and computer-based testing modes.
Pearson Vue is required to meet CPA Australia’s strict quality requirements in relation to
the selection and use of exam venues. All computer-based exams are conducted at
computer testing centres operated by Pearson Vue or its authorised representatives
around the world. Candidates are required to schedule their own exam date within the
relevant exam window, which provides flexibility for them to plan their study around work
and other commitments. In order to ensure the security and integrity of the exams all
CPA Program subjects offered throughout the exam window. now have multiple exam
forms, with each form containing randomly selected exam items,
Exam preparation process
CPA Program multiple choice exam questions are written by experienced practitioners
and academics who are specialists in each subject area. Writers are contracted by
Pearson Vue and undergo a training program designed to ensure skills in the structure,
design and analysis of testing items. CPA Australia is involved in the design and
management of the training sessions, together with educational experts from Pearson
Vue. Training is conducted using a variety of delivery methods including face-to-face
and webinars, thus ensuring ease of accessibility for writers all over the world.
13
All items are reviewed by educational experts from Pearson Vue and CPA Australia to
ensure items meet the requirements for testing learning objectives, and testing the
required levels of knowledge. Detailed ongoing analysis is undertaken to ensure all items
are performing as required during the testing process. Each exam consists of a
prescribed distribution of new and pre-tested items. All new exam items are pre-tested to
evaluate statistical reliability and performance, and these items are not included for the
purpose of a candidate’s score in the exam.
All exams are prepared in accordance with the prescribed weighting for each module
(part of a subject).
To protect the integrity of exam content, great care is taken to ensure that only those
that need to see exam content have access to it. File access and printing are restricted,
and the distribution of all information is tracked.
Security audit documentation is maintained to keep track of the flow of documents
between Pearson VUE, CPA Australia and subject matter experts throughout each
phase of exam development.
The constructed response questions in the exams for compulsory segments, Ethics and
Governance, Strategic Management Accounting and Financial Reporting, and the
capstone subject Global Strategy and Leadership, are prepared by academics and
practitioners, and reviewed by a panel of experts. Exams for Singapore Taxation and
Malaysia Taxation also utilise constructed response questions, with Singapore Taxation
comprising 100 per cent constructed response questions. These exams are prepared by
practitioners and academics who are specialists within the relevant taxation systems. All
exam papers are reviewed by a minimum of two expert members.
Exam quality process
Answers to multiple choice exam questions are marked by a computerised system.
Comprehensive audit procedures ensure the accuracy of the scanning process and
comprehensive statistical review techniques are employed to identify any anomalies in
the results.
Solutions to the constructed response section of the Ethics and Governance, Strategic
Management Accounting, Financial Reporting, and Global Strategy and Leadership
exams are developed by a panel comprising CPA Australia members (all of CPA or
FCPA status) and experts drawn from academia, commerce, industry and practice. A
detailed marking scheme is created for each constructed response question. This
process is lead by the Chief Examiner for each subject, together with the members of
the senior marking team.
The constructed response answers in Ethics and Governance, Strategic Management
Accounting, Financial Reporting and Global Strategy and Leadership are marked using
an online marking system, ePen, which is an image-based performance scoring system
that uses the internet to transmit electronic images of candidate responses to trained
markers at individual workstations. The marking system manages all aspects of the
performance assessment scoring process electronically, increasing efficiency and
flexibility.
Candidate paper exam books are scanned and images of all constructed response
items are captured and stored in an electronic file. Candidate responses captured
through computer based tests are delivered to Pearson Assessment Services for
processing through the same process as the paper based responses. The images are
routed to markers, who read and evaluate the responses in terms of the scoring rules
and marking grids developed for the assessment.
CPA Program exam markers are selected from the membership, and are of CPA or
14
FCPA status. All markers undergo detailed technical and systems training prior to the
commencement of the marking process.
The use of the online marking system provides for real-time monitoring of the
performance of all markers. Detailed quality assurance processes ensure that the work
of all markers is reviewed throughout the marking process by members of the senior
marking team. Additionally approximately 25 per cent of all papers are double marked to
ensure accuracy of marking.
The exam for Singapore Taxation was marked by a panel comprised of senior tax
specialists with experience in the Singapore tax profession.
Statistical equating methodology
As new items are seeded in exams, they are calibrated onto the existing measurement
scale according to the Rasch Simple Logistic Model. Once this has been done, it is
possible to use any of the calibrated items in the bank to generate an exam and the
resulting measure will locate a candidate along the scale. The cut-score is also located
on the scale so it is possible to compare the location of the candidate to the cut-score. If
the candidate is located at or above the cut-score then they have passed; if they are
located below the cut-score, then they have failed.
Equating of exam forms
In Semester 1 2014 CPA Australia introduced the use of multiple exam forms for each
subject in the CPA Program. Each exam form is designed with a unique set of questions
so as to minimise the risk of exam items being overexposed, and to ensure equity for all
candidates sitting exams at different times and locations. The use of multiple forms
allows individual exams to be offered over a window of time, rather than requiring each
exam to be offered on a set date. The distribution of different exam forms throughout the
testing window reduces the risk of item exposure and maintains the security of the
exams.
In order to ensure that each candidate who is administered a set of exam items receives
a statistically equivalent exam: one that is neither statistically easier nor harder than any
other candidate, an equating process is used. Item response theory is used to calibrate
scores from two or more exam forms on the same scale. Exam forms are drawn from
the calibrated item bank and no multiple choice item appears on an exam before it has
been trialled and equated to the benchmark scale. Exams which include constructed
response questions are equated using a partial-credit equating model which allows item
difficulty to be equated across exam forms. Through equating, passing standards are set
so that an equivalent level of proficiency is required to pass different forms of the exam.
Additionally, all CPA Program exams are equated between semesters to ensure that
standards are maintained in semesters where there has been no structural change to
the educational material. Generally, equating can be viewed as a means to ensure that
the passing score for the October 2014 exam paper represents the same ability as the
passing score in the May 2014 exam paper.
Scaled scoring ranges
In Semester 1, 2014 CPA Australia introduced the use of scaled scoring for all CPA
Program exams. The Scaled Score range for all CPA Program exams is 100 to 900, with
100 being the lowest possible score, and 900 being the highest possible score with all
items answered correctly. The passing scaled score for all exams is 540. In addition to
receiving their result, candidates receive their scaled score and information disclosing
the score required to pass. Additionally, candidates receive a feedback report, which
uses a graphic representation to indicate a candidate’s relative strength or weakness
against each module in the subject.
15
The scaled score allows candidates to judge how their score relates to the pass mark.
The scale does not compare their score to other candidates, it is a unique view of each
individual’s overall performance in the exam.
Setting the passing standard
Since 1994, CPA Australia has used the Angoff method to establish passing standards
for professional level exams. The Angoff method is widely used by many professional
bodies in determining pass standards. The Angoff method requires a panel of experts to
estimate a target level of proficiency (pass mark) for a set of exam questions or for a
given exam form.
A separate panel is used for each CPA Program subject. Panel members must be
knowledgeable in the specific subject area, and about the candidate population in
general, in order to form an assessment of the target level of proficiency required.
Panels are comprised of experts drawn from academia and the profession.
In light of the knowledge, skills and abilities required of candidates who achieve the
target level of proficiency, panel members are asked to individually review each
question. In reviewing the question, panel members must ask themselves what
percentage of adequately competent candidates should answer the question correctly.
The target level of competence from an individual panel member is an average of their
identified individual question difficulties and the exam as a whole. The proficiency level
is determined by statistical analysis and expert recommendation of the appropriate level
of proficiency or standard. This ensures that candidates are graded fairly from one
semester to another. The experts are representative of the membership and the
profession; consequently, the expectations are high.
Psychometric evaluation
The psychometric quality of the CPA Program exams is evaluated using item response
theory, using the Rasch simple logistic model for multiple choice questions, and the
partial credit model for constructed response items. The analysis examines overall item
performance, the relationships between candidate performance on exam questions and
their exam scores, and the relationships among candidate scores on the exam
questions themselves. The goal of the psychometric evaluation is to determine how well
each exam has differentiated between candidates of different abilities and to identify
those questions that should be reused, rewritten or discarded.
In the psychometric evaluation of multiple-choice questions, many psychometric
characteristics can be evaluated. Three of the more common are:
• level of difficulty;
• discrimination power; and
• distractor functioning.
Level of difficulty
The level of difficulty of multiple choice questions is determined by measuring the
percentage of candidates answering the question correctly. A question with moderate
difficulty will better distinguish between candidates of different abilities.
16
Discrimination power
In a four-option multiple-choice exam, questions should be asked so that the candidate
scoring higher grades is more likely to answer the question correctly than a candidate
receiving lower grades.
The categories of discrimination power are described below.
Discriminating questions
Candidates receiving higher total exam scores are more likely to get the correct answer
than candidates with total lower scores.
Non-discriminating questions
Candidates with higher total exam scores are neither more nor less likely to get the
correct answer than candidates with lower total scores.
Reverse discriminating questions
Candidates with total lower exam scores are more likely to get the correct answer than
candidates with higher total scores.
The discrimination power of a question is measured by the point biserial correlation
coefficient. The discrimination power of each multiple-choice question is measured by
correlating candidates’ aggregate scores to the answers for each option. Ideally, a
question’s point biserial correlation coefficient should be 0.20 or above.
Distractor functioning
Ideally, all the incorrect answers (distractors) of a four-option multiple-choice question
should be plausible (functioning) to a candidate of lesser ability. A distractor is
commonly deemed to be not functioning when fewer than 1 per cent of candidates
select that distractor.
Reliability
The reliability of exam scores is calculated to determine the degree to which candidates
are likely to earn similar grades on exams with different but comparable questions
covering the same subject matter. The coefficient alpha is a measure of the internal
consistency or homogeneity of the scores on a test. The reliability coefficient can range
in value from 0 to 1, and it is desirable for the level of reliability to be greater than 0.80.
CPA Program exams have consistently been highly reliable over many years. This is
due to the exam questions being written and refined by experts, the use of expert
judgment to inform proficiency standards, and the re-testing of exam questions for
equating purposes.
Table 3 shows the reliability levels of the CPA Program segment exams.
17
Table 3 Reliability
CPA exams
May 2014
October 2014
Advanced Audit and Assurance
0.88
0.88
Advanced Taxation
0.84
0.85
Contemporary Business Issues
0.83
0.83
Ethics and Governance
0.86
0.88
Financial Reporting
0.83
0.83
Financial Risk Management
0.85
0.86
Global Strategy and Leadership
0.66
0.64
Malaysia Taxation
0.71
0.68
Strategic Management Accounting
0.79
0.80
Exam Policy Advisory Committee
The Exam Policy Advisory Committee provides advice to CPA Australia on CPA
Program exam matters. It is composed of senior CPA Australia members with wide
experience in education and independent external experts in assessment, measurement
and higher education.. The Exam Policy Advisory Committee considers:
•
•
•
•
•
•
•
the structure and design of the assessment policy for all CPA Program exams;
the procedures used for the development of exam questions and the setting of
CPA Program exam papers;
the standards of and the location of exam centres used in the CPA Program;
the methods used and the security provisions implemented for the printing,
distribution and collection of CPA Program exam papers;
the procedures used for the selection of, and information provided to, exam
supervisors;
the procedures used and the timelines for the CPA Program exam grading
process; and
the statistical analysis provided on CPA Program candidate performance.
18
CANDIDATE PERFORMANCE
Foundation exams
Candidate results for Foundation exams are provided as pass or fail only. Table 4 shows
the percentage of candidates passing each exam in 2014.
Table 4 2014 exam results
Exam
Fail %
Pass %
Foundations of Accounting
61
39
Accounting Concepts and Principles
40
60
Financial Accounting and Reporting
45
55
Fundamentals of Business Law
32
68
Economics and Markets
46
54
IT and Business Processes
34
66
Business Finance
32
68
Management Accounting
27
73
Candidates sitting Foundation exams receive their results on the day of their exam.
Each candidate receives a personal report showing their performance on the scale for
the relevant exam. Candidates also receive a report showing their individual
performance against each learning objective.
19
CPA Program
In accordance with CPA Australia Professional Qualifications Advisory Committee
policy, the following grades were used:
• Fail
• Pass
• Credit
• Distinction
• High Distinction.
Table 5 shows the performance of candidates by grade for each CPA Program
professional level segment for each semester in 2014.
Table 5
Semester 1, 2014 exam results
Total exam
sits
(26 009)
Advanced Audit and Assurance
HD
%
D
%
C
%
P
%
F
%
2 111
8.0
8.1
11.1
40.6
32.2
Strategic Management Accounting
4 499
6.5
7.7
12.4
41.2
32.2
Advanced Taxation
2 288
7.1
7.7
15.4
38.8
31.0
1 757
6.8
9.1
11.6
38.5
34.0
4 673
4.2
5.1
8.6
36.1
46.0
1 801
9.2
8.9
14.2
40.4
27.3
Global Strategy and Leadership
3 590
7.0
8.7
19.3
41.7
23.3
Ethics and Governance
5 143
5.5
8.4
13.7
39.1
33.3
72
0.0
6.9
11.1
54.2
27.8
75
0.0
1.3
14.7
37.3
46.7
D
%
C
%
P
%
F
%
Financial Risk Management
Financial Reporting
Contemporary Business Issues
Singapore Taxation
Malaysia Taxation
Semester 2, 2014 exam results
Total exam
sits
(26,855)
Advanced Audit and Assurance
HD
%
2,101
7.0
7.3
11.3
38.9
35.5
4,313
7.5
7.0
10.3
40.9
34.3
Advanced Taxation
2,228
6.3
8.6
15.1
39.0
31.0
Financial Risk Management
1,807
6.4
6.1
12.3
40.0
35.2
Financial Reporting
4,881
4.9
4.3
8.2
36.1
46.5
Contemporary Business Issues
1,893
8.8
9.2
13.7
39.9
28.4
Global Strategy and Leadership
3,669
5.7
11.4
14.2
42.1
26.6
Ethics and Governance
5,786
5.0
8.2
14.8
36.8
35.2
Singapore Taxation
80
2.5
7.5
2.5
47.5
40.0
Malaysia Taxation
97
0.0
5.2
14.6
34.4
45.8
Strategic Management Accounting
20
Reporting of results
Candidate results were released through the CPA Australia website Official confirmation
of results is sent by conventional mail. Candidates who sit an exam receive a personal
candidate Feedback Report which provides their grade, their scaled score, and provides
a graphical representation of their performance against each module of the study
materials.
21
2014 COMMITTEES AND STAFF
Professional Qualifications Advisory Committee
Members
Associate Professor Foo Yin Fah FCPA
Associate Professor Bryan Howieson FCPA
Professor Zahirul Hoque FCPA
Mr Steven Hui FCPA
Professor Kim Langfield-Smith FCPA (Chair)
Mr Warwick J. Spargo FCPA
Mr Srini Vasan FCPA
Professor Kim Watty CPA
Exam Policy Advisory Committee Members
Mr Ken Devos CPA
Dr Christine Jubb CPA (Chair)
Professor Louise Kloot FCPA
Associate Professor Denis Vinen FCPA
Special Consideration Committee Members
Dr Mary Dunkley FCPA
Mr Greg Ellis CPA
Ms Jane Hamilton CPA
Ms Dianne Harvey CPA (Chair)
Ms Fay Kairn CPA
CPA Program Staff
Mr Richard Brown — General Manager Professional Programs & Pathways
Ms Desley Ward—Education Manager
Ms Kristy Grady—Professional Level Manager (maternity leave)
Ms Alexandra Wilson – Professional Level Manager
Ms Ange Rice—Foundation Level Manager
Ms Alisa Stephens—Quality Management Consultant
Ms Sarah Chinnick – Education Adviser
Ms Charlene D’Rosario—Professional Level Coordinator
Ms Jan Haverfield—Administration Executive
Ms Alex Lawrence, Foundation Level Coordinator
Ms Elise Literski—Professional Level Coordinator
Ms Mary Shevlin—Foundation Level Coordinator (maternity leave)
Mr Rohit Singh—Professional Programs Systems Coordinator
Ms Vammy Trinh – Administration Executive
Ms Belinda Zohrab-McCnnell—Technical Advisor