1 INVESTIGATION OF STUDENT COMPLAINTS AGAINST CENTRAL QUEENSLAND UNIVERSITY March 2007 Emeritus Professor W.G. Carson. 1. Background. On 16 March, 2007, I was asked by Dr. Terry Stokes, General Manager, Higher Education and Regulation Division, Department of Education and Training, to investigate allegations made against Central Queensland University’s Melbourne Campus (CQU-MIC) by a group of students who had presented themselves at 2, Treasury Place on Thursday 15 March, 2007. These complaints principally revolved around the results of an examination in Management Accounting (ACCT 20053) that had been held on February 19, 2007, but also extended to a number of other largely examination and assessment related topics. A copy of these complaints is attached as Appendix 1, as is a copy of the student complaint to CQU, itself (Appendix 2). A copy of the Examination paper is also attached (Appendix 2a). Over the weekend of 17-18 March, I conducted a preliminary website search in relation to these complaints and with Dr. Stokes’ agreement, secured the services of Mr. Michael Scorgie, Manager, Academic Development, Department of Accounting and Finance, Monash University, to assist me in the investigation. On 19 March, in consultation with Mr. Scorgie, I prepared an extensive list of questions to be put to CQU based on the students’ documentary complaint to the Higher Education Regulation Division and this document is attached as Appendix 3. It was also sent to CQU. On 20 March, Mr. Scorgie and myself, accompanied by Ms. Ewa Kuriata, Acting Manager of the Higher Education Regulation Division of the Department of Education and Training, visited the Melbourne Campus of CQU where we met with the following staff: Mr Darryl Stewart, Campus Director, CQU Melbourne International Campus Professor David Hamilton, Associate Director, Teaching and Learning and Acting Associate Director Academic, CQU Melbourne International Campus Professor Gerry Anderson, Acting General Manager Academic Programs, Divisional Office, Campus Management Services, CQU Melbourne International Campus The meeting was also joined by the following CQU officers by conference telephone from Rockhampton: Professor Gus Geursen Executive Dean, Faculty of Business and Law, CQU 2 Mrs Beth Tennent Acting Associate Dean (Teaching and Learning), Faculty of Business and Law, CQU Mr Phil Lignier, Lecturer in Accounting and Tax, Faculty of Business and Informatics, CQU The meeting lasted for some two hours and during its course the issues itemised in Appendix 3 were addressed seriatim, CQU having prepared written responses to each of my queries (Appendix 4). We were also given a copy of the CQU response to the students prepared by Professor David Hamilton (Appendix 5). CQU also provided additional documentation including the following: The examination question at issue. The Course Study Guide. The Course Profile Document. Extracts from the set textbook dealing with by products and the allocation of joint costs. Slides from relevant lectures. The weekly tutorial schedule with relevant textbook references. Slides from an audio revision lecture. The solution and marking guide for the question under dispute. These documents and a number of others were closely scrutinised by myself and Mr. Scorgie, and they are available for inspection if required. On 21 March, Ms Kuriata and I spent some three hours at 2, Treasury Place with a delegation from the complainant students. They were articulate, although still obviously distressed, and they elaborated on the substance of their complaints. They also provided a series of documents which have been examined and which will either be attached as appendices to this Report or retained for scrutiny if required. 2. The Complaints (see Appendix 1). (a) The students claimed that Question 2 in Part C of the examination for Issues in Management Accounting (ACCT 20053) held on 19 February, 2007, had been unfair because it had not been covered in the syllabus. Worth 13 out of 70 marks for the examination, they maintained that this question, which involved calculations using the theory and practice of by-products in working out Net Realisable Value and Joint Costs Allocation, had not been covered in tutorials, was not mentioned in the relevant course materials, was not included in the tutorial or review questions upon which they had been told to concentrate, and was not covered in the Review lecture given by the Course Co-ordinator to help them prepare for the examination. They alleged that allocation of a full 13 marks for this question would have averted the failure of many students in the examination (178 out of 400 candidates failed to reach the pass-mark, 122 of whom were graded SE (Supplementary Examination) and advised to re-enrol in the same course for this semester. 3 (b) The students asked for an extension of the date by which they must confirm enrolment to 26 March as the then designated date of 16 March did not allow time for the review of marks system to be completed. (c) The students complained that they have been denied access to Conceded Pass and Terminating Pass grades, on the grounds that these are restricted to undergraduates, and that Conceded Passes are not acceptable to CPA. (d) The students complained that examination irregularities had occurred before, notably in ACCT 20054, 2006, when students had to be allocated full marks for a question that had contained insufficient information to permit an answer. They claimed that, in this instance, when they pointed out the problem during the examination, the ‘lead lecturer’ told them the information was sufficient, only for them subsequently to receive an e-mail saying this had not been so and that the extra marks would be allocated. The examination for Maths 20247, 2006, was cited as another example when similar compensation was given for ‘a wrong question.’ (e) The students complained about Supplementary Examination policies because they do not take account of assignment marks and are held at the end of semester (presumably the one following that in which the course had been offered, if not examined). (f) The students complained that continuous ‘changes of grade’ take place, particularly in relation to marks allocated for assignments. (g) The students also complained about the University’s review of grades policy and process, which they see as being unnecessarily protracted, in order to secure additional fees from re-enrolment. (h) The students complained about the standard of teaching, claiming that lecturers and tutors only ‘go through the slides’ and seldom have answers to questions posed by the students. (i) The students claimed that study materials, such as the Study Guide for Taxation (LAWS 20023), for example, were not up to date. (j) Library facilities were said to be inadequate. (k) It was claimed that the process of ‘Step Marking’ for practical questions is not being followed. (l) Examination dates were claimed to be too compressed and much too close upon completion of syllabus coverage. In the discussions held with students on 21 March, they elaborated upon, and in some respects added to, this list of complaints: (m) The students conceded that they had been given some theoretical instruction with regard to by-products, but that no practical experience had been offered. 4 (n) The calculation of Net Realisable Value had been taught but not the practical aspects of by- products in Cost Allocation. (o) The English language proficiency of tutors was said to be inadequate. (p) In the review of grades process the students claimed that they have to talk to administrative staff rather than academic staff. (q) One named teacher has a continuing record of high failure rates across the different courses in which he has carried responsibility. Despite these elaborations and additions, the students agreed in their meeting with myself and Ms. Kuriata that, given certain exigencies extraneous to academic considerations, investigative priority should be given to three issues: Was the examination unfair, in that the substance of Question 2 (1), Section 3, had been inadequately covered in the syllabus and the teaching program? Should all students given an SE grade (and presumably all failing students, if not indeed all students) therefore be allocated the full 13 marks for the question at issue? Had ‘step marking’, marking each section of the question disregarding any consequential errors arising from inaccurate calculation of Part 1 of Question 2, Section 3, been employed? Should PC and PT grades be available to students awarded SE grades? 3. Discussion and Findings. (a) Was the examination unfair, in that the substance of Question 2 (1), Section 3, had been inadequately covered in the syllabus and the teaching program? Close examination of the relevant course materials revealed that the issue of by- products and cost allocation was addressed during the course. At page 6.3 of the Course Guide, one of the objectives of Module 6 on Cost Allocation is stated as being to ‘identify the characteristics of the joint cost process and identify the accounting issues regarding joint products and by – products’. The students were referred to Chapter 7 of the textbook, Hansen and Mowen, Cost Management: Accounting and Control, 2006. At page 298 of that chapter, there is a clear distinction drawn between joint products and by-products, including a succinct description of the ways in which by-products may be accounted for. At page 6.7 of the Course Guide, the issue of joint products and by-products is once again clearly addressed, with a clear explanation of the two ways of accounting for the latter. At page 5 of the lecture slides, provided to me by CQU, under the heading of Accounting for Joint Production Processes, a clear definition of by-products is provided. The audio-presented Revision Lecture, given by the Course Co- 5 ordinator shortly before the examination stated that ‘[I]ssues covered during the lectures, or in the study guide, are more likely to be examined’ and ‘practical questions will focus on areas which have been illustrated by lecture examples or covered in review questions and tutorial questions’(Slide 1). Under the heading of ‘Practical Questions’, the relevant slide indicated that ‘Question 2, worth 13 marks will be either on Cost Allocation (Mod.6.), Budgeting or Variances (Mod.7). In the event, the question was about cost allocation. Listing ‘Major Issues’ in the same preparatory revision lecture, the Course Co-ordinator’s slide includes: ‘Allocating costs in a joint product situation using different methods: Physical, Value at split-off, NRV and Constant gross Margin methods. What is a by-product? How are by-products valued?’ I believe it to be unfortunate that more attention was not paid to the practical dimensions of by-product calculations – investigation of review questions and tutorial questions did not appear to reveal any directly on this topic, and what, if any, examples were used in lectures it is not possible to determine. On the expert advice available to me, however, it would appear that the course materials included enough information for the competent student to have been able and ready to answer this question. I am further informed that all the information necessary to answer the question was contained in the question itself, a sine qua non of fairness in questions of this kind. The following extract from a letter written to one of the students by Professor David Hamilton of the Melbourne Campus of CQU on 13 March, 2007, (Appendix 5) emphasises these points: In this question, I am told, students were required to calculate the Net Realisable Value of the by-product, and to do this students are required to understand the meaning of NRV, which has been dealt with in this and other accounting courses that students will have taken in the programme. All the information needed to calculate NRV was provided in the question on the exam, and further, students were advised how to treat the NRV of the byproduct in the question itself. I am also reminded that it is quite customary to include a difficult question of this kind (the students referred to it as a trick question) to help in differentiating those candidates who have attained a full conceptual, and consequentially practical understanding of an issue, from those who have approached the subject through rote learning and an expectation of formulaic questions. I therefore conclude that the student allegation of unfairness, by the inclusion of Question 2 on the examination paper, cannot be sustained, though CQU might usefully be rather more meticulous in the notoriously sensitive matter of the advice given to students about the likely contents of upcoming examinations. This issue may be related to the structural and 6 operational relationships between CQU Rockhampton and its satellite campuses, and I comment further on this matter in Section 4, below. (b) Should all students given an SE grade (and presumably all failing students, if not indeed all students) be allocated the full 13 marks for the question at issue on the grounds that ‘step marking,’ marking each section of the question disregarding any consequential errors arising from inaccurate calculation of Part 1 of Question 2, had not been employed? As part of my investigation I requested, and received, a sample of 20 answers (randomly selected save for one ‘perfect’ answer provided as a model) by Melbourne students graded SE for the question at issue. Analysis of these answers and the marks allocated to them according to the marking guide revealed that although all twenty (save for the one model answer that achieved 13 marks) did indeed fail to achieve 50% of the marks available for the question, failure to answer part 1 (referring to the Net Realisable Value of the By-Product) correctly had not precluded students from gaining marks for subsequent parts of the question. Five of the sampled answers gave the correct answer to Part 1, thereby gaining the stipulated 1 mark, and went on to score total marks ranging from 1 to 5.5. In two cases the mode of allocating marks could not be established (itself a matter that should be of concern to CQU with its clear Solution and Marking Guide), but both scored 2 marks, indicating that failure to get over the 1 mark hurdle for Part 1 had not been a fatal impediment to answering the remainder of the question. Of the twelve candidates who scored zero on Part 1, the total marks for the question were as shown below: Two students scored 1 mark. One student scored 1.5 marks. Two students scored 2 marks. One student scored 3 marks. Two students scored 3.5 marks. One student scored 4.5 marks. One student scored 5 marks. Two students scored 6 marks. Although by no means a distinguished performance, this sample of the marks awarded indicates quite clearly that failure to answer Part 1 of the question correctly did not preclude candidates from earning marks on subsequent parts of the question. Although the idiosyncratic marking practices of individual markers can never be quantified, this would seem to indicate that, as indicated to me by CQU and advised as normal practice by the experts whom I consulted, a line is drawn at an appropriate point under effects of the original calculational error in Part 1, and the remainder of the question is then marked on the basis of appropriateness of methodology using the student’s own, albeit inaccurate calculations at Part 1. 7 I therefore conclude that the full 13 marks should not be allocated to all failed, SE or other students, an appropriate method of marking appearing on the balance of probabilities to have been employed. (c) Should Pass Conceded (PC) and Pass Terminating (PT) grades be made available to students awarded Supplementary Examination (SE) grades? In the written complaint and during the meeting held with students on 21 March, the question of PC and PT grades was raised. According to the students, the Student Handbook indicates that these grades should be available to candidates who meet the necessary criteria. Applying what might be called ‘the reasonable student rule,’ I asked a member of the Higher Education Regulation Division staff to access the CQU website in my presence with one of the CQU students present, in order to investigate this matter. The search sequence taken was as follows: 2007 Student Handbook Degrees Postgraduate Business and Tourism Master of Professional Accounting Important Information Assessment Routed, it should be noted, through the link for postgraduate degrees, this process led to the document: Student Handbook: Important Information / Assessment (Appendix 6). This document refers to PC and PT grades, explaining the conditions under which they might be available and stating that these conditions might be further restricted by a Faculty Education Committee, or more closely defined in the application for a Faculty award. A link to the Faculty of Business and Informatics was provided and, at a subsequent meeting with CQU on 22 March, I asked that this link should be opened. When it was, it led to a document entitled: Faculty of Business and Informatics Policies and Procedures: PC/PT Grades Operational Procedure (Appendix 7). The policy set out in this document could indeed be interpreted as implying that such grades are, by default as it were, available only to undergraduate students. Nowhere, however, is the exclusion of postgraduate students, coursework or other, referred to. It could also, therefore, be contended by the reasonable postgraduate student who has followed the relevant postgraduate links that the restriction did not apply to her or him. In interview, CQU officers appeared to concede that it would be advisable to refer explicitly to the exclusion of postgraduate students in this document. 8 CQU officers explained that the policy on PC/PT grades had been changed in July 2006, in such a way as to apply only to undergraduate students, and that this information had been e-mailed to all students in the Faculty (Appendix 8). Moreover, the application form for such a grade requires a student declaration that the course in question is not a postgraduate one (see Appendix 7). The reasons for this change in policy are not explained in the documentation. In an e-mail to one of the complainants, however, the Campus Director indicated that the PC grade: ‘is not available to CQU Masters of Accounting students as this would mean the qualification is not available for CPA’ - Certified Practising Accountants (Appendix 9). This information is incorrect, as attested by a letter from CPA Australia produced to me by the students (Appendix 10). This letter states inter alia that ‘(t)he number of conceded passes acceptable in a postgraduate degree or conversion course is restricted to one unless a more advanced subject in the area has been passed.’ The Campus Director’s error, notwithstanding, the grading policy of the University was indeed changed in mid-2006, the changes in question being approved by the Academic Board on 26 July of that year. As finally amended by the Executive Committee of the Academic Board on 21 February, 2007, the new policy (Appendix 11) states that ‘the grade of PC is generally not awarded in Level One courses or in any postgraduate coursework, or research award programs, and is usually confined to undergraduate courses of study’ (emphases added). A similar statement is made with reference to PT grades, and in both instances the policy states that: ‘Faculties will determine and publicize the range of marks which satisfies their requirements’ for a PC or PT. Returning to the Faculty of Business and Informatics, this process of publicization appears to have taken two main forms. One was the e-mail circulated in mid-2006 and the other was the, at least potentially, misleading document reached through the Student Handbook and the Business and Informatics trail on the website, referred to above (Appendix 7). It was also contended by CQU that students were reminded of these rules at enrolment and in informal consultation sessions. It might also be asserted, furthermore, that instead of following the web link from Student Handbook: Special Information: Assessment into the Business and Informatics document, as described above, the reasonable postgraduate student might avail herself or himself of another link included in the Assessment Document (Appendix 6). This is a link that the document claims, immediately after the listing of grades, including PC and PT, will provide ‘definitions and further information on the Assessment Policy –Common Grading System located in the Academic Policy and Procedures Manual at: http://www.cqu.edu.au/policy.’ This site does not, as far as I can ascertain, list any Common Grading Policy under Assessment, and it is only by searching through other likely headings that the policy currently in place can be located under the title Grades and Results Policy. It should also be noted that, in any event, the policy in place since mid2006 only states that the grades of PC and PT will not generally be available to postgraduate students. 9 In light of the foregoing, I am inclined to the view that the Faculty of Business and Informatics at CQU, Melbourne International Campus, has not taken adequate steps to formally notify and publicise among its students the unavailability of PC and PT grades to postgraduate students. Communication by e-mail is not in my opinion a reliable means of conveying notice of formal policy changes, and what is said at enrolment or in informal consultation remains unascertainable. Even if such steps were deemed acceptable, consequential, explicit and readily accessible, changes to web site entries should be made. Yet, at least in my opinion, the reasonable postgraduate student would not easily locate the new policy that replaces the Common Grading Policy through the web link provided in the Assessment Document; and the information provided by the Business and Informatics link is open to misinterpretation. In these circumstances I recommend that CQU be requested to consider for PC/PT grades those Masters of Professional Accounting students at the Melbourne International Campus who in all other respects would qualify for such consideration. I further strongly recommend that, for the future, the Faculty of Business and Informatics make the policy excluding postgraduates from such consideration quite explicit in its PC/PT Grades Operational Procedure document (Appendix 7). Beyond the three more urgent questions already addressed, it has only been possible within the available time-frame for this investigation to investigate some, but not all of the students’ other complaints. (d) Extension of Enrolment Date. CQU has provisionally enrolled all SE graded students in ACCT 20053 for Term1, 2007, and the University has advised them to commence the course forthwith. An initial extension of the date for confirmation of enrolment to 27 March has now been further extended to 30 March, before which date it is understood it will not be necessary for any provisionally enrolled student to pay fees. (e) Previous Examination Irregularities. The response from CQU in relation to allegations of previous irregularities in examinations ACCT 20054 (2006) and MATHS 20247 (2006) can be found at points 2, 3 and 11, of Appendix 4. While differing in some respects from the accounts offered by the complainant students, these responses do not appear to warrant further comment, save for reiteration of the obvious injunction that all possible steps should be taken to ensure that examination irregularities do not occur. Neither incident creates a precedent for the present case and, therefore, neither incident has any bearing on the question of the allocation of marks for the examination ACCT 20053 (2006). 10 (f) Supplementary Examination Policies. It is understood that CQU is attempting to grapple with the apparent logistical problems associated with bringing Supplementary Examinations forward. It may also be noted that such a development could place additional work pressure upon SE students who are not in their final semester. (g) Currency of Course Materials in LAWS 20023. I am unable to reach a firm determination on the detail of the complaint about course materials for this course. CQU officers pointed out that the textbook, The Australian Taxation Study Manual, is up-dated in every new tax year, with the inevitable result that for part of the teaching year, the previous year’s edition will be in use. The edition shown to me was the 2006 edition, which I assume will remain in use till an up-date occurs in mid-2007. On page 6 of the Course Profile for this course, students are referred to a number of texts, including The Australian Tax Casebook, 2004, co-authored by Stephen Barkoczy et al. This was the most current edition of the book available in 2006, as a new edition did not appear until 2007. However, the book of Resource Materials supplied for this course refers the students throughout to the 5th edition of this book, with page references to cases. This 5th edition was published in 2000, subsequent editions having been published in 2002 and 2004. I have not been able to check what substantive difference there may be between the different editions of this book, but there is clearly some foundation to the students’ claim of lack of synchronization. It would appear that whoever prepared the Resource Materials did not utilise the most up to date version of the relevant text. What difference this would make to the case law presented to the students it is not possible to say, since the relevant Facts, Arguments, Issues and Decisions are reprinted in the Resource Materials from the text. At worst, the newer edition of the text might contain cases which supersede, or materially elaborate, upon the case law presented to the students. This is a matter that should be attended to by the lecturers responsible for this course. (h) Library Facilities. The library facilities at the CQU Melbourne Campus, including those for the Masters of Professional Accounting Degree were inspected in November 2006 and were found to be adequate. A further inspection of these facilities could be undertaken as part of any subsequent investigation to look into matters not covered in this Report due to limitations of time. It should be noted, however, that the student complaint refers, among other things, to the unavailability of textbooks. CQU officers pointed out that students in ACCT 20053 (2006) were forewarned that they would have to purchase textbooks but many did not appear to have done so. 11 Only 88 copies of the Cost and Management Accounting textbook had been purchased in 2006, although there were 400 students enrolled. Even allowing for second-hand purchases, it was suggested, these figures seemed to indicate failure to acquire a basic text without which satisfactory completion of the course would be virtually impossible. (i) Step-Marking. On the limited number of examination scripts scrutinised in the course of this investigation, from the detailed Solution and Marking Guide and from the explanation of its use by CQU officers, it was concluded that, with the possible exception of some examiners who should be traced through the moderation process, step marking is in place. (j) Grade Reviews with Administrative Staff. CQU officers explained that the initial ‘review of grades’ took place with administrative personnel, because at this initial stage it was imperative that the moderated examination process at Rockhampton not be pre-empted by academic discussion of merits and deficiencies at the Melbourne Campus. It is assumed that this stage of the process is purely informational (i.e involving discussion of what marks have been awarded, and that any academic dispute about marks would eventually be taken up with academic staff in the course of the full review process). (k) Staff Allegations. Allegations against one named staff member on the grounds of consistently high failure rates were discussed with CQU officers who assured me that this staff member’s marks are broadly in line with those awarded by other staff. (l) Teaching Standards / Levels of English Comprehension. CQU declined, quite correctly, to provide copies of student evaluations on the grounds of the protection of privacy. I understand, however, that statistical summations of data are made. I therefore recommend that CQU be asked to provide to the Victorian Office of Tertiary Training and Education, by 30 April 2007, statistical summaries of student evaluations for all Postgraduate Accounting Courses delivered on the Melbourne Campus during 2006. (m) Juxtaposition of Teaching Term and Examinations. I understand that a number of universities take account of this problem by operating what is, in effect, a two and a half term teaching system per calendar year. This is something to which CQU might give serious consideration, making due allowance for compulsory attendance visa requirements. 12 4. Conclusion. I have found that while student complaints in relation to the availability of Conceded Passes have some validity, largely for procedural reasons, their claim for the compensatory award of 13 marks for Question 2, Part C of the examination 20053 cannot be sustained. I have examined a number of other complaints and, to the best of my ability within the time available, I have provided answers to the allegations made. I have not been able to investigate complaints about a number of administrative processes such as grade change and grade review. These are matters which CQU may well decide to investigate further on its own initiative. I believe that many of the difficulties encountered by CQU at its Melbourne Campus may stem from the difficulties inherent in delivery of centrally directed and examined academic courses at ‘remote’ sites. The degree of central control, examination moderation and quality control is commendable. But this needs to be combined with a sufficient degree of campus autonomy to allow for local exigency and accountable flexibility in terms of things like course coverage, topic emphasis and so on. I do not see these two things as incompatible and it may well be that the University needs to review this matter as an urgent issue of systemic importance. I do not believe that, at the present time, and on the basis of this investigation, there is a case for a more general audit of CQU’s compliance with the ESOS Act. I am conscious, however, that this is the second year in which difficulties with students at this campus have arisen, and I would suggest that any recurrence should in and of itself constitute sufficient cause for such an audit to take place. 5. Summary of Main Findings and Recommendations. The student allegation of unfairness by the inclusion of Question 2 on the examination paper cannot be sustained, though CQU might usefully be rather more meticulous in the notoriously sensitive matter of the advice given to students about the likely contents of upcoming examinations. An appropriate method of marking appears to have been employed in the grading of Question 2 of the Issues in Management Accounting Examination ACCT 20053, 2006, and I therefore conclude that the full 13 marks should not be allocated to all failed, SE or other students. For the reasons adumbrated at 3 (c), above, I recommend that CQU Melbourne International Campus be requested to consider for PC/PT grades those Masters of Professional Accounting students who in all other respects would qualify for such consideration. I further strongly recommend that, for the future, the Faculty of Business and Informatics make the policy excluding postgraduates from such consideration quite explicit in its PC/PT Grades Operational Procedure document (Appendix 7). 13 I recommend that CQU be asked to provide to the Victorian Office of Tertiary Training and Education, by 30 April, statistical summaries of student evaluations for all Postgraduate Accounting Courses delivered on the Melbourne Campus during 2006. I recommend that CQU give consideration to the institution of a two and a half term teaching system (with possible compulsory revision classes to meet visa requirements) in order to obviate undue proximity between the completion of teaching and the sitting of examinations. I recommend that CQU review its distribution of academic and organisational responsibilities between Rockhampton and the Melbourne Campus, with a view to balancing centralised control against local autonomy. I do NOT recommend that a full ESOS audit of CQU’s Melbourne Campus be instituted, but that such a review should be automatically put in place should there be any recurrence of the problems encountered in the last two years. Professor Emeritus W.G. Carson M.A. (Oxon), Dip. Crim. (Cantab.), LL.D. (Edinburgh).