Quality Review Final Report: MA Programmes An Coiste Feabhais Acadúil The Committee on Academic Quality Improvement The Academic Quality Assurance Programme, 2007–2008 Review of MA programmes in the College of Arts, Social Sciences and Celtic Studies Final Report (18th-20th February, 2008) This report arises from a visit by a review group to the MA programmes in the College of Arts, Social Sciences and Celtic Studies on 18th-20th February, 2008. The Programmes and Dean of the College had already prepared and submitted a 'Self Assessment Report' that, with other documentation, was made available to the Group in advance of the visit. This report arises also from additional material provided to the team during the visit and the meetings that were held with relevant academic and administrative staff and students. The review also took note of the annual set of review themes for 2007-2008 in "The Guidelines for Academic Quality Reviews 2007-2008". The Review Group consisted of: Professor Stuart Turley, Manchester Business School, University of Manchester (Chair); Dr Maeve Conrick, Department of French, University College Cork; Mr Ronnie O'Gorman, Advertiser Group, Galway; Dr Vinodh Jaichand, Irish Centre for Human Rights, NUI Galway; and Dr Aoife Boyd, Department of Microbiology, NUI Galway acting as Rapporteur. The report is structured to cover the following main topics: 1. Aims, Objectives and Planning 2. Organisation and Management 3. Research and Scholarship 4. Programmes and Instruction 5. Facilities and Infrastructure 6. Community Service 7. Summary and Recommendations The Review Group was provided with 5 programme reports to represent the MA programmes within the College. This review reflects the group's impression of the College programmes overall and outlines recommendations for the future development of MA level education within the College. The Review Group was impressed with the quality of educational provision within the individual programmes reviewed. There is a high degree of commitment from members of staff to ensure the quality of the educational experience and the students are extremely positive about their MA education experience. 1. Aims and Objectives This review of the MA programmes in the College of Arts, Social Sciences and Celtic Studies has been undertaken during a time of change in NUI, Galway structures and shortly after the creation of this new College and the new Schools within it. Following University restructuring over the past 12 months, structures within the newly formed College of Arts, Social Sciences and Celtic Studies are still being developed. The early stage of the new structure has limited the extent to which strategic and operating plans have been developed, implemented and communicated to units within the College so far. There is a pressing need to establish an overall strategy for the College which makes clear the intended role for MA level programmes within the full set of activities undertaken by the College and the Schools within the College. The creation of the new College presents a significant opportunity to establish clear direction for the future to ensure that the programmes work within and towards a common strategy, rather than to continue historically different approaches. 2 Particular matters that should be reflected in the overall College approach include the following: The basis of the educational aims on which new programmes are developed and approved - Several programmes exist because of the tremendous enthusiasm of individual staff members. While acknowledging the excellence and creativity that this brings to the portfolio of MA programmes, it is important that programmes command collective commitment and are seen to be consistent with overall College and School plans. Each programme should contribute to the common strategy and individual programmes should not have to depend on the personal “ownership” by individuals for their survival and high standards. The impact of quotas and class sizes on teaching, research, resources and staff development and staff morale - It is important that quotas operate in a manner that follows a logical planning process and gives the appropriate incentives to staff in relation to recruiting students. If quotas set in advance rather than actual recruitment determine the distribution of resources, there is a danger that imbalances can develop between the provision of, and need for, resources in practice. How the current (and prospective) MA programmes feed into PhD recruitment - A strategic objective should be to recruit high calibre students at MA level who are able to carry on to further education as PhD students as encouraged by the Higher Education Authority (HEA). This objective may not be relevant to all programmes and other professional and vocational objectives should also be recognised. 2. Organisation and Management The College has a well established culture of education at the MA level and is to be commended on the quality and range of MA programmes on offer. The management of individual programmes follows a varied pattern, and to some extent each programme follows its own management and delivery structure. This is largely a product of the historical development of the MA programmes within separate Departments. Individual programmes appear to have operated as separate identifiable entities. This has had benefits for flexibility and responsiveness, but carries the risk of diversity of practice, variation in conventions and standards and potential duplication of activity. These factors can impact differentially on the student experience. This situation can also make it difficult to judge equity in student and staff workload allocations, the viability of programmes and the use of academic resources. There is a need to establish a common framework for programme structure and the management of programmes. Particular areas of organisation and management which should be considered as part of developing a common approach across the College include: Structural framework for MA programmes Clear and transparent guidelines should be established for the requirements of introducing and maintaining a MA programme. These could include, although not 3 be restricted to, consideration of the financial viability of programmes. It is clear that some programmes are recruiting small numbers of students and there has to be a concern about their ability to generate the resources necessary for maintenance of delivery. A means of reviewing the economics of individual MA programmes, alongside their academic benefits, should be considered. The guidance should make clear the degree of standardisation and flexibility regarding the components required for a MA level programme. The sample programmes reviewed by the panel illustrated variations with respect to the credit structure associated with course units, the weighting of the dissertation component, and the overall level of credits required to complete the programme. The College and University should evaluate the potential benefits of greater standardisation in structure, possibly through common College-wide or University-wide degree regulations, which could facilitate, for example, sharing of activity across programmes (see Modularisation below in section 3). Programme Management: Clarity is needed as to which level of the University is responsible for which aspects of programme management, including the composition and responsibilities of programme boards and expectations for frequency of meeting. To avoid unnecessary proliferation of programme boards there is considerable scope for a single board to take responsibility for several programmes within a broad field, rather than requiring separate structures for every programme. Responsibility for the promotion and communication associated with student recruitment is one area that may benefit from identifying focal points within the new structures. The authority structure and responsibility for academic oversight within the College with respect to MA level education needs to be developed further. It is important that this area is represented in one of the portfolios of the senior management of the College (and is mirrored in the equivalent at School level), whether this be through the creation of a separate post of Vice Dean or in combination with another area of academic responsibility. Academic oversight of each new MA programme should be undertaken at College level when the programme is being proposed and should draw on academic staff with expertise in the area. The College should assess the academic aspects of all programmes (e.g. appropriateness of content, assessment methods and weightings, and learning outcomes), before they are approved and forwarded to central University bodies. Appropriate arrangements for cross-disciplinary programmes should be established. One potential approach would be to ensure that all such programmes are, for organisational and administrative purposes, assigned to a lead school, with related cross school representation, with the educational contributions from all schools reflected in the financial model. Arrangements should be put in place to ensure rotation of responsibilities, such as the post of Programme Director. Finances At present there seems to be a lack of clarity concerning differences in the financial position for different programmes and a lack of transparency about the overall financial position of academic groups. This situation can create uncertainties and potential misconceptions among staff. The University (and College) should continue 4 to develop a model giving greater transparency on the financial flows associated with academic activities, and indicating whether the budgetary and financial structure is to give separate recognition for individual programmes or to expect financial management to operate across a combination of activities. Review Process Members of staff participated in the review in very helpful and constructive ways, but it appeared that the review itself was seen as a one-off exercise. There would be benefits from creating a culture of regular internal review and encouraging an attitude of continuous development in response to review. It is important that review and response are seen as natural components of the regular management and development of programmes, rather than as an occasional exercise conducted for external purposes. Policies Currently it appears that there are some areas in which policy has been developed by different bodies within the University, and some areas which have been left to local practice, but the rationale for approaches to developing policy in different areas is not clear. Opportunities exist within the new structures and through the appointment of a Dean of Graduate Studies to establish a clear framework for the development of policy and practice relating to MA programmes and for the communication of policy throughout the College. Staff Development and Management Aspects of staff development and the management of staff where there appear to be potential inconsistencies in practice include the measurement and allocation of workloads, the provision of mentoring support for staff and the expectations for training and development of new staff. There is scope for developing a common framework in each of these areas, which appear to be significant causes of frustration for some members of staff. One concern regarding workloads is the lack of clarity on how teaching on interdisciplinary courses is factored into the workload of an individual academic. The Review group suggests that the College and the University develop workload models which are transparent and which take into account both the needs of individual disciplines and the need to foster interdisciplinarity. 3. Programmes and Instruction The staff contributing to the MA programmes are to be commended on sustaining a consistently high level of teaching quality that provides an optimal learning experience for students. Each programme actively engages students in the learning process and takes great care to prepare them for the future and to match curricula to necessary competencies. Past and present students responded extremely positively to the educational experience which the programmes provide for them. The remarkable commitment of the staff is reflected in their approach to the delivery of programmes. It was clear to the reviewers that many programmes exist because of the tremendous enthusiasm shown by the staff involved in the delivery of the programmes. Much of this valuable work is done at the individual programme level. 5 Staffing More contact and cooperation between staff involved in different MA programmes would encourage greater consistency in approach and delivery. The College should consider facilitating greater interaction between staff teaching on different programmes, which would lead to greater cohesion in the integration of the programmes at College level, for example in the sharing of modules, sharing good practice, sharing of expertise in developing new modules and programmes and reviewing practices in existing programmes. More involvement of academic staff at higher levels (e.g. professorial level) should be encouraged, as this is appropriate for MA courses involving a large independent research element. The delivery of teaching and dissertation supervision in MA programmes should be by academics who are at an appropriate level of appointment, research activity and seniority. Student Feedback There was evidence of significant interaction between staff and students and considerable informal feedback mechanisms for students. Programmes should consider documenting discussion of student feedback in more formal ways, and ensure that there are processes for reporting back to students informing them what actions have been taken in response to their feedback. The College may wish to consider a framework for handling student feedback at College level. Consideration might be given to the establishment of staff-student structures across all programmes, whether at the individual programme level or for clusters of related programmes. Modularisation The College should consider developing structures to facilitate modularisation of programmes, such that students have a greater range and choice of modules, without proliferating a large number of modules in separate disciplines. This approach would also maximise the use of resources and minimise the burdens on staff teaching loads. It may also help to recruit students to modules in programmes with small enrolment numbers. Module weighting would need to be agreed across college, and beyond, in order to encourage uptake by students. The student workload for some programmes could be reviewed, with a view to increasing flexibility of the programme (e.g. spreading a programme over 2 years, rather than necessitating completion with 1 year), and to give students the opportunity to enjoy, appreciate and derive greater intellectual benefit from the course and to improve their work-life balance. This may be especially relevant to those programmes with professional objectives, where study is often being pursued alongside full-time employment. Assessment Assessment is generally handled in a very professional manner, with many opportunities for students to receive feedback from their teachers. Many students referred positively to the approachability and availability of staff to discuss their progress. A range of modes of assessment is in place across the programmes. However, there are some differences and inconsistencies in the assessment within 6 programmes. Improvements could be made in agreeing assessment procedures, particularly at programme development stage. A detailed breakdown of modes of assessment for each module, how they link to the aims and objectives of the programme and their relative weighting should be provided when a new programme or module is being proposed. The University may wish to consider implementing procedures to ensure anonymity in assessment procedures, in so far as possible. The University should consider drawing up a document of best practice for ensuring anonymity in the examination and assessment procedures. The College should consider the benefits of appointing one external examiner for each programme or suite of programmes. Existing roles of subject external examiners could be extended to include responsibility for commenting on the coherence of a programme, as well as the modules within a subject area. The expectations for attendance and involvement of external examiners in programme boards needs to be evaluated and made clear. The roles of the external examiner and selection of the examiner should be clearly outlined. Physical Resources A feeling was expressed amongst many students that a dedicated space should be available for their use for teaching and self-study, even on a shared basis with students from other MA disciplines. The library staff support was very much appreciated by the students. Improvements in provision of library resources, especially electronic journals, and off-campus access was recognised as important and appreciated especially by part-time students. However the library has limitations in some of its physical collections and space. There is a need for development of the collection in certain areas relevant to specific MA programmes. The urgent need for additional library space was highlighted by students and library staff alike. 4. Scholarship and Research We note that preparation for further research degrees is aspired to by all the programmes, although, as noted earlier, the objectives of some programmes are more focused on professional development and others on research development. The role, importance and form of research training for all programmes should be clarified and, since all postgraduate programmes have a research component, all students should be encouraged to disseminate their MA research in fora such as conferences and in the media. 5. Community Service The Review Committee was impressed by the NUI, Galway Community Knowledge Initiative supporting civic engagement evidenced by students working on a voluntary basis with social needs groups in the city. Several MA programmes attract students who contribute significantly to the community and will continue to contribute to the community following their postgraduate education at NUI, Galway. Programmes in particular 7 disciplines are also concerned with relevant areas of social need and are making a considerable contribution to developing professional capacity in these areas. The University is thereby making a valuable practical and intellectual contribution to the development of the community it serves. 6. The Wider Context As outlined in section 1, this is a time of transition for the new College structure and an opportunity for the College to develop its strategy for the future progress of MA programmes in a College-wide and University-wide context to reflect the desires and needs of students and their future employers while at the same time maintaining a high standard of education, increasing flexibility and streamlining management policies. This is also an opportunity for the College to highlight the contribution of the MA programmes to the University portfolio in order to ensure the continued success of these programmes. The significant impact of MA level education on University economics should be borne in mind, both through national funding based on numbers of postgraduate students (MA and on to PhD) and through attracting fee-paying international students to help with programme viability. MA level education has an important role to play in producing graduates who will help to promote a knowledge-based economy which will have social and economic benefits for the Irish population. MA programmes are important as a means for progress to PhD studies and thereby complying with current HEA policy to increase the numbers of PhD students. Therefore MA programmes have a significant impact on the University portfolio and its contribution to the Irish population. 7. Summary and Concluding Remarks The Review Group was impressed with the quality of educational provision within the individual programmes reviewed. There is a high degree of commitment from members of staff to ensuring the quality of the educational experience and the students the Review Group met were extremely positive about the approach followed for their education. While the educational quality of individual programmes is high, the Review Group did conclude that there are several areas where the arrangements surrounding the general strategy for MA education and processes for programme management and approval could be improved. To a large extent these points reflect the still formative stage of development of the new College and Schools structures, and are the product of historical differences between units that now comprise the new College. It is extremely important that such differences are not allowed to continue into the future without explicit consideration. Staff should be encouraged to identify with the new Schools, rather than the original Departments, as the relevant structures for organisation of academic activity. The opportunity afforded by the creation of the new structures for a fresh approach to MA programmes should be taken before the College becomes long established. The individual points that the Review Group recommends be considered for action are covered in the preceding sections of this Report. They are not reproduced in detail here but, in summary, the Group recommends development in the following areas of strategy, management and delivery of MA level programmes: i. Establishing a clear overall strategy for MA education, including the relationship to 8 ii. iii. iv. v. vi. vii. viii. ix. x. xi. xii. xiii. xiv. xv. xvi. xvii. xviii. xix. doctoral studies and levels of desired recruitment. Establishing guidelines for MA programmes that include consideration of their financial viability. Establishing regulations for programme structure and credit rating. Identifying a common framework of committees having responsibility for management of programmes. Identifying oversight responsibility within the senior management team of the College. Ensuring appropriate arrangements for cross-disciplinary programmes. Providing financial and budgetary transparency. Encouraging a culture of continual review and development in programmes. Establishing (at University level) the framework for setting policies and procedures for MA education. Ensuring a comprehensive and equitable approach to staff workloads and other aspects of staff development. Encouraging cooperation and sharing of good practice between staff in different programmes. Ensuring the experience and qualifications of staff are commensurate with the roles they are undertaking at higher degree level. Establishing a systematic approach to student feedback and mechanisms for reporting back to students on actions taken. Using modularisation to maximise choice and efficient use of resources. Ensuring the organisation and workload for professional programmes takes into account the time available from students in employment. Ensuring that clear profiles of assessment are associated with each module and programme and are communicated to students. Reviewing the policy with respect to anonymity in assessment processes. Consideration of the creation of a role of Programme External Examiner. Consideration of dedicated physical space available to MA students. 8. Comments on the Methodology of the Review Process The self-assessment process should be integral to the ongoing development of programmes, rather than just for Quality Review purposes. The review committee would have welcomed evidence of engagement in a benchmarking process as is highlighted in the Guidelines for Academic Quality Review. "Insights and ideas gained from comparisons that have been made with equivalent high quality units in other universities in Ireland and abroad (benchmarking) may be invaluable". Such an exercise would have enhanced the opportunities for self-reflection. The benefits from any review process often derive as much from the process of reflection and preparation undertaken by staff as from the comments of a review group. It is helpful to reviewers if the documentation provided emphasises the evaluative views of staff in an open and honest way, rather than just describing the current situation. The documentation provided in this review was variable in this respect; some reports gave clear views of strengths and areas for development, while others gave limited insight into what staff actually thought about the quality of their programme. The University and College should seek to emphasise the importance of genuine self-evaluation in preparations for reviews. 9 The purpose of the meeting with individual members of staff on the second day of the review visit was not clear in the minds of some of the individuals themselves. The committee met with some individuals 3 times in the 2 days. If the intention of this part of the review is to allow members of staff to raise matters as individuals rather than through their capacity in a representative or management role, perhaps a general invitation could be issued to all staff, but only those who have specific comments not likely to arise in other meetings need take up the invitation. Professor Stuart Turley (Chair) Dr Maeve Conrick Mr Ronnie O'Gorman Dr Vinodh Jaichand Dr Aoife Boyd (Rapporteur) 13th May 2008 10