School of Agricultural and Veterinary Sciences, and Office of the Deputy Vice-Chancellor (Academic),
Charles Sturt University

MN Sillence - Wishing on a Star
Page
Executive summary ………………………………………………………………. 3
Background ………………………………………………………………………. 4
Introduction ………………………………………………………………………. 4
Perspectives on the purpose of the Research Quality Framework ……………….. 5
Research funding in Australian higher education ………………………………... 6
The UK Research Assessment Exercise …………………………………………. 7
Development of an Australian Research Quality Framework ……………………. 9
Implications of a new assessment system for Australian universities ……………. 9
Improving the research standing of Charles Sturt University ……………………. 11
Motivation
……………………………………………………………. 11
Skills
………………….……………………………………………… 12
Time
………………….……………………………………………… 13
Resources
…………………………………………………………….. 13
The Research Management Plan
…………………….…………………… 14
Maintaining the quality of learning and teaching at CSU ………………………... 14
Leadership and management strategies
…………………….……………… 16
Preserving diversity at CSU ……………………………………………………… 16
Leadership and management strategies
…………………………………… 16
Conclusions: a mindset for morale, growth and prosperity ……………………… 18
References ………………………………………………………………………... 20
Appendix 1: Abbreviations ………………………………………………………. 24
2

MN Sillence - Wishing on a Star
The Commonwealth Government is about to introduce a periodic review mechanism to assess the quality, quantity and impact of research conducted in Australian
Universities. The Research Quality Framework (RQF) will be based on a system that has operated in the UK for 20 years, and will inform the distribution of almost $1 billion in block funding for research. Although the cost and effort required by the
RQF is likely to be substantial, the Australian Vice-Chancellors’ Committee and several well-established universities have declared their support for the initiative, in the belief that it will lead to increased Commonwealth funding for research.
However, for new universities with a narrow research focus and a shorter trackrecord, the RQF poses a significant threat, as it is likely to divert much needed research income towards their larger competitors. Furthermore, the UK exercise has had an adverse effect on teaching quality, diversity and morale in some institutions. It is recommended that Charles Sturt University (CSU) should approach the RQF with a healthy perspective and a clear vision of what the University wants to achieve as a whole institution. Goals should be set to enhance research output, but these should be balanced by strategies to improve the value and quality of service we offer to our students and other stakeholders. Rather than trying to compete with the more powerful research-intensive universities head on, CSU should pursue its own aims, and maintain the values that make the University such a unique and important resource for regional Australia.
3

MN Sillence - Wishing on a Star
This paper describes a system about to be introduced by the Commonwealth
Government, to evaluate research quality in Australian universities. It is written from the perspective of a full-time research team leader, a part-time strategist currently on secondment to a Workforce Planning Project, and a student in university leadership and management.
The paper explores possible implications of the Research Quality Framework (RQF) for Charles Sturt University (CSU), in the light of experiences in the UK, which has employed a similar system for more than 20 years. Among the wide range of issues that are identified, three topics are used to illustrate the type of leadership and management strategies that could be adopted to maximise the benefits and minimise the threats associated with the RQF. These topics are: improving research output, maintaining the quality of teaching and learning, and preserving institutional diversity. The paper concludes with questions, actions, and a recommended mindset, that would serve the University best in the current climate.
Imagine that one day you receive a notification from an important customer - your largest customer in fact - telling you how to run your business. What is the appropriate response? This is the question that universities across Australia are struggling to answer, in a climate where the Commonwealth Government is showing less interest in funding universities 1 , but an intense interest in managing their affairs
(Karmel 2000). Following the 2002 Review of Higher Education by Dr Brendan
Nelson, Minister responsible for the Department of Education, Science and Training
(DEST), and the subsequent the passage of the Higher Education Support Act in
2003, the university sector has been the subject of an unprecedented number of reviews and reforms. These include the current workplace reforms, a discussion on transferring legislative power over universities from the States to the Commonwealth
(DEST 2004, 2005a), a new accreditation policy to enable more private providers to enter the arena (DEST 2005b), an examination of offshore teaching activities (DEST
2005c), and the introduction of a Research Quality Framework (DEST 2005d).
The Government’s claim of being committed to “reducing red tape and regulation in universities” (DEST 2003) seems to be in stark contrast to the result of its policies.
Previous measures of quality were described as “inaccurate and wasteful of time and resources” (Karmel 2000), but the raft of new measures has not been greeted warmly.
The CEO of the Australian Vice-Chancellors’ Committee (AVCC) argues that
Commonwealth reporting requirements continue to provide a significant bureaucratic burden, diverting much needed funds away from the core activities of teaching and research (Mullarvey 2005). He goes even further, in stating that Government policies threaten a long tradition of autonomy and academic freedom in Australian universities. In light of these comments, it is surprising that the official response of the AVCC to the Commonwealth’s RQF initiative is to support the proposal (AVCC
1 As a proportion of total university revenue, Commonwealth funding has declined from approximately
90% in the 1980’s to a current average of about 40% (Mullarvey 2005).
4

MN Sillence - Wishing on a Star
2005a). This is certainly not the view of all higher education providers, and the reasoning behind the AVCC position deserves closer investigation.
In May 2004 the Prime Minister announced his decision to develop a quality framework for publicly funded research. This initiative was is based on the assertion that
“currently there are no mechanisms or procedures in place to consistently assess the quality of publicly funded research in Australia, or the impact or benefits that result from public investment in that research ” (DEST 2005d, p11). Thus, from the
Government’s perspective, the RQF is intended to ensure accountability and inform funding decisions.
Others argue that universities already put a considerable effort into demonstrating the quantity of their research, through the DEST points system that currently informs
Commonwealth block funding (Group of Eight 2005). Furthermore, research quality is assessed by a peer review system that is the foundation of the competitive grants scheme. Arguably, even industry funding is likely to be sustained for only as long as universities continue to deliver value and quality outputs. Nonetheless, the
Government is not satisfied by these measures.
Despite initial reservations, the AVCC has declared its support for the RQF in the belief that it will help to “ reinforce the value of Australian research to the community ” and “ provide a basis for future investment targeted at high quality research
” (AVCC, 2005a). It should be noted, however, that without additional investment in research by the Commonwealth, the AVCC does not support the RQF on the grounds that the cost and administrative burden of the exercise will far outweigh any gains made through the redistribution of existing funds. Although one hopes that the AVCC expectation of additional funds is correct, this may be unrealistic as the Minister has made no public promise or even mention of additional funds. Some observers have suggested that additional university funding will not be channelled through the RQF, but allocated under a different scheme according to university involvement in so-called “third stream” activities. This is a role that goes beyond education and research, where universities contribute to regional and national economic growth, through activities such as commercialisation, industry partnerships, the development of business and technology parks, and so on (The Allen Consulting
Group 2005a).
Finally, in contrast to the AVCC, The Group of Eight, and many other stakeholders, the reaction of CSU management to the RQF has been less than positive. In a recent discussion paper the Vice-Chancellor stated that he is “ certain this is another strategy to further differentiate the sector into ‘research led’ and ‘teaching led’ institutions
”
(Goulter, 2005). This view was reinforced shortly afterwards by the Pro Vice-
Chancellor for Research, in an even less cheerful paper titled “ No Research at CSU: A major risk?
” (Burnett, unpublished) which paints the image of an almost inevitable and irreversible descent into a teaching only institution. Others claim that such a view is unrealistic, inappropriate and dispiriting, especially to new staff (Kemmis et al. unpublished). It is certainly true that over a number of years CSU has achieved
5

MN Sillence - Wishing on a Star several research outcomes of which it can be justifiably proud. Furthermore, many
CSU staff are motivated to conduct research through curiosity and a belief in the value of knowledge creation, not through an ambition to earn DEST points. However, following the introduction of the RQF, the University’s capacity to support these staff in a sustained research effort will depend more than ever, on the approach and actions of university leaders and managers.
Although the outcomes of the RQF will be slightly different for every university, the process is likely to produce some generic behaviours and consequences across the sector. Much can be learned from studying the experience of universities in the UK which have undergone research assessment for the past 20 years. However, before discussing the most likely outcomes and issues, it will be useful to examine the current status of research funding for higher education in Australia, and the way it is distributed. This will help to put the proposed RQF reform into proper perspective, and may go some way towards addressing the validity of the argument that Australia needs to divert a greater share of its limited research funds into high quality research.
In total, approximately $2,343 million in research funding was available to higher education institutions in Australia in 2004. This comprised about $629 million from sources other than the Commonwealth, including industry, charities and State
Governments (based on 2003 data, AVCC 2005b); $619 million awarded under
Commonwealth Competitive Grants Schemes (AVCC 2005b); and $1,095 million awarded by the Commonwealth as Institutional Block Funding (Nelson 2004). A specific aim of the RQF is to inform the distribution of this last and largest portion of funding - the block grants. However, it is reasonable to assume that when the RQF awards a quality rating to each university discipline group, this will also influence the future prospects of that group to win competitive research grants, and to attract industry funds or other research dollars. Thus, the entire spectrum of university research funding is likely to be affected to some extent, by the introduction of the
RQF.
Table 1. Distribution of Commonwealth research block funding ($ million) among various schemes, and proportion of total allocation earned by two universities at extremes of the sector
(data adapted from Nelson 2004).
Research training awards Infrastructure and small grants
Scheme total
Melbourne a
CSU b
Research
Training
Scheme
540.8
53.8
3
Australian
Postgraduate
Awards
89.3
9.7
0.45
International
Postgraduate
Awards
17.7
1.7
0.09
Institutional
Grants
Scheme
284.6
31.4
1.3
Infrastructure
Block Grants
160.3
21.9
0.84
Regional
Protection
Fund
2
0
0.05
CSU % total 0.55% 0.50% 0.51% 0.46% 0.53% 2.50% a The University of Melbourne earned the largest share of block funds, averaging 10%, just ahead of The University of Sydney and The University of Queensland. b
Charles Sturt University is ranked according to earnings at between 29 th
and 34 th
out of 38 in the sector, depending on the scheme (Burnett P, unpublished).
Totals
1094.7
108.8
5.7
0.52%
6

MN Sillence - Wishing on a Star
Table 1 illustrates the relative distribution of block funds awarded in 2004. Based on the UK model, there has always been a strong likelihood that RQF scores will be linked directly to funds allocated under the Institutional Grants Scheme (IGS). This provides money to support the operating costs of early career researchers through small and competitive grants, some research staff salaries, and some teaching relief for academic staff. Infrastructure block grants, which provide money for buildings, maintenance and large equipment to support national competitive grants, are unlikely to be linked to RQF initially, and neither is the Regional Protection Fund
2
.
The largest portion of funds awarded under the block grant system is that devoted to postgraduate training (59%), which provides both scholarships and fee support for research higher degree students. Initially, linkage of the RQF to Research Training
Awards was left open to discussion (DEST 2005d), but now it seems likely to be adopted as a result of strong support from the AVCC and other key stakeholders
(DEST 2005e). Charles Sturt University was among the few respondents to the RQF
Issues Paper to strongly oppose this (DEST 2005f). Apart from any philosophical concern about the difficulty in assessing the quality and impact of postgraduate training, the inclusion of Research Training Awards (as opposed to IGS funding alone) more than trebles the sum of money that is directly at stake under the RQF scheme (Table 1). This is an area in which the University is particularly vulnerable. It is possible to substitute industry funding for Government funding to some extent, and some private companies are will provide a stipend for postgraduate students.
However, few companies have an interest in paying postgraduate student fees of up to
$17,000 per year, and very few students can afford to meet this cost themselves.
The concept of assessing research quality and impact is not unique to Australia, and quality systems are either already in place or being developed, in countries such as
New Zealand and Belgium (The Allen Consulting Group 2005a). However, the grandfather of all schemes is the UK Research Assessment Exercise (RAE) which commenced in 1985, and had expanded by 1992 to cover the entire higher education sector. The RAE is conducted on average every four years, and universities are currently gearing up for the sixth round to be conducted in 2008 (RAE 2008).
Competitive research grants are not tied directly to the RAE, but about $2 billion in infrastructure block grants is distributed each year according to RAE outcomes. This is just over twice the sum likely to be affected in Australia, but considering the UK money is shared among more academics (145,510 staff: Association of University
Teachers 2004), it is instructive to note how influential the exercise has been in affecting the actions of UK universities and individuals. Some older universities
2 The Regional Protection Fund (RPF) was introduced as a temporary measure to cushion the impact on regional universities of earlier reforms to higher education funding. Funding is fixed for the period
2001 to 2008 and is currently distributed among only seven universities. The $2 million per year allocated to this scheme represents less than 0.2% of total block funding, and of this, Charles Sturt
University earned only 2.5% in 2004. Seventy six percent of RPS funds were awarded to just two institutions: The University of Newcastle and LaTrobe University – and this increases to 95% when combined with the earnings of James Cook University and the University of New England (Nelson,
2004).
7

MN Sillence - Wishing on a Star derive a significant proportion of their funds through the RAE. For example, in 1994-
95 research income at Oxford University was 48.2% of total income (Cradden 1995, cited in Jenkins 1995). In other new universities, the figure is generally much lower
(e.g. 7% for Oxford Brookes in the same period: Jenkins 1995), but not less important, as it represents a source of funds that had not been accessible previously.
The RAE system is based on the assessment of discipline groups rather than whole institutions or individuals. Each university may elect the disciplines that it wishes to be assessed, with Cambridge University submitting the largest number of disciplines
(51) and earning the most funds in the 2001 exercise.
Assessment is based on the use of expert panels which make their evaluation of quality on the basis of a portfolio provided by each discipline group, and by examining the top four research papers published by each nominated researcher over the previous four years. An estimate of the volume of research is made according to the number of full time academic staff in the discipline (75% weighting), with a lower weighting for other research staff, postgraduate students and research income. By multiplying the scores for quality and volume, and after a further weighting is applied according to the type of discipline, a figure is produced which determines the share of funding awarded under the RAE scheme.
Funds earned by each discipline are not paid to the discipline groups directly, but aggregated and given to the institution for use at its discretion. However, each discipline group is awarded an individual rating based originally on a five point scale, and later expanded to a seven point scale (1, 2, 3b, 3a, 4, 5, or 5* - five star). A score of 5 earns four times as much income as a score of 2 (Jenkins, 1995). Initially, even the lower score was enough to earn some money, however, it is claimed that in recent years the funding council has raised the bar. The UK Treasury has increased its total investment in research, but nowadays only 4, 5 and 5* departments are receiving a useful share of this (Beckett, 2005).
Of particular significance to the Australian RQF is that the Commonwealth
Government’s Expert Advisory Group is chaired by Sir Gareth Roberts, who also led a review of the UK RAE in 2002, concluding that it had made British research “ more focussed, more self-critical and more respected across the world
” (Roberts 2003, p.
2). However, Roberts is a professor from Wolfson College, Oxford - a University that came second only to Cambridge in the 2001 RAE. A very different analysis was presented by Williams - a professor of medicine at Liverpool University, whose research was commended by the RAE, but whose department earned a rating overall of only 3b in the 1996 exercise. In an article published by the British Medical
Journal
3
, Williams (1998) describes the RAE as “
A dysfunctional juggernaut …… threatening to crush creativity, careers and scientific inquiry
”. Others have supported
Williams’ view, calling for the scheme to be abandoned (McNay 2003). Critics accuse the RAE of “ creating a mess
” (Jenkins, 1995), and describe its implementation and the policies it has encouraged as “ crude, stupid and wrongheaded
” (Piercy 2000),
“ disastrous
” (Association of University Teachers, 2005), or “ singularly inappropriate ” (Banatvala, Bell & Symonds 2005).
3 Unfortunately, it is not recorded whether this paper was one of the obligatory four put forward by
Williams in the subsequent 2001RAE.
8

MN Sillence - Wishing on a Star
Despite his strong support for the concept of research assessment, Sir Gareth Roberts is clearly aware of the many flaws in the UK system, and hopes to steer Australia towards an RQF than avoids at least some of these problems (DEST 2005d). Indeed, debate about the RQF has been skilfully diverted away from the question of whether
Australia in general, and the Higher Education sector in particular, will be better off as a result of the new system. Instead, it has focussed on the ‘nuts and bolts’ of how the framework can be developed in a way that minimises any adverse or unintended consequences. Thus, dealing with multidisciplinary and cross disciplinary research in a discipline-based framework was the topic of one major workshop (The Allen
Consulting Group 2004), while another concentrated on the difficulties associated with measuring research impact (The Allen Consulting Group 2005a). Formal submissions were sought following the release of first Issues Paper in March this year
(DEST 2005d), where stakeholders were asked to complete a detailed questionnaire which avoided asking whether the respondent was actually in favour of an RQF. A series of workshops and stakeholders forums has been held in each capital city (The
Allen Consulting Group 2005b), followed by a National Stakeholders Forum in June
2005 (DEST 2005g).
According to the timeline published by DEST, a model RQF system is due to be trialled shortly, but at the time of writing no details of the preferred model have been announced. Based on the most recent documents to be released - the Advanced
Approaches Paper (DEST 2005e) and the report on the National Stakeholders Forum
(DEST 2005g) - it is only possible to predict that the RQF will be based on the assessment of discipline groups to be nominated by each University; that the review will be conducted by a panels that contain predominantly discipline experts, with a few others who are able to comment on broader impact; and that the outcome will be tied to a significant amount of funding, including the Institutional Grants Scheme and the Research Training Scheme.
The implications of the RQF are numerous and wide-ranging, and a detailed discussion of all the factors is beyond the scope of this paper. Table 2 summarises a few of these issues, while the sections that follow deal with three specific topics in more detail.
Table 2. Summary of issues that have arisen in the UK as a result of the Research Assessment
Exercise, and which could result from the introduction of an Australian Research Quality
Framework.
Issue
Perverse behaviour by staff and institutions
Possible outcomes
Institutions place far greater emphasis on research output when recruiting, assessing and rewarding staff. As a result, staff neglect other scholarly work such as refereeing grants and papers, or contributing to university administration.
Distrust of assessors and colleagues in other institutions increases, as the environment becomes more competitive.
References
Jenkins 1995;
Piercy 2000;
Tomlinson 2000;
Williams 1998
9

MN Sillence - Wishing on a Star
Equity – gender and early career researchers
Staff morale
Infrastructure and support services
Collaborative and cross institutional research
Damage to specific disciplines
Fairness and acceptability of the system
Cost / benefit of the exercise
Does Australia need an RQF?
Because a researcher’s track record goes with them when they move to another institution, a ‘transfer market’ emerges where universities compete frantically to hire top researchers in the lead up to each review.
A low weighting is given to postgraduate students and postdoctoral staff, even though these individuals generate most of the research. Early career academics are not given special consideration, and women are 1.9 time less likely to be judged ‘research-active’ than men. A greater institutional emphasis on research limits promotion prospects.
Many staff are labelled ‘not research-active’ while the work of others is classified according to the star system as
‘below national standard’, despite the best efforts of the researcher. Workloads for such staff increase markedly as teaching responsibilities are shifted from the top research performers. Staff witness the closure of departments and wonder if theirs will be next.
Those services that can demonstrate a direct benefit to research (e.g. libraries) can profit from the system; others suffer. General staff are encouraged to undertake more duties, freeing up time for academics to do research.
The research environment becomes more competitive and researchers become reluctant to share points with other institutions.
Several UK chemistry departments have closed.
Disciplines with a significant clinical load (e.g. medicine, nursing) fair poorly in relation to the basic sciences. Many departments become ‘teaching only’. In universities that earn a low rating overall, generic courses offered by many competing institutions are hit hardest in terms of student recruitment.
A significant emphasis is placed on journal impact factors, even though this has been shown to have no correlation with the citation rate of a specific paper in that journal.
Universities take credit for research produced at other institutions when a new researcher joins them. Dubious performance indicators used to measure research quality and impact, such as grant income, number of PhD students and best four papers.
The cost of the exercise is intended to be less than 1% of the funds distributed, but is likely to far exceed this. The direct cost to funding councils who administer the UK system is estimated at up to $67 million. In Australia, the indirect cost to institutions alone is estimated at $50 million.
Quality assessment made using expert peer review already occurs for most competitive grants and journal publications. In turn, grant income and publication output inform block funding through DEST points. Current reporting requirements for universities are already excessive. The distribution of Commonwealth research
Association of
University
Teachers 2004;
Williams 1998
Beckett 2005;
Piercy 2000
Oppenheim &
Stuart 2004;
Piercy 2000
Piercy 2000;
Tomlinson 2000;
Williams 1998
Banatvala, Bell &
Symonds 2005;
Beckett 2005;
McNay 2003;
Piercy 2000;
Smith 1998;
Tomlinson 2000;
Williams 1998
Group of Eight
2005;
Williams 1998
Mullarvey 2005;
Nelson 2004
10

MN Sillence - Wishing on a Star
How to achieve
‘quality’ research
Implications for teaching quality
Maintaining diversity funds is already skewed heavily in favour of a few elite institutions.
See below
A key aspect of the RQF is to evaluate the quality and impact , as well as the quantity of research in a particular discipline. Regardless of the variable measured, four key elements must be present for any type of research to proceed. These are illustrated in
Figure 1.
Research performance
Motivation Skills Time
Recruitment
Confidence
Support
Curiosity
Barriers
Disincentives
Training
Workload
Human
Resources
Plan
Teaching
Teaching support
Workforce
Plan
Course
Co-ordination
Other duties
Teaching relief
Research
Management
Plan
Resources
Infrastructure
Plan
Figure 1. Factors that contribute to research performance and points of influence of various management plans.
Motivation
Motivation to conduct research is the most difficult element to quantify and monitor, but is perhaps the most important. At first glance the ability to motivate staff would seem to belong to the domain of leadership, as it is well known that an important attribute of good leaders is their ability to convey a vision that is motivating and
11

MN Sillence - Wishing on a Star inspirational to others (Kotter 1990). However, motivation can come from a number of sources and is subject to the influence of management and planning, in so far as recruitment strategies that place a significant weighting on this attribute, can be incorporated as part of a human resources plan. Beyond this, confidence and curiosity have been identified as the attributes associated most closely with those who publish at CSU (Hemmings, Smith & Rushbrook 2004). While these may be classified as factors that are ‘internal’ to the individual, it can be easy to destroy a person’s motivation either through micromanagement, or by imposing upon them an excessive administrative burden in the name of compliance, accountability and quality control
4
.
Skills
Research requires a certain level of skill and training. This is another area where management and recruitment strategies are important, and over recent years the research-intensive faculties at CSU have deliberately favoured new staff that have a postgraduate research qualification. At the same time, many existing staff have been encouraged to undertake PhD programs to upgrade their skills and qualifications.
Unfortunately, this strategy does not go far enough. In many fields of research a PhD alone does equip a person adequately, to compete for limited research funds on a highly competitive national or international stage. New staff members with no postdoctoral training and few publications on their CV, face an enormous struggle to earn credibility and acquire sufficient resources to make any headway. Internal small grant schemes may help a few to get ahead, but in the amount of time that it takes a
CSU academic to develop a concept, apply for funds, conduct a small-scale project, then publish the results, a postdoctoral counterpart working in a well-funded, wellequipped laboratory in another institution, could reasonably be expected to produce six times the output
5
.
Appointing senior academic staff at level D or E in each discipline provides an opportunity for the mentoring and support of early career researchers, but it is naive to rely on this strategy alone. Many new staff harbour the expectation of academic freedom, and are eager to pursue their own research interests rather than to become absorbed by a larger unit. The important factor here is to make appointments at the appropriate level of seniority, to be prepared to seek out and pay for experience, even for junior positions, and to be realistic about what can be expected early of an earlycareer academic.
Finally, the university can and does provide some generic skills training and professional development activities in areas such as grantsmanship and statistical methodologies. These are mostly organised by the Centre for Research and Graduate
Training as part of the CRGT Triennial Plan (2003-2005). However, the value of
4 According to Zwetsloot and Bruijns (2001), the Hogeschool van Amsterdam was widely acknowledged for the excellence of its quality control systems, but did not rank highly in quality surveys. These authors believe that less attention should be directed at instruments and more directed towards behaviour, to help employees think and act according to quality standards. In the context of research at CSU, this would determine the way we deal with clients, our attitudes towards research, and the professionalism we show when responding to our client’s needs.
5 Based on the production of one journal paper from a pilot project, versus three papers per year for a full-time postdoctoral fellow with no teaching commitments.
12

MN Sillence - Wishing on a Star these exercises and their capacity to enhance research performance at CSU needs to be investigated, and should be judged using outcome indicators that go beyond the current measure of attendance rates.
Time
If a poll was conducted among academic staff, to determine the factor that limits research the most at CSU, time would be voted number one. The time available to do research is a function of workload, which at CSU is often used synonymously with teaching load. This is usually counted and reported in terms of ‘equivalent full-time student load’, but actually depends as much on other factors, as on the number of students taught.
The number of subjects a lecturer has to coordinate plays a role, as each subject carries administrative requirements, and most require the maintenance of a distanceeducation mail package. The demands of teaching vary considerably according to the discipline - whether there is a significant practical component, for example. In some cases, laboratories and tutorial rooms are too small to accommodate the entire practical class as one group, and each session must be repeated up to three times.
Furthermore, apart from teaching and its associated administrative activities, staff are expected to devote some of their time to serving the university or the community.
A workforce plan is currently being developed at CSU that aims to free up more time for research by academic staff. Reducing the number of subjects offered will be of benefit, and those subjects with low enrolments will be the first to disappear.
Consolidating the number of disciplines taught, should lead to a greater critical mass of staff within each discipline, which in turn, should lessen the burden that is often shared among co-workers when one staff member becomes unavailable for teaching.
Other initiatives include the use of general staff to perform duties traditionally expected of academics, such as aspects of course coordination. This has worked well in practice in some schools, but may breech union agreements. Another effective strategy has been to appoint a course manager (an academic who undertakes responsibility for coordinating several courses in the school in lieu of teaching duties), rather than spread the coordination load among several staff.
Finally, an obvious solution is to achieve a more favourable ratio of staff to students at CSU, which is currently among the worst in the sector. With a recent increase in subject fees, the University is in a financial position to achieve this for the first time in many years. As this is likely to be done by employing more staff rather that taking on fewer students, the human resources plan for staff recruitment will become even more critical in determining whether research performance will improve as a result of this strategy.
Resources
Resource needs vary considerably between disciplines. Apart from the financial resources that are won in the form of grants and contracts, certain disciplines depend heavily on infrastructure and equipment. Infrastructure management at CSU has tended to be reactive, especially concerning research activities when compared with developments in learning and teaching. Individuals or research groups have
13

MN Sillence - Wishing on a Star sometimes earned large grants that suddenly require space for additional staff or equipment, and University Properties has been the last section to find out about it.
Nowadays, with new leadership, a more proactive approach is being adopted, and faculties, schools, and in some cases individual research centres, are being consulted about their future plans for research development.
Central resources such as the library are equally important to research capacity, research assessment, and ultimately, research performance. Libraries possess expertise and databases that could be used to self archive CSU research papers and
CVs, or generate performance indicators such as citation rates and journal impact factors. Thus, library staff could become an important resource in facilitating an internal research assessment exercise, that would better prepare CSU for the RQF.
Oppenheim and Stuart (2004) report a correlation between institutions’ spending on their libraries, and the scores earned in the UK Research Assessment Exercise.
Although this relationship may not be causal, it is clear that the universities that value research, also value their libraries.
The Research Management Plan
The University Research Management Plan is at the heart of any strategy to improve research performance. From the previous section it should be apparent that this plan needs to be integrated with a human resources plan, a workforce plan and an infrastructure plan, and needs urgent revision in the context of the RQF if the maximum benefit is to be achieved.
While the Research Management Plan alone has the scope to provide some infrastructure (equipment grants) a little time (teaching relief via internal grants) and generic training (skills workshops), improving motivation is perhaps the most difficult challenge. In fact, it is far easier to create motivational barriers through insisting on a high level of compliance and introducing complex quality control measures as discussed earlier.
An important strategy in research management and leadership would be the development and promotion of a vision for research at CSU. Senior managers would communicate with research-active staff more often and more directly. They would learn more about the research that is going on in each field, and communicate what they have learned to others to encourage collaboration. Significant research achievements by CSU staff would be advertised shamelessly, to illustrate what it is possible to achieve. Overall, a sense of pride, commitment, enthusiasm and determination would be built at part of the University culture.
There is no doubt that of all the adverse effects caused by the UK RAE, its impact on teaching has been among the most significant - yet a surprising silence surrounds this issue (Jenkins, 1995). The impact on teaching has not been a topic high on the agenda for discussion at the various RQF forums in Australia. Nevertheless, there are several ways in which teaching can be affected by research evaluation systems.
14

MN Sillence - Wishing on a Star
Although governments may encourage universities to put an effort into both research quality and teaching quality, the incentives they provide can be very different. In the
UK, whereas $2 billion is tied to the outcomes of the RAE, no money is allocated on the basis of the Teaching Quality Assessment Exercise, even though departments that perform very poorly risk loosing their accreditation (Jenkins, 1995). Similarly, the
Australian government has announced plans to introduce performance-based funding for learning and teaching in 2008, perhaps to balance the effects of the RQF, but the money that will be tied to this scheme amounts to $113 million (Goulter 2005), which is only 10% of the funds tied to the RQF.
As good research can inform good teaching, theoretically, improved research performance could be a good thing for teaching and learning. However, in practice this is seldom the case. Strategies designed to enhance research performance often result in a greater distance between the researcher and the student, so that fewer students are taught in an environment where research is combined with teaching
(Beckett 2005). To increase their RAE scores, UK universities have been known to lure top researchers with the promise of research only positions (Jenkins 1995; Piercy
2000). An extreme example of a culture led by the research imperative can be found in many American universities, where tenure is linked to research performance.
Withholding job security is an excellent way to focus the heart and mind of an early career academic, and has resulted in a culture of formidable research effort where much undergraduate teaching is farmed out to postgraduate students and casual helpers, at the expense of quality. Accordingly, there has been considerable criticism in the USA by academics, politicians, students and taxpayers, for a general neglect of students and poor teaching quality (Jenkins, 1995).
Across the disciplines pedagogical research has suffered in the UK as journal publications in this category are either not recognised or not valued by the expert assessment panels (Jenkins 1995; McNay 2003). Also, less time is spent on other activities that do not earn RQF points, such as writing undergraduate textbooks, or developing innovative teaching methods.
The RAE has had a greater impact on some disciplines than on others. Fields that have a significant clinical component, such as medicine and nursing, have been hit much harder than the basic medical sciences, such biochemistry. Several chemistry department closures have been attributed to the RAE, including the much publicised case of Exeter, where a Noble laureate returned his honorary doctorate to the
University in disgust (Beckett 2005). Academics in education have called for the exercise to be abandoned (McNay 2003), while the head of a business school maintains that trying to improve the RAE score in his department would be
“fundamentally stupid” (Piercy 2000).
Finally, teaching is affected when a department earns a poor RAE score, and the university either looses research funds or chooses to deploy them elsewhere. In the
1996 UK exercise, only six out of twenty medical schools managed to earn a five star rating in at least one unit of assessment (Tomlinson, 2000). Thus, a small number of elite schools has emerged, while the rest have entered a spiral of decline and are on their way to becoming ‘teaching only’ departments. As research can inform teaching, there is an inevitable loss to the students.
15

MN Sillence - Wishing on a Star
Leadership and management strategies
If teaching and learning at CSU are to survive the RQF, it will not be sufficient for university leaders merely to promote the vision that ‘research informs good teaching’, they must show their commitment to this by backing teaching quality initiatives with the type of funds traditionally tied to research. Performance based funds for teaching quality provided by the government, must be supplemented to the extent that this exercise becomes as meaningful to academic staff as research.
Another strategy that will be attempted as part of the Workforce Planning Project, is to align disciplines through the selective deployment of resources, so that the areas of greatest research strength at CSU match the areas of greatest undergraduate student demand. The University could also resolve to avoid recruiting ‘star researchers’ in discipline areas where there is little undergraduate demand.
It has been suggested that all academic staff should be compelled to do a minimum amount of teaching. Conversely, all teaching staff should be encouraged to achieve a minimal level of engagement with research, perhaps by attending an annual research conference. This is very different to the strategy of research concentration or striving for research excellence in a limited number of discipline areas, as dictated by the RQF system.
Judging a variety of institutions by a fixed set of criteria creates an inevitable pressure for those institutions to start to think alike and behave alike. Regardless of the institution, experience in the UK has shown that the RAE favours basic science departments, where workers in large modern disciplines such as cellular and molecular biology publish routinely in journals that enjoy high impact factors, principally because of the size of their audience. In contrast, those in specialised areas, in applied sciences, or in clinical fields, fair poorly regardless of the intrinsic quality of their work. As a result, a consistent pattern has emerged where chemistry departments have closed down across the country, others have become ‘teaching only’, and clinical departments have had difficulty in recruiting new staff (Banatvala,
Bell & Symonds 2005). Multidisciplinary and multi-institutional work have also been threatened.
It would be suicidal for Charles Sturt University to attempt to mimic competitors such as The University of Sydney or The University of Melbourne, as runners up in the
RQF are unlikely to be treated very favourably. Instead, the University must maintain its own distinct character, and work to its own strengths.
Leadership and management strategies
Effective leaders articulate their vision and values clearly (Kotter 1990; Reser &
Sarros 2000). If the leaders at Charles Sturt University value diversity, not only does this need to be expressed - it needs to be rewarded. There must be a tangible and transparent commitment to subsidise fields that are considered important, but not selfsufficient. Similarly, if the University chooses to maintain disciplines in fields that are
16

MN Sillence - Wishing on a Star not strong in research (according to RQF criteria), it is important that the staff in those disciplines have equal employment and promotion opportunities. The appointment of non-research staff to senior, high-profile positions in such disciplines, would signal the University’s commitment, and would provide role models to inspire others.
Other Universities may now begin to look seriously at their discipline profile, as a reaction to the RQF. At CSU, the executive is taking a more prescient approach through the current Workforce Planning Project, which aims to narrow and strengthen the discipline base, and to align disciplines so that as far as possible, research is supported most in those disciplines with a strong undergraduate demand. This strategic initiative is designed to improve efficiency, sustainability, and working conditions for staff, as well as promoting the linkage between teaching and research.
When selecting those disciplines for culling or increased investment, the University needs to consider a range of factors other than the RQF exercise, such as regional importance, uniqueness and critical mass.
As long as discipline groups rely on University support, it is appropriate for the
University to apply some form of performance assessment from time to time.
However, performance should be gauged according to criteria negotiated between the discipline group and the university, rather than against criteria set by an external body such as the RQF panel. In some cases these criteria will coincide with those of the
RQF, but in other cases different criteria will signal the unique aspects of those diverse disciplines the University wants to retain.
Once the goals and targets have been set, it can be helpful if these are in plain sight at all times, so that progress can be checked regularly. In the commercial sector, Chuck
Trowbridge of Eastman Kodak was regarded as an outstanding leader, and used the simple but effective measure of posting charts in the canteen that allowed all workers to see the rate of progress on a daily basis (Kotter 1990). Some schools at CSU encourage staff to post their annual publications on a bulletin board in a conspicuous part of the building. However, as far as the writer is aware, none set this against a clear target of how many papers the school aims to publish in a given year. Some academics are likely to rebel against such crude ‘factory floor’ management tactics, but if the net result is an increase in output and morale, the exercise would be worthwhile.
Setting goals and monitoring progress regularly will only have any value if the team has the resources to do the job in the first place. Although the award of RQF money will be based on the outcome of discipline assessment, it is almost certain that universities will retain the discretion over how to spend this money (DEST 2005g), as is the current case for block funds earned under the DEST points system. This presents both an opportunity and a dilemma. There will be a strong temptation to return as much of the funds as possible to the income source, to enhance or maintain this income stream. Any other strategy carries the risk that workers in the star disciplines will loose the motivation to perform, or at least to put any serious effort into the assessment exercise. Furthermore, in cases where the research discipline is dependant on University funding to some extent, deployment of the funds elsewhere will lead to an inevitable decline in productivity and income, that will profit nobody.
On the other hand, the University will have the opportunity to use RQF funds to support early career researchers, other disadvantaged groups, and discipline areas that
17

MN Sillence - Wishing on a Star the University values, but which do not rate under the RQF. This is a mechanism the
University can use to demonstrate its commitment to diversity within the institution, although it must strike a fine balance in doing so.
Finally, for disciplines not rated under the RQF it will be essential to maintain morale and commitment. Staff in these disciplines should be encouraged to seek actively, other forms of recognition such as the response to press releases, community forums, and presentations to stakeholders and clients. It is also important that positive feedback is documented. Not only will this help to build confidence, but it may be of use in future assessment exercises, such as the examination of ‘third stream activities’ where there will be less emphasis on academic quality, and more emphasis on impact and engagement with stakeholders.
While the experience of others suggests that the RQF could be extremely damaging to
CSU, at the very least it will compel the University to consider its values and priorities. It should encourage not just strategic planning, but strategic thinking.
Apart from the general question “What is the appropriate response when your biggest client tells you how to run your business?” the University also needs to consider how each action taken in response to the RQF, will affect the services CSU offers to its students and other clients. The real cost of these actions needs to be weighed carefully against the anticipated return in terms of RQF funding. Perhaps the most important question of all is to ask
“Are the RQF goals likely to produce the type of University we want?”
The most important step that CSU can take in the short-term is to avoid ‘RQF fever’, and maintain a sense of perspective concerning the importance of research and government block funding, relative to its other activities and income streams. The
University must continue to show leadership and innovation in developing courses and research services that deliver value to its clients. We must also seek written acknowledgement and document other evidence that such services are valued. The disadvantage of not being able to post numerous RQF-awarded stars on our web site, must be countered by more imaginative and aggressive marketing strategies. Finally, the importance of gaining political support from key stakeholders in the region, and of making the government aware of this, can not be over-stated.
Most people would agree that certain institutions, such as the church and judiciary, should be autonomous of government. Academia is one such institution, which has enjoyed a long tradition of providing a challenging voice that helps to keep governments honest. Universities have also had a long tradition struggling against those who would wrest this power away (Boyd & King 1972). Setting the goals and priorities for CSU is not the business of Ministers Howard and Nelson, it is not the business of DEST, and it is certainly not the business of a panel of discipline experts drawn from our competitors. It is the business of the University Council, the business of the executive, and the business of all CSU staff who are willing to show leadership in this regard. Instead of aiming to become a five star research university based on our
18

MN Sillence - Wishing on a Star ability to impress other academics, we should aim to become a five star university based on the four pillars in our strategic plan (CSU 2004), plus one extra.

Research excellence - delivering outcomes that improve the quality of life for our constituents.

Teaching excellence - educating professional world-wide, who are committed to the growth and development of regional communities.

Regional engagement - contributing to the cultural, social and economic capital of our regions.

Resource generation and management - ensuring that CSU will still be in the business of serving Australia in the next century.

Values - demonstrating a respect for truth, integrity, equity, fairness, diversity, and above all, self-determination.
With these values and this mindset, CSU will be equipped to weather the RQF storm, and use the exercise as a spur to achieve its own goals and priorities, while becoming
Australia’s premiere regional university.
19

MN Sillence - Wishing on a Star
Association of University Teachers 2004, ‘UK academic staff 2002-3 – Gender and research activity in the 2001 Research Assessment Exercise’, viewed 29 June
2005, <http://www.aut.org.uk/index.cfm?articleid=873>.
Australian Vice-Chancellors’ Committee 2005a,
‘Enhancing Australia’s core research capacity – developing a Research Quality framework: The AVCC proposal’, viewed 1 July 2005,
<http://www.avcc.edu.au/documents/publications/policy/submissions/AVCC-
Proposal-Enhancing-Australias-core-research-capacity.pdf>.
Australian Vice-Chancellors’ Committee 2005b, ‘Australian research statistics’, viewed 25 May 2005,
<http://www.avcc.edu.au/documents/publications/stats/Research.pdf>.
Banatvala, J, Bell, P & Symonds, M 2005, ‘The Research Assessment Exercise is bad for UK medicine’, The Lancet, vol. 365, pp. 458-460.
Beckett, F 2005, ‘Star power threatens UK research’
The Guardian, 10 March 2005, viewed 29 June 2005,
<http://education.guardian.co.uk/RAE/story/0,7348,1433903,00.html>.
Boyd, W & King, EJ 1972, The history of western education , Adam and Charles
Black, pp. 125-158.
Charles Sturt University 2004, Strategic Plan , Charles Sturt University, viewed 1 July
2005, <http://www.csu.edu.au>.
Department of Education, Science and Training 2003, ‘Our Universities: Backing
Australia’s future’, Policy and legislative framework of reforms, Passed by
Parliament 5 December 2003, last updated 19 May 2005, viewed 25 July 2005,
<http://www.backingaustraliasfuture.gov.au/>.
Department of Education, Science and Training 2004, ‘Rationalising responsibility for higher education in Australia’ , Issues paper, Australian Government.
Department of Education, Science and Training 2005a, ‘Building better foundations for higher education in Australia: A discussion about re-aligning
Commonwealth-State responsibilities’ , Discussion paper, Australian
Government.
Department of Education, Science and Training 2005b, ‘Building university diversity:
Future approval and accreditation processes for Australian higher education’,
Issues paper, Australian Government.
Department of Education, Science and Training 2005c, ‘A national quality strategy for Australian transnational education and training’, Discussion paper,
Australian Government.
20

MN Sillence - Wishing on a Star
Department of Education, Science and Training 2005d, ‘Research Quality
Framework: Assessing the quality and impact of research in Australia’, Issues paper, Australian Government.
Department of Education, Science and Training 2005e, ‘Research Quality
Framework: Assessing the quality and impact of research in Australia’,
Advanced approaches paper, Australian Government.
Department of Education, Science and Training 2005f, ‘Research Quality Framework:
Submissions in response to issues paper, viewed 15 June 2005,
<http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_i ssues/research_quality_framework/nsf.htm>.
Department of Education, Science and Training 2005g, ‘National Stakeholders Forum for a Research Quality Framework’, viewed 30 July 2005,
<http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_i ssues/research_quality_framework/rqf_subs.htm>.
Goulter, I 2005, ‘Great Expectations or Bleak House? - A university for the next twenty five years - Defining/designing our preferred future’, Vice-Chancellor’s discussion paper, viewed 15 April 2005,
<http://www.csu.edu.au/division/vcoffice/vc/25years/25years.htm>.
Group of Eight 2005, ‘Research Quality Framework – Group of Eight response to the issues paper’, viewed 31 May 2005,
<http://www.go8.edu.au/news/2005/Go8%20Response%20to%20RQF%20Issu es%2004.05.05.pdf>.
Hemmings, B, Smith, E & Rushbrook, P 2004, ‘Factors differentiating between those academics who do and who do not publish refereed works’, Issues in
Educational Research, vol. 14(2), pp. 156-166.
Jenkins, A 1995, ‘The Research Assessment Exercise, funding and teaching quality’,
Quality Assurance in Education, vol 3, pp. 4-12.
Karmel, P 2000, ‘Public policy and higher education’, Australian Journal of
Management, vol. 26, Special Issue, pp. 123-143 .
Kotter, JP 1990, ‘What leaders really do’,
Harvard Business Review , vol. 68, pp. 104-
111.
McNay, I 2003, ‘Assessing the assessment: an analysis of the UK Research
Assessment Exercise, 2001, and its outcomes, with special reference to research in education’, Science and Public Policy’,
vol. 30 (1), pp. 47-54.
Mullarvey, J 2005, ‘Room to move prootes quality and relevance in higher education’, Australian Vice-Chancellors’ Committee, Articles, 16 June, 2005, viewed 11 July 2005,
<http://www.avcc.edu.au/content.asp?page=/news/articles/2005/avcc
_article_02_05.htm>.
21

MN Sillence - Wishing on a Star
Nelson, B 2004, ‘Higher education report for the 2004 to 2006 triennium’,
Australian
Government Department of Education, Science and Training, Commonwealth of Australia.
Oppenheim, C & Stuart, D 2004, ‘ Is there a correlation between investment in an academic library and a higher education institution’s ratings in the Research
Assessment Exercise?’, Aslib Proceedings, vol. 56, pp. 156-165.
Piercy, N 2000, ‘Commentary Why it is fundamentally stupid for a business school to try to improve its research assessment exercise score’,
European Journal of
Marketing, vol. 34, pp. 27-35.
RAE 2008, Research assessment exercise, Higher Education Funding Council for
Wales, Higher Education Funding Council for England, Scottish Higher
Education Funding Council, and Department for Employment and Learning
(Northern Ireland), viewed 29 June 05, <http://www.rae.ac.uk>.
Reser, CL & Sarros, JC 2000, ‘The concept of leadership’,
Faculay of Business and
Economics, Monash University, Working paper 93/100, December, pp. 2-14.
Roberts, G 2003, ‘Review of research assessment’, Report to the UK funding bodies issued for consultation May 2003, Higher Education Funding Council for
England, viewed 30 June 2005,
<http://www.hefce.ac.uk/pubs/hefce/2003/03_22.htm>.
Smith, R 1998, ‘Unscientific practice flourishes in science’
British Medical Journal,
Editorial , vol. 316, pp. 1036-1040.
Tomlinson, 2000, ‘The research assessment exercise and medical research’ British
Medical Journal, vol. 320, pp. 636-639.
The Allen Consulting Group 2004, ‘Establishing and Australian Research Quality
Framework: Summary of discussion from Multi-disciplinary workshop’ Report to The Australian Government Department of Education, Science and Training,
February 2005, viewed 29 June 2005,
<http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_i ssues/research_quality_framework/workshops.htm#Discipline-specific/crossdisciplinary_workshops>.
The Allen Consulting Group 2005a, ‘Measuring the impact of publicly funded research’, Report to the Australian Government Department of Education,
Science and Training, Commonwealth of Australia.
The Allen Consulting Group 2005b, ‘Establishing an Australian Research Quality
Framework: Summary of discussions from the eight State and Territory RQF stakeholder forums’ Report to The Australian Government Department of
Education, Science and Training, April 2005, viewed 29 June 2005,
<http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_i ssues/research_quality_framework/consultation_forum.htm>.
22

MN Sillence - Wishing on a Star
Williams, G 1998 , ‘Misleading, unscientific and unjust: the United Kingdom’s research assessment exercise’, British Medical Journal , vol. 316, pp. 1079-
1082.
Zwetsloot, R & Bruijns, V 2001, ‘From quality control to quality assurance: how to change behaviour’, Quality Research International , The Sixth Quality in Higher
Education Seminar: The End of Quality?, viewed 31 June 2005,
<http://www.qualityresearchinternational.com/eoq/abstracts.htm>.
23

MN Sillence - Wishing on a Star
AVCC
CSU
DEST
IGS
RAE
RPF
RQF
UK
Australian Vice-Chancellor’s Committee
Charles Sturt University
Department of Education, Science and Training
Institutional Grants Scheme
Research Assessment Exercise
Regional Protection Fund
Research Quality Framework
United Kingdom
24