LONDON’S GLOBAL UNIVERSITY
ACADEMIC COMMITTEE
WORKING GROUP ON STUDENT FEEDBACK PROVISION
Report
Executive Summary
The Working Group on Student Feedback Provision met three times to examine the International
Student Barometer survey and to consider how UCL could best make use of the feedback
received from it. WGSFP also considered other related matters including the uses of National
Student Survey data. This report:
-
presents the background information on the establishment of WGSFP, its remit and
rationale [paragraph 1];
-
outlines the key areas of focus considered by the WGSFP and important matters and
principles [paragraph 2];
-
reports on the discussion of the International Student Barometer [paragraph 3];
-
reports on the discussion of the National Student Survey [paragraph 4];
-
reports on other matters discussed by the WGSFP [paragraph 5];
-
informs the Committee of the conclusions of the WGSFP [paragraph 6];
-
informs the Committee of the WGSFP recommendations [paragraph 7];
-
informs the Committee of action required [paragraph 8];
-
presents the WGSFP terms of reference at [Annex 1]
Key to abbreviations:
AC
AS
DSSCC
HEI
ISB
JSSC
MIS
NSS
QMEC
WGSF
WGSSF
WGSFP
Academic Committee
Academic Services
Departmental Staff Student Consultative Committee
Higher Education Institution
International Student Barometer
Joint Staff Student Committee
Management, Information and Services
National Student Survey
Quality Management and Enhancement Committee
Working Group on Student Feedback
Working Group on Student Support Feedback
Working Group on Student Feedback Provision
WGSFP Report to AC 23 October 2007
1
Background and remit of the WGSFP
1.1
The Working Group on Student Feedback Provision followed on from the
work of two previous working groups. The first, the Working Group on
Student Feedback primarily considered feedback on teaching and learning at
departmental level and reported to the Quality Management and
Enhancement Committee in 2006. The second, the Working Group on
Student Support Feedback considered the feedback received by the UCL
corporate services and how UCL could gain an overview of it in order to
identify common student concerns and share best practice between the
support services. The WGSSF report was approved by Academic Committee
at its meeting of 6 March 2007.
1.2
At its meeting of 12 December 2006, AC considered the International Student
Barometer survey and resolved that the WGSSF, or a sub-group thereof, should
examine the survey in depth and take forward the issues raised by it and other
similar surveys. WGSSF considered that it would be more effective to establish
a small working group to examine matters around the ISB in order to keep this
work discrete from its own deliberations and from the findings of the WGSF.
1.3
The Working Group on Student Feedback Provision was established to consider
these matters and held three meetings in March, April and June 2007.
1.4
The WGSFP membership and terms of reference are at Annex 1.
2
WGSFP Key Areas of Interest
2.1
In order to facilitate its discussions, WGSFP identified a number of key
areas and points to consider in addition to the terms of reference:
2.2
(i)
the outcomes and matters arising from the WGSSF report in as far as
they relate to the new group;
(ii)
ways to embed the surveys (and findings thereof) into future UCL
activity;
(iii)
whether the ISB represented a suitable single tool to assess UCL
corporate service;
(iv)
the uses of the National Student Survey data;
(v)
dissemination of the resulting survey data to the faculties and
departments and reportage to the UCL committee structure.
WGSFP noted the importance of being realistic regarding what could usefully
be recommended regarding the ISB and other surveys. It should be noted
that all the data sources under consideration were in some ways flawed and
WGSFP considered that they should not be used in isolation from one
another. The key task was to identify useful data from the various surveys
which would help UCL to understand and utilise its student feedback
more effectively.
2
WGSFP Report to AC 23 October 2007
3
International Student Barometer
Background
3.1
The International Student Barometer, operated by the commercial
organisation I-graduate, seeks international student views on their experience
whilst at university including views on learning, the quality of the living and
social environment and the standards of service. The ISB covers
undergraduate, taught postgraduate and research students and is a highly
sophisticated tool which allows institutions to interrogate a wide range of data
with results that can be presented in a number of ways. The ISB also allows
for comparison with other institutions and enables UCL to benchmark its
performance against other Higher Education Institutions taking part, which
includes a number of Russell Group institutions.
3.2
UCL had chosen to participate in the ISB in the first and third terms. The ISB
had been budgeted for the 2007-08 session but a decision will be required as
to whether to continue with it thereafter.
3.3
WGSFP noted that the UCL Registry Educational Liaison Department was
currently responsible for the management of the ISB and considered the main
resource factors which related to the survey:
(i)
Operation of the ISB - the ISB needed to be managed on a daily basis
in preparation for and during the course of the survey, although the
processes involved have now been set-up. The estimated staff time
for one person to manage the ISB operation was three months per
annum, dependent on the amount of analysis of the data undertaken
(RT)
(ii)
Analysis of the ISB – requires significant input from staff, although
technical expertise is not essential as much of the core analysis is
carried out by I-graduate;
(iii)
Further analysis of the data – I-graduate can provide additional
analysis of the ISB data for a fee which will vary according to the
extent of the analysis required. However queries would be limited to
the parameters given to I-graduate to conduct the analysis and it was
noted that it might be more productive and effective to conduct any
additional analysis in-house;
(iv)
Overall cost of the ISB – I-Graduate currently charge £13,865 per
annum for the survey. There is also a significant additional cost to
UCL staff time and resources in dealing with the survey.
Key Questions
3.4
WGSFP noted that there were three key questions which needed to be
addressed in order to fully consider how to incorporate the ISB and the other
surveys into UCL planning and committee cycles:
(i)
which body or committee should have “ownership” of the surveys, to
receive and consider results, decide what use to make of them and
how to disseminate the findings?
3
WGSFP Report to AC 23 October 2007
(ii)
which UCL office should have responsibility for the administration of
the surveys i.e. managing the operation of the survey(s), digesting the
results and reporting them to the “owner” and if required,
disseminating the results;
(iii)
how to identify and decide on the timing issues around the results of
the surveys and how best to incorporate data and findings into UCL
planning and committee cycles?
Value of the ISB
3.5
3.6
WGSFP considered whether, given its expense, UCL should continue to
subscribe to the ISB. It was agreed that the survey was of value for a
number of reasons, which included:
(i)
the ISB provided detailed data on student views which can be
benchmarked against other higher education institutions (including a
number from the Russell Group), and thus allows for comparison. It
was further noted that the more HEIs use the ISB in future, the greater
its value would become;
(ii)
the survey provided detailed student views on the whole student
experience, not just academic matters. Valuable data was thus
received on accommodation, resources and other matters and
provided a fuller picture of student opinion than was given by currently
existing internal and external surveys;
(iii)
the survey was thus useful in supplementing and providing context for
other sources of student feedback, including the NSS and internal
student surveys (both academic and non-academic);
(iv)
the survey provided data twice each academic year, which allows for
problems to be identified and action to be taken quickly, as well as for
speedier monitoring of outcomes – this was advantageous in
comparison with annual surveys such as the NSS.
(v)
Working through I-graduate it is possible to drill-down into the data to
explore particular issues in depth with individual respondents.
WGSFP agreed that, given the above reasons, UCL should retain the ISB for
2007-08 as planned.

WGSFP recommends that UCL should retain the ISB for 2007-08
as planned (RECC 1).
Ownership and Management of the ISB
3.7
WGSFP agreed that the ISB would require a substantial resource
commitment by UCL and stressed the importance for a clear decision on the
“ownership” and responsibility for management of the ISB.
3.8
WGSFP noted that whilst it was important that AC received an overview of
the ISB results, it might not be the best placed committee to consider the data
in detail, due to the demands on its time and wider responsibilities. It was
also thought that the Senior Management Team would not be the appropriate
4
WGSFP Report to AC 23 October 2007
body to consider the data for similar reasons. WGSFP resolved that the
committee or body best equipped to consider the survey results would need
to include (or easily access) academic staff, students and staff from UCL
corporate services.
3.9
It was noted that the Joint Staff Student Committee provided a suitable point
of convergence for “ownership” of the ISB and that it might be the best placed
UCL committee to receive, filter and act on matters raised. JSSC met the
criteria for providing feedback to the students, with the added advantage of
providing them with direct input into any action arising. JSSC also has a
mechanism for reporting and feeding student concerns into management
level at UCL via its Annual Report to AC. This report outlines student
concerns, including a summary of the main concerns reported by
Departmental Staff Student Consultative Committees. Further to this, the
Chair of JSSC (and Dean of Students) wrote to departments, faculties and
UCL Corporate Services where necessary drawing attention to the student
concerns raised and seeking solutions to them. WGSFP agreed that it would
be possible to extend the JSSC Annual Report to also include the main
outcomes and concerns raised by the ISB and the NSS. It was further noted
that the JSSC members from the UCL corporate services would also enable
swift responses to any actions raised in their areas of responsibility. It would
also be possible, where appropriate, to extend membership to representatives
from other relevant corporate services.
3.10
WGSFP agreed that, given the advantages listed above, that JSSC should be
the main committee to receive data from the ISB and use it, together with
DSSCC minutes, as a primary source of student feedback in its Annual
Report to AC.

3.11
WGSFP also considered how best to provide faculties with disaggregated ISB
data, and provide advice on how to make best use of it. WGSFP agreed that
Faculty Teaching Committees would be the best receptacles for such
information and noted that they currently received data disseminated from the
NSS. It was further noted that faculties were also represented by an
academic member of staff on JSSC and that this would be another point of
contact between the ISB data and the faculties.

3.12
WGSFP recommends that JSSC should be the main committee to
receive data from the ISB and use it as a primary source of
student feedback in its Annual Report to AC (RECC 2).
WGSFP recommends that faculties should receive the ISB data
for consideration directly through their FTCs, as well as by the
faculty academic staff members of the JSSC (RECC 3).
WGSFP considered in depth where the ISB should be managed within UCL
and the most likely administrative departments to do this. It was noted that
the Educational Liaison Department of the Registry, although currently
managing the survey, did not have the remit or resources to continue to do so
and that this was indeed unsustainable in the long-term. WGSFP also agreed
that it should not be part of Academic Services’ role to manage the ISB.
Although AS was managing much of the administration and all of the analysis
and dissemination of the NSS data, it was noted that this was a HEFCE
requirement for HEIs and thus in line with the AS quality assurance role and
responsibilities. This quality assurance role is primarily focussed on the
5
WGSFP Report to AC 23 October 2007
academic experience. The ISB covers a wider range of issues and areas,
including feedback on other areas of the student experience, such as
accommodation, student finance or social or catering facilities. AS is not best
placed to deal with this wider focus and the anticipated data management and
reporting processes required to provide analysis of the student feedback
received. WGSFP also considered whether the survey could be managed
from the Dean of Student’s office (as Chair of JSSC and the driver behind a
number of UCL student feedback initiatives) and agreed that although this
was feasible, it would require substantial additional resources and staff to do
so.
3.13
WGSFP considered that in terms of managing and disseminating the ISB
data, the UCL Registry’s Management Information and Services might be the
best placed department to do so. MIS not only had the technical ability to
manage the ISB, it was also well placed within UCL Registry to understand its
context and how this related to student feedback in general. However,
WGSFP agreed that whichever department or section was chosen to manage
the ISB, a significant resource allocation (in terms of staff and funding) would
be required. Given this condition, WGSFP recommends that MIS be
considered as the UCL office responsible for the management of the ISB.

3.14
WGSFP further agreed that given the cost of the ISB in terms of the
necessary commitment of finance, resources and staff time, it was essential
to establish a process to monitor the survey and to review it at fixed intervals.
This process would need to focus on the effectiveness of the survey, its
response rate and currency amongst students, as well its value and
usefulness to UCL as an indicator of student feedback. It was also agreed
that this process would need to consider the wider usage of the ISB by other
HEIs and, where appropriate, their views on its value. WGSFP agreed that
this process would be most effective if conducted at the end of the next
session and on an annual basis thereafter (assuming that a decision was
made to retain the survey). WGSFP considered that it would be appropriate
for this review process to be conducted by the committee (or a sub-group
thereof) chosen to receive the ISB data, i.e. JSSC if Recommendation 2 is
accepted (see paragraph 3.10 above).

3.15
WGSFP recommends that subject to appropriate resource
allocation, MIS be considered as the UCL office responsible for
the administration of the ISB, managing its operation and
digesting and reporting the results (RECC 4).
WGSFP recommends that the arrangements for the ISB and other
surveys should be reviewed at the end of the 2007-08 session
and if retained, on an annual basis thereafter (RECC 5).
WGSFP agreed that the monitoring of the impact of developmental work
undertaken on future survey performance would need to be built into the
processes for management of the ISB. It was noted that should JSSC be
selected as the appropriate committee to receive the ISB data, it would be
able to monitor developments through its membership which included
academic, student and UCL service staff representatives. It would be
necessary to adjust the membership of JSSCC to ensure that appropriate
service areas were included.
6
WGSFP Report to AC 23 October 2007
3.16
WGSFP considered how best to incorporate UCL services in the
management of the ISB and identify those areas where there is potential for
developmental work which could realise a significant return. It was agreed
that this should be built into the planning process. It was noted that should
JSSC be selected as the main committee responsible for dealing with the
ISB, its members from the UCL corporate services (especially if extended to
include a representative from the Estates and Facilities Division) would be
well placed to consider the potential for development work from the survey.
4
National Student Survey
4.1
WGSFP also considered the National Student Survey, now in its third year.
The NSS took place in the second term and involved all final year
undergraduate students. The NSS is managed at UCL by Academic
Services, which has responsibility for co-ordinating the administration of the
survey in liaison with other UCL corporate services (e.g. providing the student
contact details with MIS and publicising the survey throughout UCL), as well
as reportage of the results to AC and the Quality Management and
Enhancement Committee. AS is also responsible for dissemination of the
more detailed NSS data available and supplies faculties with digests of their
results which are further considered by their FTCs.
4.2
The main NSS results are received and considered by AC each year, which
gives UCL an overview of the main trends and student concerns. The
management of the survey and the more detailed results are reported to and
considered by QMEC, which also receives all the data derived from the NSS
Dissemination Site, which in turn are distributed to faculties as digests.
4.3
WGSFP noted some reservations regarding the reliability of the NSS data,
particularly at lower levels of granularity, as institutions were not informed of
which students had completed the survey (for reasons of confidentiality) and it
was not always clear which specific programmes the response data referred
to. Despite this, the NSS clearly did contain useful information on student
views and WGSFP considered that should JSSC be the committee to receive
and consider the ISB data, it would also be useful for it to receive the overall
NSS data (i.e. the same information received by AC) in order to provide a
context to the ISB and triangulate the data received from the two surveys with
the information received from the DSSCC minutes.

WGSFP recommends that JSSC also receive and consider the
overall NSS figures in order to provide a context to the ISB and
triangulate the data received from the two surveys and the
DSSCC minutes (RECC 6).
5
Other Matters
5.1
WGSFP noted that Council was currently giving consideration to the
establishment of an annual report on the student experience, which was likely
to be partly based on data from the ISB and NSS. It was not clear what form
the report would take at present with a decision likely in the 2007-08 session.
5.2
WGSFP considered the ISB and other surveys in relation to UCL strategic
documents, including the UCL Corporate Plan 2006-2012, the International
7
WGSFP Report to AC 23 October 2007
Strategy and the Teaching and Learning Strategy. It was agreed that
although these documents were primarily concerned with overall strategic
matters (with little pertaining to specific student feedback matters), they
nonetheless provided an important background to the WGSFP’s work. In
particular, WGSFP took note of the key strategic point regarding the drive for
greater efficiency at UCL and of streamlining processes where possible and
was concerned to ensure that its recommendations avoid duplication of
already existing activity and be able to co-exist, complement and where
possible merge with both current and planned provision.
5.3
In terms of timing, WGSFP considered that the most effective time of year for
reportage of the first data from the ISB and the NSS results would be in
January each year. This would allow time for some action to be taken to
address any specific concerns arising from the surveys, although it was
acknowledged that it would require careful allocation of resources to ensure
that analysis and reportage to the committee(s) was conducted in time. It
was noted that there would be a financial cost attached to this. It was further
noted that the first JSSC meeting each year was held in November, fitting this
time-scale.
5.4
WGSFP considered the possibility of working with other HEIs using the ISB.
Though pooling resources with the other participants and sharing some of the
management processes could potentially save money and extend the range
of data to be received, any elaborate management structure was likely to
prove impractical. It was agreed instead to recommend the creation of a
relatively informal seminar for Russell Group ISB participants, hosted by UCL,
to identify areas of common interest and concern and explore the scope and
options for collaboration and sharing good practice.

5.5
WGSFP recommends that an informal seminar be established
with other Russell Group ISB participants in order to discuss
matters of common interest or concern. It was envisaged that
the seminar would be chaired by the Chair of WGSFP and QMEC
(RECC 7).
WGSFP noted that the ISB, NSS and other student feedback mechanisms
are not without cost in terms of financial and resource commitment. The ISB
cost some £11,800, with an additional cost in staff time in its management.
The NSS was also not a “free” survey, with many hidden costs in staff time
and materials, not least in preparing data, publicising the survey and reporting
and disseminating analysis of the data to UCL committees and to faculties.
WGSFP strongly urges AC to be mindful of this background when considering
the use of these sources of student feedback. The proposals in this report
are not cost-neutral. A significant resource allocation would be required in
order to successfully implement the recommendations and approval if given
should involve a commitment by UCL to resource at the appropriate level.

WGSFP asks AC to note that the successful implementation of
the above recommendations will require a significant resource
allocation (RECC 8).
8
WGSFP Report to AC 23 October 2007
6
WGSFP’s Conclusions
6.1
WGSFP discussed and agreed a number of matters and principles during its
deliberations:
(i)
not to rely on one single source of data in seeking feedback on the
student experience and to recognise the limitations of the main data
sources. The key task was to identify useful data from the various means
available and decide which could help UCL understand and utilise its
student feedback more effectively;
(ii)
to ensure that feedback from the surveys response data is conveyed back
to the students, for reasons of both transparency and credibility. Students
were more likely to take surveys seriously and complete them in future if
they were aware of the use made of the data collected and action taken;
(iii)
to involve student representatives in discussion of feedback, in order to
include them in the management processes and provide a sense of
ownership of the mechanisms involved. It was further noted that
successful feedback mechanisms tended to acknowledge this principle;
(iv)
to acknowledge the UCL strategy to strive for greater efficiency and
streamline processes where possible – the WGSFP recommendations are
intended to avoid duplication of already existing activity and be able to coexist, complement and where possible merge with both current and
planned provision;
(v)
to recognise the need for clear reporting lines within UCL for external
surveys such as the ISB, NSS and other surveys in order to enable the
student feedback obtained to inform UCL strategy and planning;
(vi)
to ensure that reporting mechanisms recognise the successes and
achievements which emerge from the surveys and other sources of
positive feedback and not to focus exclusively on areas of difficulty and
dissatisfaction.
6.2
WGSFP recognised that there were major resource issues regarding the
sources of feedback on the student experience (see paragraphs 3.7, 3.14 and
5.5). The surveys necessitate a substantial use of staff time for those
involved in their administration, not to mention the commitment of the
committees required to oversee and make use of the surveys. WGSFP
reiterates its belief that a substantial resource investment would be required
in order to implement the recommendations of this report. Although WGSFP
believes that on balance such an investment will be beneficial to UCL, it
believes that should these recommendations be implemented, close
monitoring and review of the ISB and NSS will be required.
6.3
The above matters and principles outlined informed the recommendations of
WGSFP which are given below (Section 7).
9
WGSFP Report to AC 23 October 2007
7
Recommendations
7.1
WGSFP makes the following recommendations:
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
(viii)
that UCL should retain the ISB for 2007-08 as planned (paragraphs
3.5 and 3.6);
that JSSC should be the main committee to receive data from the ISB
and use it as a primary source of student feedback in its Annual
Report to AC (paragraphs 3.9 and 3.10);
that faculties should receive the ISB data for consideration directly
through their FTCs, as well as by the faculty academic staff members
of the JSSC (paragraph 3.11);
that subject to appropriate resource allocation MIS be considered as
the UCL office responsible for the administration of the ISB, managing
its operation and digesting and reporting the results (paragraph 3.13);
that the arrangements for the ISB and other surveys should be
reviewed at the end of the 2007-08 session and if retained, on an
annual basis thereafter (paragraph 3.14);
that JSSC also receive and consider the overall NSS figures in order
to provide a context to the ISB and triangulate the data received from
the two surveys and the DSSCC minutes (paragraph 4.3);
that an informal seminar be established with other Russell Group ISB
participants in order to discuss matters of common interest or concern.
It was envisaged that the seminar would be chaired by the Chair of
WGSFP and QMEC (paragraph 5.4);
asks AC to note that the successful implementation of the above
recommendations will require a significant resource allocation
(paragraphs 5.5).
8
Action required
8.1
AC is asked to note the conclusions of the WGSFP report and comment as
appropriate.
8.2
AC is asked to further note that the work of the WGSFP is now complete and
that the group has stood down.
Professor Chris Carey and Rob Traynor
Academic Services
2 August 2007
10
WGSFP Report to AC 23 October 2007
Annex 1
Academic Committee
Working Group on Student Feedback Provision
Terms of Reference and Membership
Terms of Reference
(1)
consider the Student Barometer and other similar survey outcomes in detail
and alongside the UCL Corporate Plan 2006-2012, the International Strategy,
the Teaching and Learning Strategy and, as appropriate, supporting
strategies;
(2)
working in consultation and collaboration with service heads, identify those
areas where there is potential for developmental work which could realise a
significant return;
(3)
consider whether there is scope for working with the other Barometer
participants (for example, King’s College London) to address specific issues;
(4)
review how best to provide faculties with disaggregated Barometer and other
survey data, and advice on how to make best use of it;
(5)
monitor the impact of developmental work undertaken on future survey
performance;
(6)
consider whether the Barometer represents a suitable single tool to assess
those Corporate Support Services that interface with students.
Membership
Chair
Head of the Department of Greek and
Professor Chris Carey
Latin and Chair of the Quality Management
and Enhancement Committee
Members
Senior Tutor
Media and Communications Officer,
UCL Union
Medical School Sub-Dean
(Quality Assurance)
Dean of Students
Director of Administration, Estates
and Facilities
Director of Educational Liaison
Assistant Registrar, Registry
Information and Services, Registry
Student Representation Co-ordinator,
UCL Union
Mr Bob Allan
Mr Nick Barnard
Dr Anita Berlin
Professor John Foreman
Mr Raymond Harding
Mr Nigel Percival
Mr Andy Saffery
Mr Simon To
Secretary
Academic Services
Mr Rob Traynor
11