UK e-Science Technical Report Series ISSN 1751-5971 Review of Network Provision for Research Needs Undertaken on behalf of the JISC John Duke and Andy Jordan 03-2006 Abstract: This review examines the network infrastructure provision and use patterns in leading research departments in the UK. The information was gathered by visits, telephone interviews, and web surveys conducted with 5*-rated research departments. The associated ICT service departments were also surveyed. For a number of services, there are significant differences between groups who are aware of the Grid and groups who are not, with Grid-awareness being largely associated with science and engineering departments. Grid-aware respondents are generally happier with the speed and reliability of their network connections than non-Grid-aware respondents. They are also more likely to find their research work impaired by departmental firewalls. Grid-aware departments make greater use of departmental servers for specialist software, data storage and backup, while non-Grid-aware departments make much more use of central servers. The ICT service departments’ responses are generally similar to those of the non-Grid aware research groups. Some issues of concern arise from the survey. There is a pressing need to replace legacy networking in about 30% of institutions. Many users rely on personal media (DVDs, CDs and other external media) for data storage and backup, and this fact is not fully appreciated by ICT departments. Lack of information about Grid computing and the apparently closed nature of the Grid community are barriers to wider use of the Grid. The review makes the following recommendations: 1. 2. 3. 4. 5. The third of institutions with researchers served by coaxial networks and the rather larger group with researchers served by shared network access should ensure that they possess and implement plans to replace this with current technology. Institutions should ensure that systematic approaches are used to back up research data. JISC and UKERNA should produce clear and timely guidance for institutions on adoption of new network applications. National and institutional policies and planning need to reflect the potential impact upon working practices presented by this technology-based change. Education is required to allow users to make continuing assessment of Grid technology and a plan for its adoption is required. UK e-Science Technical Report Series Report UKeS-2006-06 Available from http://www.nesc.ac.uk/technical_papers/UKeS-2006-06.pdf Copyright © 2006 The University of Edinburgh. All rights reserved. Review of Network Provision for Research Needs Final Report May 2006 Undertaken on behalf of the JISC Jon Duke & Andy Jordan 1 Executive summary 1.1 Scope of the review 5 The review examined the network provision to the 284 research departments rated as 5* in the 2001 Research Assessment Exercise (RAE). This grouping was chosen as representative of disciplines and institutions across UK Higher Education conducting leading edge research. The study focused on the network infrastructure between the computers used by research workers and their institution’s SuperJANET5 Point of Presence as well as the provision of networkdependent services over that infrastructure. 6 1.2 7 Process used An initial group of ten research departments was visited to elicit key issues. Locally nominated contacts in a second group of about 60 departments were then telephoned to provide more detailed understandings of the issues, following which all departments were sent a web survey. The ICT service departments in the same institutions as the initial group of research departments were interviewed. The ICT service departments in all 60 institutions involved were sent a web survey for factual information on the campus network and for their perspectives on service provision. 1.3 1.3.1 1 2 3 1.3.2 1 2 Principal findings The sample of departments was found to be homogeneous in its use of some services, but for a number, it could be differentiated into two groups: those who were aware of the Grid and those who were not. These sub-populations also aligned fairly cleanly with use of different groups of research-funding sources. 3 4 Network Connections Most departmental networks provide switched links to users at 100 Mbps. About 80% of institutions have some of their research staff connected at 10 Mbps. Coaxial cabling is present in about 30% of institutions and is often a consequence of being bound in with protracted building refurbishment cycles. It is thus a cause of inadequate network provision to research workers. 5 6 Research departments are well satisfied with the speed and reliability of their network connections to departmental and institutional resources. They are also satisfied with Duke & Jordan Network services Knowledge of the Grid is largely associated with science and engineering departments. The single most common form of data backup, used by nearly 80% of users/departments, is to users’ personal media (DVDs, CDs, or portable tape drive). The most common reasons stated are: a. Inadequate quotas offered on institutional resources. b. Cost of central storage and backup services, when these are available. Recommendation 2 Institutions should ensure that systematic approaches are used to backup research data. Recommendation 1 The third of institutions with researchers served by coaxial networks and the rather larger group with researchers served by shared network access should ensure that they possess and implement plans to replace this with current technology. 4 connections into and beyond JANET, although hardly surprisingly the levels of satisfaction diminished with distance. Grid aware respondents are generally happier with the speed and reliability of network connections within their departments and institutions than are non-Grid aware respondents. This could be because a. Grid aware respondents spend more on their networks; b. Grid aware respondents have more local support for their use of the network; c. Grid aware respondents are more tolerant of weaknesses in the technology. It is the norm for institutions to possess a firewall. Firewalls, mainly through their policy aspects and their impacts upon ad hoc laptop use, impair research work in about a third of departments, most of which are Grid aware. 7 Page 2 In general, ICT service departments overestimated the use of central facilities and underestimated use of personal media. The most common place to store data is on the user’s desktop, and personal external media are widely used for storage as well as backup. The Grid aware departments make substantially greater use of departmental servers for access to specialist software as well as data storage and backup than the nonGrid aware departments, who make rather more use of central servers than the first group. For the overwhelming majority of research departments, video-conferencing in all its forms is only used to a slight extent, even amongst the Grid aware. Suite based video conferencing is the most common choice, used by about 40% of departments and is rated as generally satisfactory. Other forms of video communications (e.g. desktop videoconferencing) are little used, even though there is interest in them. Skype is used by about half of the departments contacted. The lack of central Network Provision for Research Needs, May 2006 guidance about its adoption has been interpreted in different ways by institutions, but presently the trend is towards acceptance. Recommendation 3 JISC and UKERNA should produce clear and timely guidance for institutions on adoption of new network applications. 8 Broadband ADSL and VPNs are permitting large amounts of research work to be performed from home. As well as having implications for research productivity, this will facilitate more flexible patterns of work and employment, such as use of short-term or part-time contracts. Recommendation 4 National and institutional policies and planning need to reflect the potential impact upon working practices presented by this technology-based change. 9 Amongst the science and engineering departments there is widespread use of locally clustered computing, which is commonly felt likely to provide capacity to meet needs for at least the next two years. 10 Amongst the science and engineering departments about a third of respondents felt that they would be making appreciable use of the Grid by two to four years time. 11 A substantial number of potential Grid users have consciously decided to hold back from involvement because of the learning curve involved and the availability of their own clusters. 12 Science and engineering departments with their own HPC resources can be expected to continue to continue to develop them, and their doing so will inevitably tend to offset the attractiveness of moving to the Grid. Thus the appeal of the Grid will need to grow faster than that of local resources if it is to remain as an attractive option for migration in the short or medium term. Recommendation 5 Education is required to allow users to make continuing assessment of Grid technology and plan for its adoption is required. 13 Of those replying to the question asking how access to ubiquitous unlimited network bandwidth would change their working patterns, nearly half envision improved working from home: a quarter seek more ubiquitous wireless networking and another quarter improved video conferencing services. 14 Generally the ICT service departments’ responses were similar to those of the nonGrid aware research departments but differed from those of the Grid aware departments. This could be because the Grid aware departments have more internal support and therefore do not need the central service as much as the non-Grid aware departments. Duke & Jordan Page 3 Network Provision for Research Needs, May 2006 Table of contents 1 2 3 4 5 Executive summary ..........................................................................................................2 1.1 Scope of the review................................................................................................2 1.2 Process used .........................................................................................................2 1.3 Principal findings ....................................................................................................2 1.3.1 Network Connections...................................................................................2 1.3.2 Network services..........................................................................................2 Basis of the Review..........................................................................................................7 2.1 Requirements.........................................................................................................7 2.2 Scope .....................................................................................................................7 2.2.1 The group of departments used...................................................................7 2.2.2 Focus of the study .......................................................................................7 2.2.3 Terminology .................................................................................................7 2.3 Approach ................................................................................................................8 2.4 Principles................................................................................................................8 2.5 Review process......................................................................................................8 Collection of information...................................................................................................9 3.1 Identification of departments ..................................................................................9 3.2 Selection of departments .......................................................................................9 3.3 Identifying information providers ..........................................................................10 3.3.1 The 5* departments ...................................................................................10 3.3.2 The ICT service departments ....................................................................10 3.4 Visits.....................................................................................................................10 3.5 Phone interviews..................................................................................................11 3.5.1 The 5* departments ...................................................................................11 3.5.2 The ICT service departments ....................................................................11 3.6 Web surveys ........................................................................................................11 Findings ..........................................................................................................................12 4.1 The respondents to the web surveys ...................................................................12 4.1.1 The respondents to the survey of 5* research departments .....................12 4.1.2 The respondents to the survey of ICT services departments....................13 4.1.3 Caveat........................................................................................................13 4.2 Network Infrastructure..........................................................................................13 4.2.1 Departmental level .....................................................................................13 4.2.2 University level...........................................................................................15 4.2.3 Extra-University level .................................................................................16 4.2.4 Legacy networks ........................................................................................17 4.2.5 Firewalls.....................................................................................................17 4.3 Research activities using the network..................................................................18 4.3.1 Access to data ...........................................................................................18 4.3.2 Storing data................................................................................................19 4.3.3 Data backup...............................................................................................20 4.3.4 Access to software.....................................................................................21 4.3.5 Access to compute power..........................................................................22 4.3.6 Collaboration..............................................................................................24 4.3.7 ‘Blue Sky’ thinking......................................................................................27 4.4 User perceptions ..................................................................................................28 4.4.1 Cultural issues ...........................................................................................28 4.4.2 Departmental resources ............................................................................28 4.4.3 Institutional resources ................................................................................29 4.4.4 JANET........................................................................................................34 4.4.5 The Internet................................................................................................34 4.4.6 Home .........................................................................................................35 4.4.7 Overall view ...............................................................................................35 4.5 Management ........................................................................................................36 4.5.1 Network Development ...............................................................................36 4.5.2 Network management................................................................................37 4.5.3 Network funding .........................................................................................38 Discussion ......................................................................................................................39 Duke & Jordan Page 4 Network Provision for Research Needs, May 2006 5.1 Network Infrastructure..........................................................................................40 5.1.1 Departmental infrastructure .......................................................................40 5.1.2 University level...........................................................................................40 5.1.3 Extra-University level .................................................................................40 5.1.4 Legacy networks ........................................................................................40 5.1.5 Firewalls.....................................................................................................41 5.2 Research activities using the network..................................................................41 5.2.1 Data storage and access ...........................................................................41 5.2.2 Data backup...............................................................................................42 5.2.3 Access to software.....................................................................................42 5.2.4 Access to compute power..........................................................................42 5.2.5 Collaboration..............................................................................................43 5.2.6 ‘Blue Sky’ thinking......................................................................................44 5.2.7 Home working ............................................................................................44 5.3 Management and funding ....................................................................................45 5.3.1 Differences between institutions ................................................................45 5.3.2 Differences of perception within institutions ..............................................45 6 Conclusions and recommendations ...............................................................................46 6.1.1 Network Connections.................................................................................46 6.1.2 Network services........................................................................................46 Appendix I 5* departments and universities ................................................................48 Appendix II List of topics for discussion sent out before visits to departments ............54 Appendix III List of topics for discussion sent out before phone calls to academic departments 55 Appendix IV List of topics for discussion sent out before phone calls to ICT service departments 56 Appendix V Web survey sent to academic departments ..............................................57 Appendix VI Web survey sent to service departments ..................................................76 Table of Figures Figure 1. Figure 2. Figure 3. Figure 4. Figure 5. Figure 6. Figure 7. Figure 8. Figure 9. Figure 10. Figure 11. Figure 12. Figure 13. Figure 14. Figure 15. Figure 16. Figure 17. Figure 18. Figure 19. Figure 20. Figure 21. Figure 22. Figure 23. Duke & Jordan The funding bodies.............................................................................................. 13 Proportion of institutions having some provision to researchers of type shown . 14 Connection speed to the desktop ....................................................................... 14 ICT department view of speeds and reliabilities of departmental network connections ......................................................................................................... 15 Connection speed from department network to institutional backbone .............. 15 ICT department view of speeds and reliabilities of network connections into central resources................................................................................................. 16 ICT department view of speeds and reliabilities of network connections to machines on JANET and the Internet ................................................................. 17 ICT department view of speeds and reliabilities of network connections between the Internet and the institutional network by VPN ............................................... 17 Problems with firewalls........................................................................................ 18 Services provided by ICT departments............................................................... 18 Use of the network to download data by end-users............................................ 19 Use of the network to access data storage by end-users................................... 19 Where users store data....................................................................................... 19 Where users store data - the ICT department view ............................................ 20 Where users backup data ................................................................................... 20 Where users backup data - the ICT department view ........................................ 21 Use of the network to access specialist software by end-users ......................... 22 Use of the network to access compute power by end-users .............................. 22 Clustered computing - likelihood of outgrowing local provision .......................... 23 Grid computing - current and likely future level of use........................................ 24 Collaborative tools - Email - Extent of use .......................................................... 25 Collaborative tools - Video-conferencing and IP telephony - Level of use ......... 26 Collaborative tools - Other means - Level of use................................................ 27 Page 5 Network Provision for Research Needs, May 2006 Figure 24. User perceptions of speeds and reliabilities of network connections within the department .......................................................................................................... 29 User perceptions of speeds and reliabilities of network connections within the institution ............................................................................................................. 29 Collaborative tools - Email - Satisfaction with service ........................................ 30 Collaborative tools - Video-conferencing and IP telephony - Satisfaction with service ................................................................................................................. 31 Collaborative tools - Other means - Satisfaction with service............................. 32 Collaborative tools - Email - Extent to which user needs are considered met.... 32 Collaborative tools - Video-conferencing and IP telephony - Extent to which user needs are considered met .................................................................................. 33 Collaborative tools - Other means - Extent to which user needs are considered met ...................................................................................................................... 34 User perceptions of speeds and reliabilities of connections from the desktop to machines elsewhere in the UK ........................................................................... 34 User perceptions of speeds and reliabilities of connections from the desktop to machines outside the UK .................................................................................... 35 Overall user perceptions of their network connections ....................................... 36 Balance of responsibility for planning the network.............................................. 36 Balance of responsibility for managing the network ........................................... 37 Balance of responsibility for supporting network users....................................... 38 Balance of responsibility for funding the network ............................................... 39 Figure 25. Figure 26. Figure 27. Figure 28. Figure 29. Figure 30. Figure 31. Figure 32. Figure 33. Figure 34. Figure 35. Figure 36. Figure 37. Figure 38. We gratefully acknowledge the guidance and assistance given to us throughout this piece of work by its sponsors, Brian Gilmore (Director, Edinburgh University Computing Services), Dr Bryan Lawrence (Head of the British Atmospheric Data Centre, NERC) and Prof Peter Clarke (Deputy Director, National e-Science Centre). Additionally, we thank the many people we have of necessity contacted in the course of the work for the timely and diligent assistance they so generously gave: • The researchers we contacted for information. • The staff in the ICT service departments we contacted for information. • The many people we contacted en route to obtaining names for people in the above groups. These included Pro Vice Chancellors, Heads of Department, Heads of ICT services, administrators and support staff. Jon Duke and Andy Jordan www.dukeandjordan.co.uk Duke & Jordan Page 6 Network Provision for Research Needs, May 2006 2 2.1 Basis of the Review Requirements This review has been carried out for the JISC Committee for the Support of Research (JCSR) for consideration at its May 2006 meeting. Its purpose is to investigate how well the network provision to internationally rated UK research departments meets the needs of their research workers. The study has been conducted in the context of the imminent commissioning of SuperJanet5 and has entailed considering the quality and performance of the network infrastructure between the SuperJANET5 Points of Presence and the departments. 2.2 2.2.1 Scope The group of departments used The review was planned to examine the network provision to all the internationally rated research departments in UK higher education institutions, specifically those rated as 5* in the 2001 1 Research Assessment Exercise (RAE) . There are 284 of these departments and 60 Universities are involved. This grouping was chosen as it yielded a sample of manageable size that spanned the institutions and disciplines conducting leading edge research in UK Higher Education. By way of comparison, had the nationally rated research departments with a 5 rating been included, the number would have risen to 1,000, drawn from 97 Universities. 2.2.2 Focus of the study The principal focus of the study has been the network infrastructure between the computers used by research workers and their institution’s SuperJANET5 Point of Presence as well as the provision of network-dependent services over that infrastructure. The review collected information from nominated individuals in the 5* departments to cover the areas listed below, in order to form a picture of present usage of network based resources, users’ degree of satisfaction, any constraints they experience, and their needs in the short and medium term: 1. 2. 3. 4. 5. 6. 7. The network technologies used by institutions to provide their research workers with network access. The different IT resources that research workers access and the purposes that these are used for. Any use of specialised network-based resources, including computationally intensive or Grid resources. How research workers use the network to collaborate and communicate with each other. Research workers’ views on the quality of the service provision and the network connections involved How the networking they use within their institution is funded and supported. What difference to research could result if unlimited network bandwidth were available to researchers. The review also contacted the central ICT services in all the 60 institutions to obtain the service provider’s viewpoint. These service departments were asked a very similar set of questions to those asked of the researchers, with extra questions about networking technologies and their deployment. 2.2.3 Terminology Research may be conducted by individuals working alone, by research groups, or even by whole departments working in unison. Because the RAE rating reported on departments, the study was formally couched in a departmental context, although in discussions with researchers this would often shift to that of a research group or of individuals. This was felt to be inescapable. However 1 These results are published on the HERO web site at http://www.hero.ac.uk/rae/Results/ Duke & Jordan Page 7 Network Provision for Research Needs, May 2006 in the presentation of results, although we refer to the unit of research activity as the department, it should be borne in mind that the research activity may arise from only certain groups or individuals within the department. 2.3 Approach It was planned to survey the research departments and ICT services using web surveys, but to ensure that the surveys asked the right questions, and to add depth to the reviewers’ understandings, two substantial prior stages of contact were made with the departments. In the first stage, an initial set of questions was discussed in visits to a group of about 10 departments. With lessons learnt from that, in the second stage of evidence gathering, about 60 departments were telephoned, again using a previously distributed script. In a parallel activity, which slightly lagged with the contact with departments, the relevant ICT service departments were contacted for their perspectives on network provision to researchers. A significant task within the project was the identification of suitable contacts. 2.4 Principles Our principles are to make our deliverables evidence-based and to run an open process as far as is possible. Evidence is handled confidentiality and reporting is performed unattributably: all contributors were assured that their evidence would only be used in an aggregated or anonymised form, unless prior agreement had been obtained to use it in an identifiable manner 2.5 Review process The project plan comprised seven work packages as shown in the table below. Some detail changed during the course of the project, the main change being that, because of the difficulty of obtaining names of people to speak with, rather over 60 contacts in the 5* departments were interviewed by phone, as compared with the originally intended number of about 100. Duke & Jordan Page 8 Network Provision for Research Needs, May 2006 Work Package 1 Project set-up 2 Collection of basic contact data 3 Visit a first group of departments 4 Telephone a second group of departments 5 Contact ICT services Description To confirm in detail the scope and activities of the project with its sponsors and key stakeholders and make initial preparations to run the project. Gather the contact data for the 5* departments, i.e. people, e-mail addresses and phone numbers. Visit an initial group of about ten 5* departments, using an initial interview script Undertake telephone interviews with a larger group of about 60 departments, using a script developed from the previous work package, and summarise the results of this activity Undertake telephone interviews with the ICT services of the institutions involved in work package 3 in order to obtain a description of their institutions’ network infrastructure along with their perspectives of their network based services. 6 Conduct web surveys Conduct two web surveys: 1. Using a script developed from work package 4, send a web survey to all the 5* departments. 2. Using a script developed from work package 5, send a web survey to all the ICT service departments. The results of the surveys were then analysed alongside each other. 7 Reporting the project Report on progress about halfway through the project. Deliver a final report on the work for JCSR. Make presentations of the results as required by JCSR. 3 3.1 Collection of information Identification of departments The departments were identified from the HEFCE output from the 2001 RAE exercise, which was available as an Excel spreadsheet. A small number of institutions had more than one 5* department in a particular RAE area. 3.2 Selection of departments The initial group of 10 departments to be visited was chosen so as to span the academic areas (arts and languages, engineering, humanities, medicine, science and the social sciences), the Funding Councils and the UK. From the remaining 274 departments, about 50 were then selected for interview by telephone. The selection was done using a randomising algorithm, which aimed at ensuring that institutions with 5 or more 5* departments would be called at least once and that key RAE areas would be covered by at least one call. This methodology was modified with time. The most noteworthy change was a decision to contact by telephone all the 5* departments in the institutions that had been visited. This yielded a sample of institutions where a picture of service provision could be formed from the 5* departments’ views, which could then be compared with the views of the ICT service department. However as many as possible of the 50 randomly selected departments were interviewed by phone. The web survey was sent to all those 5* departments for which contact information had been obtained: this included both departments visited and phoned. It amounted to about 250 departments. Duke & Jordan Page 9 Network Provision for Research Needs, May 2006 After the majority of the phone interviews had been carried out, a representative of the ICT service department was interviewed in each of the 10 institutions which had been selected for contact in work package 3, using either phone calls or visits. A web survey was sent to the ICT service department of each institution for which we had appropriate contact information, including those interviewed. 3.3 3.3.1 Identifying information providers The 5* departments Preliminary examinations showed that there was no straightforward way of obtaining the contact details of a person in each 5* department who would be in a good position to provide information on how well the networking provision to research workers meets their needs. As it was felt that the Heads of these departments would be in a good position to nominate such a person, it was decided to devise an approach that involved them. However, attempts to use University web sites to directly obtain contact information for Heads of department were not fruitful, so it was decided to initially approach the Universities through their Pro Vice Chancellor for Research or equivalent. Advantages of this route turned out to be: 1. It reliably yielded the correct name for the department. 2. The Pro Vice Chancellors saw that it was in their University’s interest to contribute. 2 Accordingly these peoples’ contact details were collected. The Aimhigher site was helpful in a small number of cases, but in the main this information had to be collected by telephone. Email contact with the Pro Vice Chancellors yielded contact details for the majority of Heads of Department, who were then emailed seeking the name of a contact who could speak for researchers regarding networking. For a while, there were concerns that the Pro Vice Chancellors might not be replying, so a secondary route, asking Directors of ICT services in each institution for the departmental networking contact’s names, was also used. In both routes, email reminders were found to be necessary – about half the names were obtained followed the sending of reminders. In a number of cases the telephone had to be used to bring the emails to peoples’ attention. Considerable persistence was needed in this stage of the work, which required careful record keeping to track the replies received from about 1,700 emails that were sent to about 650 people. In the event, the Pro Vice Chancellor route yielded far more names than the ICT Director one, and by the end of the task it was felt that they had just needed to be given a little time to operate their part of the process. Additionally, the telephone interview stage of the project showed that the people nominated by Heads of department in the first mentioned route were more consistently research active staff who could give authoritative views. Only one University declined to provide names for all their 5* departments. The reason given was that their research staff were too busy. Overall, contact names for 260 of the 5* departments were obtained by these means. 3.3.2 The ICT service departments Using email contact information kindly provided by UCISA and RUGIT, the Directors of ICT services at all the 60 institutions were contacted and the names of networking contacts in 58 of these department was obtained. 3.4 Visits The visits were the first stage of contact with the 5* departments and their purpose was, by making contact with a sample of different types of department, to encounter the main issues in the review area. Ten of the 5* departments were selected for this work. In the event it proved possible to arrange meetings with nine. Each visit used a script which had been sent to the networking contact beforehand. A copy of the script is shown in Appendix II. As this stage completed, the script was revised for use in the next stage of the project. 2 http://www.aimhigher.ac.uk Duke & Jordan Page 10 Network Provision for Research Needs, May 2006 3.5 3.5.1 Phone interviews The 5* departments The network contacts nominated in the 5* departments were contacted by email and telephone appointments booked. The script used is shown in Appendix III, and copies were sent to all contacts beforehand. Notes from the interviews were recorded in an online form, and a software tool was used to facilitate analysis of the information from all the interviews. 3.5.2 The ICT service departments The network contacts nominated in the ICT service departments were contacted by email and telephone appointments booked. The script used was chosen to have a number of questions in common with the one used for the 5* departments, and is shown in Appendix IV. Copies were sent to all contacts beforehand. Again, notes from the interviews were recorded in an online form to facilitate analysis 3.6 Web surveys Based upon what had been learnt in the preceding two phases, a web survey tool was used to create a survey form for the 5* departments and the ICT services departments. The survey was set up so that only one response could be received from each department. The survey to the research departments had the following organisation: 1. 2. 3. 4. 5. 6. 7. An introduction to the survey About the respondent a. Role in the department About the research department a. Sources of research income b. Its use of IT resources The department' s use of network services. a. Its use of, and satisfaction with, network services i. Transport of data in use over the network ii. Data backup over the network b. How the network is used to facilitate research collaboration c. Any firewall issues Network funding and support a. Balance of responsibility between the department and the centre Clustered computing and the Grid. a. Use and anticipated use Overall a. A ‘Blue Sky’ question (what use could be made if infinite network bandwidth were available?) b. General comments The survey to the ICT service departments had a similar structure, as shown: 1. 2. Duke & Jordan An introduction to the survey About the respondent a. Role in the department Page 11 Network Provision for Research Needs, May 2006 3. 4. 5. 6. 7. 8. About the service department a. Principal network service areas Network provision a. Deployment of particular networking technologies b. Perceptions of satisfaction of research users with network services c. Any firewall issues d. Any constraints to obtaining the benefits of SuperJANET5 Network Use by Researchers i. Transport of data in use over the network ii. Data backup over the network Clustered Computing and the Grid. a. Use and anticipated use Network Funding and Support a. Balance of responsibility between departments and the centre Overall The surveys are reproduced in full in Appendix V and VI respectively. 4 4.1 4.1.1 Findings The respondents to the web surveys The respondents to the survey of 5* research departments 130 responses were received to the web survey sent to research departments. The responses came from 42 institutions and covered 58 RAE areas. About two thirds of the respondents were research active staff, a minority of whom were heads of department. About a quarter were in administrative or support roles. The remainder felt that neither of the above roles characterised them and about a half in this category had roles with responsibilities for IT. One of the questions asked of respondents was whether or not they were aware of cluster computing or of the Grid. 61 replied that they were so aware and this was used as a measure – however rough – of the ICT involvement of the respondents’ departments. The remaining 69 respondents either replied No to the question or did not reply to it. These two almost equally sized groups were used throughout the analysis of the data. Respondents were asked for the principal funders of their research. The results are shown below. Duke & Jordan Page 12 Network Provision for Research Needs, May 2006 Figure 1. The funding bodies % of responding of all responding depts funded 0 10 20 30 40 50 AHRC BBSRC British Academy British Council EPSRC ESRC EU Government department Non-grid aw are users Industry Grid aw are users Leverhulme Trust MHRA MRC NERC PPARC Royal Society Wellcome Trust Other It is clear from this that Grid aware users are predominantly those funded by the science and engineering oriented research councils and that the non-Grid aware users are those funded by arts and humanities oriented councils. 4.1.2 The respondents to the survey of ICT services departments There were 39 responses to the survey of ICT departments. Nearly two thirds of these respondents were managers responsible for networking or technology or infrastructure. Almost all of the remainder were either Heads or Deputy/Assistant Heads of Service. 4.1.3 Caveat Not all responses included an answer to every question! For the purposes of analysis, the assumption has been made that a lack of response to a multi-choice question indicates a Don’t Know or Not Applicable response: this assumption has been made despite the fact that most questions permitted such an answer as one of the choices. In turn, the analysis usually ignores the Don’t Know answers: this equates to assuming that the responses from those giving a definite answer can be extrapolated to those answering Don’t Know. Where different assumptions have been made in the analysis, they are stated. 4.2 Network Infrastructure The survey of researchers did not ask directly about the detail of the network infrastructure. This was quite deliberate as it was clear from the telephone interviews that only a limited proportion of respondents had such knowledge. We therefore sought to obtain this information from the ICT respondents. 4.2.1 Departmental level The survey of ICT departments sought information on the nature of the network within the institution. The following graph shows how many institutions have any use of either coaxial cable or shared network connections to serve researchers. Duke & Jordan Page 13 Network Provision for Research Needs, May 2006 Figure 2. Proportion of institutions having some provision to researchers of type shown Percentage of institutions 0 20 40 60 Coaxial netw ork cable Shared netw ork connection to the desktop This shows that about 30% of institutions responding have some research active staff being served by coaxial cable. The graph also shows that between 40% and 50% of institutions serve some of their researchers with shared network connections to the desktop. The next graph shows the ICT respondents’ figures for the speeds of connections to the desktop. The graph shows each of the responses in cumulative fashion. Each line therefore represents a single institution: No institution has any connections at 0 Mbps and all have all their connections at 1 Gbps or less. Superimposed lines have been artificially separated by small amounts so that the graph can illustrate the behaviour of the whole sample. Figure 3. Connection speed to the desktop %age of researchers in institution having this speed of connection 100 0 0 Mbps 10 Mbps 100 Mbps 1000 Mbps Speed This graph shows that most institutions have the majority of desktop connections running at 100 Mbps. There are however a significant number of institutions running to some desktops at 10 Mbps together with a substantial number running to a small number of desktops at 1 Gbps. The ICT respondents were also asked about the quality, in terms of speed and reliability, of the connections from desktops to departmental servers. Respondents clearly see the reliability as generally excellent but are less sanguine about the speed of connection. Duke & Jordan Page 14 Network Provision for Research Needs, May 2006 Figure 4. ICT department view of speeds and reliabilities of departmental network connections % of institutions holding this view 0 20 40 60 80 100 From desktop machine to departmental server (all activities) - Speed Excellent Adequate Inadequate From desktop machine to departmental server (all activities) - Reliability 4.2.2 Very poor University level The ICT respondents were also asked questions similar to the above about services at an institutional level. The following graph shows what speed of link between the department and the institutional backbone serves researchers. Each response is a line showing the cumulative set of connections from zero to greater than 1 Gbps. %age of researchers in institution having this speed of connection Figure 5. Connection speed from department network to institutional backbone 100 0 0 Mbps 10 Mbps 100 Mbps 1 Gbps >1 Gbps Speed It shows that the two principal models are • Most connections are at 100 Mbps with some at 1 Gbps • Some connections are at 100 Mbps with most at 1 Gbps. There are some outliers at 10 Mbps and at greater than 1 Gbps. The following graph shows the ICT respondents view of the speeds and reliabilities for a range of links to centrally managed systems. The links cover • Desktop machine to centrally managed server including high performance computing for all activities; • Data backup from departmentally managed server to centrally managed server; Duke & Jordan Page 15 Network Provision for Research Needs, May 2006 • Wireless connected machine to institutional network. It seems that users perceive connections as not being so good for data backup as for general activities. For wireless connections, the view is that reliability is generally adequate but that the speed of such links is inadequate. Figure 6. ICT department view of speeds and reliabilities of network connections into central resources % of institutions holding this view 0 20 40 60 80 100 From desktop machine to centrally managed server including high performance computing facilities (all activities) - Speed From desktop machine to centrally managed server including high performance computing facilities (all activities) - Reliability From departmental server to centrally managed server (data backup only) Speed Excellent Adequate From departmental server to centrally managed server (data backup only) Reliability Inadequate Very poor From w ireless connected machine into institutional netw ork - Speed From w ireless connected machine into institutional netw ork - Reliability We also asked ICT respondents about bottlenecks between the desktop and the regional network: this question was designed to elicit the slowest part of the network between the user’s desktop machine and the Internet. This showed up no information that was not present in the other answers. 4.2.3 Extra-University level ICT departments were also asked about the speed and reliability of connections from desktop machines to machines on JANET and elsewhere on the Internet. Respondents clearly consider that connections to systems elsewhere in the JANET domain are better than to a different domain on the Internet. Duke & Jordan Page 16 Network Provision for Research Needs, May 2006 Figure 7. ICT department view of speeds and reliabilities of network connections to machines on JANET and the Internet % of institutions holding this view 0 20 40 60 80 100 From desktop machine to server elsew here in the JANET domain (all activities) - Speed From desktop machine to server elsew here in the JANET domain (all activities) - Reliability Excellent From desktop machine to server outside the JANET domain (all activities) - Speed Adequate Inadequate Very poor From desktop machine to server outside the JANET domain (all activities) Reliability A significant number of institutions offer VPN or SSH access for staff to their network. The following graph shows what the ICT respondents believe to be the quality of those connections. Figure 8. ICT department view of speeds and reliabilities of network connections between the Internet and the institutional network by VPN % of institutions holding this view 0 20 40 60 80 From machine on Internet via VPN into institutional netw ork - Speed Excellent Adequate From machine on Internet via VPN into institutional netw ork - Reliability Inadequate Very poor ICT respondents were asked whether either the institutional or the regional network imposed any constraints on researchers. 20% suggested that the institutional network imposes constraints and 15% that the regional network does. One respondent wrote “If there are constraints, we will address them.” 4.2.4 Legacy networks Figure 2 shows that more than 25% of institutions have some research staff being served by coaxial cable. The same graph shows that over 50% of institutions have a significant number of research staff being served to the desktop at 10 Mbps. Whether this matters may be assessed in the light of the following comment from one ICT respondent: “We are able to offer any researcher any bandwidth they require. In general the majority are happy with 10 or 100 Mbps. We get an occasional request for 1 Gbps which we have no problem with delivering anywhere on campus.” However another wrote that a “programme … to replace co-ax cabling is awaiting approval.” 4.2.5 Firewalls ICT respondents were asked about whether or not the institution provided a firewall. Only one respondent replied in the negative. This respondent wrote “We use a range of [access control lists] on the edge router to block specific ports.” Duke & Jordan Page 17 Network Provision for Research Needs, May 2006 While users were not in general asked about the details of the network, they were asked about whether network firewalls caused them problems. 31% of research respondents suggested that they did have such problems. The following graph shows the reasons for these problems: it also demonstrates that those responding to this question were predominantly Grid aware respondents. Figure 9. Problems with firewalls % of respondents answering question citing this as a problem 0 20 40 60 80 Difficulties w ith firew alls elsew here Restrictions imposed by firew all policies Video conferencing performance affected by firew all Non-grid aw are users Grid aw are users Services available to visitors'laptops constrained by firew all Other (please specify) About two-thirds of those saying firewalls cause problems blamed policies. 4.3 Research activities using the network In the ICT web survey, respondents were asked about the provision of specific services. The following graph shows the proportions of responding departments providing various services. It showed that data archiving is a minority activity. Figure 10. Services provided by ICT departments % of responding depts providing service 0 20 40 60 80 100 Specialist softw are Compute pow er Data storage Data backup Data archiving VPN or SSH connection for remote access A w ireless service 4.3.1 Access to data Users download data from a number of sources. The following graph shows where they access it from. This analysis includes all respondents. Duke & Jordan Page 18 Network Provision for Research Needs, May 2006 Figure 11. Use of the network to download data by end-users % of responding depts in sub-group making this use of network 0 20 40 60 80 Deptl srvr - Dow nloading data Centrally managed m/c - Dow nloading data Non-grid aw are users M/c elsew here in the UK - Dow nloading data Grid aw are users M/c elsew here outside the UK Dow nloading data 4.3.2 Storing data Storage of data by users can be in a number of different places. Departmental and institutional servers are equally important – and far ahead of the use of machines off campus. This analysis includes all respondents. Figure 12. Use of the network to access data storage by end-users % of responding depts in sub-group making this use of network 0 20 40 60 80 100 Deptl srvr - Access to data storage capacity Centrally managed m/c - Access to data storage capacity Non-grid aw are users M/c elsew here in the UK - Access to data storage capacity Grid aw are users M/c elsew here outside the UK - Access to data storage capacity In a separate question, researchers were asked where they store data. The answers to this match those in the previous graph well – in both graphs the use of departmental and central servers is about 60% - but also demonstrate that CDs, DVDs and the like are used extensively. Figure 13. Where users store data % of respondents in sub-group utilising this means 0 20 40 60 80 100 On CDs, DVDs or other external media On desktop m/c Non-grid aw are users Grid aw are users On Deptl srvr On centrally managed srvr The same question – where do users store data – was also asked of the ICT respondents. Duke & Jordan Page 19 Network Provision for Research Needs, May 2006 Figure 14. Where users store data - the ICT department view % of institutions with proportion of users utilising this means 0 20 40 60 80 100 Stored On CDs or DVDs Stored On desktop machine A lot Significant Slightly Stored On departmental server Very little Stored On centrally managed server One of the issues which the interviews identified was the quota of storage space allocated to individual users. One research respondent wrote “department backup is quite limited (3 months) so users must take responsibility for longer storage. We have no idea what the real lifetime of this data will be.” and another said “the huge problem we face is the absence of a departmental server and that centrally-provided storage space is so small as to be useless.” This last comment was made from an institution with sophisticated central data storage arrangements. 4.3.3 Data backup Data backup can represent a substantial load on the network in the otherwise quiet hours. Researchers were asked where they backup their data. Figure 15. Where users backup data % of respondents in sub-group utilising this means 0 20 40 60 80 100 On CDs, DVDs or other external media On desktop m/c Non-grid aw are users Grid aw are users On Deptl srvr On centrally managed srvr ICT respondents were asked to estimate the same information. Duke & Jordan Page 20 Network Provision for Research Needs, May 2006 Figure 16. Where users backup data - the ICT department view % of institutions with backup of users utilising this means 0 20 40 60 80 100 Backed up On CDs or DVDs A lot Significant Backed up On departmental server Slightly Very little Backed up On centrally managed server These graphs suggest that the ICT respondents underestimate the use of CDs and the like for data backup in comparison with other means. One ICT respondent said “Data backup is provided against system crashes only. Other backup is the end-user' s responsibility.” However it is also clear that many institutions recognise the issue and are planning developments in this area. Responses from ICT departments included the following: “[We are planning an] extension this year to [a service delivering] document management, sharing, archiving, version control and offsite storage.” “Data storage facilities do not cover all Researcher needs; where departments have specific needs they have put their own facilities in place and rely on my department to provide more generic facilities.” “In 2006 we are introducing a SAN with archiving for long term storage of research data.” “The greatest demands at [my institution] are currently around data storage, retention and retrieval.” “We are about to implement a data storage service for research groups which will obviously include data backup.” “Central data storage and archiving facilities are not sufficient and plans are in place to increase this.” Another respondent noted that “there is currently a technology lag in the types of storage that can affordably be connected to HPC clusters that deliver adequate performance and scalability.” 4.3.4 Access to software The following graph shows how many responding departments access such software across the network. Duke & Jordan Page 21 Network Provision for Research Needs, May 2006 Figure 17. Use of the network to access specialist software by end-users % of responding depts in sub-group making this use of network 0 20 40 60 80 Deptl srvr - Access to specialist softw are Centrally managed m/c - Access to specialist softw are Non-grid aw are users M/c elsew here in the UK - Access to specialist softw are Grid aw are users M/c elsew here outside the UK - Access to specialist softw are It is noteworthy that Grid aware users make greater use than the non-Grid aware users of departmental servers and less use of centrally managed servers. There is little difference between the two groups in the small amount of access of off-campus software. 4.3.5 Access to compute power 4.3.5.1 General Users were asked the extent to which their departments access compute power across the network. The following graph shows the responses. Figure 18. Use of the network to access compute power by end-users % of responding depts in sub-group making this use of network 0 20 40 60 80 Deptl srvr - Access to compute pow er Centrally managed m/c - Access to compute pow er Non-grid aw are users Grid aw are users M/c elsew here in the UK - Access to compute pow er M/c elsew here outside the UK - Access to compute pow er To be noted are the use of departmental servers and external machines for computing by Grid aware users. 4.3.5.2 Cluster computing Both users and ICT respondents were asked about current use of cluster computing (“Low”, “Medium” or “High”) and what they thought the likelihood was that demand in the future would outgrow local provision. The purpose of the question about the future was to ascertain the likelihood that access to external resources – such as through use of the Grid – would be required. The next graph shows the responses to these questions. The responses of researchers and of ICT respondents are similar in shape and show that both groups think the most likely outcome over 4 years is that local provision will be outgrown. It should be noted that the web survey allowed research department respondents to skip both this and the questions about Grid computing if they were not aware of such computing systems. Just under 40% of all research department respondents and about 70% of all the ICT respondents answered this question on clustered computing. The graph immediately below shows the proportions of these sub-groups: the Don’t Know category is included in the analysis but omitted Duke & Jordan Page 22 Network Provision for Research Needs, May 2006 from the graph. It will be seen from the graph that the Don’t Know category increases with distance ahead. There is considerable similarity in the profiles of the two groups. Figure 19. Clustered computing - likelihood of outgrowing local provision % of those responding to this question More than 4 years Within 4 years Within 2 years Within 1 year 0 20 40 60 80 100 Research respondents ICT respondents Research respondents Low ICT respondents Medium Research respondents High ICT respondents Research respondents ICT respondents 4.3.5.3 Use of the Grid Users and ICT respondents were asked similar questions about use of the Grid to those asked about cluster computing. Both the present level of use (“Low”, “Medium” or “High”) and the likely future level of use were sought. Just over 40% of all research department respondents addressed this question. The following graph shows proportions of that 40%. The number of Don’t Knows for “Current use” is largely due to those not providing any answer to Current Use. 82% of all ICT respondents answered the similar question in their survey. The graph below shows proportions of that 82%. The Don’t Know proportion of answers is high – around 40% of all the answers – but is remarkably consistent across the time period. The graphs for both sets of respondents show similar profiles. Both anticipate a substantial increase in use of the Grid from a low starting point. Duke & Jordan Page 23 Network Provision for Research Needs, May 2006 Figure 20. Grid computing - current and likely future level of use % of those responding to this question Likely level of Likely level of Likely level of use in 4 year' s use in 2 year' s use in 1 year' s time time time Current use 0 4.3.6 20 40 60 80 100 Research respondents ICT respondents Research respondents ICT respondents Low Research respondents High Medium ICT respondents Research respondents ICT respondents Collaboration Interviews showed that a major use of the network by researchers was in support of collaboration with others. Both surveys asked about the use of collaborative tools. 4.3.6.1 Email Both surveys asked about the use of email and of email with large attachments, which interviews showed to be an important means of exchanging large amounts of data. The following graph shows the importance of email to collaboration. There is great similarity between the responses of all three groups. Duke & Jordan Page 24 Network Provision for Research Needs, May 2006 Figure 21. Collaborative tools - Email - Extent of use % of respondents in sub-group to this question 0 20 40 60 80 100 Email Non-grid aw are users Grid aw are users ICT A lot Email with large attachments Significant Non-grid aw are users Slight Grid aw are users ICT One ICT respondent noted that they put a “limit on email attachment size for operational reasons.” 4.3.6.2 Video conferencing and telephony Interviews showed that video conferencing is an important tool, especially from custom built suites. The following graph shows the responses from the two surveys to questions on the extent of use of video conferencing and Skype or other Internet telephony. Duke & Jordan Page 25 Network Provision for Research Needs, May 2006 Figure 22. Collaborative tools - Video-conferencing and IP telephony - Level of use % of respondents in sub-group to this question 40 60 80 100 Grid aw are users ICT Non-grid aw are users Grid aw are users Desktop video phone between two people ICT Non-grid aw are users Non-grid aw are users Skype or other telephony over the Internet 20 Non-grid aw are users Desktop video phone involving more than two people Video conferencing using Video equipment carried conferencing from on a trolley a custom-built suite 0 A lot Grid aw are users Significant Slight ICT Grid aw are users ICT Non-grid aw are users Grid aw are users ICT This graph shows that the ICT respondents believe more use is being made of these tools than do the research respondents and that the Grid aware users make more use of these services than do non-Grid aware users. A number of ICT respondents noted that they do not allow the use of Skype: • “Because of Skype' s parasitic behaviour, we cannot allow it to be run without restrictions.” • “We have a ban on the use of Skype at the minute, mainly because of its licensing terms and …. its supernode behaviour.” • “Limit on email attachment size for operational reasons. Network supports desktop video, Skype, etc. but they are not guaranteed services.” Another noted that “in 2006 we have launched a project to identify solutions …. to deliver voice and video over IP and secure alternatives to Skype.” An interesting approach came from a research department respondent who wrote “An alternative to real time provision is the podcast, which we are now adopting.” 4.3.6.3 Other The surveys also asked about the use of Virtual Research Environments and of the Access Grid. The following graph shows that neither of these tools is currently in wide use. Duke & Jordan Page 26 Network Provision for Research Needs, May 2006 Figure 23. Collaborative tools - Other means - Level of use % of respondents in sub-group to this question Virtual research environment (analogous to a VLE) 0 20 40 60 80 100 Non-grid aw are users Grid aw are users ICT A lot Significant Slight Access Grid Non-grid aw are users Grid aw are users ICT 4.3.7 ‘Blue Sky’ thinking Both sets of respondents were asked what effect unlimited access to network bandwidth would have on the ways in which research is undertaken. We offered four examples, drawn from interviews: • High resolution viewing of artefacts held remotely; • Working at home as if you were at work; • Using specialist facilities that are linked in real-time over the network; • Accessing the network using wireless. These four choices were confirmed as of importance by the web surveys. Respondents noted the impact on jobs: • “Allowing specialists, e.g. statisticians, to be part of several projects in several institutions simultaneously [and] avoiding contract staff having to move frequently.” • “Improved teleworking by researchers in non UK universities who hold part time research appointments here.” Viewing images is important to a range of disciplines: • “Speedy and high-resolution viewing of ancient documents, medieval manuscripts, early printed books etc. held remotely. Enhanced ability to collate different editions or manuscripts of same text on-line.” • “High resolution viewing of digital images of primary text materials stored remotely.” • “Obviate the need to bring back telescope images on multiple DAT/DLT/DVD.” • From a department of electrical engineering: “Local streaming of uncompressed HighDefinition video from remote sources in real-time.” • From a department of biological science: “Using specialist facilities that are linked in realtime over the network: we have a lot of image data that cannot be viewed over [the institutional] network.” • From a Social Science department: “Better use of video as [a] data source.” • From a music department: “It' s possible our composers would be interested in real-time internet-based interaction.” • Some frequently occurring terms in researchers’ responses that are considered to be of significance are reported below: Duke & Jordan Page 27 Network Provision for Research Needs, May 2006 Term Home access or VPN Video conferencing Wireless 4.4 4.4.1 Frequency 41 18 20 User perceptions Cultural issues The JCSR members who project managed this work are all scientists or engineers as are the two people carrying out the work. This may therefore explain the occasional problems met in explaining the nature of the work to some colleagues in the arts and humanities. One philosopher for example asked “What do you mean by a network?” A respondent from a language department wrote “I have consulted with colleagues and we have to say we are somewhat bemused by what you are describing; we are a small Arts department….and are not aware of using (or even perhaps needing) the sort of thing you describe for our work.” Others indicated that our use of the phrase “research group” was essentially alien to them. As a result of the experience of the extensive interview process undertaken before sending out the web surveys, the design of the survey for research departments reflected these cultural differences. A number of technical questions were omitted because they might deter some respondents from replying to the whole survey. Such problems could perhaps have been overcome by extended explanations of the questions: this however might also have had the alternative deterrent effect of so lengthening the survey that respondents considered it excessive. The design of the surveys was therefore a compromise between asking all the questions to which we wanted answers and keeping the length of the survey acceptable. The answers received therefore need to be seen in this light. 4.4.2 Departmental resources Users were asked how good they considered the connection to be from the desktops in their department to departmental servers. The following graph shows that both speed and reliability are held in good esteem, though the Grid aware users have a higher opinion of the reliability of these connections than do their non-Grid aware colleagues. Duke & Jordan Page 28 Network Provision for Research Needs, May 2006 Figure 24. User perceptions of speeds and reliabilities of network connections within the department % of users in sub-group holding this perception Grid aware users Non-grid aware users 0 20 40 60 80 From desktop m/c to Deptl srvr - Speed From desktop m/c to Deptl srvr - Reliability Excellent Adequate Inadequate From desktop m/c to Deptl srvr - Speed Very poor From desktop m/c to Deptl srvr - Reliability 4.4.3 Institutional resources In similar vein to the question addressed in the last section, users were asked about the quality of the connections to central machines from both desktop systems and departmental servers. The following graph shows the results, which are significantly worse than the quality of links within the department. Figure 25. User perceptions of speeds and reliabilities of network connections within the institution % of users in sub-group holding this perception Grid aware Non-grid users aware users Grid aware Non-grid users aware users 0 20 40 60 80 From desktop m/c to centrally managed srvr Speed From desktop m/c to centrally managed srvr Reliability From desktop m/c to centrally managed srvr Speed Excellent From desktop m/c to centrally managed srvr Reliability Adequate From Deptl srvr to centrally managed srvr Speed Inadequate Very poor From Deptl srvr to centrally managed srvr Reliability From Deptl srvr to centrally managed srvr Speed From Deptl srvr to centrally managed srvr Reliability The surveys asked the degree of satisfaction with the quality of the collaborative tools provided for use. The same questions were also asked of the ICT respondents. In general their answers were very similar to those of their customers. Duke & Jordan Page 29 Network Provision for Research Needs, May 2006 The next graph shows the user view of the email service. The provision of the email service for large attachments is clearly not wholly satisfactory. Figure 26. Collaborative tools - Email - Satisfaction with service % of respondents in sub-group to this question 0 20 40 60 80 100 Email Non-grid aw are users Grid aw are users ICT A lot Email with large attachments Significant Non-grid aw are users Slight Grid aw are users ICT This graph omitted Don’t Knows from the analysis: there were very few of them. The next graph shows the responses to the same questions when asked about video conferencing and telephony tools. The analysis underlying this graph did include Don’t Knows of which there were a large number. Duke & Jordan Page 30 Network Provision for Research Needs, May 2006 Figure 27. Collaborative tools - Video-conferencing and IP telephony - Satisfaction with service % of respondents in sub-group to this question 40 60 80 100 Grid aw are users ICT Non-grid aw are users Grid aw are users Desktop video phone between two people ICT Non-grid aw are users Non-grid aw are users Skype or other telephony over the Internet 20 Non-grid aw are users Desktop video phone involving more than two people Video conferencing using Video equipment carried conferencing from on a trolley a custom-built suite 0 A lot Grid aw are users Significant Slight ICT Grid aw are users ICT Non-grid aw are users Grid aw are users ICT The responses to questions about user satisfaction with VREs and the Access Grid are similarly dominated by the Don’t Knows. ICT respondents rather overestimated the degree of user satisfaction with the Access Grid. Duke & Jordan Page 31 Network Provision for Research Needs, May 2006 Figure 28. Collaborative tools - Other means - Satisfaction with service % of respondents in sub-group to this question Virtual research environment (analogous to a VLE) 0 20 40 60 80 100 Non-grid aw are users Grid aw are users ICT A lot Significant Slight Access Grid Non-grid aw are users Grid aw are users ICT Users and ICT respondents were also asked to what extent collaborative tools met their needs. The next graph shows the responses for email. The responses of all three groups are remarkably similar. Figure 29. Collaborative tools - Email - Extent to which user needs are considered met % of respondents in sub-group to this question 0 20 40 60 80 100 Email Non-grid aw are users Grid aw are users ICT A lot Email with large attachments Significant Non-grid aw are users Slight Grid aw are users ICT The next graph shows the extent to which users thought their needs are met by the video conferencing and telephony tools available to them. Here the ICT respondents generally overestimate the extent to which user needs are met, though all the groups considered needs only slightly met. Duke & Jordan Page 32 Network Provision for Research Needs, May 2006 Figure 30. Collaborative tools - Video-conferencing and IP telephony - Extent to which user needs are considered met % of respondents in sub-group to this question 40 60 80 100 Grid aw are users ICT Non-grid aw are users Grid aw are users Desktop video phone between two people ICT Non-grid aw are users Non-grid aw are users Skype or other telephony over the Internet 20 Non-grid aw are users Desktop video phone involving more than two people Video conferencing using Video equipment carried conferencing from on a trolley a custom-built suite 0 A lot Grid aw are users Significant Slight ICT Grid aw are users ICT Non-grid aw are users Grid aw are users ICT The extent to which user needs are met by the VRE and access grid tools is shown in the next graph. Duke & Jordan Page 33 Network Provision for Research Needs, May 2006 Figure 31. Collaborative tools - Other means - Extent to which user needs are considered met % of respondents in sub-group to this question Virtual research environment (analogous to a VLE) 0 20 40 60 80 100 Non-grid aw are users Grid aw are users ICT A lot Significant Slight Access Grid Non-grid aw are users Grid aw are users ICT 4.4.4 JANET The user view of the connection from their desktop machines to systems elsewhere in the JANET domain is shown next. Figure 32. User perceptions of speeds and reliabilities of connections from the desktop to machines elsewhere in the UK % of users in sub-group holding this perception Grid aware users Non-grid aware users 0 10 20 30 40 50 60 70 80 90 100 M/c elsew here in the UK - Speed Excellent M/c elsew here in the UK - Reliability Adequate Inadequate M/c elsew here in the UK - Speed Very poor M/c elsew here in the UK - Reliability There is considerable agreement here between the two sub-groups 4.4.5 The Internet As the next graph shows, research respondents only perceive their connection to systems elsewhere on the Internet as Adequate with a significant tail of respondents seeing the speed as being less than adequate. As in Figure 32, the graph showing similar data for machines in the UK, there is considerable similarity in the views of the two sub-groups. Duke & Jordan Page 34 Network Provision for Research Needs, May 2006 Figure 33. User perceptions of speeds and reliabilities of connections from the desktop to machines outside the UK % of users in sub-group holding this perception Grid aware users Non-grid aware users 0 4.4.6 10 20 30 40 50 60 70 80 90 100 M/c elsew here outside the UK - Speed Excellent M/c elsew here outside the UK - Reliability Adequate Inadequate M/c elsew here outside the UK - Speed Very poor M/c elsew here outside the UK - Reliability Home Many users expressed the view that better bandwidth could enable working from home to resemble working within the workplace. It was clear from many interviews that the advent of ADSL had enabled a dramatic change to working practices. Many institutions provide a VPN service to enable home users to access the university network directly. The following examples of comments in this area are indicative of many: • “VPN service is an essential part of working from home into college: this allows access to … subscription services.” • “[I] use VPN at home and find it very good.” • “{I] would set things up so that home computing looks like the office e.g. Citrix. Everything goes through SSH (looks like VPN): this is more secure than straight VPN but setting it up is more finicky.” • “A MAJOR element in the equation …. is working at home and elsewhere using VPN connections--we do this a great deal (and, if there was the bandwidth I would sometimes send large AV files).” • “We all like working at home as if we were at work. Given that we mostly just read journals, case reports (we mostly do that online these days), and books, and send and receive emails, we can already do that from home with the [institutional] VPN, so long as our computers are good enough and we have broadband. For a wireless UK!” • “A number of long running experiments are monitored remotely/at home already. Desktops are currently routinely logged into from home.” 4.4.7 Overall view Overall users have the following view of their network connection. The modal responses of both groups of users for speed and reliability are “Adequate”. However, the Grid aware users have a better opinion of their network connections than do non-Grid aware users. Duke & Jordan Page 35 Network Provision for Research Needs, May 2006 Figure 34. Overall user perceptions of their network connections % of users in sub-group holding perception Grid aware users Non-grid aware users 0 4.5 20 40 60 80 Excellent Adequate Inadequate Very poor Speed Reliability Excellent Adequate Inadequate Very poor Management The surveys posed a series of parallel questions about the management of the network. These questions were generally framed in terms of the respondent’s view of where the responsibility lies between the local department and the central ICT service for various aspects of network management. 4.5.1 Network Development 4.5.1.1 Planning Both researchers and ICT staff were asked where they believed the balance of responsibility lies for planning the network. The following graph presents their responses and shows that each group tends to the view that their own group has the responsibility. The modal response of the Grid aware researchers is that it is a departmental responsibility and of the non-Grid aware that it is shared: the ICT modal response is that it is a central responsibility. Figure 35. Balance of responsibility for planning the network % of sub-group respondents Planning the development of departmental networks 0 20 40 Non-grid aw are users Departmental responsibility, ie not top-sliced Shared departmental and central responsibility Grid aw are users Duke & Jordan 60 Central responsibility Don' t know ICT Page 36 Network Provision for Research Needs, May 2006 4.5.1.2 Building refurbishments Building refurbishment drives the timetable for much network change. Interviewees told us • ‘There is some legacy network left, which is generally being left for refurbishment.’ • ‘We develop the network as part of refurbishment and then work closely with Estates to decide what is done.’ • ‘Building refurbishments have cabling costs in project costs. Asbestos is a problem here. They try to dovetail as best they can.’ Web responses from researchers included “Our researchers are currently in a building which is due for complete refurbishment this year. At that point I would expect to go from 10 Mbps lines to Gigabit lines [with] a consequent rise in speed and reliability.” One ICT responses said “Network equipment is largely funded centrally, either through refurbishments or [the central ICT service].” 4.5.2 Network management 4.5.2.1 Managing network equipment We also enquired about the balance of responsibility for managing network equipment within departments. The following two graphs show the responses. There was a tendency to think that “we” do it. There was a very high propensity for the ICT respondents to consider it a central responsibility while the Grid research respondents took the opposite view. The non-Grid aware research respondents were between the two but much closer in view to the ICT respondents. Figure 36. Balance of responsibility for managing the network % of sub-group respondents Managing the network equipment installed within departments 0 20 40 Non-grid aw are users 60 Departmental responsibility, ie not top-sliced Shared departmental and central responsibility Grid aw are users Central responsibility Don' t know ICT 4.5.2.2 Supporting network users Both sets of respondents were asked where the balance of responsibility lies for providing first line support for users of the network. The two graphs which follow show a similar response to that for where the responsibility for managing network equipment lies, but rather more starkly. The ICT respondents saw their own department as being predominantly responsible whereas the Grid aware research respondents were very clearly of the view that the department has the responsibility. Duke & Jordan Page 37 Network Provision for Research Needs, May 2006 Figure 37. Balance of responsibility for supporting network users % of sub-group respondents Providing first line support to users of network services 0 20 40 Non-grid aw are users 60 Departmental responsibility, ie not top-sliced Shared departmental and central responsibility Grid aw are users Central responsibility Don' t know ICT 4.5.2.3 Security management By and large, our interviews showed that the security imposed within departments is governed by university-wide policies. The principal exceptions to this were where client or patient confidentiality was at stake. As one ICT respondent noted, “Network security can have an impact on what researchers wish to do, especially if they are working with the NHS.” Because there was reasonable clarity about this from the interviews, where security was addressed directly, we did not pursue the issue per se in the web surveys, a fact which did not escape all those researchers we surveyed: “I am surprised at the omissions from this questionnaire (….network security).” There were some comments from ICT respondents about network security including • “The design of Grid technologies is naive about security. It assumes that it needs only to be concerned about access to the Grid itself.” • “There are potential conflicts between large bandwidth e-Science type of activities and effective network security, e.g. firewalls capable of wirespeed Gbit protection are very very expensive compared to those that we use for ordinary production work.” 4.5.3 Network funding In both surveys, respondents were asked about the balance of responsibility for funding the network, both in capital and recurrent terms. In the survey, the question was couched to mean that top-sliced funding within an institution was equivalent to central funding. Here the views of the three groups were more similar than in the other questions about responsibility but here too there was a division between the non-Grid aware users and ICT respondents on the one hand and the Grid aware research respondents on the other. Duke & Jordan Page 38 Network Provision for Research Needs, May 2006 Figure 38. Balance of responsibility for funding the network % of sub-group respondents Recurrent funding of departmental networks Capital funding of departmental network upgrades and new connection points 0 20 40 60 Non-grid aw are users Grid aw are users Departmental responsibility, ie not top-sliced Shared departmental and central responsibility ICT Central responsibility Non-grid aw are users Don' t know Grid aw are users ICT How funding is managed does affect the way the network is used. One ICT respondent commented that “network traffic here is moderated by the need for departments … to contribute to the [national] network charge that we distribute proportionally by [departmental] network traffic each quarter.” 5 Discussion Section 4 shows that overall, the sample of departments could be viewed as homogeneous in its use of some services, but for a number of services, a breakdown into two groups according to whether the departments were aware of the Grid or of cluster computing showed that these subpopulations behaved in different ways. The Grid aware grouping included as expected the science and engineering departments, but also included Applied Mathematics, Statistics, and Psychology departments. These sub-populations also aligned fairly cleanly with use of different groups of research-funding sources. The Grid aware departments were also characterised by typically possessing their own support staff, their own servers with developed storage and backup regimes, and a wider variety of technical needs (e.g. coping with firewalls, and more use of laptops by visitors). The ICT service departments generally understood research users’ needs well, though when the opportunity to compare presented, their views of the network and network services provision tended to align more with that of the non-Grid departments than of the Grid aware ones. This could be because the Grid aware departments have more internal support and therefore do not need the central service as much as the non-Grid aware departments. The ICT service departments also tended to overestimate the level of use of centrally provided services in cases where there were departmental alternatives. For example, in the area of data backup it was felt Duke & Jordan Page 39 Network Provision for Research Needs, May 2006 that often the ICT departments were unaware of the sheer amount of data being backed up (or not) within the research departments. 5.1 5.1.1 Network Infrastructure Departmental infrastructure While provision to the desktop at 100Mbps is most common, there is a considerable spread of provision, and as about 80% of institutions have appreciable proportions of their research users connected at 10Mbps, there is still appreciable scope for upgrading these slow links. Arts users so connected have reported concerns: • 8 ‘Speed limitations when dealing with images are noticeable’. In general, reliability of the departmental infrastructure is not an issue as evidenced by the very low level of adverse comments from both the researchers and the ICT service departments. It is noteworthy though that the ICT service departments score networks as being appreciably more reliable than did the users. This could be because a. Grid aware respondents spend more on their networks; b. Grid aware respondents have more local support for their use of the network; c. Grid aware respondents are more tolerant of weaknesses in the technology. 5.1.2 University level At the next level, that from the departmental infrastructure to the institutional network backbone, the story is similar to the above in that there is a rump of departments connected at 100Mbps, though the norm is now 1Gbps. The ICT service departments see this class of network connection as overwhelmingly excellent, though the users are rather less impressed, with about half marking the speed and reliability of this part of the infrastructure as adequate rather than excellent. 5.1.3 Extra-University level The research departments consider that the speed and reliability of connections to elsewhere within the JANET domain are mostly adequate, sometimes excellent and only in a few cases inadequate. The ICT view accorded reasonably with this except that they rated reliability considerably higher. For links beyond JANET, the picture from the research departments regarding speed is similar, mostly adequate, though slightly less commonly excellent. For reliability, both pictures are predominantly ‘adequate’, with the non-Grid aware users rating it rather better than the Grid aware group. Here, the ICT service departments’ views accorded well with those of the researchers. 5.1.4 Legacy networks Legacy networking was found in 30% of institutions surveyed, usually at the departmental level, and comments from both researchers and ICT service departments show that its removal often depends on building refurbishment cycles. • • • • • “No legacy networking. Refurbs a few years ago did this.” “All cat5 – building has been through a BIG refurb in the last 6 years.” “Since the arrival of the cat6 cabling about a year ago, things are much better – this came in on the back of their whole building’s refurb ….. Before then it was all thin Ethernet.” “Our researchers are currently in a building which is due for complete refurbishment this year. At that point I would expect to go from 10MB lines to GB lines and a consequent rise in speed and reliability.” “In buildings there is still an appreciable installed base of thin Ethernet - this is top of the [service department’s] list but stays there - funding is and remains the problem. Estimates £2M of work left to do in 3 large and 2 small buildings. Asbestos is also a problem here.” Some of the situations encountered demonstrate that maintaining an association between network upgrades and building refurbishments in order to try and save money has caused long term Duke & Jordan Page 40 Network Provision for Research Needs, May 2006 negative impact to research workers and hence to research competitiveness. The consequences extend from poor responsiveness when accessing web pages to influencing users to operate poor data backup procedures in order to minimise network use. As the budgets needed to provide departmental networking to current norms are not large, it seems sensible that this issue should be examined by institutional senior management wherever legacy networking is present. 5.1.5 Firewalls All institutions except one had a centrally managed firewall. Some were recent and a few were still in the process of implementation. About a third of departments (mostly from the Grid aware grouping) reported that firewalls were causing them difficulties. Policy issues were the main cause of problems, followed by the constraints they imposed upon laptop use by visitors. Amongst the Other specified reasons reported by respondents were: • The inability to use multicast. • Problems in using University resources from home. • Access to the NHS partner organization, which has, by necessity, to have a firewall between its network and that of the institution. • Delay in opening a port for video conferencing. • Researchers don' t always ask about restrictions or default blocks early enough! These reasons were supported by the evidence of the interviews. The interview group indicated clearly that being able to successfully host a flow of visitors is an important capability of a modern research department, and a key part of this is an ability to rapidly make a visitor productive by allowing their laptop to access the resources the user needs. A variety of means were found to be in use for permitting this, with substantial variations generally made in order to fit in with particular institutional policies. Overall, respondents showed that firewalls can cause a wide variety of problems of varying seriousness to research workers, a number of which were listed in section 4.2.5. Although there is comprehensive guidance on network security design produced by the JISC and JANET-CERT, it seems that the variety of solutions is sufficient to make arriving at a different university with a laptop something of an adventure. 5.2 5.2.1 Research activities using the network Data storage and access The research departments report that the most common place to store data is the researcher’s desktop. The ICT service departments recognise this too but underestimate the extent. The research departments indicated that departmental and central servers were used for these purposes in approximately equal measure. UK and foreign resources were used to a lower but significant extent, and used more for downloads than for storage. The phone interviews added detail to these figures, showing that overall the picture is of a community making very widespread use of data, wherever it is located in the world. The users in these departments (significant numbers of which are in the Arts area) latch on to new sources of data very quickly, and they now make routine use of research resources that are no more than a year or two old. Of concern is that many research users store data on DVDs, CDs or other personal external media rather than on resources available over the network. In some cases it was stated to be a consequence of seriously inadequate quotas being available on the servers to which they have access. • “Central [service] doesn’t seem to understand their needs (they appear to think all users’ needs are those of the average user).” An instance was found where a modern central storage and backup service had been introduced in the last two years, but that it was felt to be too expensive, because it had been designed with corporate data needs in mind, without consulting research departments at the design stage. Had Duke & Jordan Page 41 Network Provision for Research Needs, May 2006 they been, presumably they could either influenced the model and its costs, or have learnt why the costs were as they were, either which could have led them to use the service. The departments were therefore creating their own solutions alongside the central one. Patterns of use of research data can be complex. For example, with the advent of imaging technologies, Departments of Psychology are now very large scale creators and users of data. They are acquiring large servers with storage in the Terabyte range. Data files of hundreds of megabytes are stored, exchanged with collaborators, and sent abroad for some processing. Some research work involves using imaging systems at other locations, and experiments are being planned that involve simultaneous real-time use of pairs of imaging systems. This work places great demands on the network. Furthermore, some data has NHS origins and has to be handled specially. 5.2.2 Data backup Even more researchers (nearly 80%) were found to use personal external media for backup than use it for storage. Using personal media was the most common way of performing backup. These researchers are thus taking responsibility for backup upon themselves. The results also show that many users employ a variety of means to backup their data which suggests that no single entirely satisfactory approach is available to them. • “Departmental data backup is basically out of control. Lots of CDs and memory sticks around.” As a group, the ICT service departments appear to considerably overestimate the role played by central servers to back up research data and to underestimate the level of use of DVDs and CDs for this purpose by researchers. However, a number of the ICT survey responses clearly show that central provision of backup services backup is presently being addressed. Until the overall situation improves though, a lot of research data must be considered to be at much more risk than it should be. One institution was found that did not offer a data backup service, expecting departments or users to perform that task. 5.2.3 Access to software Although access to remote software is a minority activity, it is important to those who do use it. Some researchers move large data files abroad for data reduction, and move it back for subsequent analysis. 5.2.4 Access to compute power 5.2.4.1 Cluster computing The phone interviews indicated that departments with needs for high performance computing (HPC) are most commonly using local facilities, sometimes clusters of idle PCs, using Condor software, for this purpose. The web survey showed that, with nearly 40% of departments providing information on usage, the activity is widespread. Citing reasons of ease of adoption, convenience and manageability, the groups with HPC needs were most frequently using clustered computing resources within their own department or institution. They generally felt that their resources would meet their needs for a couple of years. Some of these users had often not finished exploiting all the idle PCs that were available. Under a half felt they were likely to outgrow them in the longer term, though for this response the level who could not express a view was nearly a half. 5.2.4.2 Use of the Grid There was a modest level of awareness of the Grid amongst users of high performance computing (HPC), but only a handful used it beyond a low level. About 10% of the Grid aware respondents saw themselves as making high use of the Grid in between two and four years time, with another 20% anticipating medium levels of use. Here, about a third of respondents did not offer a view. The views from the ICT service departments anticipated a similar profile of change. The phone interviews showed that awareness of the Grid’s existence is widespread amongst HPC users, though it is common for them to have made a conscious decision not to get involved with Duke & Jordan Page 42 Network Provision for Research Needs, May 2006 the Grid so far because of both the learning curve involved (with the risk of interruption to research work) and the fact that many have fairly recently started using local compute clusters. Some research users also run HPC jobs of such duration on their own clusters (e.g. up to several months) that they doubt whether a Grid resource would be suitable. The science and engineering departments can be expected to continue to develop their own HPC resources and doing so will inevitably tend to lessen the comparative attractiveness of Grid resources. Thus the latter will need to grow at least at the same rate if they are to remain as an attractive option for migration in the short or medium term. The uncertainties found in the responses to the cluster computing and Grid survey questions are not inconsistent with such a scenario. Some HPC cluster users are not convinced that the Grid will deliver resources suitable for running their jobs, for example one of six months duration on a 20 node cluster (estimated 10Pflops). It was clear that some of the Grid users were conducting or planning work that local HPC provision could not have coped with. A characterising feature of some Grid users was that the data management requirements of their work needed deep planning. Some of the small number of Grid users recognised issues in the support technologies: this included management of both certificates and virtual organisations, which were seen as nascent network services. Impressions gained were: • The Grid community can appear rather closed to those outside it. • Because the Grid is basically a concept, it remains imprecise to some. For example, people do not know whether Grid resources would cost more, or less, than other types. • There is likelihood that unless a balanced cross section of information is available about Grid computing - one that can assist rational decision making about adoption - some potential adopters will delay putting time into it, while others may experience disappointment if it turns out to interrupt research output. • For some departments using HPC, the Grid was seen as an option, but for a minority (with some Physics departments being the clearest example), the Grid was essential (for the physicists in its ability to handle data). 5.2.5 Collaboration 5.2.5.1 Email Email is reported by the research departments as an outstanding medium for facilitating research collaboration. It works and everybody uses it. It is seen as offering advantages over all other forms of contact for some purposes, such as working with others in widely different time zones. It has also been mentioned as the best way of fostering some types of research community, e.g. of a very small number of pure mathematicians sharing a particular research area. Mail lists have been cited by numbers of departments, notably in the Arts, as a very valuable means of forming and holding together larger research communities. Mail services are reported generally as very good. It was noteworthy that both the Grid aware and the non-Grid aware groups of departments were had almost exactly coincidental views on email, demonstrating the established nature and embeddedness of this technology. There were a couple of downsides. Some departments report that better access to email, including folders, would contribute to their ability to work at home. Also the use of large attachments can be an issue: these are often used to ship data between collaborators but there are often significant restrictions on size, restricting their benefits. These restrictions are often imposed for good reason and alternative means of transfer are available: however such means are not usually so convenient to the user. Duke & Jordan Page 43 Network Provision for Research Needs, May 2006 5.2.5.2 Video conferencing and telephony Video conferencing from custom built suites is used to a slight extent by about half the departments responding. No form of video conferencing has high levels of use across 5* departments as a group, even in the Grid aware departments the technology is in frequent use only by a small minority, mainly using suite-based services. Only a minority of non-Grid aware users reported on their level of satisfaction, but amongst Grid aware departments, most users were satisfied with the suite-based services available. Only a small number of departments used desktop or trolley-based systems. The ICT service departments’ views were similar, but with higher levels of use estimated generally. It should be noted that the phone interviews showed that suite based services were often provided by other service departments (e.g. Audio-Visual Services). There was little evidence of widespread use of IP telephony other than Skype. Skype has come in through the back door and enjoys some use by about half of the departments contacted. Institutional attitudes towards it vary widely. Some ICT service departments ban it, some accept it, and others turn a blind eye to it. One University revoked an earlier ban during the course of this review but has taken considerable technical steps to control it. As this review neared its 3 completion, the UKERNA Voice Advisory Group produced an information sheet on Skype . The Access Grid was mentioned by over 20 research departments. Most were at an investigatory stage, but looked towards it to meet their needs well. Some users reported problems stemming from its fixed use of IP numbers. 5.2.6 ‘Blue Sky’ thinking Numbers of survey respondents supported the examples given in the survey: • Improved working from home • More ubiquitous wireless networking • Improved video conferencing services Another group could be identified, centred upon use of high quality image data to view valuable resources such as manuscripts, books and paintings. Streaming applications were also mentioned by several departments. The departments making these suggestions were drawn from all disciplines, with the implication that there are support needs to be met if the undoubted capabilities of SuperJANET to deliver improved bandwidth are to be realised. . 5.2.7 Home working Many interviewees and survey respondents commented on the ease of working from home using ADSL lines. More than 80% of the universities responding to the ICT survey offer a VPN or SSH service for access to the institutional network. Together these facilities allow home-workers a semblance of the workplace network environment. Some respondents without VPN access noted that not all their needs were met, citing access to email folders and access to the .ac.uk domain as problems Interestingly, it was clear that working from home is perhaps the area which will benefit most from improved network bandwidths. The benefits to researchers are obvious. For example, some experimenters expressed a wish to monitor online instrumentation outside normal working hours. The same sentiment was expressed regarding monitoring long-running compute jobs. Another benefit which can be foreseen is to facilitate the employment of short term contract or part-time workers who may be able to work from a distance rather than have to move to take a post or may be able to work part-time at more than one institution. 3 http://www.ja.net/development/voip/skype&janet.pdf Duke & Jordan Page 44 Network Provision for Research Needs, May 2006 5.3 5.3.1 Management and funding Differences between institutions Our interviews showed that different institutions operate very differently. Some institutions devolve responsibility for the network within buildings almost wholly to departments whereas others concentrate responsibility in the central ICT service. In the former case there are almost invariably arrangements available so that departments can, for a price, contract the central department to manage the network for them. Some institutions are consciously changing their arrangements. One ICT respondent told us “Our model is changing at present from department responsibility for all aspects …. of [departmental] networks, to a more corporate style where Planning, Management and 1st line Support will become a joint responsibility.” Funding streams of course affect the planning cycle, particularly where funding is largely in the hands of the department. One interviewee told us that it is important to note that nearly all the department’s income is ‘soft’ and has to be reasonably proof against a lean year. This adversely affects levels of expenditure and makes medium and long term planning very difficult. Another interviewee noted that planning is an area of concern, as income streams are volatile. 5.3.2 Differences of perception within institutions The results of the two surveys show markedly different views of who is perceived as being responsible for managing and funding the network. Grid aware departments tend to the view that they carry the can whereas non-Grid aware and ICT service departments tend to the view that the network is a central responsibility. The greatest differences of view between Grid aware and ICT service respondents are in their perceptions of who plans the network and who provides first line support for it. The significant difference in view between Grid aware and ICT service departments suggests either a lack of communication or a lack of clarity about what arrangements are actually in place. It is noteworthy that one science department said to us that its central ICT service department does little for the physical sciences but probably more for the humanities. Another department at the same institution said that, though planning is entirely in departmental hands, consultation within the institution is very good. A further department at the same institution said “planning is entirely central.” The differences of view between Grid aware – often science and engineering departments - and non-Grid aware departments – usually arts and humanities departments - may be for a number of reasons: 1. 2. 3. Science and engineering departments have larger incomes and are therefore in a better position to spend more on departmental infrastructure. Science and engineering departments may consider the network more important to their work than do arts departments. Arts and humanities departments may take the view that the network infrastructure is not part of their core business and they therefore tend to leave it to the central ICT service department. All of this, however, points up the difficulties of implementing an effective and coherent planning process. The importance of planning was however stated succinctly by a respondent to the survey of researchers: “We are a tiny Department, …mostly occupied with lone-scholar text-based research. I myself have moved increasingly to web-based research, and I think that others will follow. It is incredibly difficult to plan on the basis of a current snap-shot; but in our field the cost of traditional publication is going to be a motor in transforming the way we work and [will] enormously [increase] the demands that we make on the system.” Duke & Jordan Page 45 Network Provision for Research Needs, May 2006 6 6.1.1 Conclusions and recommendations Network Connections 1 In most institutions, most departmental networks provide switched links to users at 100 Mbps. 2 About 80% of institutions have some of their research staff connected at 10 Mbps or below. 3 Coaxial cabling is present in about 30% of institutions, and is often a consequence of being allowed to be bound in with very protracted building refurbishment cycles, which are thus a cause of inadequate network provision to research workers. The reviewers were surprised and concerned to discover the extent of legacy provision, much of which would be expected to be 15 - 20 years old. Recommendation 1 The third of institutions with researchers served by coaxial networks and the rather larger group with researchers served by shared network access should ensure that they possess and implement plans to replace this with current technology. 4 The research departments are well satisfied with the speed and reliability of their network connections to departmental and institutional resources. They are also satisfied with connections into and beyond JANET, although hardly surprisingly their levels showed some decline with distance. 5 Grid aware respondents are generally happier with the speed and reliability of network connections within their departments and institutions than are non-Grid aware respondents. This could be because a) Grid aware respondents spend more on their networks; b) Grid aware respondents have more local support for their use of the network; c) Grid aware respondents are more tolerant of weaknesses in the technology. 6 It is now the norm for institutions to possess a firewall. 7 Firewalls, mainly their policy aspects and their impacts upon ad hoc laptop use, impair research work in about a third of departments, most of which were Grid aware. 6.1.2 Network services 1 Not surprisingly, knowledge of the Grid is largely associated with science and engineering departments. 2 The single most common form of data backup is to users’ personal media (DVDs, CDs, or portable tape drive), which is used by nearly 80% of users/departments. The most common reasons stated are: a) Completely inadequate quotas offered on institutional resources. b) Cost of central storage and backup services, when these are available. Recommendation 2 Institutions should ensure that systematic approaches are used to backup research data. 3 In general, the ICT service departments overestimated the use of central facilities and underestimated use of personal media. 4 The picture for data storage showed that the most common place to store data is on the user’s desktop, and that personal external media are widely used for storage as well as backup. 5 The Grid aware departments make substantially greater use of departmental servers for access to specialist software as well as data storage and backup than the non-Grid aware departments, who make rather more use of central servers than the first group. 6 For the overwhelming majority of research departments, video-conferencing (all forms) is only used to a slight extent, even amongst the Grid aware subset. Suite based video conferencing is the most common choice, used by about 40% of departments and is rated as generally satisfactory. Other forms of video communications (e.g. desktop videoconferencing) are little used, even though interest in them was found. 7 Skype enjoys some use by about half of the departments contacted. The lack of a central steer has been interpreted in different ways by institutions, but presently the trend is towards acceptance. Duke & Jordan Page 46 Network Provision for Research Needs, May 2006 JISC and UKERNA should produce clear and timely guidance for institutions on adoption of new network applications. 8 Broadband ADSL and VPNs are permitting large amounts of research work to be performed from home. As well as having implications for research productivity, this will facilitate more flexible patterns of work and employment, such as use of short-term or parttime contracts. Recommendation 4 National and institutional policies and planning need to reflect the potential impact upon working practices presented by this technology-based change. 9 Amongst the science and engineering departments there is widespread use of locally clustered computing, which is commonly felt likely to provide capacity to meet needs for at least the next two years. 10 Amongst the science and engineering departments about a third of respondents felt that they would be making appreciable use of the Grid by two to four years time. 11 A substantial number of potential Grid users have consciously decided to hold back from involvement because of the learning curve involved and the availability of their own clusters. 12 Those science and engineering departments with their own HPC resources can be expected to continue to develop them and their doing so will inevitably tend to offset the attractiveness of moving to the Grid. Thus the appeal of the Grid will need to grow faster than that of local resources if it is to remain as an attractive option for migration in the short or medium term. Recommendation 5 More education to allow users to assess Grid technology and plan for its adoption is required. 13 Of those who replied to the question asking how access to ubiquitous unlimited network bandwidth would change their working patterns, nearly half envision improved working from home (this group embraced all types of department), followed by a quarter each wishing either to have more ubiquitous wireless networking or improved video conferencing services. 14 The responses to a number of the survey questions show that generally the ICT service departments’ responses are similar to those of the non-Grid aware research departments but differ from those of the Grid aware departments. This could be because the Grid aware departments have more internal support and therefore do not need the central service as much as the non-Grid aware departments. Recommendation 3 Duke & Jordan Page 47 Network Provision for Research Needs, May 2006 Appendix I 5* departments and universities A small number of institutions have more than one 5* department in an RAE area. In such cases the number is shown in brackets after the Unit of assessment name. Institution Birkbeck College Cardiff University City University Courtauld Institute of Art Goldsmiths College Heriot-Watt University Imperial College of Science, Technology and Medicine Institute of Cancer Research Institute of Classical Studies Institute of Germanic Studies Institute of Latin American Studies Keele University King' s College London Lancaster University Duke & Jordan Unit of assessment name English Language and Literature History Iberian and Latin American Languages Civil Engineering Education English Language and Literature Other Studies and Professions Allied to Medicine Psychology Theology, Divinity and Religious Studies Town and Country Planning Music History of Art, Architecture and Design Communication, Cultural and Media Studies Sociology Mineral and Mining Engineering Applied Mathematics Biological Sciences Chemical Engineering Chemistry Civil Engineering Clinical Laboratory Sciences Computer Science General Engineering Hospital-based Clinical Subjects Mechanical, Aeronautical and Manufacturing Engineering Mineral and Mining Engineering Physics Pure Mathematics Biological Sciences Clinical Laboratory Sciences Classics, Ancient History, Byzantine and Modern Greek Studies German, Dutch and Scandinavian Languages American Studies Law Anatomy Classics, Ancient History, Byzantine and Modern Greek Studies Clinical Dentistry Community-based Clinical Subjects German, Dutch and Scandinavian Languages History Iberian and Latin American Languages (2) Philosophy Politics and International Studies Business and Management Studies Page 48 Network Provision for Research Needs, May 2006 Liverpool John Moores University London Business School London School of Economics and Political Science Loughborough University Manchester Metropolitan University Open University Oxford Brookes University Queen Mary, University of London Royal Holloway, University of London School of Oriental and African Studies The Queen' s University of Belfast University College London University of Aberdeen University of Bath University of Birmingham Duke & Jordan Physics Sociology Statistics and Operational Research Sports-related Subjects Business and Management Studies Accounting and Finance Anthropology Economics and Econometrics History Law Philosophy Social Policy and Administration Built Environment Sociology Sports-related Subjects Sports-related Subjects Geography History Iberian and Latin American Languages Law Linguistics French Geography German, Dutch and Scandinavian Languages Music History Mechanical, Aeronautical and Manufacturing Engineering Anthropology Chemical Engineering Chemistry Classics, Ancient History, Byzantine and Modern Greek Studies Economics and Econometrics English Language and Literature Geography German, Dutch and Scandinavian Languages Hospital-based Clinical Subjects (3) Italian Law Linguistics Pharmacology Psychology French Applied Mathematics Mechanical, Aeronautical and Manufacturing Engineering Pharmacy Anatomy Page 49 Network Provision for Research Needs, May 2006 University of Bradford University of Bristol University of Cambridge Duke & Jordan Chemical Engineering Clinical Laboratory Sciences European Studies French German, Dutch and Scandinavian Languages Italian Metallurgy and Materials Middle Eastern and African Studies Music Psychology Sports-related Subjects European Studies Anatomy Applied Mathematics Biological Sciences Chemistry Civil Engineering Clinical Dentistry Community-based Clinical Subjects Drama, Dance and Performing Arts Earth Sciences Education Geography Psychology Russian, Slavonic and East European Languages Social Work Statistics and Operational Research Applied Mathematics Archaeology Asian Studies Biological Sciences (2) Celtic Studies Chemistry Classics, Ancient History, Byzantine and Modern Greek Studies Clinical Laboratory Sciences Community-based Clinical Subjects Computer Science Earth Sciences English Language and Literature French General Engineering German, Dutch and Scandinavian Languages History Hospital-based Clinical Subjects Iberian and Latin American Languages Italian Law Linguistics Metallurgy and Materials Music Philosophy (2) Page 50 Network Provision for Research Needs, May 2006 University of Dundee University of Durham University of East Anglia University of Edinburgh University of Essex University of Exeter University of Glasgow University of Kent at Canterbury University of Leeds University of Leicester University of Liverpool University of Manchester Duke & Jordan Physics Psychology Pure Mathematics Statistics and Operational Research Biological Sciences Clinical Laboratory Sciences Applied Mathematics Chemistry English Language and Literature Geography History Law Communication, Cultural and Media Studies Environmental Sciences History Computer Science Electrical and Electronic Engineering English Language and Literature Geography German, Dutch and Scandinavian Languages Hospital-based Clinical Subjects Middle Eastern and African Studies Philosophy Pure Mathematics Economics and Econometrics Politics and International Studies Sociology German, Dutch and Scandinavian Languages English Language and Literature European Studies Psychology Sports-related Subjects Social Policy and Administration Statistics and Operational Research Electrical and Electronic Engineering English Language and Literature Food Science and Technology Italian Mechanical, Aeronautical and Manufacturing Engineering Town and Country Planning Biological Sciences English Language and Literature Mechanical, Aeronautical and Manufacturing Engineering Physiology Accounting and Finance Biological Sciences Computer Science French German, Dutch and Scandinavian Languages Iberian and Latin American Languages Page 51 Network Provision for Research Needs, May 2006 University of Manchester Institute of Science & Technology University of Newcastle University of Nottingham University of Oxford University of Reading University of Salford University of Sheffield Duke & Jordan Metallurgy and Materials Music Pharmacy Pre-Clinical Studies Sociology Theology, Divinity and Religious Studies Metallurgy and Materials Other Studies and Professions Allied to Medicine Biological Sciences Clinical Laboratory Sciences Music Psychology American Studies German, Dutch and Scandinavian Languages Iberian and Latin American Languages Music Theology, Divinity and Religious Studies Archaeology Asian Studies Chemistry Classics, Ancient History, Byzantine and Modern Greek Studies Clinical Laboratory Sciences Community-based Clinical Subjects Earth Sciences English Language and Literature French General Engineering Hospital-based Clinical Subjects Italian Law Linguistics Metallurgy and Materials Music Pharmacology Philosophy Physics Politics and International Studies Psychology Pure Mathematics Russian, Slavonic and East European Languages Statistics and Operational Research Theology, Divinity and Religious Studies Archaeology English Language and Literature Environmental Sciences Italian Psychology Built Environment Library and Information Management Biological Sciences (2) Page 52 Network Provision for Research Needs, May 2006 University of Southampton University of St Andrews University of Surrey University of Ulster University of Wales, Aberystwyth University of Wales, Bangor University of Wales, Swansea University of Warwick University of York Warburg Institute Duke & Jordan Electrical and Electronic Engineering (2) Library and Information Management Metallurgy and Materials Politics and International Studies Pre-Clinical Studies Russian, Slavonic and East European Languages Civil Engineering Computer Science Electrical and Electronic Engineering European Studies Law Mechanical, Aeronautical and Manufacturing Engineering Music Physics English Language and Literature Psychology Electrical and Electronic Engineering Other Studies and Professions Allied to Medicine Sociology Celtic Studies Other Studies and Professions Allied to Medicine Celtic Studies Politics and International Studies Celtic Studies Psychology Civil Engineering Applied Mathematics Business and Management Studies Drama, Dance and Performing Arts Economics and Econometrics English Language and Literature Statistics and Operational Research Computer Science English Language and Literature Psychology Classics, Ancient History, Byzantine and Modern Greek Studies Page 53 Network Provision for Research Needs, May 2006 Appendix II List of topics for discussion sent out before visits to departments 1. Which Research Councils fund their work. 2. The role computing plays in the research activity. 3. The research group’s uses for the network present and expected. 4. Basic network provision to the research group (i.e. to its computers). We would expect this to cover a) Performance and reliability b) Any policies about the standards of networking provision to research workers. c) Any policies about security and access. d) Support of the research group’s networking needs by its own department e) Central network support provided by the contact’s institution f) Anything more about adequacy of basic network services. 5. Networking technology issues. This may include problems with firewalls or use of the Grid. 6. Budgets for the research group’s networking needs. 7. How access to unlimited network access bandwidth would change the way in which your department performs research. 8. Anything else you want to tell us. Duke & Jordan Page 54 Network Provision for Research Needs, May 2006 Appendix III List of topics for discussion sent out before phone calls to academic departments 1. The principal funding bodies for the research work. 2. The role computing plays in the research activity. 3. The research group’s uses for the network and whether the Grid is involved. (This includes processing, data access, software, data storage and back-up). 4. Network provision to the research workers, covering such as technology, bandwidth, reliability and firewalls. Any outstanding needs. 5. Collaboration with others. The nature of services used here (e.g. email, video conferencing and Skype). Any outstanding needs. 6. Policies on network security and access and whether they affect research work. 7. How network planning is conducted, e.g. involvement of central ICT services. 8. Division of network support between central and departmental provision. 9. Mechanisms used for network funding. 10. How access to unlimited network access bandwidth would change the way in which the department performs research. 11. Any other networking issues. Duke & Jordan Page 55 Network Provision for Research Needs, May 2006 Appendix IV List of topics for discussion sent out before phone calls to ICT service departments 1. Network provision to the research workers, covering such as technologies, bandwidth, reliability and firewalls. 2. The research groups’ uses for the network. (This includes processing, data access, software access, data storage, data back-up and possibly the Grid). 3. Services used to facilitate collaboration with others (e.g. email, video conferencing and Skype). 4. Policies on network security and access and whether they affect research needs. 5. How network planning is conducted, e.g. involvement of departments. 6. Division of network support between central and departmental provision. 7. Mechanisms used for network funding. 8. How access to unlimited network access bandwidth might change the way in which the departments perform their research. 9. Any other networking issues that are relevant to research. Duke & Jordan Page 56 Network Provision for Research Needs, May 2006 Appendix V Web survey sent to academic departments ! " #$ ! " ! " !% #$ " ! & " " ' * + " ()* * * * * ! " !% " " & Duke & Jordan " - " " " , ! ! ! ! ! Page 57 Network Provision for Research Needs, May 2006 %$#& ' " ($' ) " 1$2 $ , Duke & Jordan ! 3& , , *$' " +$' ,- . /0 /0 ,4 . , & , , Page 58 $ Network Provision for Research Needs, May 2006 $' 5$2 ! , , " !, . # ! " ! 4 .&/ " 6$ , Duke & Jordan , ! 7 0 1 $ Page 59 Network Provision for Research Needs, May 2006 $ 8$ , ,. ! " $ ! . & / ( 2,3 4 3 , 24 2,5 4 5 ,6224 , , . 1 " 62 7" 622 7" 68" 62 7" " " " 622 7" " " " 68" " " " 8 " 68" " Duke & Jordan " Page 60 Network Provision for Research Needs, May 2006 9$ , " " ! ! . / 62 7" 62 , 622 7" 9 622 7" Duke & Jordan / , , 2,3 4 3 , 24 2,5 4 5 ,6224 . " . $ ! 1 " 9 8 . , , 68" 68" Page 61 Network Provision for Research Needs, May 2006 (:$; , , / . , , . ! " . 4 " : + - + - 1/; + - 1/; + - " - : : : : + : .&/ : (($ ." ,, & ! , &, 4 Duke & Jordan Page 62 Network Provision for Research Needs, May 2006 (*$< 7 & , ,$ < / = 1 (+$ & Duke & Jordan " , . = ! 7 & , ,/!, , , "$ Page 63 Network Provision for Research Needs, May 2006 (1$< 7 . . & , , ,& . , "! $ < / " " ( $ " Duke & Jordan & . , "! & & , , , . Page 64 , , , . , ,& 4 Network Provision for Research Needs, May 2006 $ = (5$ , $ " & " , , . 9 , , . ! "!, 1 & > . 9 9 9 Duke & Jordan Page 65 Network Provision for Research Needs, May 2006 (6$ , $ " " , , , , . ! "!, . 9 1 & & ! > . 9 9 (8$ Duke & Jordan ! . /!, "! Page 66 $ Network Provision for Research Needs, May 2006 (9$ & !$ *:$< 7 , " ! ." & , /!, !, ,, & , ,& , ,, / , / / , , . & ! " $) / $ # ? ;( = ; ; . . ," * " . + .?;1 Duke & Jordan 8 Page 67 Network Provision for Research Needs, May 2006 *($ , , ,& Duke & Jordan , , & . , "! $ & ! Page 68 , Network Provision for Research Needs, May 2006 )$ , "! * . > " > 1 **$ , ! &, $ $ 8 "! , , . 8 ! , ! / 4 , , , " " ! ? % 6 % 3 % @ 7 Duke & Jordan @ Page 69 Network Provision for Research Needs, May 2006 *+$ , , , ,> ! ! $ # ? 6 > ? 3 > ? @ > *1$ /! Duke & Jordan . , , 8 , , . & ! , "! . > $ Page 70 Network Provision for Research Needs, May 2006 $ * $< ) 7 & . , , " ? !! &, ! !! &, & ! " ) , , < $ ! " , . !$ , ! . , /!, , ,& , " & ! " / $ !- , . , ! &, $ A & 7 * & Duke & Jordan Page 71 Network Provision for Research Needs, May 2006 *5$ ) , Duke & Jordan . , , < & . &, !! Page 72 ! &, & 4 ! " Network Provision for Research Needs, May 2006 >$; *6$< & . , , @ %, / , B - ?. A7 , , -2 . -= . ! -# Duke & Jordan . $ ," 4 , . , " "!, & , " , , , . , - " , Page 73 Network Provision for Research Needs, May 2006 *8$# . , "" " ! 4 *9$0 < +:$ Duke & Jordan !! . , ! 4 / , 7 ' /!, . Page 74 " , $ Network Provision for Research Needs, May 2006 < Duke & Jordan " "!, . $ , , Page 75 , $ Network Provision for Research Needs, May 2006 Appendix VI Web survey sent to service departments ! " #$ ! " ! " #$ !% " ! & " " ' * , * " ()* - " * * " * + " !% " " Duke & Jordan " ! ! " & ! ! ! ! Page 76 Network Provision for Research Needs, May 2006 %$#& ' " ($' ) " 1$2 $ , Duke & Jordan ! 3& , , *$' " +$' ,- . /0 /0 ,4 . , & , , Page 77 $ Network Provision for Research Needs, May 2006 $' 5$2 ! , , " !, . # ! " ! 4 .&/ " 6$ , Duke & Jordan , ! 7 0 1 $ Page 78 Network Provision for Research Needs, May 2006 $ 8$ , ,. ! " $ ! . & / ( 2,3 4 3 , 24 2,5 4 , , 5 ,6224 . 1 " 62 7" 622 7" 68" 62 7" " " " Duke & Jordan Page 79 Network Provision for Research Needs, May 2006 9$ , " " ! ! . / 62 7" 62 , 622 7" 9 622 7" Duke & Jordan / , , 2,3 4 3 , 24 2,5 4 5 ,6224 . " . $ ! 1 " 9 8 . , , 68" 68" Page 80 Network Provision for Research Needs, May 2006 (:$; , , / . , , . ! " . 4 " : + - + - 1/; + - 1/; + - " - : : : : + : .&/ : (($ ." ,, & ! , &, 4 Duke & Jordan Page 81 Network Provision for Research Needs, May 2006 (*$< 7 & , ,$ < / = 1 (+$ & Duke & Jordan " , . = ! 7 & , ,/!, , , "$ Page 82 Network Provision for Research Needs, May 2006 (1$< 7 . . & , , ,& . , "! $ < / " " ( $ " Duke & Jordan & . , "! & & , , , . Page 83 , , , . , ,& 4 Network Provision for Research Needs, May 2006 $ = (5$ , $ " & " , , , , . ! "!, . 9 1 & > . 9 9 9 Duke & Jordan Page 84 Network Provision for Research Needs, May 2006 (6$ , $ " " , , , , . ! . 9 "!, 1 & & ! > . 9 9 (8$ Duke & Jordan ! . /!, "! Page 85 $ Network Provision for Research Needs, May 2006 (9$ & !$ *:$< 7 , " ! ." & , /!, !, ,, & . , ,& , ,, / , / / , , & ! " $) / $ # ? ;( = ; ; . . ," * " . + .?;1 Duke & Jordan 8 Page 86 Network Provision for Research Needs, May 2006 *($ , , ,& Duke & Jordan , , & . , "! $ & Page 87 ! , Network Provision for Research Needs, May 2006 )$ , "! * . > " > 1 **$ , ! &, $ $ "! , , . 8 8 ! , ! / 4 , , , " " ! ? % 6 % 3 % @ 7 Duke & Jordan @ Page 88 Network Provision for Research Needs, May 2006 *+$ , , , ,> ! ! $ # ? 6 > ? 3 > ? @ > *1$ /! Duke & Jordan . , , 8 , , . & ! , "! . > $ Page 89 Network Provision for Research Needs, May 2006 $ * $< ) 7 & . , , ? " !! &, ! !! &, & ! " ) , , < $ ! " , . !$ , ! . , /!, , ,& , " & ! " / $ !- , . , ! &, $ A & 7 * & Duke & Jordan Page 90 Network Provision for Research Needs, May 2006 *5$ ) , Duke & Jordan . , , < & . &, !! Page 91 ! &, & 4 ! " Network Provision for Research Needs, May 2006 >$; *6$< & . @ %, / , B - ?. . -= . ! Duke & Jordan A7 , , -2 -# , , . $ , . , " ," 4 "!, & , " , , , . , - " , Page 92 Network Provision for Research Needs, May 2006 *8$# . , "" " ! 4 *9$0 < Duke & Jordan !! . , ! 4 / Page 93 Network Provision for Research Needs, May 2006 +:$ < Duke & Jordan , " 7 "!, ' . /!, $ , . " , Page 94 , , $ $ Network Provision for Research Needs, May 2006