University of NSW Communication Services Out-of-Hours Support Description of IT at UNSW • Like many other large organizations, UNSW has oscillated between a centralised and distributed support model. • At the moment we have a hybrid model with support for centralized services and infrastructure as well as local faculty support (that report to the CIO) for school- or faculty-level desktops and applications etc. Servicedesk • One number / single point of contact for IT support • Normal Phone and Email Support hours are 8AM to 8PM weekdays. • There is a skeleton crew of (student) casuals after hours. Users are directed to the Servicedesk manager’s mobile if no staff are available. Communications Services • Our group looks after all aspects of the voice and data network infrastructure • Given that the voice and data networks are critical infrastructure, our team is required to also provide limited out-of-hours support Out of Hours Support: The Old Way • In the past, this was done in a fairly ad-hoc manner; the Servicedesk would call the Comms manager who would then try and contact the relevant support staff via SMS or mobile phone.. i.e. there were no proper on-call arrangements. • There were a few problems with this earlier approach: – The Comms manager was a single point of failure – what if he/she was out of contact? – There was no guarantee that support staff would be available – The burden of responsibility was not shared fairly – one or two people ended up doing it all. Current Out of Hours Support • Given the increasing reliance on central infrastructure, it was decided that we should come up with a better out-of-hours support model. This was done in consultation with the groups/teams who provide that support. • In the new support model, most senior staff in the group are on-call on a weekly basis. Non-technical staff (including the manager) share the role of resolution manager. Current Out of Hours Support • The role of the resolution manager is similar to that which AARNet uses in that they are responsible for contacting the relevant on-call technical resource, make sure that the incident is dealt with properly and to filter calls (we only provide on-call support for unscheduled severity 1 incidents). • As far as the roster goes, each week there is one resolution manager (out of a pool of 3), one network resource (pool of 4) and one voice resource (pool of 2) on-call. Current Out of Hours Support • The Servicedesk is given a roster of who to call on what week. Note that they only call the resolution manager but there are details on who to call if the primary person can’t be contacted. • Since we are only on-call for severity 1 incidents, junior staff aren’t used in the on-call system as they do not yet have the skills to fix major problems in any part of the network. • On-call support is voluntary but since we had input on how we provide out-of-hours support and because we have very few severity 1 incidents outof-hours, nobody has opted out. On-Call Allowances / Overtime • Staff that are on-call are paid an allowance of 30% of their base salary for the week they are on-call. This is not part of the Enterprise Agreement – it’s an informal agreement between the staff, management and HR. • Anyone who gets called when they are on-call get a minimum of 4 hours overtime paid per the Enterprise Agreement. • Note that if a project requires a network resource to be on-call during scheduled changes then this is separate to the normal on-call arrangements and is paid as overtime. Miscellaneous • All on-call staff are provided with the resources they need to provide support from whenever they are including a home broadband connection and 3G/HSDPA modem for their laptop. • There is a budget for on-call/overtime arrangements but given that the network is highly redundant and it is very rare for us to get called out of hours (less than once per month), we never spend it all. • We also have an SMS alert service for critical infrastructure.