University Information Technology Services Enterprise Infrastructure Division proudly presents Smart Administration Services What are Smart Administration Services? SMART Administration Services, provided by the Enterprise Infrastructure Division of UITS, are best described as comprehensive, complimentary technology services for the University virtualized hosting environment. It is the intent of SMART Administration Services to simplify the day to day operations of departmental technology professionals by off-loading repeatable, operational activities to UITS EI. Built within the framework of the nationally acclaimed Intelligent Infrastructure (II), Smart Administration Services provides operating system administration support for both Microsoft Windows and RedHat Enterprise Linux as well as database administration services for Oracle, Microsoft SQL Server and My SQL. What types of Smart Administration Services are offered? Smart Administration Services are currently offered in two basic forms, namely, System Administration (SA) and Database Administration (DA) services. Leveraging the University’s deep server administration expertise, technology professionals have an option to offload common operational activities such as operating system installation, database installation, security patching, backup and monitoring. You pick how much server administration or database administration services you want and need. Server administration is provided by the Enterprise System Administration (ESA) and database administration is provided by the Database Administration (DBA) units within the Enterprise Infrastructure Division of UITS. Both units are prepared to customize the level of support you require. When you choose to purchase DA services, the SA services are automatically included (ESA manages the DBA servers). You can also choose to only purchase SA services. How to engage Smart Administration Services? You can engage UITS EI Smart Administration Services by contacting either Dan Young (Manager of Enterprsie Database Administration) – youngdj@iu.edu Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 1 or Denise Craig (Manager of Enterprise System Administration) – dmcraig@iu.edu Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 2 University Information Technology Services Enterprise Infrastructure Division Smart Administration Services Service Offerings SMART Administration Services, provided by the Enterprise Infrastructure Division of UITS, are best described as comprehensive, complimentary technology services for the University virtualized hosting environment. It is the intent of SMART Administration Services to simplify the day to day operations of departmental technology professionals by off-loading repeatable, operational activities to UITS EI. Built within the framework of the nationally acclaimed Intelligent Infrastructure (II), Smart Administration Services provides operating system administration support for both Microsoft Windows and RedHat Enterprise Linux as well as database administration services for Oracle, Microsoft SQL Server and My SQL. SYSTEM ADMINISTRATION CORE SERVICES Enterprise Redhat Linux (5.X & 6.X) Windows 2008 R2 & Windows 2012 Supported Hardware Platforms and Specifications Notes: Assumes VMs have been purchased through IU Intelligent Infrastructure. Processors OS Max OS Max Platform Dependent RAM OS Max OS Max Platform Dependent Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 3 Disk Space Unlimited Unlimited Installation Services Define, Document, Recommend and Review VM build specifications Install Operating System To IU standards and best practices. Apply current OS security patches As applicable, identified by IU WSUS and determined by Microsoft and IU RHEL repository Install and setup up local host and Network (NOC) firewalls To IU standards and best practices Request and obtain server IP and DNS Request, evaluate and resolve issues relating to initial UISO security scan Install and patch any Windows or Linux OS products, as required by the customer Windows products such as but not limited to Print Server, Terminal Server and IIS Linux products within the IU RHEL repository. SYSTEM ADMINISTRATION MAINTENANCE SERVICES Administration Services Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 4 Ongoing maintenance of Operating System Perform Annual Disaster Recovery Testing Performed on selected number of DR restores on production environment virtual machines using SAV DR snapshots Monitor server logs and resolve errors impacting system availability. Resolution to be jointly completed by the Customer and UITS EI Provide Production System Troubleshooting and Critical Issue Resolution All Hours with 1 hour response. Critical being defined as a production system which is unavailable, unusable or presenting data privacy concern. Provide Production/Test/Development Troubleshooting and Non-Critical Issue Resolution Normal Business Hours (8am to 5pm), with 4 hour response subject to normal issue submission and resolution procedures Provide Server OS Level Performance Tuning Memory, I/O, CPU, etc Provide VM Cloning for the support of Systems Development. For Development/Test/Production Systems; applicable to Linux environments only Provide ESA liaison who will be appointed to be point of contact for customer Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 5 Provide a way to submit requests for assistance via IU Footprints ticketing system for tracking. Security Services Application of Operating System patches and upgrades Adhere to current UITS EI standard patching schedule Review and evaluation of monthly scheduled UISO security scans Resolution to be jointly completed by the Customer and UITS EI Account Management Completed in partnership with Account Management Team and Customer. This only pertains to server access. Application access is the responsibility of the customer. Network Based Access Filtering Commonly known as 'IP Filtering' Configuring Encryption Technologies Completed in partnership with UITS EI and Customer, per customer requirements. Backup Services If TSM backup option was purchased via IU Intelligent Infrastructure. Installation and maintenance of TSM Client Provide setup of cross-site, replicated backups Monitoring of nightly backup and issue resolution Enterprise Infrastructure Smart Administration Services – FY 2013-2014 Resolution to be jointly completed by the Customer and UITS EI (7/25/13) Page 6 DATABASE ADMINISTRATION CORE SERVICES Oracle (11G) Sql Server (2008 , 2012) MySQL (5.0) Supported Hardware Platforms and Specifications Notes: Assumes VM purchase through IU II and that ESA Core Services have been provisioned. Processors OS Max OS Max OS Max Platform Dependent RAM OS Max OS Max OS Max Platform Dependent Database Size Unlimited Unlimited ~200MB Virtualized Windows Virtualized Linux Installation Services Define, Document, Recommend and Review Database Installation Specifications Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 7 Install RDBMS according to RDBMS Vendor Specifications Validate RDBMS connectivity and operation Install database monitoring scripts and integrate with centralized alerting services. Monitoring includes: System Availability, Processor Utilization, Physical Memory and Storage Capacity Automated Workload Reporting (AWR) and/or database monitoring access via Confio Ignite DATABASE ADMINSTRATION MAINTENANCE SERVICES Administration Services Create, Configure and Schedule RDBMS Backup Scripts, per Customer Specifications. Perform Annual Recovery Testing Monitor RDBMS System Logs and resolve critical errors impacting system availability. Monitor and provide consultation regarding non-critical messages or errors with the RDBMS System. Resolution to be jointly completed by the Customer and UITS EI Provide Production System Troubleshooting and Critical Issue All Hours with 1 hour response. Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 8 Resolution. Critical being defined as a production system which is unavailable, unusable or presenting data privacy concern. Provide Production/Test/Development Troubleshooting and Non-Critical Issue Resolution Normal Business Hours (8am to 5pm) with 4 hour response subject to normal issue submission and resolution procedures. Perform RDBMS Vendor Management Activities, including creation of Service Requests, License Management, etc. Server Level Performance Tuning Memory, I/O, CPU, etc. Database Cloning and Data Migration for the support of Systems Development. For Development/Test/Production Systems. Security Services Install applicable RDBMS Software Upgrades and Patches During Standard, Scheduled Maintenance Windows. Database Account Management Completed in partnership with the Service Owner as appropriate Network Based Access Filtering Commonly known as 'IP Filtering' Configuring RDBMS Encryption Technologies Network and Data Encryption, per customer requirements. Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 9 Maintain central repository and connection file template for Client Connectivity. Centralized TNS Names and Oracle Internet Directory Optional SMART Database Administration Services The services noted in the matrix below are not part of the general monthly services offered by the Enterprise Database Administration team. However, the team can bring certain expertise to the services noted below when requested by the customer, with fees for service charged on an hourly basis in addition to the standard monthly rates. OPTIONAL DATABASE SERVICES Oracle (11G) Sql Server MySQL (2008, 2012) (5.0) Notes: Service Application Performance Tuning (SQL Based Query Tuning) Enterprise Infrastructure Smart Administration Services – FY 2013-2014 Ultimately, the Application Service Owner is responsible for the design and performance of SQL executed against the RDBMS. The Database Administration team can assist in these activities, particularly in the identification of long running (7/25/13) Page 10 queries, but cannot be responsible for performance over which we have no design or control. Automated Shell Script Development and Schedule Advising (CRON, BRTE.) Data Modeling and Design. (Best Practices) RDBMS Procedure Development (Functions, Procedures, etc.) Client Connectivity Software (Application Server) Enterprise Infrastructure Smart Administration Services – FY 2013-2014 Includes the Oracle Customer, JDBC/ODBC connectivity and Cobol Pre-compilers. (7/25/13) Page 11 SMART Administration Service Pricing The following is the pricing model for the services listed in the matrixes above. The pricing is “per virtual server” and would be in addition to the traditional IU II virtual system and backup service fees. Core Services (one time setup fee) Smart System Administration Services Smart Database Administration Services $420 (up to 4 hours labor) $640* (up to 4 hours labor) * Includes Smart System Administrative Services Maintenance Services (monthly fee) $110 / month (up to 2 hours labor) $240 / month* (up to 2 hours labor) Optional Services (hourly fee) $85 / hour $85 / hour Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 12 SMART Administration Service Terms Smart Administration Service Terms (SMART) Termination of Services Consulting and Troubleshooting Services Problem Resolution Either the customer or Smart Administration Services may terminate this agreement with 60 day notification. Upon termination of SMART, the administrative teams will assist the customer in moving the SMART system to a customer designated location. This process will minimally include securing a new IP address, VLAN modifications and updating network firewall rules. Optionally, for SMART database systems, an appropriate RDBMS export or backup file will be produced to allow the customer to easily migrate the data from a SMART system to a customer managed system. At completion of the termination process, all SMART administrative scripts, processes and access will be removed with formal customer acknowledgement that the customer has taken full responsibility for administrative activities. Consulting and troubleshooting services are available upon request to SMART Administrators. Response time will vary based upon personnel resources and volume of pending requests. In general, all customers will receive the same priority for consulting services. SMART Administrators reserve the right to reprioritize pending requests for services as circumstances may require. There is no additional fee for routine consulting and troubleshooting up to the monthly limit of 2 hours. Extensive consulting and troubleshooting services beyond the noted monthly limit may result in additional fees. Additional fees will be communicated with and agreed to by the customer in advance of rendering services. If a SMART system experiences a problem identified as critical (i.e. A production system is Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 13 deemed unavailable, unusable, or presenting data privacy concern), a designated customer contact may immediately notify the SMART Administration Team as noted in the ‘SMART Issue Notification Guidelines’ section below. Critical issues are applicable to production systems only. Data Backup and Restoration In the event of a non-critical application or server issue, the customer must first consult the internal application service support personnel in attempt to resolve the situation. If the problem cannot be resolved by the customer's internal support personnel and the issue is believed to be operating system or database related, the customer may contact SMART Administrators for assistance as noted in the ‘SMART Issue Notification Guidelines’ section below. Problem reports made between the hours of 8am and 4pm on a standard business day will be investigated on that same business day. Problems reported outside of normal business hours will be investigated as quickly as possible on the next business day. When customers or users report a problem, the following information is needed: o Identify that the virtual server is part of the II SMART Environment. o Include the name of the virtual server and the application. o The name, email address, and phone number of who to contact during troubleshooting. o A description of the problem, including details such as: o The complete URL to the problem page. o The UNC path to the share containing the file with the problem. o The server and database name if the problem is a database connection. o A description of the aberrant behaviors. The IU II utilizes FalconStor Virtual Tape Libraries and Tivoli Storage Manager (TSM) for system backup and restoration. TSM backups are used to backup data volumes only with the following defaults: Files on file shares (volumes not located on Disk 0) are backed up as follows: o First, TSM copies all your files and keeps that copy of each file until you modify or delete Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 14 Maintenance Security it. o The previous 13 versions of your files are each kept for 30 days from the day a new version is created. o When you delete a file up to 13 versions are kept for 30 days, as above, and the last version is kept for 60 days. o Backups are first written to disk and then written to 2 tape pools. One tape pool is stored in Indianapolis and one is kept in Bloomington. Backups of files located on Disk 0 (OS disk) are backed up only at the request of the customer. General Maintenance o When necessary, maintenance to development and pre-production systems will occur on Tuesdays between 12:00 PM and 5:00 PM. Maintenance to production systems will generally occur on any Sunday of the month between 12:00 AM and 8:00 AM. Emergency Maintenance o Emergency maintenance will occur as needed. The first priority will be to prevent service loss or to restore service. Consequently, emergency maintenance may be performed without advance notice to customers. Customers will be notified of the emergency maintenance as soon as possible, before or after the event, as the situation allows. o SMART Administrators are on call 24 hours a day, 7 days a week. During an emergency the administrators will give their best effort to restore service, however, there is no guaranteed response or recovery time during an emergency. System Security Patching Maintenance Activities Linux Systems: o If necessary, security updates to non-production servers will occur every Tuesday and a reboot of the servers will occur between the hours of 5am and 6am on Wednesday. If necessary, security updates to production systems will occur the second Sunday of the month between the hours of 12:00 AM and 8:00 AM. As a matter of standard practice, a system reboot occurs between the hours of 5:00 AM and 6:00 AM during the second Sunday maintenance window. The servers or affected services will be down for only the Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 15 amount of time necessary to update the system files and restart the machine. Windows Systems: o If necessary, security updates to non-production servers will occur on the Wednesday following the second Tuesday of every month between 12:00 PM and 5:00 PM. If necessary, security updates to production severs will occur between 12:00 AM and 8:00 AM the following Sunday. The servers or affected services will be down for only the amount of time necessary for the updates – usually for only a few minutes. Environment Architecture Application Security Activities o Applications hosted within the IU intelligent infrastructure and machine rooms are subject to security scanning by the University Information Security Office. The customer is responsible for requesting and fixing vulnerabilities revealed by a security scan. The customer is strongly encouraged to request application scan prior to the introduction of a system into the production environment and following any major code upgrades or changes. o The University Information Security Office reserves the right to audit the security of any system residing in its facilities, through periodic security scans. Per Policy IT-12 “Proactively seek out and apply vendor-supplied fixes necessary to repair security vulnerabilities, within a timeframe commensurate with the level of risk (i.e., within 24 hours for high-risk, with 48 hours for medium-risk, and within 72 hours for low-risk).” Application Security Compromise o If a customer system experiences a security compromise, SMART Administrators will immediately remove the system from the network and notify the customer. Network access to the system will not be restored until the customer and SMART administrators have resolved the situation and the resolution has been certified by the University Information Security Office. Minimally, SMART customers are required to have one pre-production environment and one production environment of identical configurations. Customers may also have additional environments, such as ‘development’, ‘sandbox,’ etc. at their discretion. Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 16 SMART Administration Service Issue Notification Guidelines When problems arise, customers should contact SMART Administrators as follows: Normal Issue Notification Protocol System Administration Support Database Administration Support During normal business hours, 8:00 AM – 4:00 PM, service or problem requests should be submitted via the web at: ESA Service Request During normal business hours, 8:00 AM – 4:00 PM, service or problem requests should be submitted via the web at: DBA Service Request The description of the request should contain the following information: The description of the request should contain the following information: Critical Issue Identify that the virtual server is part of the II SMART Environment. Include the name of the virtual server and the application. The name, email address, and phone number of who to contact during troubleshooting. A description of the problem, including details such as: The complete URL to the problem page. The UNC path to the share containing the file with the problem. The server and database name if the problem is a database connection. At any time, in the event of a critical Enterprise Infrastructure Smart Administration Services – FY 2013-2014 A description of the problem, including details such as: The complete URL to the problem page. The UNC path to the share containing the file with the problem. The server and database name if the problem is a database connection. A description of the aberrant behaviors. The name, email address, and phone number of who to contact during troubleshooting. At any time, in the event of a critical production (7/25/13) Page 17 Notification Protocol production issue, a page can be issued by contacting Computer Operations at 812-8559910 and requesting an ESA Page. issue, a page can be issued by contacting Computer Operations at 812-855-9910 and requesting a DBA Page. Customer’s Scope of Responsibilities: When you subscribe to UITS EI SMART services, the customer assumes full responsibility for the installation and security of the applications and data residing within the environment as well as for any end user support. The customer responsibilities and/or requirements include: Customer Responsibility Funding Responsibility Description A fiscal account contact and account number needs to be established prior to the beginning of any services. It is expected that upon any changes to either account number or account contact will be communicated to UITS EI. Designated Contacts The customer must designate at least one individual as contact for all matters related to the use of this service. UITS EI or Data Center Operations will only act upon requests made by or approved by the designated contact. A list of designated customer personnel with contact information (email and phone numbers) will need to be provided to UITS EI. It is expected that these contacts will be updated upon any personnel or responsibility changes. Active Directory Services (ADS) Group An ADS group needs to be requested, created and maintained by the customer. This group will be used to grant user access Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 18 to the enviroment. VPN Group A VPN group needs to be requested and maintained by the customer. This VPN group will be used to grant computer/workstation access to the server(s). Access to the Environment The customer is completely responsible for controlling access to the software application and associated workspace. Data Management Data management, as prescribed by university policies and state and federal laws and regulations in respect to protection of, access to, and confidentiality of institutional or personal data residing on or processed by the system, is the responsibility of the customer. Assure you are not storing the following personal information anywhere in your workspace without approval of University Counsel, Internal Audit, the University Information Security Office, the University Information Policy Office and the University Data Stewards. o o o o o o Enterprise Infrastructure Smart Administration Services – FY 2013-2014 Social Security Numbers Credit card numbers Financial account numbers Debit card numbers Security codes, access codes and passwords Driver’s license numbers (7/25/13) Page 19 o State identification card numbers o HIPAA Regulated Data Application Installation & Maintenance The customer is responsible for the installation and security of the software application. If specific software licensing is necessary to execute the application, the license must be obtained by the customer. Once the application is installed, the customer will be responsible for obtaining and applying security and maintenance updates as necessary. The customer is responsible for the installation and security of any third-party software required by the application. Application End User Support The customer is responsible for providing the application end user support. Application Security The customer is responsible for applying appropriate industries data security practices and responsibilities in accordance with UISO policy. The customer is responsible for having their application scanned by the UISO application scanner. Enterprise Infrastructure Smart Administration Services – FY 2013-2014 (7/25/13) Page 20