Connect. Communicate. Collaborate Installing and Configuring the Click to edit Master title style perfSONAR Services COURSE OBJECTIVES By the end of this course you will be able to: • Describe key perfSONAR services. • Install, configure and deploy the following perfSONAR services: • The Lookup Service • The BWCTL Measurement Point • The SSH / Telnet Measurement Point • The RRD Measurement Archive • The SQL Measurement Archive • Identify how to interact with the Service Desk. 2 THE PERFSONAR ADMINISTRATION GUIDE Please note that these slides do not contain detailed instructions about how to install the perfSONAR services. For detailed instructions, please refer to the perfSONAR Administration Guide. 3 COURSE OUTLINE Module 1 – perfSONAR Technical Overview Module 2 – General Installation Considerations Module 3 – Installing and Configuring the Lookup Service Module 4 – Installing and Configuring the BWCTL MP Module 5 – Installing and Configuring the SSH / Telnet MP Module 6 – The Service Desk Module 7 – Installing and Configuring the RRD MA Module 8 – Carrying out an Installation on Debian Module 9 – Installing and Configuring the SQL MA Module 10 – Feedback on perfSONAR Installation and Configuration 4 Connect. Communicate. Collaborate MODULE 1: perfSONAR TECHNICAL Click to edit Master title style OVERVIEW WHAT IS PERFSONAR? perfSONAR is: • A project consisting of a variety of organisations and individuals • A set of protocols that: • Assume a set of services based on defined roles. • Define their communication syntax and semantics. • Allow anyone to develop an implementation of a service. • A set of code • Service implementations 6 WHAT ARE THE PERFSONAR SERVICES? The perfSONAR services form an interoperable, distributed performance measurement middleware framework. perfSONAR stands for PERformance Service Oriented Network monitoring Architecture. 7 DESIGN OBJECTIVES perfSONAR is designed to be: • Decentralised and Scaleable • Large number of networks and services, large volume of data • Each domain can set its own security policy • Dynamic and ‘Self-Discovering’ • Add and remove components during operation • Components ‘automatically’ become aware of one another • Secure • Will not put participating networks at risk of attack or congest them • Modular • Allows discrete module development 8 THREE-TIER ARCHITECTURE The perfSONAR framework: • Is middleware. • Is distributed between domains. • Facilitates inter-domain performance information sharing. perfSONAR services ‘wrap’ existing measurement tools. perfSONAR Visualization Tools Domain D perfSONAR service perfSONAR service perfSONAR service perfSONAR service perfSONAR service perfSONAR service Measurement Tool Measurement Tool Measurement Store Measurement Tool Measurement Tool Measurement Store Domain C Key Domain A Domain B = perfSONAR protocols 9 WHAT IS A SERVICE? A Service is a tightly defined, independent entity that has a well defined interface and can be accessed directly. 10 ARCHITECTURAL PRINCIPLES Postulate: all measurement systems contain a combination of: • Measurement tools • Data Storage • Security and Policy implementation • Topology information • Visualization Services have been identified that are: • Based on these ‘roles’ • Based on requirement for other functionality such as Command line tools Measurements in file system Router Measurement Database BWCTL Router • Service discovery, resource protection, data formatting 11 THE PERFSONAR SERVICES FRAMEWORK (1) perfSONAR divides measurement system tools into generic ‘families’ or ‘services’. Each service has a protocol. Key Enabling Services = perfSONAR service Authentication Services Lookup Services Resource protectors Measurement Points Transformation Services Measurement Archives Measurement Tools = existing measurement tool or measurement data store = Registration Measurement Stores Performance Data Services Domain 12 THE PERFSONAR SERVICES FRAMEWORK (2) Client Authentication Service Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service 13 THE CLIENT Client Authentication Service Lookup Service Transformation For example the perfSONAR Service Visualisation User Interface Measurement Point Service Measurement Archive Service Resource Protector Service 14 THE LOOKUP SERVICE Client Authentication Service Transformation Registers services including their Service capabilities. Lookup Service Facilitates complex searches. Participates in network of Lookup Measurement Services Point Service Measurement Archive Service Resource Protector Service 15 THE LOOKUP SERVICE (1) Purpose: all other services must register with the lookup service in order to participate in the framework. 16 THE LOOKUP SERVICE (2) Other services (including measurement points) register their existence with a lookup service, by delivering ‘lookup information’: • Location • Type of Service Each domain has an instance of the lookup service • These instances (will) communicate with one another Clients find other services by querying the lookup service. • All the client needs to know is the URL of a Lookup Service 17 THE AUTHENTICATION SERVICE Client Authentication Service Lookup Service Transformation Service Provides authentication for clients and protects privacy. Measurement Point Service Can be federated. Measurement Archive Service Resource Protector Service 18 THE AUTHENTICATION SERVICE Purpose: ensures client-privacy and domain security by using role-based authentication and authorisation. 19 THE MEASUREMENT POINT SERVICE Client Exposes measurement tools and Authentication publishes their data. Service Transformation Service Measurement Point Service Lookup Service Measurement Archive Service Resource Protector Service 20 MEASUREMENT POINTS Measurement Points: • Belong to domains (domain = a network) • Each measurement point implementation maps to a tool that provides one or several metrics • Examples: – One-way-loss – Jitter – TCP throughput – Show commands on routers 21 THE MEASUREMENT ARCHIVE SERVICE Client Authentication Exposes measurement databasesService and file stores. Publishes measurement data Lookup Service Transformation Service Avoids queries to multiple Measurement Point Services Measurement Point Service Measurement Archive Service Resource Protector Service 22 MEASUREMENT ARCHIVES Purpose: expose measurement data held in databases or file systems. • Wrapper for any type of storage mechanism (SQL Databases, RRD files, etc) • Provides access to recent and stored data • Collects information from Measurement Points, Transformation Services or other Measurement Archives. – i.e. it ‘subscribes’ to these other services • Can also write to databases and file systems 23 THE TRANSFORMATION SERVICE Transforms data in a variety of ways (e.g., aggregation, filtering, Authentication correlation). Service For future development. Precise role needs to be defined. Client Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service 24 THE RESOURCE PROTECTOR SERVICE Controls the comsumption of limited resources (e.g. network bandwidth). Client Authentication Service Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service 25 THE PERFSONAR SERVICES FRAMEWORK (3) Each service has a specific function. Each instance of a service belongs to an administrative domain. 26 PROTOCOLS perfSONAR has developed a set of protocols for sharing performance data. These: • Assume the services set out in the framework. • Define their communication syntax (schema) and semantics (business logic). • Allow anyone to develop an implementation of a defined service. • Are compliant with the Global Grid Forum’s Network Measurement Working Group (NM-WG) schema specification. • Are based on XML over SOAP. You can see the protocols as ‘rules and tools’ for participating in the perfSONAR framework. 27 GENERIC SERVICES AND SERVICE IMPLEMENTATIONS Imple Measurement Archive Service ic fu specif lo o t tion: menta added nction ality Implementation: toolsp ecific functionality added Common functionality, common protocol RRD Measurement Archive service implementation Round Robin Database files SQL Measurement Archive service implementation SQL database Imple Measurement Point Service Common functionality, common protocol ty tionali c n u f pecific tool-s : n io t menta added Implementation: toolsp ecific functionality added BWCTL Measurement Point service implementation BWCTL measurement tool SSH / Telnet Measurement Point service implementation SSH / Telnet measurement tool 28 SERVICE IMPLENTATIONS The perfSONAR project has also developed a set of service implementations that use the defined protocols. • Some of these have been developed by JRA1, within the GEANT2 project for the perfSONAR pilot: • The Lookup Service • The BWCTL Measurement Point • The SSH / Telnet Measurement Point • The RRD Measurement Archive • The SQL Measurement Archive 29 DATA COLLECTION, NORMALISATION AND SHARING VIA THE FRAMEWORK Sy da BWCTL Tool perfSONAR SSH / Telnet Measurement Point (Web) Service Existing Measurement Tools Measurement Data SSH / Telnetspecific Code Tool-specific commands Measurement Data BWCTL Toolspecific Code Standardised Schema Data normalization / conversion to XML Data normalization / conversion to XML Data normalization / conversion to XML perfSONAR BWCTL Measurement Point (Web) Service Tool-specific commands Data normalization / conversion to XML St an ax nt Sy da rd ed is is ed rd Standardised Schema perfSONAR Measurement Framework an St nt ax Other perfSONAR Services (e.g. measurement archive) SSH / TELNET 30 FRAMEWORK FACILITATES NORMALISED END-TO-END PERFORMANCE DATA (SIMPLIFIED DEPICTION) Client (e.g. Visualisations Tool) Network 3 Lookup Service RRD Measurement Archive Authentication Service SSH / Telnet Measurement Point Lookup Service SQL Measurement Archive Authentication Service Network 4 SSH / Telnet Measurement Point Data Data Network 1 Router Network 2 Router 31 THE PERFSONAR PILOT: MEASUREMENT AND ARCHIVING SERVICES Network 2 RRD Measurement Archive service implementation SQL Measurement Archive service implementation BWCTL Measurement Point service implementation SSH / Telnet Measurement Archive service implementation Round Robin Database files link utilisation data SQL database utilisation data and path status BWCTL measurement tool - available bandwidth SSH / Telnet measurement tool - router commands Network 4 Network 3 Network 1 Network 5 32 THE PERFSONAR PILOT Key Enabling Services Authentication Services Lookup Services Service implementations included in pilot Resource protectors Partial service implementation included in pilot BWCTL & SSH / Telnet Measurement Points Transformation Services Measurement Tools RRD & SQL Measurement Archives Measurement Stores Performance Data Services Service implementation not included in pilot Existing measurement tools / data stores Domain 33 THE ROADMAP Enhanced Lookup Service: • Lookup Services (ideally one per domain) peer with one another for increased ease of data discovery. Authentication Service: • Collaborating with JRA5 and Edugain to produce an implementation of the service. • A data subscriber (client) and a data producer (server) can communicate directly across domains provided that they are authorised to do so. Further service implementations using Netflow information or packet capture features 34 TECHNICAL OVERVIEW: SUMMARY perfSONAR will provide an infrastructure to: • Locate data sources • Authenticate and authorise clients • Protect resources and ration their usage • Retrieve, normalise, transform and share data • Only a partial infrastructure is implemented in the pilot perfSONAR is a flexible and open framework: • perfSONAR services can ‘wrap around’ existing data collection tools 35 THE PERFSONAR SERVICES FRAMEWORK - RECAP How do we expose measurement tools? • Measurement point services How do we expose measurement databases and file stores? • Measurement archive services How do we transform data (aggregate, correlate, filter etc.)? • Transformation services How do we locate all these services and their capabilities? • Lookup Services How do we protect resources? • Resource Protection Services How do we ensure a client is allowed to access a service? • Authorisation and Authentication Services 36 MODULE 2 – GENERAL INSTALLATION CONSIDERATIONS GENERAL INSTALLATION PREREQUISITES (1) All of the perfSONAR services require the following: • Recommended operating system: Redhat Linux / Fedora. • You can install on other platforms, but perfSONAR has not been tested on these • Installations on Windows are not supported • ‘wget’ command must be available on the OS – Required by installers to download software • Perl module: LWP • Perl version 5.6.1 or higher These prerequisites must be manually installed before you begin installation of the perfSONAR services. 38 GENERAL INSTALLATION PREREQUISITES (2) All of the perfSONAR services except for the BWCTL Measurement Point (a non-Java application) require the following: • Java Developers’ Kit (JDK) version 1.5 or higher • Already installed for you on the training server • Note that the RRD MA must have JDK version 1.5 (not any other) • Apache Ant 1.6.x • Tomcat application server – Jakarta Tomcat These prerequisites must be manually installed before you begin installation of the perfSONAR services. – Note: Tomcat can be automatically installed by the bundle installer, but it recommended that you download Tomcat from the Apache web-site and manually install it before running the bundle installer. 39 OTHER INSTALLATION PREREQUISITES In addition to the General Installation prerequisites, each service has one or more prerequisites that are specific to it. Some must be installed manually by you before you begin to install a service: • These are listed in subsequent course modules and are also documented in the Installation Manual. Some can be installed automatically by the perfSONAR Bundle Installer: • These are documented in the perfSONAR Administration Guide. 40 THE ROLE OF THE ADMINISTRATION GUIDE The perfSONAR Administration Guide: • Lists all pre-requisite software. • Provides step-by-step instructions explaining how to install the perfSONAR services. • Will be used extensively during this course. • Will be handed out to you in hard copy. • Should be used as your guide when you are installing the services on your own servers. During the exercises, please point out any inconsistencies or errors in the guide to your trainer. 41 THE ROLE OF THE SERVICE DESK The Service Desk is being set up as a single point of contact for all issues relating to perfSONAR installation, configuration and use. More information about the service desk is provided in module 6 of the course. 42 INSTALLATION STEPS To install the perfSONAR services: • Unzip and untar the bundle installer (perfSONAR-2.0.tar.gz) • Empty the CLASSPATH variable. • Execute the bundle installer • Follow the on-screen dialogue For detailed instructions, refer to the perfSONAR Administration Guide. 43 HOW DOES THE BUNDLE INSTALLER WORK? • The bundle installer will ask you whether you want to: • Install a new service • Modify or test an existing service • Give feedback to the perfSONAR team • If you choose to install a new service: • The installer will ask you to choose from the six available services • It will then: – Automatically download the appropriate installation files and execute the installation steps in sequence – Ask you ‘interactive questions’ about the installation 44 CONFIGURING THE SERVICES – STITCHING Before they can be used, several of the perfSONAR service implementations require a type of configuration known as ‘stitching’. Stitching: • Is the process of configuring metadata for your service. • Metadata is data that describes other data. – E.g. data units, interface name, direction of traffic etc. • Usually involves the creation of a ‘metadata configuration file’. 45 STITCHING IN CONTEXT (1) POS-6/0 router rm rfo e P Instance of RRD Measurement Archive Service fo Per an r ma Perfor m Pe rfo rm d ce ata MRTG RRD file MRTG d n ce ance ata data GEO/0 router Cricket an ce In Out da RRD file ta Cricke t In Out Stitching allows a service to understand the data it is dealing with. Examples: Which router? Inbound or outbound traffic? What data units? Etc. 46 STITCHING IN CONTEXT (2) mySQL or PostgreSQL database Instance of SQL Measurement Archive Service Path Status data Router B NonperfSONAR application Router D Router A Path Status data perfSONAR Measurement Point Router C Stitching allows a service to understand the data it is dealing with. Examples: Which link? Which interfaces? Location of interfaces? Etc. 47 THE ADVANTAGES OF STITCHING Ultimately, stitched metadata tells the perfSONAR framework precisely what kind of data your service sends or can receive. Why this approach? • The ability to perform stitching makes service implementations flexible. • E.g. the RRD Measurement Archive can handle data from RRD files in different networks that are structured in different ways. • Stitching allows service implementations to deal with multiple ‘flavours’ of the same kind of data. • E.g. a single instance of the SQL Measurement Archive service can deal with both inbound and outbound traffic since you can mark each data source as providing information about either inbound or outbound traffic. 48 OTHER CONSIDERATIONS If you are installing multiple services in the same Tomcat instance, using the same eXist XML database (as in training): • Each service requiring XML database access must have a different username and password • You must ensure that each service is allocated a unique name for its own collection • Must be manually created for the Lookup service Ensure that you only install one instance of ant on your machine • Putting multiple ant installations in the same path causes problems 49 BACKUPS Once you have successfully installed and configured a service, back it up. • Backup the ‘webapps’ directory related to the service • Located by default within your perfSONAR bundle directory • Can be used to restore the service • Backup your metadata configuration files 50 ACTIVITIES (1) Installing Tomcat • Install • Change ports in Tomcat’s .conf file (Two ports - must be unique for each participant) • Start up Installing Ant • Install • Change path variable in bash profile to include Ant location • Start up 51 ACTIVITIES (2) Notes: • Java Developers’ Kit (JDK) is already installed on the server • A UNIX account has been created for each of you • In your UNIX account’s home directory you will find: • Tomcat installation files • Ant installation files • The perfSONAR bundle installer 52 MODULE 3 – INSTALLING AND CONFIGURING THE LOOKUP SERVICE THE LOOKUP SERVICE (1) All other services must register with the lookup service in order to participate in the framework. 54 THE LOOKUP SERVICE (2) Other services (including measurement points) register their existence with the lookup service, by delivering ‘lookup information’: • Location (URL) • Type of Service • Service-specific information • For example an Measurement Point will tell the Lookup Service what kind of measurements it can take Clients find other services by querying the lookup service. • All the client needs to know is the URL of the Lookup Service 55 THE LOOKUP SERVICE (3) The LS keeps Lookup Information in Lookup Storage (LSSTORE), an XML database. • Format of information described in NMWG schema. 56 THE LOOKUP SERVICE (4) 57 LOOKUP MESSAGES Other services can interact with the lookup service to: • Register with it. • De-register. • Update registration details. • Keep-alive registration details. • Query the lookup store. 58 LOOKUP SERVICE-SPECIFIC PREREQUISITES The following prerequisites are necessary for Lookup Service installation: • Java Developer’s Kit (JDK) version 1.5 or later • Apache Ant 1.6.x • eXist XML Database version 1.0.1 or 1.1.1 • Can re-use an existing eXist XML database • Install new eXist XML database as a webapp via Tomcat • Some configuration is required after installation • You must set your JAVA_HOME environmental variable to your Java directory 59 THREE-STAGE INSTALLATION The Installer will work through three stages: • Pre-Install • Establishes information required for the rest of the process – E.g. the application server port and service directory • Configure • Set important parameters: – E.g. Service type, name, URL, XML database username and password etc. • Deploy • Deploys the service on the application server 60 HINTS AND TIPS • Change the password for the eXist database admin user after the service installation is complete • Prevents use of the eXist client application to alter data • Not necessary in training, but important in a ‘live’ context • If you change Tomcat’s default port, ensure you configure the Lookup Service to use the amended port number. • For any installation on Linux: • Recommended that you download Tomcat from the Apache web-site • Recommended that you do not use the version of Tomcat supplied with the distribution 61 TESTING In order to find out whether the service has been successfully set up, perform the following test: • ant client-echo • This contains an XML Database connectivity test Subsequently, you can perform the following tests: • ant client-register • ant client-query • ant client-deregister Note that test results are not printed on the screen, but are put into an XML file. 62 INTERACTION WITH THE SERVICE DESK (1) If you encounter problems during or after installation, contact the Service Desk with the following information: • A description of the problem • Software versions for the following: • Lookup Service • eXist XML DB • Java • Ant – Continued on next slide… 63 INTERACTION WITH THE SERVICE DESK (2) If you encounter problems during deployment of the service or runtime please give the service desk the following information: • Log files (especially sonar.log) • Configuration files • service.properties • const.properties • log4j.properties • components.properties • Request / response files (if run) • The result of “ant client-echo” (if run) 64 LOOKUP SERVICE INSTALLATION OVERVIEW Check that all of the required prerequisite software is installed. Check that you have the right version of each prerequisite. Download and install prerequisite software if necessary. • Java • Ant • eXist XML database • Tomcat application Server Install the Lookup Service • Follow the instructions in the perfSONAR Administration Guide 5) Test your installation 65 ACTIVITIES Lookup service Installation and Configuration • Demonstration • Exercise • Dependency Checks – are all of the Required Prerequisites installed? • Installing Manual Prerequisites – the eXist XML Database • Preparing for the Installation • Carrying out the Installation • Testing the Installation • Feedback 66 MODULE 4 – INSTALLING AND CONFIGURING THE BWCTL MEASUREMENT POINT MEASUREMENT POINTS (1) Purpose: expose measurement tools to provide three types of performance measurement data: • Active measurements • Passive measurements • Network state information 68 MEASUREMENT POINTS (2) Measurement Points: • Belong to domains (domain = a network) • Each measurement point implementation maps to a tool that provides a specific metric • Examples: – Active delay – One-way-loss – Jitter – Available bandwidth 69 BWCTL MP SPECIFICS The BWCTL MP: • Is implemented as typical UNIX daemon. • Is a wrapper for the BWCTL tool. • Receives client requests to trigger BWCTL tests. • Sends these requests to the BWCTL tool, which executes them. • Returns test results to the client. • Implementation could be adapted for use with other command line tools • Change parsing of input and output parameters as necessary 70 THE BWCTL MP: USER-ADVANTAGES Using the BWCTL Measurement point offers you two major advantages: • You don’t have to be logged on to the machine where the BWCTL tool is installed • You don’t have to configure BWCTL keys Additionally, since the BWCTL MP ‘plugs-in’ to the perfSONAR framework, it makes measurements available to the perfSONAR community, subject to local security policies. 71 FOUR STAGE INSTALLATION The Installer will work through four stages: • Pre-Install • Establishes information required for the rest of the process – E.g. the installation directory • Configure • Confirms which necessary Perl modules already exist on the server and which need to be installed. • Deploy • Asks for the user and group ID the service will be started as • Test • Offers the opportunity to test your installation 72 INSTALLATION PREREQUISITES In addition to the general prerequisites required for all services, the BWCTL MP requires: • BWCTL Tool version 1.1b or higher. • Iperf Tool version 2.0.2 or higher. 73 HINTS AND TIPS If required Perl Modules are not found in the local Perl installation, then they will be installed into the installation directory of the BWCTL MP. • You could choose to use a system tool Check your network connectivity! 74 TESTING You can use the supplied test script to check whether the service daemon and the init script have been correctly installed. • The test script will only work if the init script is installed properly. • You need root privileges for this. 75 ISSUES WHEN INSTALLING UNDER DEBIAN The included init script has been designed for installation on Fedora Linux, but should also work on other LSB compliant systems. It is known NOT to work on Debian 3.1 and earlier. It was not tested on Debian 4.0. The test script will not work out of the box on Debian, because it depends on the init script. 76 ADDING MEASUREMENTS WITH OTHER TOOLS The BWCTL MP source code is structured in a modular fashion. This approach makes it relatively easy to link further command-line measurement tools to the service. • OWAMP functionality has already been added. 77 INTERACTION WITH THE SERVICE DESK When things go wrong: • Typically you will need to supply the service desk with the following information: • Which operating system you are using, including the version • Which version of Perl you are using • Which version of BWCTL / IPERF you are using • The installation stage at which you began to experience problems • The log output of the installation scripts • Whether there is anything unusual about the way in which your system is configured 78 BWCTL MP INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. • BWCTL Tool • IPERF • Perl 4) Install the perfSONAR BWCTL Measurement Point • Follow the instructions in the perfSONAR Administration Guide 5) Test your installation 79 ACTIVITIES BWCTL Measurement Point Installation and Configuration • Demonstration • Exercise: • Dependency Checks – are all of the Required Prerequisites Installed? • Preparing for the Installation • Carrying out the Installation • Testing the Installation • Feedback 80 MODULE 5 – INSTALLING AND CONFIGURING THE SSH / TELNET MEASUREMENT POINT SSH / TELNET MP SPECIFICS (1) The SSH / Telnet Measurement Point acts as a central contact point inside a network. It: • Is able to retrieve information from routers • uses standard protocols such as SSH or Telnet • Issues ‘show like’ commands – Discovers configuration information: routing tables, interface configuration etc. • Can only issue pre-configured commands and parameters • Prevents usage for malicious attacks 82 SSH / TELNET MP SPECIFICS (2) The SSH / Telnet MP is the back-end of the Looking Glass user interface • Similar to existing Looking Glasses on the web (traceroute.org). 83 SSH / TELNET MP: AVAILABLE COMMANDS A client can issue the SSH / Telnet MP two types of command: • MetadataKeyRequest: • client asks the MP what its capabilities are • SetupDataRequest: • Client issues a command with its parameters through the MP for execution on a specific device 84 SSH / TELNET MEASUREMENT POINT USE CASES Example use cases: • Retrieval of a routing table entry for a specific network • Traceroute command • Ping What you can do depends upon the commands and parameters that are configured for each instance of the measurement point. 85 SSH / TELNET MP PREREQUISITES The SSH / Telnet MP is a Java application that is deployed using Axis and Apache Tomcat. For communication with routers SSHTools [J2SSH], JSch [JSCH] or Telnet/SSH/Terminal for Java application [JTA] are required. 86 SUPPORTED NETWORK DEVICES • Quagga (Telnet) • Cisco (Telnet) • Cisco (SSH) • Juniper (Telnet) • Juniper (SSH) 87 THREE STAGE INSTALLATION The Installer will work through three stages: • Pre-Install • Establishes information required for the rest of the process – E.g. installation path, Tomcat path • Configure • Set important parameters: – Give path of configuration (stitching) file or create the file interactively • Deploy • Deploys the service on the application server 88 STITCHING For the SSH / TELNET Measurement Point, stitching is the process of defining the available devices and the commands and parameters that users can issue to them through the measurement point. These settings are held in the service.properties file. The service.properties file can be: • Automatically created via a wizard-like script that runs during installation. or: • Manually created and then imported during configuration. 89 AN EXAMPLE OF STITCHING Devices: • Cisco1 (10.10.3.24) • JuniperBerlin (10.10.1.14) Commands: • PING • Ping • 1 parameter • Syntax (reg exp): ^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+(\\/[0-9]{1,2})?$ 90 HINTS AND TIPS (1) To complete the installation and configuration you will need to know: • The routers and other devices that you want to make available via the SSH / Telnet MP. • The commands that you want to make available for these routers and devices. 91 HINTS AND TIPS (2) In order to protect the SSH / Telnet Webservice you should: • Configure the regular expressions that can be used in requests • I.e. configure acceptable commands and parameters etc. • Configure a reasonable access rate for a device. • The access rate is the time between two requests in which no other request can be handled. 92 HINTS AND TIPS (3) For any installation on Linux: • Recommended that you download Tomcat from the Apache web-site • Recommended that you do not use the version of Tomcat supplied with the distribution 93 TESTING Check in browser at the MP’s URL if the service is running. If so, run the ant test command. • creates a SetupDataRequest for every command of every device, and sends it to the MP. • Results coming back are automatically checked. 94 ISSUES WHEN INSTALLING UNDER DEBIAN • No known issues when installing under Debian. • Service has been deployed and tested completely on a Debian setup. 95 INTERACTION WITH THE SERVICE DESK When problems arise during installation, please provide the following information: • Java Version • An URL which we can use to test remotely • The service.properties file 96 SSH / TELNET MP INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. • Java • Ant • Tomcat application Server 4) Install the perfSONAR SSH / Telnet Measurement Point • Follow the instructions in the perfSONAR Administration Guide 5) Test your installation 97 ACTIVITIES SSH / Telnet Measurement Point Installation and Configuration • Demonstration • Exercise • Dependency checks – are all of the Prerequisites Installed? • Preparing for the Installation • Carrying out the Installation • Examining the service.configuration file • Testing the Installation • Feedback 98 MODULE 6 –THE SERVICE DESK THE ROLE OF THE SERVICE DESK The Service Desk is a single Point of Contact for: • The 5 MDM pilot deployers • GEANT2 community NOC and PERT users Through the service desk you can: • Report incidents about the installation, configuration, operation and utilisation of the web services and visualisation tools. • Ask questions: • About the MDM service • About the installation, configuration and operation of perfSONAR web-services and visualisation tools • Raise enhancement Requests 100 THE DUTIES OF THE SERVICE DESK (1) The duties of the service desk will include: • Logging all calls, events and requests. • Acting as the ‘first layer’ of incident management. • Taking overall ownership of incidents • Escalating and re-assigning them until they are resolved • Monitoring the MDM service, the web-services and the visualisation tools. 101 INCIDENT MANAGEMENT EXAMPLE – KNOWN ERRORS Several incident management processes have been defined. This is one example. 102 THE DUTIES OF THE SERVICE DESK (2) The duties of the service desk will include: • Gathering feedback on the service provided. • Generating regular reports on: • Incidents. • Lessons learned. • Missing pieces (continuous improvement). • Offering a managed service for FCCN and GEANT2. 103 SUCCESS FACTORS Critical success factors are: • Well defined and efficient support processes • Trouble ticketing system, CMDB, monitoring tools • Documentation • Training 104 SUPPORTED SOFTWARE The Service Desk will support the: • Installation, configuration and operation use of: • SQL MA, RRD MA, SSH / Telnet MP, BWCTL MP, L2 status MP, LS. • Use and configuration of CNM and E2EMON. • Installation and use of the perfSONAR UI and of the Looking Glass. • Use of: • The Hades MA. • The Hades Visualisation Tool. • Use and operation of Hades monitoring tools. 105 L2 STATUS Please note that the L2 status MA cannot be supported until released. • E2EMon visualisation will be supported at the same time. 106 INFORMATION TO PROVIDE TO THE ASD NREN MDM Contact detail Deployed web-services IP addresses, URL, locations, the GPS installation quotes When web-services are installed, so that the ASD can start monitoring it For equipment shipment When a planned maintenance will affect the MDM service 107 CONTACT DETAILS E-mail: asd@geant2.net Phone: +44 1223 371 380 (available from 18th of June) • In the meanwhile, please call +44 1223 371 3xx 108 MODULE 7 – INSTALLING AND CONFIGURING THE RRD MEASUREMENT ARCHIVE MEASUREMENT ARCHIVES Purpose: Measurement archives expose measurement data held in databases or file systems. They: • Are wrappers for any type of storage mechanism (SQL Databases, RRD files, etc). • Provide access to recent and stored data. • Can also be used to write information to databases and file systems. 110 THE RRD MEASUREMENT ARCHIVE (1) The Round Robin Database (RRD) Measurement Archive is a wrapper for binary files of the RRDTool format. 111 THE RRD MEASUREMENT ARCHIVE (2) The RRD Measurement Archive has two main functions: • Writing and storing measurement data in RRD files • E.g. information from perfSONAR Measurement Points collected as a result of regularly scheduled or on-demand measurements • Publishing measurement data held in RRD files to client applications • E.g. Measurements that have been stored in RRD files by nonperfSONAR applications such as MRTG (Multi Router Traffic Grapher) or Cricket 112 THE RRD MEASUREMENT ARCHIVE IN CONTEXT Key perfSONAR client (e.g. User Interface or Transformation service) RRD Measurement Archive = perfSONAR service or UI = existing measurement tool or measurement data store = performance data flow RRDTool Files Measurement Point (e.g. BWCTL) Measurement Tool (e.g BWCTL) MRTG or Cricket etc. 113 RRD MA INSTALLATION OVERVIEW Follow these steps: • Install the perfSONAR RRD Measurement Archive Service • Generate and populate metadata configuration files • Deploy the Web Service • Test the deployed service to see if it is working 114 THE RRD MA: THREE STAGE INSTALLATION The Installer will work through three stages: • Pre-Install • Establishes information and performs tasks required for the rest of the process – Collects information such as installation path, Tomcat port, eXist admin user password – Compiles the RRD J tool • Configure • Set important parameters: – E.g. name and path of metadata configuration file, location of file-store, whether or not to automatically register with a lookup service • Deploy • Deploys the service on the application server 115 TESTING You can execute a test script that sends a series of test requests to the service. • The metadata configuration file used by the service must be the test metadata configuration file that is supplied with the installation files. • The command to run is “ant test” Once the test is complete, you can analyse the responses generated by the service. • If there are problems, then clear error messages will be displayed in the responses. • If there is no response, then there is a problem. 116 STITCHING FOR THE RRD MEASUREMENT ARCHIVE In the context of a measurement archive, stitching: • Is the process of configuring metadata that underlies the performance data handled by your archive. • Metadata is data that describes other data – E.g. data units, interface name, direction of traffic etc. • Involves the creation of a ‘metadata configuration file’. Ultimately, the metadata configuration file tells the perfSONAR framework what kind of data the archive stores. 117 RRD MA STITCHING IN CONTEXT POS-6/0 router rm rfo e P Instance of RRD Measurement Archive Service fo Per an r ma Perfor m Pe rfo rm d ce ata MRTG RRD file MRTG d n ce ance ata data GEO/0 router Cricket an ce In Out da RRD file ta Cricke t In Out Stitching allows a service to understand the data it is dealing with. Examples: Which router? Inbound or outbound traffic? What data units? Etc. 118 THE METADATA CONFIGURATION FILE The Metadata Configuration File: • Is an xml file that will expose information describing your network’s RRD files. • Conforms to NMWG’s XML schema. The default perfSONAR installation provides samples of: • A metadata configuration file • An RRD file The sample metadata configuration file: • Is provided as a template only and should be changed to correctly describe your own RRD archives. 119 STITCHING FOR THE RRD MA – THREE STEP PROCESS Step 1 – Understand the metadata configuration file structure Step 2 – Create your own metadata configuration file Step 3 – Apply your metadata configuration file to your instance 120 STITCHING FOR THE RRD MA – STEP 1 To Understand the metadata configuration file structure you should: • Refer to the metadata configuration file guide • Study the sample metadata configuration file • Gather information about each data source in your RRD files. • A data source usually equates to a measurement of an interface’s traffic in a single direction. • You should create a a metadata ‘chain’ for each data source. • There is no limit to the number of metadata chains you configure. 121 STITCHING FOR THE RRD MA – CHAINS A chain: • Describes measurement data for a single interface, for one direction only • Consists of two connected parts: • ‘Metadata’ – Host Name (DNS entry of the router containing the interface) – IPV4 interface address – Interface’s name – Interface’s description – Traffic direction (in or out) – Authentication realm – Capacity / interface speed • ‘Data’ – Name and path of the RRD file – Data source within the RRD file – Data storage unit (example: bps or Bps) 122 RRD MA METATDATA CHAINS – EXAMPLE OF METADATA 123 RRD MA METATDATA CHAINS – EXAMPLE OF DATA 124 STITCHING FOR THE RRD MA: STEP 2 Create your metadata configuration file • Can be created by manual file editing. • Can be generated using contribution scripts or your own scripts • Contribution scripts are available for some tools – E.g. MRTG • If scripts do not already exist for your tool, you can create your own script based on existing contribution scripts • Manual file creation for a large number of interfaces would be labourintensive and time-consuming • When something changes in your network, update your file • Recommended that you automate this process 125 STITCHING FOR THE RRD MA: STEP 3 Apply your metadata configuration file to your instance • Three possible methods: • Use the eXist XML database web-based User Interface – Recommended method • Use the perfSONAR installer – Not covered in this course; refer to the installation guide for details • Use the service installation scripts – Not covered in this course; refer to the installation guide for details 126 MAKING THE MEASUREMENT ARCHIVE AVAILABLE TO THE VISUALISATION TOOLS A user client application needs to know the address of the Measurement Archive. • This information can be taken from the Lookup Service • Contact Andreas Hanemann at DFN in order to get CNM to use your service. • When the Lookup Service is installed, please notify the perfSONAR UI team. 127 INSTALLATION PREREQUISITES In addition to the general prerequisites required for all services, the RRD MA requires: • RRDTool version 1.2.x • The RRD MA can be installed on any Linux platform 128 HINTS AND TIPS • During the installation, you will need to supply the location of the RRD Tool. • It is better to store metadata configuration information in an XML database than in a text file as this improves performance. • To run the RRD MA on a 64-bit machine, you must be consistent in your use of 32 bit or 64 bit software • i.e. either all software (Java, rrdjtool, rrdtoo libs, rrd files) must be compiled for 32-bit or all software must be compiled for 64-bit. • For any installation on Linux: • Recommended that you download Tomcat from the Apache web-site • Recommended that you do not use the version of Tomcat supplied with the distribution 129 ISSUES WHEN INSTALLING UNDER DEBIAN The librrd2-dev package must be installed. 130 INTERACTION WITH THE SERVICE DESK When problems arise during installation, please provide the following information as a minimum: • Log files • Configuration files • Request / response messages 131 RRD MA INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. • Java • Ant • RRDTool • eXist XML database • Tomcat application Server 4) Install the perfSONAR RRD Measurement Archive • Follow the instructions in the perfSONAR Administration Guide 5) Test your installation 132 ACTIVITIES RRD Measurement Archive Installation and Configuration • Demonstration • Exercise • Dependency Checks – are all of the Required Prerequisites Installed? • Preparing for the Installation • Carrying out the Installation • Testing the Installation • Stitching – Editing the RRD Metadata Configuration File • Testing the Results of Stitching • Feedback 133 MODULE 8 – CARRYING OUT AN INSTALLATION ON DEBIAN OVERVIEW OF DEBIAN • Very popular OS among the Open-Source alternatives • Stability • Powerful package management, Easy upgrade • Huge Community • Comes in 3 flavours • Stable: Integrate robust package. (Not always recent but… Security patched !! AKA ETCH) • Testing: The next Stable version (AKA LENNY) • Unstable: Bleeding edge package (AKA SID) • “Stable” is the way to go • Security • Robust service due to package stability 135 RRD MA INSTALLATION: DEBIAN BASE INSTALLATION • Get the ISO from www.debian.org • Either download the full distribution but … • If possible prefer the NETINSTALL ISO (163 Meg) • NETINSTALL provides up to date packages • Install the minimum distribution (server/standard install) • A rule of thumb for production environment is … • To always use the KISS method • “If you don’t need it, don’t install it” 136 REMINDER: APT, DPKG, LOCATE ARE ALL FRIENDS • apt • Package/Distribution management tool • After “minimum install”, update source.list, chose your favorite mirror and add “non-free” repository • Then … apt-get update; apt-get dist-upgrade • apt-cache search , if you are lost ! • dpkg • -l <pkg_name>: List all packages installed • -L<pkg_name>: List all files part of a package • locate • Locate <file_name> “Where is <file_name>” • Updatedb when to need to locate a file on the file system 137 PERFSONAR BUNDLE INSTALLER / RRD MA DEPENDENCIES • ssh • java • rrdtool and … librrd2-dev • wget • perl • libwww-perl (AKA LWP) • gcc • ant and don’t forget… ant-optional ! • Debian Tomcat or PerfSONAR Tomcat… It’s up to you ! 138 TOMCAT • Debian Tomcat • Disable security manager • Enable TOMCAT but define a TOMCAT security policy • Tomcat from bundle installer • Preferred method • Easy upgrade • Security manager disabled 139 POTENTIAL ISSUES • Environment variable • LD_LIBRARY_PATH No RRD-GRAPH in PerfSONARUI • JAVA_HOME not set TOMCAT won’t start • Enable system wide environment variable • Put it in /etc/profile • export LD_LIBRARY_PATH … • Export JAVA_HOME … • Tomcat refuse to install the services on “packaged Tomcat” • Disable Tomcat security manager or … • Add security policy 140 FINAL TOUCH • Starting PerfSONAR service at system startup • /etc/init.d/rc.local script • Run the service as perfsonar user ! (Avoid root …) • Diagnostic tools • netstat –a | grep LISTEN check that Tomcat is listening at port defined and also when the MA is interrogated • ps –def | grep perfsonar check the process is running 141 LAST BUT NOT LEAST…. SECURITY • PerfSONAR services provide access to sensitive infomation so … • General rules • Permit « anyone» that wants to access the service ONLY • Permit remote administration using SSH from your LAN • Permit ICMP echo request from 194.141.0.9 • Available tools on RRD-MA host • TCPD (AKA TCP wrapper) • IPTABLES • Tripwire • Available tools on the local LAN • Router access-list • Switch VACL/Private VLAN • Etc. 142 QUESTIONS? Comments and suggestions are of course welcome ! 143 MODULE 9 – INSTALLING AND CONFIGURING THE SQL MEASUREMENT ARCHIVE THE SQL MEASUREMENT ARCHIVE (1) The SQL Measurement Archive is a wrapper that allows perfSONAR to access data stored in a database. • utilisation and path status are currently supported • Supports MySQL or PostgreSQL databases • Theoretically other databases can be used, but these have not been tested 145 THE SQL MEASUREMENT ARCHIVE (2) The SQL Measurement Archive has two main functions: • Publishing measurement data from a database to client applications • Writing and storing measurement data to a database When installing, you can either: • Setup a new database • Schema setup scripts included in installation • Use an existing database • Configure the SQL MA to work with your existing database 146 STITCHING FOR THE SQL MEASUREMENT ARCHIVE In the context of a measurement archive, stitching: • Is the process of configuring metadata that underlies the performance data handled by your archive. • Metadata is data that describes other data – E.g. data units, interface name, direction of traffic etc. • Involves the creation of a ‘metadata configuration file’. Ultimately, the metadata configuration file tells the perfSONAR framework what kind of data the archive stores. 147 SQL MA STITCHING IN CONTEXT mySQL or PostgreSQL database Instance of SQL Measurement Archive Service Path Status data Router B NonperfSONAR application Router D Router A Path Status data perfSONAR Measurement Point Router C Stitching allows a service to understand the data it is dealing with. Examples: Which link? Which interfaces? Location of interfaces? Etc. 148 THE METADATA CONFIGURATION FILE Your Metadata Configuration File will expose information describing your network’s: • Interfaces and their utilisation data. • Links and link status The default perfSONAR installation provides samples of: • A metadata configuration file • A SQL database The sample metadata configuration file: • Is provided as a template only. • Should be changed to correctly describe the your own SQL database. 149 STITCHING FOR THE SQL MA – THREE STEP PROCESS Step 1 – Understand the metadata configuration file structure Step 2 – Create your own metadata configuration file Step 3 – Apply your metadata configuration file to your instance 150 STITCHING FOR THE SQL MA – STEP 1 To understand the metadata configuration file structure you should refer to: • The metadata configuration file guide • The sample metadata configuration file There are two types of SQL MA metadata: • Metadata describing interface utilisation • Similar to the RRD MA’s metadata • Metadata describing path status • Unique to the SQL MA 151 SQL MA ARCHITECTURE 152 UNDERSTANDING PATH STATUS METADATA SQL MA path status metadata can be subdivided into: • Node metadata • Link metadata First create metadata describing each node. Then create metadata describing each link. 153 LINK METADATA AND NODE METADATA When creating link and node metadata: • You need to create a metadata chain for each link that you want to collect data about. • Within the link’s metadata chain, associate two nodes with the link. – I.e. the start and demarcation points of the link • The same node can be referred to in the metadata of multiple links. 154 STITCHING – METADATA FOR NODES AND LINKS Node metadata includes: • Node ID and name • Node’s country, city and institution • Node’s latitude and longitude Link Metadata includes: • Link name and global name • Name of related nodes • Roles of related nodes: • I.e. end point or demarcation point 155 STITCHING – PATH STATUS CHAINS A path status chain describes measurement data for a single link and refers to multiple nodes. Consists of two connected parts: • ‘Metadata’ • For Nodes • For Links • ‘Data’ • Name of relational database configuration file • Path ID 156 SIMPLIFIED LINK PATH METADATA EXAMPLE 157 NODE METADATA EXAMPLE 158 LINK CHAINS – AN EXAMPLE 159 LINK CHAIN DATA – AN EXAMPLE 160 SQL MA ARCHITECTURE 161 STITCHING FOR THE SQL MA– INTERFACE UTILISATION CHAINS Interface utilisation chains: • Describes measurement data for a single interface, for one direction only. • Are the same as RRD Measurement Archive link chains • However the data is different because the storage type is different • Consist of two connected parts: • ‘Metadata’ – Host Name (DNS entry of the router containing the interface) – IPV4 interface address – Interface’s name – Interface’s description – Traffic direction (in or out) – Authentication realm – Capacity / interface speed • ‘Data’ – Name of relational database configuration file 162 STITCHING FOR THE SQL MA: STEP 2 Create your own metadata configuration file. • Can be created by manual file editing. • Generate your file using contribution scripts or your own scripts. • When something changes in your network, update your file. • Recommended that you automate the process 163 STITCHING FOR THE SQL MA: STEP 3 Apply your metadata configuration file to your instance • Three methods: • Use the eXist XML database web-based User Interface – Recommended method • Use the perfSONAR installer – Not covered in this course; refer to the installation guide for details • Use the service installation scripts – Not covered in this course; refer to the installation guide for details 164 THREE STAGE INSTALLATION OF THE SQL MA The Installer will work through three stages for the SQL MA: • Pre-Install • Establishes information required for the rest of the process – E.g. installation path, Tomcat port, eXist admin user password • Configure • Set important parameters: – E.g. setup the database – username and password for database – name and path of metadata configuration file – location of file-store – whether or not to automatically register with a lookup service • Deploy • Deploys the service on the application server 165 MAKING THE MEASUREMENT ARCHIVE AVAILABLE TO THE VISUALISATION TOOLS A user client app needs to know the address of MA (can be taken from LS). 166 TESTING You can execute a test script that sends a series of test requests to the service. • The metadata configuration file used by the service must be the test metadata configuration file that is supplied with the installation files. • The command to run is “ant test” Once the test is complete, you should analyse the responses generated by the service. 167 INSTALLATION PREREQUISITES In addition to the general prerequisites required for all services, the SQL MA requires: • One of the following relational databases: • Mysql version 5.0 Or • PostgreSQL version 8.x 168 HINTS AND TIPS The SQL MA can be installed on any Linux platform. Metadata Configuration must be held in XML database. • For any installation on Linux: • Recommended that you download Tomcat from the Apache web-site • Recommended that you do not use the version of Tomcat supplied with the distribution 169 INTERACTION WITH THE SERVICE DESK When problems arise during installation, please provide the following information as a minimum: • Log files • Configuration files • Request / response messages 170 SQL MA INSTALLATION OVERVIEW 1) Check that all of the required prerequisite software is installed. 2) Check that you have the right version of each prerequisite. 3) Download and install prerequisite software if necessary. • Java • Ant • MySQL (or postgreSQL) database • eXist XML database • Tomcat application Server 4) Install the perfSONAR SQL Measurement Archive • Follow the instructions in the perfSONAR Administration Guide 5) Test your installation 171 ACTIVITIES SQL Measurement Archive Installation and Configuration • Demonstration • Exercise • Dependency Checks – are all of the Required Prerequisites Installed? • Preparing for the Installation • Carrying out the Installation • Testing the Installation • Stitching – Editing the SQL Metadata Configuration File • Testing the Results of Stitching • Feedback 172 MODULE 10 – FEEDBACK ON PERFSONAR INSTALLATION AND CONFIGURATION ACTIVITIES Please use the forms provided to give us feedback about the perfSONAR installation and configuration process. • This asks for your feedback about the installation process and the associated software, not about the training course • You will be asked for feedback about the training course separately • For both the BWCTL MP and the JAVA Services • Provide three positive points and three things to enhance about the installation and configuration process – No inter-personal issues – No generalities. Be specific. – Make suggestions about how to improve • Provide answers to quantitative questions 174 FOR MORE INFORMATION • www.geant2.net • www.dante.net • For latest news and factsheets http://www.geant2.net/media • For research activities http://www.geant2.net/research • The perfSONAR FAQ and mailing list can be found at www.perfsonar.net. • The WIKI at wiki.perfsonar.net is also a valuable source of information. 175 RECAP OF COURSE OBJECTIVES By the end of this course you will be able to: • Describe key perfSONAR services. • Install, configure and deploy the following perfSONAR services: • The Lookup Service • The BWCTL Measurement Point • The SSH / Telnet Measurement Point • The RRD Measurement Archive • The SQL Measurement Archive • Identify how to interact with the Service Desk. 176