FELIX: FEderated Test-beds for Large-scale Infrastructure eXperiments Bartosz Belter bartosz.belter@man.poznan.pl Focused Technical Workshop: International OpenFlow/SDN Testbeds Florida International University 31 March – 2 April 2015 FELIX in a nutshell Facts Objectives • EC (EU) & NICT (JP) collaborative project - Collaboration between European and Japanese partners • Started in April 2013. To run till March 2016 • To createa large-scale testbed (experimental infrastructure), federated across two continents • To define a common software architecture for testbeds EU coordinator JP coordinator NL BE PL DE JP IT ES Federating existing SDN-based testbeds in EU and JP Existing testbeds • OFELIA (OpenFlow in Europe: Linking Infrastructure and Applications) First large-scale OpenFlow-enabled experimental facility in Europe), continuously evolving through other FIRE projects (FIBRE, Fed4FIRE, ALIEN, etc.) www.fp7-ofelia.eu • RISE (Research Infrastructure for large-Scale network Experiments) OpenFlow testbed over JGN-X (the largest testbed network in Japan), with wide-area coverage from US West coast to Southeast Asia www.jgn.nict.go.jp/rise/english Why to federate? • To increase mutual benefits of European and Japanese researchers by creating more complex environments for specialized research and experiments • This opens up new opportunities for experiments due to geographical dispersion of testbeds FELIX Use Cases – Towards a FELIX Architecture Two major clusters of use cases Based on different applicability area and stakeholders, use cases are divided into: • Data Domain use cases Optimize the use of the interconnectivity between testbeds to realize data transfer 1. Data on Demand – delivery of distributed data by setting data flows 2. Pre-processing and delivery of nearly real-time [satellite] data 3. High quality media transmission over long-distance networks • Infrastructure Domain use cases Optimize the use of the infrastructure as a whole, including the migration of entire data processing workloads 4. Data mobility service by SDN technologies 5. Follow the Sun/Moon – green energy in data centers 6. Disaster recovery by migrating IaaS to a remote data center FELIX Data Domain – Use Case #1 Data on Demand – delivery of distributed data by setting data flows over the network A user wants to run an algorithm at a given site (Data Processor) over different data sets contained in data storage locations around the world. Research question. How can we efficiently retrieve large amounts of data stored along different and distributed sites and process it on demand? • Key entity: Data Processor (algorithms could be proprietary) to control the testbed network elements and define the flow paths • Key solution aspect: Dynamic infrastructure interconnection and reconfiguration through network resource managers to set up NSI and OpenFlow paths FELIX Data Domain – Use Case #2 Pre-processing and delivery of nearly real-time [satellite] data to distant locations Satellite sources generate huge amounts of nearly real-time data, later to be transmitted to a number of distant locations interconnected by transit networks. Research question. Can we reduce the size of data to be delivered across the transit network to improve the overall performance? • Key entity: FELIX middleware to allocate the proper computing and caching resources at data source and destination(s) ends, also to configure the network segments on-demand • Key solution aspect: Coordinated control of computing and network resources through resource orchestration and the involved resource managers FELIX Data Domain – Use Case #3 High Quality Media Transmission over long-distance networks The federation of SDN testbeds interconnected through NSI-enabled domains opens up opportunities for testing media streaming over existing network technologies in backbone networks (DWDM, MPLS, etc.) in conjunction with an OpenFlow control plane deployed in the testbeds. Research question. Can we determine the behavior of the transmission mechanisms when streaming high resolution media content at very long distances? • Key entities: Experiment control and performance monitoring • Key solution aspect: NSI, OpenFlow control and monitoring for automatic adjustment of connection path(s) through path parameters, depending on media streams and QoS & QoE Experimenter Domain A Media Consumers (End users) Domain B Testbed operator Domain C Media provider FELIX Infrastructure Domain – Use Case #1 Data Mobility Service by SDN Technologies (Inter-Cloud use case) A user of a service provided by a cloud system moves to a remote location (e.g. due to a business trip). Research question. Can the cloud system monitor performance and “move” selected data closer to the remote location? • Key entity: User experience monitoring and manager • Key solution aspects: Monitoring (detection of the “closest” location that guarantees required service quality, availability & performance), NSI and OpenFlow managers (configure paths and flows), resource management (migration of user data, services and applications) FELIX Infrastructure Domain – Use Case #2 Follow the Sun/Moon Some considerations on energy consumption: • Internet usage curves following a similar daily pattern around the world • Cooling is reported to amount up to 50% of the total energy bill of data centers • Prices of renewable energy may strongly depend on the availability of wind and solar energy Research question. How can we move the compute workflow to the nearest & “greenest” data center available in a federation? • Key entity: Workload control and management • Key solution aspects: Monitoring (detect energy cost for operation), OpenFlow & NSI managers for dynamic, on-demand, end-to-end network connections and resource management for (re-)provisioning FELIX Infrastructure Domain – Use Case #3 Disaster recovery by migrating IaaS to a remote data center After some disaster occurs, it will be highly beneficial to immediately continue the service at a remote site,even if the data is slightly obsolete – up to hours or days. BCP: Business continuity is sometimes more important than data integrity. Research question. Can the entire IaaS platform (cluster of VMs & management software) be migrated to remote, differently configured data centers for business continuity? • Key entity: Hardware as a Service (HaaS) coordination software to configure and provide virtual resources by using technologies such as KVM/XEN, OpenFlow, etc. • Key solution aspects: NSI managers to configure the transit links, resource management for migration FELIX Use Cases – Recap Why use cases? • To use as a guide for the architecture definition - Translate into requirements - Help to identify architectural components • To validate the features of the testbed More details? See paper & deliverable D2.1 www.ict-felix.eu What shall FELIX provide to networking experimenters? • Access to a large SDN experimental facility between locations from Europe and Japan • Applicability of similar use cases What shall FELIX architecture provide to testbed facility administrators? • Federation with other GENI-enabled testbeds • Easier deployment of new internal GENI-based resource managers • Improved access control and resource monitoring The FELIX Architecture (1) • • • The slice concept is adopted in FELIX – Experimental facilities to be provided dynamically on top of the FELIX physical infrastructure (federated testbeds). All the experimental facilities will be controlled programmatically – Facilities are composed of computing and network resources (CR and NR) belonging to distributed SDN islands in FELIX infrastructure – Orchestrate resources in a multi-domain environment – In a slice, facilities are interconnected via TNS-controlled domains (transit network). User has access and control of a slice provided. The FELIX Architecture (2) Request configuration of slice(s) • The FELIX Space provides users with slices for their own use. Users request slices to an RO. – RO: Resource Orchestrator – RM: Resource Manager – PHY RES: physical resources (testbed) • The User Space consists of any tools and applications that a user wants to deploy to control a slice or execute particular operations Users Manage slice Control slice Resource controllers in User Space may communicate with FELIX Space components The FELIX Architecture (3) KEY FEATURES Resource Orchestration •Orchestration of various virtualized resources (compute, network and storage) provided by multiple domains Domain Resource Management •Coordination of various resources provided by heterogeneous resource management systems within a domain Resource Allocation Planning •Consider both computing and network resources, user and resource administrator aspects, such as, for instance, cost, energy consumption and load balancing Provisioning •A virtual flat environment, just like a dedicated cluster, using dynamic resource information Authentication and Authorization •All actions are performed (only) by authenticated actors •Ensure all actors are who they claim to be; keep track of all authenticated action Monitoring •Information from multiple domains is aggregated and provided to the user User Access/GUI •Allow for easy interaction between the experimenter and the FELIX testbed federation Inter-domain networking through NSI RO: Resource Orchestrator, MRO: Master RO, VM: Virtual Machine, AM: Aggregate Manager, RM: Resource Manager, SE-RM: Stitching Entity-RM, TN-RM: Transit Network-RM Relevant Standards for Interoperability and Portability OpenGridForum (OGF) http://www.ogf.org – Network Services Interface Workign Group (NSI-WG) • • • An interface to request a multi-domain dynamic network service Being currently standardized in OGF To be deployed by R&E networks in Europe, Asia and US In FELIX: A means to achieve inter-domain connectivity for SDN Islands, contribution to SDN extensions in NSI – Network Modeling Language Working Group (NML-WG) • a standardised network description ontology and schema, facilitating interoperability between different research projects and existing initatives In FELIX: ontology for SDN and NSI resources Internet Research Task Force (IRTF) https://irtf.org/ – Software-Defined Networking Research Group (SDNRG) • • • • Classification of SDN models (Definitions, Taxonomies, Relationship to work ongoing in the IETF and other SDOs) SDN model scalability and applicability Multi-layer programability and feedback control systems Network description languages, abstractions, interfaces and compilers In FELIX: contribution to multi-domain SDN architectures, interfaces and description lang. Achievements to date & Future plans Architecure • Six project use cases have been identified and described (September 2013) • FELIX architecture has been released (February 2014) Implementation • Prototype implementations of the architectural framework released (January 2015) – Software page: http://www.ict-felix.eu/wp-content/uploads/2014/03/FELIX-D2.1.pdf http://www.ict-felix.eu/?page_id=824 – GitHub: https://github.com/dana-i2cat/felix/wiki • FELIX testbed deployed Experimental validation • FELIX experiments and implementation of project use cases just started (Q1-2015) http://www.ict-felix.eu/wpcontent/uploads/2014/03/FELIX_D2.2_General_Architecture_and_Functional_Blocks.pdf PARTNERS Poznan Supercomputing and Networking Center Poland National Institute of Advanced Industrial Science and Technology Japan Nextworks Italy Fundacio Privada i2CAT, Internet I Innovacio Digital A Catalunya Spain SURFnet bv Netherlands European Center for Information and Communication Technologies Gmbh Germany iMinds VZW Belgium KDDI Japan