An Introduction to OSLC and Linked Data OSLC: Lifecycle integration inspired by the web Objectives and Pre-requisites Objectives: You will gain an understanding of the OSLC community, linked data, and the OSLC specifications – OSLC = Open Services for Lifecycle Collaboration Pre-requisites: none Recommended reading: 1. http://www.linkeddatatools.com/introducing-rdf 2. http://3roundstones.com/linked-data-101/linkeddata-tutorial/ 2 What’s next The OSLC community Linked data and RDF An example The OSLC specifications 3 The Integration Problem Point-to-point Integrations don’t scale Monocultures lock you in Maintenance, management, and change costs go up over time Ongoing and unexpected costs drain resources End-user productivity suffers: Either stuck with the wrong tool, stuck doing manual integration; often stuck doing both Creating new integrations is unpredictable Past choices restrict present action and future vision Integrations consume more of the IT budget: integration failures are the top 2 causes of software project delays* More limited ability to respond to change Constrained by exhausted IT budget and lower productivity 4 * Commissioned study conducted by Forrester Consulting on behalf of IBM. OSLC and Open Community: open collaboration, better integration Identify Scenarios Call it a specification Iterate on working drafts Gain technical consensus, http://open-services.net 5 The Basics: What is OSLC, and why should I care? OSLC is an open community building practical specifications for integrating software Tool Maker • create software using reusable and open assets that will interoperate with other tools both inside and outside your influence providing time and cost savings Tool Manager • reduce the complexity and risk of increasingly complex software infrastructures, and improve the value of software across a broader set of internal and external stakeholders Tool User • choose the best tools for your job and have them interact seamlessly to achieve traceability and visibility with the rest of your organization Systems Integrator • focus energy and resources on higher-value customizations, deliver more business value to your clients, and increase client satisfaction OSLC is beneficial to many stakeholders 6 OSLC’s Simple Solution Users can work seamlessly across their tools Architecture of the Web Standard Interfaces Automation Linked Data “Just Enough” integration Increased reuse Increased traceability Monitoring Decreased maintenance costs Better visibility OSLC is an open and scalable approach to lifecycle integration. It simplifies key integration scenarios across heterogeneous tools 7 OSLC’s Big Picture Tests, Libraries, Samples, Examples, Reference Implementations The Resource for OSLC Implementers LINKED DATA PLATFORM WORKING GROUP Open Services for Lifecycle Collaboration Lifecycle integration inspired by the web Scenario-driven & Solution-oriented Leading choice for strategic integration technology OSLC: 8 Generally applicable: specs available for many domains covering ALM, DevOps, ISM, and PLM Inspired by the web Free to use and share Changing the industry Proven Open Innovative Eclipse Lyo Eclipse project created with the goal of providing tools to enable adoption of OSLC specifications. Content includes Code libraries (Java, Perl, others under development) – Give developers tools to ease OSLC implementations Reference implementations of specifications – Provide a starting point for new integrations Test suites and test reporting. Covers OSLC Core, CM, QM, RM and Asset today. – Accelerate and assess development Samples, tutorials and documentation – Working samples of OSLC integrations with Bugzilla, Excel, Jazz tools and more. http://eclipse.org/lyo 9 What’s next The OSLC community Linked data and RDF An example The OSLC specifications 10 OSLC turns data into... Requirements Validation Tests Design Implementation R1 D1 I1 R2 D2 I2 Tool C Tool D T1 T2 Tool A 11 Tool B ...connected information... Requirements Validation Tests Design Implementation T1 validates validates R1 satisfy satisfy R2 D1 D2 implements implements I1 I2 validates validates T2 Tool A Tool B Does every requirement have a test to validate it? 12 Tool C Tool D Which requirements for the UI are related to test cases that failed on their last run? ...that can facilitate applied knowledge Requirements Validation Tests Design Implementation User Interface T1 validates validates R1 satisfy D1 I1 implements Release satisfy R2 D2 T2 Tool B Why is the number of failed test cases for the UI increasing in each iteration? 13 I2 validates validates Tool A implements Processing Engine Tool C Tool D How much faster is work progressing on the UI versus the Processing Engine? OSLC links lifecycle data 14 Linked Data and RDF Tim Berners-Lee’s four principles for linking data: 1. Use URIs as names for things 2. Use HTTP URIs so that people can look up those names 3. When someone looks up a URI, provide useful information using the standards (RDF, SPARQL) 4. Include links to other URIs so that they can discover more things “Instead of defining a new data model, OSLC’s resource and property-value approach is based on industry standard Resource Description Framework (RDF) data model.” Adapted from: http://open-services.net/bin/view/Main/ OslcCoreSpecification 15 RDF Concepts OSLC applies some RDF key concepts: 1. Graph data model 2. URI-based vocabulary 3. Format - Serialization syntaxes (RDF/XML, Turtle, JSON) 4. Datatypes 5. Literals 6. Expression of simple facts 7. Entailment We’ll briefly look at some of them. 16 1. OSLC uses an RDF graph data model Subject Amanda Test Case 1 Predicate owns validates Object Car Requirement 1 The predicate provides the property or relationship between the subject and object. Adapted from: http://www.w3.org/TR/2004/REC-rdf-concepts-20040210/#section-data-model 17 2. OSLC uses a URI-based vocabulary When there is a need to identify anything in OSLC, use a URI (there are a few exceptions). Using URIs allows everything to be linked together. It also allows common agreed-upon meaning for relationships and for resource types <http://...Test Case 1> <http://...validates> <http://...Requirement 1> OSLC Core URI Naming Guidance: http://open-services.net/wiki/core/OSLC-Core-URI-Naming-Guidance/ 18 3. OSLC allows different RDF formats The RDF model provides for describing RDF triples. There are various supported formats. Some are specialized for RDF (Turtle) and others are derived from existing formats (XML, JSON). These formats can be exchanged between different applications (tools). OSLC allows different types of format: RDF/XML Turtle JSON OSLC Core Specification: http://open-services.net/bin/view/Main/OslcCoreSpecification 19 Examples of different OSLC notations Subject <http://...Test Case 1> Predicate <http://...validates> Object <http://...Requirement 1> <http://example.com/TestCases/1> a oslc_qm:TestCase ; oslc_qm:validatesRequirement <http://example.com/Requirements/1> Turtle { "rdf:about": "http:\/\/example.com\/TestCases\/1", "rdf:type": [ { "rdf:resource": "http:\/\/open-services.net\/ns\/qm#TestPlan" } ], JSON "oslc_qm:validatesRequirement": { "rdf:resource": "http:\/\/example.com\/Requirements\/1" } } <oslc_qm:TestCase rdf:about="http://example.com/TestCases/1"> <oslc_qm:validatesRequirement rdf:resource="http://example.com/Requirements/1"/> </oslc_qm:TestCase> 20 RDF/XML What’s next The OSLC community Linked data and RDF An example The OSLC specifications This example is adapted from http://3roundstones.com/linkeddata-101/linked-data-tutorial/ [David Wood, 3RoundStonesInc. November2011] 21 Here’s a fictional project Existing product: Lunar Rover 3.0 New Release: Lunar Rover 3.1 Main goal is to improve remote steering Release to orbit date: September 20, 2014 22 Let’s look at the requirements domain Requirement 28465 Improve Remote Steering owner Bob release Lunar Rover 3.1 priority High owned by Iris created on November 24, 2011 state 23 release to orbit date September 20, 2014 Implemented The same information as before, in tabular form Requirement Owner Priority Created on State Release Requirement 28464 Add rear FIDO mast Linda Low October 18, 2012 New Lunar Rover 3.1 Requirement 28465 Improve Remote Steering Bob High November 24, 2011 Implemented Lunar Rover 3.1 Requirement 28466 Rebuild wheels for soil excavation Tanuj Medium September 9, 2012 Reviewed Lunar Rover 3.1 24 Rover Release Owned by Release to orbit date Lunar Rover 3.0 Cheng August 16, 2011 Lunar Rover 3.1 Iris Sept 14, 2014 Let’s look at the quality domain Test Case 35645: Test Steering owner Janet release Lunar Rover 3.1 priority High owned by Iris created on December 7, 2011 September 20, 2014 state Executed result 25 release to orbit date pass Let’s add more relationships validated by Test Case 35645: Test Steering release Requirement 28465 Improve Remote Steering owner owner priority release Bob Lunar Rover 3.1 High owner priority created on Iris Janet High December 7, 2011 created on November 24, 2011 state 26 Implemented release to orbit date state September 20, 2014 result Executed pass The same information as before, in tabular form Requirement Owner Priority Created on State Release Validated by Requirement 28464 Add rear FIDO mast Linda Low October 18, 2012 New Lunar Rover 3.1 Requirement 28465 Improve Remote Steering Bob High November 24, 2011 Implemented Lunar Rover 3.1 Test Case 35645: Test Steering Requirement 28466 Rebuild wheels for soil excavation Tanuj Medium September 9, 2012 Reviewed Lunar Rover 3.1 Lunar Rover 3.1 Rover Release Owner Release to orbit date Test Case Owner Priority ... Lunar Rover 3.0 Cheng August 16, 2011 Test Case 35645 Test Steering Janet High ... Lunar Rover 3.1 Iris Sept 14, 2014 Lunar Rover 3.1 Iris 27 ... RDF triple (subject-predicate-object) Triple Subject = Resource = always a URI Requirement 28465 Improve Remote Steering Predicate = Relationship or property = Always a URI validated by priority 28 Object = Could be a URI (which could refer to a resource) or a literal value (value to work with and show users) Test Case 35645: Test Steering High RDF triple (subject-predicate-object) Triple Subject = Resource = always a URI Predicate = Relationship or property = Always a URI <http://...require ment28465_ improve_remote steering> <http://...validatedby> <http://...priority> 29 Object = Could be a URI (which could refer to a resource) or a literal value (value to work with and show users) <http://...testcas e35645_test_ste ering> “High” Let’s add more relationships Work Item 38759 implements validated by release Test Case 35645: Test Steering Requirement 28465 Improve Remote Steering owner owner Janet release Bob Lunar Rover 3.1 priority High priority High owner created on December 7, 2011 Iris created on November 24, 2011 state 30 Implemented release to orbit date state Executed September 20, 2014 result pass There is a web of URIs around a development effort <http://.../build> <http://.../testresult > <http://.../change request> <http://.../build> <http://.../test case> <http://.../req> <http://.../build> <http://.../testresult > <http://.../workitem > <http://.../test case> <http://.../bug> <http://.../workitem > <http://.../bug> <http://.../test case> <http://.../change request> <http://.../req> <http://.../build> <http://.../workitem > <http://.../req> <http://.../release> <http://.../build> <http://.../bug> validate <http://.../build> <http://.../workitem > <http://.../req> <http://.../change request> <http://.../workitem > <http://.../test case> <http://.../req> <http://.../testresult > <http://.../test case> <http://.../change request> <http://.../testresult > <http://.../change request> 31 <http://.../build> <http://.../change request> <http://.../testresult > OSLC principles Tim Berners-Lee’s four principles applied to OSLC: Use URIs as names for things – In OSLC, each artifact in the lifecycle (for example, requirements, change requests, test cases...) is identified by a URI. Use HTTP URIs so that people can look up those names. – In OSLC, each artifact in the lifecycle is an HTTP resource. Standard HTTP methods (GET, PUT, POST, DELETE) are used to interact with them. When someone looks up a URI, provide useful information using the standards (RDF*, SPARQL) – Each OSLC resource has an RDF representation. OSLC resources can be queried using SPARQL. Include links to other URIs so that they can discover more things. – OSLC lifecycle artifacts are linked by relationships (for example, validatesRequirement or testedByTestCase) which are defined by URIs. 32 What’s next The OSLC community Linked data and RDF An example The OSLC specifications 33 Anatomy of a specification OSLC Core Specification How What OSLC Change Mgt Specification OSLC Requirements Specification OSLC Domain X Specification Core: Specifies the primary integration techniques for integrating lifecycle tools – the standard rules and patterns for using HTTP and RDF that all the domain workgroups must adopt in their specifications Domain: Defines integration scenarios for a given lifecycle topic and specifies a common vocabulary for the lifecycle artifacts needed to support the scenarios. Example: The Core specification describes Delegated UIs and Creation Factories and states that OSLC service providers MAY provide them. The Change Management specification states that CM service providers MUST provide them. http://open-services.net/bin/view/Main/OslcCoreSpecification 34 OSLC defines the following technical areas: 1. Discovery of capabilities 6. UI Previews for Resource Links 5. Delegated UI for Create and Select 35 2. HTTP C.R.U.D. for resources 3. Standard resource representations 4. Querying for resources First, some terminology (before we get to #1) OSLC Service Provider catalog provides OSLC Service Provider provides an implementation of OSLC Service These catalogs are used in the discovery of OSLC service providers. They help to simplify the configuration of tools that will integrate with providers of OSLC-defined services. example: IBM Rational Team Concert A product or online service offering that provides an implementation of one or more OSLC Services, which may themselves implement different OSLC Domain specifications Set of capabilities that enable a web client to create, retrieve, update and delete resources managed by an ALM or PLM product or online service offering and associated with one OSLC Domain manages OSLC Resource 36 Managed by an OSLC Service, may have properties and may link to other resources including those provided by other OSLC Services. example: IBM Rational Team Concert project area example: Change Management capability example: work item (bug, defect, enhancement request) 1. Discovery of capabilities Starting from the catalog you can discover services and their capabilities. This is a common pattern in OSLC. OSLC capabilities: Delegated UI Dialog allows you to create or find resources using a UI provided by the OSLC tool Creation Factory allows you to create resources programmatically Query Capability allows you to query for resources 37 2. HTTP C.R.U.D OSLC allows manipulation of resources using standard HTTP C.R.U.D Create = POST Request = GET Update = PUT Delete = DELETE 38 Resource Creation (Create) Create a resource using HTTP POST, with the resource body in format of choice URI for doing the POST is defined in the oslc:ServiceProvider in the oslc:creationFactory service Response is a 201-Created with Location HTTP header indicating URI for resource Request may be rejected for any number of reasons Insufficient permissions Missing required values Invalid data choices ...and … and ... Valid resource formats for creation are defined by: domain specifications service provider may define its own resources and formats optionally, by resource shape associated with creation factory 39 Resource Retrieval (Request) Use HTTP GET and standard HTTP content negotiation Client uses HTTP Accept request header to specify desired resource formats Accept: application/json, application/xml Use standard content(MIME) types Partial representations can be requested via HTTP URL key=value pair as ?oslc.properties= Allows for minimal retrieval of properties Get Defect 123 (all properties) GET http://bugs/123 Get Defect 123 (just title and status) GET http://bugs/123?oslc.properties=dcterms:title,oslc_cm:status 40 Resource Modification (Update) Use HTTP GET to get resource properties to be updated You’ll get an ETag back Change only the property values you need to change Clients must preserve unknown content Use HTTP PUT to send updated resource Use If-Match HTTP request header with ETag, services may reject your request without it HTTP PUT will completely replace the resource representation We are moving towards PATCH – new HTTP verb http://tools.ietf.org/html/rfc5789 It is possible to update only selected properties 41 Resource Deletion (Delete) Use HTTP DELETE on the resource identifier May not be allowed Response usually: • • • • 42 200-OK 204-No-Content 400-Bad-Request 403-Forbidden 3. Resource representations OSLC services should handle any type of resource Not just those defined by OSLC Resources defined by OSLC use RDF data model therefore are simply defined by their set of properties OSLC services MUST produce and consume RDF/XML representations Clients and services MUST NOT assume any subset of RDF/XML Other representations are allowed such as: XML: OSLC defined format that allows for consistent formats and is RDF/XML valid JSON: Rules for representing namespaces and QName properties Turtle: No constraints, use as is (may be preferred by future specs) Atom Syndication Format: <atom:content> SHOULD be RDF/XML 43 A few words on Resource Linking Links are properties where the property values are URIs Turtle format for a bug resource (abbreviated) <http://example.com/bugs/2314> a oslc_cm:ChangeRequest ; dcterms:relation <http://server/app/bugs/1235> ; Don't make assumptions about the target of links OSLC supports an open model Needed to achieve goal of “loosely coupled” integrations Clients need to be flexible and expect anything Sometimes we need to provide additional data about links: label, owners, and so on. Special cases where links need more representation 44 4. Querying for resources Query capability has base URI Clients form query URI and HTTP GET the results OSLC services MAY support OSLC Query Syntax http://open-services.net/bin/view/Main/OSLCCoreSpecQuery 45 Query syntax overview Filter results by appending “oslc.where=” with query clause to query base URI Only boolean operation allowed is “and” which represents conjunction “or” for disjunction is not defined in the interests of keeping the syntax simple. Retrieve just what you want with “oslc.select=” Defined ordering using “oslc.orderBy=” Full-text search via “oslc.searchTerms=” 'in' operator: Test for equality to any of the values in a list. The list is a comma-separated sequence of values, enclosed in square brackets: in [“high”,”critical”] 46 Comparison Operators = test for equality != test for inequality < test less-than > test greater-than <= test less-than or equal >= test greater-than or equal Query syntax example Find high severity bugs created after April fools day http://example.com/bugs?oslc.where= cm:severity="high" and dcterms:created>"2010-04-01" Find bugs related to test case 31459 http://example.com/bugs?oslc.prefix=qm= <http://qm.example.com/ns>& oslc.where=qm:testcase=<http://example.com/tests/31459> Find all bugs created by John Smith http://example.com/bugs?oslc.where= dcterms:creator{ foaf:givenName="John" and foaf:familyName="Smith"} 47 5. Delegated UI renders the source app UI in the target app A delegated UI renders the source application UI in the target application. This example shows the contributed/delegated Rational Team Concert Work Item search dialog being rendered in an OSLC Quality Management application. 2. iframe's src set to delegated UI's URL 1. Click to launch delegated UI 4. Click OK. Sends message (link+label) to parent window 48 3. Selection made Delegated UI key points Delegated UIs support both creation and selection of resources Two communication protocols are supported for iframes: HTML5 postMessage() ← preferred method – Supported in most modern browers Window object's window.name – Supported in older browsers and Eclipse embedded web widget Consumer selects which protocol to use, informs provider via fragment identifier Tremendous value for resource creation Traditionally most service logic was communicated to client and new dialog built Now the rules for creation and dialog change as needed Prefilling of creation dialog done by “creating” a dialog resource HTTP POST of resource format to creation dialog URL, response is URL of dialog prefilled 49 6. UI Preview Scenario supported: hover over link to get in context preview of resource Simple resource format defined and retrieved using HTTP content negotiation Hover over link 50 OSLC specifications also include Common property and resource definitions covering Resource shapes Resource typical properties: title, description, identification, … Leveraging Dublin Core and FOAF Discussion/comments OSLC services MAY offer OAuth 1.0a Three legged OAuth for webapp to webapp authentication Two legged OAuth for client to webapp authentication OSLC services MAY offer HTTP BASIC Auth User name and password sent “in the clear” with Base64 encoding Should be done via HTTPS only 51 OSLC Specifications Cover Many Domains Core & Common • Core, Configuration Management, Reporting Application Lifecycle Management (ALM) • Change Management, Quality Management, Requirements Management, Asset Management, Architecture Management, Automation (Software) Project Management • Estimation and Reporting Product Lifecycle Management (PLM) • ALM-PLM Interoperability Integrated Service Management (ISM) • 52 Performance Monitoring, Reconciliation Get Connected with OSLC Learn more at http://open-services.net http://eclipse.org/lyo Get started with your OSLC “Hello World!” with Eclipse Lyo and existing specs Join the community and contribute scenarios Jump in and create specifications that matter to you Open Services for Lifecycle Collaboration Lifecycle integration inspired by the web 53 Resources OSLC Web Site http://open-services.net OSLC Primer http://open-services.net/primer OSLC Tutorial http://open-services.net/tutorial OSLC Core and Domain specifications http://open-services.net/specifications/ Eclipse Lyo http://www.eclipse.org/lyo/ Even more OSLC resources http://open-services.net/resources/ 54 BACKUP 55 Need better integration approaches Past integration approaches have provided limited choice and coverage. Single repository Point-to-point integrations “Can I really expect one vendor to provide all the functionality I need? And what about my existing tools?” “How can I ever upgrade one tool without breaking everything else?” Past integration approaches have been disruptive and slow to emerge. Universal metadata standard Standard implementations “How did I ever think all those vendors would be able to agree?” “Did I really believe that every vendor would rewrite their tools on a single framework?” 56 Need open collaboration on solutions Past collaborations have fallen short because of limited and restrictive participation. Limited to small set of business partners No open process for others to join in Limits solution to particular use cases and technologies Solution design goals and approach No consensus driven approach No external review 57 No visibility into solution Point-to-point integrations Built after the fact with limited product APIs Solution focuses on two tools in hand Restrictive licenses and intellectual property License fees Fear of giving up IP Forces alternative solutions OSLC community Wide range of interests, expertise, participation Growing list of implementations Vendors, end users, industry consortia 40+ organizations have had employees participate in specification development efforts 100s of developers have used assets from Lyo Collaborating on solutions for ALM, DevOps, ISM, PLM Implementations from IBM Rational, Oracle, IBM Tivoli and open source 3rd party adapters from IBM, Kovair, Tasktop, and open source Dozens of end users enabling homegrown tools Specifications for many domains Change Management, Quality Management, Requirements Management, Asset Management, Architecture Management, Automation Product Lifecycle Management, Configuration Management Performance Monitoring, Reconciliation 58 OSLC website at http://open-services.net Register 59 Learn Adopt Browse Participate