China Research Lab Introduction to IBM Cloud Products and Cloudbased Analysis Platform Research at CRL Chen Wang (王晨) IBM China Research Laboratory IBM Confidential © 2006 IBM Corporation China Research Lab About Me Received Ph.D. degree of computer science from Fudan University in 2005 Joined IBM China Research Lab since 2005 Major research areas – Database and data mining [2002 to 2005] • Focus: to solve scalability and performance issues of querying and mining over graph like data • Representatives : KDD’04, SIGMOD’05, PAKDD’04, JCST, WWW J, DKE J – Semantic web and data management [2005 to 2008] • Focus: to develop scalable and efficient RDF data store in a single node • Representatives : SIGMOD’08, ICDE’09, WWW’08, WWW’07, VLDB’07 – Semantic web + data mining + cloud computing [Now!] • Focus: to deliver knowledge discovery system over RDF data in cloud environment to enable business intelligence in healthcare area • Representatives : … Homepage – http://www.research.ibm.com/people/w/wang IBM Confidential © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings (1 hour and 30 min) – CloudBurst (BlueCloud) – Tivoli service automation management (TSAM) Break (30 min) Cloud-based analysis platform research at CRL (50 min) Q & A (10 min) IBM Confidential © 2003 IBM Corporation China Research Lab IT infrastructure is reaching a breaking point 85% idle In distributed computing environments, up to 85% of computing capacity sits idle. 1.5x 70¢ per $1 Explosion of information driving 54% growth in storage shipments every year. 70% on average is spent on maintaining current IT infrastructures versus adding new capabilities. 40 billion 33% Consumer product and retail industries lose about $40 billion annually, or 3.5 percent of their sales, due to supply chain inefficiencies. 33% of consumers notified of a security breach will terminate their relationship with the company they perceive as responsible. IBM Confidential © 2003 IBM Corporation China Research Lab Video: Cloud Computing in IBM Expert’s eyes Irving Wladawsky-Berger discusses cloud computing with Tech Web TV IBM Confidential © 2003 IBM Corporation China Research Lab We are talking about a way to significantly lower cost… Internal Private Cloud Traditional Infrastructure • x86 servers – one application per server • 5% hardware utilization • Manual operations & maintenance versus • x86 servers – full virtualization • Embedded service management system • Automated self service Internal Private Cloud drives cost savings Can reduce IT labor cost by 50% in configuration, operations, management and monitoring Can improve capital utilization by 75%, significantly reducing license costs Reduce provisioning cycle times from weeks to minutes Can reduce end user IT support costs by up to 40% Unit cost Traditional Infrastructure Internal Cloud Scale IBM Confidential (IBM projections based on customer work) © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst • Hardware • Software • Architecture – Tivoli service automation management (TSAM) IBM Confidential © 2003 IBM Corporation China Research Lab What Is IBM CloudBurst? IBM CloudBurst leverages an IBM pre-integrated service delivery platform that include the hardware, storage, networking, virtualization and service management software to create a private cloud computing environment. IBM Confidential © 2003 IBM Corporation China Research Lab An industry standard rack cabinet External dimensions: – Height: 1999 mm (78.7 in) – Width: 605 mm (23.8 in) – Depth: 1001 mm (39.4 in) – Weight: 124.7 kg (275 lb) Comes with a front door, rear doors, and side panels installed. Has the following features: – Front stabilizer foot – Six 1-U side-wall compartments – Lockable doors and side panels IBM S2 42U – Room for cable management IBM Confidential © 2003 IBM Corporation China Research Lab Housing a cloud computing infrastructure management server … FRONT VIEW x3650M2 – Form: • 2U – Processor: • Dual Quad Core Intel® Xeon® X5520 • 2.26GHz, 8MB L2 cache, 80W – Memory: HDD Up to 12 REAR VIEW • 24GB RAM – Storage: • HDD 6x 146GB SAS 10K – Ethernet interfaces: • Four 1 Gbps NICs • Dual port 4 Gbps FC IBM x3650M2 rack-mount server IBM Confidential © 2003 IBM Corporation China Research Lab … a cloud compute platform … HS22 – Form: • 1U – Processor: • Dual Quad Core Intel® Xeon ® X5560 4C • 2.8GHz, 8MB L2 cache, 95W – Memory: • 48GB RAM – HDD: • Diskless – Ethernet interfaces: • Two 1 Gbps NICs • Dual port 4Gbps FC – USB: • VMware ESXi Hypervisor™ on embedded USB key IBM BladeCenter HS22 IBM Confidential © 2003 IBM Corporation China Research Lab FRONT VIEW … imbedded into a chassis BladeCenter H chassis – Form: • 9U – Features: • Up to 14 blade bays • Advanced Management Module standard, plus optional module for redundancy • USB-based keyboard, video, mouse (KVM), Ethernet, USB • Optical drive and two USB connections • High-speed fabric with eight data channels to every blade, four of which can be 10 Gb • Supporting industry-standard I/O switches • Light path diagnostics • Predictive Failure Analysis • Support for IBM System Storage solutions (Including DS and NAS family of products) and many widely adopted non-IBM storage offerings External storage Example: 8 blade servers followed by 6 empty slots REAR VIEW Example: on the left two FC switches and two Ethernet switches. On the right side a management module with console cables. IBM BladeCenter H Chassis IBM Confidential © 2003 IBM Corporation China Research Lab The rack also includes IBM system storage … DS3400 FC attached storage – Dual active RAID controller – Host Interface: • 4 Gbps Fiber Channel (FC) interface technology – Direct-attach storage (DAS) or SAN solution—start with a DAS configuration and seamlessly transition to a FC SAN when ready IBM System Storage DS3400 – Easy to deploy and manage with the DS3000 Storage Manager – 5.4 TB of storage capacity with twelve 450 GB hot-swappable 15K drives IBM Confidential © 2003 IBM Corporation China Research Lab … and network components BladeCenter H (I/O) – Redundant 4G FC Networking - Qlogic FC SM • QLogic 20-port 4 Gb Fibre Channel Switch Module – Redundant 1G Ethernet Networking – SMC 8126L2 IBM Confidential © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst • Hardware • Software • Architecture – Tivoli service automation management (TSAM) IBM Confidential © 2003 IBM Corporation China Research Lab Cloud computing infrastructure management server Operating System Windows2003R2 Enterprise Edition 32 bit (includes MS Active Directory) Database Server MS SQL Server Enterprise Edition MS SQL Server Express (imbedded w/ VMWare VirtualCenter (now vCenter Server) Platform Management Tools IBM Systems Director 6.1.1 BladeCenter Open Fabric Manager (BOFM) IBM BOFM v3 plugin Active Energy Manager IBM Active Energy Manager 4.0 Server Management Tools IBM ToolsCenter 1.1 Storage Management Tools IBM DS Storage Manager for DS4000 10.36 LSI SMI-S provider for DS3400 LMI SMI-S Virtualization Management Tools VMware vCenter Server 2.5 * (requires MS Active Directory) * Previous product name was VMware VirtualCenter 2.5u4 IBM Confidential © 2003 IBM Corporation China Research Lab CloudBurst software stack Bare Metal Hypervisor on HS22 blade VMWare ESXi 3.5 U4 Hypervisor Operating System Novell SUSE Linux Enterprise 10 SP 2 Network File System NFS Database Server IBM DB2 Enterprise Server Edition V9.1 for Linux Web Application Server, including IBM HTTP server IBM WebSphere Application Server Network Deployment 6.1.0.13 Directory Server IBM Tivoli Directory Server 6.1.0.1 Monitoring IBM Tivoli Monitoring 6.2.1 (OS pack) IBM Tivoli Monitoring for Energy Mgmt 6.2 Storage Management Client IBM Tivoli Storage Manager client 6.1 Provisioning Management IBM Tivoli Provisioning Manager 7.1 CloudBurst application IBM Tivoli Service Management pack including appliance wizards IBM Confidential © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst • Hardware • Software • Architecture – Tivoli service automation management (TSAM) IBM Confidential © 2003 IBM Corporation Virtual Center SQL Server ToolsCenter IBM Storage Mgr Active Energy Customer Ethernet Network Customer SAN Network BOFM IBM Director . China Research Lab Windows Server 3650 M2 Server 1G Ethernet Switch 1G Ethernet Switch MSIM-L Bay 9 MSIM-L Bay 7 Mid-plane ITM ITDS 24 port 1Gps Ethernet Switch 24 port 1Gps Ethernet Switch 20port FC SM 20port FC SM Bay 3 PDU 1 Bay 4 DB2 ESE PDU 2 WAS ND AMM2 Mid-plane TPM AMM2 HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade HS22 Blade CloudBurst x3650 M2 SUSE & NFS Controller A VMWare ESXi Hypervisor IBM Confidential Controller B IBM Storage © 2003 IBM Corporation China Research Lab Built-in Virtualization Understand virtual and physical resource usage Dynamically manage virtual workloads to optimize resource usage Automatically migrate virtual machines across systems to maintain service levels …Increases utilization for lower capital expense IBM Confidential © 2003 IBM Corporation China Research Lab Base configuration 1 BladeCenter H with 4 blades* BladeCenter H 1 2 3 4 5 6 7 8 9 10 11 12 13 14 DS3400 3650M2 Network Components 42U Standard Rack *First HS22 blade server is reserved for the CloudBurst software stack IBM Confidential © 2003 IBM Corporation China Research Lab CloudBurst software stack shipped as VMware images to run on the first HS22 blade server BladeCenter H 1 2 3 4 5 6 7 8 9 10 11 12 13 14 HS22 Blades CloudBurst TPM ITDS WAS ND ITM DB2 ESE DB2 ESE BlueCode VM SUSE BlueCode VM SUSE VMWare ESXi Hypervisor CloudBurst VM IBM Confidential VMWare ESXi Hypervisor Monitoring VM BlueCode VM SUSE + NFS + TFTP VMWare ESXi Hypervisor Image Server VM © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst – BlueCloud – Tivoli service automation management (TSAM) IBM Confidential © 2003 IBM Corporation China Research Lab IBM to Unveil Plans for “Blue Cloud” on 11/15 Family of ready-to-use cloud computing offerings – Based on open standards and open source software together with IBM software, systems technology and services – First offering to support Power and x86 processors – Plans to support System z and highly dense rack clusters Reduces IT management complexity and increases business responsiveness Supports both existing and emerging, data-intensive workloads Vietnamese Ministry of Science and Technology plans to leverage Almaden Research Center’s cloud environment to run country innovation portal Initial Blue Cloud offering targeted for Spring, 2008 availability IBM Confidential © 2003 IBM Corporation China Research Lab Forces Driving Cloud Computing Explosion of data intensive applications on Internet Increased network capacity and availability Fast growth of mobile commerce Advances in computer architecture and price/performance Requires massively scalable cloud infrastructures to serve billions of heterogeneous browser-based clients IBM Confidential © 2003 IBM Corporation China Research Lab Blue Cloud Initial Offering – Targeted for Spring 2008 Delivers a massively scalable and flexible compute platform for hosting both existing and emerging data-intensive workloads. Apache Virtual Virtual Machine Machine BladeCenter Virtual Machine Virtual Machine Tivoli Monitoring Agent Linux with Xen Monitoring IBM Monitoring v.6 DB2 Provisioning Baremetal & Virtual Machines Provisioning Manager v.5.1 WebSphere Application Server Provisioning Management Stack Virtualized Infrastructure Based on Linux & Xen • Based on open standards and open source software • Includes IBM software, systems technology and services • Supports both Power and x86 processors • Web 2.0 resource reservation system IBM Confidential © 2003 IBM Corporation China Research Lab IBM and Massive Scale Computing IBM is uniquely qualified to lead in cloud computing Blue Cloud builds off of IBM’s decades of experience developing and leading massive-scale computing: – Parallel Sysplex – IBM’s Deep Blue SP Cluster – Blue Gene – Grid Computing IBM Confidential © 2003 IBM Corporation China Research Lab Business Benefits of Blue Cloud • Cost efficient model for creating and acquiring information services • Reduces IT management complexity • Increases business responsiveness with real-time capacity reallocation as demand for compute power grows • Powers both existing and emerging data-intensive workloads IBM Confidential © 2003 IBM Corporation China Research Lab Technology Incubation Cloud for IBM Employees 70 active incubations 10 new products IBM Confidential © 2003 IBM Corporation China Research Lab IBM Research Computing Cloud (RC2) A ‘living lab’ to advance research strategies Provides self service ‘on demand’ delivery solution for research computing resources Integrates existing assets and products with SOA Zero touch support for the full life cycle of service delivery – Order creation – Approval process – E-mail notification – Automated provisioning – Monitoring IBM Confidential RC2 Self - Service Portal Business Process Workflow Application Business Process Workflow Management Business Service Platform Enterprise Service Bus Virtualized Infrastructure © 2003 IBM Corporation China Research Lab Vietnam Ministry Leverages Cloud to Run Innovation Program VISTA Innovation Portal (VIP) VIP pilot hosted on IBM’s Blue Cloud computing infrastructure at Almaden Students Teachers Blogs Profiles Wikis Forums Social Tagging Information Discovery Researchers IBM Confidential VIP, powered by IBM Innovation Factory, provides a platform to foster collaborative innovation among major universities and research institutes. IBM Innovation Factory © 2003 IBM Corporation China Research Lab Summary IBM unveiling new cloud computing offerings to respond to customer needs: – Explore extreme scale quickly and easily – Share infrastructure resources efficiently – Simplify IT management – Handle new and emerging workloads – Provide a platform to encourage open collaboration A white paper, “Cloud Computing” from High Performance on Demand Solutions available at: http://www.ibm.com/developerworks/websphere/zones/hipods/library.html IBM Confidential © 2003 IBM Corporation China Research Lab 蓝云成功案例1:中国无锡科教产业园云计算中心 2008年2月1日,IBM在 中国无锡太湖新城科教产 业园建立第一个云计算中 心; 该中心将为中国新兴软 件公司提供接入一个虚拟 计算环境的能力,从而鼎 力支持其开发活动; 是全球第一个实现商业运 营的云计算中心。 IBM Confidential © 2003 IBM Corporation China Research Lab 蓝云成功案例1:中国无锡科教产业园云计算中心 该平台可以直接为园区的软件外包公 司以及周边的企业提供数据中心的服务; 通过在云计算平台上搭建完整的IBM Rational软件交付平台,可以为各软件 公司提供高品质的开发和测试等外包服 务; 该云计算平台可协助园区实现软件许 可证租借、培训等商业服务模式。 IBM Confidential © 2003 IBM Corporation China Research Lab 蓝云成功案例2:与Google进行的学术合作 培养具备下一代“云计算”技能的人才 IBM研究中心 • 倡导大学推动开放式标准和新兴并行计算模型 • 联合提供硬件、软件、服务支持新的并行计算课 程 • 三大计算 “云团” 谷歌 首批入选大学名单 美国华盛顿大学 IBM Confidential 35 © 2003 IBM Corporation China Research Lab 蓝云成功案例3:荷兰iTricity IDC • 连接当地5个数据中心 • 托管多个客户的业务系统 • 基于订阅和使用计费 • 基于IBM System x 和 System p • 增加的蓝云功能 – 部署SAN – 网络虚拟化 – 发布到多个数据中心 – 部署复杂应用 优势 – 按使用计费,不是一次投入 – 设备和位置的独立性 • 通过多个数据中心的云计算资源池提供可靠性 商业模式 • 可利用其他数据中心的处理能力增加扩展性 • “按小时计费” • 符合ISO27002标准的安全性 • 把基础架构作为服务,客户可以迅速获得或释放资源 • 云计算作为传统托管服务的补充,并将逐渐增长 IBM Confidential © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst – BlueCloud – Tivoli service automation management (TSAM) IBM Confidential © 2003 IBM Corporation China Research Lab Cloud Computing - Definition from IBMs BlueCloud Architecture Board It is a user experience and a business model • Cloud computing is an emerging style of computing in which applications, data, and IT resources are provided as services to users over the network. It is a infrastructure management methodology • Cloud computing is way of managing large numbers of highly virtualized resources such that from a management perspective, they resemble a single large resource. This can then be used to deliver services. Monitor & Manage Services & Resources Cloud Administrator IT Cloud Service Catalog, Component Library 38 IBM Confidential Datacenter Infrastructure Service Consumers Access Services Component Vendors / Software Publishers Publish & Update Components, Service Templates © 2003 IBM Corporation China Research Lab Service Request & Operations Service Creation & Deployment IT Infrastructure & Application Provider Architectural Model for Cloud Computing End User Requests & Operators Service Management User Request Management/Self Service Portal Service Lifecycle Management Image Lifecycle Management Provisioning Performance Management Security: Identity, Access, Integrity, Isolation, Audit & Compliance Service Oriented Architecture … Service Catalog Request UI Operational UI Availability/Backup/ Restore Usage Accounting License Management Information Architecture Design & Build Image Library (Store) Optimized Middleware (image deployment, integrated security, workload mgmt., high-availability) Deployment Virtualized Infrastructure Virtual Resources & Aggregations Server Virt. System Resources SMP Servers Blades IBM Confidential Virtual Image Management Storage Virt. Storage Servers Network Virt. Storage Operational Lifecycle of Images Network Hardware August 24, 2008 © 2003 IBM Corporation China Research Lab Service Management Center for Cloud Computing Service Management Center for Cloud Computing Admin Provision a new server with OS and applications Modify a provisioned server Start/stop a server, change memory and CPUs Add/remove node to cluster AIX Linux on z Linux on x End User Virtualized Infrastructure AIX Linux System p LPARs IBM Confidential Linux Win VMWare z/OS System z © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst – BlueCloud – Tivoli service automation management (TSAM) • Request driven provisioning • Service lifecycle manager • User experience IBM Confidential © 2003 IBM Corporation China Research Lab Request Driven Provisioning (RDP2) Project End User Request for Provisioning 1 n Virtual Server P5/6 LPARs with AIX, RH Linux Intel VMWare ESX with RH Linux or Windows 1 1 SW Stack IBM Confidential © 2003 IBM Corporation China Research Lab RDP Scenarios Create virtual servers within a new project Deploy new virtual servers in an existing project Install a SW stack (OS and applications) Destroy a virtual server Increase/decrease CPU of a virtual server Increase/decrease RAM of a virtual server Cancel a project and destroy all associated virtual servers IBM Confidential © 2003 IBM Corporation China Research Lab Create Virtual Servers in a new Project 2 SRM Catalog Selects “Create a new Project” List of offerings: Create new Project End User Add server to existing project 1 The End User logs into TSAM Project Name Description userGroup Server Type # of servers # CPUs Memory DSL List of SW stacks 6 5 Select SW stack 4 Specifies Project details Specifies Server Details 7 Submit Request IBM Confidential © 2003 IBM Corporation China Research Lab Add Server to an Existing Project SLM List of projects 3 SRM Catalog List of offerings: Create new Project End User Add server to existing project 1 Selects “Add Server to an Existing Project” The End User logs into TSAM Project Name Description userGroup Server Type # of servers # CPUs Memory DSL List of SW stacks 6 5 Select SW stack Selects an existing project 2 4 Specifies Project details Specifies Server Details 7 Submit Request IBM Confidential © 2003 IBM Corporation China Research Lab Modify a Server SLM List of projects 3 SRM Catalog End User List of offerings: Modify a Server Delete a Server Cancel Project 1 2 Selects an existing project Selects “Modify a Server” The End User logs into TSAM List of Servers in the project Server # CPUs Memory 4 Selects a specific Server 5 Modify Server Details 7 Submit Request IBM Confidential © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst – BlueCloud – Tivoli service automation management (TSAM) • Request driven provisioning • Service lifecycle manager • User experience IBM Confidential © 2003 IBM Corporation China Research Lab Sample Configuration SLM R1 WAS-based Application Service (structure + mgmt plans defined within template used for instantiation) WAS Deployment Manager AIX manages Administrator System p LPAR WAS Cluster Member Central TPAP (aka Maximo)-based GUI “Launch in Context” to other Mgmt Software DB2 AIX System p LPAR zOS Management Server on AIX or z/Linux IBM HTTP Server WAS Cluster Member AIX AIX System p LPAR System p LPAR z/VM or zLPAR • WAS Cluster Provisioned & Configured according to best practices Monitored by ITM / Omegamon End User IBM Confidential © 2003 IBM Corporation China Research Lab Some sample Topologies and Monitors DB2 WAS Deployment Manager DB2 IBM HTTP Server WAS Deployment Manager z/Linux zOS z/VM z/VM WAS Cluster Member IBM HTTP Server WAS Cluster Member WAS Cluster Member Linux AIX AIX z/VM System p LPAR System p LPAR Development System IBM Confidential Testsystem WAS Deployment Manager (standby) WAS Deployment Manager Linux Linux z/VM z/VM DB2 DB2 zOS zOS IBM HTTP Server IBM HTTP Server Linux Linux z/VM z/VM WAS Cluster Member WAS Cluster Member AIX AIX System p LPAR System p LPAR System z LPAR Parallel Sysplex PreProduction system © 2003 IBM Corporation China Research Lab Example: Deployment of an Application Landscape – System z Service Management Center for System z Tim, Application Test Team (End User) 1 Work Order “IT Service Order” Lacking z skills (network, security, monitoring, etc.) , Tim is pleased to see a complete WebSphere landscape offered exactly what he needs for an upcoming test of his newly developed SOA application. Tim doesn’t want to bother with the details of the underlying topology Service work order process Automation for service fulfillment, services inventory, tracking, work order, approvals 2 TPM workflow, best practice mgmt configurations, install images ( OS, middleware, mgmt agents) 3 Serice Definitions Inventory Tim’s request together with all parameters describing service levels and other details about the desired WebSphere Landscape are routed to Ann for approval. After Ann verified the validity of Tims request, she initiated the automated deployment of the landscape. Work order Application Landscape Manager Landscape Mgr Process Assembles topology information, build and management plans, best practices templates, thresholds and policies to initiate deployment Deployment automation TPM Install Images, best practice configuration TPM provisions all landscape components including OS, DB, MW, etc. together with management agents and best practices templates, thresholds and policies IBM Confidential Ann, Service Landscape Deployment Planner OMEGAMON 4 Tim receives notification that the landscape is ready for use OMEGAMON DB2 on z/OS DB2 on z/OS z/OS z/OS OMEGAMON WebSphere Linux on System z z/VM © 2003 IBM Corporation China Research Lab Service Definition and Provisioning Same Service Definition Process can be used to – Build and manage a (Development-)System that can be ordered and gets instantiated right-away (Request Driven Provisioning) – Define a (Production-)System that is mapped to the infrastructure involving various subject matter experts ( security, HA, performance ….) and gets instantiated in the next maintenance window The Service Instantiation time and quality is improved by – Collecting all required parameters upfront – Using a standardized and repeatable process – Using Automation where possible / desired IBM Confidential © 2003 IBM Corporation China Research Lab Managing the IT Landscape for a Service Management can be done on a Service Instance – that was build using the Service Definition / Request process Typical Management Actions – Start / Stop – Resizing ( add/remove Cluster-Member ) – Incident Handling (next chart) Adaptable Framework – Customer can include existing management tools – Customer can adopt to existing processes IBM Confidential © 2003 IBM Corporation China Research Lab Operational Management Scenario Threshold exceeded indicating a potential Response Time Problem Close ticket TPAP Service Lifecycle Manager 5 Correlate with information about service, proposed Mgmt. Plans, etc. CMDB MBOs 2 3b Ann: DB Specialist John, Operations Specialist Analyze Incident Analyze Problem on DB Select Automation Packages or route Problem 3a 1 Simplify typical operational management scenarios by providing Best Management Practices for the specific application landscape across the entire spectrum including OMPs, thresholds, policies, job plans, etc. ITM Agent: Threshold exceeded ISM Job Plan Execution 1. Identify and stop ran-away pid 2. Add Application Server 3. <customer best practice> 4. Adapt WAS Heap Size 4 Ticket handling through service process (out of scope) 6 Close incident automatically or through user after manual action Composite Application - Distributed WAS on p, DB2 on z IBM Confidential © 2003 IBM Corporation China Research Lab Agenda IBM cloud offerings – CloudBurst – BlueCloud – Tivoli service automation management (TSAM) • Request driven provisioning • Service lifecycle manager • User experience IBM Confidential © 2003 IBM Corporation China Research Lab Logon / Select Service Defintion IBM Confidential © 2003 IBM Corporation China Research Lab Management Plans IBM Confidential © 2003 IBM Corporation China Research Lab Monitoring and Event Management Good Practices for determining the cause of the Event and possible corrective actions (if required) Good Practices for corrective management actions IBM Confidential © 2003 IBM Corporation China Research Lab Resource / Topology Defintion IBM HTTP Deployment Server Manager AppServer WebSphere Cluster Instance Managed Node ... AppServer Database Instance Instance Managed Node DBMS Server (1 .. 5) WebSphere Cell IBM Confidential © 2003 IBM Corporation China Research Lab Resource Definitions – Connection to DB2 Database Instance DBMS Server IBM Confidential © 2003 IBM Corporation China Research Lab Topology Node Operations Deployment Manager IBM Confidential © 2003 IBM Corporation China Research Lab Service Instances Self Service GUI Existing Instances IBM Confidential © 2003 IBM Corporation China Research Lab Instantiation Steps 1/3 IBM HTTP Deployment Server Manager zLinux zLinux AppServer AppServer WebSphere Cluster Instance Topology Customization Cardinality Resource Allocation Template Colocation Instance ... Managed Node zLinux Managed Node zLinux (1 2.. 5) Logical Server 1 WebSphere Cell Logical Server 2 IBM Confidential Monitoring Configuration (optional) AppServer Database Instance Instance Managed Node DBMS Server © 2003 IBM Corporation China Research Lab Instantiation Steps – Step 2/3 IBM HTTP Deployment Server Manager zLinux zLinux Topology Customization Cardinality Resource Allocation Template Colocation Monitoring Configuration (optional) Resource Allocation AppServer AppServer Instance Instance Managed Node zLinux LNXSOA02 Managed Node zLinux LNXSOA04 Logical Server 1 Logical Server 2 CMDB IBM Confidential © 2003 IBM Corporation China Research Lab Instantiation Steps – Step 3/3 IBM HTTP Deployment Server Manager Topology Customization Cardinality Resource Allocation Template Colocation Monitoring Configuration (optional) Resource Allocation AppServer AppServer Database Edit Job Plan (optional) Instance Instance Instance Approval (Scheduled) Provisioning Managed Node Managed Node LNXSOA02 LNXSOA04 DBMS Server To TPM IBM Confidential © 2003 IBM Corporation China Research Lab Shortcuts for less complex / controlled Environments Preapproved Instantiation Fixed Cardinality of cluster members / target OS Fixed server layout Fixed monitoring setup Non-editable TPAP job plans IBM Confidential © 2003 IBM Corporation China Research Lab 30 Min Break IBM cloud offerings (1 hour and 20 min) – CloudBurst (BlueCloud) – Tivoli service automation management (TSAM) Break (30 min) Cloud-based analysis platform research at CRL (1 hour) Q & A (10 min) IBM Confidential © 2003 IBM Corporation China Research Lab Agenda Cloud-based analysis platform research at CRL – Semantic web – Healthcare applications – Clod based analysis platform IBM Confidential © 2003 IBM Corporation China Research Lab Today’s Web Most of today’s Web content is suitable for human consumption – Even Web content that is generated automatically from databases is usually presented without the original structural information found in databases Typical Web uses today people’s – seeking and making use of information, searching for and getting in touch with other people, reviewing catalogs of online stores and ordering products by filling out forms IBM Confidential © 2003 IBM Corporation China Research Lab New Challenge What We Say to Dogs – from Gary Larson cartoon (local link) ... • "Stay out of the garbage! Understand, Ginger? Stay out of the garbage!" – What Dogs Understand • "... blah blah blah blah GINGER blah blah blah blah ..." What Computers Understand on Current Web – " ... blah blah blah <a href=http://www.xwz.com/foo.html> link </a> blah blah blah . . . ."* IBM Confidential © 2003 IBM Corporation China Research Lab Semantic Web The Semantic Web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. “Facilities to put machine-understandable data on the Web are becoming a high priority for many communities. The Web can reach its full potential only if it becomes a place where data can be shared and processed by automated tools as well as by people. For the Web to scale, tomorrow's programs must be able to share and process data even when these programs have been designed totally independently.” W3C Semantic Web Activity IBM Confidential © 2003 IBM Corporation China Research Lab What Does RDF Data Look Like University:hasName xsd:Boolean University:Person University:hasName rdfs:subClassOf Subject Predicate University:Person1 xsd:String “Jack”^xsd:String University:hasName University:Person1 University:Professor University:hasGender University:teachOf University:hasName University:Person2 University:Person2 Object rdfs:subClassOf Male^xsd:Boolean University:hasCourseName “Rose”^xsd:String University:hasGenderUniversity:Course Female^xsd:Boolean University:GraduateStudent University:teachOf University:CourseA University:takeCourse University:Person1 University:Person2 University:takeCourse “Maths”^xsd:Strin RDF:typeOf gUniversity:hasCourseName RDF:typeOf University:CourseA Take.PhUniversity:Professor .D course University:hasCourseName RDF:typeOf University:Person1 University:CourseA rdfs:subClassOf “Maths”^xsd:String University:Person2 RDF:typeOf University:GraduateStudent RDF:typeOf University:CourseA University:Person1 RDF:typeOf University:CourseA University:Course University:Professor rdfs:subClassOf University:teachOf University:GraduateStudent University:hasGender University:Professor rdfs:subClassOf University:hasName University:GraduateStudent …… Male^xsd:Boolean University:teachOf University:takeCourse University:Person University:takeCourse University:Person University:Course University:Person2 University:Course University:hasName University:hasGender “Jack”^xsd:String “Rose”^xsd:String IBM Confidential Female^xsd:Boolean © 2003 IBM Corporation China Research Lab LOD (Linking Open Data), DBpedia, IMDb, Freebase…… LOD: over 30 datasets which consisted of 2 billion triples till Apr 2008 DBpedia: over 100 million triples till Sep 2008 – Entities (over 2 million), e.g., • • • • 80,000 persons 70,000 places 35,000 music albums 12,000 films – Relationships, e.g., • 657,000 links to images • 1,600,000 links to relevant external web pages • 180,000 external links into other RDF datasets – Classifiers, e.g., • 207,000 Wikipedia categories • 75,000 YAGO categories Freebase: – Over 4 million resources. IMDb (LinkMDM): over 30K overlapping movies, soundtracks IBM Confidential © 2003 IBM Corporation China Research Lab RDFS RDF Schema is a Vocabulary Description Language – it allows specification of domain vocabulary and a way to structure it – Class, Property, subClassOf, subPropertyOf, domain, range Formal semantics add simple reasoning capabilities: rdf:Property rdf:type name – class and property subsumption – domain and range inference rdfs:domain rdfs:Class rdf:type Person rdfs:subClassOf Researcher rdf:type person001 IBM Confidential © 2003 IBM Corporation China Research Lab OWL <?xml version="1.0" encoding="UTF-8" ?> <rdf:RDF xml:base = "http://www.ibm.com/crl#" xmlns:rdf = "http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:owl="http://www.w3.org/2002/07/owl#"> <owl:Ontology rdf:about=""> <rdfs:comment>An example to show differences between owl-lite, DLP and owl-DL</rdfs:comment> </owl:Ontology> <owl:Class rdf:about=#Faculty> Faculty <owl:unionOf rdf:parseType="Collection"> <owl:Class rdf:about="#Professor" /> <owl:Class rdf:about="#Post-Doc" /> OWL:UnionOf <owl:Class rdf:about="#Lecturet" /> </owl:unionOf> </owl:Class> Professor Post-Doc Lecture <owl:Class rdf:ID=“Ph.D Student"> <rdfs:subClassOf> <owl:intersectionOf rdf:parseType="Collection"> <owl:Class rdf:about="#Person" /> <owl:Restriction> <owl:onProperty rdf:resource="#take" /> Ph.D Student rdfs:subClass <owl:someValuesFrom> <owl:Class rdf:about="#Ph.D course" /> </owl:someValuesFrom> </owl:Restriction> </owl:intersectionOf> </rdfs:subClassOf> </owl:Class> IBM Confidential Blank node OWL:intersectionOf Person Take.Ph.D course © 2003 IBM Corporation China Research Lab Agenda Cloud-based analysis platform research at CRL – Semantic web – Healthcare applications – Clod based analysis platform IBM Confidential © 2003 IBM Corporation China Research Lab Scenario: Adverse Effect Discovery for Drugs The Pharmacovigilance Process (FDA) Traditional Methods Insight from Outliers Type A (Mechanism-based) Type B (Idiosyncratic) IBM Confidential Data Mining Detect Signals Generate Hypotheses Refute/Verify Public Health Impact, Benefit/Risk Estimate Incidence Act Inform Change Label Restrict use/ withdraw © 2003 IBM Corporation China Research Lab Scenario: Adverse Effect Discovery for Drugs AERS Schema Pieces (FDA) Reporter country Patient death date Receipt date Qualification – Physician – Pharmacist – Health professional – Lawyer – Consumer Seriousness – death – lifethreatening – hospitalization – disabling – congenitalanomaly – Other Patient set age Medicinal product Patient weight Drug batch numb Patient gender Reaction medDRA PT IBM Confidential © 2003 IBM Corporation China Research Lab Scenario: Adverse Effect Discovery for Drugs Statistical Methodology Most use variations of 2-way table statistics No. Reports Target AE Target Drug a Other AE b Other Drug c d nOD nTA nOA n Total Total nTD Basic idea: Flag when R = a/E(a) is “large” Some possibilities Reporting Ratio: E(a) = nTD nTA/n Proportional Reporting Ratio: E(a) = nTD c/nOD Odds Ratio: E(a) = b c/d IBM Confidential © 2003 IBM Corporation China Research Lab Scenario: Adverse Effect Discovery for Drugs AERS Data Statistics (FDA) Spontaneous Reporting System began in late 1960’s, and had received over 2.7 million reports till June, 2004. For now, 300,000 reports per year were added continuously (approximately 1,000 reports per day) . > 9,000 event codes (MedDRA Preferred Terms) > 7,000 drug/biological products by trade names > 3,000 by generic names (generic names + combination products), from health professionals, suspect products only > 63,000,000 drug-event combinations possible! For all possible quadruples (e.g., drug-drug-drug-event or drugdrug-event-event) 20,000,000,000,000,000 combinations are possible! IBM Confidential © 2003 IBM Corporation China Research Lab Agenda Cloud-based analysis platform research at CRL – Semantic web – Healthcare applications – Clod based analysis platform IBM Confidential © 2003 IBM Corporation China Research Lab Architecture Application HTTP Server Repository Data Requirement Parsing PMML+ Parsing Importing & Mining In-memory Data Model In-memory Analysis Model RDF2X Mining RDF2File QueryGen RDF2Flattentable HBase SPARQL Queries Distributed File System (HDFS) Map/Reduce Running System (Hadoop) Master IDB SeDA I1 Queries Results DataImport P1 P2 gk gk g1 IBM Confidential g2 g1 g2 I2 … Pn gk g1 g2 © 2003 IBM Corporation China Research Lab Example Data ISR Drug Seq Reaction PT Outcome Cod Age Gender Wt. Occu. 5701501 1009771744 PNEUMONIA DE 83 M HO 49 M 103KG CN DEAFNESS DS 68 F 142LBS MD SYNCOPE HO MD HO 5710206 1009806917 ABDOMINAL DISCOMFORT ABDOMINAL PAIN UPPER DIZZINESS HYPOTENSION NAUSEA VOMITING 5908666 1010540206 IBM Confidential © 2003 IBM Corporation China Research Lab Lifecycle of a Cloud Service Service Offering Subscription & Instantiation •Select Service, specify parameters and SLA’s •Automatically instantiate the Service Subscriber (e.g. Line of Business) Service Offering Creation & Registration •Define Service based on Template and register it in the Catalog Manual or Autonomic Execution of Management Plans leveraging Automation and Virtualization Administrator / SLM Ensure SLA Conformance Cloud Service Service Catalog Manager Service Instance Termination OS Virtualizers (e.g. PHYP) Ensemble Hardware Individual Servers IBM Confidential Management LPAR / VM Management LPAR / VM Management LPAR / VM Ensemble Hardware … OS … Subscriber (e.g. Line of Business) App App Virtualizers (e.g. z/VM) OS System x Ensemble App App OS App App … Power Ensemble App App OS App App •Create Build- and Management Plans for Service IBM / ISV / IT Dept App App Service Template Definition System z Ensemble Destroy Service and free up resources OS Virtualizers (e.g. Xen) Ensemble Hardware Storage Ensembles © 2003 IBM Corporation China Research Lab Q&A IBM Confidential © 2003 IBM Corporation China Research Lab IBM Confidential © 2003 IBM Corporation