managing new workload applications in the cloud
Rich Garcia rggarcia@us.ibm.com
1 © 2013 IBM Corporation

2
 New & Smarter Job Scheduling
– New Business Models
– Extended Business Partners
–
IT Service & Delivery
– End User self service
 New Applications, job types
– Cloud
– Big Data Analytics
 New LOB Development Models
– App/Dev, Dev/Ops

New Users
– Private Clouds (remote development groups, contractors)
– Public & Hybrid Clouds (Business Partners, POS, Off-Shore Dev)
 Roadmap: Next Generation of Workload Automation success
– IT responsive to change
• Advances in hardware
• e-2-e cross platform solutions
– Consolidated Operations groups
– IT process automation
– Un-attended scheduling process
– Centralized Monitoring, visualization & control
– Infrastructure Matters
© 2013 IBM Corporation

NEW Industry Drivers for Business, IT, Applications and Workload Automation
Gartner Magic Quadrant – yard stick
Abstraction levels
IT Consolidations, Automation
Componentized solution, innovation-ready
Integrated technologies, no boundaries of technical expertise
Single point for vision, control, automation
New IT- Business
On demand,service based
Policy-based and SLA-based execution services
Asset optimization and on-demand provisioning of resources - maximized resource utilization and minimized energy consumption
Business-driven service management
Cost Management priority
New programming models
Leveraging collective intelligence
Adoption of new processes, applications, workloads, and technology
Collaboration across the IT organization by integrating service management process workflows with production service schedules
3
Complexity growing
Rapid IT changes
Near realtime services
Data Center adaptability
Reduce IT Spend
Labor cost control
Underutilized systems
Global collaboration, B2B,
Business Parners
Outsource / insource / rightsource
© 2013 IBM Corporation

Batch Scheduling & Automation Transformation to SBS
 Automate calendar-based production batch – workload planning
 Orchestration of work flows
 Centralized management
 Optimize use of existing resources
 Provide increased availability, reduce errors
 Constant productivity increase on batch window

Governance across scheduling points through a consolidated view
 Event-based scheduling, on demand

Provide proactive alerts on workload risks
 Transform complexity in platform/applcation flexibility

Integration across Tivoli
 Follow the ERP evolution
•
For example load balancing in SAP
 Use of virtualization technology
 Service Mgmt automation
– Workload Service
Assurance
– Align to Business
Processes
 Flexible deployment
– Flexible End-to-end and convergence to common code
– Pluggable Applications product support
– Solution-based offering
 Dynamic topology, HA
 Web Services - Schedule for the Cloud
4
Batch scheduling
We started here
Workload Automation
Event Driven
Service based scheduling
Newer Workloads
© 2013 IBM Corporation

How job scheduling has changed along the years
Automate calendarbased and event-based into plandriven production batch
Advanced orchestration of workflows, monitoring and reporting
High availability, scalability, fault tolerance , flexibility, advanced automation
End-to-end consolidation z-Centric, d-
Centric, peer2peer schedulers
SLA management
Proactive
Monitoring and
Workload
Service
Assurance
Heterogeneous
Workloads &
Application
Extensibility
Resource
Virtualization &
Dynamic
Scheduling
Automate calendarbased into plandriven production batch
Orchestration of workflows, efficient monitoring
High availability, scalability automatic recovery
5
2001 2003 2006 2008 2009 2010 2011 2012 2013
© 2013 IBM Corporation

Cloud reaches beyond traditional IT & production control
Application & scheduling plans
Spikes in demand, access IT
New Users of IT
Automation
Audit Governance
Optimize with Cloud
Extend to
Mobile Devices
Deploy Smarter
Physical Infrastructures
Protect &
Manage Data
6
6
Speeding service delivery Ad Hoc
Schedule change requests
Respond to new End
User service requests
– access computing resources
System z is Cloud Ready, end-2-end IT
Automation critical maintenance- i.e. backup critical data
© 2013 IBM Corporation

VISIBILITY e-2-e centralized
Monitoring workload
CONTROL
Unattended
Operations
New
Workloads
AUTOMATION
Achieve desired business outcomes
© 2013 IBM Corporation

All about the Business, End Users, Applications,
AND…IT Service, Support & Delivery
Working Smarter
“Intelligent Use of Computers”
8
Interconnected users and smart devices
Highly virtualized, serviceoriented architectures
Private & public delivery options, like Cloud & SaaS
© 2013 IBM Corporation

9
• New Business Drivers
• Private – Public – Hybrid Clouds
• IT & LOB Transformations
• All about the End Users
• Extended (remote) Business Partners
• Speed time to market, TTV
• Applications & Development LCM
• Real Time Access to information
• IT Optimization,
• How we manage
• systems
• workload
• Service Delivery
• Reduce cost
• work consolidation
© 2013 IBM Corporation

 New Batch

Data Management
 ETL, ERP-SAP
 Analytics db
 Access
 Availability

Security
10 © 2013 IBM Corporation

11
Source: “Consider Scheduling Tools for Batch Application Integration”, Gartner.
Batch jobs are no longer run within a batch window, but rather 24x7 in
"micro-batches".
Businesses are moving their applications to the
CLOUD
© 2013 IBM Corporation

 Cloud, Web Services, JAVA, Internet Facing Mobile Apps

Revenue Generating Systems
– Critical Business Focus
– ETL, ERP

Business Intelligence & Analytics , DB Applications
Big Data, analytics, cloud, mobile, and security:
– Emergency Services, Law Enforcement, Homeland Security, US Customs,
Health Care/Pharmaceuticals/H-Insurance, Wall Street Traders, POS Retail,
Banking -> ATMs -> Mobile

Real Time 24x7 User Access to information
 Business today - Global WW Enterprise systems
© 2013 IBM Corporation 12

Financial HealthCare
 Banking
 Credit Card
 Loans
 Credit Bureaus
 Doctors/Hosp
 Lab Services
 Pharmacies
Insurance
 Life
 Medical
 Auto
 Home
Automotive Govt Fed, State &
Local
 GM, Ford, Chrys
 Homeland Security
 POS Dealerships
 Fire/Rescue
 Police
 US Customs Serv
© 2013 IBM Corporation 13

New Workload requirements
•
Availability always
•
Resource based scheduling
•
HA resources assurance
• Automatic optimization, priority pools
• Automatic provisioning, scaling
• Dynamic on-demand scheduling capabilities
•
Full function access to run jobs
•
Variable substitution on the fly
•
Centralized scripts repository
•
New End-User Support interfaces
• TDWC User GUI
• Self Service Delivery
•
Mobil Device support
• New Application Development API interfaces
14 © 2013 IBM Corporation

Heterogeneous and unattended workloads and applications
Infrastructure matters
• Deploying applications in the cloud requires
– Knowledge of the business logic
– Knowledge of data needed
 WA automatically moves data within the application
–
Transforms data into useful information
–
Satisfy the business goals

WA deploys on top of the business infrastructure and platforms to give business
–
Production ready dynamic applications
– Leverage the flexibility & scalability of cloud computing

Integration & Automation of process
15 © 2013 IBM Corporation

New end-to-end Applications Architecture
User GUI
Scheduler controller
Distributed systems
System z
Controller
JOB definitions are stored as XML files in the JCL library. supports pluggable Job
Executors, implemented as OSGI bundles distributed servers
16
© 2013 IBM Corporation

• Support business growth mapping new types of workloads for simplified management
• Expand automation into new applications to take advantage of processes in a managed approach
• Share infrastructure among application, with no need to learn the individual applications interface
1
A file is tranferred and loaded into
Hadoop
Provide an extensible framework through application plug ins to extend the reach of automation to any new workload type
2
Data is manipulated through Big
Data
3
SAP uses the data for financial processing
Business reports are generated 4
Datastage
© 2013 IBM Corporation 17

1
Advanced
Workload
Support
Vendor supported integrations
2
Administrator can build its own application plug-ins
18
Customize for in-house apps
!
Executors, implemented as
OSGI bundles
Solutions available on open nets
BigInsights
System Automation
© 2013 IBM Corporation

What you should offer to 3 parties (Clients, BP, System Integrators) for new
Application Plug-ins implementation
• Provide an Integration Workbench UI
• Takes you through the creation of your plug-ins
• Two main phases:
• Panels creation (AUIML file)
• Execution method creation (JSDL)
Application
Plug-in
(jar file)
• After deployment of new Application Plug-in, you will be able to manage the new job type in the same fashion as all other job types creation
19 © 2013 IBM Corporation

20
 Integrate batch scheduling into business applications
 Batch becomes “built-in” rather than an afterthought
 Cloud service strategy on-demand
 Automate frequent ad-hoc job submission
 Variable Table Support
© 2013 IBM Corporation

21
•
Provide Development an embedded mobile device friendly interface for Self-
Service Catalog access
•On-demand 24x7 user/developer access to IT resources and information
• Workload Service Catalog
• Integration with any Service Desk application
•Eliminates log-on to OS
•Platform Agnostic end-2-end workload automation
© 2013 IBM Corporation

22
Submit Ad-Hoc Jobs
Value proposition

New WA provides a self-service front-end for business users to trigger and control “Their
Own ” ad-hoc tasks
Deliverable shall:
 Provides a web-based portal, also available on smart devices:
 Menu of pre-defined cataloged business services
 Easy-to-use self-service interface
 Control of end-to-end business service lifecycle
 Service Request
 Approval
 Notifications status
Business Value
 Streamline request of business services through easy-to-use online catalog
 Submit ad-hoc service from anywhere at anytime
© 2013 IBM Corporation

23
Real Time Application Status, process notifications, Alerts, job completion status, self initiated job restarts, take action
© 2013 IBM Corporation

«Clouds are a large pool of easily usable and accessible virtualized resources
(such as hardware, development platforms and/or services).
[Vaquero et al , 2009]
Elastic scaling is the ability to provision resources in response to changes in demand.
Scale up when demand increases
Scale down when demand subsides
© 2013 IBM Corporation

Elastic Scaling / Automated virtualized provisioning / Pooled Resources/Static
Schedule to the DDM, automate selection of system
 Load balancing within a pool

Automatic discovery of new resources

Optimize use of existing resources

Eliminate manual intervention to add and remove dynamic agent from a pool
 Proactive provisioning

SLA-driven Automation, resource allocation ondemand
 Ready for cloud and virtualized architecture integration
© 2013 IBM Corporation

New Infrastructure support and provisioning
Platforms supported:
 Distributed systems
 Linux on System Z
Business benefits
 Scalability
 Integrated Virtualization support
 Standardized access controls and user definitions
IT Administrator
 Workload Automation Execution service
 Ideal for dev/test applications which run for short time in month end (i.e. Payroll)
 Automatic provisioning (and deprovisioning) of a “WAready” execution environment in a cloud, to save resource costs, as well as installation and configuration costs
 Elastic scaling – Production workload assurance
 Tight SLAs with business penalities need to adjust the environment and avoid any miss
 Automatic scaling-up (and scaling-down) of WA environment to recover from errors or prevent SLA breaching
WA Administrator
Model, schedule, monitor and manage WA jobs
Enterprise
Scheduler
Create VM Templates
Define time-windows reqs
Define QoS/SLAs
Cloud Mgmt
Platform
Configure
Scheduler
Templates
Llibray
Provision environment
Monitor environment
Elastic scale
Deprovision
Servers pool
Reporting App
Accounts DB Database
© 2013 IBM Corporation

Workload Service Assurance:
Ensure End-User satisfaction
Customer Care
• Applications respond faster
• Fewer outages
Application Owners
• Peaks are absorbed, service levels are met
• Visibility to application health to achieve incremental improvements
Infrastructure
• Failovers can happen without loosing customer sessions
Customer’s Experience
• Sub second response time
• Mobile Device support
• Faster and better customer experience with every channel!
4/15/2020
Page 27

Organizations are now moving beyond virtualization to higher value stages of Cloud Computing
Virtualization underpins
Cloud
Cloud focuses on eased service consumption & management
Consumption based metering and dynamic capacity optimization
End-to-end real time monitoring and optimization
Business service catalogs & self service
Service delivery automation
Virtualization management
© 2013 IBM Corporation

29
Enhanced IT agility. Reduce cost and risk.

Efficiently manage workload in the cloud
 Improved LOB Application Services
 APM Performance & HA assurance - Production
 WA integration for optimized workloads, Process consolidation

Automated Resource Provisioning

Centralized EU self service support
© 2013 IBM Corporation

30
 Set of optional tooling, processes and best practices to transform existing assets and reusing them through modern technologies, so transforming
Cloud challenges are reality.
© 2013 IBM Corporation

© 2013 IBM Corporation