If you knew what I know
or
CloudWave - Improving services in the Cloud
through collaborative adaptation
Eliot Salant
salant@il.ibm.com
IBM Haifa Research
CloudWave Project Coordinator
Where things are today
Grid computing – resources allocated to nodes
Cloud Computing
Elasticity
Scale up
Scale out
So, what’s the problem?
Hint…
Evolution of software delivery
Runs cost money!
Fail early, fail cheaply
Extensive alpha and beta testing
Release times (wks) –
Windows vs. app
The DevOps paradigm
Development
Operations
“A large segment of DevOps tools delivers automation and
configuration to relax stress on developers and operators during
continuous delivery… but to support smooth operation data
analytics will need to step up to the plate.”
http://siliconangle.com/blog/2014/12/23/predictions-for-devops-in-2015-the-year-of-smart-devops/
How can the Cloud better
support DevOps-style
development AND adopt
DevOps concepts itself?
The CloudWave idea
Infrastructure behavior
Application behavior
CloudWave overview
www.cloudwave-fp7.eu
3 year project sponsored by the EU’s FP7
Just finished the first year
10 partner organizations
6.3 Meuro budget
Main project concepts
Infrastructure
monitoring
Application monitoring
FDD
Holistic Cloud
events db
Adaptation engine
Some CloudWave Challenges
Execution Analytics
•
•
•
•
Effective monitoring of infrastructure
Effective monitoring of applications
Event filtering, consolidation
Complex event processing
Coordinated Adaptation
• Machine Learning techniques
• Modeling adaptation scenarios
Feedback Driven Design
• Insights to aid developers
• Test plan evolution through analytics
Strategy
Research
Implementation
Use cases
Functional Decomposition
CW DevOp engineer
Feedback
Development Env
App changes
Deployment
Feedback
Runtime Environment
Configuration
Level 1 decomposition
Feedback
CloudWave Admin
Status visual.
Administration Env.
Administration
Terminology
OpenStack – Open Source cloud computing platform
Heat – Orchestration tool for deployment on OpenStack cloud
HOT – Heat Orchestration Template
Ceilometer – OpenStack resource monitoring tool
Enactment point – Sets the state of the application for adaptation
Concept
Monitoring data
OpenStack action
Enactment point definition
Enactment trigger
Application adaptation
request
User input
Physical machine
Application
and monitoring
environment
Application
and monitoring
environment
Cloud Stack
Mgr
Living State
Manager
Heat Engine
FDD
Monitoring
collection and
Analysis
Coordinated
Adaptation
CW Monitoring
Physical machine: Nova Compute Node
Application
code
OpenStack Controller Node
Application
logging
tools
CW probe
CW.so library
VM
Celiometer Collector
Celiometer Agent
CW
Pollister
Pollster
1…
Pollster
N
CWE
dispatcher
Mongo
db
Adding analytics
CelioEsper
Esper
Other CEP
Engines
Celiometer Collector
From monitoring
CWE
dispatcher
Mongo
db
OpenStack Controller Node
To Living State
Manager
Living State Manager
HOT++
CW Grunt
Heat Engine
Adaptation Engine
From Ceiloesper
All together now
Coordinated
Adaptation
Directions for Coordinated
Adaptation
• Machine learning to react to enactment
point triggers
• Adaptation of both infrastructure and
application
• Determination of new enactment points
Some challenges
• Multiple layers for adaptation
App
Middleware
Virtual machine
Physical machine
Data center
Cloud
Federated clouds
Coordinated adaptation
challenges
• Ultimate effect of adaptation actions at
different levels not always clear
• Sample set for machine learning
• Standardizing application adaptations
• …
Example of potential
coordinated adaptation
• Computations on a mobile phone vs. in
Cloud
• IoT devices – autonomy vs. centralized
control
Feedback Drive Design
• Better monitoring information and analysis
to help developers
• Analysis of Adaptation Engine efficiency
• Feedback driven testing
– Evolution of testing
• Problem recreation
FDD Challenges
• Effective feedback visualization
• Intelligent hints to developers (analysis)
• What-if analysis
In summary…