System S Project | IBM Research
Building User-defined Runtime
Adaptation Routines for
Stream Processing Applications
Gabriela Jacques-Silva, Bugra Gedik,
Rohit Wagle, Kun-Lung Wu,
Vibhore Kumar
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Outline
 Motivation
 Challenge for orchestration
 Building a management policy
 Use case
2
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
In System S, users build streaming applications as
data flow graphs
composite
operator
stream
connection
processing element (PE)
3
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Streaming applications have single deployment
mode
submit
application
Application
Manager
start PEs
4
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Users want adaptability, but currently this requires
ad-hoc solutions
submit
application
Application
Manager
start PEs
sleep, cat,
grep, cut, …
5
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Orchestrator controls the life-cycle of an
application via event handling
submit
orchestrator
onStart:
Application
Manager
submitJob(MyApp);
orchestrator
onOperatorMetric:
submit
application
…
Use cases
onPEFailure:
- Recovery coordination on operator …
onUserEvent:
failures
- Operator adaptation due to metric …
measurement values
- Incremental job deployment based on
runtime learning
6
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Events and actions occur at the physical level, but
users are familiar with logical view
Logical view
Physical view
Host A
op4’
op1
op3’
op4’
op6’
op1
op5’
composite1
op3’
PE 1
op5’
Host B
PE 4
op2
op4’’
op6’’
op3’’
op6’
op2
op4’’
op2
PE 3
op5’’
composite2
op2
op6’’
op3’’
PE 2
op5’’
Orchestrator maintains an in-memory
application representation to disambiguate
logical and physical views when necessary.
7
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Developers register events of interest based on the
logical view of the application
void MyOrca::handleOrcaStart(
const OrcaStartContext & context) {
PEFailureScope greenS(“green”);
PEFailureScope blueS(“blue”);
greenS.addCompositeInstanceFilter(“sample::greenComp”);
_orca->registerEventScope(greenS);
blueS.addOperatorInstanceFilter(“sample::blueOp”);
_orca->registerEventScope(blueS);
id = submitJob(sample::MyMainComp);
}
8
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
On event delivery, we provide a context to
understand logical and physical views
void MyOrca::handlePEFailure(
const PEFailureContext & context)
if(context.hasScope(“green”)) {
list<PEID> pes = new list<PEID>();
_orca->getPEsWithCompositeInstance(
pes, “sample::greenComp”);
_orca->restartPEs(pes);
} else {
_orca->executeCommand(“./myRecoveryHelper”);
_orca->restartPE(context.getPEId());
}
}
9
PE failure context:
- Matched scopes
- Application name
- Job ID
- PE ID
- Reason
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Orchestrator allows management of multiple
streaming applications
-
10
Dependency relations
Uptime requirements
Garbage collectable
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Developer configures application requirements and
specifies its dependency
void MyOrca::handleOrcaStart(
T, 60
10.0
T, 60
const OrcaStartContext & context) {
AppConfig fb(“Facebook”, “sample::Facebook”,
true, 60.0);
AppConfig tw(“Twitter”, “sample::Twitter”,
F, 0
10.0
false);
AppConfig sc(“Social”, “sample::SNetwork”,
true, 60.0);
_orca->registerDependency(sc, fb, 10.0);
_orca->registerDependency(sc, tw, 10.0);
_orca->submitJob(sc);
}
11
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Orchestrator starts applications according to their
dependencies and uptime requirements
T, 60
10.0
T, 60
void MyOrca::handleOrcaStart(
const OrcaStartContext & context) {
…
_orca->registerDependency(sc, fb, 10.0);
F, 0
10.0
_orca->registerDependency(sc, tw, 10.0);
_orca->submitJob(sc);
}
12
Orchestrator log
-10:00:00 AM – “Facebook” submitted
-10:00:02 AM – “Twitter” submitted
-10:00:12 AM – “Social” submitted
© 2012 IBM Corporation – All Rights Reserved
System
S Project | IBM Research
Unused
applications
are automatically cancelled
according to their configurations and
dependencies
T, 60
10.0
T, 60
void MyOrca::handleSomeEvent(
const SomeEventContext & context) {
…
_orca->cancelJob(sc);
F, 0
10.0
}
Orchestrator log
-11:00:00 AM – “Social” cancelled
-11:01:00 AM – “Facebook” cancelled
13
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Example demonstrator uses custom operator
metrics to adapt a Sentiment Analysis applications
Sentiment
Analysis
Product
Cause
Sentiment
iPhone
screen
-
iPhone
AppStore
+
iPhone
iPhone
Model
computation
14
iPhone
?
camera
?
-
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Orchestrator helped streaming application to adapt
to the incoming data distribution
Unknown to known cause ra o
1.4
1.2
Actuation threshold
1.0
MapReduce
job submitted
0.8
0.6
New model
loaded
0.4
new stream of
tweets injected
0.2
0.0
0
50
100
150
200
250
300
350
400
Epoch
15
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Summary
 System S hides the complexity of parallel and
distributed systems
– Orchestrator must open this abstraction to allow
appropriate event handling and actuation
 APIs for event registration based on logical view
 Event delivery includes context to understand
physical view
 Future work
– Add orchestrator concepts to SPL
– Use static analysis to automatically infer event scope
16
© 2012 IBM Corporation – All Rights Reserved
System S Project | IBM Research
Building User-defined Runtime
Adaptation Routines for
Stream Processing Applications
Gabriela Jacques-Silva, Bugra Gedik,
Rohit Wagle, Kun-Lung Wu,
Vibhore Kumar
© 2012 IBM Corporation – All Rights Reserved