Modeling and Reasoning about
Software Systems Containing
Uncertainty and Variability
Marsha Chechik
University of Toronto
September 11, 2015
TASE’15, Nanjing, China
Acknowledgements
Many thanks for colleagues
in Toronto
and elsewhere in the world
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Michalis Famelis
Rick Salay
Julia Rubin
Alessio DiSandro
Yi Li
Benson Quach
Cynthia Disenfield
Andrej Wazowski
Aws Albarghouthi
Michal Antkewitz
Krzysztof Czarnecki
Levi Lucio
Gehan Selim
Gabrielle Taenzer
Daniel Struber
2
Why Models?
• Traditional Engineering Approach
– Abstract & Precise
– Amenable to analysis
– Complexity: Model << System
• Pre-development and pre-deployment analysis
– Early detection -> cheaper fixes
• Cost < Benefit
3
Software Engineering Models
Abstraction
Structure
Static Design
Behaviour
Architecture
Concepts
Requirements
Deployment
Use Cases
4
Model-Based Software Engineering
(MBSE) Toolbox (partial list!)
Support for:
Development,
analysis,
refinement,
transformation of models
+
Communication,
documentation
5
• Model-driven engineering under presence of
uncertainty
• Model-driven engineering under presence of
variability
• Uncertainty and variability combined?
• Some current/future work
6
Uncertainty
• Sources
• Modeling
• Verification of models containing uncertainty
– Technology
– Scalability / effectiveness
– Dealing with analysis results
• Other parts of MDSE toolbox
• Tool support
7
This is Natalie.
Natalie is a modeler.
Natalie faces uncertainty in her everyday work
8
Alternative Designs
Hmm, I don’t
know which
one, yet.
9
Conflicting Stakeholder Opinions
What do I do
until they
decide?
10
Incomplete Information
I don’t know
everything about
this, yet.
11
Uncertainty in Software Development
Many design alternatives
Incomplete information
Conflicting stakeholder
opinions
Uncertainty about the
content of the model.
12
MBSE Toolbox for Natalie?
Existing tools do not apply
Options:
– Wait until uncertainty gets resolved (how long?)
– Make a provisional decision and run with it (need backtracking!)
We propose:
– Defer resolution of uncertainty but incorporate
uncertainty handling into the development process
to allow progress
– Lift standard MDE tools to handle models with uncertainty
– Add new tools:
• Help Natalie express her uncertainty
• Help Natalie determine what she does not yet know
• Help Natalie manage and reduce uncertainty
Modeling Uncertainty: Example
Solver
Notation means “inner
class” in UML
Unsure if it
should be an
inner class.
SolverException
+ effect : String
Unsure if we need
this field.
14
Some Decisions to Make
Options for assigning values to states of confidence
Quantitative
Qualitative
Examples: 0, 0.3, 0.75
Examples: yes/no/don’t know
strongly
… strongly
agree
disagree
Good for capturing likelihood of events
Good for capturing one’s own level of
uncertainty
Hard to capture one’s own uncertainty
Less precise than quantitative
15
How to Represent Uncertainty?
Ask users
• Goals: model uncertainty explicitly, keep syntax familiar
[MiSE’13]
16
Representing Uncertainty
Solver
Unsure if it
should be an
inner class.
SolverException
Partial Models
• Points of uncertainty
(“May elements”)
explicated using
syntactic annotations
+ effect : String
Unsure if we need
this field.
[ICSE12]
17
Representing Uncertainty
Solver
X
SolverException
Partial Models
• Points of uncertainty
(“May elements”)
explicated using
syntactic annotations
+ effect : String
Y
Propositional variables:
“the element exists”
[ICSE12]
18
Representing Uncertainty
Solver
Solver
Solver
SolverException
SolverException
x=F, y=F
x=T, y=F
X
SolverException
+ effect : String
Y
Solver
SolverException
+ effect : String
x=F, y=T
4 concretizations: 4 ways to resolve uncertainty.
Solver
SolverException
+ effect : String
x=T, y=T
19
Representing Uncertainty
Solver
Partial Models
• Points of uncertainty
(“May elements”)
explicated using syntactic
annotations
X
SolverException
+ effect : String
Y
• Restrictions to the set of
concretizations can be
captured in the “May
formula”
XvY
20
Representing Uncertainty
Solver
Solver
Solver
SolverException
SolverException
x=F, y=F
x=T, y=F
X
SolverException
+ effect : String
Solver
SolverException
+ effect : String
Y
XvY
x=F, y=T
Solver
SolverException
+ effect : String
x=T, y=T
21
Partial Models
A partial model represents the entire set of
possible concrete models
• Benefits
– Partial model refinement = uncertainty reduction
– Formal: allows analysis and automation
– Semantics-independent – applies to models of different types
– Use instead of concrete models …
… to allow deferral of decisions without delaying progress
22
Uncertainty
• Sources
• Modeling
• Verification of models containing uncertainty
– Technology
– Scalability / effectiveness
– Dealing with analysis results
• Other parts of MDSE toolbox
• Tool support
23
Model Checking
Property P
Model M
Does M satisfy P?
yes
counterexample
the
property
holds!
Properties to Verify
Example property:
“Every inner class has at
least one attribute”
Class.allInstances()->forAll( c |
not c.nestedIn->isEmpty()
implies
not c.ownedAttributes->isEmpty() )
Natalie’s favorite
analysis
technique
How can you apply it to partial models?
25
Model Checking for Models with
Uncertainty
Property P
Partial Model M
Does M satisfy P?
yes
counterexample + concretizations
where property fails
the property
holds in all
concretizations!
Verification: the Naïve Approach
Example property:
“Every inner class has at least one attribute”
Enumerate:
27
Verification: Lifting
Example property:
“Every inner class has at least one attribute”
Property holds for…
…all
concretizations
…some but
not all
Natalie’s favorite
analysis technique,
LIFTED
…none
• Applies directly to the partial model
• Does not enumerate concretizations
• Computes result using three-valued logic
28
Verification: Lifting
• Checking syntactic properties [ICSE12]:
– Language independent
– SMT-based
29
SMT Encoding
;Concretizations
(define-sort NodeType () Int)
(define-sort Node () Int)
(declare-fun node (Node) Bool)
(declare-fun nodeType (Node) NodeType)
(define-sort EdgeType () Int)
(define-sort Edge () Int)
(declare-fun edge (Edge) Bool)
(declare-fun edgeType (Edge) EdgeType)
(declare-fun src (Edge) Node)
(declare-fun tgt (Edge) Node)
(declare-const CLASS NodeType)
(assert (= CLASS 1))
(declare-const ATTRIBUTE NodeType)
(assert (= ATTRIBUTE 2))
(declare-const OPERATION NodeType)
(assert (= OPERATION 3))
(declare-const DEPENDENCY EdgeType)
(assert (= DEPENDENCY 4))
(declare-const ASSOCIATION EdgeType)
(assert (= ASSOCIATION 5))
(declare-const NESTEDINREFERENCE EdgeType)
(assert (= NESTEDINREFERENCE 6))
(declare-const SUPERCLASSREFERENCE EdgeType)
(assert (= SUPERCLASSREFERENCE 7))
;Model
(declare-const Solver Node)
(assert (= Solver 8))
(declare-const SolverException Node)
(assert (= SolverException 9))
(declare-const Y Node)
(assert (= Y 10))
(declare-const X Edge)
(assert (= X 11))
;End Model
(assert (= (nodeType Solver) CLASS))
(assert (= (nodeType SolverException) CLASS))
(assert (= (nodeType Y) ATTRIBUTE))
(assert (= (edgeType X) NESTEDINREFERENCE))
(assert (= (src X) SolverException))
(assert (= (tgt X) Solver))
(assert (=>
(edge X)
(and (node SolverException) (node
Solver))
))
;Model is Complete
(assert
(forall ((c Node)) (=>
(node c)
(and
(>= c 8)
(<= c 10)
)
)))
(assert
(forall ((c Edge)) (=>
(edge c)
(and
(>= c 11)
(<= c 11)
)
)))
;Solver Exists
(assert (node Solver))
;SolverException Exists
(assert (node SolverException))
;May Formula
(assert (or X Y))
30
Verification: Lifting
• Checking syntactic properties [ICSE12]:
– Language independent
– SMT-based
• Checking semantic properties:
– Depends on language semantics
– Requires custom adaptation of verification technique
– Examples:
• Goal satisfaction in i* models [RE12, RE14]
• MTSA: Model checking and other analyses over MTSs [ASE08]
31
Scalability of Verification
• Q: Is verification of models with uncertainty feasible?
– How does it compare with checking each model individually?
– What about comparison with the classical verification?
• Q: How does the level of uncertainty affect this
feasibility?
Experiments: with random inputs and real case studies
Random inputs:
varying size of the model and level of uncertainty
speedup = time for set of conventional models
time to check partial model
32
Results
33
Results of Evaluation
Reasoning with Partial models
vs
Reasoning with a set of conventional models
• Is there a speedup?
- Yes, it is consistently faster than reasoning with the set.
• How is speedup affected by changing model
size and levels of uncertainty?
- Speedup decreases with model size
- Speedup increases with level of uncertainty
- No slowdowns!
36
Uncertainty
• Sources
• Modeling
• Verification of models containing uncertainty
– Technology
– Scalability / effectiveness
– Dealing with analysis results
• Other parts of MDSE toolbox
• Tool support
37
Analysis Results
Solver
Class.allInstances()->forAll( c |
not c.nestedIn->isEmpty()
implies
not c.ownedAttributes->isEmpty() )
Show user an example:
SolverException
+ effect : String
-- A concretization where the
property holds
Analysis Results
Solver
Class.allInstances()->forAll( c |
not c.nestedIn->isEmpty()
implies
not c.ownedAttributes->isEmpty() )
Show user a counterexample:
SolverException
+ effect : String
-- A concretization where the
property does not hold
Using Analysis Results
Solver
X
SolverException
+ effect : String
Class.allInstances()->forAll( c |
not c.nestedIn->isEmpty()
implies
not c.ownedAttributes->isEmpty() )
Generate the diagnostic core:
-- Partial model of concretizations
where property does/does not hold
Tool Support for Responding to
Analysis
Greyed out: not
part of the
diagnostic core
Part of the core
Tool Support for Responding to
Analysis
Offending May
elements
removed
Uncertainty
• Sources
• Modeling
• Verification of models containing uncertainty
– Technology
– Scalability / effectiveness
– Dealing with analysis results
• Other parts of MDSE toolbox
• Tool support
44
Uncertainty-Reducing Refinement
• Systematically incorporate
new information…
– … Manually
Unsure if it
should be an
inner class.
• Edit the uncertainty
annotations
Unsure if we
need this
field.
– … Using automated
transformations
XvY
I want to get rid of
effect.
Verify that a transformation
is always uncertainty
reducing.
[FASE12, JOT15]
45
Transformations
Like every
good MBE
practitioner,
Natalie uses
a variety of MTs
The transformations assume
inputs that don’t contain
uncertainty
But Natalie’s models do!
[MODELS13]
46
Transforming Models with Uncertainty
Natalie should be able
to use model
transformations
[MODELS13]
47
Transforming Models with Uncertainty
Natalie should be able
to use model
transformations
Existing transformation techniques
do not support this!
We need to lift Natalie’s transformations so that they
can apply to models with uncertainty
[MODELS13]
48
Goal: “Lifting”
class1
class2
class1
+ attribute : type
class1
NAC
- attribute : type
+ getAttribute():type
LHS
RHS
Solver
Solver
SolverException
+ effect : String
Solver
SolverException
-effect : String
+getEffect() : String
Solver
SolverException
SolverException
Solver
Solver
SolverException
+ effect : String
SolverException
+ effect : String
49
Goal: “Lifting”
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
Solver
SolverException
+ effect : String
Solver
X
SolverException
SolverException
Solver
SolverException
+ effect : String
a + - effect :YString
b +getEffect() : String
( X∧ ¬Y ∧ ¬a ∧ ¬b) v
(¬X∧ Y ∧ ¬a ∧ b) v
( X∧ Y ∧ a ∧ ¬b)
50
Goal: “Lifting”
class1
class2
class1
+ attribute : type
class1
NAC
Solver
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
X
X
SolverException
SolverException
+ effect : String
Y
XvY
a + - effect :YString
b +getEffect() : String
( X∧ ¬Y ∧ ¬a ∧ ¬b) v
(¬X∧ Y ∧ ¬a ∧ b) v
( X∧ Y ∧ a ∧ ¬b)
51
Lifting Technique Summary
Step 1:
Determine applicability (Henshin + Z3 SMT Solver)
Step 2:
Transform graph (Hanshin)
Step 3:
Transform formula (Java + Z3 input strings)
Solver
Solver
X
X
SolverException
SolverException
+ effect : String
Y
XvY
a + - effect :YString
b +getEffect() : String
( X∧ ¬Y ∧ ¬a ∧ ¬b) v
(¬X∧ Y ∧ ¬a ∧ b) v
( X∧ Y ∧ a ∧ ¬b)
52
Making the Toolbox...
MMINT: Model Management INTeractive [MODELS15]
–
–
A platform for developing (typed) model management
tools
Model management environment:
•
Diff, refine, merge, etc.
Available at https://github.com/adisandro/MMINT
CSP
SAT/SMT
Henshin
Alloy
53
Making the Toolbox... Uncertaintyaware
MMINT: Model Management INTeractive
–
–
A platform for developing (typed) model management tools
Model management environment:
•
Diff, refine, merge, etc.
Available at https://github.com/adisandro/MMINT
MU-MMINT [ICSE15] – lifting of model management operators + new
ones
MU-MMINT
(pronounced
“Moomin”)
CSP
SAT/SMT
Henshin
Alloy
54
Summary: Uncertainty Modeling and
Analysis
• Our choice:
– Qualitative, possibilistic
uncertainty modeling
– Keeping concrete syntax
familiar
– Encoding an exact set of
potential classical models
– Refinement – reducing the
set of possibilities
– Verification – lifting of
classical approaches
Explicit uncertainty modeling
with Partial Models + May
formulas
Partial model
Concretizations
55
• Model-driven engineering under presence of
uncertainty
• Model-driven engineering under presence of
variability
• Uncertainty and variability combined?
• Some current/future work
56
Variability
• Context
• Modeling
• Verification of models containing variability
– Technology
– Scalability / effectiveness
– Dealing with analysis results
• Other operators for handling variability
57
Product Lines
• Goal: Help develop, manage, reuse a large number
of similar but different artifact variants (products)
• Example: Washing Machine Co.
…
58
Software Product Line Engineering
a discipline that promotes planned and
predictive software reuse
K. Pohl et al., Software Product Line Engineering:
Foundations, Principles, and Techniques, 2005
P. C. Clements and L. Northrop, Software
Product Lines: Practices and Patterns, 2001
59
Modeling: Product Line Structure +
Terminology
• Product line (annotative) represented by
– Domain Model – combined parts from all products, annotated by
features (presence conditions)
– Feature Model – shows possible features and restrictions for product
combinations
• Example : Washing Machine Co.
Domain Model
Feature Model
Heat
excludes
Wash
Delay
Dry
60
Product Derivation (or Configuration)
• The process of deriving a specific software
product from:
– … artifacts with variability
– … feature to artifacts association
– … a product definition
61
Product Configuration – Example 1
• +Heat product
– Feature configuration: {Wash, Heat}
Feature Model
Heat
excludes
Wash
Delay
Dry
62
Product Configuration – Example 2
• +Dry/Delay product
– Feature configuration: {Wash, Dry, Delay}
Feature Model
Heat
excludes
Wash
Delay
Dry
63
Washing Machine
Product Line
Feature Model
Heat
excludes
Presence Conditions
Domain Model
Wash
Heat
Locking
Heat
Delay
Delay
[heatEnabled;delayEnabled]
Heat ∨ Delay
/ HeaterOn() Heat
Waiting
Dry
Heat ∨ Delay
Heat ∨ Delay
Delay
/ HeaterOff(); Heat
wash.Start(); Heat ∨ Delay
[not heatEnabled;not delayEnabled]/
wash.Start();
Heat
/ QuickCool()
Unlocking
¬Dry
Washing
entry/TempCheck() Heat
Dry
Dry
/ QuickCool()
Dry
Dry
Drying
64
Washing Machine Product
Line: Configuring a Product
Domain Model
Heat
Heat
excludes
Wash
Heat
Locking
Feature Model
Delay
Delay
[heatEnabled;delayEnabled
Heat ∨ Delay
]/ HeaterOn() Heat
Waiting
Dry
Heat ∨ Delay
Heat ∨ Delay
Delay
/ HeaterOff(); Heat
wash.Start(); Heat ∨ Delay
[not heatEnabled;not
delayEnabled]/ wash.Start();
Heat
/ QuickCool()
Unlocking
¬Dry
Washing
entry/TempCheck() Heat
Dry
Dry
/ QuickCool()
Dry
Dry
Drying
65
Result: +Dry/Delay State Machine
+Dry/Delay Variant
[
delayEnabled]
Waiting
Locking
[
not delayEnabled]/
wash.Start();
/
wash.Start();
Washing
Unlocking
/ QuickCool()
Drying
66
Result: +Dry/Delay State Machine
+Dry/Delay Variant
[delayEnabled]
Waiting
Locking
[not delayEnabled]/
wash.Start();
/wash.Start();
Washing
Unlocking
/ QuickCool()
Drying
67
Variability
• Context
• Modeling
• Verification of models containing variability
– Technology and its scalability / effectiveness
– Dealing with analysis results
• Other operators for handling variability
68
Model Checking
Property P
Model M
Does M satisfy P?
yes
counterexample
the
property
holds!
Model Checking for Product Lines
Product Line
Model M
Property P
Does M satisfy P?
yes
counterexample + a list of
violating products
the property
holds for all
products!
Methods for Checking Systems with
Variability
• Input: typically featured transition systems and LTL
properties
– “Don’t crash into a wall.”
¬ overlap(vehicle, wall)
– “Each request is answered.”
 (request ⇒ response)
• Methods:
– Special-purpose model-checker [Classen – ICSE10]
– Transformation to ..
• … Spin [Classen – STTT12]
• … NuSMV [Classen – ICSE11]
• … IC3 [Ben-David – ICSE15]
71
Dealing with Analysis Results
• Model-checking returns a list
of violating products
• Methods:
– Remove the offending products
(easy to automate)
– Debug and reverify to ensure
that property holds for all
products (no specific support)
…
72
Variability
• Context
• Modeling
• Verification of models containing variability
– Technology
– Scalability / effectiveness
– Dealing with analysis results
• Other operators for handling variability
73
74
Lifting Transformations
[ICSE14]
Other PL Operations
[SPLC13, STTT15]
75
Feature Location and Migration Across
Products
Given a history (sequence of commits) and a
functionality exercised by tests, find a “semantic slice”
that is
well-formed
and can pass the same tests
Locating features in cloned variants
[ASE12, FASE14, ongoing]
[ASE15]
76
Detecting and Resolving Feature
Interaction
• Goal: compositional detection and
resolution (using priorities) of behavioral
interference properties between
features
• Given a selection of features,
understanding of the spec of the
resulting system
[In progress]
77
Summary: Variability Modeling and
Analysis
• Product lines – an industrially-relevant method
for compactly encoding sets of similar but
different products
– Difference is explicated using features
• Verification of product lines:
– Determine in which products a property fails
– Resolution – reduction of variability or ad-hoc fixing of
products
• Other PL operations:
– “lifted” versions of classical
– … or special-purpose
Product Line
Products
78
• Model-driven engineering under presence of
uncertainty
• Model-driven engineering under presence of
variability
• Uncertainty and variability combined?
• Some current/future work
79
Two common kinds of choices in SE
Uncertainty
Variability
Reason
Incomplete information, design
alternatives, stakeholder
conflicts, etc.
Market demands for product
variants
Granularity
Decisions
Features
Expression
Partial models
Product line (PL) models
Semantics
Set of artifacts produced by
combinations of decisions
Set of artifacts produced by
combinations of features
80
Goal
Enable modeling and analysis for design-space
exploration of different product lines
i.e., given a space of product lines, which one should
be selected (and why)?
Decision
combinations
produce
possible PLs
𝑃𝑟𝑜𝑑1
𝑃𝐿1
𝑃𝐿2
⋱𝑃𝐿𝑚
𝑃𝑟𝑜𝑑2
…
Feature combinations produce
possible products
𝑃𝑟𝑜𝑑𝑛
81
Uncertainty + Variability
• Sources
• Modeling
• Verification of models containing uncertainty
– What are relevant properties?
– How to check these properties?
– Dealing with property violations
• Current Status
82
Uncertainty in a Washing
Machine Product Line
Decision point Mutex
Not sure whether Heat and Delay
Domain Model
areHeat
mutually exclusive.
Delay
[heatEnabled;delayEnabled]
Heat ∨ Delay
/ HeaterOn() Heat
Waiting
Locking
Feature Model
Heat
excludes
Wash
Delay
Dry
Heat ∨ Delay
Heat
Heat ∨ Delay
Delay
/ HeaterOff(); Heat
wash.Start(); Heat ∨ Delay
[not heatEnabled;not delayEnabled]/
wash.Start();
Heat
/ QuickCool()
Unlocking
¬Dry
Washing
entry/TempCheck() Heat
Dry
Dry
/ QuickCool()
Dry
Drying
Dry
Decision point HasSpin
Not sure whether to put a guard
NoSpin on transition. 8383
A Design Space of Product Lines
Decision combinations
produce possible PLs
e.g., {¬Mutex, HasSpin}
𝑃𝐿1
𝑃𝐿2
⋱ 𝑃𝐿𝑚
Goal: Use properties to explore the design space of
PL’s
𝑃𝑟𝑜𝑑1
𝑃𝑟𝑜𝑑2
…
Feature combinations produce possible products
e.g., {¬Delay, Heat, Dry}
𝑃𝑟𝑜𝑑𝑛
84
Constraining the Design Space
using Properties
• For a product property P
– Use All for critical properties and Some for
desirable properties
– Use Necessary when you are sure it is needed and
Possible when unsure but don’t want to exclude
the possibility
Necessary
for product line
Possible
for product line
All
products have P
All products in
All product lines
All products in
Some product line
Some
products have P
Some product in
All product lines
Some product in
Some product line
85
Necessary-Some
All
product lines
Some
product
…
86
Possible-Some
Some
product line
Some
product
…
87
Possible-All
Some
product line
All
products
…
88
Necessary-All
Example Prop: State Unlocking must always be reached
- a washing machine that violates this is unacceptable
All
product lines
All
product
Example Analysis
Check: does prop hold?
…
89
Uncertainty in a Washing
Machine Product Line
Decision point Mutex
Not sure Heat and Delay are
Domain Model
mutually
exclusive.
Heat
Delay
[heatEnabled;delayEnabled]
NO!
Heat ∨ Delay
/ HeaterOn() Heat
Waiting
Locking = 𝑡𝑟𝑢𝑒 and 𝐷𝑟𝑦 = 𝑡𝑟𝑢𝑒
If 𝐻𝑎𝑠𝑆𝑝𝑖𝑛
then
Feature Model
Heat
excludes
Wash
Delay
Dry
Heat ∨ Delay
Heat ∨ Delay
Heatguard may
Delayprevent state Unlocking
the
/ HeaterOff();
[not heatEnabled;not delayEnabled]/
fromwash.Start();
being reached.
wash.Start();
Heat
/ QuickCool()
Unlocking
¬Dry
Heat
Heat ∨ Delay
Washing
entry/TempCheck() Heat
Dry
Dry
/ QuickCool()
Dry
Drying
Dry
Decision point HasSpin
Not sure whether to put guard
NoSpin on transition. 9090
Responses to Property Violation
Necessary-All?

…
92
Response 1: Relax Constraint
Necessary-All?
Possible-All?

…
93
Response 2: Reduce Uncertainty
Necessary-All?

Reduce
Uncertainty
e.g., decide 𝐻𝑎𝑠𝑆𝑝𝑖𝑛 = 𝑓𝑎𝑙𝑠𝑒
…
94
Response: Reduce Uncertainty
Necessary-All?

…
95
Response: Reduce Variability
Necessary-All?

…
Reduce Variability
e.g., remove feature 𝐷𝑟𝑦
96
Response 3: Reduce Variability
Necessary-All?

…
97
Implementation and Status
• Implementation
– Proof of concept implementation in Clafer for
property checking and feedback generation
– Exploring Higher-Order Alloy*
• Current Work
– Evaluating scalability of the tools and effectiveness of
the methodology
– Better automation of response strategies
– Case studies
• Power Window
• “Real” Washing Machine
98
Summary: Uncertainty + Variability
• Uncertainty and Variability are similar but different
• They can be used effectively together to explore SPL
design space
• We defined:
– Four natural classes of properties that can be checked
– Responses to property violations
Uncertainty
Partial model
Variability
Concretizations
Product Line
Products
99
• Model-driven engineering under presence of
uncertainty
• Model-driven engineering under presence of
variability
• Uncertainty and variability combined?
• Some current/future work
100
Understanding Sources of Uncertainty:
Mining Developer Conversations
Mine conversations
Gather Analytics
Generate Action
Recommendations
[RSSE14]
101
Understanding Sources of Uncertainty:
Mining Developer Conversations
• Identify unique Points of
Uncertainty from question
utterances
• Associate them with Proposed
Alternatives
• Track the stakeholders’ arguments
• Recognize the Resolution of
Uncertainty
[RSSE14]
102
Variability-Aware Transformations
Problem:
Graph-based transformation
Lots of similar yet somewhat different rules
How to represent these rules so that the transformation
can be done efficiently (e.g., save time on matching to
determine applicability)
How to produce variability-aware rule sets from lots of
rules
[FASE15, additional work in progress]
75
Lifting Code Analyses
A terrific body of work by Christen Kaestner on reinterpreting
various code analyses – one at a time- on 150% code models
(with #ifdefs)
Problem:
Given an analysis method on programs, reinterpret it on
150% representations, together with proofs of correctness (that
the method gives correct analysis on each variant)
Specifically, trying to lift analysis behind UFO [CAV12]
a combination between over- and under-approximation
[in progress]
76
Representation of Sets
Uncertainty
Partial model
Metamodels
Concretizations
Product Lines
Product Line
Metamodel
Instances
Megamodels
Products
Megamodel
Members
105
Support for Megamodeling
• Model management for
collection of models
• Additional operations:
– Map
– Reduce
– Filter
[MODELS15]
106
To Summarize
107
References, part 1
108
[MiSE13] Michalis Famelis, Stephanie Santosa: MAV-Vis: Notation for Model Uncertaint. MiSE13 @
ICSE13.
[ICSE12] Michalis Famelis, Rick Salay, Marsha Chechik: Partial models: Towards modeling and reasoning
with uncertainty. ICSE 2012: 573-583
[RE12] Rick Salay, Marsha Chechik, Jennifer Horkoff: Managing requirements uncertainty with partial
models. RE 2012: 1-10
[RE14] Jennifer Horkoff, Rick Salay, Marsha Chechik, Alessio Di Sandro: Supporting early decision-making
in the presence of uncertainty. RE 2014: 33-42
[ASE08] Nicholas D’Ippolito, Dario Fishbein, Marsha Chechik, Sebastian Uchitel: MTSA: The Modal
Transition System Analyzer. ASE 2008: 475-476
[FASE12] Rick Salay, Michalis Famelis, Marsha Chechik: Language Independent Refinement Using Partial
Modeling. FASE 2012: 224-239
[JOT15] Rick Salay, Marsha Chechik, Michalis Famelis: A Methodology for Verifying Refinements of Partial
Models. Journal of Object Technology 2015.
[MODELS13] Michalis Famelis, Rick Salay, Alessio Di Sandro, Marsha Chechik: Transformation of Models
Containing Uncertainty. MoDELS 2013: 673-689
[ASE12] Julia Rubin, Marsha Chechik: Locating distinguishing features using diff sets. ASE 2012: 242-245
[FASE14] Daniel Strüber, Julia Rubin, Gabriele Taentzer, Marsha Chechik: Splitting Models Using
Information Retrieval and Model Crawling Techniques. FASE 2014: 47-62
[ASE15] Yi Li, Julia Rubin, Marsha Chechik: Semantic Slicing of Software Version Histories: ASE 2015. to
appear.
[MODELS15] Rick Salay, Sahar Kokaly, Alessio Di Sandro, Marsha Chechik: Enriching Megamodel
Management with Collection-Based Operators. MODELS 2015 to appear
108
109
References, Part 2
[CAV12] Aws Albarghouthi, Yi Li, Arie Gurfinkel, Marsha Chechik: Ufo: A Framework for Abstraction- and
Interpolation-Based Software Verification. CAV 2012: 672-678
[FASE15] Daniel Strüber, Julia Rubin, Marsha Chechik, Gabriele Taentzer: A Variability-Based Approach to
Reusable and Efficient Model Transformations. FASE 2015: 283-298
[ICSE15] Michalis Famelis, Naama Ben-David, Alessio Di Sandro, Rick Salay, Marsha Chechik: MUMMINT: An IDE for Model Uncertainty. ICSE (2) 2015: 697-700
[MODELS15Tool] A. Di Sandro, M. Famelis, R. Salay, S. Kokaly, M. Chechik. MMINT: A Graphical Tool for
Interactive Model Management: MODELS 2015 Demos, to appear
[SPLC13] Julia Rubin, Krzysztof Czarnecki, Marsha Chechik: Managing cloned variants: a framework and
experience. SPLC 2013: 101-110
[STTT15] Julia Rubin, Krzysztof Czarnecki. Cloned Product Variants: From Ad-Hoc to Managed Software
Product Lines. STTT 2015.
[ICSE14] Rick Salay, Michalis Famelis, Julia Rubin, Alessio Di Sandro, Marsha Chechik: Lifting model
transformations to product lines. ICSE 2014: 117-128
[RSSE14] Ahmed Shah Mashiyat, Michalis Famelis, Rick Salay, Marsha Chechik: Using developer
conversations to resolve uncertainty in software development: a position paper. RSSE@ICSE 2014: 1-5
[Classen-ICSE10] Andreas Classen, Patrick Heymans, Pierre-Yves Schobbens, Axel Legay, Jean-François
Raskin: Model checking lots of systems: efficient verification of temporal properties in software product
lines. ICSE (1) 2010: 335-344
[Classen-ICSE11] Andreas Classen, Patrick Heymans, Pierre-Yves Schobbens, Axel Legay: Symbolic
model checking of software product lines. ICSE 2011: 321-330
[Classen-STTT12] Andreas Classen, Maxime Cordy, Patrick Heymans, Axel Legay, Pierre-Yves
Schobbens: Model checking software product lines with SNIP. STTT 14(5): 589-612 (2012)
[Ben-David-ICSE15] Shoham Ben-David, Baruch Sterin, Joanne M. Atlee, Sandy Beidu: Symbolic Model
109
Checking of Product-Line Requirements Using SAT-Based Methods. ICSE (1) 2015: 189-199
110
Uncertainty-Reducing Refinement
• Systematically incorporate
new information
Unsure if it
should be an
inner class.
Two options:
• Manual
Unsure if we
need this
field.
– Edit the uncertainty
annotations [FASE’12]
XvY
You know what? I
want to get rid of
effect.
• Using automated
transformations
– How do you verify that a
transformation is always
uncertainty reducing?
[VOLT’12]
111
Analysis Example
• P1: State Unlocking must always be reached
– Mandatory, Certain
• Check P1 on washing machine product line WM
– Is query ∀𝑈1, 𝑈2 ∀𝐻𝑒𝑎𝑡, 𝐷𝑒𝑙𝑎𝑦, 𝐷𝑟𝑦 (𝑊𝑀 ⇒ 𝑃1) true?
– No! If U2 = True and Dry feature is selected then the
guard may prevent Unlocking from being reached.
• Possible corrective actions:
– Resolve uncertainty U2 to False
– Modify state machine to have alternate path to Unlocking
74
Tool Support for Responding to
Analysis
Offending May
elements
removed
Tool Support for Responding to
Analysis
Greyed out: not
part of the
diagnostic core
May elements
part of the core
Reminder: Model Transformations
With Graph Rewriting (“Classical”)
Example rule:
class2
EncapsulateVariable refactoring:
Make fields private and add getter methods
unless they belong to some inner class
class1
+ attribute : type
class1
Negative
Application
Condition
Left
Hand
Side
class1
- attribute : type
+ getAttribute() :type
Right
Hand
Side
115
Graph Rewriting: Example Input Model
class2
class1
class1
- attribute : type
+ getAttribute() :type
+ attribute : type
class1
NAC
LHS
RHS
Match
SolverException
Solver
+ effect : String
116
Graph Rewriting: Example Input Model
class2
class1
class1
+ attribute : type
class1
NAC
LHS
NAC also matches!
- attribute : type
+ getAttribute() :type
RHS
ABORT !
SolverException
Solver
+ effect : String
117
Graph Rewriting: Example Input Model
2
class2
class1
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute() :type
RHS
Match
SolverException
+ effect : String
118
Graph Rewriting: Example Input Model
2
class2
class1
class1
+ attribute : type
class1
LHS
NAC
- attribute : type
+ getAttribute() :type
RHS
Delete
SolverException
+ effect : String
119
Graph Rewriting: Example Input Model
2
class2
class1
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute() :type
RHS
Add
SolverException
- effect : String
120
Graph Rewriting: Example Input Model
2
class2
class1
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute() :type
RHS
Add
SolverException
- effect : String
+ getEffect() : String
121
Graph Rewriting: Example Input Model
2
class2
class1
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute() :type
RHS
No more LHS
matches.
SolverException
Stop.
- effect : String
+ getEffect() : String
122
And Now: Transforming Models With
Uncertainty
Natalie wants to apply
the rule to an input
with uncertainty
123
Why Transforming Models with
Uncertainty is Hard?
class1
class2
class1
- attribute : type
+ getAttribute() :type
+ attribute : type
class1
LHS
NAC
RHS
Match???
Solver
X
SolverException
+ effect : String
XvY
Y
124
Why Transforming Models with
Uncertainty is Hard?
class1
class2
class1
- attribute : type
+ getAttribute() :type
+ attribute : type
class1
LHS
NAC
RHS
Match???
Solver
X
SolverException
+ effect : String
XvY
Y
125
Why Transforming Models with
Uncertainty is Hard?
class1
class2
class1
- attribute : type
+ getAttribute() :type
+ attribute : type
class1
LHS
NAC
RHS
Should
we add?
Should we delete?
Solver
X
SolverException
+ effect : String
XvY
Y
126
Why Transforming Models with
Uncertainty is Hard?
class1
class2
class1
- attribute : type
+ getAttribute() :type
+ attribute : type
class1
LHS
NAC
RHS
Existing transformation
techniques cannot be used.
Solver
X
SolverException
+ effect : String
XvY
Y
127
Goal: “Lifting”
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
X
SolverException
+ effect : String
Y
XvY
?
128
Goal: “Lifting”
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
SolverException
+ effect : String
Solver
SolverException
Solver
SolverException
?
+ effect : String
129
Lifting Technique
Step 1:
Determine
applicability
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
(a) Find Match
X
SolverException
+ effect : String
Y
XvY
130
Lifting Technique
Step 1:
Determine
applicability
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
(a) Find Match
X
SolverException
+ effect : String
Y
XvY
(b) Make sure the
rule applies to
at least one
concretization
(requires solving
a SAT problem)
131
Lifting Technique
Step 1:
Determine
applicability
Step 2:
Transform graph
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
Solver
X
X
(a) Copy over unchanged
parts
SolverException
+ effect : String
Y
XvY
SolverException
+ effect : String
Y
132
Lifting Technique
Step 1:
Determine
applicability
Step 2:
Transform graph
class1
class2
class1
+ attribute : type
class1
NAC
LHS
- attribute : type
+ getAttribute():type
RHS
Solver
Solver
X
X
(a) Copy over unchanged
parts
SolverException
+ effect : String
Y
XvY
(b) Perform additions and
deletions
SolverException
a + - effect :YString
b +getEffect() : String
Added and deleted
elements become Maybe
133
Lifting Technique
Step 1:
Determine
applicability
Step 2:
Transform graph
Step 3:
Transform formula
class1
class2
class1
+ attribute : type
class1
NAC
LHS
SolverException
+ effect : String
Y
XvY
RHS
Solver
Solver
X
- attribute : type
+ getAttribute():type
Constrain Maybe elements
to ensure each that
concretization
is correctly affected.
X
SolverException
a + - effect :YString
b +getEffect() : String
( X∧ ¬Y ∧ ¬a ∧ ¬b) v
(¬X∧ Y ∧ ¬a ∧ b) v
( X∧ Y ∧ a ∧ ¬b)
134
Transforming Models with Uncertainty
Natalie should be able
to use model
transformations
Existing transformation techniques
do not support this!
We need to lift Natalie’s transformations so that they
can apply to models with uncertainty
[MODELS13]
135