Trust analysis
Atle Refsdal
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 1
Atle Refsdal
The goal
• Facilitate model based risk analysis of systems
whose behavior depends on trust
• We will present a method and a language for
performing such analysis
- Subjective STAIRS
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 2
Atle Refsdal
Overview
• Model based risk analysis
• Examples of systems whose behavior depends
on trust
• Definition of trust
• Example case
- Introduce and illustrate the method and the language
• Conclusion
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 3
Atle Refsdal
Models
• A model is a representation of a system
- Captures important aspects of the system
- Simplifies or omits the rest
• Refinement techniques allow us to present
models at different levels of abstraction
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 4
Atle Refsdal
Model based risk analysis
• Risk analysis: A systematic process for assessing
a system in order to identify, evaluate, and
mitigate risk
- Typically led by a risk analysis expert
- Different kinds of stakeholders typically take part in the
analysis: end-users, decision makers, engineers,...
• In model based risk analysis, models serve as
- input to the analysis, and/or
- documentation of analysis results (e.g. risk scenarios)
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 5
Atle Refsdal
Systems whose behavior depends on
trust
• Actors make choices based on their trust in other
actors/entities
- Based on incomplete information about the actors/entities
• Downloading software
- An employee decides whether to download software on the
company computer
• Access control
- An employee decides whether to open the door for someone who
claims to have forgotten his/her entrance card
• Granting of loans in a local bank
- A bank employee decides whether to grant a loan
• Internet shopping
- A customer wanting to buy an item decides whether to send
advance payment
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 6
Atle Refsdal
What is trust?
• Trust is the subjective probability by which an
actor, the trustor, expects that another entity, the
trustee, performs a given transition on which its
welfare depends.
- Based on Gambetta: “Can We Trust Trust?” (1988)
and Jøsang et al: “Can We Manage Trust?” (2005)
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 7
Atle Refsdal
What is trust?
• A relation between trustor and trustee
• The trustor may lose or gain from engaging in the
transaction
- reduction/increase of the trustor’s asset values
- risk and prospect
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 8
Atle Refsdal
Example case
• Business idea: Buy old items (antiques) on the
internet, then sell them from a shop at a higher
price
• A purchaser agent (Billy) surfs the internet to find
suitable items to buy for the shop
- Billy finds offers on various sites
- Sellers frequently require advance payment
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 9
Atle Refsdal
Example case
• Billy’s decision whether to pay depends on to
what degree he trusts that the item will be sent
- Simplification: other factors, such as price, are ignored
• Potential loss: Seller does not send the item after
receiving payment
• Potential gain: The item produces profit
• Billy is the trustor, the seller is the trustee
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 10
Atle Refsdal
Analysis method
1. Modeling of target
2. Analysis of target
i.
ii.
iii.
iv.
Identify critical decision points
Evaluate well-foundedness of trust
Estimate impact of alternative behavior
Evaluate and compare alternative behavior
3. (Capturing a policy to optimize target)
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 11
Atle Refsdal
Step 1: Modeling of target
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 12
Atle Refsdal
Target description with probabilistic
sequence diagrams
• Probabilities may be based for
example on frequencies of
observed behavior
probabilistic
alternative
• Billy talks to the seller on
the phone before
making the decision
• In 60% of the cases, Billy
decides to send the advance
payment.
• The item is received in 80% of
the cases where payment is
sent. Otherwise the money is
written off.
• ...what about trust?
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 13
Atle Refsdal
Basic idea
• Express trust by subjective probabilistic
sequence diagrams
- expresses the belief of an actor (the subject)
- probabilities represent the estimates made by the
subject
• Two kinds of diagrams:
- Objective diagrams show actual behavior
- Subjective diagrams show probability estimates made
by an actor
• The objective diagram refers to the subjective
- shows what probability estimates that are made, and
- how these estimates influence the choice of action
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 14
Atle Refsdal
Subjective sequence diagram (ssd)
• Billy believes that the
probability is 0.9 that the
item will be received, and
0.1 that the money must
be written off (w.r.t. the
specified scenario).
The lifeline making
the estimate
• How does this trust
(subjective belief)
influence Billys behavior?
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 15
Atle Refsdal
Integrating trust considerations in the
target description
Subjective diagram showing Billy’s
probability estimate.
Use a variable x since the estimate
varies from seller to seller
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 16
Atle Refsdal
Integrating trust considerations in the
target description
Objective diagram.
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 17
Atle Refsdal
Integrating trust considerations in the
target description
est.x≥0.5 holds in
60% of the cases
The first alternative is chosen
only if est.x≥0.5
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 18
Atle Refsdal
Integrating trust considerations in the
target description
The objective diagram refers to a
variable in the subjective diagram
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 19
Atle Refsdal
Trust related questions
How good are Billy’s
probability estimates?
Is the threshold right?
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 20
Atle Refsdal
Step 2: Analysis of target
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 21
Atle Refsdal
Step 2.i: Identify critical decision points
This is it!
• Identify points that need to
be looked into
• Typically points where
decisions are made based
on trust
• Could also be points
where one could benefit
from introducing new trustbased decisions
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 22
Atle Refsdal
Step 2.ii: Evaluate well-foundedness of
trust
• To what degree does the subjectively estimated
probabilities correspond to the actual (objective)
probabilities?
• We need more information:
- What would have been be the probability of receiving
the items in the cases where Billy chose to cancel the
deal?
• This information can be obtained from a model
that shows what happens if Billy sends advance
payment in all cases
• Such a model could for example be obtained from
an experiment
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 23
Atle Refsdal
The experiment
• Billy is instructed to
- send advance payment for
all items for a certain period
of time, and to
- write down his probability
estimate in every case
• The table records the
result
• The next slide shows a
model that could be
obtained from this
experiment
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 24
Item Estimated Item
probability received?
#1
0.9
Yes
#2
0.5
Yes
#3
0.7
No
#4
0.4
Yes
#5
0.7
No
#6
0.9
Yes
...
...
...
Atle Refsdal
Step 2.ii: Evaluate well-foundedness of
trust
Each operand represents an
interval of estimated probability.
The number of intervals/operands
depends on desired granularity.
‘est’ is the subjective diagram
from the previous slide
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 25
Atle Refsdal
Step 2.ii: Evaluate well-foundedness of
trust
Probability estimates can be
compared to objective values
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 26
Atle Refsdal
Step 2.ii: Evaluate well-foundedness of
trust
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 27
Atle Refsdal
Step 2.ii: Evaluate well-foundedness of
trust
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 28
Atle Refsdal
Step 2.iii: Evaluate impact of alternative
behavior
• Examples of alternative
behavior:
- obtaining information from a
reputation system
- arranging the transaction
through a trusted third party
- ...
Here: What would be the result
of using a higher or lower
decision threshold?
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 29
Atle Refsdal
Impact of alternative thresholds
The table shows the
outcomes from using
different thresholds
Note: % of all items
considered
0.2
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 30
0.2*0.9=0.18
Atle Refsdal
Impact of alternative thresholds
The table shows the
outcomes from using
different thresholds
Note: % of all items
considered
0.2+0.4=0.6
0.2*0.9+0.4*0.75=0.48
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 31
Atle Refsdal
Step 2.iv: Evaluate and compare
alternative behavior
• From the table, the analysis team decides which
threshold gives the most desirable result.
• Other considerations, such as asset values,
could also be taken into consideration
- At what price are items bought and sold?
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 32
Atle Refsdal
What has been achieved?
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 33
Atle Refsdal
Evaluation
• Analytical evaluation
- Subjective STAIRS is based on probabilistic STAIRS
- Probabilistic STAIRS is supported by a formal semantics
- Based on this semantics the concepts of of refinement and
compliance are defined
- Mathematical properties allowing stepwise and modular
refinement have been proved, as well as preservation of
properties under refinement
• Empirical evaluation
- Case study: buying items on the internet. Presented in “Extending
UML sequence diagrams to model trust-dependent behavior with
the aim to support risk analysis”, STM’07
- Case study: granting loans. Presented in “A UML-based Method
for the Development of Policies to support Trust Management”,
IFIPTM’08
- Industry case (in progress): Analysis of the effect implementing a
Validation Authority service for digital certificates, in cooperation
with DNV
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 34
Atle Refsdal
Analyzability
• Subjective STAIRS facilitates analysis
of systems whose behavior depends on trust
- Trust considerations are modeled explicitly
• Subjective probability estimates
• Decision thresholds
- ⇒ Easier to identify vulnerabilities and treatments
• Subjective STAIRS facilitates analysis of
mechanisms designed to control, restrict and
support trust dependent behavior
- Build one model where the mechanism is assumed to
be implemented, and one where it is not
- Compare the models
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 35
Atle Refsdal
Expressiveness
• It is possible to express to what degree
an actor trusts another actor with respect to a
certain transaction
- Expressed by a probability in a subjective diagram
• It is possible to express how trust considerations
influence a choice made by an actor between
different courses of action
- Expressed by a guard in an objective diagram referring
to a subjective diagram
• It is possible to describe the behavior
of the actors and their interaction
- Expressed by an objective diagram
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 36
Atle Refsdal
Conclusion
• UML sequence diagrams have been extended to
capture trust dependent behavior
- Subjective probability estimates
- Decision thresholds
• From the new models the analysis team should
be better able to identify trust related
vulnerabilities
- How good are the estimates?
- What about the decision threshold?
• Treatments can then be identified, and
their effect evaluated
SARDAS-seminar 21.05.2008 / Trust analysis / Slide 37
Atle Refsdal