IT & Society
Exercise 10
Misleading Information and
Fake Advice
Simeon Ivanov, Una Petrovic, Luc Fässler
Technical University of Munich
Department of Informatics
Chair of Cyber Trust
Karl Max von Bauernfeind Hörsaal, 07. July 2023
Agenda
1.
2.
3.
4.
5.
Recap Quiz
Effectiveness and Perception of Dark Patterns
Legal Regulations on Dark Patterns
Deepfakes
Wrap-up
IT & Society | SS 2023 | Exercise 10
2
Join at slido.com
#1635234
ⓘ
Click Present with Slido or install our Chrome extension to display joining
instructions for participants while presenting.
Which of the following
statements on post-truth is
false?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
What does "endowment
effect" mean?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
What does "dark pattern"
mean?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Which dark pattern is used in the following
example: "Joe enters an online shop and
sees a very nice pair of shoes on discount,
which is only available for the next 30
minutes."?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Which dark pattern is used in the following
example: "Joe opens a website and there
is a pop-up ad. To close it, he has to click
on “No thanks, I hate saving money!” or
“Yes, I want to save money!” if he wants to
go to the advertiser’s webpage."?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Which dark pattern is used in the following
example: "Joe is registering on a website
and has to tick a box, which states: “I
would like to pay for a membership &
agree to the Terms & Conditions & receive
emails & other promotional offers""?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Which dark pattern is used in the following
example: "Joe is registering on a website.
There is a checkbox and text, which says
“Don't uncheck the box if you prefer not to
receive email updates”"?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Effectiveness and Perception
of Dark Patterns
IT & Society | SS 2023 | Exercise 10
18
What are Dark Patterns?
Dark Patterns are subtle design features embedded in websites that prey on human
psychology to steer consumers into making decisions that, if fully informed or under optimal
conditions, they might not make and which go beyond acceptable marketing practices.
How do Dark Patterns differ from Nudges?
Dark patterns are ‘dark nudges’ with the intent to steer customers towards disadvantageous
decisions (e.g., impulse purchase or accepting privacy policies), while nudges could be used in
a positive sense (e.g., health or energy savings).
Examples:
● Forced Continuity
● Sneak in to basket
● Countdown-Timer
● etc. (see lecture for more)
Source: [1] Sin, R., Harris, T., Nilsson, S., & Beck, T. (2022). Dark patterns in online shopping: do they work and can nudges help mitigate impulse buying?. Behavioural Public Policy, 1-27.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
19
How do Dark Patterns work?
●
●
●
They evoke System 1 thinking rather than System 2 thinking
They do this by exploiting cognitive biases like scarcity bias or social proof
This has a strong influence on impulse buying decisions
Source: [1] Sin, R., Harris, T., Nilsson, S., & Beck, T. (2022). Dark patterns in online shopping: do they work and can nudges help mitigate impulse buying?. Behavioural Public Policy, 1-27.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
20
System 1 vs. System 2 thinking
System 1 thinking is a near-instantaneous process; it happens automatically, intuitively,
and with little effort. It’s driven by instinct and our experiences.
System 2 thinking is slower and requires more effort. It is conscious and logical.
Image Source: https://morecustomersfaster.com/wp-content/uploads/2015/10/branding-and-your-lazy-brain-morecustomersfaster.jpg
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
21
Social Proof
Definition: Individuals determine the correct action and behavior for themselves in a given
situation by examining the action and behavior of others.
Exploit:
Use this influence to accelerate user decision- making and purchases.
Source: [1] Lecture 10: Fake and Misleading Information and Fake Advice
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
22
Can Dark Patterns influence buying decisions?
Study by Sin et al. (2022)
Four Hypotheses were tested:
1) Consumers exposed to social proof dark pattern (a customer testimonial) will have
significantly higher buying impulse than the control group
2) Consumers exposed to supply-related scarcity messaging (‘Only 5 left in stock – Order
soon’) will have significantly higher buying impulse than the control group
3) Consumers exposed to demand-related scarcity messaging (‘Item in high demand – Order
soon’) will have significantly higher buying impulse than the control group
4) Consumers exposed to supply-related scarcity messaging will have significantly lower
buying impulse than those exposed to the demand-related scarcity messaging.
Source: [1] Sin, R., Harris, T., Nilsson, S., & Beck, T. (2022). Dark patterns in online shopping: do they work and can nudges help mitigate impulse buying?. Behavioural Public Policy, 1-27.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
23
Can Dark Patterns influence buying decisions?
Study by Sin et al. (2022)
Setup:
● Survey participants were presented with a hypothetical online shopping experience
involving Red Yeast Rice.
● Each participant was shown a scientific information sheet about the product, then actual
product information from Amazon and lastly one of three dark patterns (High-Demand,
Positive Testimonials, Limited-Quantity).
● After each piece of information they were asked about their urge to purchase.
Source: [1] Sin, R., Harris, T., Nilsson, S., & Beck, T. (2022). Dark patterns in online shopping: do they work and can nudges help mitigate impulse buying?. Behavioural Public Policy, 1-27.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
24
Can Dark Patterns influence buying decisions?
Study by Sin et al. (2022)
Results:
Significantly higher purchase
impulsivity across all treatment
groups, no difference between the
different treatment conditions.
Source: [1] Sin, R., Harris, T., Nilsson, S., & Beck, T. (2022). Dark patterns in online shopping: do they work and can nudges help mitigate impulse buying?. Behavioural Public Policy, 1-27.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
25
Are more aggressive Dark Patterns more effective?
Study by Luguri & Strahilevitz (2021) – Study 1
Setup:
● A representative sample was surveyed about demographic information, living situation, and
attitude and opinion on data privacy (this took up most of their time).
● They were then told that, based on previous responses, the system had identified them as
a person with a heightened concern about privacy. Therefore, they were automatically
offered an identity theft protection plan for which they would receive a 6-month free trial
and then be billed monthly for the service.
● The survey participants were then exposed to different dark patterns while evaluating this
offer.
● After accepting or declining the offer participants were surveyed about their mood and
whether they want to engage in follow-up surveys.
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
26
Are more aggressive Dark Patterns more effective?
Study by Luguri & Strahilevitz (2021) – Study 1
Setup (continued):
● Control group: Just “Accept” and “Decline” options.
● Mild dark patterns: “Accept and continue (recommended)”, which was selected by default
or “Other options”, if they chose “Other options” the participant had to select why they do
not want this offer (confirmshaming answer options) or could choose “On second thought,
please sign me up for 6 months of free credit history monitoring and data protection
services.”
● Aggressive dark patterns: Same structure as for mild dark patterns, but after they chose an
option for declining the offer, the participants were presented additional information about
identity theft which could not be skipped for ten seconds. At the end, they were presented a
trick questions about whether they want to accept the offer or not.
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
27
Are more aggressive Dark Patterns more effective?
Study by Luguri & Strahilevitz (2021) – Study 1
Results: Both types of Dark Patterns led to a significant increase of acceptance rate.
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
28
Does effectiveness differ among dark patterns?
Study by Luguri & Strahilevitz (2021) – Study 2
Setup:
● Similar to study 1; difference: different dark patterns were included during initial offer
(Hidden information, Social Proof, Scarcity, Confirmshaming).
● Additionally, different options for ‘Accept or Decline’ conditions (control, default,
recommendation, and ‘More Options’).
● Same follow-up scenarios for the ‘More Options’ condition as in study 1.
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
29
Does effectiveness differ among dark patterns?
Study by Luguri & Strahilevitz (2021) – Study 2
Results:
The study found significant differences in acceptance rates among different dark patterns.
Note: These findings are opposing to the results of the study by Sin et al. (2022).
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
30
Have dark patterns influenced your
(future) decision not to use a particular
website or service?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
How do people perceive Dark Patterns?
Study by Luguri & Strahilevitz (2021)
●
●
●
Aggressive dark patterns led to a significant higher dropout rate (9 vs. 65 participants) in
the second study.
Reasons?
- The more participants were exposed to dark patterns, the more likely they were to feel
forced into accepting the offer.
Control group had significant more interest to participate in follow-up surveys compared to
mild and aggressive dark patterns. However, there was no difference between mild and
aggressive dark patterns.
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
33
Conclusion on the Effectiveness of Dark Patterns
●
●
●
Dark patterns have a significant influence on purchasing decisions and can deceive
customers to make unintentional purchases.
Some dark patterns are more effective than others.
Dark patterns do not, opposing to our own perception, lead to backlash if applied
moderately.
Is regulation required?
Source: [1] Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43-109, [2] Sin, R., Harris, T., Nilsson, S., & Beck, T. (2022). Dark patterns in online shopping: do
they work and can nudges help mitigate impulse buying?. Behavioural Public Policy, 1-27.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
34
Legal Regulations on Dark Patterns
IT & Society | SS 2023 | Exercise 10
35
1.
2.
3.
4.
Dark Patterns under GDPR
Dark Patterns under UCPD
Dark Patterns under DMA
Dark Patterns under DSA
IT & Society | SS 2023 | Exercise 10
36
Dark Patterns under GDPR
●
No explicit definition of “dark patterns” in the General Data Protection Regulation
●
The European Data Protection Board (EDPB) defines dark patterns as “interfaces and user
experiences implemented on social media platforms that lead users into making
unintended, unwilling and potentially harmful decisions in regards to their personal data
with the aim of influencing users’ behaviors”1.
EDPB also provides Guidelines on how to not implement dark patterns in social media
interfaces; best practices:
- Use of Examples
- Explaining Consequences
- Providing Definitions
●
Source: [1] Guidelines 03/22 on deceptive design patterns in social media platform interfaces: how to recognise and avoid them https://edpb.europa.eu/our-work-tools/documents/public-consultations/2022/guidelines-32022-dark-patterns-social-media_en;
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
37
Possible Dark Pattern-related Breaches of GDPR
●
“any freely given, specific, informed and unambiguous indication of the data subject's
wishes by which he or she, by a statement or by a clear affirmative action, signifies
agreement to the processing of personal data relating to him or her” - Article 41
●
“If the data subject’s consent is given in the context of a written declaration which also
concerns other matters, the request for consent shall be presented in a manner which is
clearly distinguishable from the other matters, in an intelligible and easily accessible
form, using clear and plain language.” - Article 71
●
“Taking into account the [...] severity for rights and freedoms of natural persons posed
by the processing, the controller shall […] implement appropriate technical and
organisational measures […] to integrate the necessary safeguards into the processing in
order to meet the requirements of this Regulation and protect the rights of data
subjects.” - Article 251
Source: [1] General Data Protection Regulation - https://gdpr-info.eu/;
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
38
First Fine Caused by Dark Patterns
●
Announced on April 17, 2023, The Italian Supervisory Authority (“Garante”) imposed a fine
of 300.000 EUR1
●
Visual interference used in the cookie banner - “unclear communication patterns with
particular regard to the graphic design of the interfaces and how the process of signing up
for services was carried out.”
●
Violation of Art. 7 and 25 from GDPR
Source: [1] https://www.gpdp.it/web/guest/home/docweb/-/docweb-display/docweb/9870014;
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
39
1.
2.
3.
4.
Dark Patterns under GDPR
Dark Patterns under UCPD
Dark Patterns under DMA
Dark Patterns under DSA
IT & Society | SS 2023 | Exercise 10
40
Dark Patterns under UCPD
●
No explicit definition of “dark patterns” in the Unfair Commercial Practices Directive:
“The term ‘dark pattern’ does not have a legal definition in the Directive”1
●
“...the term ‘dark pattern’ is used to refer to a type of malicious nudging, generally
incorporated into digital design interfaces.”
●
Covers dark patterns related to business-to-consumer relationships:
- Professional diligence requirement - Visual Interference
- Misleading practice - Trick Phrases
- Aggressive practice - Confirmshaming
Source: [1] Commission Notice – Guidance on the interpretation and application of Directive 2005/29/EC of the European Parliament and of the Council concerning unfair business-to-consumer commercial practices in the
internal market - https://edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-032022-deceptive-design-patterns-social-media_en;
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
41
1.
2.
3.
4.
Dark Patterns under GDPR
Dark Patterns under UCPD
Dark Patterns under DMA
Dark Patterns under DSA
IT & Society | SS 2023 | Exercise 10
42
Dark Patterns under DMA
●
●
●
●
Digital Markets Act entered into force on November 1, 2022
Companies have to comply by March 6, 2024
Affects only “gatekeepers” - companies that:
- achieve a certain annual turnover in the European Economic Area (EEA) and provide
a core platform service in at least three EU Member States or
- provide a core platform service to more than 45 million monthly active end users
established or located in the EU and to more than 10,000 yearly active business users
established in the EU or
- met the second criterion during the last three years1
“Gatekeepers should not design, organise or operate their online interfaces in a way
that deceives, manipulates or otherwise materially distorts or impairs the ability of end
users to freely give consent.” 2
Source: [1] https://ec.europa.eu/commission/presscorner/detail/en/ip_22_6423; [2] https://eur-lex.europa.eu/eli/reg/2022/1925;
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
43
1.
2.
3.
4.
Dark Patterns under GDPR
Dark Patterns under UCPD
Dark Patterns under DMA
Dark Patterns under DSA
IT & Society | SS 2023 | Exercise 10
44
Dark Patterns under DSA
●
●
●
●
●
Digital Services Act came into force on November 16, 2022
Companies have to comply by February 17, 20241
“Providers of online platforms shall not design, organise or operate their online
interfaces in a way that deceives, manipulates or otherwise materially distorts or
impairs the ability of recipients of their service to make free and informed decisions.” Article 25
DMA and DSA should fill the gap, left by GDPR and UCPD:
“The prohibition in paragraph 1 shall not apply to practices covered by Directive
2005/29/EC or Regulation (EU) 2016/679.” - Article 252
“...the Commission should be empowered to supplement the provisions of this Regulation
by adopting delegated acts, where necessary.” 2
Source: [1] https://digital-strategy.ec.europa.eu/en/policies/digital-services-act-package; [2] https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32022R2065;
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
45
Deepfakes
IT & Society | SS 2023 | Exercise 10
46
What is the first thing that
comes to mind when hearing
the term 'deepfake'?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Introduction to deepfakes1
Deepfakes: The use of neural network (type of ML model)
trained on two faces to replace one face with
another.
Mostly used in:
- Entertainment
- Parody
- Video games
- Education
- Advertisement
Source: [1] Exploring Deepfakes. by Bryan Lyon, Matt Tora. Released March 2023. Publisher(s): Packt Publishing. ISBN: 9781801810692.
Image source: https://www.premiumbeat.com/blog/deepfake-technology-future-of-film-industry/
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
49
Creating deepfakes1 (I /II)
●
●
●
●
●
Require a significant amount of training data to accomplish its task
Generate an image with new details that were not in the original image using AI
The model is continuously trained on images
The training consists of four steps:
- Encode
- Decode
- Calculate the loss
- Modify (backpropagate)
Once completed, the whole process starts back
at the encoder again
Source: [1] Exploring Deepfakes. by Bryan Lyon, Matt Tora. Released March 2023. Publisher(s): Packt Publishing. ISBN: 9781801810692.
Image Source: https://static.packt-cdn.com/products/9781801810692/graphics/image/B17535_01_002.jpg
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
50
Creating deepfakes1 (II/II)
●
●
●
To provide best results, deepfakes require training on every face pair to be swapped.
Making a realistic deepfake from scratch – alongside many hours needed for model
training – requires lots of hours of human assistance and technology expertise.
However, once the model has been trained for a particular face, a deepfake can be
generated in the matter of minutes.
Available deepfake softwares:
- Faceswap2
- DeepFaceLab3
- Reface4, etc.
Source: [1] Exploring Deepfakes. by Bryan Lyon, Matt Tora. Released March 2023. Publisher(s): Packt Publishing. ISBN: 9781801810692.
[2] https://Faceswap.dev/, [3] https://github.com/iperov/DeepFaceLab, [4] https://reface.ai/, Image source: https://ars.els-cdn.com/content/image/1-s2.0-S0007681319301600-gr3.jpg
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
51
What is your biggest
concern regarding
deepfakes on a societal
level?
ⓘ
Click Present with Slido or install our Chrome extension to activate this
poll while presenting.
Threats1
Believability: The brain’s visual system, despite being largely robust in natural settings, can be
targeted for misperception. If we see something with our own eyes, we believe it
to exist or to be true - even if it is unlikely.
Accessibility: Creating deepfakes is becoming easier and is allowing anyone to create them
without significant investments.
Areas of threat:
- Reputation
- Politics
- Economics
Source: [1] Kietzmann, J., Lee, L. W., McCarthy, I. P., & Kietzmann, T. C. (2020). Deepfakes: Trick or treat?. Business Horizons, 63(2), 135-146. 10.1016/j.bushor.2019.11.006.
Image source: https://i2-prod.dailystar.co.uk/incoming/article15819781.ece/ALTERNATES/s1200b/421871
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
54
Deepfake examples
https://www.zdf.de/nachrichten/video/panorama-fake-video-selenskyj-100.html
IT & Society | SS 2023 | Exercise 10
https://www.youtube.com/watch?v=NbedWhzx1rs
Q&A: #1635234
55
Prevention measures1
●
●
●
There is no foolproof way to make yourself immune to deepfakes.
Possible solutions to minimise the threat:
- Starving the model of data by controlling your media presence;
- Deepfake detection;
- Authenticating any genuine media;
- Raising public awareness.
There are still little economic incentives for companies to provide a thorough deepfake
detection solution.
Source: [1] Exploring Deepfakes. by Bryan Lyon, Matt Tora. Released March 2023. Publisher(s): Packt Publishing. ISBN: 9781801810692.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
56
Public’s opinion on deepfakes
Research1 on US citizens’ perception of deepfakes:
Furthermore, 77% say steps should be taken to restrict altered videos and images that are
intended to mislead, with 49% stating that they have shared information online thinking it was
real, and only later on finding out the information was altered.
Source: [1] Pew Research Center, June, 2019, “Many Americans Say Made-Up News Is a Critical Problem That Needs To Be Fixed”
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
57
Future of deepfakes1
●
●
●
Technology behind deepfakes is just beginning to take off.
Departments that may benefit the most from deepfakes in the future include: the music,
cinematic, and entertainment industry.
Growing number of papers and articles are being published, constantly revealing newly
developed features of deepfakes and their creation methods.
https://www.oneusefulthing.org/p/a-quick-and-sobering-guide-to-cloning
Source: [1] Exploring Deepfakes. by Bryan Lyon, Matt Tora. Released March 2023. Publisher(s): Packt Publishing. ISBN: 9781801810692.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
58
Wrap-up
IT & Society | SS 2023 | Exercise 10
59
Wrap-Up
●
●
●
●
●
●
●
Dark patterns have a significant influence on purchasing decisions and can deceive
customers to make unintentional purchases.
Some dark patterns are more effective than others.
Dark patterns do not, opposing to our own perception, lead to backlash if applied
moderately.
The legal regulations on dark patterns are complicated – they are covered by different
Directives and Acts, depending on the exact case.
The goal of The European Commission is to fill the legal gaps on dark patterns.
Technologies being developed with the intention of misleading people are only beginning
to take off – deepfakes being one of the primary examples.
General public is of the opinion that something should be done regarding misleading
information online and raise their concerns on this topic.
IT & Society | SS 2023 | Exercise 10
Q&A: #1635234
60