we are dynamo: overcoming
stalling and friction in
collective action for
crowd workers
Niloufar Salehi, Lilly Irani, Michael Bernstein,
Ali Alkhatib, Eva Ogbe, Kristy Milland,
@niloufar_s
niloufar@cs.stanford.ed
u
Many collective action efforts, online, never
succeed.
[Hill 2011, Keegan et al. 2010, Kraut et al. 2012]
Success cases:
Exception, rather than the norm.
decentralized characteristics
gather
make it easy to disperse
easy to
The majority of collective action efforts fail.
failures
successes
[Hill 2011, Keegan et al. 2010, Kraut et al. 2012]
We engaged with crowd-workers on
Amazon Mechanical Turk.
dynamo
www.wearedynamo.org
dynamo
Guidelines for Academic Requesters
http://guidelines.wearedynamo.org
159 workers and 55 requesters signed.
dynamo
Letter Writing Campaign
http://wearedynamo.org/dearjeffbezos
movements faced failure
Stalling:
losing
momentum
Friction: critiques
and
disagreements
challenges
dynamo
stalling & friction
labor of action
challenges
dynamo
stalling & friction
labor of action
challenges of online work
• crowdsourcing platforms as computational
infrastructure
• invisible workers
• struggle to find good work or assert their
rights
[Irani and Silberman 2013, Kittur et al. 2013]
turker collectives
• resources for good work and mutual
support
turker collectives
• resources for good work and mutual
support
forums struggled
• open ended, sometimes unclear, goals
• require public debate
• sharing of labor
what about a turker union?
• Turkers found it impractical in their
environment:
– high turnover
– short work contracts
– personal independence
How can we support an online collective not
just to discuss, but to agree on a common
goal and execute it?
challenges
dynamo
stalling & friction
labor of action
dynamo
www.wearedynamo.org
dynamo: our design process
• interactive and iterative
• engaged with Turkers
• designed and built Dynamo together
dynamo: our design process
• interactive and iterative
• engaged with Turkers
• designed and built Dynamo together
dynamo: activity
•
•
•
•
•
532 registered Turkers
22 ideas for action, 2 campaigns
7,000 unique visitors
32,000 views
longest discussion took 181 posts and had
more than 1,800 views
dynamo: registration
•
•
•
•
registration code
requires 100 tasks
prove worker status
one voice
dynamo: registration
• anonymity
• new space for
agreement
• randomly generated
screen names:
– e.g. light_dragonfly,
excited_iguana, and
dark_bird_of_paradise
1. assemble a collective
2. mobilize
dynamo: in action
Guidelines for ethical research
[Irani and Silberman 2013]
ethical guidelines
• Turkopticon abused by researcher
• researchers lack exposure to Turkers’
issues and vulnerabilities
ethical guidelines
• publicly available ethical guidelines:
– guide ethical behavior
– back Turker claims
ethical guidelines
from wanting ethical guidelines to creating it
 idea
✔ vote
28
mobilize
• 25 upvotes & more upvotes than
downvotes
• discuss, take action, track progress
ethical guidelines
• http://guidelines.wearedynamo.org
• published on September 7th (Labor day)
ethical guidelines
•
•
•
•
http://guidelines.wearedynamo.org
23 page write-up
accessed 43’000 times
fair payment, Turker privacy, etc.
ethical guidelines
• support from researchers
• 159 Turkers and 55 academic requesters
• info@wearedynamo.org
ethical guidelines
letter writing campaign
• 30 public letters at:
• http://wearedynamo.org/dearjeffbezos
letter writing campaign
• 30 public letters at:
• http://wearedynamo.org/dearjeffbezos
While both these campaigns were
successful, they both almost failed.
challenges
dynamo
stalling & friction
labor of action
stalling
loosing momentum
stalling
• example: letter writing campaign stalled
“So, it seems no one is interested […]. [A Turker]
just says we're doing it wrong, but won't say how
to do it right, and no one else has input.”
– A Turker
stalling
• example: letter writing campaign stalled
“So, it seems no one is interested […]. [A Turker]
just says we're doing it wrong, but won't say how
to do it right, and no one else has input.”
– A Turker
friction
active criticism with negative emotion
friction
• example: disappointment over one
paragraph of the guidelines
“I’m sorry to see the core document get
excessively mired in technical detail in what
purports to be a high-level document on ethics”
– A Turker
friction
• Reflect arguments and suggest ways of
addressing them.
• Turn friction into constructive criticism
instead of flames.
twin pitfalls
Stalling:
losing
momentum
Friction: critiques
and
disagreements
twin pitfalls
Stalling:
losing
momentum
Friction: critiques
and
disagreements
Online collectives have more reasons to fail
at acting together than to succeed.
challenges
dynamo
stalling & friction
labor of action
debates with deadlines
debates with deadlines
“We've spent so much time and energy on this, we
need a last effort to reach consensus […] We need
your help for that to happen. Do you think setting a
deadline would help?”
– [Dynamo team]
“I do agree we should wrap things up soon, if
possible without unnecessary sacrifices.”
– A Turker
debates with deadlines
Participants know when they have to gear
towards building consensus.
act and undo
act and undo
“the purpose of Dynamo is/should be to add to the
rights/recourse of Turkers, not to limit/replace
them.”
– A Turker
act and undo
Take action, but leave space for objections
and undo your action if necessary.
labor of action
Structured mechanisms to guide an online
collective to success.
HCI envisions how design and technology
can affect positive changes in the world
the social in social software
software
people
the social in social software
software
people
the social in social software
software
people
challenges
dynamo
stalling & friction
labor of action
we are dynamo
Thank you!
http://wearedynamo.org
Special thanks to Turkers who participated in, and
helped shape this project. This work was
supported by a National Science Foundation
award IIS-1351131, an Open Society Foundation
grant, and a Stanford Graduate Fellowship.
Niloufar Salehi, Lilly Irani, Michael Bernstein,
Ali Alkhatib, Eva Ogbe, Kristy Milland,
@niloufar_s
niloufar@cs.stanford.ed
u