Regulating Behavior
KSE 652 Social Computing Systems Design
Uichin Lee
Oct. 9, 2012
Encouraging Commitment: Summary
• Affective Commitment
– Identity-based affective commitment: e.g., social identity, group
norms, homogeneous groups, naming, common fate/goal/task,
in vs. outgroups
– Bond-based affective commitment: e.g., leveraging social ties,
social interaction mechanisms, profiles, pseudonyms
– Reducing repelling forces: e.g., large group, diversity, off-topic
• Normative Commitment
– purpose, other's commitment, reciprocity
• Needs-based Commitment
– matching benefits with motivation, competitors, lock-in (sunk
cost)
Contents
•
•
•
•
Introduction to normative behavior
Limiting effects of bad behavior
Coerced compliance: limits on bad behavior
Encouraging voluntary compliance
– Making norms clear and salient
– Enhancing compliance
– Rewards and sanctions
• Summary
Normative Behavior
• Normative Behavior
– Rough consensus emerges about the range of
behaviors that the managers and most members
consider acceptable
– Ex) A new article “a rape in a cyberspace” describes an
example of unacceptable behavior
• Communities differ on which behaviors are
normative
– Wikipeida expects writers to adopt a neutral point of
view when writing articles
– Huffington (www.huffingtonpost.com) post expects
guest bloggers to express view points
Normative Behavior
• Help community to achieve its mission
– Being supportive, on-topic, etc.
• Behavioral norms of handling conflicts
– Edit wars in Wikipedia  three-revert rule
• Destructive behavior: troll and manipulators
– Troll/griefers like to destruct the community
• E.g., inflammatory comments, corpse camping (WoW)
– Manipulators like to produce some particular
outcome
• E.g., pumping up ratings of a particular venue in Yelp.com
using multiple “shill” accounts
Normative Behavior
• Insiders “do” violate
– New comers do not know much about rules
– Some people may have some cognitive/social impairments
• Those who have hard time inferring rules
– Social dilemma
• Everyone is better off if everyone complies with the norms; if no one does, each
individual is better off if he/she does not comply and the others do
• Public good problem vs. public bad problem (common pool resource; or tragedy of the
common problem)
• Online community: “people’s attention” is the limited (common pool) resource
• Learning design principles from institutions that have successfully selfgoverned common pool resources (Ostrom 1990)
– Participation of rule making, monitoring, graduated sanctions, conflictresolution mechanisms
• Regulation describes any efforts to decrease the frequency of nonnormative behaviors or lessen their impacts on the community
– Law, norms, markets, technology; in online communities, hard to deal with
laws; our focus is on the latter three categories
Contents
•
•
•
•
Introduction to normative behavior
Limiting effects of bad behavior
Coerced compliance: limits on bad behavior
Encouraging voluntary compliance
– Making norms clear and salient
– Enhancing compliance
– Rewards and sanctions
• Summary
Limiting effects of bad behavior
• Design Claim 1: Moderation systems that pre-screen,
degrade, label, move, or remove inappropriate
messages limit the damage they cause.
– Ex) deleting messages, disemvoweling, labeling/rating
messsages (e.g., rating in slashdot)
• Design Claim 2: Redirecting inappropriate posts to
other places will create less resistance than removing
them.
– People may not agree with moderator’s decision; it’s
better for people to take it outside (thus, not really
watching flaming)
Limiting effects of bad behavior
• Design Claim 3: Consistently applied moderation criteria, a chance
to argue one’s case, and appeal procedures increase the legitimacy
and thus the effectiveness of moderation decisions.
– People are more satisfied if they go though fair procedure (source of
legitimacy comes from procedural justice)
– Interestingly, people likes to take a more severe punishment after they
had had their day in court than a milder punishment without any
hearing (Tyler 1990)
• Design Claim 4: Moderation decided by people who are members
of the community, are impartial, and have limited or rotating power
will be perceived as more legitimate and thus be more effective
– Moderator: community selected; those with expertise
– Ex) Slashdot “promote quality, discourage crap”; it also has “metamoderation system to address issue of unfair moderators
Limiting effects of bad behavior
• Design Claim 5: Revision tools limit the damage disrupters
can inflict in production communities.
• Design Claim 6: Filters or influence limits can limit the
damage of shill raters in recommender systems, but only at
the cost of ignoring some useful information from honest
raters.
– Algorithms of detecting suspicious patterns (e.g., too many in a
short term or insufficient variability
– Influence limiter partially discounts contribution (discount
declines as the system gains confidence)
• Design Claim 7: A widely followed norm of ignoring trolls
will limit the damage they can do.
– E.g., “DNFTT: do not fee the troll”
Contents
•
•
•
•
Introduction to normative behavior
Limiting effects of bad behavior
Coerced compliance: limits on bad behavior
Encouraging voluntary compliance
– Making norms clear and salient
– Enhancing compliance
– Rewards and sanctions
• Summary
Coerced Compliance
• Design Claim 8: Activity quotas allow people to
participate in a community, but prevent repetitive,
spam-like activity
• Design Claim 9: Gags and bans can limit the continuing
damage of a bad actor, but only if it is hard for the bad
actor to use a different account (“expensive” account)
or if the ban is disguised
• Design Claim 12: Limiting fake accounts with CAPTCHAs
or identity checks limits automated attacks.
Coerced Compliance
• Design Claim 10: Consistently applied criteria for gags
and bans, a chance to argue one’s case, and appeal
procedures increase the legitimacy (“procedural
justice”) and thus the effectiveness of gags and bans.
• Design Claim 11: Paying to take actions in the
community with currency accumulated through normal
participation will reduce the ability for trolls and
manipulators to act.
– Ex) OSS’s vetting process, Omidyar’s internal currency,
Influence Limiter
Contents
• Introduction to normative behavior
• Limiting effects of bad behavior
• Coerced compliance: limits on bad behavior
• Encouraging voluntary compliance
– Making norms clear and salient
– Enhancing compliance
– Rewards and sanctions
• Summary
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Making norms clear and salient
• How to learn community norms?
– Observing other people and consequences of their behavior
– Seeing instructive generalizations or codes of conduct
– Behaving and directly receiving feedback
• Descriptive (one’s perception) vs. injunctive (others’ perception)
norms
• People may want to fit in by doing what others are doing (some
cases, descriptive ~ injunctive)
• Power of observing others (how a group learn norms)
– Auto-kinetic effect (Sherif 1936): each person speak out distance, and
people’s perception of illusion converged to 3 inches
– Presentation of self in everyday life (Goffman 1959) explains people
tend to act predictably within norms in typical social interactions (e.g.,
hospital, parties, elevator, etc.)
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Making norms clear and salient
• Design Claim 13: Publicly displaying examples of
appropriate behavior on the site will show
members what is expected and increase their
adherence to those expectations.
– Making people’s normative behavior (descriptive
norms) to all members of community
• Ex) reducing binge drinking among students by publishing
accurate information about how much the typical student
drinks (Perkins et al., 1999)
– Online communities have many ways of explicitly
displaying descriptive norms (rather than random
encounters): e.g., post of the week in an online forum
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Making norms clear and salient
• Design Claim 14: Publicly contrasting examples of
inappropriate behavior in the context of a descriptive
norm of appropriate behavior will highlight the
descriptive norm and increase people’s adherence to it.
– Cialdini’s focus theory (1991): people learn norms from
salient behaviors—actions that stand out (e.g., negative
behavior) and point out to people what is appropriate to
do in a situation
• Ex) seeing littering behavior in dirty vs. clean environments
– Online communities: e.g., moderator flag, disemvowel
• Design Claim 15: Publicly displaying many examples of
inappropriate behavior on the site will lead members
to believe this is common and expected.
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Making norms clear and salient
• Design Claim 16: Displaying feedback of members to others
increases members’ knowledge of community norms and
compliance with them; formal feedback is more effective
than informal feedback
– Observing common online behavior (descriptive norms) vs.
observing feedback to such behaviors teaches injunctive norms
(i.e., approved/disapproved behaviors)
– Informal (e.g., “thank you” comments) vs. formal feedback (e.g.,
points, stars); formal feedback is known to work better in
practice
• Design Claim 17: In large communities, displaying statistics
that highlight the prevalence of normative behavior will
increase members' adherence to normative behavior
– Ex) low percentage of messages flagged for violating the rule
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Making norms clear and salient
• Design Claim 18: Explicitly stating rules and guidelines
increase the ability for community members to know
the norms, especially when it is less clear what others
think is acceptable.
(Motorists returning to their cars were given
a hand bill with different statements; Cialdini, Kallgren, Reno 1991)
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Making norms clear and salient
• Then how prominently to display guidelines and rules?
– Design Claim 19: prominently displayed or excessively detailed
rules may also convey the wrong descriptive norm
• Reddit.com’s “reddiquette”; articles that describe “norms/rules” in
detail
• But this has negative impression (e.g., newcomers)
• Design Claim 20: Offering people reminders at the point of
an action that may violate norms will reduce the number of
offenses
– E.g., Eudora’s MoodWatch s/w automatically cautions users who
are about to send an email containing flame words (Shankland
2000)
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Enhancing compliance
•
Design Claim 21: In more cohesive groups to which members are more committed,
members will be more likely to spontaneously comply with the norms.
– How to improve group cohesion?
•
Design Claim 22: Community influence on rule making (i.e., participatory rule
design; or collective choice) will increase compliance with the rules.
– In general, externally imposed rules and monitoring tend to be viewed as unfair and to lead to
conflict (Ostrom 2000)
– Management’s top-down “play nice” rules was not that helpful in a multi-player game,
Everquest (Yee 2001)
•
Design Claim 23. Face-saving (or ignorance pleading) ways to correct normviolations will increase compliance.
– Violators are often willing to stop bad behavior and correct errors if they can do so without
having to admit that they deliberately violated the norms; provide an opportunity of face
saving
– Ex) “UYA notice in MIT network”; “someone using your account”  “did you notice that…”
– Ex) Notification like “You may not have been aware of this guideline, but we have a stated
policy of […]. No big deal, but please stick to this in the future (plead ignorance)
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Rewards and Sanctions
• Deterrence theory in criminology
– People with criminal disposition will violate the
rules only when it pays
– They perform informal cost-benefit analysis
(punishment vs. benefit)
– For spammers, manipulators and trolls, simply
limiting the effectiveness of their actions reduces
the incentives to participate
• What kinds of sanctions?
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Rewards and Sanctions
•
Design Claim 24: Telling search engines not to follow links will discourage
spammers from posting links.
– E.g., nofollow attributes in Blog comments to prevent search engine spamming
•
Design Claim 25: Verified identities and pictures will reduce the incidence of norm
violations.
– Social approbation (people are very sensitive to public impression); Goffman 1959
•
Under-reporting lying, stealing, drug use with “in-person” interviews (vs. online survey)
– Anonymity and larger group size both lead to antinormative behavior
•
•
De-individuation in a group as being submerged in it (Festinger 1952); individuals are not seen or paid
attention to as individuals (and not stand out)
People are more likely to violate norms when they can conceal their identities under white robes
(Zimbardo 1969)
– Relative anonymity of online communications reduces normative pressure online (Bordia
1997)
•
Design Claim 26: Reputation systems, which summarize the history of someone’s
online behavior, help to encourage good behavior and deter norm violations.
– Reputation may influence future interactions as well (e.g., posting articles, etc.)
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Rewards and Sanctions
• Design Claim 27: Prices, bonds, or bets that make
undesirable actions more costly than desirable actions will
reduce misbehavior.
• Maintaining effective sanctions with cheap pseudonym
– Design Claim 28: Increasing the benefits of participating with a
long-term identifier (due to reputation or some other reasons)
will increase the community’s ability to sanction misbehavior.
– Design Claim 29: Imposing costs for or preventing pseudonym
switching increases the community’s ability to sanction
misbehavior.
– Design Claim 30: Forcing newcomers to post bonds that may be
forfeited if the newcomers misbehave or forcing newcomers’
sponsors to stake their own reputations increases the
community’s ability to sanction misbehavior
Encouraging voluntary compliance



Making norms clear and salient
Enhancing compliance
Rewards and sanctions
Encouraging voluntary compliance
Rewards and Sanctions
• Design Claim 31: Graduated sanctions increase the legitimacy and thus the
effectiveness of sanctions.
– Sanctions proportional to the offence are perceived as fair and legitimate
(Ostrom 1990)
– More persuaded to comply through expertise and judgment than commands
and force (Koslowsky 2001)
– Graduated sanctions that begin with persuasion based on expertise and
judgment and proceed to more forceful measure can be effective
• Design Claim 32: Peer reporting or automatic detection of violations
increases the deterrent effect of sanctions.
– Mild but certain punishment is more effective than severe but uncertain
punishment
• Design Claim 33: Increased community cohesion, graduated sanctions,
explicit rules, identifiable perpetrators, formal sanctioning roles, and antiretaliatory measures increase the likelihood that sanctions will be applied
and thus increase the deterrent effect of sanctions.
Summary
• Limiting effects of bad behavior
– Moderation, effective moderation, revision tools,
filters/influence limiters (reputation systems)
• Coerced compliance: limits on bad behavior
– Activity quota, gags/bans, CAPTCHAs
• Encouraging voluntary compliance
– Making norms clear and salient
• Descriptive (explicit display) vs. injunctive norms (feedback)
– Enhancing compliance
• Cohesive groups, collective choice, face-saving
– Rewards and sanctions
• Link spam protection, identitiability, reputation, handling cheap
pseudonym