--�··
� .. �--
ANNALS OF TECHNOLOGY
ANTISOCIAL MEDIA
Reddit,free speech, and the struggle to detoxify the Internet.
By Andrew Marantz
W
hich Web sites get the most
net," Steve Huffman, Reddit's C.E.O.,
Some of the conspiracy theorists left
traffic? According to the
said recently. "These days, I tend to say
Reddit and reunited on Voat, a site
ranking service Alexa, the
that we're a place for open and honest
made by and for the users that Reddit
top three sites in the United States,
conversations-'open and honest' mean­
sloughs off (Many social networks have
as of this writing, are Coogle, You­
ing authentic, meaning messy, meaning
such Bizarre networks, which brand
Tube, and Facebook.(Porn, somewhat
the best and worst and realest and weird­
themselves as strongholds of free speech
hearteningly, doesn't crack the top
est parts of humanity."
and in practice are often used for hate
ten.) The rankings don't reflect every­
On November 23,2016, shortly after
speech. People banned from Twitter
thing-the dark Web, the nouveau­
President Trump's election, Huffman
end up on Gab; people banned from
riche recluses harvesting bitcoin-but,
was at his desk, in San Francisco, pe­
Patreon -end up on Hatreon.) Other
for the most part, people online go
rusing the site. It was the day before
Pizzagaters stayed and regrouped on
where you'd expect them to go. The
Thanksgiving.Reddit's administrators
r/The_Donald, a popular pro-Trump
only truly surprising entry, in fourth
had just deleted a subreddit called
subreddit. Throughout the Presiden­
place, is Reddit, whose astronomical
r/Pizzagate, a forum for people who
tial campaign,The_Donald was a hive
popularity seems at odds with the fact
believed that high-ranking staffers of
of Tr ump boosterism. By this time, it
that many Americans have only vaguely
Hillary Clinton's Presidential cam­
had become a hermetic subculture, full
heard of the site and have no real un­
paign, and possibly Clinton herself,
of inside jokes and ugly rhetoric. The
derstanding of what it is.A link aggre­
were trafficking child sex slaves. The
community's most frequent comment­
gator? A rnicroblogging platform? A
evidence, as extensive as it was unper­
ers, like the man they'd helped propel
social network?
suasive, included satanic rituals, a map
to the Presidency, were experts at test­
To its devotees, Reddit feels proudly
printed on a handkerchief, and an elab­
ing boundaries. Within minutes, they
untamed, one of the last Internet gi­
orate code involving the words "cheese"
started to express their outrage that
ants to resist homogeneity.Most Red­
and "pizza." In only fitf een days of ex­
Pizzagate had been deleted.
dit pages have a throwback aesthetic,
istence, the Pizzagate subreddit had
Redditors are pseudonymous, and
with a few crudely designed graphics
attracted twenty thousand subscrib­
their pseudonyms are sometimes pref­
and a tangle of text: an original post,
ers. Now, in its place, was a scrubbed
aced by
comments on the post, responses to
white page with the message "This
is Spez. As he scanned The_Donald,
the comments, responses to the re­
community has been banned."
he noticed that hundreds of the most
"
"
u,
for "username." Huffman's
sponses. That's pretty much it. Red­
The reason for the ban, according
dit is made up of more than a million
to Reddit's administrators, was not
"fuck u/spez"
individual communities, or subred­
the beliefs of people on the subred­
'\Jspez is complicit in the coverup"
dits, some of which have three sub­
dit, but the way they'd behaved­
"u/spez supports CHILD RAPE"
scribers, some twenty million. Every
specifically, their insistence on pub­
subreddit is devoted to a specific kind
lishing their enemies' private phone
"u/SPEZ IS
of content, ranging from vital to triv­
numbers and addresses, a clear viola­
hundred and ten times in a row.
popular -comments were about him:
One commenter simply wrote
"
A CUCK,
in bold type, a
ial: r/News, r/Politics, r/Trees (for
tion ofReddit's rules.The conspiracy
Huffman, alone at his computer,
marijuana enthusiasts), r/Marijuana­
theorists, in turn, claimed that they'd
wondered whether to respond. "I con­
Enthusiasts (for tree enthusiasts),
been banned because Reddit admin­
sider myself a troll at heart," he said
r/Mildlylnteresting ("for photos that
istrators were part of the conspiracy.
later. " M aking people bristle, being
are, you know, mildly interesting").
(Less than two weeks after Pizzagate
a little outrageous in order to add
Some people end up onReddit by ac­
was banned, a man fired a semiauto­
some spice to life-I get that. I've done
cident, find it baffling, and never visit
matic rifle inside a D.C.pizzeria called
that." Privately, Huffman imagined
again.But people who do use it-red­
Comet Ping Pong, in an attempt to
The_Donald as a misguided teen-ager
ditors, as they're called-often use it
"self-investigate" claims that the restau­
who wouldn't stop misbehaving. "If
all day long, to the near-exclusion of
rant's basement was a dungeon full of
your little brother flicks your ear, maybe
anything else. "For a while, we called
kidnapped children. Comet Ping Pong
you ignore it," he said. "If he Hicks your
ourselves the front page of the Inter-
does not have a basement.)
ear a hundred times, or punches you,
58
THE NEW YOI\KEI\, MAII.CH 19, 2018
"The Internet started as a bastionforfree expression," aformer Reddit C.E. 0. wrote. These days, "the trolls are winning."
ILLUSTRATION BY JAVIER JAEN
THE NEW YOI\�ER. MAI\CH 19. 2018
59
the Supreme Court heard a case about
whether it was constitutional to bar
registered sex offenders from using so­
cial media. In order to answer that
question, the Justices had to ask an­
other question: What is social media?
[n sixty minutes of oral argument, Face­
book was compared to a park, a play­
ground, an airport terminal, a polling
place, and a town square.
It might be most helpful to com­
pare a social network to a party. The
party starts out small, with the hosts
and a few of their friends. Then word
gets out and strangers show up. Peo­
ple take cues from the environment.
\
Mimosas in a sun-dappled atrium sug­
gest one kind of mood; grain alcohol
in a moldy basement suggests another.
Sometimes, a pattern emerges on its
own. Pinterest, a
simple photo-sharing
site founded by three men, happened
to catch on among women aspiring to
an urbane life style, and today the front
page is often a collage of merino scarves
and expensive glassware. In other cases,
"He's unbeatable on carpet."
the gatekeeping seems more premed­
itated. If you're fourteen, Snapchat's
•
user interface is intuitive; if you're
•
twenty-two, it's intriguing; if you're
over thirty-five, it's impenetrable. This
then maybe you give him a little smack
to show you're paying attention."
Although redditors didn't yet know
Huffman can no longer edit the site
encourages old people to self-deport.
indiscriminately, but his actions laid
Huffman and his college roommate,
bare a fact that most social-media com­
Alexis Ohanian, founded Reddit a few
it, Huffman could edit any part of the
panies go to great lengths to conceal­
weeks after graduating from the Uni­
site. He wrote a script that would auto­
that, no matter how neutral a platform
versity of Virginia, in 2005. The first
matically replace his usemame with those
may seem, there's always a person be­
people to show up were, like t h e
ofThe_Donald's most prominent mem­
hind the curtain.''I fucked up,"Huffman
co-founders, the kind of strong-headed
bers, directing the insults back at the in­
wrote in an apology the following week.
young men who got excited about com­
sulters in real time: in one comment,
"More than anything, I want Reddit
puter programming, video games, and
"Fuck u/Spez"became "Fuck u/ Trump­
to heal, and I want our country to heal."
edgy, self-referential humor. Reddit's
shaker"; in another, "Fuck u/Spez" be­
Implicit in his apology was a set of
system
came "Fuck u/MAGAdocious."
questions, perhaps the central ques­
is to say anarchic. Anyone could post
was
purely democratic, which
T he_Donald's users saw what was
tions facing anyone who worries about
happening, and they reacted by spin­
the current state of civic discourse. Is
ning a conspiracy theory that, in this
it possible to facilitate a space for open
of a page. At the time, Facebook was
case, turned out to be true.
dialogue without also facilitating
available only to college students, and
before joining it you had to provide
any link, and the ones that got the
most "upvotes" would rise to the top
"Manipulating the words of your
hoaxes, harassment, and threats of vi­
users is fucked," a commenter wrote.
olence? Where is the line between au­
your real name, your birthday, and a
"Even Facebook and Twitter haven't
thenticity and toxicity? What if, after
valid school e-mail address-the equiv­
stooped this low."
technology allows us to reveal our inner
alent of being carded at the door. To
"Trust nothing."
voices, what we learn is that many of
join Reddit, all you needed was a user­
T he incident became k n o w n a s
us are authentically toxic?
name that hadn't been claimed yet. You
Spezgiving, a n d it's still invoked, in­
ternally and externally, as a paradig­
could start as many anonymous ac­
he only way to understand the In­
counts as you wanted, which gave rise
ternet, at least at first, was by met­
to creativity, and also to mischief.
reach. Social-media platforms must
T
aphor. "Web" and "page" and "super­
Back then, Ohanian was ungainly
do something to rein in their users,
highway"are metaphors. So are "link,"
and clean-shaven, and he was often
the consensus goes, but not that.
"viral," "post," and "stream." Last year,
photographed in a hoodie and with a
matic example of tech-executive over­
60
THE NEW YOI\KEI\, MAII.CH 19, 2018
goofy smile.At his wedding, last year,
Y ishan Wong, an engineer who
report it and the company would re­
wearing a beard and an Armani tux­
had worked at Facebook, was then
move it. In retrospect, this seems like
edo, he was nearly unrecognizable.
Reddit's C.E.O. He implied that he'd
a straightforward business decision,
(The paparazzi weren't too interested
banned Jailbait only because the sub­
but some redditors treated it as the
in him, though, given that his bride
reddit had violated U.S. law. "We
first in an inevitable parade of hor­
was Serena Williams.) Huffman, on
stand for free speech," he wrote i n
rors. "This rule is stupid and sup­
the other hand, has always looked
a n internal post, i n 2012. Reddit's
presses our rig hts," u/penisfucker­
mcgee commented.
more or less the same: bright-blue
goal, he continued, was to "become
eyes, chipmunk teeth, and a thatch of
a universal platform for human dis­
A few months later,Reddit banned
blond hair.
course." Therefore, "it would not do
five of its most egregious communi­
ties, including r/FatPeopleHate and
A few months after Reddit launched,
if, in our youth, we decided to cen­
Huffman created the first constraints.
sor things simply because they were
r/ShitNiggersSay. Again redditors were
People were posting links to vulgar
distasteful."
apoplectic ("We may as well take a one
and violent content-which was fine,
At the time, Wong's free-speech
except that Huffman wanted users to
absolutism was ubiquitous in Silicon
every day, strident misogynists called
have some idea of what they were
Valley.Twitter's executives referred to
Pao a tyrant, an ''Asian slut," or worse.
way ticket to North Korea"). Almost
about to click on, so that they could
their company as "the free-speech
(Yes, it gets worse.) She resigned in
avoid, say, inadvertently opening porn
wing of the free-speech party." Face­
July, 2015. "The Internet started as a
in front of their bosses. Huffman la­
book's original self-description, "an
bastion for free expression," she wrote
belled some content N.S.F.W.-not
online directory that connects people
in the Washington Post. "But that bal­
safe for work-and separated it from
through social networks at colleges,"
ancing act is getting harder.The trolls
everything else.That was the end of
had evolved into a grandiose mission
are winning."
pure democracy.
statement: "Facebook gives people the
In 2006, Ohanian and Huffman
power to share and make the world
sold Reddit to Conde Nast, a media
more open and connected."With the
0
ver time, social networks have
turned into institutions. More
conglomerate that owns more than
Arab Spring fresh in everyone's mind,
than two billion people now use Face­
twenty magazines, including this one.
few questioned the assumption that
book. I n other words, the company
(Reddit now operates independently.)
"giving people the power" would in­
has achieved its mission of making
The sale made them twenty-two-year­
evitably lead to social progress. Barack
the world more connected. In 2016,
old mill ionaires, but they didn't fit in
Obama, who had been carried into
that meant, among other things, mak­
at a large corporation, and three years
office by a social-media groundswell,
ing the American electorate more
connect·ed to white supremacists,
later they left. In their absence, the
often expressed a similar optimism
party got bigger and weirder, and om­
about the salubrious effects of the In­
armed militias, Macedonian fake­
inous cliques started to gather in the
ternet. "In the twenty-first century,
news merchants, and micro-targeted
corners. One popular subreddit, r!Jail­
information is power," Obama said in
campaign ads purchased in rubles. "I
bait, was devoted to sexually sugges­
a 2011 speech on Middle East policy.
continue to believe Mr.Trump will
tive photos of young-looking women.
"The truth cannot be hidden.... Such
This was profoundly creepy, but prob­
not be President," Obama said that
year,despite the mounting aggression
ably not illegal-the subreddit's users
in some online forums."And the rea­
swore that all the women in the pho­
son is because I have a lot of faith in
tos were eighteen or older-and Red­
the American people." (In response
dit allowed the community to grow.
to Obama's remarks,a commenter on
In September of 2011, Anderson Coo­
The_Donald wrote, "FUCK THAT
per discussed the subreddit on CNN.
LOW
"It's pretty amazing that a big corpo­
ENERGY
CUCK!")
Shortly after the election,Brad Par­
ration would have something like this,
scale, the Trump campaign's top dig­
which reflects badly on it," he said.
ital strategist, told Wired, "Facebook
Traffic to Jailbait quadrup]ed over­
open discourse is important even if
and Twitter were the reason we won
night. Twelve days later, after some­
what is said does not square with our
this thing."Reddit was also an impor­
one in the group apparently shared a
worldview."
nude photo of a fourteen-year-old
Wong left the company in 2014,
tant part o f Trump's strategy. Parscale
wrote-on Reddit, naturally-that
girl, the community was banned.And
after two and a half years. His suc­
"members here provided considerable
yet the founder of Jailbait, an infa­
cessor was Ellen Pao, a former ven­
growth and reach to our campaign."
The_Donald, in particular, proved a
mous troll who went by u/V iolen­
ture capitalist.She lasted eight months.
tacrez, was allowed to stay on Red­
Early in her tenure,Reddit announced
fecund host cell for viral memes. On
dit, as were some four hundred other
a crackdown on involuntary pornog­
July 2, 2016, Trump tweeted a photo
communities he'd created-r!Jewmer­
raphy. If you found a compromising
collage of Hillary Clinton, piles of
ica, r/ChokeABitch, and worse. (Yes,
photo of yourself circulating on Red­
cash, and the phrase "Most Corrupt
it gets worse.)
dit without your consent, you could
Candidate Ever!" written inside a
THE NEW YOI\�ER, MAI\CH 19, 2018
61
"
'
wE RE THE MEDIA
NOW").
of throwing Communists out of heli­
ics called attention to the image's anti­
T he_Donald, with more than half
copters, is alt-right slang for murder.
Semitic implications, The_Donald's
a million subscribers, is by far the big­
T he_Donald accounts for less than
users rushed to Trump's defense, post­
gest pro-Trump subreddit, but it ranks
one per cent of Reddit's traffic, but it
ing photos of other six-pointed stars
just below No. 150 on the list of all
occupies far more than one per cent
in innocuous contexts. "Where is the
subreddits; it's roughly the same size
of the Reddit-wide conversation. Trolls
set a cunning trap. By ignoring their
six-pointed star. When Trump's crit­
Don:
outrage from the liberal left on this
as r!CryptoCurrency and r!Comic­
one?" a user wrote, beneath a photo of
Books. "Some people on The_Donald
provocations, you risk seeming com­
a "Frozen"-themed sticker book with
are expressing their genuine political
plicit. By responding, you amplifY their
a star on its cover. A few hours later,
beliefs, and obviously that's something
message. Trump, perhaps the world's
Trump tweeted the same photo, with
we want to encourage,"Huffman said.
most skilled troll, can get attention
a version of the same question, fol­
"Others are maybe not expressing sin­
whenever he wants, simply by being
lowed by "Dishonest media! #Frozen."
cere beliefs, but are treating it more
outrageous. Traditional journalists and
During the campaign, Trump, or
like a game-If I post this ridiculous
editors can decide to resist the bait,
someone typing on his behalf, par­
or offensive thing, can I get people to
and sometimes they do, but that op­
ticipated in Reddit's signature inter­
upvote it? And then some people, to
tion isn't available on user-generated
view format-an A.M.A., for "ask m e
quote 'The Dark Knight,' just want to
platforms. Social-media executives
anything." In response to a question
watch the world burn."On some smaller
claim to transcend subjectivity, and
they have designed their platforms to
about the "protected class of media
far-right subreddits, the discourse is
elites," Trump wrote, " I have been
more unhinged. One, created in July
be feedback machines, giving us not
very concerned about media bias and
of 2016, was called r/Physical_Removal.
what we claim to want, nor what might
the total dishonesty of the press. I
According to its "About Us" section, it
be good for us, but what we actually
think new media is a great way to get
was a subreddit for people who believe
pay attention to.
out the truth." This drew hundreds
that liberals "qualifY to get a helicop­
There are no good solutions to this
of jubilant comments (u!RAINBOW_
"
DILDO: "daddy YES ; u/CantConthe-
ter ride.""Helicopter ride," an allusion
problem, and so tech executives tend
to Augusto Pinochet's reputed habit
to discuss it as seldom as possible, and
only in the airiest of platitudes. Twit­
ter has rebuffed repeated calls to ban
President Trump's account, despite his
many apparent violations of company
policy. (If tweeting that North Korea
"won't be around much longer" doesn't
break Twitter's rule against "specific
threats of violence," it's not clear what
would.) Last fall, on his Facebook page,
Mark Zuckerberg addressed-sort of,
obliquely-the widespread critique that
his company was exacerbating politi­
cal polarization. "We'll keep working
to ensure the integrity of free and fair
elections around the world, and to en­
sure our community is a platform for
all ideas and force for good in democ­
racy," he wrote, then stepped away as
a global howl of frustration grew in
the comments.
I asked a few social-media executives
to talk to me about all this. I didn't ex­
pect definitive answers, I told them; I
just wanted to hear them think through
the questions. Unsurprisingly, no one
jumped at the chance. Twitter mostly
ignored my e-mails. Snapchat's P.R. rep­
resentatives had breakfast with me once,
then ignored my e-mails. Facebook's
representatives talked to me for weeks,
asking precise, intelligent questions, be­
fore they started to ignore my e-mails.
Reddit has more reason to be trans-
parent. It's big, but doesn't feel indis­
r/Coontown. "There was pushback,"
all of them himself on the first day."My
pensable to most Internet users or, for
Huffman told me."But I had the moral
political views might not be exactly
that matter, to most advertisers. More­
over, Anderson Cooper's CNN seg­
authority, as the founder, to take it i n
what you'd predict," he said. "I'm a gun
stride." I f Pao w a s like a forbearing
owner, for example. And I don't care
ment was hardly the only bit of vividly
parent, then Huffman's style was closer
all that much about politics, compared
terrible press that Reddit has received
to "I brought you into this world, and
to other things." He speaks in quick
over the years.All social networks con­
I can take you out of it.""Yes, I know
bursts, with an alpha-nerd combina­
tain vitriol and bigotry, but not all
that it's really hard to define hate
tion of introversion and confidence. His
social networks are equally associated
speech,and I know that
opinion about Trump is
with these things in the public imag­
any way we define it
that he is incompetent and
ination. Recently, I typed "Reddit is"
has the potential to set
that his Presidency has
into Google.Three of the top suggested
a dangerous precedent,"
mostly been a failure. But,
auto-completions were "toxic," "can­
he told me."I also know
he told me, ''I'm open to
cer," and "hot garbage."
that a community called
counterarguments."
Huffman,after leaving Conde Nast,
Coontown is not good
T h a t af te rnoo n , I
spent a few months backpacking in
for Reddit." In most
watched Huffman make
Costa Rica, then founded a travel com­
cases, Reddit didn't sus­
a sales pitch to a group of
pany called Hipmunk. In July, 2015, he
pend individual users'
executives from a New
York advertising agency.
returned to Reddit as C.E.O. In a post
a c c o u n t s , H u ff m a n
about his "top priority" in the job, he
said: "We just took
wrote, "The overwhelming majority
away the spaces where they liked
of content on reddit comes from won­
to hang out, and went, 'Let's see if
derful, creative, funny, smart, and silly
this helps."'
audience into a stable revenue stream,
and its representatives spend a lot of
time trying to convince potential ad­
communities.There is also a dark side,
communities whose purpose is repre­
Like many platforms,Red­
dit has struggled to convert its huge
eddit's headquarters, in a former
vertisers that Reddit is not hot gar­
radio tower in downtown San
bage. Huffman sat at the head of a long
gation to support them .. .. Neither
R
Fran cisco, look like a stereotypical
table, facing a dozen men and women
Alexis nor I created reddit to be a bas­
startup office: high concrete ceilings,
tion of free speech."This was shock­
a large common area with beer and
in suits.The "snarky, libertarian" ethos
ing, and about half true.When free­
kombucha on tap. Each desk is deco­
from me as a twenty-one-year-old. I've
speech absolutism
since grown out of that, to the relief of
hensible, and we don't have any obli­
of early Reddit, he said, "mostly came
in vogue, Red­
rated aggressively with personal flair-a
dit's co-founders were as susceptible
"Make Reddit Great Again" hat, a
everyone."The executives nodded and
to its appeal as anyone. In 2012, a Forbes
glossy print magazine called Meme In­
chuckled. "We had a lot of baggage,"
reporter asked Ohanian how the
sider. Working at Reddit requires pay­
he continued. "We let the story get
Founding Fathers might have reacted
ing close anthropological attention to
was
away from us. And now we're trying
to Reddit. "A bastion of free speech on
the motley tastes of redditors, and it's
the World Wide Web? I bet they would
not uncommon to see groups of fit,
Later, Huffman told me that get­
like it," Ohanian responded. "I would
well-dressed employees cheerfully dis­
ting Reddit's shit together would re­
love to imagine that 'Common Sense'
cussing the most recent post on r!Cat­
quire continual intervention. "I don't
would have been a self-post on Red­
Dimension or r/PeopleFuckingDying.
dit, by Thomas Paine, or actually a red­
The first morning I visited the office,
Friday and go,'Mission accomplished­
to get our shit together."
think I'm going to leave the office one
I ran into Huffman, who was wearing
we fixed the Internet,"' he said. "Every
Still, Ohanian and Huffman never
jeans, a T -shirt, and Adidas indoor­
day, you keep visiting different parts
took their own rhetoric too literally.
soccer shoes, as he tried to persuade an
of the site, opening this random door
The site's rules were brief and vague,
employee to buy a ticket to Burning
or that random door-'What's it like
and their unwritten policy was even
Man. Huffman is far more unfiltered
in here? Does this feel like a shitty
simpler. "We always banned people,"
than other social-media executives, and
place to be? No, people are generally
every time he and I talked in the pres­
having a good time, nobody's hatch­
ditor named T_Paine."
Huffman told me."We just didn't talk
about it very much." Because Reddit
ence of Reddit's head of P.R., he said
ing any evil plots, nobody's crying.
was so small, and misbehavior rela­
at least one thing that made her wince.
O.K., great.' And you move on to the
tively rare, Huffman could do most of
"There's only one Steve," Ohanian told
next roon1."
"It wasn't well thought out or even ar­
for better or worse, that's the Steve
ticulated, really. It was That guy has
you're gonna get." I had a list of deli­
the banning himself, on an ad-hoc basis.
me. "No matter when you catch him,
n January, Facebook announced that
the N-word in his username? Fuck
cate topics that I planned to ask
I
that.' Delete account."
Huffman about eventually, including
designed to connect friends and fam­
ily-and it has excelled at that," a prod­
it would make news less visible in its
users' feeds. "Facebook was originally
As C . E.O., Huffman continued
allegations of vote manipulation on
the trend Pao had started, banning a
Reddit's front page and his personal
uct manager named Samidh Chak­
few viciously racist subreddits such as
feelings about Trump. Huffman raised
rabarti wrote on a company blog. "But
THE NEW YOI\�ER, MAI\CH 19, 2018
63
as unprecedented numbers of people
challenge" for himself: "enforcing our
enthusiastic about Bernie or Trump
channel their political energy through
policies and preventing misuse of our
in real life, so they go on Reddit and
this medium, it's being used in unfore­
tools."This seems to be a reversal for
talk about how much they like Ber­
seen ways with societal repercussions
Zuckerberg, who was once a fake­
nie or Trump. So far, so good."Then
that were never anticipated." It wasn't
news truther.Two days after the 2016
he laid out the dog hypothesis, which
the most effusive mea culpa in history,
election, he said, "The idea that fake
his fellow social-media executives al­
but by Facebook's standards it amounted
news on Facebook, of which it's a very
most never acknowledge-that real­
to wailing and gnashing of teeth. "We
small amount of the content, in­
ity is also a reflection of social media.
want to make sure that our products are
fluenced the election in any way, I
"All sorts of weird things can happen
not just fun, but are good
think, is a pretty crazy
online," he said. "Imagine I post a
for people," Mark Zuck­
idea. Voters make deci­
joke where the point is to be offen­
er berg told the Times.
sions based on their lived
sive-like, to imply, 'This is some­
Direct pronouncements
experience." This was a
thing that a racist person would say'­
from him are so rare that
pretty crazy idea, and
but you misread the context and think,
even this pabulum was
Zuckerberg has been
'Yeah, that racist guy has a good point.'
treated as push-alert­
walking it b a c k ever
That kind of dynamic, I think, ex­
worthy news.
In retrospect, although
since. It's obvious that
plains a lot of what happened on
w h a t we s e e online
The_Donald, at least in the early
Facebook denies this, it
affects how we think and
days-someone keeps pushing a joke
seems clear that the com­
feel. We know this i n
or a meme to see how far they can
part because Facebook
take it, and the answer turns out to
pany was preparing for a
blow that was about to land. On Feb­
has done research on it. In 2012, with­
ruary 16th, the special counsel Robert
out notice or permission, Facebook
Leftist commu n ities on Reddit
Mueller filed an indictment against
tweaked the feeds of nearly seven hun­
often implore the company to ban
several Russian individuals and busi­
dred thousand of its users, showing
The_Donald. So far, Huffman has de­
nesses, including the Internet Research
one group more posts containing "pos­
murred."There are arguments on both
Agency, a company aligned with the
itive emotional content" and the other
sides," he said, "but, ultimately, my
Kremlin. The indictment mentioned
more "negative emotional content."
view is that their anger comes from
Facebook thirty-five times, and not in
Two years later, Facebook declassified
feeling like they don't have a voice, so
be Pretty fucking far."
ways that made the platform seem like
the experiment and published the re­
it won't solve anything ifl take away
a "force for good in democracy." Ac­
sults. Users were livid, and, after that,
their voice." He thought of something
cording to recent reporting by the Daily
Facebook either stopped conducting
else to say, but decided against it.Then
Beast, the Internet Research Agency
secret experiments or stopped admit­
he took a swig of beer and said it any­
way. "I'm confident that Reddit could
also seeded Reddit with disinforma­
ting to them. But the results of the
tion during the 2016 election. (A group
experiment were clear: the people with
sway elections," he told me. " W e
of impostors even tried to set up an
happier feeds acted happier, and vice
wouldn't d o it, o f course. And I don't
A.M.A.) Last Monday, the Washing­
versa. The study's authors called it
know how many times we could get
ton Post reported that the Senate In­
"massive-scale emotional contagion."
away with it. But, if we really wanted
telligence Committee will question
Since then, social media has only
to, I'm sure Reddit could have swayed
Reddit executives about this; the same
grown in size and influence, and the
at least this election, this once.''That's
day, Huffman admitted that the com­
persuasive tools available to advertis­
a terrifYing thought. It's also almost
pany had "found and removed a few
ers, spies, politicians, and propagan­
certainly true.
hundred accounts" associated withRus­
dists have only become sharper.During
sian propaganda. (A Reddit represen­
the 2016 election, a few Russian im­
tative told me that the company has
postors affected many Americans' be­
0
n August nth, Huffman's alma
mater, the University of Virginia,
been cooperating with congressional
liefs and, presumably, votes. With an­
was overrun by white nationalists car­
investigators "for months," although
other election coming up, most of the
rying torches. "I was on a plane when
they haven't spoken about it publicly.)
loopholes that the Russians exploited
I saw the news, and I got really emo­
As in all such disinformation cam­
have not been closed, and the main
tional," Huffman said. He told his
paigns, the Russians did not act alone:
loophole-the open, connected, mas­
employees, "If any of these people are
their messages were upvoted and re­
sively contagious world of social
on Reddit, I want them gone. Nuke
peated by thousands of unsuspecting
media-might not be closable.
'em."This felt cathartic, but personal
Americans. "I believe the biggest risk
W hen I raised this issue with
catharsis is an awful way to make pol­
we face as Americans is our own abil­
Huffman over dinner last summer,
icy. "Luckily, my team knew me well
ity to discern reality from nonsense,"
he said, "I go back and forth o n
enough to go, 'Steve, you're pissed off
Huffman wrote. "I wish there was a
whether Reddit i s the tail o r the dog.
right now. Let's talk about it more ra­
solution as simple as banning all pro­
I think it's a bit of both." First, he
tionally on Monday.'"
paganda, but it's not that easy."
laid out the tail hypothesis: "Reddit
Early the next week,Reddit banned
is a reflection of reality. People are
Physical_Removal. In Charlottesville,
Zuckerberg recently set a "personal
64
THE NEW YOI\KEI\, MAII.CH 19, 2018
James Alex Fields, one of the white
scribed, respectively, as good cop, bad
symbol.But, even so, there's a lot that's
nationalists, had driven a car into a
cop, and RoboCop. Community stays
clear-cut." I asked whether the same
logic-that the Nazi flag was an in­
crowd of counterprotesters, injuring
in touch with a cross-section of red­
nineteen and killing a woman named
ditors, asking them for feedback and
herently violent symbol-would apply
Heather Heyer."This is a good thing,"
encouraging them to be on their best
to the Confederate flag, or the Soviet
the top post on Physical_Removal
behavior. When this fails and reddi­
flag, or the flag under which King
read. "They are mockeries of life and
tors break the rules, trust and safety
need to fucking go." Reddit had a rule
punishes them. Anti-evil, a team of
Richard fought the Crusades. "We
can have those conversations in the
prohibiting content that "encourages
back-end engineers, makes software
future," Ashooh said. "But we have
or incites violence," and this was a vi­
that flags dodgy-looking content and
to start somewhere."
olation of that rule. Huffman said,
sends that content to humans, who de­
"We'd had our eye on that commu­
cide what to do about it.
At 10 A.M., the trust-and-safety
team posted the announcement and
nity for a while, and it felt good to
Ashooh went over the plan for the
get rid of them, I have to say. But it
day. All at once, they would replace
ting me do DylannRooflnnocent,"
still didn't feel like enough."
the old policy with the new policy,
one employee said. "That was one of
the ones I really wanted."
began the purge."Thank you for let­
" Encouraging or inciting violence"
post an announcement explaining the
was a narrow standard, and Huffman
new policy, warn a batch of subred­
"What is ReallyWackyTicTacs?"
and his team agreed to expand it.Four
dits that they were probably in vio­
another employee asked,looking down
the list.
words became thirty-six: "Do not post
lation of the new policy, and ban an­
content that encourages, glorifies, in­
other batch of subreddits that were
"Trust me, you don't want to know,"
cites, or calls for violence or physical
flagrantly, irredeemably in violation.
Ashooh said."That was the most un­
harm against an individual or a group
I glanced at a spreadsheet with a list
pleasant shit I've ever seen, and I've
of people; likewise, do not post con­
of the hundred and nine subreddits
spent a lot of time looking into Syr­
ian war crimes."
tent that glorifies or encourages the
that were about to be banned (r/KKK,
abuse of animals."This, too, required
r!KillAIJews,
I
r/KilltheJews, r/Kill­
Some of the comments on the an­
interpretation, and forced the com­
theJoos), followed by the name of the
nouncement were cynical. "They don't
pany to create a non-exhaustive list of
employee who would carry out each
actually want to change anything," one
exceptions ("educational, newsworthy,
deletion, and, if applicable, the rea­
redditor wrote, arguing that the bans
artistic, satire, documentar y"). Still, it
son for the ban ("mostly just swasti­
were meant to appease advertisers. "It
made the team's intentions clearer.Jes­
kas?"). "Today we're focussing on a
was, in fact, never about free speech,
sica Ashooh, Reddit's head of policy,
lot of Nazi stuff and bestiality stuff,"
it was about money. " One trust-and­
safety manager, a young woman wear­
spent four years as a policy consultant
Ashooh said. "Context matters, of
in Abu Dhabi. "I know what it's like
course, and you shouldn't get in trou­
ing a leather jacket and a ship cap­
to live under censorship," she said."My
ble for posting a swastika if it's a his­
tain's cap, was in charge of monitoring
internal check, when I'm arg uing for
a restrictive policy on the site, is Do
torical photo from the 1936 Olym­
the comments and responding to the
pics, or if you're using it as a Hindu
most relevant ones. " Everyone seems
I sound like an Arab government? If
so, maybe I should scale it back." On
the other hand, she said, "people hide
behind the notion that there's a bright
line between ideology and action, but
some ideologies are inherently more
violent than others."
In October, on the morning the new
policy was rolled out, Ashooh sat at a
long conference table with a dozen
other employees. Before each of them
was a laptop, a mug of coffee, and a
few hours' worth of snacks. "Welcome
to the Policy Update War Room," she
said. "And, yes, I'm aware of the irony
of calling it a war room when the point
is to make Reddit less violent, but it's
too late to change the name. "The job
of policing Reddit's most pernicious
content falls primarily to three groups
of employees-the community team,
the trust-and-safety team, and the anti­
evil team-which are sometimes de-
"Are we gonna have to scrape the Daddy decal off the minivan?"
The idea behind riP/ace was to put up a very simple microcosm ifthe Internet and see what happens.
computer scientists at three univer­
to be taking it pretty well so far," she
said, standing up. "How many cheese
said. "There's one guy,freespeechwar­
sticks is too many in one day? At what
sities published a study called "You
rior,who seems very pissed,but I guess
point am I encouraging or glorifYing
Can't Stay Here:The Efficacy ofRed­
that makes sense,given his username."
violence against my own body?"
dit's 20�5 Ban Examined Through
"People are making lists of all the
Nazi subs getting banned, but nobody
"It all depends on context," Ashooh
said.
Hate Speech." They parsed a data
set of a hundred millionReddit posts.
Did the ban "diminish hateful be­
has noticed that we're banning besti­
I understood why other compa­
ality ones at the same time," Ashooh
nies had been reluctant to let me see
havior " over all, or did it merely "re­
said.
something like this. Never again
locate such behavior to different parts
of the site"?They concluded that the
"No one wants to admit it," an em­
would I be able to read a lofty phrase
ployee said." 'Guys,I was just browsing
about a social-media company 's shift
ban had worked: "Users participat­
r/HorseCock and I couldn't help but
in policy-"open and connected," or
ing in the banned subreddits either
"encouraging meaningful interac­
left the site or (for those who re­
notice ..."'
The woman in the captain's cap
tions "-without imagining a group
mained) dramatically reduced their
said,"O.K., someone just asked,'How
of people sitting around a conference
hate speech usage."
will the exact phrase "kill yourself
" be
room, eating free snacks and making
Melissa Tidwell, Reddit's general
fallible decisions.Social networks, no
counsel, told me,"I
"It all depends on context," Ashooh
matter how big they get or how fa­
ple who repeat the mantra 'Free speech!'
said. "They're going to get tired of
miliar they seem,are not ineluctable
but then have nothing else to say.Look,
hearing that, but it's true."
forces but experimental technologies
free speech is obviously a great ideal to
handled?' "
am
so tired of peo­
"Uh-oh, looks like we missed a
built by human beings. We can tell
strive toward.Who doesn't love freedom?
bestiality sub," the woman in the
ourselves that these human beings
Who doesn't love speech? But then, in
captain's cap said."Apparently, Sex­
aren't gatekeepers, or that they have
practice, every day,gray areas come up."
WithDogs was on our list, but Dog­
cleansed themselves of all bias and
Earlier that day,I<i watched Tidwell and
Sex was not."
emotion, but this would have no re­
a colleague spend several minutes debat­
lation to reality. "I have biases, like
ing whether a soft-core porn subreddit,
"Did you go to DogSex?" Ashooh
everyone else,"Huffman told me once.
r/GentlemenBoners,should be included
"Yep."
"I just work really hard to make sure
in standard search results.
"And what' s on it?"
that they don't prevent me from doing
"I mean . . ."
what's right."
said.
"Are there people having sex with
In a perfect world, a thirty-four­
"Does free speech mean literally
anyone can say anything at any time?"
Tidwell continued. "Or is it actually
year-old in soccer shoes wouldn't have
more conducive to the free exchange
"Oh, yes, very much."
such fearsome power. In the world
of ideas if we create a platform where
dogs?"
"Yeah, ban it."
we live in,the least social-media ex­
women and people of color can say
"I'm going to get more cheese
ecutives can do is acknowledge that
what they want without thousands of
sticks," the woman in the captain's cap
power. Last November, a group of
people screaming,'Fuck you,light your-
66
THE NEW YOI\KEI\, MAII.CH 19, 2018
self on fire, I know where you live'? If
supporters, patriotic libertarians, and
said. ''A lot of memes, some Pokemon,
your entire answer to that very diffi­
pre-poHtical teen-agers-decided to
and a barrage of clicks."
cult question is 'Free speech,' then, I'm
draw an American flag in the center
"If there's ever a Reddit musical,
sorry, that tells me that you're not re­
of the square. They congregated at
that wouldn't be a bad ti tle," Huffman
ally paying attention."
r/ AmericanFlaglnPlac e, where they
said. "Maybe I should write a memoir
hashed out the exact dimensions, the
called 'A Barrage of Dicks."'
t has become a tradition for tech
shapes ofthe stars and stripes, and strat­
companies to release elaborate, self­
egies for repelling invaders. Meanwhile,
referentialjokes every April Fools' Day.
a group ofnihilists at r/BlackVoid pre­
Lunch was served: a shrimp-and­
The point is to generate some free pub­
pared to blot out whatever the other
lentil salad and a vegan bean fricassee.
licity that will make the company seem
groups created.
People stood in the common area, hold­
I
"I have faith in our people," Slowe
said.
quirky and relatable, but it can also have
Wardle went to great lengths to show
the opposite effect, especially when the
me that Place was a pure democracy­
premise of the joke is Silicon Valley's
the algorithm was design ed so that,
One employee, reading the com­
unprecedented power. A few years ago,
once it went lve,
i
all he could do was
ments, said, "A bunch of people are
Twitter announced that it would start
watch, along with everyone else. Now
finding swastikas and then telling ev­
charging for vowels. Last year, Coogle
that he was watching, he seemed deeply
eryone else where they are, so that peo­
shared a mockup of its new data cen­
nervous. "The idea was 'Let's put up a
ple can go get rid of them."
ter on Mars, and Amazon revealed
very simple microcosm ofthe Internet
"I just saw it!" another said. H e
voice-recognition sofrware that would
and just see what happ ens, " ' he said.
pointed to a section of the screen. As
we watched, one swastika was erased
ing paper plates, watching a live feed
of Place on a wall-mounted TV.
take commands from pets. The com­
"Reddit itself is not the most complex
panies hadn't actually initiated any of
idea. It's sort of a blank canvas. Th e
and another was modified to become a
these projects, but they probably could,
co mmunit y takes that and does all sorts
one day, if they wanted to. Get it?
of creative things with it."
Windows 95 logo. Eventually, the swas­
tika-makers got bored and moved on.
Last April Fools', instead of a par­
''And some terrible things," I said.
At one point, the American flag
ody announcement, Reddit unveiled a
He paused."I'm pretty confident,"
caught on fire; the fire was stamped
genuine social experiment. Itwas called
he said.
r/Place, and it was a blank square, a
hundred per cent confident." Already,
"Feels like watching a football game
thousand pixels by a thousand pixels.
one of the top comments on Place read,
in extreme slow motion," one said.
"Or likewatching the election results."
''I'd be lying
if I said I was a
out, and the Reddit employees cheered.
In the beginning, all million pixels were
"I give this an hour until swastikas."
white. Once the experiment started,
Later, one of Wardle's colleagues told
anyone could change a single pixel,
me, "That was what kept Josh up at
Toward the end, the square was a
anywhere on the grid, to one ofsixteen
night. Before this went live, he was lit­
dense, colorful tapestry, chaotic and
"Oh, God, don't say that."
colors. The only restriction was speed:
erally calculating, '0. K., it takes a min­
strangely captivating. It was a collage
the algorithm allowed each redditor to
imum of seventeen pixels to make a
of hundreds of incongruous images:
alter just one pixel every five minutes.
swastika-what if we open this up to
logos of colleges, sports teams, bands,
"That way, no one person can take
the world, and the headline the next
and video-game companies; a tran­
over-it's too slow," Josh Wardle, the
day is "REDDTT :
scribed monologue from "Star Wars";
A PLACE TO DRAW
SWASTIKAS ON THE INTERNET"?"'
likenesses of He-Man, David Bowie,
Place, explained. "In order to do any­
The upper-left corner was a choppy,
the "Mona Lisa," and a former Prime
thing at scale, th ey're gonna have to
cooperate."
flickering purple, as the Blue Empire
Minister of Finland. In the final hours,
and the Red Empire battled for dom­
shortly before the experiment ended
and the image was frozen for poster­
Reddit product manager in charge of
Place had been active for about
inance. A graffiti arti st, or artists, wrote,
rwenry minutes when I stopped by, and
"9/11 was an inside job"; a few minutes
ity, BlackVoid launched a surprise at­
Wardle was huddled over his laptop,
later, the "was" turned into "wasn't," and
tack on the American flag. A dark fis­
fran tic ally refreshing dozens of tabs.
the "an" became "anime." Elsewhere,
sure tore at the bottom ofthe flag, then
So far, the square was mostly blank,
"Dick butt" became "Dick butter," then
overtook the whole thing. For a few
minutes, the center was engulfed in
with a few stray dots blinking in and
"Dick buffet." And then the swastikas
out of existence. But redditors were
app eared-just a few of them, but
darkness. Then a broad coalition ral­
making plans in the comments and, in
enough to make Wardle raise the hood
lied to beat back the Void; the stars
true Reddit fashion, clinging to those
of his sweatshirt, retreat into an empty
and stripes regained their form, and,
plans with cultish intensity.The Blue
conference room, and shut the door,
in th e end, the flag was still there .
Empire was conspiring to turn the
looking pallid.
The final image contained n o visi­
In his office, Huffman met with
ble hate symbols, no violent threats, not
vowed to make it red; already, they were
Chris Slowe, Reddit's first employee,
even much nudity. Late in the day,
on a war footi ng. Other groups planned
who is now the chief technical officer.
Wardle emerged from hiding, poured
whole square blue; the Red Empire
elaborate messages, fractal patterns,
and references to various memes. A
multi-partisan group-leftis ts, Trump
"How is Place going?" Huffman
asked.
"Pretty much as expected,'' Slowe
himself a drink, and pushed back his
hood. "It's possible that I will be able
to sleep tonight," he said. +
11-iE NEW YOI\�EI\. MARCH 19. 2018
67