18 APRIL 22, 2011 | THE BOSTON PHOENIX | THEPHOENIX.COM
S COT T M . L AC E Y
NEST BUILDERS
From left, Tristan Jehan,
Jim Lucchese, and Brian
Whitman in the Echo
Nest’s Davis Square
headquarters.
THE MOST IMPORTANT
MUSIC COMPANY ON EARTH
THE ECHO NEST IS READY TO FLY
_ BY CH RIS FA R AO N E
f
Music is data. A shitload of it packed
in every single song. To people, music
equals entertainment. To a computer, it’s a
precise stream of ones and zeros.
If you could teach computers to understand those ones and zeros — to digest them
like we do — then you could manipulate
sounds in ways even Prefuse 73 hasn’t
dreamed of. You could use your iPhone as
a violin. You could settle scores over who
liked an artist first — you, or your obnoxious
friend with the impossibly skinny jeans. You
could take your favorite Jay-Z album, tweak
one cut into a two-step swing remix, add
cowbell to another, and determine which
bangers the Grateful Dead would have been
most likely to cover in the early 1970s — all
on your smart phone, while in line for a
Metallica show.
You can do all this now. And you can
thank the Echo Nest: a small Somerville
company founded by two audiophilic tech
heroes, who have given developers everywhere the tools to shape the future of music.
The Echo Nest is so far ahead of the rest
of the music industry that their 15-minute
demo at the South by Southwest tech summit last month looked like a magic trick —
even to some of the highest-paid engineers
from the world’s biggest multimedia behe-
moths. After the Echo Nest’s presentation,
attendees rushed the microphones — all asking some form of the question, How the fuck did
you do that?
The short answer: the Echo Nest has
analyzed tens of millions of songs, as well
as virtually every fan and artist profile, blog
post, comment, and article about music
that’s available online. Through that process, they’ve amassed an incredibly dynamic
war chest of data — their Application
Programming Interface (API) — that can
power music apps in radical new ways. With
these extensive resources in their clutches,
the Echo Nest is the platform on which the
next generation of music fans will discover,
follow, remix, and recommend their favorite
bands. Most of what the company’s intelligence can do hasn’t even been invented yet.
CEO Jim Lucchese says the Echo Nest
doesn’t want to be the next MTV, or the next
Spotify, or the next Island Def Jam (IDJ).
Instead, they’ve sealed watershed deals
with those companies and many more, all
of which are tapping the Echo Nest API to
gauge what’s hot and what’s not for who,
what, when, and where. In its search of
the next killer app, IDJ has even given Echo
Nest’s community of 7000 independent
developers access to more than one million
songs to play with.
Yet despite receiving tons of tech press,
the Davis Square company remains in the
shadows of pop culture. They’re not even as
well-known as Pandora, which, according
to Lucchese, has taken a decade to index the
same number of songs that his crew could
crunch in two hours. For their scrapper status, Lucchese credits the fact that Echo Nest
provides no direct-to-consumer products,
and concedes that the business is hardly run
by marketing mavens. But he suspects they
won’t be underdogs for much longer.
“If there’s one thing that we’ve done a
terrible job of, it’s driving more mainstream
awareness to the awesome shit that people
are doing on our platform,” says Lucchese, an
attorney. “Maybe it’s because we’re founded
by two MIT PhDs and run by a lawyer. In
the developer community, we’re past being
considered the ‘smart little guys,’ and we’re
being recognized as a best-in-class platform.
Granted that it’s still kind of the early days
for [everybody else] to really understand what
we do, but now it looks like mobile apps are
changing that.”
AGREE TO DISAGREE
In the late 1990s, Brian Whitman was a frustrated computer scientist and electronic mu-
sician living in New York City, and performing under the name Blitter. He played regular
gigs and even dropped some vinyl, but spent
most of his time imagining ways to get his
music in front of potential fans. At the time,
he noticed that online message boards were
becoming increasingly populated by people
who were anxious to discuss — and, more
often, argue about — music trends and stylings. Somehow, Whitman thought, these
were not disparate musings.
“In hindsight,” he says, “this is all very
obvious. But when these communities were
forming, the people who were doing music
recommendation and retrieval weren’t looking at [blogs and message boards] — they
were just looking at the audio signals. . . .
I set out to prove that the more you know
about a community, the more you understand peoples’ preferences.”
Following his interest in the conversation around music, Whitman’s explorations brought him to the world-renowned
MIT Media Lab, where he eventually met
his philosophical nemesis and future business partner, Tristan Jehan: a soft-spoken,
French-born computer scientist, amateur
keyboardist, and researcher who cut his
teeth at UC-Berkeley’s Center for New Music
and Audio Technologies.
THEPHOENIX.COM | THE BOSTON PHOENIX | APRIL 22, 2011 19
Jehan’s view of music analysis was the
opposite of Whitman’s: he thought that
relationships between songs should be
derived by extracting and analyzing musical
metadata. Jehan came to MIT to prove that
sounds — as opposed to the worldwide dialogue about music — were the best barometers of listener taste.
“I’d been working on how to make computers better understand music,” says Jehan.
“Brian was looking at how computers could
understand music in the context of how
people speak about it on the Web.” Adds
Whitman: “You can’t just look at the audio
signals, and at the same time you can’t
ignore the audio — you have to know
what the song sounds like, and understand the conversation around it. You
need to do both.”
By the time that Whitman and
Jehan earned their doctorate degrees (in
machine listening and media arts and
sciences, respectively) in 2005, both were
considered all-stars in their parallel fields.
So it was fitting that they agreed to disagree,
and partnered to launch the Echo Nest out of
a small office in the same building where the
company now occupies several suites.
Using their complementary research as a
foundation, they wrote programs that crawl
the Internet (and streaming services like last.
fm), analyzing everything from comments
and discussion about songs and artists, to
the rhythm, harmony, and timbre of millions of actual tracks. Within two years,
they’d built the most powerful interactive
music database ever indexed as a single platform — an API they would come to call the
“Musical Brain.”
But while Whitman and Jehan had ideas
that could significantly impact listening
habits the world over, they still hadn’t figured out how to use their Musical Brain to
make money. “It was a pretty classic example
of two scientists running a business,” says
Whitman.
Luckily for them, the right people noticed
Echo Nest relatively early on. They attracted
investors like Barry Vercoe, a music-programming icon and a co-founder of the Media
Lab, and Don Rose, who co-founded the New
England–based imprint Rykodisc.
“They didn’t have any paying clients, and
they didn’t have any products, but I immediately liked them,” says Rose. “Before I did
anything with them, though, I wanted to
see if they could make it work. So I had them
sniff my iPod to make recommendations,
and lo and behold, what Pandora did manually over time, they were able to do in less
than a day.”
Rose’s son, Scotty, who now works at the
Echo Nest, was in a band with Lucchese. After
Lucchese learned about the company, he left
New York to join the Somerville operation fulltime and help write a business plan. He was
familiar with the music-licensing gauntlet
from working with platinum artists at the
mega-firm Greenberg Traurig, and had the
perfect tough-but-friendly attitude to navigate
the industry for them.
With Lucchese on board, in 2007 the
company was awarded more than $500,000
in grant money by the National Science
Foundation, and that year also attracted
the attention of the Boston-based Kelso
Management Company investment fund.
From there, the Echo Nest launched its developer API at the DEMO conference in Palm
Desert, California, in September 2008.
At the time, it was becoming ever more
clear that music search-and-discovery
tech was of paramount importance to the
industry. The leader in that arena was
Pandora, which had just launched a mobile
version of its software.
People tapped Pandora to direct them
toward music based on the songs they
already liked. But Pandora’s patented Music
Genome Project employs actual humans to
manually index tracks according to audio
criteria. Decisions as to what songs fit which
parameters are subjective, made by a person.
The Echo Nest’s approach, alternatively,
couldn’t be any more different. It is entirely
algorithmic. The Musical Brain makes those
judgment calls better than any teams of
humans could, Jehan and Whitman say
— and much, much faster.
Still, despite the Echo Nest’s booming
popularity in tech circles, it would take a
calculated sales pitch to entice big labels and
media companies.
“Everybody had already seen a lot
of PhDs give them the bullshit line
about having the science to understand
music,” says Lucchese, “so we had to
separate ourselves from the scores of illconceived companies that were doomed
to fail from the jump.”
RETHINKING
MUSIC
‘It’s fun, it’s
free, and
people who
aren’t music
geeks are now
able to do some
incredible stuff.’
API IS THE NEW MP3
On a now-storied fall weekend in late 2009,
about 200 techies gathered at Microsoft’s
New England Research and Development
Center on Memorial Drive in East Cambridge for the first stateside Music Hack
Day, which, according to Lucchese, is
“essentially a jam session” for music-tech
visionaries. For two straight days — some
participants stayed awake on Red Bull and
coffee the whole time — hackers, programmers, and developers tapped the Echo Nest
API (and those of other participating companies) in order to build apps that change
the way people find and digest music.
“You used to have all of these gatekeepers,” says Paul Lamere, director of the Echo
Nest’s developer community. “There were
the A&R folks, the producers, the labels,
the old-time DJs, the record store buyers —
they were all the tastemakers. Now that’s
all gone, which means people have to be
able to search through everything on their
own and make these decisions — and that’s
where the Echo Nest comes in . . . [Music
Hack Day] is a business-free zone, but it’s
great because kernels of ideas from there
have found their ways into real products
that help fans find what they want.”
Lamere sits at the helm of Boston’s
annual hack day, and of others in Cannes,
Stockholm, Barcelona, Amsterdam, Berlin,
London, and San Francisco. A guru’s
guru in music innovation, Lamere came
to the Echo Nest in early 2009 from Sun
Microsystems, where he’d spent the previous five years researching music exploration. Though excited about his work there,
Lamere, whose kids describe his music
taste as “dad-core,” says he became frustrated that Sun never intended to introduce his
inventions to the marketplace, and defected to the Echo Nest as a result. Within
three months of his arrival, the company
inked a deal with the music streaming
service Spotify to assist with playlist and
discovery services.
At hack days and on a daily basis, Lamere
helps guide inventors toward apps that are
already revolutionizing the commercial space.
Last December, the Echo Nest teamed with
Viacom for the MTV Music Meter, which pegs
the hottest rising artists on the Web, then
helps pair those acts with mtv.com readers
who might dig them. That same month,
they also announced a similar venture with
the British Broadcasting Company that helps
visitors to the BBC’s Music Showcase hub find
favorable clips and videos.
“We don’t care where the songs come
from,” says Lucchese. “What we do is sit on
top of the data and enable the experiences
through which people access [music]. We’ve
built our whole business on the idea that,
one day, kickass independent developers
will be able to build apps and release them
legally without ever talking to a lawyer.”
The Echo Nest’s biggest landmark deal
so far came this February, when they linked
with IDJ to bring developers closer to license
holders. With Lamere assisting, app wizards
can now write programs using the Echo Nest
platform — for everything from PCs to smart
phones — and release them commercially
with no hassle. Considering the depth of
IDJ’s roster, which includes everyone from
Ghostface to Fall Out Boy, the relationship
between the Echo Nest and IDJ has the potential to yield countless retail opportunities.
“My company is always looking to innovate,” says IDJ senior vice-president of digital and business development Jon Vanhala,
who helped shepherd the deal through.
“There’s no silver bullet in the app world
right now — nobody can tell you what’s
absolutely going to be cool or not. You have
to have some faith, and then let the market
decide. . . . We believe this community of
developers is exceptionally creative, and [the
Echo Nest] gives us access to all of that.”
SOMETHING FOR EVERYONE
If hack days are the Seattle grunge scene
2.0, then Echo Nest is Sub Pop circa 1991. At
least that’s how it seems now that the larger
music world is recognizing how integral app
advancements are to steering the industry
forward. On April 28, MTV will host its first
annual O Music Awards (“to celebrate online
creativity”), where Lamere and his teenage
daughter Jennie are up for Best Music Hack.
Their nominated app, Jennie’s Ultimate
Road Trip, uses the Songkick and Echo Nest
APIs to help music fans find tour dates for
their favorite artists in each town along any
travel route.
The Lameres engineered their app at
last year’s Boston Music Hack Day, as
did two of the other OMA nominees for
Best Music Hack. In fact, these powwows
have become ground zero for cataclysmic
developments, from the aforementioned
Invisible Instruments iPhone violin app,
to sQRatchLive, which allows users to
instantaneously buy whatever track a DJ is
playing in the club.
Moving forward, the Echo Nest’s new
visual remix tech will allow users to personalize music videos in real time, so that your
friends’ ugly faces — rather than Kanye’s
— will fill your screen. Equally awesome are
their soon-arriving “query-by-description”
apps, which are essentially digital versions
of the grease-ball snots behind record-store
counters to whom customers hum songs.
To keep tabs on all this, the Echo Nest even
launched its own blog, evolver.fm, to serve
as a critical clearinghouse for breaking
music-app developments.
“There is something for everyone out
there — it’s just a matter of finding what
works for you,” says evolver.fm editor Eliot
Van Buskirk. A former writer for CNET and
Wired, Van Buskirk has surveyed the digital
landscape for more than a decade; Echo
Nest hired him to review music apps and
evaluate the marketplace. “I’m testing an
app right now that quizzes you about what
you’re doing this weekend, and then picks
songs to go with that. And that’s kind of the
point to a lot of this — it’s fun, it’s free, and
people who aren’t music geeks are now able
to do some incredible stuff.”
“We understand the entire language
of music better than anyone else to date,”
says Lucchese, “and now we’ve put that
research and analysis into the hands application developers. . . . To us, there’s no
doubt about it — this is the new class of
creative people who are reshaping the role of
music in our lives. With them, we’re aiming toward where things are heading, and
making sure that we’re still around when it
explodes.” ^
Chris Faraone can be reached at cfaraone@phx.com.