Big data accessible to all
Ryan Shuttleworth, AWS Evangelist
Big Data
Compute
Lorem ipsum dolorStorage
sit
met,
consectetur
Bring compute capacity to the data
dipiscing elit. Etiam
uis ligula neque, eget
enenatis
sem. Personal
Suspendisse non eros
ulla, at placerat nibh.
Cras id lectus mattis est
llamcorper
blandit.
Proin ut nisi vitae enim
ulputate
tempor.
Phasellus id commodo
ros.
Mauris
nec
ignissim turpis. Nunc
Very large dataset
Cras id lectus mattis
est
ullamcorper
seeks
strong
&
consistent compute for
short term relationship,
possibly longer. GSOH a
plus
aws.amazon.com
Lorem ipsum dolor
amet,
consecte
adipiscing elit. Etia
quis ligula neque, eg
venenatis
se
Suspendisse non er
nulla, at placerat nibh
Cras id lectus mattis
est
ullamcorper
blandit. Proin ut nisi
vitae enim vulputate
tempor. Phasellus id
commodo
eros.
Mauris nec dignissim
turpis. Nunc
Storage
Big Data
Compute
Data has gravity
App
Data
App
http://blog.mccrory.me/2010/12/07/data-gravity-in-the-clouds/
Storage
Big Data
Compute
…and inertia at volume…
Data
http://blog.mccrory.me/2010/12/07/data-gravity-in-the-clouds/
Storage
Big Data
Compute
…easier to move applications to the data
Data
http://blog.mccrory.me/2010/12/07/data-gravity-in-the-clouds/
Storage
Big Data
From one instance…
Compute
Storage
Big Data
…to thousands
Compute
Storage
Big Data
and back again…
Compute
have data
have data
can store
have data
can store
can analyse
economically
fast
Who is your customer really?
What do people really like?
What is happening socially with
your products?
How do people really use your
products?
15
Lesson 1: don’t leave your Amazon
account logged in at home
Lesson 1: don’t leave your Amazon
account logged in at home
Lesson 2: Drive proactive business
processes based upon data you have
1 instance for 100 hours
=
100 instances for 1 hour
Small instance = $8
1 instance for 1,000 hours
=
1,000 instances for 1 hour
Small instance = $80
Achieving economies of scale
100%
Time
Achieving economies of scale
100%
Reserved capacity
Time
Achieving economies of scale
100%
On
On-demand
Reserved capacity
Time
Achieving economies of scale
100%
Spot
On
On-demand
Reserved capacity
Time
EMR with spot instances
Scenario #1
Job Flow
Duration:
14 Hours
#1: Cost without Spot
4 instances *14 hrs * $0.50 = $28
EMR with spot instances
Scenario #1
Scenario #2
Job Flow
Job Flow
Duration:
14 Hours
#1: Cost without Spot
4 instances *14 hrs * $0.50 = $28
Duration:
7 Hours
EMR with spot instances
Scenario #1
Scenario #2
Job Flow
Job Flow
Duration:
14 Hours
#1: Cost without Spot
4 instances *14 hrs * $0.50 = $28
Duration:
7 Hours
#2: Cost with Spot
4 instances *7 hrs * $0.50 = $14 +
5 instances * 7 hrs * $0.25 = $8.75
Total = $22.75
EMR with spot instances
Scenario #1
Scenario #2
Job Flow
Job Flow
Duration:
Time Savings: 50%
Cost Savings: ~22%
14 Hours
#1: Cost without Spot
4 instances *14 hrs * $0.50 = $28
Duration:
7 Hours
#2: Cost with Spot
4 instances *7 hrs * $0.50 = $14 +
5 instances * 7 hrs * $0.25 = $8.75
Total = $22.75
Easily and rapidly analyze
petabytes of data
Introducing Amazon
Redshift
Data Warehousing the AWS Way
1/10 the cost of traditional data
warehouses
Automated deployment &
administration
Compatible with popular BI tools
Internal Testing:
At Least 10X Faster for a Fraction of the Cost
Our Test
On-premises retail
data warehouse
32 nodes, 4.2 TB of
RAM, 1.6 PB of disk
Several million dollars
2 billion row data set
& 6 most complex
queries
Amazon
Redshift
Two 16 TB /
128 GB RAM nodes
$3.65 / hour
Get insight faster for less
Utility
Accessible
Pay-as-you-go
Big Data Cloud Economics
Unlimited
Low price
Performant