WHITE PAPER
Outgrowing
Amazon
This paper explores
the AWS model of
shared public cloud
infrastructure and
how organizations can
determine if and when
it’s time to consider
alternative solutions.
www.tierpoint.com
Outgrowing Amazon
WHITE PAPER
Table of Contents
Executive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
What is the Amazon/AWS Model?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Application Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Performance Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
It’s all about Amazon’s Infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
The “Noisy Neighbor” Phenomenon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Transparency Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Cost Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
What Does All This Mean?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Is It Time to Take Off the Training Wheels? . . . . . . . . . . . . . . . . . . . . . . . . . . 12
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
www.tierpoint.com
2
Outgrowing Amazon
WHITE PAPER
Executive Summary
Amazon Web Services (AWS) offers businesses of all sizes – from one-person
startups to giant enterprises like Netflix – the resources and tools they need
to develop and work with “cloud computing services.” Amazon can boast a
number of very high profile, successful customers who have examined Amazon’s
offerings and found them to be a good fit. In fact, it’s an excellent resource for
companies as long as the fit stays right.
But recent years have also seen other major customers – like Instagram, Target,
Dropbox, General Motors, Zynga and Moz – leave the AWS ecosystem wholly
or partially in favor of dedicated solutions.i ii
For many such companies, the built-in limitations of AWS makes Amazon like a
cloud provider with training wheels, which, at a certain point, starts inhibiting or
even preventing growth. But how do you know if and when it’s time to take off
the training wheels, and what do you do then?
Amazon’s vast but largely closed ecosystem of products and services may
or may not be able to support specific applications and use-case scenarios;
likewise, AWS specific architecture can impact whether it can meet unique
performance requirements. Why? Because Amazon has designed AWS
effectively as cloud-as-commodity. As a result, it offers few “concierge-type”
services, which can leave some customers stumbling when dealing with
performance and support issues.
Additionally, commoditizing the cloud hasn’t necessarily rendered it less costly.
Many AWS customers assume Amazon’s economics of scale guarantee the
lowest price among cloud providers; but at scale, Amazon actually becomes
disproportionately expensive.
This paper provides a framework for considering all of these issues, along with
specific recommendations on evaluating your own needs.
www.tierpoint.com
3
Outgrowing Amazon
WHITE PAPER
What is the Amazon/AWS model?
AWS is a robust ecosystem of cloud storage and compute products, services
and solutions. At its heart is a giant, sophisticated public cloud whose
customers utilize shared resources. That kind of co-tenancy means that most
AWS customers do not get their own dedicated hardware, network options
or storage. Instead, Amazon’s public cloud infrastructure is shared among all
of its clients. However, AWS is not limited to public clouds; its vast ecosystem
encompasses a multitude of options, like its Virtual Private Cloud (VPC)
product for deploying dedicated clouds.
AWS and its infrastructure are both enormous and growing rapidly. Market
research firm Gartner found that AWS had “five times the computing
capacity of its 14 nearest competitors combined.”iii Meanwhile, they’re adding
functionality and products to its ecosystem at a breathtaking pace. According
to AWS Senior Vice President Andy Jassy, after launching the Amazon RedShift
product in February 2013, the company added 56 new features in its first 13
months. And RedShift itself is but one among some 1,400 AWS products.iv
Initially, this economies-of-scale approach benefits AWS customers as it allows
companies to quickly ramp up their back-end infrastructure at reduced costs.
The following formula works very well for Amazon’s many clients:
• Responsiveness to new feature requests have kept AWS highly functional
• That functionality is adaptive and highly scalable, almost instantaneously
• Scale and automation, in turn, generate tremendous efficiencyv
The bad news? AWS is not a long-term solution for everyone forever,
particularly for small and mid-size enterprises who:
• Have learned which of their applications should be in a public cloud versus
a private or hybrid cloud environment (think test and development vs.
production; or any I/O intensive app)
• Require network optimization to meet their application’s needs, like latencysensitive apps
• Have gained experience with the public cloud and now have
the data to more accurately calculate and analyze the OpEx vs. CapEx
question
How did those companies realize that they’d outgrown Amazon, and what
were their options then? To answer these questions, we’ll look at four major
considerations: applications, performance, transparency
and cost.
www.tierpoint.com
4
Outgrowing Amazon
WHITE PAPER
Application Considerations
Amazon’s massive ecosystem of services (such as RedShift, messaging queuing,
analytics, etc.) enable engineers to build their infrastructure on top of Amazon.
Further, Amazon has rolled out several APIs for VMWare VCenter that allows
customers to integrate their existing platforms for faster deployment and
easier management. For many customers, that means AWS offers a rich and
robust ecosystem through which they can deploy efficient, powerful clouds.
However, some customers will run into a big problem: “The major downside
to Amazon’s ecosystem,” says TierPoint sales engineer Joe Conlin, “is that
once an organization architects around Amazon’s services, they are locked in.
AWS services are so specific, that even though they work well, it’s a massive
undertaking to re-engineer for a private, dedicated solution.”
Customers must consider their unique cloud use-case scenarios, which can
vary dramatically. Consider the compliances, standards and best practices that
can apply to an organization in any given field. AWS offers products compliant
with many federally mandated privacy laws, like HIPAA; but the complex (and
ever-changing) stew of federal, state and municipal regulations combined with
international standards and industry best practices can turn compliance with
clouds into a legal and logistical minefield. Ultimately, regulations and guidelines
sometimes end up outright dictating what kinds of clouds are permissible. For
instance, federal security requirements mean that private cloud spending by the
U.S. federal government is expected to outpace public clouds by $1.7 billion to
just $118.3 million in 2014.vi
Other use-case scenarios can also affect whether a given cloud provider is a
good fit. For example, AWS can provide an excellent arena for early testing. Not
having to worry about issues like latency means that a company can test new
applications or operating environments without having to invest resources to
build, expand or customize a private cloud.
But what happens if, after testing, it makes more sense to migrate the app to
customized private or hybrid clouds optimized for performance? Similarly,
resource-intensive applications (like Apache Hadoop for large-scale data
processing) are typically better suited to a private cloud, where costs can be
better contained while meeting performance requirements.
Indeed, performance is often a prime determinant of what kind of cloud setup
will best fit a company’s needs.
www.tierpoint.com
5
Outgrowing Amazon
WHITE PAPER
Performance Considerations
In its early days, prominent inbound marketing platform HubSpot found
Amazon’s cloud “fast, beautiful, unbridled [which] let us do things we could
never do [otherwise],” CIO Jim O’Neill told IT World.vii But as the company
grew to serving over 11,000 customers in 70 countries, HubSpot began to
struggle with aspects of the AWS ecosystem, which eventually bled over into
service quality. For example, at the large scale they were utilizing, the number
of “zombie servers,” or servers for which the HubSpot paid but didn’t use,
became a “significant problem.”
“We find companies learn from Amazon what their
infrastructure really needs to be, to grow and be scalable
within their environment.As they outgrow Amazon’s model,
dedicated cloud environments become a superior fit.”
– John Holland, TierPoint Senior Vice President, Sales.
It’s all about Amazon’s infrastructure.
Sharing resources is an efficient and cost-effective way to leverage the cloud
for companies whose needs mesh well with AWS architecture; but at scale – as
HubSpot discovered – it can cause technical headaches. For example, EC2 –
Amazon’s Elastic Compute Cloud, which provides scalable computing capacity in
the AWS cloud – works very well for distributed work but may work poorly or
be prohibitively expensive for other use-case scenarios (e.g., heavy I/O). Another
company saw a 50% drop in their application’s average response time after
moving to a private cloud, where they were able to use a smaller cluster. Once on
a private cloud, latency times decreased and performance fluctuations evened out.
AWS provides tools to help its customers fine-tune performance, but AWS
turns clouds into commodities. Customers with unique service needs may run
into major pitfalls, particularly around how AWS interacts with customers’ own
private networks. For example, AWS offers a special “direct connect” service
where they charge a usage-based backhaul if the customer can utilize their
limited selection of telco hotels. By contrast, says Conlin:
“A carrier neutral Cloud Hosting Provider like TierPoint provides
customers with the flexibility to plug their cloud solution into their
existing network, rather than trying to build a network around their
cloud. For enterprises with 10G fiber rings running Active/Active data
centers, or clients with global MPLS networks, this type of flexibility
provides a faster ramp to cloud adoption and higher rates of success.”
Once again, it’s all about fitting specific customer needs.
www.tierpoint.com
6
Outgrowing Amazon
WHITE PAPER
Some performance problems are inherent,
like the “Noisy Neighbor” phenomenon.
One issue that can arise with a public cloud – one based on shared resources
– is the “Noisy Neighbor” problem. In a public cloud, a single server can host
multiple virtual machines (VMs). Certain resources, like CPU, can be allocated
between VMs efficiently and effectively; but with other resources, it’s more
difficult. With the “Noisy Neighbor” issue, a cloud customer’s neighbors on
that server are utilizing a disproportionately large amount of disk I/O, which
degrades performance for all of the other VMs hosted on that server. And it just
gets worse at scale.
A large organization has the option to build and deploy its own infrastructure,
so that their own cloud performance does not rely on any single server or VM.
In fact, they can usually automate the fault detection and correction processes
so that admins don’t even have to do anything when performance is affected.
Small and medium businesses (SMBs) rarely have the resources needed to
implement such a solution; and when performance starts falling due to a “Noisy
Neighbor,” they must manually configure a solution. In extreme cases, it can
force a migration of all data from that one VM to a new one.
As Netflix says,
“Co-tenancy can introduce variance in throughput at any level of the
stack.You’ve got to either be willing to abandon any specific subtask,
or manage your resources within AWS to avoid co-tenancy...” viii
Easy for Netflix to say. Hard for most businesses, especially SMBs,
to do.
Of course, performance issues can happen anywhere; and Amazon does offer
a highly adaptive and flexible computing environment to fix such problems…
that is, when you even know what to fix. This is an underlying concern that
complicates performance problems: transparency.
www.tierpoint.com
7
Outgrowing Amazon
WHITE PAPER
Transparency Considerations
AWS provides an enormous amount of supporting data and documentation to
its customers: it posts service issues publicly; posts rafts of technical, security
and reference architecture documentation; and even their pricing is right out
on their site for anyone to read.
AWS also provides tools for customers to monitor their environments and
suggest sizing increases/decreases so they can right-size both their workloads
and their bill to match what they are actually using at any given point in time.
That’s a lot of great information, and for many customers, it’s plenty.
But remember that AWS is built, deployed and operated on a massive scale; it is
cloud-as-a-commodity, which means performance problems that fall to a single
instance or machine, like Noisy Neighbors, can fly under Amazon’s radar and go
unidentified, reported or remediated until well after performance has degraded.
It’s often up to the customer to identify the issue and find a stop-gap resolution,
if not a final solution, themselves. Not all companies have the resources of a
Netflix to deal with these troubles.
Of course, AWS provides an enormous amount of support … to the customers
who pay for it. As Paul Mazzucco, TierPoint’s Chief Security Officer, explains:
“With the Amazon model, you literally just buy an
unmanned and unmanaged slice of their infrastructure
for whatever you want. If you want security wrapped in,
it’s a whole new layer. If you want them to do any sort
of patching or upgrading, again, it’s a whole other layer of
contract. And so on.”
That brings us to our final consideration: cost.
www.tierpoint.com
8
Outgrowing Amazon
WHITE PAPER
Cost Considerations
We mentioned earlier that Seattle-based search engine marketing giant Moz
(formerly SEOMoz) has moved away from AWS. Here’s why:
“Moz’s goal for the end of Q1 2014 is to be paying $173,000/month
for their own environment plus $100,000/month for elastic AWS
cloud usage. If they remained entirely on AWS, it would work out at
$842,000/month.” ix
Emphasis ours. That’s a difference of over half a million dollars, or a cost savings
of 67.58%. Moz is not alone. HubSpot is another example. By moving off AWS
onto a private cloud build, HubSpot slashed their cloud costs by two-thirds,
according to Wired Magazine.
No wonder executives from Cisco and Redapt told IT World, “Cost
is a reason that some businesses are leaving public clouds and
going private.”
Josh McKenty, former cloud architect for NASA, is blunt in his assessment:
“AWS in this sense is like a crack dealer.The first hit is free, [but as
you scale up, particularly in storage], Amazon will cost you $80,000 a
year for something you can buy for $3,000.” xii
Once again, it’s all about fit, and the same is true of pricing as of performance.
Even low raw prices – even those after the AWS price cut in early 2014, in
response to Google and Microsoft making aggressive overtures in the market
– can skyrocket at scale.xiii We observed earlier that Amazon’s contractual
arrangements can add up quickly; so, too can the costs. With Amazon, you don’t
just pay for EC2 instances; you pay for everything extra, separately, from storage
to I/O costs. Every granular cost piles onto the monthly invoice. Dell cloud
executive Nnamdi Orakwue says companies often start looking at alternatives
when their monthly AWS bill hits $50,000.xiv
www.tierpoint.com
9
Outgrowing Amazon
WHITE PAPER
What does all this mean?
AWS offers a powerful and sophisticated solution to its customers. They’re
pioneering bleeding edge technologies and policies around the cloud, and
they’re accomplishing great things as a result. With last year’s major price cuts,
they’re more affordable than ever. However, Amazon has trained its customers
to think they are the cheapest provider of a magical solution, so that when an
organization finds itself paying $30,000/month at Amazon, they just assume the
cost would be higher and performance no better elsewhere.
That is a faulty assumption, says TierPoint’s Holland:
“Once you reach a certain level with Amazon, it
becomes exponentially more expensive and frustrating
to stick with that model and keep that much
infrastructure at Amazon than it is to move into a
dedicated cloud environment.”
Once the infrastructure has gotten past the rapid-prototype or proof-ofconcept phase, and organizations hit the “always on” critical mass, they will start
to realize the benefits of dedicated private cloud infrastructure immediately
because it’s not as expensive as they may think it is.
Consider personal cloud storage powerhouse Dropbox. It had been a dedicated
AWS customer from the beginning, up until March 2014. According to GigaOm,
the company has since begun moving IT infrastructure away from AWS.xv Ryan
Floyd, managing director of investment firm Storm Ventures, is not surprised:
“The economics of scale doesn’t support it.” xvi
What once was a great fit, may eventually no
longer fit at all.
At that point, dedicated cloud environments become the choice solution to
rein in costs, enhance performance, and fine-tune control. Problems inherent
to public, shared resources (like noisy neighbors) evaporate. Customers can
get exactly the cloud build, setup and services they need – all hosted from
powerful, purpose-built data centers that can match or even beat Amazon’s
own, multiplying power at a fraction of the cost.
www.tierpoint.com
10
Outgrowing Amazon
WHITE PAPER
Is It Time to
Take Off the
Training Wheels?
Four Recommendations:
1
If performance metrics aren’t hitting your goals, it’s time to take Amazon’s
measure. When performance suffers, a private or hybrid cloud from a managed hosting provider, with
2
Don’t let transparency gaps translate into problems. As a “mass market” cloud,
3
In fact, consider meeting your cloud team. Select a cloud solution provider that allows
4
www.tierpoint.com
dedicated boxes and custom configurations, can start paying dividends in performance at competitive
prices. As you watch your AWS performance, remember not just to track raw analytics but also to
measure performance against cost as usage scales, so you don’t pay comparable monthly fees for degraded
performance. A dedicated cloud can give you the fine-tuned control you need to power up performance.
Amazon is offering a commodity; it’s not delivering concierge-type services to enhance your experience.
Private, dedicated cloud environments, particularly combined with managed services, deliver in-depth
insight to power smart business decisions.
and encourages you and your auditors to visit the data center location and shake hands with the facility
operators and GMs. This interaction builds a bond that the provider is an extension of your team. Even
with AWS’s compliant and dedicated solutions, AWS customers can’t get that unless they’re the U.S.
government.
If you’re paying $30,000+/month, start considering alternatives, period. Once
a company hits that level of usage – especially if workload begins reaching a predictable, fixed point
– it starts making much more sense to switch from an OpEx to a CapEx cost model. The cost of the
hardware or a managed services agreement will be cheaper than continuing to pay AWS.
11
Outgrowing Amazon
WHITE PAPER
References
i.
King, R. (2013, May 20). Why Target ditched Amazon. The Wall Street Journal
http://blogs.wsj.com/cio/2013/05/20/why-target-ditched-amazon/
ii.
Romano, B. (2014, January 31). Moz Dumps Amazon Web Services, Citing Expense and
‘Lacking’ Service. Xconomy.
http://www.xconomy.com/seattle/2014/01/30/moz-dumps-amazon-web-services-citingexpense-and-lacking-service/
iii.
Hesseldahl, A. (2014, January 31). Is IBM’s $4.4 Billion Cloud Bigger Than Amazon’s? Not
Quite. Recode.net.
http://recode.net/2014/01/31/is-ibms-4-4-billion-cloud-bigger-than-amazons-not-quite/
iv.
Asay, M. (2014, August 15). AWS innovation is what’s driving adoption, not price. TechRepublic.
http://www.techrepublic.com/article/aws-innovation-is-whats-driving-adoption-not-price/
v.
Verge, J. (2014, April 24). How Amazon Stays On Top in the Cloud Wars. Data Center
Knowledge.
vi.
Humphreys, J. (2013, November 22). How to determine if your application is suitable for the
cloud. Network World.
http://www.networkworld.com/article/2172168/tech-primersow-to-determine-if-yourapplication-is/tech-primers/how-to-determine-if-your-application-is-suitable-for-the-cloud.
html
vii.
Gohring, N. (2013, April 27). HubSpot latest to leave Amazon Web Services -- mostly. Should
Amazon worry? IT World.
www.itworld.com/cloud-computing/352750/more-talk-customers-leaving-aws-shift-pointsexperience-not-trouble-amazon
viii.
Ciancutti, J. (2010, December 16). 5 Lessons We’ve Learned Using AWS. Netflix Blog.
http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html.
ix.
Mytton, D. (2013, December 7). Saving $500k per month buying your own hardware.
ServerDensity.com.
https://blog.serverdensity.com/saving-500k-per-month-buying-your-own-hardware-cloud-vscolocation/
x.
McKenty, J. (2013, September 10). Three signs it’s tie to get off Amazon’s cloud. Wired.
http://insights.wired.com/profiles/blogs/three-signs-it-s-time-to-move-off-aws
xi.
Gohring, 2013 April 27.
xii.
Gohring, 2013 April 9.
xiii. Lardinois, F. (2014, March 26). In Response To Google, Amazon Announces Massive Price Cuts
For S3, EC2, ElastiCache, Elastic MapReduce And RDS. TechCrunch.
http://techcrunch.com/2014/03/26/in-response-to-google-amazon-announces-massive-pricecuts-for-s3-ec2-and-rds/
xiv.
Darrow, B. (2013, October 10). Amazon Web Services: should you stay or should you go?
GigaOm.
http://gigaom.com/2013/10/10/amazon-web-services-should-you-stay-or-should-you-go/
xv.
Darrow, B. (2014, July 25). AWS in fight of its life as customers like Dropbox ponder hybrid
clouds and Google pricing. GigaOm.
http://gigaom.com/2014/07/25/aws-in-fight-of-its-life-as-customers-like-dropbox-ponderhybrid-clouds-and-google-pricing/
xvi. Darrow, 2014.
www.tierpoint.com
12
Outgrowing Amazon
About TierPoint
TierPoint is a leading national provider of cloud, colocation and managed
services designed to help organizations improve business performance and
manage risk. With corporate headquarters in St. Louis, Mo., TierPoint operates
13 highly-redundant, Tier III plus data centers in the states of Washington, Texas,
Oklahoma, Pennsylvania, Maryland, New York, Massachusetts and Connecticut.
To find out how TierPoint can help you with your cloud, colocation
and managed services initiatives — call 877.859.TIER (8437), email
sales@tierpoint.com, or visit us at www.tierpoint.com.
TierPoint
520 Maryville Centre Dr.
St. Louis, MO 63141
www.tierpoint.com
© 2015TierPoint, LLC. All Rights Reserved.
WHITE PAPER