Amazon Web Services
Justin DeBrabant
CIS 570 - Advanced Systems - Fall 2013
Cloud Computing
• A “pay what you use” service for
hardware/software resources
– analogy: utilities
• Amazon Web Services (AWS) is a cloud
computing provider
– there are many others
CIS 570 - Advanced Systems - Fall 2013
Benefits of Cloud Computing
• elasticity
– scale out or scale up easily*
• *can be difficult depending on your application
– scale down if demand decreases
• no upfront capital
– IT costs can be high, prohibitively so for a startup
CIS 570 - Advanced Systems - Fall 2013
*Slide Courtesy Jeff Barr, Amazon
CIS 570 - Advanced Systems - Fall 2013
Available Services
• S3 (Simple Storage Service)
– store and retrieve data
• EC2 (Elastic Computer Cloud)
– scalable compute capacity
• EBS (Elastic Block Store)
– predictable storage volumes that can be attached
to a running EC2 instance
– highly available, highly reliable
CIS 570 - Advanced Systems - Fall 2013
Available Services (cont’d)
• Databases
– Relational: RDS (Relational Database Service)
– NoSQL: DynamoDB
– Scale-Out Caching: ElastiCache
• EMR (Elastic MapReduce)
– Hadoop-based
– uses S3 and EC2
CIS 570 - Advanced Systems - Fall 2013
Other Important Terms
•
•
•
•
vCPU (virtual CPU) – 1 hardware thread
ECU – Amazon’s compute unit
instance – EC2 virtual machine
instance storage – amount of persistent
storage for an instance
• AMI (Amazon Machine Image) – bundled
software to be deployed on an instance
CIS 570 - Advanced Systems - Fall 2013
Various Storages
• S3
– available to instances within same region
– analogy: DropBox
• Instance Storage
– available during the life of a specific instance, not shared across
instances
– analogy: local disk
• EBS
– persistent block storage that can be added to local instances,
but is preserved across instance lifetimes
• can be attached to only one running instance
– analogy: removable storage media (e.g. external hard drive)
CIS 570 - Advanced Systems - Fall 2013
AWS Regions
• Amazon maintains different clusters worldwide
– helps provide locality as well as improved geographic redundancy
• each AWS service is available in specific regions
– e.g. S3
•
•
•
•
•
•
•
•
us-east-1 (Northern Virginia)
us-west-1 (Northern California)
us-west-2 (Oregon)
eu-west-1 (Ireland)
ap-southeast-1 (Singapore)
ap-southeast-2 (Sydney)
ap-northeast-1 (Tokyo)
sa-east-1 (Sao Paolo)
• http://docs.aws.amazon.com/general/latest/gr/rande.html
CIS 570 - Advanced Systems - Fall 2013
S3
• data organized into buckets
– buckets names must be unique across S3
– data in bucket can be organized any way you wish
• direct mapping between buckets and AWS
subdomains
– e.g. bucket name: mybucket, file: myfile.txt
• http://mybucket.aws.amazon.com/myfile.txt
• must specify region of bucket
CIS 570 - Advanced Systems - Fall 2013
S3 (cont’d)
• Guarantees:
– 99.999999999% durability
– 99.99% availability
– designed to be recoverable against two
simultaneous data losses
• SLAs
– if above guarantees are not met, partial refunds
are given
CIS 570 - Advanced Systems - Fall 2013
S3 Pricing
• storage used
– pay per GB
• tiered, firstTB is most expensive
– can reduce cost by reducing redundancy
– http://aws.amazon.com/s3/pricing/
• data transfer
– all data transfer in is free
– data transfer out is not
• pay per GB
CIS 570 - Advanced Systems - Fall 2013
EC2
• machines classified as instances
• each instance is a virtual machine
• every instance type has different resources
– vCPU
– ECU
– network bandwidth
– memory
– instance storage
CIS 570 - Advanced Systems - Fall 2013
Instance Families
• General Purpose
– balanced, all-purpose machines with up to 8 vCPUs
and 30 GB of memory
• Memory Optimized
– up to 244 GB of memory
• Compute Optimized
– up to 32 vCPUs and 10GBit network perf.
• Storage Optimized
– SSD and big disk backed instance storage
CIS 570 - Advanced Systems - Fall 2013
Instance Software
• Instances are virtual machines, and can
support a wide range of operating systems
– Linux (free)
– Windows (have to pay for additional license)
• Can create or use existing AMIs
– prepackaged software bundle
• includes OS and key software
– e.g Oracle, MySQL, Apache WebServer
– for this class, we’ll use existing AMIs
CIS 570 - Advanced Systems - Fall 2013
Buying Instances
• On-Demand Instances
– request an instance, use it, terminate it
• Reserved Instances
– akin to buying a machine
– permanently own instance
• Spot Instances
– bid for unused EC2 capacity
– price can vary greatly
– can set maximum bid and wait for available machines
at or below that price
CIS 570 - Advanced Systems - Fall 2013
On-Demand Instance Pricing
• billed per instance hour
– rounded up at per instance granularity!
– e.g. 2 instance for 20 mins = 2 instance hours
• free tier
– 750 hours with micro instance
– 30 GB EBS storage
– 1 GB of data transfer
• http://aws.amazon.com/ec2/pricing/
CIS 570 - Advanced Systems - Fall 2013
EMR
• easy interface to launch Hadoop jobs using
EC2
– just upload runnable jar containing Hadoop job
• pricing
– standard instance pricing + EMR instance pricing
– e.g. 8 node cluster of m1.small for 1 hour
• (8 * .06) + (8 * .015)
CIS 570 - Advanced Systems - Fall 2013
Accessing AWS Resources
• Console
– easy interactive GUI
– can launch EC2 instances, create S3 buckets, and
start EMR jobs (among other things)
– demo!
• CLI (command line interface)
– must install tools
• http://docs.aws.amazon.com/AWSEC2/latest/UserGuid
e/SettingUp_CommandLine.html
CIS 570 - Advanced Systems - Fall 2013
Accessing a Running Instance
• use SSH
– must have private key file for key specified at
instance launch
– ssh –i <private key> ec2-user@public-dns
• once SSH’d in, can use like any other machine
• use SCP to copy in and out files
– Warning: will be stored on ephemeral instance
storage!
CIS 570 - Advanced Systems - Fall 2013
Questions?
CIS 570 - Advanced Systems - Fall 2013