Version Date Description of revision 0.1 17 March 2014 Early draft for internal review 0.2 19 March 2014 Initial draft for review by the client 0.3 31 March 2014 Revised draft including further suggestions from Lee Gillam 0.4 7 April 2014 Revised draft including comments from EPSRC and Jisc 0.5 14 April 2014 Final draft for approval by EPRSC and Jisc 1.0 30 June 2014 Issue version, including additional non-cloud comparison Introduction 1 A previous report, published in early 2012, provided advice and guidance on the financial implications of using ‘the cloud’ for research computing,1 including an analysis of the prices offered by a range of then current cloud providers. That report noted that cloud computing service offerings and prices would most likely be subject to rapid change. This report provides an update of cloud computing costs, based on research finalised in early – mid March 2014, just ahead of the ‘price war’ that started in late March 2014 (see paragraph 6). It should be read in conjunction with the previous report. Price 2 There seems to be a dynamic market for cloud services suitable for research. Further suppliers have entered the market (eg Google Compute Engine and HP Cloud) and new instance types are available. Both have been included to allow comparison with the other cloud providers and existing instance types. 3 With some exceptions, there has been a reduction in the price per core-hour across the various compute categories2 considered since autumn 2011. For those cloud providers that have reduced prices, this is equivalent to an average reduction of ~2p – ~5p per core-hour, depending on the compute category considered. There is also an additional benefit because the instances used in March 2014 typically represent an unquantified improvement in compute capability. 4 The price for storage (£/month/GB) has either stayed the same or decreased for most cloud providers considered. The average decrease in price is ~1.4p/month/GB. 5 Fewer cloud providers now charge for inbound data transfers (7 out of 12 in autumn 2011 and 3 out of 12 in March 2014). The average reduction is ~3.3p/GB. Most cloud providers continue to charge for outbound data transfers and there has only been a very modest average reduction in price (~0.2p/GB) with some cloud providers increasing the price. 6 Since the research for this report was carried out in early – mid March 2014, Google-Compute Engine has announced a 32% reduction in prices for its cloud services across all regions, sizes and classes from 1 April 2014, together with a simplification of the charging structure. Amazon EC2 and Azure have responded with significant price reductions. At the time of finalising this report, it is too early to say what the implications are for the cloud market as a whole, but it is entirely possible that other cloud providers will also respond. Comparison with institutional research computing 7 The reduction since autumn 2011 for on-demand cloud compute costs changes the comparison of cloud computing with that of institutional research computing.3 The majority of the cloud computing instances, rented on an on demand hourly basis, are less expensive or the same price per core-hour as well managed, locally provided clusters in modern data centres operating at high utilisation levels.4 1 Final report to EPSRC and JISC: cost analysis of cloud computing for research, CC497D002-1.2, 22 February 2012, 2 3 4 http://www.epsrc.ac.uk/SiteCollectionDocuments/Publications/reports/CC497D002-1_2FinalReportToEPSRCandJISC.pdf. ‘Compute category’ refers to the various types of computing for research considered (see paragraph 1.4.6). Representative instances have been selected for each cloud provider and each compute category. It is assumed that the estimated cost of institutional research computing of 5p – 7p per core-hour found by the previous study remains true in March 2014. The previous report stated “between one-and-a-half to two times…” CC551D001-1.0 1 8 It is perhaps more accurate to say that the price ranges of public cloud providers and institutions overlap and have a similar upper bound. Under some circumstances, cloud will offer a ‘better’ solution and that for others a traditional data centre will remain more appropriate. The final decision will depend on the precise performance of the selected approach. The trend of price reductions such as the recent ones by Amazon, Google Compute-Engine and Azure reinforces these findings. 9 Importantly, this comparison takes no account of the performance implications of the different infrastructures for specific research tasks. The degree of overlap will be different for each institution. 10 Spot and reserved options provide an opportunity for significant price reductions (20% - 80%) over on-demand prices. This does need to be considered in the context of disadvantages such as increased commitment, increases in elapsed processing time, overnight working, etc. 11 Given the trend of continual price discounting and improved offerings, alongside alternate pricing approaches such as reserved instances and spot markets, it is now clear that use of cloud computing can often be more economic than using institutional resources and should be considered alongside institutional computing when planning research. Orchestration services such as those provided by CycleCloud may also prove useful in achieving further price reductions. Getting best value for money 12 There are many cloud service providers offering a bewildering array of options and prices from which to choose. Choices will also depend on how well a particular application or code runs on a particular architecture or technology. Moreover, there have been considerable changes since autumn 2011 in the cloud offering (eg more and better instance types), in the pricing models, and even in the cloud providers (eg SoftLayer is now an IBM Company and IBM SmartCloud is being closed). This makes it likely that a researcher will have neither the time nor the inclination to properly analyse and compare the different options – and, of course, price may not necessarily be the most important factor in deciding which provider to use. 13 Making the optimum choice applies at both the individual research project and at the research level. On the one hand, a piecemeal approach is unlikely to benefit from the economy of scale needed to get the best value for money across research. On the other hand, the introduction of the ‘dead hand of bureaucracy’ may slow the take-up of cloud computing for research and may affect the potential quality of research and concomitant outputs. 2 CC551D001-1.0 Executive summary 1 List of abbreviations and glossary 4 1 1.1 1.2 1.3 1.4 1.5 Introduction General Background Objectives and scope Approach Overview of this report 5 5 5 5 6 9 2 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Cloud prices Introduction Prices for compute categories Storage Data transfer Reserved and spot instances Comparison with cost of institutional research computing How prices have changed Google Compute-Engine, Amazon and Azure prices from 1 April 2014 10 10 10 11 13 14 16 17 20 A A.1 A.2 A.3 A.4 Cost model for an owned server Introduction Cost model Cost per core-hour Assessment 23 23 23 24 24 B B.1 B.2 B.3 B.4 B.5 B.6 Cloud prices at March 2014 Introduction Compute Storage Data transfer Reserved and spot instances Pricing models 25 25 25 29 31 31 32 C C.1 C.2 C.3 C.4 Cloud prices at autumn 2011 Introduction Compute Storage Data transfer 35 35 35 37 38 CC551D001-1.0 3 EPSRC Engineering and Physical Sciences Research Council GB Gigabyte (109 bytes) GiB Gibibyte (230 bytes) GPU Graphics Processing Unit GRS Geographically Redundant Storage (Windows Azure) HDD Hard Disk Drive HEI Higher Education Institution HPC High Performance Computing IaaS Infrastructure as a Service IO Input Output IOPS IO Operations per Second LRS Locally Redundant Storage (Windows Azure) RA-GRS Read-Access Geographically Redundant Storage (Windows Azure) RRS Reduced Redundancy Storage (Amazon S3) SAN Storage Attached Network SAS Serial Attached SCSI (drive interface standard) SATA Serial AT Attachment (drive interface standard) SSD Solid State Disk 4 CC551D001-1.0 1 1.1 General 1.1.1 Curtis+Cartwright Consulting Ltd, with support from John O’Loughlin and Dr Lee Gillam of the University of Surrey, undertook this ‘Refresh of the cost analysis of cloud computing for research’ on behalf of the Engineering and Physical Sciences Research Council (EPSRC) and Jisc. This document is the final report. 1.2 Background 1.2.1 A previous study that reported in early 2012 developed advice and guidance on the financial implications of using ‘the cloud’ for research computing.5 This study included an analysis of the prices offered by a range of then current cloud providers. The research for that analysis was carried out in autumn 2011. That report noted that cloud computing service offerings and prices would most likely be subject to rapid change. This report provides an update of the analysis, based on research finalised in March 2014. It needs to be read in conjunction with the previous report. 1.3 Objectives and scope 1.3.1 The primary audience is the EPSRC and Jisc. However, the discussion and findings in this report may be of interest to anyone in the HE community with an interest in cloud computing, and especially: – Researchers interested in using cloud computing for their research. – Higher Education Institutions (HEIs) and research computing managers, who may be considering providing access for their researchers to cloud computing services. – Funding bodies in addition to EPSRC, who fund research computing through grant awards, and the reviewers of grant applications. 1.3.2 The report looks at Infrastructure as a Service (IaaS) through the providers of public cloud. Most such public clouds provide access to virtual machines. Some cloud providers provide “bare metal” options where the users have full access down to the “bare metal”. 1.3.3 Based on discussions with research computing managers, and the information they shared in confidence, the previous report estimated that the costs of institutional research computing were around 5p to 7p per core-hour.6 While investigating these figures is out of scope for this study, it is assumed that they will not be significantly different. A worked example of the costs for an institutional server broadly equivalent to an Amazon EC2 instance that supports this assumption is provided at Annex A. It is also vital to recognise that costs at institutions do vary significantly. 5 Final report to EPSRC and JISC: cost analysis of cloud computing for research, CC497D002-1.2, 22 February 2012, 6 http://www.epsrc.ac.uk/SiteCollectionDocuments/Publications/reports/CC497D002-1_2FinalReportToEPSRCandJISC.pdf. This is estimate was based on conversations and information provided by research computing managers. As supporting material, it should be noted that Appendix A of the HPC-SIG’s 2010 report found that 5 of 11 respondents to their survey reported FEC rates per CPU hour of 0-5p, 4 of 11 had costs of 6-10p and 2 respondents had costs in the 11-15p range. See http://www.hpc-sig.org/Home?action=AttachFile&do=get&target=HPC-SIG_Report.pdf [accessed 21 March 2014]. CC551D001-1.0 5 1.4 Approach 1.4.1 The basic approach for this work was to investigate the current prices of the cloud providers that were considered by the original study. This was both to look at trends in prices between the two times and to compare the prices across a range of cloud providers. 1.4.2 Ideally, we would have used the same (or equivalent) instances to allow comparison between the late 2011 and the March 2014 findings. However, this has rarely been possible since most cloud providers have changed their offering considerably, introducing new instances with newer processor technology and subsequently withdrawing older offerings. A similar issue arises for comparing prices between cloud providers. Here, differences in the underlying processor technology, configuration and choice of RAM and storage mean that a meaningful comparison can only be achieved through detailed benchmarking. 1.4.3 Moreover, further suppliers have entered the market (eg Google Compute Engine and HP Cloud) and new instance types are available. Both have been included to allow comparison with the other cloud providers and existing instance types. 1.4.4 The team has used its skills and understanding of the use of the cloud computing for research to pick compute instances that have roughly the same or better overall performance for the various compute categories and cloud providers. These can then be used both for tracking prices over time and for comparison of prices between cloud providers. 1.4.5 Some cloud providers have various reduced price or free programmes for cloud use.7 While institutions should be encouraged to take advantage of offers of free or reduced price cloud computing, these are not included as part of the comparison. Compute categories 1.4.6 The following instance categories are presented for each cloud provider:8 – High Performance Computing (HPC) Low and HPC High: these are designed to be appropriate for CPU-bound scientific problems. Differentiated on CPU where possible, otherwise by RAM. – Low: the lowest price per core-hour available. This excludes some “micro” type instances where the instance does not have an allocated virtual core proportion. – High: Aiming for a specification similar to Amazon’s erstwhile High CPU extra-large instance type (c1.xlarge). – High Memory Large: An instance with the largest amount of RAM. – GPU: instances offering Graphics Processing Unit (GPU) cards as a processing resource. 1.4.7 The first four of these were also discussed in the original report; the last two reflect current trends in the use of the cloud for research computing. In addition, if a cloud provider offers HPCspecific nodes, this is highlighted. 7 These include grants available from Amazon (see http://aws.amazon.com/education/) and the Amazon free usage tier (see http://aws.amazon.com/aws-free-usage-tier0/) [all accessed 14 April 2014]. The specific instances chosen for each cloud provider are listed at Table B-3 for the March 2014 analysis and Table C-2 for the autumn 2011 analysis. 8 6 CC551D001-1.0 Prices 1.4.8 Compute prices are calculated in £/core-hour with the lowest commitment, so bulk discounting is not considered. The focus is on CPU rather than RAM, and storage and data transfer costs are excluded wherever possible. Prices are also given for storage (£/GB/month) and for input and output of data (£/GB). The report also examines the current situation for spot and reserve instances. 1.4.9 Public cloud providers often quote prices in US dollars. For this report, we have chosen to give all prices in pounds sterling using a nominal exchange rate of $1.6 to £1.9 Note on prices 1.4.10 Section 2 contains prices for a range of current cloud providers. However, the purpose of this work is not to offer a robust price comparison and, although we have tried to ensure that prices are accurate, we make no guarantees as to their veracity. It is important to understand that offerings and prices, and even the pricing model have changed frequently and are likely to continue to do so. Moreover, different cloud providers have very different charging and billing approaches. Readers must perform their own price assessment of cloud services tailored to the specifics of their research tasks and circumstances. However, we hope that the information provided and the issues highlighted in this report make their task easier. 1.4.11 The prices were last checked fully on 12 March 2014. Since then, both Google Compute Engine and Amazon EC2 have announced price reductions that apply from 1 April 2014.10 In addition, Azure has announced similar price reductions that will apply from 1 May 2014.11 All prices are based on the use of Linux. Prices exclude VAT. Cloud providers considered 1.4.12 There were 12 cloud providers examined in the earlier report. This time 15 cloud providers are considered.12 Cycle Computing 1.4.13 Cycle Computing is included as it was covered in the autumn 2011 report, but it now operates in a different way - as an orchestration layer to help achieve savings through use of reserved and spot instances. Cycle Computing has been omitted from the price change analysis of subsection 2.5 because of this different approach. This is discussed further at paragraph 2.5.4. 9 This is the same as the nominal exchange rate used in the previous report. See Amazon HALVES cloud storage prices after Google's shock slash, http://www.theregister.co.uk/2014/03/26/amazon_price_cut/ [accessed 28 March 2014]. See Microsoft Azure Matches Amazon’s Price Cuts and Introduces New “Basic” Tier, http://techcrunch.com/2014/03/31/microsoft-azure-matches-amazons-price-cuts-and-introduces-new-basic-tier/ [accessed 2 April 2014]. The additional cloud providers are Google Compute Engine, HP Cloud, Rackspace and SoftLayer. IBM acquired SoftLayer Technologies Ltd in 2013 and is now investing in this and moving from its previous cloud offering (IBM SmartCloud Enterprise) to a Softlayer-based offering (see http://www.bloomberg.com/news/2014-01-17/ibm-to-invest-1-2-billionto-build-on-softlayer-cloud-services.html [accessed 7 March 2014]). In addition, NewServers has changed its name to Baremetalcloud. 10 11 12 CC551D001-1.0 7 13 Provider URLs Amazon EC2 Notes G:http://aws.amazon.com/ec2/ I: http://aws.amazon.com/ec2/instance-types/#instance-details I: http://aws.amazon.com/s3/details/ P: http://aws.amazon.com/ec2/pricing/ P: http://aws.amazon.com/s3/pricing/ G: http://www.windowsazure.com/en-us/solutions/ I: http://msdn.microsoft.com/en-us/library/windowsazure/dn197896.aspx P: http://azure.microsoft.com/en-us/pricing/details/virtual-machines/ P: http://www.windowsazure.com/en-us/pricing/details/storage/ Formerly Windows Azure G: http://www.baremetalcloud.com/index.php/en/hardware/dedicated-servers.htm I: https://noc.baremetalcloud.com/web.html P: https://noc.baremetalcloud.com/web.html Formerly NewServers Cycle Computing http://www.cyclecomputing.com/cyclecloud/overview CycleCloud offers an orchestration layer to help with the process of establishing and operating an HPC Cloud Eduserv G: http://assets-production.govstore.service.gov.uk/G4/Eduserv0244/51f978b012a2fcb9e8000979/QD1/CloudCompute(IL0-2)_ServiceDefinition_Eduserv.pdf I: http://govstore.service.gov.uk/cloudstore/cloud-compute-il0-il2-4-g4-0244-213 P: http://assets-production.govstore.service.gov.uk/G4/Eduserv0244/51f978b012a2fcb9e8000979/QD4/CloudCompute(IL0-2)_Pricing_Eduserv.pdf ElasticHosts G: http://www.elastichosts.co.uk/ I: http://www.elastichosts.com/cloud-servers/virtual-machines/ I: http://www.elastichosts.com/support/faq/ P: http://www.elastichosts.co.uk/cloud-servers-quote/ Flexiscale G: http://www.flexiscale.com/products/flexiscale/ I: http://www.flexiscale.com/products/flexiscale/specifications/ I: http://www.flexiscale.com/products/flexiscale/faq/ P: http://www.flexiscale.com/products/flexiscale/pricing/ GoGrid G: http://www.gogrid.com/products/cloud-servers I: http://www.gogrid.com/products/cloud-servers P:http://www.gogrid.com/pricing Google Compute Engine G: https://developers.google.com/compute/docs/ I: https://developers.google.com/compute/pricing P: https://developers.google.com/compute/pricing New for March 2014 HP Cloud G: http://www.hpcloud.com/products-services/compute I: http://www.hpcloud.com/pricing P: http://www.hpcloud.com/pricing New for March 2014 Joyent G: http://www.joyent.com/products/compute-service I: http://www.joyent.com/products/compute-service/pricing P: http://www.joyent.com/products/compute-service/pricing Penguin G: http://www.penguincomputing.com/services/hpc-cloud/pod I: http://docs.pod.penguincomputing.com/wiki/POD_MT1 P: https://pod.penguincomputing.com/services Warning: the offering has changed frequently throughout the study Rackspace G: http://www.rackspace.co.uk/cloud I: http://www.rackspace.co.uk/cloud/servers/pricing P: http://www.rackspace.co.uk/cloud/servers/pricing Compute prices new for March 2014. Softlayer G: http://www.softlayer.com/cloudlayer/computing/ I: http://www.softlayer.com/cloudlayer/computing/# P: http://www.softlayer.com/cloudlayer/build-your-own-cloud New for March 2014 Previously IBM SmartCloud Terremark vCloud Express G: http://vcloudexpress.terremark.com/default.aspx I: http://vcloudexpress.terremark.com/specifications.aspx P: http://vcloudexpress.terremark.com/pricing.aspx Azure Baremetalcloud Table 1-1: Cloud service providers considered for the cost comparison and related URLs 13 URLs are categorises as General (G), Instance Type (I) or Pricing (P). General covers an introduction/overview of the offering. Instance type provides information on the processors, RAM and disk storage for instances. Pricing covers the details of the price for the selected instances. Sometimes instance descriptions and pricing are combined on the suppliers’ website and sometimes not. All URLs correct as at 12 March 2014. 8 CC551D001-1.0 1.5 Overview of this report 1.5.1 This report is structured as follows: – Section 2 lists the prices for the compute categories listed at paragraph 1.4.6, and storage and data transfer for each of the cloud service providers identified at Table 1-1. It also discusses how prices have changed since the last report. – Annex B and Annex C document the specific instance types selected for the compute categories together with prices as at March 2014 and autumn 2011 for these instance (£/core-hour) together with prices for data storage (£/month/GB) and for data transfers (£/GB). CC551D001-1.0 9 2 2.1 Introduction 2.1.1 This Section lists the prices for the compute categories listed at paragraph 1.4.6, and storage and data transfer for each of the cloud service providers identified at Table 1-1. It also discusses how prices have changed since the last report. 2.2 Prices for compute categories 2.2.1 The prices for the compute categories listed at paragraph 1.4.6 are set out for March 2014 (no shading) and autumn 2011 (with shading) at Table 2-1. Details of the instances used can be found at Table B-3 and Table B-4 for March 2014 and Table C-2 for autumn 2011. The key features of on-demand charging for each cloud provider considered are summarised at Table B-9. Provider HPC low Low 0.047 Amazon EC2 Baremetalcloud (was NewServers) HPC high HPC nodes High memory large Data centre used for 14 price GPU 0.038 0.070 0.047 Y 0.068 0.051 0.047 0.053 0.053 0.10 Y - - 0.038 0.039 0.038 0.038 N 0.103 N 0.075 0.075 0.075 0.075 N - - 0.054 0.047 0.054 0.054 N 0.054 N 0.047 0.047 0.069 0.098 N - - 15 Azure High US East US East Standard price 0.047 0.043 0.092 0.046 N 0.185 N 7.1×10–4 0.012 0.032 0.067 N - - 0.188 0.080 0.012 0.188 N 0.188 N 0.026 0.026 0.04 0.037 N - - 0.045 0.018 0.045 0.027 N 0.027 N 0.033 0.022 0.055 0.055 N - - 0.038 0.038 0.038 0.038 N 0.038 N 0.12 0.11 0.12 0.12 N - - 0.041 0.065 0.065 0.065 N 0.076 N - - - - - - - 0.094 0.019 0.070 0.127 N 0.211 N - - - - - - - 0.045 0.035 0.075 0.075 N 0.127 N 0.053 0.053 0.175 0.10 N - - 0.075 0.075 0.075 0.075 Y - - Standard price/corehour across range 0.193 0.068 0.193 0.19 Y - - Standard price 0.125 0.030 0.030 0.125 N 0.125 N - - - - - - - SoftLayer (IBM SmartCloud) 0.037 0.030 0.049 0.037 N 0.037 N 0.052 0.052 0.084 0.054 N - - Terremark vCloud Express 0.050 0.022 0.050 0.073 N 0.073 N 0.008 0.008 0.1 0.15 N - - Eduserv ElasticHosts Flexiscale GoGrid Google Compute Engine HP Cloud Joyent Penguin Rackspace UK UK UK - priced in Flexiscale units Standard price US Standard price Standard price UK Europe Standard price Table 2-1: March 2014 and autumn 2011 prices (£/core-hour) for various compute categories (no shading is for March 2014 data and with shading is for autumn 2011 data) 14 15 10 Some cloud provides have different prices for the same instance from each data centre (eg US East and EU (Ireland) data centres for Amazon EC2). Other cloud providers have standard prices for each instance. This has been amended from the previous report to reflect a change of view on hyperthreading for this instance and for a rounding/typographic error. CC551D001-1.0 2.2.2 The autumn 2011 data was analysed for the previous report (see Annex C.2). It was found that, while noting that researchers need to trade-off application performance with cost and choice of cloud instance, they could expect to pay up to 19p per core-hour for instances most suitable for CPU-bound tasks, with the majority of providers charging in the region of ~7p to ~10p per corehour. 2.2.3 The analysis has been repeated for the March 2014 data (see Annex B.2) with a very similar result. In particular, it was found while noting that researchers need to trade-off application performance with cost and choice of cloud instance, they can expect to pay up to 19p per corehour for instances most suitable for CPU-bound tasks, with the majority of providers charging ~6.5p to ~7.5p per core-hour). 2.3 Storage 2.3.1 The cloud providers considered offer many types of data storage16 with differing characteristics (eg performance, capacity, local or geographical redundancy, etc). This can makes comparison difficult. Storage can be ephemeral (where data held vanishes once the related instance is stopped) or persistent. This report categorises persistent storage as:17 – Block: This provides instance-attached persistent storage; once an instance is stopped, the data is retained until deleted (eg Amazon EBS, Azure and Google Compute-Engine). – Object: This provides the means to store data and virtual machine images, accessible from anywhere until deleted (eg Amazon S3, Azure and Google Compute-Engine). – Virtual disk: This covers persistent provisioned HDD or SSD storage (eg Eduserv, ElasticHosts). – Other: This covers other storage types, where available, such as archive storage (eg Amazon Glacier) or snapshot storage. 2.3.2 Indicative prices for storage for each of the cloud service providers considered are set out at Table 2-2 for March 2014 (no shading) and autumn 2011 (with shading). Complete information across the categories was not collected for autumn 2011. 16 This excludes RAM. This is based on OpenStack storage concepts ops/content/storage_decision.html [accesses 28 March 2014]. 17 CC551D001-1.0 (see http://docs.openstack.org/trunk/openstack- 11 18 Cloud provider Amazon EBS 20 Amazon S3 Storage charge (£/month/GB) Object 0.031 - - - Standard: flat rate plus £0.031 per million IO requests for 1GB-1TB 0.060 - - - Standard: flat rate plus £0.031 per million IO requests for 1GB-1TB 0.078 - - - - - 0.095 Snapshots stored on S3: flat rate - Standard: tiered structure - 0.038 – 0.053 - - 0.031 – 0.043 - 21 Azure ElasticHosts Flexiscale - - RRS: tiered structure - - 0.006 Glacier archive storage: flat rate - - - - - 0.039 – 0.045 LRS: tiered structure 0.040 – 0.055 0.046 – 0.061 - 0.046 - 0.061 GRS: tiered structure 0.052 – 0.071 0.059 – 0.077 - 0.059 – 0.077 RA-GRS: tiered structure - - - - - - - - Tiered, lower range corresponds to 10TB target - - Tiered, lower range corresponds to 10TB target - Current offering is 36GB, 73GB, 146GB or 300GB. 0.054 - 0.180 - - - 0.091 – 0.245 - - For a Storage Attached Network (SAN) volume - 0.054 - 0.183 - 0.020 - For a dedicated 250GB drive - - 0.300 Tier 1 (SAS or SATA and SSD hybrid storage): flat rate - - 0.175 Tier 2 (SATA storage with SSD metadata acceleration): flat rate - - 0.067 Assumed Tier 2 equivalent - - 0.110 Tier 3 (SATA storage supporting archiving): flat rate - - 0.300 SSD: flat rate - - 0.060 HDD: flat rate - - 0.060 - - 0.045 - - 0.075 GoGrid IOPS: flat rate plus £0.063/provisioned IOPS/month for 1GB-1TB 0.032 – 0.032 0.054 -0.180 Eduserv Other 0.032 - 0.044 0.088 Baremetalcloud (was NewServers) Attached 19 disk Notes Block - 0.046- 0.055 HDD: flat rate - Flat rate for disk storage, plus £0.018 per GB transferred - Flat rate for disk storage, plus £0.018 per GB transferred - - Block storage: flat rate - 0.063 – 0.074 - - Cloud storage: tiered with first 10 GB/month free - 0.094 - - Cloud storage: tiered with first 10 GB/month free Table 2-2: March 2014 persistent storage costs for a range of commercial cloud providers (part 1 of 2) 18 19 20 21 12 The first element of a range given for a tiered structure gives the price for 500TB/month, unless otherwise stated (eg, where 500TB of storage is not offered). The second element gives the price for lowest charged-for tier. Attached disk can be virtual or physical, depending on the offering. Standard S3 storage offers multiply redundant, highly available storage. Reduced Redundancy Storage (RRS) provides a lower level of redundancy. Glacier provides highly available archive and backup storage (see http://aws.amazon.com/s3/details/ [accessed 28 March 2014]). Azure offers Locally Redundant Storage (LRS), Geographically Redundant Storage (GRS) and Read-Access GRS (RA-GRS). Other refers to prices for NoSQL database storage. CC551D001-1.0 22 Cloud provider Google Compute Engine HP Cloud Storage charge (£/month/GB) Object - - 0.025 - Persistent disk: flat rate - - - 0.078 Snapshot storage: flat rate - - - - - - - - Flat rate plus £0.062 per million IO requests - 0.056 - - Flat rate plus £0.006 per 10,000 object requests - - - 0.063 Snapshot stored in HP object storage: flat rate - - - - - 0.094 Joyent 0.040 – 0.054 23 SoftLayer (IBM SmartCloud) Terremark vCloud Express NFS (NetApp) - - Manta: tiered, this is the price per copy with the default being 2 copies but 1 – 6 copies possible, spread across 3 data centres in US-East - 0.229 - - - - 0.063 - - Free for instance allocated storage. Uses CEPH distributed object storage system 0.000 0.125 - - - 0.370 - - - SSD volume: flat rate, 100GB to 1 TB, up to 14 volumes - 0.090 Rackspace Other 0.063 - Penguin Attached disk Notes Block - - Standard volume: flat rate, 100GB to 1TB,up to 14 volumes - 0.058 – 0.070 - - Cloud files: tiered structure - 0.110 - - Cloud files - - - 0.070 Snapshot: flat rate - 0.063 - - Object storage: flat rate, in 20GB increments up to 300GB as additional instance drive - 0.068 - - Object storage 0.469 - - - SAN (iSCSI): flat rate, in 20GB increments - - 0.313 - NAS: flat rate, in 20GB increments - - 0.156 - Flat rate: up to 15 virtual disks (including system) up to 512GB each - - 0.156 - - Table 2-2: March 2014 persistent storage costs for a range of commercial cloud providers (part 2 of 2) 2.4 Data transfer The prices for data transfer for each of the cloud service providers considered are set out at Table 2-3 for March 2014 (no shading) and autumn 2011 (with shading). 22 23 The first element of a range given for a tiered structure gives the price for 500TB/month, unless otherwise stated (eg, where 500TB of storage is not offered). The second element gives the price for lowest charged-for tier. Storage charge is based on $0.012 per day and 30.5 days in a month. CC551D001-1.0 13 Cloud provider Data transfer in from internet (£/GB) Data transfer out to internet (£/GB) 0.000 0.037 – 0.075 Outbound has tiered structure with first 1GB free, plus costs to transfer between availability zones for outbound 0.000 0.03 - 0.07 Plus costs to transfer between availability zones for outbound 0.000 0.044 – 0.080 Charges for data transfer out are tiered with first 5GB out free 0.000 0.09375 - 0.125 Amazon EC2 Azure Baremetalcloud (was NewServers) Eduserv ElasticHosts Flexiscale GoGrid Google Compute Engine HP Cloud Rackspace SoftLayer (IBM SmartCloud) Terremark vCloud Express - 0.000 0.000 Free Internap 0.0625 0.0625 - 0.000 0.105 Flat rate 0.000 0 - 0.100 0.200 0.100 0.200 Committed in advance Burst above committed level 0.100 - 0.200 0.100- 0.200 - 0.045 0.045 Flat rate 0.018 – 0.022 0.018 – 0.022 - 0.000 0.053 – 0.075 Free private network data transfer between servers within a single data centre plus free inbound data transfer Outbound has tiered structure with first 1GB free 0.181 0.181 - 0.000 0.050 – 0.075 For egress to EMEA Tiered structure, 0.050 applies for more than 10TB/month - - - 0.000 0.049 – 0.100 Tiered structure with first 1GB free, Low corresponds to 250TB/month - - - 0.000 0.037 – 0.075 Tiered structure with first 1 GB out free 0.080 0.080 Free 20TB internet transfer per 24 month, then £0.08 per GB/month 0.000 0.000 Covered by log in node charges (if any) 0.000 0.000 - 0.000 0.046 - 0.080 Tiered structure 0.000 0.120 - 0.063 0.063 Flat rate 0.050 - 0.094 0.050 - 0.094 - 0.106 0.106 Flat rate 0.106 0.106 - Joyent Penguin Notes Table 2-3: March 2014 and autumn data transfer costs for a range of cloud providers 2.5 Reserved and spot instances 2.5.1 Reserved and spot instances provide a means of obtaining a discount on the price in return for an increased commitment or other trade-offs (eg only running a job when the spot price is below a pre-fixed level). The potential discounts achievable need to be considered in the context of these trade-offs. There may also be other obstacles to the use of such instances (eg the previous report observed that institutional policies might currently demand a tendering procedure be followed to take advantage of reserved and spot instances). 24 This option was withdrawn in September 2013. 14 CC551D001-1.0 Reserved instances Some cloud providers offer discounts if customers agree to purchase resources for a fixed period – for example, Microsoft Azure offers approximately 20% off compute prices for the instances set out at Table B-3 for a 6 to 12 month plan. Probably the most complex of these offerings are Amazon’s reserved instances. Three tiers are available, intended for light, medium, and heavy usage, and each may be reserved for a term of 1 or 3 years. An up-front fee is charged, leading to reduced hourly charges. The previous report considered the example of the Amazon EC2 cc2.8xlarge. The on demand and the reserved instance prices have not changed between autumn 2011 and March 2014. Figure 2-1 sets out the total price of each reserved instance type compared with the on-demand version at different utilisation levels25 for a one-year and threeyear reserved cc2.8xlarge (Cluster Compute) instance. 2.5.2 25,000 25,000 20,000 20,000 15,000 15,000 10,000 10,000 5,000 5,000 - - % uptime over 1 year On demand Light Medium % uptime over 3 years Heavy On demand Light Medium Heavy Figure 2-1: Total cost over 1 and 3 years at different levels of utilisation for the reserved and ondemand instances of the Amazon EC2 cc2.8xlarge cluster compute instance26 Spot instances 2.5.3 Spot instances provide an opportunity to bid for use of instances when the market price of such instances is below a user-specified value - with the price set dynamically and instances dropped when the price goes above the user-specified value. A snapshot of the spot prices for the Amazon EC2 compute categories (see Table B-3) is set out at Table B-8. This illustrates that use of spot instances can achieve savings of 70% – 80% over the on-demand prices, but at the risk of work being lost if the price changes suddenly. Cycle Computing 2.5.4 Cycle Computing is included as it was covered in the autumn 2011 report. However, it now operates in a different way. Its CycleCloud offering is a software layer that helps orchestrate the creation of technical computing and HPC clusters in the cloud. CycleCloud makes it easier for researchers and scientists to deploy, secure, automate, and manage running calculations dynamically. This includes managing spot and reserved instances to achieve value for money (eg, users can specify the maximum cost per core-hour). Cycle Computing claims savings of 25 – 40% over on-demand instances. Users typically have a separate account with a cloud services provider and pay a monthly fee to Cycle Computing for this service. The fee is tiered with the lowest level providing up to 26,667 core-hours per month for $800. 25 For a reserved instance, utilisation is the proportion of time that the instance is running, rather than the proportion of time that a scientific load is running on it. The 3-year heavy utilisation line was incorrect in the earlier report and this has been corrected. 26 CC551D001-1.0 15 2.6 Comparison with cost of institutional research computing 2.6.1 Figure 2-2 shows the on-demand price per core-hour costs for the High and HPC high instances for autumn 2011 and March 2014.27 In the previous report is was found that: “On a pure price comparison, the more powerful cloud computing instances, rented on an hourly basis, appear to be one-and-a-half to two times the price per core-hour of wellmanaged, locally-provided clusters in modern data centres operating at high utilisation levels.” March 2014 0.200 0.175 0.175 0.150 0.150 £ per core-hour £ per core-hour Autumn 2011 0.200 0.125 0.100 0.075 0.050 0.125 0.100 0.075 0.050 0.025 0.025 - - High High HPC High HPC High Institutional low range per core-hour Institutional low range per core-hour Institutional high range per core-hour Institutional high range per core-hour Figure 2-2: Autumn 2011 and March 2014 comparison per core-hour price (£) for on-demand High and HPC High compute categories with estimated institutional research computing costs 2.6.2 Examination of Figure 2-2 suggests a different answer for March 2014. The previous report (see paragraph 1.3.3) estimated that the costs of institutional research computing were around 5p to 7p per core-hour. Using the estimate from the previous report that the costs of institutional research computing were around 5p to 7p per core-hour (see paragraph 1.3.3), Table 2-4 sets out the number of instances for the HPC Low, Low, High and HPC High compute categories below 5p, 7p and 7.7p per core-hour. The latter price is a 10% uplift on the upper range. The majority of the instances (48/56) for these compute categories are now below the 7.7p per core-hour compared to 31/44 in autumn 2011. Similarly, 33/56 instances are now below 5p per core-hour compared to 13/44 in March 2011. This represents a significant reduction in on-demand prices since the previous report. Provider When HPC low Low High HPC high Instances below 5p per corehour Autumn 2011 5/11 5/11 2/11 1/11 March 2014 9/14 11/14 7/14 6/14 Instances below 7p per corehour Autumn 2011 7/11 9/11 5/11 4/11 10/14 12/14 11/14 8/14 Instances below 7p per corehour + 10% Autumn 2011 8/11 10/11 6/11 5/11 11/14 13/14 13/14 11/14 March 2014 March 2014 Table 2-4: Comparison of costs per core-hour for on demand instances with institutional research computing 27 16 Low and HPC Low compute categories have been omitted to simplify the Figures. CC551D001-1.0 Overall, it seems that in March 2014, the revised statement should be: 2.6.3 “On a pure price comparison, the majority of the cloud computing instances, rented on an on demand hourly basis, are less expensive or the same price per core-hour as wellmanaged, locally-provided clusters in modern data centres operating at high utilisation levels.” 2.6.4 It is perhaps more accurate to say that the price ranges of public cloud providers and institutions overlap and have a similar upper bound. Under some circumstances, cloud will offer a ‘better’ solution and that for others a traditional data centre will remain more appropriate. The final decision will depend on the precise performance of the selected approach. The trend of price reductions such as the recent ones by Amazon, Google Compute-Engine and Azure reinforces these findings. 2.6.5 Importantly, this comparison takes no account of the performance implications of the different infrastructures for specific research tasks. The degree of overlap will be different for each institution. 2.6.6 The above analysis reflects on-demand prices only. The comparison with estimated institutional research computing costs for reserved or spot instances is illustrated in Figure 2-3, which shows the comparison if reductions of 20% and 80% can be extrapolated for each on-demand price (see subsection 2.5). There are disadvantages of reserved and spot instances such as increased commitment, increases in elapsed processing time, overnight working, etc, which need to be taken into account when making a decision. However, it is clear that use of the cloud computing can be more economic than using institutional resources. March 2014 80% discount 0.200 0.175 0.175 0.150 0.150 £ per core-hour £ per core-hour March 2014 20% discount 0.200 0.125 0.100 0.075 0.125 0.100 0.075 0.050 0.050 0.025 0.025 - - High High HPC High HPC High Institutional low range per core-hour Institutional low range per core-hour Institutional high range per core-hour Institutional high range per core-hour Figure 2-3: March 2014 comparison of per core-hour price (£) for reserved and spot instances for High and HPC High compute categories with estimated institutional research computing costs 2.7 How prices have changed Price per core-hour 2.7.1 The changes in the price per core-hour between autumn 2011 and March 2014 for each cloud provider and for each compute category are shown at Figure 2-4. With some exceptions, there has been a reduction in the price per core-hour across the various compute categories since autumn 2011. For those cloud providers that have reduced prices, this is equivalent to a reduction of ~2p – ~5p per core-hour on average across the compute categories. CC551D001-1.0 17 2.7.2 Relative prices also change between the different compute categories for the same cloud provider. For example, Flexiscale has changed its prices so that the HPC high category is now cheaper than the HPC Low category.28 2.7.3 It is important to understand that the instances used to represent the compute categories in March 2014 (Table B-3) and autumn 2011 (Table C-2) are of necessity different because of changes in the offering. It follows that the reductions need to be seen in the context of the otherwise unquantified improvements in compute capability represented by these changed instances. Change (£ per core-hour) -0.100 0.100 -0.200 0.200 Amazon EC2 Azure Baremetalcloud (previously NewServers) Eduserv ElasticHosts Flexiscale GoGrid Joyent Penguin SoftLayer previously IBM SmartCloud) Terremark vCloud Express HPC Low Low High HPC High Figure 2-4: Comparison of per core-hour prices (£) between autumn 2011 and March 201429 Storage and data transfer 2.7.4 The changes in the price for storage and data transfer between autumn 2011 and March 2014 are set out at Figure 2-5.30 2.7.5 The price for storage (£/month/GB) has either stayed the same or decreased for nearly all31 cloud providers considered. The average decrease in price is ~1.4p/month/GB. 28 29 30 31 18 This leads to the shaded bar in Figure B-1. Cycle Cloud has been excluded from the comparison because of its different business model (see paragraph 2.5.4). The comparison is only given where there is an offering in both autumn 2011 and March 2014. The measure used is the difference between the midpoints of the price ranges in autumn 2011 and March 2014. Eduserv and Amazon EC2 have increased prices for storage. The increase for the latter is very small and is possibly the result of a difference in rounding for the previous report. CC551D001-1.0 2.7.6 In autumn 2011, 5 out of 12 cloud providers considered did not charge for inbound data transfers. In March 2014, 9 out of 12 cloud providers considered in autumn 2011 did not charge for these transfers. The average reduction is ~3.3p/GB. Most cloud providers continue to charge for outbound data transfers and there has only been a very modest average reduction in price (~0.2p/GB) with some cloud providers increasing32 the price. -0.200 -0.100 Change (£/GB) - 0.100 0.200 Amazon EBS Azure Baremetalcloud (previously NewServers) Eduserv ElasticHosts Flexiscale GoGrid Joyent Penguin Rackspace SoftLayer previously IBM SmartCloud) Terremark vCloud Express Block Object Attached Transfer in Transfer out Figure 2-5: Comparison of storage and data transfer prices (£/GB) between autumn 2011 and March 201433 Reserved and spot instances 2.7.7 The only reserved instance that was documented in autumn 2011 was the Amazon EC2 cc2.8xlarge instance with Linux from the US East data centre. Neither the on-demand nor the reserved instance prices have changed for this instance. As many other instances and prices have changed between autumn 2011 and March 2014, this is unlikely to represent the general case. 2.7.8 In autumn 2011, the spot prices for the same instance had been offered at lows of £0.021 per core-hour, but with peak prices of £0.113 and £0.141 per core-hour compared to the on-demand 32 Eduserv and Amazon EC2 have increased prices for outbound data transfers. Again, the latter price might be due to a difference in rounding for the previous report. Cycle Cloud has been excluded from the comparison because of its different business model (see paragraph 2.5.4). 33 CC551D001-1.0 19 price of £0.094 per core-hour. Over the period 22 February to 23 March 2014, the spot price for this instance was £0.011 per core-hour with peaks of £0.0145 and £0.013 per core-hour. Is cloud computing competitive? 2.7.9 The previous report found that there is no easy answer to the question ‘what does cloud computing cost compared to other means of providing research computing?’ “It is a question that is specific to each and every institution depending on the performance of the local hardware and its capital and operating costs and priorities. Public cloud computing providers’ charges are at least transparently published, even if it is not always clear how they stack-up in real-life research situations or what ‘bang for the buck’ is actually possible. However, comparative costing is made more difficult by the complicated system of research computing funding – and the costing models can change from institution to institution. So far as specific comparisons can be made, there is enough evidence to suggest that cloud computing can be cost-effective for research – at least for some researchers in some institutions. For other researchers, the cloud cannot compete on price alone with institutional services. However, in this dynamic environment the balance of relative costs for any given research task can change. For example, the competitive environment of public cloud computing may push costs down, but on the other hand, it is possible that profitdriven organisations may raise charges, for example in the face of higher energy bills or taxes as relate to their areas of operation.” 2.7.10 This report concurs with this finding but notes that the general trend in price reduction found and the potential of reserved and spot prices mean that the cloud is likely to be more competitive for research computing than in autumn 2011. 2.8 Google Compute-Engine, Amazon and Azure prices from 1 April 2014 2.8.1 Since the research for this report was carried out in early – mid March 2014, Google-Compute Engine has announced a 32% reduction in prices for its cloud services across all regions, sizes and classes from 1 April 2014, together with a simplification of the charging structure. Amazon EC2 has responded with significant price reductions from 1 April 2014, as has Azure from 1 May 2014. Prices for compute categories 2.8.2 20 The changes in instance pricing for Amazon EC2, Google Compute-Engine and Azure for the compute categories are set out in Table 2-5. CC551D001-1.0 Provider HPC low Low High HPC high HPC nodes High memory large GPU Data centre used 34 for price Amazon EC2 – March 0.047 0.038 0.070 0.047 Y 0.068 0.051 US East Amazon EC2 – April 0.039 0.028 0.044 0.033 Y 0.068 0.051 US East Reduction 16.7% 26.7% 37.8% 30.0% - 0.0% 0.0% Google Compute Engine March 0.041 0.065 0.065 0.065 N 0.076 N US Google Compute Engine April 0.028 0.044 0.044 0.044 N 0.051 N US 32.6% 32.7% 32.4% 32.5% - 32.8% - Azure - March 0.038 0.039 0.038 0.038 N 0.103 N US East Azure – May 0.038 0.039 0.038 0.038 N 0.080 N US east Percentage reduction 0.0% 0.0% 0.0% 0.0% N Percentage reduction 35 -22.4% - - - - Table 2-5: Comparison of March and post May 2014 prices for Amazon, Google-ComputeEngine and Azure (£/core-hour) for various compute categories Storage 2.8.3 For storage, Amazon has reduced the prices for Amazon S3 storage (Table 2-6) and left Amazon EBS unchanged. Google Compute-Engine has left storage prices unchanged. 36 Cloud provider Storage charge (£/month/GB) Block Object Attached disk Notes Other Amazon S3 March - 0.038 – 0.053 - - Standard: tiered structure - 0.031 – 0.043 - - RRS: tiered structure Amazon S3 April - 0.018 – 0.019 - - Standard: tiered structure - 0.015 – 0.015 - - RRS: tiered structure - 52.8 – 64.7% - - Standard: tiered structure - 52.8 – 64.7 - - RRS: tiered structure Reduction Table 2-6: Comparison of March and April 2014 persistent storage costs for Amazon S3 Data transfer 2.8.4 Neither Amazon nor Google-Compute Engine has changed data transfer prices. Reserved and spot instances 2.8.5 Amazon has also reduced some reserved instance prices in line with the changes in prices for the compute categories. The reserved instances prices for the Amazon EC2 CC2.8xlarge instance used as an exemplar for the prices savings available by using reserved instances(Figure 2-1) is unchanged, with the exception that the light usage tier appears to be no longer available. 2.8.6 No new pricing for spot instances is yet available. 34 Some cloud provides have different prices for the same instance from each data centre (eg US East and EU (Ireland) data centres for Amazon EC2). Other cloud providers have standard prices for each instance. Azure has reduced the price of memory intensive instants from 1 May 2014, which corresponds only to the GPU compute category. Azure has also introduced a new basic compute tier from 2 April 2014 that removes load balancing and auto-auto scaling from the standard compute tier. The first element of a range given for a tiered structure gives the price for 500TB/month, unless otherwise stated (eg, where 500TB of storage is not offered). The second element gives the price for lowest charged-for tier. 35 36 CC551D001-1.0 21 Implications for findings of this report 2.8.7 22 It is too early to say what the implications are for the cloud market as a whole, but it is entirely possible that other cloud providers will respond. The trend of price reductions reinforces the findings paragraph 2.7.10. CC551D001-1.0 A.1 Introduction A.1.1 This Annex provides a worked example of the cost of providing a broadly equivalent configuration to an Amazon EC2 m3.2xlarge instance. This is based on Dell prices available on the Internet for a PowerEdge 720 rack server with an Intel E5-2670V2 processor with 10 cores. The m3.2xlarge instance is based on slightly earlier technology (the Intel E5-2670 with 8 cores}. A.2 Cost model A.2.1 The cost model for the server is set out at Table A-1. Item Description Basic package Dell PowerEdge R720 rack server with an Intel Xeon E5-2670V2 2.5GHz processor (10 cores), 32GiB, Red Hat Linux 6.4, H710p Raid with 2x146GB SAS drives + 2x200GB SSDs with Broadcomm 57800 2x10Gb DA/SFP+2*1Gb BT daughter network card Dell NetShelter 42U Rack Less Dell discount Less 2x146GB SAS hybrid drives Less 2x200GB SSDs Element cost (£) Subtotal 5,728 900 (897) (462) (1,126) 2x100GB SSD 6Gbit/s 748 Dual hot-plug redundant power supply 147 4,891 Reliability package Additional power cord 8 Dell Smart-UPS 1,500VA 660 Estimated as 1 week of support staff effort @£25k per annum, including salary, pension, employer NICs and office space 561 815 Installation and set up 561 Total up-front cost Annual running costs 6,267 Ubuntu 0 Dell ProSupport with 8 hour mission critical support for 3 years (annual rate = 3 year rate/3) 483 Security, hardware, software, operations admin support (@10% of up-front costs) 627 Power (based on 80% of 495W used and Power Usage Efficiency of 170%) at 10p per Kw-hour 472 Space (1 m2 @ £50 per ft (£538 per m2 per year) 538 2,120 Total cost over 3 years Total up-front cost + 3* annual running cost plus contingency of 20% 15,151 Table A-1: Cost breakdown for a server providing broadly equivalent capability to an Amazon EC2 m3.2xlarge instance A.2.2 Assumptions and comments are set out below. 1) The price was calculated using the Dell inline customisation tool and included a discount of £897.10. Education discounts may also be available. 2) The precise configuration of 2x80GB SSDs is not available; this has been modelled by assuming 2x146GB SSDs could be negotiated with Dell rather than the HDD and SSD combination available. CC551D001-1.0 23 3) RAID, redundant power supply and UPS have been included to provide a similar level of availability and data integrity to the m3.2xlarge instance. 4) The cost of space is based on that charged by a university research park. 5) The utilisation of 80% and power usage efficiency of 170% are taken from a web article on power costs for large-scale data centres.37 The figure 10% of capital costs for admin support is from the same source. These may be somewhat low for a small-scale institutional system. 6) It is assumed that Ubuntu will be used as the operating system at no cost. 7) The cost per core-hour could be reduced by packing more servers into the rack, which has 42U of space and could thus take 21 servers. 8) Inevitably, there additional hardware, software elements and support will be required to make a working system, especially for a low number of servers. Accordingly, a 20% contingency has been included. A.3 Cost per core-hour A.3.1 Table A-2 sets out the per core-hour cost based on the cost model of Table A-1. This assumes utilisation of 80%. For comparison, the price per core-hour for 100% is 5.93p and that for 50% is 11.05p. Description Element cost (£) Number of hours per year Element cost (p) 8,760 Utilisation 80% Number of hours used in 3 years at this utilisation 21,024 Cost per core-hour for 10 cores, made up of: 7.21 Basic package 2.33 Reliability package 0.39 Installation and set up 0.27 Software licence 0.00 Hardware and software support 0.69 Admin support 0.89 Power 0.67 Space 0.77 Contingency 1.20 Table A-2: Breakdown of per core-hour for a server providing broadly equivalent capability to an Amazon EC2 m3.2xlarge instance A.4 Assessment A.4.1 The core-hour cost matches the 5p-7p range found last time. It is towards the top of the range because there is little economy of scale. What is interesting is that the headline hardware cost is ~2p per core-hour, but the true costs, including security, admin and software operations, together with power, cooling and maintenance support are around 7p per core-hour. 37 See cost of power in large-scale data centres at http://perspectives.mvdirona.com/2008/11/28/CostOfPowerInLargeScaleDataCenters.aspx [accessed 24 April 2014]. 24 CC551D001-1.0 B.1 Introduction B.1.1 This Annex sets out the instances selected and assumptions used to generate the price information in Section 2. The data was sourced from cloud providers’ websites and converted to pounds sterling using an exchange rate of $1.60 to £1. B.2 Compute Compute categories B.2.1 The compute prices in £/core-hour are provided at Table B-238 for the instances listed at Table B-3. These reflect the lowest commitment, so bulk discounting is not considered. The focus is on CPU rather than RAM, and storage and data transfer costs are excluded wherever possible. Six prices are presented for each cloud provider: – HPC Low and HPC High: these are designed to be appropriate for CPU-bound scientific problems. Differentiated on CPU where possible, otherwise by RAM. – Low: the lowest price per core-hour available. This excludes some “micro” type instances where the instance does not have an allocated virtual core proportion. – High: Aiming for a specification similar to Amazon’s erstwhile High CPU extra-large instance type (c1.xlarge). – High Memory Large: An instance with the largest amount of RAM. – GPU: instances offering Graphics Processing Unit (GPU) cards as a processing resource. B.2.2 In addition, when a cloud provider offers HPC-specific nodes, this is highlighted. B.2.3 The elements of the instance description are described using the template explained at Table B-1. The prices are for the Linux operating system as shown in Table B-4. The changes in cloud offering between autumn 2011 and March 2014 mean that new instances have typically been selected for the compute categories that have roughly the same or better overall performance. Item Notes 1 Instance name 2 Number of virtual CPUs 3 Processor memory included Can be in Gibibyte (GiB) or Gigabyte (GB) 4 Number and type of disks Can be SSD or HDD 5 Network bandwidth Gbit/s or a bandwidth descriptor (eg high) Table B-1: Instance description template 38 The Table is in the same order as the March 2014 analysis, with the same names. CC551D001-1.0 25 Provider HPC low Low High HPC high HPC nodes High memory large GPU Data centre used for price Amazon EC2 0.047 0.038 0.070 0.047 Y 0.068 0.051 US East Azure 0.038 0.039 0.038 0.038 N 0.103 N US East Baremetalcloud (was NewServers) 0.054 0.047 0.054 0.054 N 0.054 N Standard price 0.063 1.386 0.202 0.063 Y 0.073 N The example is based on US East Amazon EC2 instances with a 20% utilisation and 30% cost savings generated by the use of CycleCloud’s orchestration software (see paragraph 2.5.4) Eduserv 0.047 0.043 0.092 0.046 N 0.185 N UK ElasticHosts 0.188 0.080 0.012 0.188 N 0.188 N UK 0.045 0.018 0.045 0.027 N 0.027 N UK - priced in Flexiscale units GoGrid 0.038 0.038 0.038 0.038 N 0.038 N Standard price Google Compute Engine 0.041 0.065 0.065 0.065 N 0.076 N US HP Cloud 0.094 0.019 0.070 0.127 N 0.211 N Standard price Joyent 0.045 0.035 0.075 0.075 N 0.127 N Standard price 0.075 0.075 0.075 0.075 Y 0.075 N Standard price per core-hour across the range Rackspace 0.125 0.030 0.030 0.125 N 0.125 N UK SoftLayer (IBM SmartCloud) 0.037 0.030 0.049 0.037 N 0.037 N Amsterdam – Western Europe Terremark vCloud Express 0.050 0.022 0.050 0.073 N 0.073 N Standard price Cycle Computing 39 CycleCloud Flexiscale Penguin Table B-2: March 2014 on-demand prices (£/core-hour) for various compute categories 39 26 The Cycle Cloud price assumes savings of 30% for 50% utilisation for each compute category and with core-hours being within the lowest price tier of $800 per month. This model is realistic for each compute category, except for low. This is because the savings assumed are much less than the monthly charge in this case. CC551D001-1.0 Provider HPC Low Low High HPC High Amazon EC2 cc2.8xlarge 32 vCPU (16 cores ex hyper threading) 60.5GiB 4x840GB 10Gbit/s m1.small 1 vCPU 1.7GiB 1x160GB m3.2xlarge 8 vCPU 30GiB 2x80GB SSD High c3.8xlarge 32 vCPU 60GiB 2x320GB SSD 10Gbit/s Azure Extra large (A4) 8 vCPU 14GB Small (A1) 1 vCPU 1.75GB Large (A3) 4 vCPU 7GB Extra large (A4) 8 vCPU 14GB Baremetalcloud (was NewServers) 2x2.00GHz Gainestown E5504 8 cores 48GB 2x147GB 1x2.66 GHz Woodcrest E5150 2 cores 2GB 73GB 2x2.00GHz Gainestown E5504 8 cores 48GB 2x147GB 2x2.00GHz Gainestown E5504 8 cores 48GB 2x147GB Cycle Computing CycleCloud HPC Nodes Y High memory large GPU cr1.8xlarge 32 vCPU 244GiB 2x120GB SSD 10Gbit/s g2.2xlarge 8 vCPU 15GiB 1x60GB SSD High N A7 8 vCPU 56GB N N 2x2.00GHz Gainestown E5504 8 cores 48GB 2x147GB N As for Amazon EC2 as an example Eduserv 32 vCPU 60.5GB 1 vCPU 1.7GB 8 vCPU 30GB 32 vCPU 60GB N 32v CPU 244GB N ElasticHosts 8 vCPU (20,000 core MHz) 32GB 1 vCPU (2,000 core MHz) 1GB 4 vCPU (10,000 core MHz) 7GB 8 vCPU (20,000 core MHz) 32GB N 8 vCPU (20,000 core MHz) 32GB N Flexiscale 4 vCPU 8GB 1vCPU 0.5GB 4 vCPU 8GB 8vCPU 8GB N 8vCPU 8GB N GoGrid XX-Large 16 vCPU 16G 800GB X-Small 0.5 vCPU 0.5GB 25GB X-Large 8 vCPU 8GB 400GB XXX-Large 24 vCPU 24GB 1,200GB N XXX-Large 24 vCPU 24GB 1,200GB N Google Compute Engine n1-highcpu-16 16 vCPU 14.4GB 0GB n1-standard-1 1 vCPU 3.75GB 0GB n1-standard-8 8 vCPU 30GB 0GB n1-standard-16 16 vCPU 60GB 0GB N n1-highmem-16 16 vCPU 104GB 0GB N Standard 4XL 12 vCPU 60GB 900GB Standard extra small 1 vCPU 1GB 20GB Standard 2XL 8 vCPU 30GB 570GB Standard 8XL 16 vCPU 120GB 1,800GB N High Memory 2XL 4 vCPU 60GB 570GB N Joyent High CPU 7 vCPU 7GiB 263 GB up to 1Gbit/s Standard 1 vCPU 1.75GiB 56GB up to 1Gbit/s Standard 8 vCPU 30GiB 1,683GB up to 1Gbit/s Standard 8 vCPU 30GiB 1,683GB up to 1Gbit/s N High memory 8 vCPU 68.375GiB 1,122 GB up to 1Gbit/s N Penguin H30 queue 32 vCPU 128GB 1TB scratch on login node 10GigE data network M40 queue 1 vCPU 4GB 1TB scratch 10GigE data network M40 queue 8 vCPU 32GB 1TB scratch on login node 10GigE data network H30 queue 32 vCPU 128GB 1TB scratch on login node 10GigE data network Y M40 queue 32 vCPU 128GB 1TB scratch on login node 10GigE data network N Performance flavours class 2 24 vCPU 90GB 3x300GB SSD 7.5Gbit/s Performance flavours class 1 1 vCPU 1GB SSD 1x20GB 200Mbit/s Performance flavours class 1 8 vCPU 8GB SSD 1x40GB SSD +1x80GB SSD 1.6Gbit/s Performance flavours class 2 32v CPU 120GB 1x40GB SSD + 1x1,200GB SSD 10Gbit/s N Performance flavours class 2 32 vCPU 120GB 1x40GB SSD + 1x1,200GB SSD 10Gbit/s N SoftLayer (IBM SmartCloud) 16 vCPU 48GB 100GB 1 vCPU 1GB 100GB 8 vCPU 32GB 100GB 16 vCPU 48GB 100GB N 16 vCPU 48GB 100GB N Terremark vCloud Express 8 vCPU 8GB 1 vCPU 0.5GB 8 vCPU 4GB 8 vCPU 16GB N 8 vCPU 16GB N HP Cloud Rackspace Table B-3: Instances used for March 2014 costs CC551D001-1.0 27 40 Provider Notes and further information Amazon EC2 Linux included in price Azure Linux and Windows included in price Baremetalcloud (was NewServers) OS of choice included in per hour price, including Ubuntu VMWare+ OpenStack The instances selected for Baremetalcloud are all bare metal for comparability with autumn 2011. Baremetalcloud now also provides virtual servers. Cycle Computing CycleCloud As per chosen cloud solution Eduserv Linux included in base price ElasticHosts Debian 6.0, Ubuntu 12.04.1 LTS and CentOS Linux included in base price ElasticHosts does not have fixed instance types. The instances in Table B-3 are chosen as exemplars Flexiscale Linux included in base price GoGrid Windows Server, and Red Hat Enterprise Linux included in base price Google Compute Engine Linux included in base price HP Cloud Linux included in base price Joyent Linux included in base price Penguin Bare-metal servers Scyld ClusterWare with support for SUSE Linux Enterprise included in base price Support included in price Data (or Network) transfers both in and out included in price Access via CLI (command line interface) or GUI Web Portal Rackspace Linux included in base price SoftLayer (IBM SmartCloud) CentOS Linux included in base price Terremark vCloud Express Unlicensed servers (assumed to mean Linux) Table B-4: Operating systyems selected for March 2014 instance prices B.2.4 Figure B-1 presents graphically for each cloud provider the range of prices set out at Table B-2. The thin bars represent the range of prices (Low to High in Table B-2) that would be appropriate for CPU-bound tasks. The wide bars represent the range of prices for instances likely to be most suitable for CPU-bound tasks (HPC low to HPC high). These are typically larger instances with more virtual cores, and in the case of Amazon and Penguin, they represent distinct HPC nodes with superior characteristics. It is important to note that prices are not normalised by performance. B.2.5 The conclusion drawn from this analysis in March 2014 is that, while noting that researchers need to trade-off application performance with cost and choice of cloud instance, they can expect to pay up to 19p per core-hour for instances most suitable for CPU-bound tasks (with the average of the ranges being ~6.5p to ~7.5p per core-hour). 40 All servers are virtualised unless stated otherwise. 28 CC551D001-1.0 Cost per core-hour £0.20 £0.15 £0.10 £0.05 £0.00 Figure B-1: Range of cloud prices, not normalised for performance; thick bars indicate instances of most relevance for CPU-bound application (see paragraph B.2.3) B.3 Storage B.3.1 Table B-5 shows indicative persistent storage costs for a range of cloud providers. The cloud providers use either a flat rate or a tiered structure for pricing storage. Small amounts of storage (eg first 1GB) can be free. For a tiered structure, the range represents the prices for 500TB of storage, unless specified otherwise, and the smallest charged volume (ie the next GB, where the first GB is free). CC551D001-1.0 29 41 Cloud provider Amazon EBS 43 Amazon S3 Storage charge (£/month/GB) Object 0.031 - - 0.078 - - - - - 0.095 Snapshots stored on S3: flat rate 44 Baremetalcloud (was NewServers) Cycle Computing CycleCloud Eduserv ElasticHosts Flexiscale GoGrid `HP Cloud - - Standard: tiered structure - - RRS: tiered structure - 0.006 Glacier archive storage: flat rate - 0.032 - 0.044 0.032 – 0.032 - 0.039 – 0.045 LRS: tiered structure 0.040 – 0.055 0.046 – 0.061 - 0.046 - 0.061 GRS: tiered structure 0.052 – 0.071 0.059 – 0.077 - 0.059 – 0.077 RA-GRS: tiered structure 0.054 -0.180 - 0.054 - 0.180 - - Tiered, lower range corresponds to 10TB target - - Tiered, lower range corresponds to 10TB target - Current offering is 36GB, 73GB, 146GB or 300GB. 0.091 – 0.245 Amazon EC2 use for example - - 0.300 Tier 1 (SAS or SATA and SSD hybrid storage): flat rate - - 0.175 Tier 2 (SATA storage with SSD metadata acceleration): flat rate - - 0.110 Tier 3 (SATA storage supporting archiving): flat rate - - 0.300 SSD: flat rate - - 0.060 HDD: flat rate - - 0.045 - Flat rate for disk storage, plus £0.018 per GB transferred 0.075 - - - Block storage: flat rate - - Cloud storage: tiered with first 10 GB/month free 0.063 – 0.074 0.053 0.025 - Persistent disk: flat rate - - - 0.078 Snapshot storage: flat rate 0.063 - - - Flat rate plus £0.062 per million IO requests - 0.056 - - Flat rate plus £0.006 per 10,000 object requests - - - 0.063 Snapshot stored in HP object storage: flat rate 0.094 NFS (NetApp) 0.040 – 0.054 - - Manta: tiered, this is the price per copy with the default being 2 copies but 1 – 6 copies possible, spread across 3 data centres in US-East - 0.063 - - First 1GB free, thereafter flat rate 0.370 - - SSD volume: flat rate, 100GB to 1 TB, up to 14 volumes - - Standard volume: flat rate, 100GB to 1TB,up to 14 volumes - - Cloud files: tiered structure - Terremark vCloud Express - At cost from cloud provider 0.090 SoftLayer (IBM SmartCloud) IOPS: flat rate plus £0.063/provisioned IOPS/month for 1GB-1TB 0.031 – 0.043 - Rackspace Standard: flat rate plus £0.031 per million IO requests for 1GB-1TB 0.038 – 0.053 Joyent Penguin - - - Google Compute Engine Other Azure Attached 42 disk Notes Block 0.058 – 0.070 - - - 0.070 Snapshot: flat rate - 0.063 - - Object storage: flat rate, in 20GB increments up to 300GB as additional instance drive 0.469 - - - SAN (iSCSI): flat rate, in 20GB increments - - 0.313 - NAS: flat rate, in 20GB increments - - 0.156 - Flat rate: up to 15 virtual disks (including system) up to 512GB each Table B-5: March 2014 storage costs for a range of commercial cloud providers 41 42 43 44 30 The first element of a range given for a tiered structure gives the price for 500TB/month, unless otherwise stated (eg, where 500TB of storage is not offered). The second element gives the price for lowest charged-for tier. Attached disk can be virtual or physical, depending on the offering. Standard S3 storage offers multiply redundant, highly available storage. Reduced Redundancy Storage (RRS) provides a lower level of redundancy. Glacier provides highly available archive and backup storage (see http://aws.amazon.com/s3/details/ [accessed 28 March 2014]). Azure offers Locally Redundant Storage (LRS), Geographically Redundant Storage (GRS) and Read-Access GRS (RA-GRS). Other refers to prices for NoSQL database storage. CC551D001-1.0 B.4 Data transfer B.4.1 Table B-6 shows indicative prices for data transfer to or from the Internet for a range of providers. The cloud providers use either a flat rate or a tiered structure for pricing data transfer. Low data transfer volumes (eg first 1GB) can be free. For a tiered structure, the range represents the prices 500TB of data transfer, unless specified otherwise, and the smallest charged transfer (eg the next GB transferred after the first free GB). Cloud provider Data transfer in from Internet (£/GB) Data transfer out to Internet (£/GB) Notes Amazon EC2 0.000 0.037 – 0.075 Outbound has tiered structure with first 1GB free, 0.037 applies to 500TB/month Azure 0.000 0.044 – 0.080 Charges for data transfer out are tiered with first 5GB out free. 0.044 applies to 500TB/month Baremetalcloud (was NewServers) 0.000 0.000 Free internap, first 3GB per hour of local bandwidth free Cycle Computing CycleCloud At cost from cloud infrastructure provider Amazon EC2 use for example Eduserv 0.000 0.105 Flat rate ElasticHosts 0.100 0.200 0.100 0.200 Committed in advance Burst above committed level Flexiscale 0.045 0.045 Flat rate GoGrid 0.000 0.050 – 0.075 Free private network data transfer between servers within a single data centre plus free inbound data transfer Outbound has tiered structure with first 1GB free Google Compute Engine 0.000 0.050 – 0.075 For egress to EMEA Tiered structure, 0.050 applies for more than 10TB/month HP Cloud 0.000 0.049 – 0.100 Tiered structure with first 1GB free, Low corresponds to 250TB/month Joyent 0.000 0.037 – 0.075 Tiered structure with first 1 GB out free.0.037 applies to 500TB/month Penguin 0.000 0.000 Covered by log in node charges (if any) Rackspace 0.000 0.048 - 0.080 Tiered structure, 0.048 applies to 500GB/month SoftLayer (IBM SmartCloud) 0.063 0.063 Flat rate Terremark vCloud Express 0.106 0.106 Flat rate Table B-6: March 2014 data transfer costs for a range of cloud providers B.5 Reserved and spot instances Reserved instances B.5.1 Reserved instances offer lower instance prices for an up-front charge and a reduced hourly charge. For comparison with the previous report, Table B-7 provides this information for the Amazon EC2 cc2.8xlarge instance (see Table B-3). This is in fact the same as for autumn 2011. CC551D001-1.0 31 Reserve type Up front (£) On-demand Cost per instance-hour (£) 0 1.500 Light usage 1 year 1,101 0.565 Light usage 3 years 1,694 0.565 Medium usage 1 year 2,591 0.338 Medium usage 3 years 3,986 0.338 Heavy usage 1 year 3,125 0.226 Heavy usage 3 years 4,794 0.226 Table B-7: March 2014 and reserve prices (£ per instance hour) for Amazon EC2 cc2.8xlarge instance Spot instances B.5.2 A snapshot of the spot prices for the Amazon EC2 compute categories (see Table B-3) is set out at Table B-8.45 This illustrates that use of spot instances can achieve savings of 70% – 80% over the on-demand prices, but at the risk of work being lost or delayed if the price changes suddenly. There does seem to be considerable variation possible in spot prices;46 on some occasions, the spot price has significantly exceeded the on-demand price. Provider HPC low Low High HPC high High memory large GPU Amazon EC2 - on-demand 0.047 0.038 0.070 0.047 0.068 0.051 Amazon EC2 – spot 0.005 0.005 0.010 0.010 0.007 0.013 Reduction (%) 88.7 88.0 86.3 78.6 89.5 73.7 Table B-8: Spot prices (£ per core-hour) for Amazon EC2 compute categories on 20 March 2014 B.6 Pricing models B.6.1 The different cloud providers have different approaches to charging for use that can lead to significant differences in the bill. Table B-9 outlines the key features of on-demand charging for each cloud provider considered. 45 This was calculated from the spot prices for US East on 20 March 2014. The prices are given on http://aws.amazon.com/ec2/purchasing-options/spot-instances/ [accessed 20 March 2014]. At the time of writing, http://awsspotprices.com/ provides a tracker for spot prices by Amazon EC2 instance type for the last 30 days [accessed 20 March 2014]. 46 32 CC551D001-1.0 Cloud provider Pricing approach For details see See billing topic on http://aws.amazon.com/ec2/faqs/ http://aws.amazon.com/ec2/pricing/ Amazon EC2 Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours There is no data transfer charge between two Amazon Web Services within the same region (eg US East) 47 Amazon EBS standard and IOPS volumes are charged per GB/month for provisioned storage until the storage is released Amazon EBS snapshots to S3 are charged per GB/month for data stored Amazon S3 Pricing is based on the average storage used throughout the month http://aws.amazon.com/s3/faqs/ Instance pricing is on a per minute basis http://www.windowsazure.com/enus/pricing/details/virtual-machines/ Storage capacity is billed in units of the average daily amount of data stored (in GB) over a monthly period http://www.windowsazure.com/enus/pricing/details/storage/ Baremetalcloud (was NewServers) Compute instances charged by the hour Each target includes 1 million IO requests per hour. Additional requests cost £0.063 per million Each server hour includes 3GB of internet transit. There is no charge for bursting up to full port speed (1Gbit/s) http://documentation.baremetalcloud.com/display/b mc/Billing Cycle Computing CycleCloud As per infrastructure provider Azure Charged on total usage per month Volume- and commitment-based discounts available Eduserv http://assetsproduction.govstore.service.gov.uk/G4/Eduserv0244/51f978b012a2fcb9e8000979/QD1/CloudComput e(IL0-2)_ServiceDefinition_Eduserv.pdf http://assetsproduction.govstore.service.gov.uk/G4/Eduserv0244/51f978b012a2fcb9e8000979/QD4/CloudComput e(IL0-2)_Pricing_Eduserv.pdf ElasticHosts Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours Storage charged based on virtual drive provisioned http://www.elastichosts.co.uk/pricing-information/ http://www.elastichosts.co.uk/?s=disk+pricing Resources are charged in Flexiscale units, which need to be bought in advance. Flexiscale unit use is charged for every hour the compute resource is available Storage charged based on virtual drive provisioned http://www.flexiscale.com/products/flexiscale/pricing/ Flexiscale Table B-9: Key features of the on-demand charging models for each cloud provider (part 1 of 2) 47 IOPS volumes included a specified number of IO operations per month. CC551D001-1.0 33 Cloud provider GoGrid Google Compute Engine HP Cloud Pricing approach https://wiki.gogrid.com/index.php/Billing_Model All machine types are charged a minimum of 10 minutes. After 10 minutes, instances are charged in 1-minute increments rounded up to the nearest 1 minute https://developers.google.com/compute/pricing#mac hinetype Persistent disks are charged for provisioned space per disk. IO operations are included in charges for provisioned space https://developers.google.com/compute/pricing#persi stentdisk Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours http://www.hpcloud.com/pricing Storage pricing is based on average amount of virtual storage used each month, the amount of data transferred, and the number of requests (Get, Put, Post, Copy, or List) made http://www.hpcloud.com/pricing#Block Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours http://wiki.joyent.com/wiki/display/jpc2/Provisioning+ a+Machine System storage is included in instance charges. Additional storage charged per GB/month on provisioned level Joyent Manta storage is charged per hour or part hour. The hourly rate is 1/730 of the GB/month charge Penguin Rackspace SoftLayer (IBM SmartCloud) Terremark vCloud Express For details see Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours Not found on website – call 1 (0) 855 456 9368 Compute instances charged by usage, metered down to 0.001 of a corehour https://pod.penguincomputing.com/about Storage is charged per GB/month, for allocated storage, tracked daily https://pod.penguincomputing.com/services Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours http://www.rackspace.co.uk/calculator Block storage is charged per GB/month for provisioned volumes http://www.rackspace.co.uk/cloud/blockstorage/pricing Cloud files are charged per GB/month on actual amount used http://www.rackspace.co.uk/cloud/files Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours Not found on website – call 1 (0) 866 398 7638 Storage charged per GB/month for provisioned volumes Compute instances charged by the hour for any provisioned resource. Partial instance-hours used are charged as full hours System storage is included in instance charges. Additional storage charged per GB/month on provisioned level https://community.vcloudexpress.terremark.com/enus/product_docs/w/wiki/am-i-billed-for-my-serverswhile-they-are-powered-off.aspx http://vcloudexpress.terremark.com/pricing.aspx Table B-9: Key features of the on-demand charging models for each cloud provider (part 2 of 2) 34 CC551D001-1.0 C.1 Introduction C.1.1 This Annex repeats the data and assumptions used to generate the autumn 2011 cloud prices. The data was sourced from cloud providers’ websites and converted to pounds sterling using an exchange rate of $1.60 to £1. C.2 Compute C.2.1 The compute prices in £/core-hour are provided at Table C-148 for the instances listed at Table C-2. These reflect the lowest commitment, so bulk discounting is not considered. The focus is on CPU rather than RAM, and storage and data transfer costs are excluded wherever possible. Four prices are presented for each cloud provider: C.2.2 – HPC Low and HPC High: these are designed to be appropriate for CPU-bound scientific problems. Differentiated on CPU where possible, otherwise by RAM. – Low: the lowest price per core-hour available. This excludes some “micro” type instances where the instance does not have an allocated virtual core proportion. – High: Aiming for a specification similar to Amazon’s erstwhile High CPU extra-large instance type (c1.xlarge). In addition, when a cloud provider offers HPC-specific nodes, this is highlighted. Provider HPC low Amazon EC2 0.047 0.053 0.053 0.10 Y US East Azure 0.075 0.075 0.075 0.075 N US East Baremetalcloud (was NewServers) 0.047 0.047 0.069 0.098 N Standard price 0.108 0.064 0.064 0.12 Y Example based on Amazon EC2 US East Eduserv 7.1×10–4 0.012 0.032 0.067 N UK ElasticHosts 0.026 0.026 0.04 0.037 N UK 0.033 0.022 0.055 0.055 N UK - priced in Flexiscale units GoGrid 0.12 0.11 0.12 0.12 N Standard price Google Compute Engine - - - - - - - - Joyent 0.053 0.053 0.175 0.10 N Standard price Penguin 0.193 0.068 0.193 0.19 Y Standard price N UK SoftLayer (IBM SmartCloud) 0.052 0.052 0.084 0.054 N Europe Terremark vCloud Express 0.008 0.008 0.1 0.15 N Standard price 49 Low High HPC high HPC nodes Cycle Computing CycleCloud Flexiscale HP Cloud Rackspace Data centre used for price Table C-1: Autumn 2011 prices (£/core-hour) for various compute categories 48 49 The Table is in the same order as the March 2014 analysis, with the same names. This has been amended from the previous report to reflect a change of view on hyperthreading for this instance and for a rounding/typographic error. CC551D001-1.0 35 Provider Amazon EC2 HPC Low Low High HPC High Cluster Compute 8 Extra Large US East cc2.8xlarge Calculated on the basis of 16 cores (ignoring hyper threading) Small instance, US East m1.small High CPU Extra Large US East c1.xlarge Cluster Compute Quad Extra Large US East cc1.4xlarge Azure All Azure core-hours have the same cost Baremetalcloud (was NewServers) Jumbo Jumbo 24 vCPU, 0.5GB RAM ElasticHosts 20 core GHz (~10 cores), 1GB RAM Flexiscale HPC Nodes Y N Small Fast N 1 vCPU, 0.5GB RAM 8 vCPU, 8GB RAM 24 vCPU, 48GB N 20000 core MHz (~10 cores), 1GB RAM 2000 core MHz, (~1 core), 1GB RAM 20 core GHz (~10 cores), 8GB RAM N 4 Cores, 8GB RAM 8 Cores, 0.5GB RAM 4 cores, 8GB RAM 8 Cores, 8GB RAM N All same price 0.5 core, 0.5GB RAM Google Compute Engine - - - - HP Cloud - - - - - Medium High CPU Small XXL 32GB High CPU XL 8GB N Compute node ($0.2/CH) + login node ($79.95/month = $0.11/hour) Just a login node Compute node ($0.2/CH) + login node ($79.95/month = $0.11/hour) Compute node ($0.2/CH) + login node ($79.95/month = $0.11/hour) Y - - - - - Gold 32bit SUSE Gold 32bit SUSE Copper 64bit SUSE Platinum 64bit SUSE N 8 Cores, 0.5GB RAM 8 Cores, 0.5GB RAM 8 Cores, 8GB RAM 8 Cores, 16GB RAM N Cycle Computing Eduserv GoGrid Joyent Penguin Rackspace SoftLayer (IBM SmartCloud) Terremark vCloud Express Amazon EC2 + 20% Y All same price N - Table C-2: Autumn 2011 instances used for cost comparison C.2.3 Figure C-1 presents graphically for each cloud provider considered the range of prices set out at Table C-1. The thin bars represent the range of prices (Low to High in Table C-1) that would be appropriate for CPU-bound tasks. The wide bars represent the range of prices for instances likely to be most suitable for CPU-bound tasks (HPC low to HPC high). These are typically larger instances with more virtual cores, and in the case of Amazon and Penguin, they represent distinct HPC nodes with superior characteristics. It is important to note that prices are not normalised by performance. C.2.4 The conclusion drawn from this analysis in autumn 2011 was that, while noting that researchers need to trade-off application performance with cost and choice of cloud instance, they could expect to pay up to 19p per core-hour for instances most suitable for CPU-bound tasks (with the average of the ranges being ~7p to ~10p per core-hour). 36 CC551D001-1.0 Cost per core-hour £0.20 £0.15 £0.10 £0.05 £0.00 Figure C-1: Range of cloud prices, not normalised for performance; thick bars indicate instances of most relevance for CPU-bound application (see paragraph C.2.3) C.3 Storage C.3.1 Table C-3 shows indicative storage costs for the cloud providers considered. 50 Cloud provider Storage charge (£/month/GB) Attached disk Notes Block Object Amazon EBS 0.060 - - - Standard: flat rate plus £0.031 per million IO requests for 1GB-1TB Amazon S3 - - - - - Azure 0.088 - - - - Baremetalcloud (was NewServers) - - - For a Storage Attached Network (SAN) volume - - 0.020 - For a dedicated 250GB drive Eduserv - - 0.067 ElasticHosts - - 0.060 Flexiscale - - GoGrid - 0.094 Google Compute Engine - - HP Cloud - - Joyent - 0.229 Penguin 0.000 Rackspace SoftLayer (IBM SmartCloud) Terremark vCloud Express 0.054 - 0.183 0.046- 0.055 Other Assumed Tier 2 equivalent HDD: flat rate - Flat rate for disk storage, plus £0.018 per GB transferred - - Cloud storage: tiered with first 10 GB/month free - - - - - - - - - 0.125 - - - - 0.110 - - Cloud files - 0.068 - - Object storage - - 0.156 - - 51 Table C-3: Autumn 2011 storage costs for a range of commercial cloud providers 50 51 The first element of a range given for a tiered structure gives the price for 500TB/month, unless otherwise stated (eg, where 500TB of storage is not offered). The second element gives the price for lowest charged-for tier. Storage charge is based on $0.012 per day and 30.5 days in a month. CC551D001-1.0 37 C.4 Data transfer C.4.1 Table C-4 shows indicative data transfer costs for the cloud providers considered. Cloud provider Data transfer in from internet (£/GB) Data transfer out to internet (£/GB) Amazon EC2 0.000 0.03 - 0.07 Azure 0.000 0.09375 - 0.125 Baremetalcloud (was NewServers) 0.0625 0.0625 Cycle Computing CycleCloud At cost from cloud provider Eduserv 0.000 0.000 ElasticHosts 0.100 - 0.200 0.100- 0.200 Flexiscale 0.018 – 0.022 0.018 – 0.022 GoGrid 0.181 0.181 Google Compute Engine - - HP Cloud - - Joyent 0.080 0.080 Penguin 0.000 0.000 Rackspace 0.000 0.120 SoftLayer (IBM SmartCloud) 0.050 - 0.094 0.050 - 0.094 Terremark vCloud Express 0.106 0.106 Notes Plus costs to transfer between availability zones for outbound Amazon EC2 use for example 2 units per month per GB Free 20TB internet transfer per month, then 0.08 Table C-4: Autumn 2011 data transfer costs for a range of cloud providers 38 CC551D001-1.0