Windows Azure Architecture, Services and Programming Model Ilian Iliev Telerik Corp. www.telerik.com What is Windows Azure? Platform Overview and Architecture The Microsoft Cloud ~100 Globally Distributed Data Centers Quincy, WA Chicago, IL San Antonio, TX Dublin, Ireland Generation 4 DCs 3 The Microsoft Cloud Highly available, scalable, and consistent application fabric environment 4 How Your Ideas are Shipped without the Cloud? Lets say you have an idea? What do you need to manage to ship your idea to potential customers? Managing environment Network Hard drive failure Routers Installing OS Load balancing Storage Memory Updating Patching Server acquisition 5 How Windows Azure solves your problems? Windows Azure runs your application code Inside virtual machines Managed by the fabric controller 6 The Microsoft Cloud Categories of Services Application Services Software Services Platform Services Infrastructure Services 7 Windows Azure Platform Application Services Frameworks Secure Token Service Security Connectivity Relational Database Data Personal Data Repository Application Marketplace Information Marketplace Services Hosting Workflow Hosting Distributed Cache Declarative Policies Claims-Based Identity Federated Identities Service Bus Composite Applications On-Premise Bridging ADO.NET, ODBC, PHP Transact-SQL Synchronization C / C++ Compute Storage Data Win32 Unstructured Data Blobs Message Queues Distributed File System VHD Content Distribution 8 Windows Azure Platform Application Services “Dublin” Frameworks “Velocity” Security Access Control “Geneva” Connectivity Service Bus Project “Sydney” SQL Azure Data Sync Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 9 Windows Azure Compute Development, service hosting, & management environment Application Services .NET, Java PHP, Python, Ruby, native code (C/C++, Win32, etc.) “Dublin” “Velocity” Frameworks ASP.NET providers, FastCGI, memcached, Access Control “Geneva” Security MySQL, Tomcat Full-trust – supports standard languages and APIs Project Service Bus Connectivity Secure certificate store “Sydney” Management API’s, and logging and diagnostics SQL Azure Data Data Sync systems Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 10 Windows Azure Compute(2) Multiple roles – Web, Worker, Virtual Machine (VHD) Application Services Multiple VM sizes 1.6 GHz CPU x64, 1.75GB RAM, 100Mbps network, “Dublin” “Velocity” Frameworks 250GB volatile storage Security Small (1X), Medium (2X),Access LargeControl (4X), X-Large (8X)“Geneva” In-place rolling upgrades, organized Service Bus by upgrade domains Connectivity Data Walk each upgrade domain one at a time Project “Sydney” SQL Azure Data Sync Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 11 Windows Azure Diagnostics Configurable trace, performance counter, Windows Application Services event log, IIS log & file buffering Local data buffering quota management “Dublin” Frameworks “Velocity” Query & modify from the cloud and from the Access Control “Geneva” Security desktop per role instance Project Service Bus & on-demand“Sydney” Connectivity Transfer to storage scheduled Data Filter by data type, verbosity & time range SQL Azure Data Sync Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 12 Windows Azure Storage Application Services Rich data abstractions – tables, blobs, queues, drives, CDN “Dublin” “Velocity” Frameworks Capacity (100TB), throughput (100MB/sec), transactions (1K req/sec) Access Control “Geneva” Security High accessibility Supports geo-location Project Service Bus Connectivity “Sydney” Language & platform agnostic REST APIs URL: http://<account>.<store>.core.windows.net SQL Azure Data Client libraries for .NET, Java, PHP, etc. Data Sync Compute Storage 13 Windows Azure Table Storage Application Services Designed for structured data, not relational data Data definition is part of the application “Dublin” “Velocity” Frameworks A Table is a set of Entities (records) An Entity is a set of Properties (fields) Access Control “Geneva” Security No fixed schema Each property is stored as a <name, typed Project Service Bus Connectivityvalue> pair “Sydney” Two entities within the same table can have SQL Azure Data different properties Data Sync No schema is enforced Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 14 Windows Azure Blob Storage Application Services Storage for large, named files plus their metadata Block Blob “Dublin” “Velocity” Frameworks Targeted at streaming workloads Each blob consists of a sequence of blocks a Block ID Control “Geneva” Security Each block is identified byAccess Size limit 200GB per blob Page Blob Project Serviceworkloads Bus Connectivity Targeted at random read/write “Sydney” Each blob consists of an array of pages Each page is identified by its offset from the start of SQL Azure Data Data Sync the blob Size limit 1TB per blob Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 15 Windows Azure Queue Application Services Performance efficient, highly available and provide reliable message delivery “Dublin” Frameworks Asynchronous work dispatch “Velocity” Inter-role communication AccessFIFO Control Security Polling based model; best-effort data structure Queue operations Connectivity Create Queue Data > Message operations Service Bus Message • Add Delete Queue • Get Message(s) List Queues • Peek Message(s) Get/Set Queue Metadata • Delete Message “Geneva” Project “Sydney” SQL Azure Data Sync Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 16 Windows Azure Drive Application Services Provides a durable NTFS volume for Windows Azure applications to use “Dublin”drive “Velocity” Frameworks Use existing NTFS APIs to access a durable Durability and survival of data on application failover Access Control “Geneva” Security Enables migrating existing NTFS applications to Project Service Bus Connectivitythe cloud “Sydney” Drives can be up to 1TB; a VM can dynamically mount up to 8 drives SQL Azure Data Data Sync Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 17 Windows Azure Drive(2) Application Services A Windows Azure Drive is a Page Blob “Dublin” Frameworks Example, mount Page Blob as X:\ Security “Velocity” http://<account>.blob.core.windows.net/< Access Control “Geneva” container>/<blob> Project Service Busdurable to the“Sydney” Connectivity All writes to drive are made Page Blob Data Drive made durable through standard Page Blob replication Compute Storage Table Storage Blob Storage Queue Drive SQL Azure Data Sync Content Delivery Network 18 Windows Azure Content Delivery Network Application Services Provides high-bandwidth global blob content delivery 18 locations globally (US, Europe, Asia, Australia and “Dublin” “Velocity” FrameworksSouth America), and growing Blob service URL vs. CDN URL Access Control “Geneva” Security Blob URL: http://<account>.blob.core.windows.net/ CDN URL: http://<guid>.vo.msecnd.net/ Support for custom domain names Project Service Bus Connectivity Access details “Sydney” Blobs are cached in CDN until the TTL passes SQL Azure Use per-blob HTTP Cache-Control policy for TTL (new) Data Data Sync CDN provides only anonymous HTTP access Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 19 SQL Azure Highly available, scalable, and consistent distributed relational database; geo-replication and geo-location of data Application Services Relational database, provided as a service Highly symmetrical development and tooling experience “Dublin” “Velocity” Frameworks (use TDS protocol and T-SQL) Highly scaled out, on commodity hardware foundation Access Control “Geneva” Security Built on the SQL Server technology Editions: Web (1GB), Business (10GB) Project “Sydney” Service Bus Connectivity SQL Azure Data Sync Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 20 SQL Azure Data Sync Application Services Frameworks Security Tools for data synchronization with SQL Azure “Velocity” Connect on-premises apps“Dublin” with the cloud Create cached-mode clients Built on Microsoft Sync Framework and “Geneva” Access Control ADO.NET Sync Services Project “Sydney” Service Bus Connectivity SQL Azure Data Sync Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 21 AppFabric Service Bus Securely connect applications Over the internet Across any network topology Across organizational “Dublin” “Velocity” boundaries Primary application patterns Eventing – notify applications Access Control “Geneva” and/or devices Service Remoting – securely Project services Service Busproject on-premises“Sydney” out to the cloud Tunneling – app-to-app SQL Azure communication with Data Sync NAT/Firewall traversal Application Services Frameworks Security Connectivity Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 22 AppFabric Access Control Provides outsourcing of claimsbased access control for REST web services Key capabilities: “Dublin” Usable from any “Velocity” platform Low friction way to onboard new clients Access Control “Geneva” Integrates with ADFS v2 Supports OAuth WRAP Project / Service BusSWT “Sydney” Enables simple delegation SQL Azure Used today by Service Bus and Data Sync “Dallas” Application Services Frameworks Security Connectivity Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 23 “Geneva” > > Next generation identity and access management platform Application Services • Claims-based access and single sign-on for onpremise and cloud-based applications in the “Dublin” Frameworks enterprise, across organizations, and on the Web “Velocity” “Geneva Access Foundation) Control Security Framework“ (Windows Identity “Geneva” • Provides .NET development tools, which includes pre-built, user-access logic that Service externalizes Bus Connectivity authentication from applications. It helps developers build claims-aware .NET applications, Data plus build custom security token services (STS) > Project “Sydney” SQL Azure Data Sync “Geneva Server“ Compute • An STS for IT that issues and transforms claims and other tokens, manages user access and Table Storage and Blob Storage Queue Drive Storage enables federation access management for simplified single sign-on Content Delivery Network 24 “Geneva” “CardSpace Geneva” Application Services helping users navigate between multiple logons for simplified single sign-on Frameworks “Dublin” providing complete user control and transparency for how personal information is Access Control Security shared “Geneva” Project “Sydney” Service Bus Connectivity “Velocity” SQL Azure Data Sync Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 25 “Velocity” Distributed in-memory cache platform for Application Services developing scalable, available, and highlyperformance applications Frameworks “Dublin” Security Caches any serializable CLRAccess object and Control provides access through simple cache APIs Connectivity Supports enterprise scale: tens Serviceto Bus hundreds of computers Data Configurable to run as a service accessed over the network Compute Supports dynamic scaling and automatic load balancing Storage Table Storage Blob Storage Queue Drive “Velocity” “Geneva” Project “Sydney” SQL Azure Data Sync Content Delivery Network 26 “Velocity” Supports continuous availability of cached data Application by storingServices copies on separate cache hosts Integration with administration and monitoring “Dublin” Frameworks tools such as Event Tracing for Windows (ETW) and System Center. “Velocity” Security Access Control “Geneva” Connectivity Service Bus Project “Sydney” SQL Azure Data Sync Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 27 “Dublin” Evolution of WAS/IIS and Windows App Application Services Server role to run and manage WF and WCF services Frameworks Enhances the hosting of Windows Communication Foundation (WCF) and Windows Workflow (WF) inAccess the Control Security Windows Process Activation Service (WAS) Service Bus Connectivity Simplified deployment and management of WCF and WF Data services hosted in WAS Compute Configuration of persistence for durable workflows Storage Table Storage Blob Storage Queue “Dublin” “Velocity” “Geneva” Project “Sydney” SQL Azure Data Sync Drive Content Delivery Network 28 “Dublin” Customizable tracking profiles with Application Services dedicated, queryable storage for tracked data Frameworks Customizable monitoring of hosted services “Dublin” “Velocity” Security Access Control “Geneva” Connectivity Service Bus Project “Sydney” SQL Azure Data Sync Data Compute Storage Table Storage Blob Storage Queue Drive Content Delivery Network 29 “Dallas” Application Services Frameworks “Dublin” Information-as-a-service Discover, acquire, and consume structured, blob, Access Security and real-time data to power anyControl application – on any platform and any screen size Service Bus Connectivity Brokerage Business – partner driven ecosystem and global reach to deliver data and functionality to Data developers and information workers Analytics and Reporting – single click analysis to augment private data with public data Compute Built on Windows Azure and SQL Azure Storage Table Storage Blob Storage Queue Drive “Velocity” “Geneva” Project “Sydney” SQL Azure Data Sync Content Delivery Network 30 Azure Pricing Computing Nodes Shared CPU, 768 MB RAM $0.02 / hour ($15 / month) 1 Core, 1.75 GB RAM $0.12 / hour ($90 / month) (as of April 2012) Storage $0.125 / GB + $0.01 / 10000 operations SQL Azure Database 100 MB – $0.0067 / hour ($5 / month) 1 GB – $0.0133 / hour ($10 / month) 31 AppFabric Service Bus 32 AppFabric Access Control 33 Table Storage Concepts Accounts Tables Entities Email = … Name = … Users Email = … Name = … moviesonline Movies Genre = … Title = … Genre = … Title = … 34 Queue Storage Concepts Accounts Queues Messages 128 x 128 http://... thumbnailjobs 256 x 256 http://... sally http://... traverselinks http://... 35 Blob Storage Concepts Accounts Container Blob PIC01.JPG images PIC02.JPG sally movies MOV1.AVI 36 Blob Storage Concepts Adding Blocks and Pages Accounts Container Blob Blocks PIC01.JPG images PIC02.JPG sally movies MOV1.AVI Block or Page 1 Block or Page 1 Block or Page 1 37 Windows Azure Questions? http://clouddevcourse.telerik.com Homework 1. Install Windows Azure SDK for your favorite programming language E.g. C#, Java, PHP or JavaScript 2. Create a "Hello World" Azure application Run it in the Azure Compute emulator 3. Create an application for uploading, listing and downloading files from Azure Blobs Run it in the Azure Storage emulator As a result of your homework submit the source code of your Azure applications. 39