From Numbers And Data To Insights And Value Microsoft Solutions for Big Data | Oct 17th 2013 Roger Breu SQL Server PDW Solution Sales Microsoft Western Europe roger.breu@microsoft.com The world of data has changed 10x 4.3 increase every five years connected devices per adult 85% from new data types Data explosion Consumerization of IT 27% using social media input The Large Hadron Collider produces 1 PB/sec But, my customer isn’t CERN and doesn’t have a Large Hadron Collider… But your customers do have… Sensors Clicks Logs Transactional records Call centers Images Documents Signals from social media Simulations A Definition Traditional Data is Highly Structured traditional databases are organized around planned queries Big Data is All This and More: volume, variety, velocity (and volatility & variability) Common Big Data Customer Scenarios IT infrastructure optimization Legal discovery Social network analysis Traffic flow optimization Web app optimization GAIN COMPETITIVE ADVANTAGE BY MOVING FIRST AND FAST IN THEIR INDUSTRY Churn analysis Fraud detection Life sciences research Natural resource exploration Weather forecasting Healthcare outcomes Advertising analysis Equipment monitoring Smart meter monitoring Twitter Analytics Demo with Microsoft Excel Demo Is this Big Data? Is this Enterprise Ready? Big Data is more than just new BI! Big Data is more than just Hadoop! The Big (Data) Picture, 4 main topics: Alerts, Notifications SQL Server StreamInsight Big Data Sources (Raw, Unstructured) Data & Compute Intensive Application SQL Server FTDW Data Marts Load Sensors Summarize & Load SQL Server Reporting Services Fast Devices HDInsight on Windows Azure Bots Business Insights HDInsight on Windows Server SQL Server Parallel Data Warehouse Historical Data (Beyond Active Window) Interactive Reports Integrate/Enrich SQL Server Analysis Server Crawlers Performance Scorecards Azure Market Place Enterprise ETL with SSIS, DQS, MDS ERP CRM LOB Source Systems APPS The Traditional Data Warehouse … data warehousing has reached the most significant tipping point since its inception. The biggest, possibly most elaborate data management system in IT is changing. – Gartner, “The State of Data Warehousing in 2012” Data sources 14 The Traditional Data Warehouse 2 1 Increasing data volumes 3 Real-time data New data sources & types Data sources Non-Relational Data 4 Cloud-born data 15 The Modern Data Warehouse Data sources Non-Relational Data Combining Structured and Semistructured Data with SQL Server PDW and Polybase Demo Combining Structured and Semistructured Data with SQL Server PDW and Polybase Demo And remember, it’s not just working with Twitter data DATA SYSTEMS APPLICATIONS Socket for Big Data: Hadoop Access Made Simple SQL Server PDW Microsoft Applications HDInsight CREATE TABLE Customers ([user_id] INT, name NVARCHAR(50)… DATA SOURCES CREATE EXTERNAL TABLE ClickEvent (url varchar(50), event_date date, user_ID varchar(50) ) WITH LOCATION =‘hdfs://MyHadoop:5000/clickstream/click.txt’; SELECT COUNT(*) FROM Customers cNew Sources Traditional Sources JOIN ClickEvent e ON (web logs, email, sensor data, (RDBMS, OLTP, OLAP) c.[user_ID] = e.[user_ID] social media) WHERE c.Name = 'Jones'; Polybase Feature in SQL PDW Pillars of Microsoft’s Big Data Strategy Simplicity & Manageability Combine internal and external data Broaden accessibility of Big Data Analytics to all users Develop once, deploy onpremises or in the cloud Whats in it for you as Partner? Data Platform market is estimated to grow 9% YoY Business Intelligence market growing 16.4% YoY Data Warehousing growing 13.5% YoY Big Data market estimated to grow 39.4% YoY Source: IDC Software Consulting Hardware Whats in it for you as Partner? Leverage pre-engineered Data Warehouse and Hadoop Appliance solutions from Microsoft (and it’s hardware partners) • De-risk your projects • Streamline deployments • Modernize your customers Data Warehouse/BI Environments >> Make them Big Data Ready! Leverage and connect the cloud • Streamline deployments • Decommission fast if smaller hadoop (HDInsight) cluster is sufficient Thank you © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentations. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. Additional Resources • SQL Server Parallel Data Warehouse (PDW) Landing Page: • www.microsoft.com/PDW • Microsoft Big Data Landing Page: • www.microsoft.com/bigdata • Introduction to Polybase: • http://www.microsoft.com/en-us/sqlserver/solutions-technologies/datawarehousing/polybase.aspx • Twitter Analytics for Excel: • http://extendedresults.com/products/twitteranalytics/