Real Time BI with
Windows Azure
Windows Azure Conference 2014
Agenda
 Why Cloud BI
 Change Data Capture – A BI Scenario
 Cloud BI Self Service Overview
Windows Azure Conference 2014
Why move to the Cloud ?

Reduced Organizational IT costs

Built-in scalability

High Availability

Maximum “future proofing”
This applies to Business Intelligence Solutions too

Addresses several key concerns in DW / BI hardware estimates

Reduces significant hardware costs and associated planning effort

Potentially shortens implementation time
Windows Azure Conference 2014
Common Cloud Services Model
 Infrastructure as a Service (IaaS)
 Platform as a Service (Paas)
 Software as a Service (SaaS)
Windows Azure Conference 2014
Business Intelligence in the Cloud
Public Cloud Delivery Options
IaaS
PaaS
SaaS
Hosted Virtual
Server DB
Build Your Own
Platform
BI as a “service”
apps
• Deploy your own
• Cloud based platform services
Solution in the Cloud
• Low cost, pay per use
• Low upfront costs
• Allows / requires custom
(licensing of VM,
development to some extent
Software etc)
• No cloud based horizontal
Scaling; must be done in DB and
Application architecture
Windows Azure Conference 2014
• Pre Packaged BI Solutions
• “Easy” management and
Implementation
• No design flexibility
• Custom apps are off the table
Business Intelligence in the Cloud
Other delivery options
Private
Cloud
•
•
•
•
Offers the most control but also Greater cost
Design your own solution
Manage your own cloud
An option if your organization
Owns and maintains a private cloud
Windows Azure Conference 2014
Hybrid
Cloud
• Combination of Private Cloud and Public Cloud
• Leverage the public for “spikes” in either availability
Or Processing
• More complicated solution to accomplish load balance
BI Lifecycle
Data Sources
Data Preparation
Data Storage
Data Analysis
Data Access
Extract
OLTP Database
Transform
Reports
Load
OLAP
- Tablix
- Drill
- Visualisations
Data Mart
ERP Database
Graphs
Filter
Departmental
Database
Data Warehouse
Cleanse
Schedule
Windows Azure Conference 2014
In-Memory
Analytics
- Charts
- Ad-hoc
- Dynamic
Dashboard
Data Mining
- Mash-up
- Collaborate
- Search
Real-time BI Platform - CDC Overview
Change-Data-Capture is a mechanism to trickle feed changes in
data from a source database to multiple destinations in near realtime.
Approaches such as Replication (which are trigger based) can induce
performance overhead on source systems and be intrusive in
implementation.
Change-Data-Capture (which is log based) mitigates the above by
capturing changes directly from source system logs and
propagating the change delta.
Windows Azure Conference 2014
Setting up CDC
• SELECT [name], database_id, is_cdc_enabled FROM sys.databases
• EXEC sys.sp_cdc_enable_db
• EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name = N'TestTable1',
@role_name = NULL
GO
Windows Azure Conference 2014
Creates CDC Schema
Creates CDC System Tables
Creates CDC Jobs
Creates CDC Functions
Enabling CDC
Returns list of captured columns
Contains indexes
associated with
Change tables
Returns list of tables
which are enabled for
CDC
Changes of all DDLs
after CDC is enabled
Windows Azure Conference 2014
What if
http://blogs.msdn.com/b/mattm/archive/2012/03/26/cdc-fororacle-in-sql-server-2012.aspx
Windows Azure Conference 2014
CDC Concept Architecture
SQL Server 2012 has built-in CDC for Oracle source
Oracle
1.
ACTIVE Archive
REDO Log File
A_1
:
:
2.
Log
Miner
SQL Server
CDC Service
3.
4.
INACTIVE
Archive REDO
Log File A_n
SQL Server CDC
Staging
Database
5.
CDC SSIS
Package
6.
Windows Azure Conference 2014
SQL Server / DB 2
/ Oracle / PDW
Value Proposition
Technology Benefit
•
•
Low-overhead, granular (column level) change propagation in real-time
No installation, temporary staging or manual customization on source system
Implications
•
•
Elimination of delay resulting from DW refresh in batch cycles
Improved ROI on data analytics/BI investments due to greater relevance,
accuracy and completeness of information
Business Benefit
•
Competitive differentiation through real-time business visibility and improved
decision-making agility
Windows Azure Conference 2014
Real-time BI Concept Architectures – Azure & On-Premise
On-Premise
Oracle
On-Premise
SQL Server CDC
(OLTP)
On-Premise
Oracle
(OLTP)
SQL Server
DW / PDW /
DB2 DW etc.
Azure
SQL Server CDC
Windows Azure Conference 2014
Oracle
(OLTP)
Azure
SQL Server CDC
SQL
Server DW
Azure
SQL Server
DW
Oracle
(OLTP)
SQL Server CDC
SQL
Server DW
14
Oracle OLTP -> DB2 DW Case-Study
Punjab National Bank with more than 119 years of strong existence and over 6000 branches including 5 overseas branches,
6460 ATMs, 5047 Business Correspondents and 2165 Ultra Small Branches, serves more than 82 million esteemed customers.
Business Requirements
Quantified Results
 End-of-day bulk upload from Customer:
Online Payments System
 Up-to-the minute refresh of EDW
(OPS) to Enterprise Data
resulting in timely decision
Warehouse (EDW) took 6
making and analytics
hours
 Reduction in labor cost due to
 Minimum 6 hour delay in
elimination of extended work
reconciliation of actual cashhours
flow position in EDW thus
resulting in delayed decision Microsoft:
 $500K+ SQL License Revenue
making
 IT personnel have to work till
mid-night to refresh EDW
thus incurring additional cost
and inconvenience
Windows Azure Conference 2014
Microsoft Team
Vivek Ravindra (STU Lead)
Raghavendra Prabhu (PPS)
Sandeep Katyal (SSP)
Prajyot Saran (TSP)
Himanshu Gupta (TSP)
Arun Sharma (HCL PAM)
Udaya Shankar (IMDM)
Rakesh Sinha (IMDM)
N. Raja (DCM)
Sanjeev Sharma (CATM)
A powerful new way to work with data
Self-service business intelligence with familiar Excel and the power of the cloud
Windows Azure Conference 2014
Power BI components
•
Power Query
• Easily search and access public and organizational data within excel
Windows Azure Conference 2014
Power BI components
•
Power Pivot
• Tools for creating relationships
• hierarchies
• Calculated Measures
• KPIs
•
Power View
– Tools for creating interactive charts, graphs and other visual
representatives of data
Windows Azure Conference 2014
Power BI components
•
Power Map
• 3D Visualization tool for mapping, exploring and interacting with geographic data
Windows Azure Conference 2014
Share & Collaborate with Power BI for
Office 365
Windows Azure Conference 2014
Power BI Sites
•
•
•
•
SharePoint Online sharing of Power BI content via new site type
Reports can be set to refresh data on schedule or demand
Power BI sites can be browsed from mobile device or laptop
Add reports to mobile favorites to view on tablet or smartphone
Windows Azure Conference 2014
To conclude
Not all business intelligence needs are the same; if they
were, this world would be a very boring place. With
Microsoft's BI offerings in Azure, you have the flexibility to
create the BI environment that best fits your needs.
Whether you need traditional SQL Server Reporting
Services, PowerPivot, or full-blown Power View, Azure
provides the building blocks for your solution.
Windows Azure Conference 2014
Customer Interaction Zone (CiZ)
Any question on Anything Azure!
Book your slot now! @ the
registration desk at the CiZ
Windows Azure Conference 2014
Windows Azure Conference 2014
Windows Azure Conference 2014
Windows Azure Conference 2014
Competing against Oracle Golden-Gate
Cost
•
•
•
Risk
•
•
•
Golden-gate is an Oracle DB option, that needs to be licensed for both Source and Target
What really hurts customers is that they need to pay Oracle even when the Source is Oracle DB!
SQL Server CDC (or our partner solution from Attunity) is not licensed by Source or Target
Golden-gate requires installation of services or temporary tables or temporary files on both Source and
Target
For mission-critical or sensitive environments, customers don’t want to risk installing anything on the
Production server
SQL Server CDC (or our partner solution from Attunity) does not require installation of services or
temporary tables or temporary files on either the Source or the Target
Ease-of-Use
•
SQL Server CDC can be set-up with a few clicks
•
Using SSIS CDC components, processing of change-data can be automated to refresh OLAP cubes
Windows
Azure
Conference 2014
EPG | MICROSOFT
CONFIDENTIAL
25