SQL Server 2008 R2
Parallel Data Warehouse
Steve Busby
Data Warehouse Technology Specialist
Microsoft Corporation
Steve.Busby@microsoft.com
Facebook: msstevebus
Twitter: @BamaSteveB
About Steve
 14 year MSFT veteran
 MCS, then MTC
 Appdev/SQL background (OLTP)
 But Infrastructure too
 Just moved back “home”
 Graysville, Minor High (‘89)
 BSEE – UAB
 Former Southern Company Services
Some caveats
 This presentation is about V1 of PDW
 V2 is releasing December’ish
 New HW and SW
 will be launched at SQLPASS
 Can’t talk publicly about V2 (NDA only)
 Significant performance increases and cost decreases
 Incentive to buy now though (procs vs cores)
One more caveat!
•
I’m not new to Microsoft,
but new to role (and I’m
rusty on DW)
•
You guys get to be my lab
rats 
MANAGE ANY DATA, ANY SIZE, ANYWHERE
010101010101010101
1010101010101010
01010101010101
101010101010
A Complete Business Analytic Platform
Integration with SQL
Server BI
Complementary
tools
Non-Microsoft BI &
ETL Tools
SQL Server Integration
Services, Analysis
Services
PowerPivot & Power
View
Hadoop
StreamInsight
Master Data Services
Informatica
Business Objects
Microstrategy and SAS
6
Microsoft & Data Warehouse
Alerts, Notifications
SQL Server
StreamInsight
Big Data
Sources (Raw,
Unstructured)
Data & Compute
Intensive Application
Business
Insights
SQL Server FTDW
Data Marts
Sensors
Load
Summarize &
Load
Devices
Fast
SQL Server
Reporting Services
Hadoop on
Windows Azure
Bots
Hadoop on
Windows Server
Interactive
Reports
SQL Server Parallel
Data Warehouse
Historical Data
(Beyond Active Window)
Integrate/Enrich
SQL Server Analysis
Server
Crawlers
Performance
Scorecards
Enterprise ETL with
SSIS, DQS, MDS
ERP
CR
M
LOB
Source Systems
APPS
MICROSOFT DATA WAREHOUSING SOLUTIONS
Reference Architectures
offering best price
performance
Appliance for Data
Warehousing requiring
highest scalability,
performance or
complexity
Ideal for data marts or small
to mid-sized DWs with scan
centric workloads
Offers flexibility in
hardware and architecture
Software only
Reference Architectures
(Software and Hardware)
DW Appliance
(Fully integrated Software
and Hardware)
Scale-Up DW
Scale-Up DW
Scale-Out DW with MPP
Scalable and reliable
platform
Ideal for data marts or small
to mid-sized EDWs
Tier 1 Services and Support
Some SQL Data Warehouses today
Big SAN
Big 64-core Server
Connected together
What’s wrong with this picture?
Answer: system out of balance
• This server can consume 16 GB/Sec of IO, but the
SAN can only deliver 2 GB/Sec
− Even when the SAN is dedicated to the SQL Data
Warehouse, which it often isn’t
• System is typically IO bound
• Queries are slow
Result: significant investment, not delivering performance
The Alternative: A Balanced System
• Design a server + storage configuration that can
deliver all the IO bandwidth that CPUs can
consume when executing a SQL Relational DW
workload
• Avoid sharing storage devices among servers
Fast Track Data Warehouse Components
Balanced across all components
CPU Feed Rate
FC
HBA
A
B
FC
HBA
SQL Server
Read Ahead Rate
A
B
A
B
DISK
HBA Port Rate
DISK
A
A
B
STORAGE
CONTROLLER
CACHE
A
B
FC SWITCH
CPU CORES
WINDOWS
SQL SERVER
CACHE
SERVER
SQL Server 2008 R2 Potential Performance Bottlenecks
LUN
A
B
DISK
DISK
B
LUN
Switch Port Rate
SP Port Rate
LUN Read Rate
Disk Feed Rate
Two SQL DW Infrastructure Options:
SQL Classic DW or Fast Track SQL DW
SQL Classic DW
Architecture
Leverages Shared SAN
Enterprise Shared
SAN Storage
Fast Track SQL DW Architecture
Architecture modeled after DW Appliances
“ Appliance Like” solutions
Uses Dedicated SAN arrays and Network
Shared Network
Bandwidth
Dedicated
Network
Bandwidth
Dedicated SAN
SQL 2008 Data Warehouse
SMP Server
OLTP Applications
SQL Fast Track DW supports “Scan Centric” DW
workloads that are index light
Balanced System Scale UP with Fast Track v3
Peak Scan rates 14GBs/sec – Proliant DL980
CPU
Socket
(8 Core)
CPU
Socket
(8 Core)
CPU
Socket
(8 Core)
CPU
Socket
(8 Core)
CPU
Socket
(8 Core)
CPU
Socket
(8 Core)
CPU
Socket
(8 Core)
Fiber
Switch
Storage Processor
Storage Processor
Storage Enclosure
Storage Processor
Dual
8GB
Ports
X4 per
P2000
CPU
Socket
(8 Core)
Storage Processor
Storage Processor
Storage Processor
Storage Processor
Storage Processor
Storage Processor
Storage Processor
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Enclosure
HBA
Storage Processor
Storage Processor
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Enclosure
HBA
HBA
RAID-1
RAID-1
Storage Enclosure
HBA
Server
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Enclosure
Storage Processor
HBA
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Enclosure
HBA
HBA
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Enclosure
Storage Processor
HBA
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Processor
Storage Processor
RAID-1
RAID-1
RAID-1
RAID-1
RAID-1
Storage Enclosure
SQL Server Parallel Data Warehouse
A data warehouse appliance with massive scalability
What do you do when
the combination of
your analytic workload
and data volumes
exceed the capabilities
of SQL Server 2008 R2
and Fast Track
Reference Architecture?
PDW - Massive Scale and Performance
Complete Business
Analytics Platform
Massive Scale &
Performance
Simplicity & Value
Benefits
16
Parallel Data Warehouse
An appliance experience
 All hardware from a single vendor
 Multiple vendors to chose from
 Orderable at the rack level
 Vendor will:
 Assemble appliances
 Image appliances with OS, SQL Server, and
PDW software
 Appliance installed in 1 – 2 days
 Support:
 Microsoft provides first call support
 Hardware partner provides onsite break/fix
support
Linear Scalability
1 Data Rack
•
•
•
•
10 Servers
20 Procs
240 Cores
1.28TB RAM
Control Rack
DataRack
Expand to 4 data racks and
quadruple your performance or capacity!
Dell/Microsoft PDW Architecture
Scales for Resilience and High Performance, with Low Cost of Entry
Control Rack
Data Racks (up to 4)
PowerVault MD3620f
Storage Nodes
PowerEdge R610
Database Servers
Control Nodes (R710)
Active / Passive
Data Center
Monitoring
Dual Infiniband
Management Servers (R610)
Dual Fibre Channel
Client Drivers
Landing Zone (R510)
ETL Load Interface
Backup Node (R710 and
MD3600f w/MD1200’s)
Corporate Backup
Solution
Spare Database Server
Corporate Network
Private Network
PDW 1 Data Rack Appliance
Control Rack
Data Rack
Compute Nodes
Designed Specifically for
DW Workloads
Control Nodes
Storage Nodes
SQL
Active / Passive
SQL
SQL
SQL
True MPP shared
nothing architecture
SQL
Linear scalability by
adding additional data
racks.
Landing Zone
Dual Infiniband
SQL
SQL
SQL
SQL
High Availability
Built in
Backup Node
Dual Fiber Channel
Management Servers
SQL
SQL
Spare Compute Node
Control Rack– Designated Server Workloads
Control Rack An Appliance = 1 Control Rack and 1 to 4 Data Racks
Control Nodes
Active / Passive
Control Node
Takes submitted statement and creates
D-SQL plan
SQL
Landing Zone
Backup Node
Dual Infiniband
Management Servers
Management Server
Patch Management, Active Directory,
HPC monitoring
Landing Zone
Parallelizes loads. Tested to 1-2 TB an
hour. LZ has 6.5 TB storage
Backup Node
Has enough storage to match data
configuration.
Backup Speeds 4-6 TB an hour
Power of 10 Compute Nodes
Data Rack
Compute Nodes
Each Compute Node complete
independent SQL server.
True MPP architecture.
24 cores, 128GB Ram, per node
Storage Nodes
SQL
SQL
SQL
Data Rack:
1280 GB RAM
240 Cores
Rack Capacity:
up to 153 TB
Dual Infiniband
SQL
SQL
SQL
SQL
Scan Rates:
up to 3.6 TB min
Dual Fiber Channel
SQL
SQL
SQL
Spare Compute Node
PDW Benefits – Massive Parallel Processing
Storage Nodes
Compute Nodes
Active / Passive
SQL
Management Servers
Landing Zone
Backup Node
Dual Infiniband
Query 1
?
?
?
?
?
?
?
?
?
?
SQL
SQL
SQL
SQL
SQL
SQL
SQL
SQL
Dual Fiber Channel
Control Nodes
Query 1 is
submitted to SQL
Server on Control
Node
SQL
SQL
Spare Compute Node
Query is executed
on all 10 Nodes
Results are sent
back to client
Built In High Availability
Failover Clusters
Mirrored drives
Dual networks
Hot swap drive
Dual power
supplies
Dual cooling fans
Price your appliance knowing you have no hidden costs for HA.
More secret sauce….
MPP Data Layout Options
Replicated
• A table structure that
exists as A full copy
within each discrete
DBMS instance.
Distributed
• A table structure that is
hashed on a single
column and uniformly
distributed across all
nodes on the appliance.
Each distribution is A
separate physical table in
the DBMS.
Basic Physical DB Design in PDW
Time Dim
Product Dim
Date Dim ID
Calendar Year
Calendar Qtr
Calendar Mo
Calendar Day
Prod Dim ID
Prod Category
Prod Sub Cat
Prod Desc
SQL
Sales Facts
Date Dim ID
Store Dim ID
Prod Dim ID
Mktg Camp Id
Qty Sold
Dollars Sold
Store Dim
Store Dim ID
Store Name
Store Mgr
Store Size
SQL
SQL
Mktg
Campaign
Dim
Mktg Camp ID
Camp Name
Camp Mgr
Camp Start
Camp End
SQL
Basic Physical DB Design in PDW
Distributed Tables
Time Dim
Product Dim
Date Dim ID
Calendar Year
Calendar Qtr
Calendar Mo
Calendar Day
Prod Dim ID
Prod Category
Prod Sub Cat
Prod Desc
Store Dim
Store Dim ID
Store Name
Store Mgr
Store Size
SQL
SQL
Sales Facts
Date Dim ID
Store Dim ID
Prod Dim ID
Mktg Camp Id
Qty Sold
Dollars Sold
Larger Fact Table is
Hash Distributed
Across All Compute
Nodes
SQL
Mktg
Campaign
Dim
Mktg Camp ID
Camp Name
Camp Mgr
Camp Start
Camp End
SQL
SS
SS FFFF11
11
SS
SS FFFF- 1
1
21
SS
SS FFFF- 1
1
31
Create Table Syntax
CREATE TABLE FactSales
(
ProductKey
INT NOT NULL
,
OrderDateKey
INT NOT NULL
,
DueDateKey
INT NOT NULL
,
ShipDateKey
INT NOT NULL
,
ResellerKey
INT NOT NULL
,
EmployeeKey
INT NOT NULL
,
PromotionKey
INT NOT NULL
,
CurrencyKey
INT NOT NULL
,
SalesTerritoryKey INT NOT NULL
,
SalesOrderNumber VARCHAR(20) NOT
NULL,
…
) WITH
(
DISTRIBUTION = HASH(ProductKey),
CLUSTERED INDEX(OrderDateKey) ,
PARTITION
(OrderDateKey RANGE RIGHT FOR
VALUES
( 20010601,
20010901,
…
) ) );
Control Node
Create table metadata
on Control Node
Send Create Table SQL to each compute node
Create Table FactSales_A
Create Table FactSales_B
Create Table FactSales_C
……
Create Table FactSales_H
Compute Node 1
Compute Node 2
…
Compute Node 10
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_A
s_A
s_A
s_A
s_A
s_A
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_A
s_A
s_A
s_A
s_A
s_A
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_A
s_A
s_A
s_A
s_A
s_A
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_B
s_B
s_B
s_B
s_B
s_B
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_B
s_B
s_B
s_B
s_B
s_B
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_B
s_B
s_B
s_B
s_B
s_B
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_C
s_C
s_C
s_C
s_C
s_C
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_C
s_C
s_C
s_C
s_C
s_C
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_C
s_C
s_C
s_C
s_C
s_C
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_D
s_D
s_D
s_D
s_D
s_D
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_D
s_D
s_D
s_D
s_D
s_D
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_D
s_D
s_D
s_D
s_D
s_D
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_E
s_E
s_E
s_E
s_E
s_E
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_E
s_E
s_E
s_E
s_E
s_E
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_E
s_E
s_E
s_E
s_E
s_E
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_F
s_F
s_F
s_F
s_F
s_F
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_F
s_F
s_F
s_F
s_F
s_F
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_F
s_F
s_F
s_F
s_F
s_F
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_G
s_G
s_G
s_G
s_G
s_G
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_G
s_G
s_G
s_G
s_G
s_G
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_G
s_G
s_G
s_G
s_G
s_G
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_H
s_H
s_H
s_H
s_H
s_H
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_H
s_H
s_H
s_H
s_H
s_H
FactSale
FactSale
FactSale
FactSale
FactSale
FactSale
s_H
s_H
s_H
s_H
s_H
s_H
Basic Physical DB Design in PDW
ReplicatedTables
Time Dim
Product Dim
Date Dim ID
Calendar Year
Calendar Qtr
Calendar Mo
Calendar Day
Prod Dim ID
Prod Category
Prod Sub Cat
Prod Desc
Sales Facts
Date Dim ID
Store Dim ID
Prod Dim ID
Mktg Camp Id
Qty Sold
Dollars Sold
Store Dim
Store Dim ID
Store Name
Store Mgr
Store Size
Mktg
Campaign
Dim
Mktg Camp ID
Camp Name
Camp Mgr
Camp Start
Camp End
Smaller Dimension
Tables are Replicated
on Every Compute
Node
SQL
T
D
S
D
P
D
M
D
SS
SS FFFF11
11
SQL
T
D
S
D
P
D
M
D
SS
SS FFFF- 1
1
21
SQL
T
D
S
D
P
D
M
D
SS
SS FFFF- 1
1
31
SQL
T
D
S
D
P
D
M
D
SS
SS FFFF- 1
1
41
Result: Fact -Dimension
Joins can be performed
locally
Storage Node – Drive Array (User Databases)
User Database
Small Form Factor Drives (SFF) (24) – 8 RAID 1 groups containing a single LUN
Large Form Factor Drives (LFF) (11) – 4 RAID 1 groups containing 2 LUNs each
LUN1
LUN2
LUN3
LUN4
LUN5
LUN6
LUN7
LUN8
LUN1
LUN2
LUN3
LUN4
LUN5
LUN6
LUN7
LUN8
LUN 9
LUN 9
LUN 9
SFF – 6 disk in a RAID 1+0 group
LFF – 2 disk in a RAID 1 group
Log
LUN 9
LUN 9
LUN 9
SFF - 2 Hot Spares
LFF – 1 Hot Spare
Hot
Spare
Hot
Spare
(SFF)
32
User Database
Storage Node – Physical File Layout
LUN1
LUN2
LUN3
LUN4
LUN5
LUN6
LUN7
LUN8
FG UDB_Dist_A
FG UDB_Dist_B
FG UDB_Dist_C
FG UDB_Dist_D
FG UDB_Dist_E
FG UDB_Dist_F
FG UDB_Dist_G
FG UDB_Dist_H
UDBDist.ndf
UDBDist.ndf
UDBDist.ndf
UDBDist.ndf
UDBDist.ndf
UDBDist.ndf
UDBDist.ndf
UDBDist.ndf
UDB Replicated FG
UDBRepl.ndf
LUN 9
UDBRepl.ndf
LUN 9
UDBRepl.ndf
LUN 9
Log
LUN 9
LUN 9
LUN 9
UDBRepl.ndf
UDBRepl.ndf
UDBRepl.ndf
UDBRepl.ndf
UDBRepl.ndf
8 distribution filegroups
1 replicated filegroup with 8 files
1 log filegroup
33
Parallelism with Distributed Tables
2 Data Rack system
• PDW, distributes a single large logical table across 160 tables across the 20
servers.
• The distribution is performed by selecting a column in each table and applying
a hash function to it.
FactOrders
OrderID
OrderDateKey
OrderTimeKey
Clustered Index =
OrderDateKey
FactTrades
Partition =
OrderDateKey
TradeDateKey
TradeTimeKey
Partition =
OrderDateKey
CurrentOrderID
CustomerKey
TotalQty
TradeID
Clustered Index =
OrderDateKey
Distribution =
OrderID
OldOrderID
…
Distribution =
CurrentOrderID
34
Parallelism with Distributed Tables
Distribution Compatible Join
Node 1
Fact Orders Fact Trades
Fact Orders
=
TradeID
OrderID
OrderDateKey
OrderTimeKey
CustomerKey
Node 2
Fact Orders Fact Trades
=
….
TotalQty
FactTrades
TradeDateKey
TradeTimeKey
CurrentOrderID
OldOrderID
Node 20
Fact Orders Fact Trades
=
SELECT OrderID, COUNT(TradeID) FROM FactOrders JOIN FactTrades
ON FactOrders.OrderID = FactTrades.CurrentOrderID
WHERE ……
35
Parallelism with Distributed
Distribution Incompatible Join
Node 1
Shuffle Move
Fact Orders Fact Trades
Fact Orders
=
OrderID
OrderDateKey
OrderTimeKey
Node 2
Fact Orders Fact Trades
CustomerKey
….
TotalQty
TempDB
=
FactTrades
TradeID
TradeDateKey
TradeTimeKey
CurrentOrderID
TempDB
OldOrderID
Node 20
Fact Orders Fact Trades
=
TempDB
SELECT OrderID, COUNT(TradeID) FROM FactOrders JOIN FactTrades
ON FactOrders.OrderID = FactTrades.OldOrderID
WHERE ……
36
Massive Scale and Performance with Parallel Data
Warehouse
?
?
?
Compute Nodes
?
?
Control Nodes (
Active / Passive
? ? ??
?? ??
Management
Servers
Load
Data Rack
Landing Zone
LLLLL
Backup Node
???? ? ???
???? ? ???
? ??? ???
???? ? ???
?? ?? ? ???
???? ? ???
???? ? ???
???? ? ???
?? ?? ? ???
?? ?? ? ???
Storage Nodes
Dual Fiber Channel
?
Control Rack
Dual Infiniband
?
?
Spare Database Server
Multiple queries
are
simultaneously
executed across
all nodes.
PDW
supports
querying
while data is
loading.
PDW Data Loader Illustrated
Distributed Tables
Control Rack
Data Rack
Control Node
Active/Passive
Database Server Nodes
Storage Nodes
DWLoader
Invoked from
Command line
DMS Reads Load
File and buffers
records to Sends
to Compute
Nodes roundrobin
SQL
Server
PDWEngine
Load Manager
Creates Staging
Tables
Load
Manager
Export
Manager
DMS
Manager
DMS
Infiniband
DMS
DMS Ser er
Converter
Sender
Receiver
Writer
Each row is converted
for bulk insert and
optionally hashed
Hashed row is sent to
appropriate node
receiver for loading
Landing Zone
Load
File
Load
Client
DMS
DMS
Converter
Sender
Receiver
Writer
Received
row is
pushed
onto writer
thread
Row is bulk
inserted into
staging table
Distributed Data Warehouse Architecture
Departmental
Reporting
Central EDW Hub
Accessible from
Anywhere
SQL Server
SQL
Database
Landing Zone
High-Performance
Reporting
Regional
Reporting
SQL Server
Analysis Services
SQL Server
FastTrack
3rd Party
Data
Integration
ETL Tools
Data
Files
3rd Party
RDBMS
Getting you there with DW Appliance Support
 With Microsoft SQL Server 2008 R2 Parallel Data Warehouse
you can choose from two support packages:
Maintenance
 Maintenance + PMC

 Maintenance provides core support





Single point of contact for the appliance (hardware and software)
DELL 7x24 4hr call to repair1
Unlimited support incidents
New Product Version rights
Appliance centric software servicing and support lifecycle
 Premier Mission Critical3 for appliances is designed to maximize
business continuity for your most important of solutions:

Architectural/Operations review2


Maintain



Dedicated Service Engineer (400hrs per year)
1 Risk and Health Assessments per year
Enhanced Repair





1
Designed to improve the overall success of the project
30 minute response (priority phone number access)
Enhanced Critical Situation escalation process
Escalation Manager
Executive incident visibility
Faster access to the software engineering team
Provided by the hardware vendor support group.
Included in the MCAP- PMC pricing at no cost between July 2010 and June 2011 – May be delivered by MCS or a Microsoft Partner – will also be provided to customers who do not purchase PMC
3 PMC features listed is basic PMC offering, PMC can be customized and configured to the customers environment.
2
PDW Momentum – Customer Wins in FY12
DIRECTEDGE gets 142X Performance Gain with Parallel
Data Warehouse (PDW) Appliance
Benefits
42
Benefits
43
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.