DataWarhouse rješenje za banke - IBM Netezza
Leader in Data Warehouse Appliances
Robert Božič
robert.bozic@si.ibm.com
© 2013 IBM Corporation
Information Management
The IBM Netezza Appliance: Revolutionizing Analytics
What is Netezza?
© 2013 IBM Corporation
Information Management
Appliances make it simple,
completely transforming the user experience.
 Dedicated device
 Optimized for purpose
 Complete solution
 Fast installation
 Very easy operation
 Standard interfaces
 Low cost
3
© 2013 IBM Corporation
Information Management
The IBM Netezza Appliance: Revolutionizing Analytics

Purpose-built analytics engine

Integrated database, server & storage

Standard interfaces

Low total cost of ownership

Speed: 10-100x faster than traditional systems

Simplicity: Minimal administration

Scalability: Peta-scale user data capacity

Smart: High-performance advanced analytics
© 2013 IBM Corporation
Information Management
Some of the customers
 Zavarovalnica Maribor
 Zavarovalnca Triglav
 Telekom Slovenije
 Tuš Mobil
 Petrol
 Informatika
 NLB
 MKZ
 Fabrika Duvana Sarajevo
© 2013 IBM Corporation
Information Management
Digital Media
Financial
Services
Government
Health & Life
Sciences
Retail /
Consumer
Products
Telecom
Other
Page 6
© 2013 IBM Corporation
Information Management
Business case – financial institution
Evolution of DWH environment
 Year 2000 – first “real” data warehouse
– 1 power user, 20 common users, 1 IT developer
 Years 2001 – 2005
– Number of users increased – 3 IT professionals, 100 common users, 10 power
users
– Enlargment of the data warehouse by factor 3
– DWH/BI becomes invloved in all crucial processes
 Years 2005 – 2007
– Enlargment of the data warehouse by 100
– Increasing the number of common users to 250
– By end of 2007 DWH/BI declared as a key business process at ZM
© 2013 IBM Corporation
Information Management
Users expectations
 High demand for information from the DWH system – the users expect
that the system contains everything
 “Immediate” responsiveness
• Daily or even shorter frequencies for data loading
• Real time BI
• Sophisticated iterative analysis: marketing analysis, product or
customer profitability analysis etc.
• Quick ad-hoc reporting for various purposes
© 2013 IBM Corporation
Information Management
Main issues with existing DWH
• More and more administration: database servers, application servers, BI
tools, BI applications...
• Higher and higher ownership costs (HW; SW licences...)
• Smaller and smaller time window for ETL
• More and more end users
• Continously increased complexity of the reports and analysis - killer
queries which can only be run in the evening hours , “freeze” the
server, even on DWH server.
© 2013 IBM Corporation
Information Management
Main challenges with building fully mature DWH/BI infrastructure
• Cost boost
• Entering in the upper cost-range of DWH servers
• Unfavorable licensing policy for database SW (number of users,
CPU licensees)
• Complex administration of database servers
• Required 1 - 4 employees to manage DWH
© 2013 IBM Corporation
Information Management
Benefits
• Considered by the management as one of the best IT investments in the
last 10 years
• The same DWH/BI team is capable to manage even larger BI
infrastructure.
• Vast simplification of many complex queries and batch jobs
• Development of new BI solutions
• Users completely changed the way of thinking what they can get out
of DWH
© 2013 IBM Corporation
Information Management
Smart
Predicts what shoppers are
likely to buy in future visits
Coupon redemption rates as high
as 25%
“Because of (Netezza’s) in-database technology,
we believe we'll be able to do 600 predictive models
per year (10X as many as before) with the same staff."
Eric Williams,
CIO and executive VP
© 2013 IBM Corporation
Appliance Simplicity

© 2013 IBM Corporation
Information Management
Managing The Netezza Appliance
No software installation
No storage administration
No database tuning
Less DBA drudgery,
More applications
© 2013 IBM Corporation
Information Management
OLE-DB
The Netezza Appliance – Loading
Data Integration
Ab Initio
JDBC
Business Objects/SAP
Composite Software
Expressor Software
GoldenGate Software (Oracle)
Data In
IBM Information Server
Sunopsis (Oracle)
ODBC
Informatica
SQL
WisdomForce
© 2013 IBM Corporation
Information Management
OLE-DB
The Netezza Appliance – Querying
© 2013 IBM Corporation
ODBC
Data Out
SQL
Actuate
Business Objects/SAP
Cognos (IBM)
Information Builders
Kalido
KXEN
MicroStrategy
Oracle OBIEE
QlikTech
Quest Software
SAS
SPSS (IBM)
Unica (IBM)
JDBC
Reporting & Analysis
Information Management
Simple to Deploy and Operate

Operations
Simply load and go .… it’s an appliance
 Installation to Business Value in ~2 days
 Ease of Evaluation and Perform As Advertised


BI Developers
Data model agnostic
 No configuration or physical modeling
 No indexes or tuning – out of the box performance
 Focus on business value, not physical design


ETL Developers
Faster load and transformation times
 No aggregate tables needed – simpler ETL logic
 In-database transformation – ‘ELT’


Business Analysts
On-Stream processing by 100’s of nodes
 Train of thought analysis – 10 to 100x faster
 True ad hoc queries
 Lower latency – load & query simultaneously

17
© 2013 IBM Corporation
Information Management
Appliance Architecture

© 2013 IBM Corporation
Information Management
IBM Netezza True Appliance Architecture
SOLARIS
AIX
TRU64
HP-UX
WINDOWS
LINUX
Client
Database
Server
Storage
DATA
SQL
ETL Server
DBA CLI
Source
Systems
3rd Party
Apps
SQL
I/O
CACHE
I/O
CACHE
CACHE
Data
High
Performance
Loader
© 2013 IBM Corporation
Information Management
IBM Netezza True Appliance Architecture
Client
SOLARIS
AIX
TRU64
HP-UX
Database
WINDOWS
Storage
Server
LINUX
ODBC 3.X
JDBC Type 4
SQL-92
SQL-99
Analytics
ETL Server
DBA CLI
Source
Systems
CACHE
3rd Party
Apps
I/O
CACHE
Database,
Server,I/O
Storage - in one
High
Performance
Loader
© 2013 IBM Corporation
CACHE
Information Management
IBM Netezza True Appliance Architecture
Optimized
Hardware+Software
Streaming Data
Purpose-built for high
performance analytics;
requires no tuning
Hardware-based query
acceleration for blistering
fast results
True MPP
Deep Analytics
All processors fully utilized
for maximum speed and
efficiency
Complex analytics
executed in-database for
deeper insights
21
© 2013 IBM Corporation
Information Management
IBM Netezza True Appliance Massively Parallel Processing
Client
SOLARIS
AIX
TRU64
HP-UX
WINDOWS
1
LINUX
S-Blade
Processor &
ODBC 3.X
JDBC Type 4
OLE-DB
SQL/92
streaming DB logic
SQL
Compiler
2
S-Blade
Processor &
streaming DB logic
Query
Plan
Execution
Engine
3
S-Blade
Processor &
streaming DB logic
Optimize



Admin
ETL Server
High-Speed
Loader/Unloader
960
DBA CLI
Source
Systems
Front End
DBOS
High-Performance
Database Engine
Streaming joins,
aggregations, sorts
S-Blade
Processor &
streaming DB logic
3rd Party
Apps
SMP Host
High
Performance
Loader
© 2013 IBM Corporation
Network
Fabric
Massively Parallel
Intelligent Storage
Information Management
IBM Netezza True Appliance Massively Parallel Processing™
Client
SOLARIS
AIX
TRU64
HP-UX
WINDOWS
1
LINUX
SQL
SQL
Compiler
S-Blade
Processor &
2
Snippets
1 2 3
3
streaming DB logic
2
S-Blade
2
3
Processor &
streaming DB logic
Query
Plan
Execution
Engine
3
S-Blade
2
3
Processor &
streaming DB logic
Optimize
ETL Server
High-Speed
Loader/Unloader



Admin
SQL
960
DBA CLI
Source
Systems
Front End
DBOS
High-Performance
Database Engine
Streaming joins,
aggregations, sorts
S-Blade
Processor &
2
3
streaming DB logic
3rd Party
Apps
SMP Host
High
Performance
Loader
© 2013 IBM Corporation
Network
Fabric
Massively Parallel
Intelligent Storage
Information Management
IBM Netezza True Appliance Massively Parallel Processing™
SOLARIS
AIX
TRU64
HP-UX
Client
WINDOWS
1
LINUX
S-Blade
Processor &
2
Consolidate
3
streaming DB logic
SQL
Compiler
2
S-Blade
2
3
Processor &
streaming DB logic
Query
Plan
Execution
Engine
3
S-Blade
2
3
Processor &
streaming DB logic
Optimize



Admin
ETL Server
High-Speed
Loader/Unloader
960
DBA CLI
Source
Systems
Front End
DBOS
High-Performance
Database Engine
Streaming joins,
aggregations, sorts
S-Blade
Processor &
2
3
streaming DB logic
3rd Party
Apps
SMP Host
High
Performance
Loader
© 2013 IBM Corporation
Network
Fabric
Massively Parallel
Intelligent Storage
Information Management
Our Secret Sauce
select DISTRICT,
PRODUCTGRP,
sum(NRX)
from
MTHLY_RX_TERR_DATA
where
MONTH = '20091201'
and
MARKET = 509123
and
SPECIALTY = 'GASTRO'
FPGA Core
Uncompress
Project
CPU Core
Restrict,
Visibility
Complex ∑
Joins, Aggs, etc.
Slice of table
MTHLY_RX_TERR_DATA
(compressed)

select DISTRICT,
where MONTH = '20091201'
PRODUCTGRP,
and
MARKET = 509123
sum(NRX)
and
SPECIALTY = 'GASTRO'
© 2013 IBM Corporation
sum(NRX)
Information Management
Appliance family for data life-cycle management
26
Skimmer
N1001/N2001
Cruiser
Dev & Test System
Data Warehouse
High Performance
Analytics
Queryable Archiving
Back-up / DR
1 TB to 10 TB
1 TB to 1.5 PB
100 TB to 10 PB
© 2013 IBM Corporation
Advanced Analytics

© 2013 IBM Corporation

Information Management
Advanced Analytics the Netezza Way
SPSS
 complex analytics
SAS, SPSS, R, Java, etc
 implicit parallelism
 petabyte scalability
 appliance simplicity
SQL
Demand
Forecasting
Fraud
Detection
R, S+
SQL
© 2013 IBM Corporation
Pre-Built In-Database Analytics
Statistics
 Descriptive Statistics+
 Distance Measures*
 Hypothesis Testing*
 Chi-Square &
Contingency Tables*
 Univariate &
Multivariate
Distributions+
Transformations
 Data Profiling /
Descriptive Statistics+
 General Diagnostics
Time Series
 Autoregressive+
 Forecasting*
 Statistics+
 Sampling
 Data prep
 Monte Carlo
Simulation*
Data Mining
Predictive
Mathematical
 Basic Math*
 Permutation and
Combination*
 Greatest Common
Divisor and Least
Common Multiple*
 Conversion of Values*
 Exponential and
Logarithm*
 Gamma and Beta
Functions
 Matrix Algebra+
 Area Under Curve*
 Interpolation Methods*
Geospatial
 Association Rules+
 Linear Regression+
 Geospatial Data Type
 Clustering+
 Logistic Regression+
 Geometric Functions
 Feature Extraction+
 Classification
 Geometric Analysis
 Discriminant
Analysis*
 Bayesian
 Sampling
 Model Testing
© 2013 IBM Corporation
* Fuzzy Logix
DB Lytix
capabilities
+ Netezza
Analytics and
Fuzzy Logix
DB Lytix
capabilities
IBM Netezza: bold claims, backed up

© 2013 IBM Corporation
Information Management
Bold Claims, but...We Prove Them!
 We
prove we are simpler
 We
prove we deliver performance
 We
prove we work within your environment
 We
prove we integrate with your 3rd party tools
 We
prove we are “easy to do business with”
 We
prove we have the lowest TCO
 We
prove business value
© 2013 IBM Corporation