BIG DATA & ANALYTICS
Because the speed of business is money
Héctor Colmenares, IBM SW
Core Database Competitive Sales Leader, IM SPGI
hector.colmenares@es.ibm.com
Tendencias del mercado
Demanda de respuestas en real time es una necesidad.
• Vivimos en la generación del “NOW”.
• Los datos crecen constantemente, y los usuarios esperan respuestas más
rápidas.
• Movilidad y el uso “democrático” de la información y la analítica hacen
de la tecnologia in-memory y mensajería distribuida un requerimiento
obligado. La velocidad cambia el negocio.
• Nuevas y modernas arquitecturas de datawarehouse dinámicas
reemplazarán los modelos tradicionales de datos por la demanda de
datos en real-time.
2
© 2014 IBM Corporation
Big Data y analítica del negocio
La Revolución de los datos
30 billion RFID
12+ TBs
tags today
(1.3B in 2005)
of tweet data
every day
4.6
billion
camera
phones
world
wide
data every day
? TBs of
IT Logs
80%
De los datos mundiales
NO ESTRUCTURADOS
log data
every day
76 million smart
© 2014 IBM Corporation
devices
sold
annually
2+
billion
25+ TBs of
3
100s of
millions
of GPS
enabled
meters in 2009…
200M by 2014
people
on the
Web by
end 2011
Big Data: todo son datos
Paradigma para extraer valor
Transaccional &
Datos Aplicativos
4
Contenido
Empresarial
Dato
Social
Data
Sensores
• Volumen
• Variedad
• Variedad
• Velocidad
• Estructurado
• No estructurado
• No estructurado
• Estructurado
• Entrada / Salida
• Volumen
• Veracidad
• Ingestión
© 2014 IBM Corporation
Big Data: todo son datos
Gestión de los datos: No es única
E-commerce
5
Mobile
Storefront
Sales
Analysis
Demand
Analysis
Key 1
JSON doc 1
Meter 1
Data series 1
Key 2
JSON doc 2
Meter 2
Data series 2
Transactional Database
JSON
Database
Analytics
Data Warehouse
Transaction
Processing
Mobile
Data Serving
Reporting
and Analytics
© 2014 IBM Corporation
Real Time
Fraud Detection
Operational
Data Warehouse
Operational
Analytics
Time Series
Database
Sensor Data
Analysis
Analítica dá la clave para incrementar la
competitividad
Compañías que realizan analíticas sofisticadas superan a su
competencia
2.6x
1.6x
mas rendimiento
que sus iguales del
sector
Mas ingresos
260%
estar entre los
mejores del
sector
2.5x
Valorización del
precio del stock
Source: The New Intelligent Enterprise, a joint MIT Sloan Management Review and IBM Institute of Business Value analytics research partnership. Copyright © Massachusetts Institute of
Technology 2011. Outperforming in a data-rich, hyper-connected world, IBM Center for Applied Insights study conducted in cooperation with the Economist Intelligence Unit and the IBM
Institute of Business Value. 2012
6
© 2014 IBM Corporation
Cuantificar valor de la analítica
Speed equals 39% faster payment*
59
reduced days
to payment
business
growth
days to payment
increased
cash flow
36
2011
2012
2013
*Based on analysis done by Xero, a SaaS company specialising
in accounting software, 2014. Link to blog & infographic: HERE
Average closing of accounts
Source: SAP value engineering study
7
© 2014 IBM Corporation
Speed of Business Process, Is Money
La lógica del Data Warehouse
Contemplar componentes, propósitos, zonas
Vertical Industry Accelerators
Advanced Application Capabilities
Machine and
sensor data
Image and video
Actionable
insight
Real-time processing & analytics
Data types
Operational
systems
Deep
analytics &
modeling
Exploration,
landing and
archive
Predictive
analytics
and modeling
Trusted data
Enterprise
content
Reporting &
interactive
analysis
Transaction and
application data
Decision
management
Reporting,
analysis, content
analytics
Social data
Third-party data
Logical Data Warehouse
Information Integration & Governance
8
© 2014 IBM Corporation
Discovery and
exploration
La lógica del Data Warehouse
Ejemplo de la Solución de IBM
Vertical Industry Accelerators
BigSQL and SQL based applications
Advanced Application Capabilities
Federation
and In-memory
Real-time
processing
& analyticsfederated cube
Data types
Machine and
sensor data
Image and video
Operational
systems
Deep
analytics &
modeling
Exploration,
landing and
archive
Trusted data
ORACLE
MicrosoftReporting &
interactive
Teradata
analysis
Enterprise
content
Transaction and
application data
Actionable
insight
Decision
management
Predictive
analytics
and modeling
Reporting,
analysis, content
analytics
Social data
BigMatch
Third-party data
Logical Data Warehouse
Streams
9
© 2014 IBM Corporation
Information Integration & Governance
Discovery and
exploration
¿Qué hace diferente BLU Acceleration?
Innovacionees de IBM Research & Developments Labs.
Next Generation In-Memory
Analyze Compressed Data
In-memory columnar processing with
dynamic movement of data from storage
Patented compression technique that preserves order
so data can be used without decompressing
C1 C2 C3 C4 C5 C6 C7 C8
Encoded
CPU Acceleration
Data Skipping
Multi-core and SIMD parallelism
(Single Instruction Multiple Data)
Instructions
10
© 2014 IBM Corporation
Results
Skips unnecessary processing of irrelevant data
Data
BLU Shadow Tables
 Dedicated analytics and reporting
 Operational analytics
 Mixed workload analytics with OLTP
OLTP Indexes Analytical Indexes
Traditional row-based tables, with indexes for,
for tables dedicated to OLTP or Operational
Analytics
+
Simple BLU tables ( columnar ) for tables
dedicated to analytics and reporting workloads
OLTP Indexes
+
+
Single Server
All 3 scenarios in a single database
11
© 2014 IBM Corporation
Traditional row-based tables, with indexes and
BLU Shadow Tables for tables with mixed
workloads
• Power of BLU
• Faster analytics and reporting
• Faster OLTP
• Simpler environment
Oportunidad: Big Data y Analytics
30 billion RFID
tags today
(1.3B in 2005)
TBs
 12+
PROBLEMAS
DE RENDIMIENTO
of tweet data
every day
data every day
? TBs of
IT Logs
SAP BW
CARGAS ANALÍTICAS
 COMPETENCIA:
80%
SAP con HANA
ORACLE con EXADATA
TERADATA Of world’s data
is unstructured
MS-SQL
25+ TBs of
log data
every day
PREMISA: AHORRO DE COSTES
76 million smart
12
© 2014 IBM Corporation
meters in 2009…
200M by 2014
4.6
billion
camera
phones
world
wide
100s of
millions
of GPS
enabled
devices
sold
annually
2+
billion
people
on the
Web by
end 2011
DB2 with BLU vs Microsoft SQLServer
Query Response Time: (In Seconds, Less is Better)
DB2 (extrapolated to 8 cores, 80% scalability) v/MS SQL (8 cores)
90
80
70
Query time (s)
60
50
40
30
20
10
0
1
2
3
4
5
6
7
8
9
Query Runs
DB2 BLU Factored
13
© 2014 IBM Corporation
MS SQL
7X-8X Better Performance with equal cores
10
DB2 with BLU vs Microsoft SQLServer
Query Response Time: (In Seconds, Less is Better)
BestOffer
Less Cores & Licenses but Much more Performance => Better SLA
DB2 BLU (2 cores) v/MS SQL (8 cores)
90
80
Query Time (s)
70
60
50
40
30
20
10
0
1
2
3
4
5
6
7
8
9
Query Runs
DB2 BLU
14
© 2014 IBM Corporation
MS SQL
“56% better performance, with 25% of the cores !
Wow, that’s great !”
10
Estimated HW Infrastructure for Production – Year 1 and Year 5
assumption yearly 20% growth
Source Oracle database 8 TB on BW 7.0 (non-unicode)
DB2 on 2-tier architecture (on one server all components)
HANA on 3-tier rachitecture (database and application on different servers)
15
15
© 2014 IBM Corporation
Huge savings through DB2 Technology
Often DB2 BLU needs 70-95% less HW
COMPARATIVA ENTRE IBM POWER + DB2 contra ORACLE EXADATA
NECESIDAD INICIAL: Sistema SAP (180 sistemas, 48 entornos de producción)
CONTINUIDAD DEL NEGOCIO: Contingencia en 2 centros separados
NOTA:
• El ejercicio de sizing se ha basado en la metodología SAP con entornos para Producción, Pre-Producción y Desarrollo/Q
• El nivel de rendimiento SAPS ha sido el mismo en ambos casos
• La infraestructura IBM es POWER8 + AIX + DB2 10.5 y la de Oracle es EXADATA (INTEL`+ Oracle Linux + Oracle DB)
• La opción de IBM permite virtualización
• El ejercicio es una estimación y está orientada a mostrar las diferencias de infraestructura entre ambas soluciones
Customer runs DB2 on POWER
- 180 systems, 48 production
- 26 HA (LPM*) + 26 DR (PowerHA)
- 2 x data centers
Possible Exadata implementation **
- 180 systems, 48 production
- 26 HA + 26 DR clusters
- 2 x BIGGER or more data centers
 6 full racks for production + HA
 6 full racks for DR
 6 full racks for test/QA
 6 full racks for dev
 6 full racks for the rest 36 systems
 4 x POWER servers (160 cores)
 ~30 full racks (5760 cores)
16
16
© 2014 IBM Corporation
* LPM - AIX live partition mobility
** No virtualization + limited number of databases per rack (e.g. 8 database servers per full rack, max 24 processor per database)
COMPARATIVA ENTRE IBM POWER + DB2 contra SAP HANA
NECESIDAD INICIAL: Sistema SAP (180 sistemas, 48 entornos de producción)
CONTINUIDAD DEL NEGOCIO: Contingencia en 2 centros separados
NOTA:
•
•
•
•
•
•
El ejercicio de sizing se ha basado en la metodología SAP con entornos para Producción, Pre-Producción y Desarrollo/Q
El nivel de rendimiento SAPS ha sido el mismo en ambos casos
La infraestructura IBM es POWER8 + AIX + DB2 10.5 y la de SAP HANA (INTEL+ Linux + HANA)
La opción de IBM permite virtualización
El ejercicio es una estimación y está orientada a mostrar las diferencias de infraestructura entre ambas soluciones
1 HANA UNIT = 64 Gb RAM = 13 K€
Customer runs DB2 on POWER
- Customer runs DB2 on POWER
- 180 systems, 48 production
- 26 HA (LPM*) + 26 DR (PowerHA)
2 x data centers
Possible HANA implementation **
- 180 systems, 48 production
- 26 HA + 26 DR clusters
- 2 x BIGGER or more data centers
 48 appliance servers for production
 52 appliance servers for HA+DR clusters
 up to 48 appliance servers for test/QA
 up to 48 appliance servers for dev
 up to 36 appliance servers for rest
 4 x POWER servers (160 cores)
 101-232 x HANA servers
(4040-9400 cores)
17
17
© 2014 IBM Corporation
* LPM - AIX live partition mobility
** No virtualization + limited number of databases per rack (e.g. 8 database servers per full rack, max 24 processor per database)
Balluff GmbH – Game-changing boost to information
delivery with IBM DB2 enables rapid insight
98 percent
faster access to complex reports
30 percent
typical report speed increase
50 percent
faster SAP ERP response times
Solution Components
•SAP® Business Warehouse, SAP ERP, SAP ERP HCM,
SAP CRM, SAP NetWeaver® Enterprise Portal, SAP PI
•IBM® AIX®, DB2® for Linux, UNIX and Windows with
BLU Acceleration, PowerHA® SystemMirror®,
PowerVM®, System Storage SAN Volume Controller,
Tivoli® Storage FlashCopy® Manager, IBM® Power®
750, FlashSystem™ 840, XIV®, IBM System and
Technology Group Lab Services, IBM Software Group
Services
Business challenge: Balluff knew that slow access to finance and business
reports threatened productivity and potential growth. How could executives gain
fast insight into critical data to make better business decisions?
The solution: The company moved its SAP Business Warehouse to IBM®
DB2® with BLU Acceleration, running on IBM Power Systems™ with IBM AIX®
and IBM PowerHA®.
“IBM DB2 with BLU Acceleration is the ideal solution for us because we
can gain new insights into business data more rapidly. Deploying IBM
DB2 with BLU Acceleration was a low-risk project; implementation was
quick and easy without affecting availability.”
—Bernhard Herzog, Team Manager Information Technology SAP, Balluff
Deep Blue IBM Solution SAP Stack
18
© 2014 IBM Corporation
IBM SAP Alliance
© 2014 IBM Corporation
Big Data & Analytics
Big Data & Analytics
The Business Value, Why Speed is Money
Business Analytics Accelerator
Increase productivity & drive business growth
Adding Value, not complexity
Dynamic
Query
Dynamic
Cubes
Infrastructure
That
Matters
82X
más rápido
C1 C2 C3 C4 C5 C6 C7 C8
DB2 with BLU
vs. Competitor Row Store Database on
Ivy Bridge (x86)1
19
© 2014 IBM Corporation
Compatible
Query
1) Based on IBM internal tests as of April 7, 2014 comparing IBM DB2 with BLU Acceleration on Power with a comparably tuned competitor row store database server on x86 executing a materially identical 2.6TB BI workload in a controlled laboratory environment. Test measured 60 concurrent user report
throughput executing identical Cognos report workloads. Competitor configuration: HP DL380p, 24 cores, 256GB RAM, Competitor row-store database, SuSE Linux 11SP3 (Database) and HP DL380p, 16 cores, 384GB RAM, Cognos 10.2.1.1, SuSE Linux 11SP3 (Cognos). IBM configuration: IBM S824,
24 cores, 256GB RAM, DB2 10.5, AIX 7.1 TL2 (Database) and IBM S824, 16 of 20 cores activated, 384GB RAM, Cognos 10.2.1.1, SuSE Linux 11SP3 (Cognos). Results may not be typical and will vary based on actual workload, configuration, applications, queries and other variables in a production
environment.
82x calculation based on geometric mean calculation giving equal weighting to the report per hour (RPH) improvements in the three categories of simple, intermediate, and complex reports. GEOMEAN(RPH_simple,RPH_intermediate,RPH_complex) = GEOMEAN(18.85,40.07,747.63)=82.66
POWER + BLU Acceleration
pero y ….
¿Cómo avanzamos?
Elegir una opción
de las dos configuraciones
Small Configuration
Large Configuration
Existing Data Warehouse
Existing Data Warehouse
•
•
Supports up to 5 TB uncompressed
active data (1 TB compressed data)
Supports up to 10 TB uncompressed
active data (3 TB compressed data)
Hardware Configuration
Hardware Configuration
•
Power S814: 8 cores, 3.72 GHz
•
Power S824: 24 cores, 3.52 GHz
•
Memory: 256 GB DRAM
•
Memory: 1 TB DRAM
•
Storage:
•
Storage:
•
•
146 GB (RAID 1) (OS/PGM VG)
2.4 TB HDD (RAID 5) (Data VG)
•
AIX Standard Edition
•
PowerVM Enterprise Edition
Software Configuration
•
DB2 Advanced Workgroup Edition
•
2.4 TB HDD (RAID 5) (Data VG)
•
1.55 TB SSD (DB2 overflow cache)
•
146 GB (RAID 1) (OS/PGM VG)
•
AIX Standard Edition
•
PowerVM Enterprise Edition
Software Configuration
•
DB2 Advanced Workgroup Edition
PRECIO MUY INTERESANTE ** PREGUNTE A SU VENDEDOR DE IBM **
21
© 2014 IBM Corporation
Configuración Try&Buy, incluye HW & SW
New BLU POC Program

Stack: Power 8, DB2 10.5 with BLU Acceleration
and Cognos BI 10.2 with Dynamic Cubes

Target Use Cases: “Business Intelligence
Workload Accelerator”

Provides acceleration for BI workloads in
competitive or back-level environments
Can work in any environment

Commodity hardware accounts

Microstrategy, Business Objects

SQLServer, Oracle

Is an accelerator to your existing environment
Configurations

22
5 pre-defined Power 8, DB2 BLU and Cognos BI
configurations to support various workload sizes
© 2014 IBM Corporation
 Announcement Highlights
– Announced: June, 2014
– Leverages established Power POC
processes
– Define the free trial period
– Try with your own data
 Owners for Additional Information
– lking@ca.ibm.com
– kschlamb@ca.ibm.com
GRACIAS