A New Era Of Analytic
Ömer Sever (omers@tr.ibm.com)
IBM SWG TR
Enterprise Content Management
Agenda
• The Myth About Big Data
• Case Studies on Big Data
• How To Start With A Big Data Project ?
• Q&A
2
© 2012 IBM Corporation
The Myth About Big Data
 Big Data Is New
 Big Data Is Only About Massive Data Volume
 Big Data Means Hadoop
 Big Data Need A Data Warehouse
 Big Data Means Unstructured Data
 Big Data Is for Social Media & Sentiment Analysis
3
© 2012 IBM Corporation
Big Data Is..
It is all about better Analytic on a
broader spectrum of data, and
therefore represents an opportunity
to create even more differentiation
among industry peers.
4
© 2012 IBM Corporation
Where Is This “Big Data” Coming From ?
100s of
millions
of GPS
enabled
devices
sold
annually
25+ TBs
of
log data
every day
5
camera
phones
world
wide
data every
day
? TBs of
12+ TBs
of tweet data
every day
4.6
billion
30 billion RFID
tags today
(1.3B in 2005)
2+
billion
76 million smart
meters in 2009…
200M by 2014
people
on the
Web by
end 2011
© 2012 IBM Corporation
With Big Data, We’ve Moved into a New Era of Analytics
12+ terabytes
5+ million
of Tweets
create daily.
100’s
of different
types of data.
trade events
per second.
Volume
Velocity
Variety
Veracity
Only
1 in 3
decision makers trust
their information.
6
© 2012 IBM Corporation
The number of organizations who see analytics
as a competitive advantage is growing.
63%
2010
business initiative
2011
2012
BUSINESS
IMPERATIVE
IQ
7
© 2012 IBM Corporation
Studies show that organizations competing
on analytics outperform their peers
substantially outperform
IBM IBV/MIT Sloan Management Review Study 2011
Copyright Massachusetts Institute of Technology 2011
1.6x
Revenue
Growth
2.5x
Stock Price
Appreciation
2.0x
8
© 2012 IBM Corporation
EBITDA
Growth
Four Characteristics of Big Data
Cost efficiently
processing the
growing Volume
50x
2010
35
ZB
Responding to the
increasing Velocity
30
Billion
RFID
sensors and
counting
Collectively
Analyzing the
broadening Variety
80% of the
worlds data is
unstructured
2020
Establishing the
Veracity of big
data sources
1 in 3 business leaders don’t trust
the information they use to make
decisions
9
© 2012 IBM Corporation
Analytic With Data-In-Motion & Data At Rest
Opportunity Cost Starts Here
Data Ingest
Adaptive
Analytics
Model
Forecast
Nowcast
01011001100011101001001001001
0110100101010011100101001111001000100100010010001000100101 11000100101001001011001001010
01100100101001001010100010010
01100100101001001010100010010
11000100101001001011001001010
01100100101001001010100010010
Bootstrap
01100100101001001010100010010
01100100101001001010100010010
Enrich
01100100101001001010100010010
11000100101001001011001001010
01100100101001001010100010010
01100100101001001010100010010
01100100101001001010100010010
01100100101001001010100010010
01100100101001001010100010010
11000100101001001011001001010
01100100101001001010100010010
01100100101001001010100010010
01100100101001001010100010010
11000100101001001011001001010
10
© 2012 IBM Corporation
Agenda
• Myth About Big Data.. What Is It ?
• Case Studies on Big Data
• How To Start With Big Data Project ?
• Q&A
11
© 2012 IBM Corporation
The 5 Key Big Data Use Cases
Big Data Exploration
Find, visualize, understand
all big data to improve
decision making
Enhanced 360o View
of the Customer
Security/Intelligence
Extension
Extend existing customer
views (MDM, CRM, etc) by
incorporating additional
internal and external
information sources
Lower risk, detect fraud
and monitor cyber security
in real-time
Operations Analysis
Data Warehouse Augmentation
Analyze a variety of machine
data for improved business results
Integrate big data and data warehouse
capabilities to increase operational efficiency
12
© 2012 IBM Corporation
Big Data Exploration: Needs
Find, visualize, understand all big data
to improve decision making
Struggling to manage
and extract value from
the growing 3 V’s of
data in the enterprise;
Need to unify
information across
13
federated sources
Inability to relate “raw”
data collected from
system logs, sensors,
clickstreams, etc., with
customer and line-ofbusiness data managed
in enterprise systems
Risk of exposing
unsecure personally
identifiable information
(PII) and/or privileged
data due to lack of
information awareness
© 2012 IBM Corporation
Big Data Exploration: Value & Diagram
Relational
Data
File
Systems
Content
Management
Email
Data Explorer
Application/
Users
CRM
Supply
Chain
ERP
Find, Visualize & Understand
all big data to improve
business knowledge
• Greater efficiencies in business
processes
• New insights from combining and
analyzing data types in new
ways
• Develop new business models
with resulting increased market
presence and revenue
RSS Feeds
Cloud
Custom
Sources
14
© 2012 IBM Corporation
Big Data Exploration: Customer Example
Airline Manufacturer
• Exploring 4 TB to drive point business solutions
(supplier portal, call center, etc.)
• Single-point of data fusion for all employees to use
• Reduced costs & improved operational performance for the business
Key Questions to Ask
 Can you separate the “noise” from useful content?
 Can you perform data exploration on large and
complex data?
 Can you find insights in new or unstructured data
types (e.g. social media and email)?
15
 Can you navigate and explore all enterprise
and external content in a single user interface?
 Can you quickly identify areas of data risk?
 Do you have a logical starting point for your
big data initiatives?
Product Starting Point: InfoSphere Data Explorer
© 2012 IBM Corporation
The 5 Key Big Data Use Cases
Big Data Exploration
Find, visualize, understand
all big data to improve
decision making
Enhanced 360o View
of the Customer
Security/Intelligence
Extension
Extend existing customer
views (MDM, CRM, etc) by
incorporating additional
internal and external
information sources
Lower risk, detect fraud
and monitor cyber security
in real-time
Operations Analysis
Data Warehouse Augmentation
Analyze a variety of machine
data for improved business results
Integrate big data and data warehouse
capabilities to increase operational efficiency
16
© 2012 IBM Corporation
Enhanced 360º View of the Customer: Needs
Extend existing customer views (MDM, CRM,
etc) by incorporating additional internal and
external information sources
Need a deeper
understanding of
customer sentiment
from both internal and
external sources
17
Desire to increase
customer loyalty
and satisfaction
by understanding
what meaningful
actions are
needed
Challenged getting the
right information to the
right people to provide
customers what they need
to solve problems, crosssell & up-sell
© 2012 IBM Corporation
Enhanced 360º View of the Customer: Value & Diagram
SOURCE SYSTEMS
CRM
Name:
J Robertson
Address:
35 West 15th
Address: Pittsburgh, PA 15213
ERP
Name:
Janet Robertson
Address:
35 West 15th St.
Address: Pittsburgh, PA 15213
Legacy
Name:
Jan Robertson
Address:
36 West 15th St.
Address: Pittsburgh, PA 15213
Master
Data
Management
360 View of
Party Identity
First:
Janet
Last:
Robertson
Address:
35 West 15th St
City:
Pittsburgh
State/Zip:
PA / 15213
Gender:
F
Age:
48
DOB:
1/4/64
18
Unified View of Party’s Information
BigInsights
Streams
Unified View of Customer’s Information
© 2012 IBM Corporation
Warehouse
Enhanced 360º Customer View: Customer Example
Confidential, Internal
Use Only
• Create “Facebook”
• Identify 200+ different customer profiles to help in fulfillment &
marketing efforts
• Leverage new data types in customer analysis
Key Questions to Ask
 Can you identify and deliver all data as it relates to a
customer, product, competitor to those to need it?
 Can you gathering insights about your customers
from social data, surveys, support emails, etc.?
 Can you combine your structured and unstructured
data to run analytics?
19
 How are you driving consistency across your
information assets when representing your
customer, clients, partners etc.?
 How can a complete view of the customer
enhance your line of business users and
result in better business outcomes?
Product Starting Point: InfoSphere MDM Server, Data Explorer, BigInsights
© 2012 IBM Corporation
The 5 Key Big Data Use Cases
Big Data Exploration
Find, visualize, understand
all big data to improve
decision making
Enhanced 360o View
of the Customer
Security/Intelligence
Extension
Extend existing customer
views (MDM, CRM, etc) by
incorporating additional
internal and external
information sources
Lower risk, detect fraud
and monitor cyber security
in real-time
Operations Analysis
Data Warehouse Augmentation
Analyze a variety of machine
data for improved business results
Integrate big data and data warehouse
capabilities to increase operational efficiency
20
© 2012 IBM Corporation
Security/Intelligence Extension: Needs
Security/Intelligence Extension enhances
traditional security solutions by analyzing all
types and sources of under-leveraged data
Enhanced
Intelligence &
Surveillance
Insight
Real-time Cyber
Attack Prediction
& Mitigation
Crime prediction
& protection
Reduce
Customer Churn
Analyze data-in-motion & at rest to:
• Find associations
• Uncover patterns and facts
• Maintain currency of information
Analyze network traffic to:
• Discover new threats early
• Detect known complex threats
• Take action in real-time
Analyze Telco & social data to:
• Gather criminal evidence
• Prevent criminal activities
• Proactively apprehend criminals
• Customer Retention
21
© 2012 IBM Corporation
© 2013 IBM Corporation
Asian Telco reduces
billing costs and
improves customer
satisfaction.
Capabilities:
Stream Computing
Analytic Accelerators
Real-time mediation and analysis of
6 Billion CDRs per day
Data processing time reduced from
12 hrs to 1 sec
Hardware cost reduced to 1/8th
Proactively address issues
(e.g. dropped calls) impacting
customer satisfaction.
22
22
© 2012 IBM Corporation
Asian Government
Agency
National Intelligence
Platform
Capabilities:
Stream Computing
• Analyze all Internet traffic
(social media, email, etc)
• Track persons of interest
(drug/sex traffickers,
terrorists, illegal
refugees/immigrants) and
civil/border activity
23
23
© 2012 IBM Corporation
The 5 Key Big Data Use Cases
Big Data Exploration
Find, visualize, understand
all big data to improve
decision making
Enhanced 360o View
of the Customer
Security/Intelligence
Extension
Extend existing customer
views (MDM, CRM, etc) by
incorporating additional
internal and external
information sources
Lower risk, detect fraud
and monitor cyber security
in real-time
Operations Analysis
Data Warehouse Augmentation
Analyze a variety of machine
data for improved business results
Integrate big data and data warehouse
capabilities to increase operational efficiency
24
© 2012 IBM Corporation
Operations Analysis: Needs
Analyze a variety of machine
data for improved business results
Business Challenges:
•Complexity and rapid growth of machine data
•Difficult to capture small fraction of machine for better
decision
•In-ability to analyze machine data and combine it with
enterprise data for a full view analysis
Benefits:
• Gain real-time visibility into operations,
customer experience, transactions and
behavior
• Proactively plan to increase operational
25
efficiency
• Identify and investigate anomalies
• Monitor end-to-end infrastructure to
proactively avoid service degradation
or outages
© 2012 IBM Corporation
Raw Logs and Machine Data
Operations Analysis: Value & Diagram
Indexing, Search
Only store
what is needed
Statistical Modeling
Machine Data
Accelerator
Root Cause Analysis
Real-time Analysis
26
Federated
Navigation &
Discovery
© 2012 IBM Corporation
OPERATIONAL - ANALYSIS
Capabilities:
Hadoop & Stream Computing
• Intelligent Infrastructure
Management: log analytics, energy bill
forecasting, energy consumption
optimization, anomalous energy usage
detection, presence-aware energy
management
27
• Optimized building energy
consumption with centralized
monitoring; Automated preventive and
corrective maintenance
© 2012 IBM Corporation
The 5 Key Big Data Use Cases
Big Data Exploration
Find, visualize, understand
all big data to improve
decision making
Enhanced 360o View
of the Customer
Security/Intelligence
Extension
Extend existing customer
views (MDM, CRM, etc) by
incorporating additional
internal and external
information sources
Lower risk, detect fraud
and monitor cyber security
in real-time
Operations Analysis
Data Warehouse Augmentation
Analyze a variety of machine
data for improved business results
Integrate big data and data warehouse
capabilities to increase operational efficiency
28
© 2012 IBM Corporation
Data Warehouse Augmentation: Needs
Integrate big data and data warehouse
capabilities to increase operational efficiency
Need to leverage variety of data
• Structured, unstructured, and streaming
data sources required for deep analysis
• Low latency requirements
(hours—not weeks or months)
• Required query access to data
29
Extend warehouse infrastructure
• Optimized storage, maintenance and
licensing costs by migrating rarely used data
to Hadoop
• Reduced storage costs through smart
processing of streaming data
• Improved warehouse performance by
determining what data© 2012
to feed
into it
IBM Corporation
Data Warehouse Augmentation: Customer Example
Internal Use Only
• Creates pre-processing hub and performs ad hoc analysis
• Hadoop-based landing zone used to store, manage and analyze structured,
semi-structured and multi-structured data before moving to the warehouse
• Benefits: Data warehouse optimized for workload and performance
• Utilized InfoSphere BigInsights, InfoSphere DataStage
Key Questions to Ask

Are you drowning in very large data sets (TBs to  Do you have a lot of cold, or low-touch, data driving
up costs or slowing performance?
PBs) that are difficult and costly to store?

Are you able to utilize and store new data types?  Do you want to perform analysis of data in-motion to
determine what should be stored in the warehouse?
Are you facing rising maintenance/licensing
 Do you want to perform data exploration on all data?
costs?
 Are you using your data for new types of analytics?
Do you use your warehouse environment as a
repository for all data?


30
Product Starting Point: BigInsights, Streams
© 2012 IBM Corporation
Agenda
• Myth About Big Data.. What Is It ?
• Case Studies on Big Data
• How To Start With Big Data Project ?
• Q&A
31
© 2012 IBM Corporation
IBM Big Data Strategy: Move the ANALYTICS Closer to the Data
Analytic Applications
BI /
Exploration / Functional Industry Predictive Content
Reporting Visualization
App
App
Analytics Analytics
IBM Big Data Platform
Visualization
& Discovery
Application
Development
Systems
Management
Accelerators
Hadoop
System
Stream
Computing
Data
Warehouse
BigInsights
certified
Apache Hadoop
Information Integration & Governance
New analytic applications drive the
requirements for a big data platform
•
Integrate and manage the full variety,
velocity and volume of data
•
Apply advanced analytics to
information in its native form
•
Visualize all available data for ad-hoc
analysis (even in motion!)
•
Development environment for building
new analytic applications
•
Workload optimization and scheduling
•
Security and Governance
And grow and evolve on your
current IT infrastructure
32
© 2012 IBM Corporation
Four Entry Points of Big Data
Analytic Applications
Unlock Big
Data
BI /
Exploration / Functional Industry Predictive Content
Reporting Visualization
App
App
Analytics Analytics
IBM Big Data Platform
Visualization
& Discovery
Application
Development
Systems
Management
Simplify
Your
Warehouse
Accelerators
Hadoop
System
Stream
Computing
Data
Warehouse
Information Integration & Governance
Preprocess
Raw Data
Analyse
Streaming
Data
33
© 2012 IBM Corporation
Thank you
34
© 2012 IBM Corporation