The Enormous Data From
the Internet of Things
New Insights, Challenges, and a
New World of Opportunity for IT
Tom Bradicich, PhD
VP and GM, Servers and IoT Systems
Hewlett Packard Enterprise
twitter.com/tombradicichphd
linkedin.com/in/tombradicichphd
tombradicichphd.tumblr.com
April 2016
“I learned from watching the Oscars - that when you
take the podium, you can express your opinion on
any random topic unrelated to the event.”
T.M.S. Bradicich, PhD
2
Science, Discovery, and Innovation
1 - Thousand years ago:
Experimental
Science
2 - Last few hundred years:
Theoretical Science
− Description of natural phenomena
T
I
M
E
− Newton’s laws, Maxwell’s equations
3 - Last few decades:
Computational
Science
4 - Today:
Data Intensive
− Simulation of complex phenomena
− Unify theory, experimentation
− Data captured by instruments
− Data generated by simulators, machines
Jim Gray, PhD, The Fourth Paradigm
3
The Rise of New Pop “Marketectures” . . .
Factory of the
Future
Industrial
Internet of
Things (IIoT)
Industrie
4.0
Programmable
World
Smart
Production
Intelligent
Systems
Smart
Buildings
Brilliant
Machines
Cyber-physical
Systems
Big Analog
Data™
Solutions
New stress on
data centers
and clouds
New insights
from new
sources of data
New IT
products and
services
T.M.S. Bradicich, PhD
4
Big Data Characterized
Sources of Big Data:
Traditional IT
data sources
− Enterprise data: ERM, CRM HR
− Stock tics, medical data, inventories
− Events, logs, process, control
New / emerging
data sources
“Things” data,
analog and
natural sources
− Social data, behaviors, sentiments
− Tweets, posts, comments
− Physical world
− Natural phenomenon
− DAQ, A/D
T.M.S. Bradicich, PhD
5
“Things” Data Characterized – Big Data is not just in the data center
“Things” Data
−
Can be Big Data, derived from
the physical analog world
−
Is usually sourced from nature,
people, electrical and
mechanical devices,
environment, and objects
−
Is mostly sensor acquired and
digitized via A/D conversion
T.M.S. Bradicich, PhD
6
“Things data” sources are everywhere,
presented from the Things in the IoT
The “Things” can be . . .
Devices, machines, people, tools,
cars, animals, clothes, environment,
toys, buildings, etc.
T.M.S. Bradicich, PhD
7
Where IT meets OT
8
Two Dynamics at Play . . . Where IT Meets OT
New waves of IoT data, moving
into data centers and clouds
the IoT
#1
B
I
G
D
A
T
A
(data center / cloud)
T.M.S. Bradicich, PhD
9
Two Dynamics at Play . . . Where IT Meets OT
the IoT
How big is this Big Analog Data from the Things?
Consider the Industrial Internet of Things:
Smart Power Grid:
10TB / Month
Energy Turbine Test:
20-80TB / Day
Jet in Flight:
20TB / Hour
Smart Car:
500MB / Minute x
how many cars???
Experimental
Measurements:
40TB / Second
Big Analog Data is a trademark
of National Instruments
T.M.S. Bradicich, PhD
10
“Things data can be ‘Big Analog Data’, which is the oldest,
fastest, and biggest of all other big data --- combined."
T.M.S. Bradicich, PhD
11
Two Dynamics at Play . . . Where IT Meets OT
New waves of IoT data, moving
into data centers and clouds
the IoT
#1
B
I
G
D
A
T
A
#2
I
T
S
Y
S
T
E
M
S
New and optimized IT, moving
out the IoT edge
(data center / cloud)
T.M.S. Bradicich, PhD
12
“The cloud is the first off premises.
The IoT edge is now the ‘other off premises’."
T.M.S. Bradicich, PhD
13
Harnessing IoT Data and Extracting
Insights and Value
14
The 4 Stage IoT Solutions Architecture:
The “Things”
Sensors/Actuators
(wired, wireless)
Primarily
analog data
sources
Devices,
machines,
people, tools,
cars, animals,
clothes, toys,
environment,
buildings, etc.
Stage 2
Internet Gateways,
Data Acquisition
Systems
(data aggregation, A/D,
measurement, control)
The Edge
Stage 1
The IoT “Edge” is where the action is . . .
where the “Things” are . . .
Its the “other off premises” . . .
Sports arena
Electrical power grid
Planes, trains, and automobiles
Medical facility
Residences
People, animals, nature
Manufacturing floor
Product test facility
Wind farm
Energy power plant
Ships, boats, submarines
Crop fields
Retail stores
T.M.S. Bradicich, PhD, 2/2013, 6/2015
15
The 4 Stage IoT Solutions Architecture:
The “Things”
Sensors/Actuators
(wired, wireless)
Primarily
analog data
sources
Stage 2
Internet Gateways,
Data Acquisition
Systems
(data aggregation, A/D,
measurement, control)
The Edge
Stage 1
Stage 3
Stage 4
Edge IT
Data Center / Cloud
(analytics, preprocessing)
(analytics,
management, archive)
Devices,
machines,
people, tools,
cars, animals,
clothes, toys,
environment,
buildings, etc.
T.M.S. Bradicich, PhD, 2/2013, 6/2015
16
The 4 Stage IoT Solutions Architecture:
The “Things”
Sensors/Actuators
(wired, wireless)
Primarily
analog data
sources
Stage 2
Internet Gateways,
Data Acquisition
Systems
(data aggregation, A/D,
measurement, control)
The Edge
Stage 1
Stage 3
Stage 4
Edge IT
Data Center / Cloud
(analytics, preprocessing)
(analytics,
management, archive)
Devices,
machines,
people, tools,
cars, animals,
clothes, toys,
environment,
buildings, etc.
T.M.S. Bradicich, PhD, 2/2013, 6/2015
17
The 4 Stage IoT Solutions Architecture:
The “Things”
Stage 2
Sensors/Actuators
(wired, wireless)
Primarily
analog data
sources
Internet Gateways,
Data Acquisition
Systems
The Edge
Stage 1
(data aggregation, A/D,
measurement, control)
Stage 3
Stage 4
Edge IT
Data Center / Cloud
(analytics, preprocessing)
(analytics,
management, archive)
Devices,
machines,
people, tools,
cars, animals,
clothes, toys,
environment,
buildings, etc.
Visualization
SW Stacks:
Data Flow:
Control Flow:
Analytics
Management
Control
Analytics
Management
Control
Analytics
Management
Control
T.M.S. Bradicich, PhD, 2/2013, 6/2015
18
“If you’re not executing toward your corporate vision, then
you’re competing with just your imagination, and ultimately
customers don’t buy imagination."
T.M.S. Bradicich, PhD
19
Real World IoT Solutions
20
End-to-End IoT Solution
Rail In-Vehicle Monitoring
Sensors
Data Acquisition & Analysis
Systems
Edge IT
Data Center
Corporate IT
The Edge
The Things
64 sensor
signals per tram
3 DAQ
systems per tram
1 Ethernet switch
1 edge computer
per tram
21
End-to-End IoT Solution
Machine Asset & Condition Monitoring
The Things
Turbines
BOP Equipment
Data Acquisition &
Analysis Systems
Data Center
Edge IT
The Edge
-
Sensors
Sensors
−
−
−
−
−
−
−
−
−
−
−
Vibration
Temp
Oil
Motor
Ultrasound
IR
Leak Detection
Press
DGA
EMI
Partial Discharge
Operator Rounds
−
−
−
National
Instruments
cRIO
Edge IT Server
Corporate IT
Data Center
Cameras
Smell Sensors
Microphones
Images Courtesy of Duke Energy
22
End-to-End IoT Solution
Smart Power Grid Phase Measurements - PMU
Data Center
Edge IT
Data Acquisition &
Analysis Systems
Sensors
The
Edge
Edge Servers
Images Courtesy of Intel
and National Instruments
23
End-to-End IoT Solution
Smart Factory
Smart
Glasses
Aircraft body –
4000 holes,
1000
tools/settings
HD camera
embedded on
operator glasses
The Things
Data Flow:
The Things
Control
Smart Tools
Flow:
automatically
calibrated and set
SW Stacks:
Images Courtesy of Airbus
Holes sensed
by image
recognition
SW
24
“A good company aggressively follows the trends.
A great company sets the trends"
T.M.S. Bradicich, PhD
25
The 7 Principles of the IoT
26
The 7 Principles of IoT :
http://blog.iiconsortium.org/2015/07/the-7-principles-of-the-internet-of-things-iot.html
“The next ‘V’ of Big Data is ‘Visibility’ – access to data
from remote and disparate locations around the globe.”
28
Traditional “V’s” of Big Data:
Variety
The Next “V” of Big Data:
Visibility
− Mix of structure and format
− Access from disparate geographic locations
Volume
− Large amounts
Velocity
− High speed, high sample rates
Value
− Importance of analysis, which was
previously limited by technology
http://blog.iiconsortium.org/2015/07/the-7-principles-of-the-internet-of-things-iot.html
29
End-to-End IoT Solution
Engine Analysis and Test
Today, each of the “3 A’s” can be done at different geographic locations. . .
2. Analysis
The “3 A’s” of Big Data:
1. The Acquisition
2. The Analysis
2. Analysis
3. Action
“Time-to-Insight”
3. The Action
2. Analysis
Graphic take from an actual
IIoT customer quote:
3. Action
1. Acquisition
30
End-to-End IoT Solution
Engine Analysis and Test
Today, each of the “3 A’s” can be done at different geographic locations. . .
2. Analysis
The “3 A’s” of Big Data:
− Access from disparate
geographic locations
2. Analysis
2. Analysis
1. The Acquisition
2. The Analysis
Visibility
2. Analysis
3. Action
2. Analysis
“Time-to-Insight”
3. The Action
2. Analysis
Graphic take from an actual
IIoT customer quote:
3. Action
1. Acquisition
Data access by
data scientists
around the globe
31
“Insights from data will be derived within a spectrum of
places and times in the end-to-end IoT solution.”
32
4 Compute
Domains:
1) At the sensor,
in the I/O channel
2) At the gateway
or DAQ device
3) At the edge
server or PC
4) At the data
center or cloud
Spectrum of Insight
T.M.S. Bradicich, PhD, 2/2013, 6/2015
33
Analytics are no longer done in the only the data center ...
4 Compute
Domains:
1) At the sensor,
in the I/O channel
2) At the gateway
or DAQ device
3) At the edge
server or PC
4) At the data
center or cloud
Spectrum of Insight
A new vocabulary is needed: “inaugural
analytic”
- Begins the
analysis
“intermediate
analytic”
- Partial analysis,
no result
“concluding “intermediate “concluding
analytic”
analytic”
analytic”
- Renders a
conclusion
T.M.S. Bradicich, PhD, 2/2013, 6/2015
34
“Real time for the IoT begins at the sensor,
at the time of data acquisition.”
35
4 Compute
Domains:
1) At the sensor,
in the I/O channel
t0 - real time
for IoT data
2) At the gateway
or DAQ device
3) At the edge
server or PC
4) At the data
center or cloud
t0 - real time
for IT data
T.M.S. Bradicich, PhD, 2/2013, 6/2015
36
“Data center class IT will shift out to the IoT edge.”
37
End-to-End IoT Solution
Auto Racing Analysis and Optimization
Objectives:
The checkered flag! - optimizing speed, position, RPM, brakes, pit timing
Simplify and enhance trackside analytics and visualization – analytics at the edge
Automate and coordinate SoMe coverage
The Things
Sensors
Data
Acquisition
Virgin Racing Car
The Edge
•
•
•
The Pit
Edge IT
HPE Moonshot – As an Edge Server
•
•
Analytics
• Structured data- HPE Vertica
IT Infrastructure
• HPE Moonshot,
• 3PAR Storage, StoreOnce
Images Courtesy of Virgin Racing
38
End-to-End IoT Solution
The Pit
Data center IT
has shifted
out to the IoT edge
39
Where IT meets the IoT . . .
- Recent example activity . . .
Data analytic pre-cloud processing at the edge
Broadcast systems management in edge
Security and failure alerting at the edge and cloud
T.M.S. Bradicich, PhD, 2/2013, 6/2015
40
Where IT meets the IoT . . .
- Recent example activity . . .
HPE Edgeline Systems for the edge – watch this space . . .
T.M.S. Bradicich, PhD, 2/2013, 6/2015
41
Parting Advice
42
“With innovation, its important to challenge common belief.
For example, there is no direct evidence in the written account
of his great fall, that Humpty Dumpty was an anthropomorphic
chicken egg --- yet such is universally accepted.”
43
The Enormous Data From
the Internet of Things
New Insights, Challenges, and a
New World of Opportunity for IT
Tom Bradicich, PhD
VP and GM, Servers and IoT Systems
Hewlett Packard Enterprise
twitter.com/tombradicichphd
linkedin.com/in/tombradicichphd
tombradicichphd.tumblr.com
April 2016