DATA MANAGEMENT FOR THE
INTERNET OF THINGS
February, 2015
 Peter Krensky, Research Analyst,
Analytics & Business Intelligence
Report Highlights
p2
Data challenges
p4
Managing data at
the edge
p6
Time series,
geospatial, and
unstructured data
This report examines organizations leveraging data generated by
the Internet of Things and managing a constant flow of
information.
p7
Target areas for
improvement
Data Management for the Internet of Things
2
As organizations
embark on new IoT
initiatives and work
to attract more
insight from swelling
data volumes, a new
data management
approach is called
for.
Aberdeen’s 2014 Big Data survey
collected responses from 205
organizations in a variety of
industries. From this respondent
pool, 68 organizations reported their
use of sensor and machine-tomachine data to develop an Internet
of Things analytical infrastructure.
The Internet of Things (IoT) has made the leap to become a
mainstream topic. This growing recognition is due to the impact
the IoT has had on business analytics and the potential that still
remains untapped. Each day, new machines, sensors, and
devices come online and feed information into data systems. As
organizations embark on new IoT initiatives and work to extract
more insight from swelling data volumes, a new data
management approach is called for. Traditional databases and
analytics architectures will always be vital, but the IoT calls for
specific capabilities to handle diverse data constantly streaming
from untold numbers of sources. IoT data is complex, vast, and
fast-moving. This report examines the current state of data
management and details the capabilities needed to manage IoT
data and maximize value.
-
RT
Adapt or Drown in Data
Aberdeen examined organizations with the ability to collect,
integrate, and analyze data generated by the Internet of Things.
These “IoT organizations” seek to leverage the glut of
information generated by disparate devices, systems, and other
sources to better understand operations and overall
performance. Past Aberdeen research has called for companies
to invest in improving infrastructure and data management
capabilities to handle the challenges and opportunities
presented by the IoT. Aberdeen’s survey of 68 IoT organizations
revealed the areas where organizations struggle and hope to
improve:
•
The average IoT organization’s total volume of data
grew by 30% over the past year.
•
54% of IoT organizations reported that their current
data analysis capabilities are insufficient.
•
50% of IoT organizations failed to improve time-todecision over the past year.
www.aberdeen.com
3
Sensors are more affordable than ever before and prices on
connected devices continue to drop. New devices and machines
that transmit data come online every day and full-scale data
generation from the IoT is now economically feasible.
Organizations that previously derived the majority of their
insight from transactional data are shifting their focus to IoT
data. All of this analytical development generates swelling data
volumes, with IoT organizations averaging 30% data growth
year-over-year. Other estimates put data growth rates higher
across all industries. Even conservatively, enterprise data will
double within three years.
Not only is data growing, it is also diversifying. More than half of
IoT organizations are concerned that their analytical tools and
infrastructure are not equal to modern data demands. Many
organizations lack the tools and infrastructure needed to
leverage non-traditional data formats, such as unstructured and
geospatial data. Decision makers know they have the data they
need, but they cannot yet convert it into insight. Organizations
staring down the daunting task of IoT data management seek
features that enable them to process, store, and analyze the
crush of information they can now generate. Finally, many IoT
organizations cannot currently react fast enough to streaming
data and have failed to improve time-to-decision over the past
year. As data accelerates, so does the pace of business. Even as
analysts work with huge data volumes and perform more
complex analysis, data-driven decisions need to be made faster.
IoT organizations need data management solutions that
facilitate rapid decisions, no matter how many end points are
involved.
-
Data Management for the Internet of Things
In an Internet of Things (IoT)
environment, nearly every object,
device, and consumer good is
connected to networks and / or the
public internet. These “things” or
smart objects can be individually
identified, tracked, and managed,
and can be connected to networks
through a variety of methods. In an
Internet of Things enabled business,
everything is connected, creating
new capabilities and increased data
awareness.
IoT organizations need
data management
solutions that facilitate
rapid decisions, no
matter how many end
points are involved.
Data growth
www.aberdeen.com
Data Management for the Internet of Things
4
Managing Data at the Edge
Data management is
moving from the central
data repository towards
the edge of the network.
As devices and sensors multiply and data volumes swell, legacy
data management infrastructure and techniques will no longer
be sufficient to fully leverage the IoT. IoT organizations
demonstrate the direction that data management needs to take.
Traditional centralized databases will always have a role to play
in analytics. However, as IoT initiatives continue to gain
momentum, data management is moving from the central data
repository towards the edge of the network (Figure 1). IoT
organizations are nearly twice as likely as all other organizations
to have automated data capture. These organizations embed
data management into the devices and sensors generating data
to facilitate a smooth and steady stream of information. Data is
managed as soon as it is generated. This is especially important
for real-time data feeds. Streaming data can be aggregated at
the edge and delivered to the central databases as averages of
manageable periods of time. As data constantly pours in from
sensors at the edge, the central database only has to handle the
influx at a controlled rate, such as once per minute. Also,
organizations still have the opportunity to employ tools that
perform real-time analytics on the comprehensive volume of
data in motion.
Figure 1: IoT Data Management Capabilities
n=205
Source: Aberdeen Group, September 2014
www.aberdeen.com
5
Data Management for the Internet of Things
Many IoT organizations add intelligence to the edge to
streamline data processing and management. Thirty-nine
percent (39%) of IoT organizations have automated data
indexing and classification, compared to 19% of all other
organizations. These organizations deploy algorithms to filter
data before it reaches a centralized database. Automated
indexing at the edge ensures that important data and timely
alerts are processed and delivered to the right decision makers.
Properly classified information is also easier for users to find and
incorporate into analysis, whether they are working with
streaming data or pulling historical data. Additionally,
impertinent and superfluous information is left at the edge and
kept from clogging up the central system. Organizations with IoT
aspirations should follow this example and invest in analytics at
the edge to enable better data management. Automated
aggregation and classification at the edge accelerates the
generation of insight from IoT data and protects databases from
overwhelming data volume and velocity.
Properly managed IoT data is most valuable when it is
connected and complemented with other data sources. IoT
organizations are 58% more likely than all others to have realtime data integration tools. Integrating IoT data in real-time
offers analysts immediate context and helps them uncover
correlations between current sensor data and historical data.
Analysts can ingest information as it arrives from any number of
sources. IoT data also becomes instantly accessible via multiple
interfaces. Aberdeen’s report, Big Data Becomes Fast Data with
Accelerated Integration, details the best practices of
organizations that have invested to get information to the right
people at the right time. Managing data at the edge does not
mean eliminating current database architecture. IT decision
makers should find a way to incorporate IoT-optimized data
 Related Research:
“Big Data Becomes
Fast Data with
Accelerated
Integration"
www.aberdeen.com
Data Management for the Internet of Things
6
management into their current systems. IoT data should gel with
cloud databases and all backend analytical platforms.
Handling Data Variety
 Related Research:
“The Analytical Eye
in the Sky:
Tracking LocationBased Insight"
Aberdeen examined the types of data that organizations are able
to manage and analyze. All of the survey respondents defined as
IoT organizations capture time series data from sensors and
other smart devices. Many IoT organizations have developed
data infrastructure flexible enough to handle additional data
types (Figure 2). The vast majority of IoT organizations work with
geospatial data, whereas just 27% of all other organizations
incorporate location information into their analysis. Geospatial
data can be integrated with time series data to source problems
and differentiate trends from localized anomalies. Time series
data provides the what and the when, but users need to know
the where to develop comprehensive insights in an IoT
environment. Aberdeen’s report, The Analytical Eye in the Sky:
Tracking Location-Based Insight, found that users with access to
geospatial data were more satisfied with the relevance of their
analytical capabilities to their job role. Users can focus on the
locations that fall under their purview rather than facing the
entire stream of IoT data.
Figure 2: Going Beyond Time Series Data
n=205
Source: Aberdeen Group, September 2014
www.aberdeen.com
7
As IoT organizations mature, they will be able to handle more
complex data types all within a cohesive data management
system. To gain the full value of IoT analysis, organizations need
to go beyond the standardized data formats they have handled
in the past. Many IoT databases are flexible enough to enable
analysis of unstructured data. The majority of IoT organizations
are able to analyze both internally and externally generated
unstructured data. Unstructured data offers intelligence that
traditional data formats cannot. Organizations must manage
this data so it can smoothly flow through analytical
infrastructure and be combined with structured data to provide
deeper intelligence. Aberdeen’s report, Unstructured Data and
the New Frontier of Fact-Based Insight, details improvements in
data visibility and time-to-decision achieved by organizations
using unstructured data.
User Demand for Better Data Management
Data Management for the Internet of Things
 Related Research:
“Unstructured
Data and the New
Frontier of FactBased Insight"
All of this work and investment around the Internet of Things has
its benefits. IoT organizations are still in the early stages of
proving out the value of their analytical efforts. Although IoT
organizations offer a glimpse into the best practices of the
future, they retain plenty of room for improvement. Users and IT
personnel at IoT organizations are generally dissatisfied with the
current state of analytics (Figure 3). Ease-of-use should be a
major area of focus for developing IoT organizations. Data
processes (such as data capture and indexing at the edge)
should be automated wherever possible. For example, in
manufacturing, IoT data is used for predictive maintenance.
Machines and sensors produce data that indicates where plant
floor managers should direct their attention and resources.
Users need to be able to easily access and analyze that data in
order to keep operations running smoothly. Also, facile
integration of machine performance data with geospatial data
www.aberdeen.com
Data Management for the Internet of Things
8
 Related Research:
“Data Visualization
for the Internet of
Things"
can alert decision makers if the same problems are all tied to a
single location. Intelligent data management around the IoT
ensures that new information is available at all times. This
steady stream of potential insights leads to optimized business
decisions. Recent Aberdeen research on the IoT revealed that
organizations with strong data interfaces and easy access to
streaming data markedly improved time-to-decision over the
past year.
Figure 3: “Satisfied” or “Very Satisfied” Users
n=205
Source: Aberdeen Group, September 2014
Less than a third of users and IT personnel are satisfied with their
data access. Users especially feel that there are troves of IoT
data they cannot get at. IoT organizations must invest to make
data available in multiple interfaces for easy access and analysis.
Aberdeen’s report, The State of Data Availability: All the Right
Data in all the Right Places, provides additional insights on how
best to manage and deliver large data volumes. Finally, IoT
organizations need to improve the speed of information delivery.
The faster analytical minds can spot issues and opportunities,
the faster the appropriate individuals can respond intelligently.
Mature IoT organizations manage data so that users can read
only pertinent information directly from the stream.
www.aberdeen.com
9
Data Management for the Internet of Things
Organizations should strive for real-time analytics that enable
users to work with data as it arrives from any number of sources.
For example, in telecommunications, IoT data from smart grids
will instantly reveal outages. The responsible parties need that
information immediately so they can minimize downtime and
keep customers happy. Better yet, rapid IoT data can reveal
early signs of lagging performance and the problem can be fixed
before it becomes a major issue that impacts customer
experience.
Key Takeaways
The current capabilities and analytical achievements of IoT
organizations are just the beginning. Enlightened organizations
will continue to invest to improve the processing, storage, and
querying of IoT data. Decision makers should consider the
challenges and best practices of data management for the IoT:
 Data is swelling and analytical demands are
mounting. The average organization’s data will double
within three years. As the flood of information deepens,
analytical minds call for additional capabilities and faster
access. IoT organizations are concerned that their current
analysis is insufficient and time-to-decision is not
improving.
 Live on the edge. IoT organizations are leading the way
in managing data before it is stored in a centralized
database. More mature organizations automatically filter
and classify data at the edge. This ensures that users get
pertinent information and prevents overwhelming
databases.
 Embrace diverse data types. IoT organizations are
significantly more likely than all others to have analytical
capabilities for geospatial and unstructured data.
www.aberdeen.com
Data Management for the Internet of Things
10
Combined with time series data, these data types offer
valuable context. Data management should be flexible
enough to handle diverse data and integrate with more
traditional information formats.
 Invest to satisfy users and IT. Currently, the majority of
users and IT personnel at IoT organizations are not
satisfied with their data access, the speed of information
delivery, and the ease-of-use of their data systems.
Demonstrated improvement in these three areas is an
excellent measure of value for potential data
management investments. Organizations should
consider automation initiatives to hasten data processes
and simplify the user experience.
The potential of the IoT is only as great as an organization’s
ability to manage data and fully harness the constant flow of
information.
For more information on this or other research topics, please visit www.aberdeen.com.
Related Research
Data Visualization for the Internet of Things;
December 2014
The State of Data Availability: All the Right Data in
All the Right Places; November 2014
Unstructured Data and the New Frontier of FactBased Insight; November 2014
The Internet of Things: Connecting the Enterprise
and the Customer; October 2014
Big Data Becomes Fast Data with Accelerated
Integration; August 2014
The Analytical Eye in the Sky: Tracking LocationBased Insight; July 2014
Author: Peter Krensky, Research Analyst, Analytics & Business Intelligence
(peter.krensky@aberdeen.com)
www.aberdeen.com
11
Data Management for the Internet of Things
About Aberdeen Group
For 26 years, Aberdeen Group has published research that helps businesses worldwide improve performance. We
identify Best-in-Class organizations by conducting primary research with industry practitioners. Our team of
analysts derives fact-based, vendor-agnostic insights from a proprietary analytical framework independent of
outside influence. The resulting research content is used by hundreds of thousands of business professionals to
drive smarter decision making and improve business strategy.
Aberdeen's content marketing solutions help B2B organizations take control of the Hidden Sales Cycle through
content licensing, speaking engagements, custom research, and content creation services. Located in Boston, MA,
Aberdeen Group is a Harte Hanks Company.
This document is the result of primary research performed by Aberdeen Group. Aberdeen Group's methodologies provide for objective fact-based research and represent the best
analysis available at the time of publication. Unless otherwise noted, the entire contents of this publication are copyrighted by Aberdeen Group, Inc. and may not be reproduced,
distributed, archived, or transmitted in any form or by any means without prior written consent by Aberdeen Group, Inc.
www.aberdeen.com