Can IoT Disrupt the Product-based Business Model in Light of 5G? Dr Shoumen Palit Austin Datta Research Affiliate, School of Engineering, MIT ● Massachusetts Institute of Technology Senior Vice President, Industrial Internet Consortium ● www.iiconsortium.org Eliminate “roadmap” and embrace “compass” The Principles and Practice of Connectivity The Elusive Quest for Monetization Executive Summary COM ANALYTICS EDGE SERVICE COGNITION TIME IoT CPS It is likely to be an evolution … potential for revolution is slim. 5G What is absent from this illustration? HEALTHCARE 5G Re-visiting POTS? Issues and Politics on the Road Ahead D S R C WiFi 4G L T E https://docbox.etsi.org/workshop/2014/201402_ITSWORKSHOP/S04_RADIO/TOYOTA_SPAANDERMANforKENNEY.pdf CR Frequency GHz www.slideshare.net/zahidtg/ltebased-unlicensed-carrier-offloading-huawei http://s3.amazonaws.com/sdieee/205-LTE+Direct+IEEE+VTC+San+Diego.pdf http://bit.ly/LTE-D http://bit.ly/LTE-V Social integration of autonomous vehicles with public road traffic? 2033-2035 The Wealth of Nations ● Nature of the Firm (Transaction Cost Economics) Economic history and data related to Textile, Railway, Automobiles and Computers taken from work by Norman Poire Adoption 1853 Textile 1800 1913 Railway 1853 1969 Auto 1913 2020 Computer 1969 2061 Agents ML 2005 2081 Nanotech DLV, CPS 3DP / IoT 2020 2100 IoT / IIoT Cyberwar Hydrogen Autonomous Cars, Fusion 2040 2035 1771 1825 Technology Surfaces Industrial Revolution Atoms 1886 1939 1977 1959 AI 1991 2010 1995 2007 Public Autonomous Internet Vehicles Systems IoS Knowledge Driven Economies of Scale Bits It takes about 28-30 years for an idea to be socialized before it is accepted and adopted. 1999 was the birth year for IoT concept. Expect exponential growth of IoS ~ 2025-2026. Simple Problem EXAMPLE plastic bag How does an autonomous vehicle understand the difference between an object without threat in a run time collision avoidance context? Without algorithmic solutions, even a harmless plastic bag in the air may cause an accident. Oh I see a plastic bag What is the “brain” of the autonomous vehicle thinking? Where is the computational brain? In the cloud? In the fog? In the mist? Critical tools for real-time image identification and semantic context of object Neural Image Caption (NIC) Generator Translates images to natural language http://arxiv.org/pdf/1411.4555v1.pdf To translate languages, Recurrent Neural Network (RNN) transforms a French sentence into a vector representation, and a second RNN uses that vector representation to generate a target sentence in German. Replace first RNN and input words with deep Convolutional Neural Network (CNN) trained to classify objects in images and add known classes of objects in semantic baffles with corresponding behavior (plastic bag versus wooden plank) with assigned probability of object in the image (environment). Feed CNN’s rich encoding of the image into a RNN designed to produce phrases. We can then train the whole system directly on images and their captions, so it maximizes the likelihood that descriptions it produces best match the training descriptions for each image. The natural language spoken by human (inside vehicle) better trains the algorithms. Author’s idea is adapted from → http://googleresearch.blogspot.co.uk/2014/11/a-picture-is-worth-thousand-coherent.html Mist Computing “Edge Intelligence” “Mist Computing” doesn’t exist. It’s a suggestion by the author. Instantiate Open Mist to complement Open Fog + Open Cloud. Siamese Networks – Paraphrase Detection Siamese Networks – Paraphrase Detection T This Thin is a bags plastic may be harmless bag to vehicle Siamese Networks – Paraphrase Detection T This is a plastic bag Attributes Semantics Semantics Thin Semantics bags may be Semantics harmless Semantics to vehicle Connected Vehicle Mist Computing Tool Support Vector Machine SVM distinguishes gazelles, ostrich, trees and ground in Namibia, Africa www.cs.toronto.edu/~hinton/csc2515/notes/lec10svm.ppt www.epfl.ch ▪ Patrick Meier at www.qcri.com Support Vector Machines for ITS in China 5G Latency 2ms Jitter ?? Semantics of Time Edge Intelligence (CNN, RNN) Real-Time Convergence V2V Intersection Movement Assist Warning Scenario DSRC APPLICATIONS PRIVATE PUBLIC SAFETY • • • • • • • • • • • • • • • • • • • • • • • • • • • APPROACHING EMERGENCY VEHICLE (WARNING) ASSISTANT (3) EMERGENCY VEHICLE SIGNAL PREEMPTION ROAD CONDITION WARNING LOW BRIDGE WARNING WORK ZONE WARNING IMMINENT COLLISION WARNING (D) CURVE SPEED ASSISTANCE [ROLLOVER WARNING] (1) INFRASTRUCTURE BASED – STOP LIGHT ASSISTANT (2) INTERSECTION COLLISION WARNING/AVOIDANCE (4) HIGHWAY/RAIL [RAILROAD] COLLISION AVOIDANCE (10) COOPERATIVE COLLISION WARNING [V-V] (5) GREEN LIGHT - OPTIMAL SPEED ADVISORY (8) COOPERATIVE VEHICLE SYSTEM – PLATOONING (9) COOPERATIVE ADAPTIVE CRUISE CONTROL [ACC] (11) VEHICLE BASED PROBE DATA COLLECTION (B) INFRASTRUCTURE BASED PROBE DATA COLLECTION INFRASTRUCTURE BASED TRAFFIC MANAGEMENT – [DATA COLLECTED from] PROBES (7) TOLL COLLECTION TRAFFIC INFORMATION (C) TRANSIT VEHICLE DATA TRANSFER (gate) TRANSIT VEHICLE SIGNAL PRIORITY EMERGENCY VEHICLE VIDEO RELAY MAINLINE SCREENING BORDER CLEARANCE ON-BOARD SAFETY DATA TRANSFER VEHICLE SAFETY INSPECTION DRIVER’S DAILY LOG • • • • • • • • • • • • ACCESS CONTROL DRIVE-THRU PAYMENT PARKING LOT PAYMENT DATA TRANSFER / INFO FUELING (A) – ATIS DATA – DIAGNOSTIC DATA – REPAIR-SERVICE RECORD – VEHICLE COMPUTER PROGRAM UPDATES – MAP and MUSIC DATA UPDATES – VIDEO UPLOADS DATA TRANSFER / CVO / TRUCK STOP ENHANCED ROUTE PLANNING & GUIDANCE (6) RENTAL CAR PROCESSING UNIQUE CVO FLEET MANAGEMENT DATA TRANSFER / TRANSIT VEHICLE (yard) TRANSIT VEHICLE REFUELING MANAGEMENT LOCOMOTIVE FUEL MONITORING DATA TRANSFER / LOCOMOTIVE ATIS - Advanced Traveler Information Systems CVO - Commercial Vehicle Operations EV - Emergency Vehicles IDB - ITS Data Bus THRU – Through V-V – Vehicle to Vehicle (#) – Applications Submitted by GM/Ford/Chrysler (A- Z) – Applications Submitted by Daimler-Chrysler Basic Safety Message - SAE J2735 ● www.sae.org/standardsdev/dsrc/ 5G Latency 2ms Jitter ?? Semantics of Time Edge Intelligence (CNN, RNN) Without time synchronization (time-centric software) the promise of 5G may be limited in execution of CPS Failure of abstraction Semantics of time absent from instruction set architecture (ISA) Prof Edward A Lee, UC Berkeley 37 Autonomous Vehicle Medical Device Prof Edward Lee, UC Berkeley ● www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-8.html Essentially these are all sensor / communication networks Cloud Mist Fog Neonatal ICU at Oulu Health Labs http://bit.ly/OULU-HEALTH-2015-CNN Single Incision Laparoscopic Surgery DOI 10.1109/TMECH.2015.2506148.IEEE/ASME WSN In-Network Processing Query Result www.cs.colorado.edu/~rhan/CSCI_7143_001_Fall_2003/agr24.ps WSN In-Network Processing WSN In-Network Processing EDGE INTELLIGENCE EYERISS Convolutional Neural Networks – At the Edge http://arxiv.org/pdf/1411.4555v1.pdf To translate languages, Recurrent Neural Network (RNN) transforms a French sentence into a vector representation, and a second RNN uses that vector representation to generate a target sentence in German. Replace first RNN and input words with deep Convolutional Neural Network (CNN) trained to classify objects in images and add known classes of objects in semantic baffles with corresponding behavior (plastic bag versus wooden plank) with assigned probability of object in the image (environment). Feed CNN’s rich encoding of the image into a RNN designed to produce phrases. We can then train the whole system directly on images and their captions, so it maximizes the likelihood that descriptions it produces best match the training descriptions for each image. The natural language spoken by human (inside vehicle) better trains the algorithms. Author’s idea is adapted from → http://googleresearch.blogspot.co.uk/2014/11/a-picture-is-worth-thousand-coherent.html 5G Latency 2ms Jitter ?? Semantics of Time Edge Intelligence (CNN, RNN) Real-Time Convergence Transportation Coordination - Emergency “Crash to Care” Response Transportation of real-time data key to emergency search and rescue drones US Park Service Drones US DHS Drone US Fire Service Drone Time, Sensors, Network & Data Synchronization Dhananjay Anand, NIST 5G in other modes of transportation RAIL ENGINE On-Board 5G 5850-5.925 GHz Multi-Application OBU (360 deg horizontal pattern) Built-in and connected to an SAE1708, 1939, or EIA-709 LonWorks interface bus Interface Devices (Built-in Display, Annunciator, Microphone, Keypad, etc. connected to SAE-1708, 1939 or EIA-709 LonWorks interface bus) Freight train arrives in Tehran (Feb 15, 2016) after travelling 14 days from China www.scmp.com/comment/insight-opinion/article/1913441/chinas-new-silk-road-designed-cut-russia-out-eurasian-trade Data is the new coal US $1 Artificial Intelligence of Autonomous Driving BILLION 56 4 SEP 2015 Emerging Frontier Data Curation – sorting out what we need CURATE MONEY Innovation in Curation Algorithms How to extract ambient intelligence? Postgres Introduced the object-relational model, effectively merging DB with abstract complex data types eg CAD, geospatial, so-called big data www.csail.mit.edu/node/2459 32-G922 CSAIL MIT on 9th April 2015 Photograph taken by Shoumen Datta Michael Stonebraker Turing Award 2015 Challenges in Data Curation ● Noise obscures signal ● Data acquired is a blend of noise with signal ● Signal volatility introduces noise which is often proportional to signal → How do we correct/reduce the error due to this “noisy channel” factor? → Can novel algorithms reduce/deconstruct data to subtract “noise” and reconstruct the signal? → What about the application of the principles of (Shannon, Kalman-Bucy) error correcting algorithms? https://en.wikipedia.org/wiki/Kalman_filter http://news.mit.edu/2010/explained-shannon-0115 http://www.cs.cmu.edu/~guyb/realworld/errorcorrecting.html http://www.cs.cmu.edu/~aarti/Class/10704/lec16-shannonnoisythrm.pdf Data Curation Concepts from Laminar Flow Data is not an unique issue in this context. This is applicable across all data types and domains. This is a data curation problem. We are observing related signal/noise issues in big data analytics. Are there any concepts related to data curation which may be triggered by laminar flow? Watch: http://bit.ly/LAMINAR-FLOW-DATA-CURATION-CONCEPT Data Saves Lives 2.5 million falls 2013 734,000 hospitalized 25,500 died from fall $34 billion direct cost 66 Professor Dina Katabi (MIT) presenting RF Reflection to President Obama (White House Demo, 4 August 2015) WHITE HOUSE VIDEO ● http://bit.ly/President-Obama-with-Dina-Katabi http://newsoffice.mit.edu/2015/president-obama-meets-mit-entrepreneurs-white-house-demo-day-0806 You can’t build an elephant using the mouse as a model The pursuit of “large scale” test beds and deployments • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • •• • •• • • • • • • • • • • • •• • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • • • • • • • • • • •• • •• •• •• • • • • • •• • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • •• •• •• • ••• • • • • • • • • • • • • • • • •• • •• • • • • • • • • • • • • •• • • •• • • • • • • • • • • • • • • •• • • • •• • • • • • • •• • • •• • • • •• • • •• • •• • • • • • • • • •• • • • • •• • • • • • •• • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • •• • • • • • • •• •• • • • • • • • • • • • •• •• • • •• • • • • • • • • • • • • • • • • • • • • • • • •• • • • • •• • • • • • • •• • • •• • • • •• • • • • • • • •• • • •• • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • •• •• • • •• • • • • • • •• • • • • • •• • • • • • • • • • • • • • • • •••• •• • • • • • • • • • • • •• • ••• • • • •• • • • • • • • • • • • • • •• • • • • • • • • • • ••• • • • • • • •• • • • •• • ••• • • • • • • • • • • • •• • • • • •• • • •• • • • • • • • • ••• • • • • • • •• •• • • •• • •• •• • • ••• • • •• • • • • • • • • •• • • • • • • •• • • • • • • • • • • • • • • •• • • • • • • • • • •• • • • • • • • • • • • • • • •• • • • •• • •• • • •• •• • • •• •• • ••• • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • •• • • • • • • • • •• ••• ••• • • • • • • • • •• • • • • • •• • • • • • •• • •• • • • •• • • •• • • • • • • • • • • • •• • • •••• •• • •• • • • •• • • • • •• • •• • • • • •• •••• •• • • •• •• • • •• • ••• •• • • •• • • •• • • • • • ••• • ••• ••• • • • • • •• • • • • • •• •• • • • • • • • • • • • • • •• • • • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • •• • • • • • • • • • • • • • • • • •• ••• • • • • • • • • •• • • ••• • • • • • • ••• •• • • • • • • • • • •• • • • • •• •• •• • • • • •• • •• •• • •• • • •• • • • •• • • • • • • • • • • • ••• • • • • • • • • •• • • •• • • • •• • •• • • •• • • • •• • • • • •• • •• • • • • •• • • • •• • • • • • •• • • •• •• • • • • • •• • • • • • • •• • • • • •• • •• • • • • • • • •• • • •• •• • • • • • • • • ••••• • • • • • • • • • • • • ••• • •• • • •• • • • • • • • •• • • • • • • • • • • • • • ••• •• • • •• • • • • • • • • • • • • • • • • • •• Young’s Double Slit Experiment with Electrons Dr. Akira Tonomura, Hitachi Research Laboratories, 1-280, Higashi-Koigakubo, Kokubunji-shi, Tokyo 185-8601, Japan • • •• •• • • • • • • •• • • • • •• • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • •• • •• • • • • •• • •• • • • • •• • • • • • ••• • • • •• • • • • • • • • • •• • • •• • • •• • • • • • • • • • •• • • • • •• • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • •• •• • • • • • • •• • • • • • • • • • • • • • • • • •• • • •• • •• • • •• •• • • • • • • • • • • • • •• • • • • • • • • • • • • • •• • • •• • •• • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • •• • • • • • • •• • • •• • • • • •• •• • •• • •• • • • • • • • • • • • • • • • • • • • •• •• • • •• •• • • • • •• • • • • • • • • • • • • • • •• • • • •• • • • • • • • •• • • • • • • • • • • • • • • • • •••• •• • • • • • • • • • • • • •• • ••• • • •• • • •• • • • • • • • •• •• • • •• • • • • • • •• • •• • • • •• • • •• • ••• • • • • • • • • • •• • • •• • • • •• • • • • • • • • • • • • ••• • • • • •• • •• •• • • •• • •• • •• • • •• • • • •• • • • • • • • •• • • • • • • •• • • • • • • • • • • • • • • •• • • • • • • • • • • •• • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • •• • • •• •• • ••• • • • •• • • • • • • • • • • • •• • • • • •• • • •• • • • • • • • •• •• • • • •• • • • • • • • • •• •••• ••• • • • • • • •• • • • •• • • •• • • •• • • •• • • • • •• • •• • • •• • • • • •• •• • • • • • •• • •• • •• • • • •• • • • • •• • •• •• • • •• •••• •• • • •• •• • • • • •• •• • • • •• • • •• • • • • • • • • •• •• • • • • • • • • •• • • • • •• •• • • • • • • • • • • • • • •• • • • • •• •• • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • •• • •• • • • • • • • • • • • • • • • •• • • • •• •• ••• • • • • • • • • •• • •••• • • • • • • •••• • • • • • • • • • • •• • • • • • •• •• •• • • • • •• • •• •• • • • • • •• • • • •• • • • • • • • • • • • • •• • • • • • • • • •• • • •• • • • •• • •• • •• • •• • • •• • • • •• • •• • • • • •• • • • • • • • • • • •• • • ••• •• • • • • • •• • • • • • • •• • • • • • •• • •• • • • • • • •• • • •• •• • • • • • • • • ••••• • • • • • • • • • • • ••• • • •• • • • •• • • • • • • • •• • • • • • • • • • • • • • ••• •• • • • • • • • • • • • • • • • • • http://blogs.scientificamerican.com/guest-blog/what-does-the-new-double-slit-experiment-actually-show/ slit double slit Content restricted to IIC Members Not for External Publication 73 Data is the new coal Autonomy and Algorithms are essential catalysts The business model of communication is inextricably linked to service. http://gsacom.com/wp-content/uploads/2015/11/151118-GSA-the-Road-to-5G.pdf For more information http://bit.ly/MIT-IOT This URL will take you to my MIT Libraries Dspace page at MIT Please scroll down the page (long list of items) to the bottom Find 5G-MWC - zipped folder with key references and papers Find 5G-MWC-TALK - this presentation (a complete copy) Find REVIEW-IOT if you are interested in the big picture Dr Shoumen Datta ● shoumen@mit.edu Dr Shoumen Datta ● datta@iiconsortium.org Thank you I have created nothing new This document suggests ideas which are neither original nor the outcome of the author’s research or creativity. The synthesis of existing facts and weaving them to provoke dialogue may be attributed to the author. The author has no claim or rights over the data, visuals and graphics used in this document. The material is sourced from the world wide web and expressly used for the sole purpose of explaining thoughts presented in this document. This presentation may be shared with anyone and disseminated or used for any non-commercial purpose. The comments and opinions are solely due to the author. (shoumen@mit.edu)