Interplay of Big Data and IoT - StampedeCon 2016

@TonyShan #IoTandBigData
Interplay of Big Data
and IoT
Tony Shan
July 27, 2016

© Tony Shan. All rights reserved. Info from multiple sources are adapted and incorporated. No distribution or reproduction is allowed without author’s prior written permission.Interplay of Big Data and IoT 2
State of the Art
Big Data Anatomy
IoT Anatomy
Synergy
Interplay
Case Study
Conclusion

Explosion of Data Size

Explosion of Device Quantity

Impact
50 billion devices
are predicted to
be on the
Internet by 2020.
The data volume
in the world is
projected to
reach 40
Zettabytes in
2020
IoT + Big
Data 
changing
the world

Big Data

Viewpoints
Big
Data
What
Which
Why
Where
When
Who
How
How
Much
7© Tony Shan. All rights reserved. Info from multiple sources are adapted and incorporated. No distribution or reproduction is allowed without author’s prior written permission.Interplay of Big Data and IoT

Concept of Big Data
8
Unstructured
Interactive and Machine-
generated Data
Structural and Semi-
StructuralTransaction
Data
Distributed parallel
processing with linear
scalability
© Tony Shan. All rights reserved. Info from multiple sources are adapted and incorporated. No distribution or reproduction is allowed without author’s prior written permission.Interplay of Big Data and IoT
Big
Data
What
Which
Why
Where
When
Who
How
How
Much

Characterization of Big Data
Big
Data
What
Which
Why
Where
When
Who
How
How
Much
Volume
• TB, PB, EB, ZB
• Records
• Transactions
• Files
Variety
• Structured
• Unstructured
• Semi-structured
• Multi-structured
Velocity
• Batch
• Sparse
• Interval
• Near Real-time
• Real-time

Key Benefits
Big
Data
What
Which
Why
Where
When
Who
How
How
Much

Use Cases
11
Celestial body
Exobiology
Inheritance
Sequence of
cancer
Advertisement
Finding
communities
SNA
Finding
communities
Data Mining
Consuming habit
Changing router
Content delivery
© Tony Shan. All rights reserved. Info from multiple sources are adapted and incorporated. No distribution or reproduction is allowed without author’s prior written permission.Interplay of Big Data and IoT
Big
Data
What
Which
Why
Where
When
Who
How
How
Much

Evolution
Big
Data
What
Which
Why
Where
When
Who
How
How
Much

Big
Data
What
Which
Why
Where
When
Who
How
How
Much
Who

Pragmatic Approach
Foundation Applicability Strategization Taxonomy Tooling Roadmap Architecture Convergence Knowledgebase
Big
Data
What
Which
Why
Where
When
Who
How
How
Much

TCO for 5Years
Big
Data
What
Which
Why
Where
When
Who
How
How
Much

IoT

Viewpoints
IoT
What
Which
Why
Where
When
Who
How
How
Much

Concept
Internet of things
Syllabification (Inter•net of things)
noun
A proposed development of the Internet in which
everyday objects have network connectivity, allowing
them to send and receive data:
[Oxford Dictionary]
IoT
What
Which
Why
Where
When
Who
How
How
Much

Characterization
Devices Things, in the IoT, can refer to a wide variety of devices such as heart monitoring
implants, biochip transponders on farm animals, electric clams in coastal
waters, automobiles with built-in sensors, or field operation devices that assist fire-fighters
in search and rescue. Current market examples include smart thermostat systems and
washer/dryers that utilizeWiFi for remote monitoring.
Connectivity IoT is expected to offer advanced connectivity of devices, systems, and services that goes
beyond machine-to-machine communications (M2M) and covers a variety of protocols,
domains, and applications.The interconnection of these embedded devices (including smart
objects), is expected to usher in automation in nearly all fields, while also enabling advanced
applications like a Smart Grid.
Big amount
of info
Besides the plethora of new application areas for Internet connected automation to expand
into, IoT is also expected to generate large amounts of data from diverse locations that is
aggregated and very high-velocity, thereby increasing the need to better index, store and
process such data.
IoT
What
Which
Why
Where
When
Who
How
How
Much

Why
IoT
What
Which
Why
Where
When
Who
How
How
Much

Where
IoT
What
Which
Why
Where
When
Who
How
How
Much

Use Cases
• GlowCaps fit prescription bottles and via a wireless chip provide services
that help people stick with their prescription regimen; from reminder
messages, all the way to refill and doctor coordination.
REMEMBERTO
TAKEYOUR MEDS
• Smart thermostats like the Nest use sensors, real-time weather forecasts,
and the actual activity in your home during the day to reduce your monthly
energy usage by up to 30%, keeping you more comfortable, and offering to
save you money on your utility bills.
HEATYOUR
HOME
EFFICIENTLY
• With the use of installed sensors, mobile apps, and real-time web
applications like those provided in Streetline’s ParkSight service, cities can
optimize revenue, parking space availability and enable citizens to reduce
their environmental impact by helping them quickly find an open spot for
their cars.
STOP DRIVING IN
CIRCLES
IoT
What
Which
Why
Where
When
Who
How
How
Much

IoT
What
Which
Why
Where
When
Who
How
How
Much

Who
IoT
What
Which
Why
Where
When
Who
How
How
Much

How
IoT
What
Which
Why
Where
When
Who
How
How
Much

How much
Source: survey byVodafone
IoT
What
Which
Why
Where
When
Who
How
How
Much

Big Data
and IoT

4Vs meet 4Cs
Big Data
• Volume
• Velocity
• Variety
• Veracity
IoT
• Connectivity
• Collection
• Context
• Cognition
Large
amounts
Real-time
Structured
and
unstructured
Uncertain
provenance
Connection
to network
Data
processing
Situational
semantics
Intelligence
to action

Synergy
Source: Date generation from IoT
Source: Data collection from IoT
Source: Predictive analytics in Big
Data
Source: Deep learning in Big Data
IoT Big
Data
Target: Data store in Big Data
Target: Data aggregation in Big
Data
Target: Edge analysis at IoT
Target: Intelligent action at IoT

Mapping
Connectivity Collection Context Cognition
Volume Total bandwidth to
handle the data pipeline,
embedded capacity
Sheer amount of data
generated and collected
in sensors, mobile
devices, wearables, etc.
Sort out data, cleansing,
ingest, extract,
transform, load
New insights via
descriptive, diagnostic,
predictive and
prescriptive analysis
Velocity Network latency, high
speedWi-Fi, Internet
connection, HaLow
One-way traffic from
devices to backend
systems in fast
requests/responses
Analysis in timely
processing, streaming,
near real time, seamless
scale-out
Just-in-time analytics,
on-demand intelligence,
hybrid
Variety Flexibility on different
protocols, e.g. Http,
CoAP, MQTT, webSocket
Highly diverse formats of
data, heterogeneous
devices, proprietary apps
Nontraditional data -
clickstream, location,
user behavior, emotion
Sift through all data for a
360◦ view and conduct
persuasive analytics
Veracity Shut down devices or
disconnect the network
connection as needed
Shield the noise or
bad/useless data
Filter dirty/false data and
correlate applicable data
for semantics
Yield smart
recommendations
with engaging, relevant,
and trustworthy sources

Healthcare examples
• Personal medical records, radiology images, clinical trial data FDA
submissions, human genetics and population data genomic sequences
Volume
• Data is accumulated in real-time and at a rapid pace
• The constant flow of new data accumulating at unprecedented rates
Velocity
• The structured data in EMRs and EHRs include familiar input record fields
such as patient name, data of birth, address, physician’s name, hospital
name and address, treatment reimbursement codes, and other information
easily coded into and handled by automated databases
• The point of care generated unstructured data: office medical records,
handwritten nurse and doctor notes, hospital admission and discharge
records, paper prescriptions, radiograph films, MRI, CT and other images
Variety
• Data assurance: error-free and credible
• Inaccurate “translations” of poor handwriting on prescriptions
• Is this the correct patient/hospital/payer/reimbursement code/dollar
amount? Other veracity issues are unique to healthcare: Is the info captured
correctly: diagnoses, treatments, prescriptions, procedures, and outcomes?
Veracity
• Wearable device, e.g. Fitbit
• Bluetooth
• Wi-Fi/USB
Connectivity
• Data stored locally
• Raw data on edge
• Aggregated data pushed to cloud
Collection
• Tracking activities for health and fitness
• Time and places of sleep and exercises
• Related activities
Context
• Identifying patterns and issues
• Monitoring red flags
• Suggest recommending actions
Cognition

Case Study – Connected Cars

Connected Cars Solution

Big Data is about
processing
voluminous data, and
extracting insights from
the correlated info.
IoT is about connecting
devices to generate
useful data streams,
and taking intelligent
actions via devices.
In a nutshell

Conclusion

• © Copyright Tony Shan. All rights reserved. All materials, content
and forms contained in this presentation are the intellectual
property of Tony Shan and may not be copied, reproduced,
distributed or displayed without author's express written permission.
• Other streams of data and information from Internet are adapted
and incorporated in the presentation for reference and illustration
purposes. Some sources are not mentioned on the slides due to
space and time constraints.
• The author does not warrant, either expressly or implied, the
accuracy, timeliness, or appropriateness of the information
contained in this deck. The author disclaims any responsibility for
content errors, omissions, or infringing material, and disclaims any
responsibility associated with relying on the information provided in
this document. The author also disclaims all liability for any material
contained in other resources linked to this file.
•Contact:
mail@TonyShan.com

Interplay of Big Data and IoT - StampedeCon 2016

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Interplay of Big Data and IoT - StampedeCon 2016 (20)

More from StampedeCon (20)

Recently uploaded (20)

Interplay of Big Data and IoT - StampedeCon 2016