From Data Visualization to Visual Analytics

Dr. Andreas S. Maniatis

Commercial Manager / Head of BI
CyberStream LTD

Athens Big Data Meetup

Friday, March 18th 2016
The CUBE – Athens, Greece
CyberStream LTD
The human face of technology …

Systems Integrator
Business process
Established in 2000 automation

Strong software
engineering skills Digital
Unified
communications
Signage
Focus on development
of custom & reusable solutions
Partner relationship with
established technology
companies Enterprise
E-Learning networking
Involvement in ΒΙ-DW-
Analytics since 1997
Portals

© Copyright 2000-2016 CyberStream LTD

Agenda

PART I
Visualization in Big Data [Business Intelligence, Business Analytics]

PART II
Data Visualization Techniques and Practices

PART III
Visual Analytics in Practice

© Copyright 2000-2016 CyberStream LTD

Part I
Visualization in Big Data [Business Intelligence,
Business Analytics]
Issues to be discussed

• The Big Data hype …

• The many Vs for Big Data

• Some definitions on Visualization and Analytics

• Visualization concepts and Principles

© Copyright 2000-2016 CyberStream LTD
The Data Science Hype Cycle [2014]

Gartner Hype Cycle

http://www.gartner.com/technology/research/methodologies/hype-cycle.jsp

© Copyright 2000-2016 CyberStream LTD

The Data Science Hype Cycle [2015]

Gartner Hype Cycle

http://www.gartner.com/technology/research/methodologies/hype-cycle.jsp

© Copyright 2000-2016 CyberStream LTD

Key usability differentiators for Visual
Analytics (I)

•Factors demanding for usability differentiators (I):

• Flood of data – The “BIG DATA” era:
• Social Networks, Media (Facebook, Twitter, LinkedIn etc.)
• Search engines (Google etc)
• Variety of new data types / streams:
• 3 (4) V’s: Volume, Velocity, Variety, (Veracity)
• More V’s: Viability, Value, Volatility, Virtualization …
• 1 V only: Variety (Damianos Chatziantoniou)

© Copyright 2000-2016 CyberStream LTD

IBM: Big Data Defined by 4 Vs

IBM Data Hub

http://www.ibmbigdatahub.com/sites/default/files/infographic_file/4-Vs-of-big-data.jpg

© Copyright 2000-2016 CyberStream LTD

IBM: Big Data Defined by 5 Vs

IBM Data Hub

http://www.ibmbigdatahub.com/infographic/extracting-business-value-4-vs-big-data

© Copyright 2000-2016 CyberStream LTD

“V” for Visualization

•Why Visualization is the biggest “V” of them all:

• Visualization is the key to making Big Data an integral part
of decision making
• Visualization is the only way to make Big Data accessible
to a large audience
[Storytelling and Narration]
• Visualization is essential to the analysis of Big Data so it
can be of highest value

© Copyright 2000-2016 CyberStream LTD

Key usability differentiators for Visual
Analytics (II)

•Factors demanding for usability differentiators (II):

• Faster cycle times
• New devices (Smartphones, Tablets, Touch and Voice
Activated Displays, Video Walls etc.)
• The introduction of Analytics by itself
• Involve more power users in exploratory data analysis

© Copyright 2000-2016 CyberStream LTD

Biology in Visual Perception

© Copyright 2000-2016 CyberStream LTD

The importance of senses

•Seeing 83 percent
•Hearing 11 percent
•Touching 3 ½ percent
•Smelling 1 ½ percent
•Tasting 1 percent
•6th Sense … ?
Source: Oklahoma State University
http://www.oces.okstate.edu/washita/uploaded_files/4h_Learning_Styles.doc

© Copyright 2000-2016 CyberStream LTD

The importance of context

The Washington Metro Experiment with Joshua Bell

http://www.washingtonpost.com/lifestyle/style/joshua-bell-is-playing-in-the-metro-again-this-time-you-can-plan-to-be-there/2014/09/23/7a699e28-4282-11e4-9a15-137aa0153527_story.html

© Copyright 2000-2016 CyberStream LTD

A Rabbit or a Duck?

© Copyright 2000-2016 CyberStream LTD

What is Data Visualization

Kim Rees - Founding partner of Periscopic, a socially-conscious data visualization firm

https://about.me/krees

© Copyright 2000-2016 CyberStream LTD

The Art of Visualizing Data

© Copyright 2000-2016 CyberStream LTD

Visual Analytics: A Definition

“... the science of analytical reasoning

facilitated by interactive visual interfaces …”

Relative Terms:
• Visual Analysis
• Visual Data Analysis
• Visual Data Mining
Illuminating the Path: Research and Development Agenda for Visual Analytics
James J. Thomas, Kristin A. Cook
IEEE Press, 2005

© Copyright 2000-2016 CyberStream LTD

The Scope of Visual Analytics

© Copyright 2000-2016 CyberStream LTD Source: http://www.visual-analytics.eu/faq/

Research Fundamentals for Visual
Analytics

© Copyright 2000-2016 CyberStream LTD Source: http://www.visual-analytics.eu/faq/

The Power of Visual Analytics

1281736875613897654698450698560498286782
9809858453822450985645894509845098096565
9091830208805989595772588875050678904567
8845789809821677654872664908560912949686

1281736875613897654698450698560498286782 =8
9809858453822450985645894509845098096585
980985845382245098564589450984509809656 =8
9091830208805989595772588875050678904567 =8
8845789809821677654872664908560912949686 =8
32

© Copyright 2000-2016 CyberStream LTD

Part II
Data Visualization Techniques and Practices
Issues to be discussed

• Some History …

• Data Visualization Techniques

• Visualization practices to avoid

• The Visual Analytics Mantra

• Interactive Dynamics for Visual Analysis

© Copyright 2000-2016 CyberStream LTD

Some History (1)
Map from 600 BC - Babylon

© Copyright 2000-2016 CyberStream LTD

Some History (2)
Meet … Jon Snow (1 of 2)

© Copyright 2000-2016 CyberStream LTD

Some History (2)
Meet … John Snow (2 of 2)

© Copyright 2000-2016 CyberStream LTD

Some History (2)
Cholera Outbreak in London (1854)

Sources: “The Guardian” Interactive Map

http://www.theguardian.com/news/datablog/interactive/2013/mar/15/cholera-map-john-snow-recreated
© Copyright 2000-2016 CyberStream LTD
Some History (4)
Napoleon’s March to Moscow (C.J.Minard, 1869)

© Copyright 2000-2016 CyberStream LTD

 Sources: http://www.extremepresentation.com
http://annkemery.com/essentials/ | http://labs.juiceanalytics.com
Visualization Practices to Avoid (1)

Can you interpret the Chart? This is better …

3D Bar Chart Bar Chart

40

35
35

30
30

25
25

20 20

15 15

10 10

5 5

0 0
A B C A B C
Creating More Effective Graphs
Naomi B. Robbins
Statistical Society of Ottawa, Ottawa – Ontario, December 8th 2009

© Copyright 2000-2016 CyberStream LTD

Visualization Practices to Avoid (2)

Can you sort the data? This is better …

Pie Chart Bar chart sorting

40

35

30

25

20

15

10

0
A B C D E C E B D A
Creating More Effective Graphs
Naomi B. Robbins
Statistical Society of Ottawa, Ottawa – Ontario, December 8th 2009

© Copyright 2000-2016 CyberStream LTD

Visualization Practices to Avoid (Final)

© Copyright 2000-2016 CyberStream LTD

Visualization Practices to Follow (2)

Gauges? Bullet Graph - This is better …

Gauges: The Black Sheep of Data Visualization

http://www.dundas.com/blog-post/gauges-the-black-sheep-of-data-visualization/

© Copyright 2000-2016 CyberStream LTD

Devour the Pie Chart

© Copyright 2000-2016 CyberStream LTD

Visualization Practices to Follow (Final)

Curious, what's your preference

If city-specific of these 4is
comparison ways
not to show Uber
needed, thenvwhy
Taxi over-complicate?
wages? Know a better way?

Sometimes ONE Number Tells the Whole Story

© Copyright 2000-2016 CyberStream LTD

The Visual Analytics Process

© Copyright 2000-2016 CyberStream LTD Source: http://www.visual-analytics.eu/faq/

Part III
Visual Analytics in Practice
Issues to be discussed

• Requirements for a Data Analytics Platform Architecture

• New methodology for Visual Applications

• Live Visual Analytics with TIBCO Spotfire

• The Data Scientist as a Story Teller

© Copyright 2000-2016 CyberStream LTD

We are surrounded by data, yet most people
still make decisions without it
“ …visualization-based data discovery tools have far-reaching implications for how
“
business information is consumed….end-user organizations should adopt use as a way to
improve the success of their BI program. - Gartner, June 2011
Technical Differentiators

Enterprise-Class
Dimension-Free Predictive &
Data Exploration Event Driven

• Unmatched
Contextual Performance
• Massive
• Visual Data Mashup Collaboration Scalability
• Interactive • Data at Rest & In Motion • 24x7
• No Constraints • Open Source & 3rd Party Expertise
• “Two Second Advantage”

• Combine Data Sources • Bookmarks

• No Scripting • Guided Apps
• IT Free • Portals & Social Platforms

© Copyright 2000-2016 CyberStream LTD

IT Centric BI

IT Business
Developer User
Gather
Source Load Specifications
the data the data

Design
Data Models Publish
ERP To User

Data BI
E
Warehouse Platform
CRM T
Meta
L Data Build
Reports
OLAP Construct
BI Metadata
Cube
SCM
If specifications change, repeat the process!

© Copyright 2000-2016 CyberStream LTD

Business User centric Analysis

IT Business
Developer User
Gather
Source Load Specifications
the data the data
… and/or
Multiple
Design
Data Sources
Data Models Publish
ERP To User
Publish/Share
w/ others
Connect to
Data Data Source BI
E
Warehouse Platform
CRM T
Meta
L Data & Build
Interact
Visualize Data Reports
OLAP Construct
BI Metadata
Cube
SCM
If specifications change, repeat the process!

© Copyright 2000-2016 CyberStream LTD

Case Study
Environmental Analytics
Problem Description …

• Best Pilot Data Visualization on Air Quality in Athens

• Sample data from 2 measuring stations

• Want to locate “Outliers” and gain “Insights”

• Build an Interactive Visualization Dashboard

© Copyright 2000-2016 CyberStream LTD

ENVisage:
Exploratory Visual Analytics on Open Access
Environmental Data

the CyberGreen Team

Athens Green Hackathon

14/12/2012 – 16/12/2012
Presentation @ Athinais Cultural Centre
© Copyright 2000-2016 CyberStream LTD
Turning legacy reporting examples …

© Copyright 2000-2016 CyberStream LTD

Project Metrics

Data Interactive
Extraction, Visual
Cleaning and Analytics
Transforming Application

70% 30%

© Copyright 2000-2016 CyberStream LTD

… into exemplary Visual Analytics paradigms

© Copyright 2000-2016 CyberStream LTD

LIVE DEMO

© Copyright 2000-2016 CyberStream LTD

New Skills for the Data Scientist

• Data Scientist, Analyst or Statistician?

– Technical Skills
– Teamwork Skills
– Communication Skills
– Business Skills (with a little bit of empathy)
– Tool Mastery
• Story Teller?

Institute for Advances Analytics

© Copyright 2000-2016 CyberStream LTD http://analytics.ncsu.edu/
Modern Data Scientist
• Math & Statistics • Programming And Database
• Machine Learning • Computer Science Fundamentals
• Statistical Modelling • Scripting Languages (i.e., Python etc.)
• Experiment design • Statistical Computing Packages (i.e., R)
• Bayesian Interface • Databases: SQL and NoSQL
• Supervised Learning: Decision Trees, • Relational Algebra
Random Forests, Logistic Regression • Parallel Databases and Parallel Query
• Unsupervised Learning: Clustering, Processing
Dimensionality Reduction • MapReduce Concepts
• Optimization: Gradient Descent and • Hadoop and Hive/Pig
Variants • Custom reducers
• Experience with xaaS
• Communication & Visualization
• Able to Engage with Senior Management • Domain Knowledge & Soft Skills
• Story telling skills • Passionate about the Job
• Translate Data-Driven Insights into • Curious about Data
Decisions and Actions • Influence without Authority
• Visual Art Design • Hacker Mindset
• R packages like ggplot and lattice • Problem Solver
• Knowledge of any of Visualizations Tools • Strategic, Proactive, Creative, innovative
& Platforms (i.e., TIBCO Spotfire, and Collaborative
Tableau, Flare, d3.js etc.)
© Copyright 2000-2016 CyberStream LTD
A Visual Narration:
Are we Better Off than we Think?

@HansRosling
https://www.youtube.com/watch?v=jbkSRLYSojo

© Copyright 2000-2016 CyberStream LTD

 Thank you!

Dr. Andreas S. Maniatis

Commercial Manager / Head of BI
[email protected]
http://www.linkedin.com/in/andreasmaniatis
www.cyberstream.eu