Exploratory data analysis data visualization

Exploratory Data Analysis/Data
Visualization
Prepared by
Dr. Hamdan Al-Sabri

Outlines
 What is EDA
 EDA Goals
 EDA Philosophy
 EDA/Data Visualization History
 Data Analysis Approaches
 Data Visualization
 Data Visualization Steps
 Data Visualization Applications
 Data Visualization Examples
Dr. Hamdan M. Al-Sabri, CCIS-KSU

What is EDA? [1]
Exploratory Data Analysis (EDA) is an
approach/philosophy for data analysis that employs a
variety of techniques (mostly graphical) to
1. Maximize insight into a data set.
2. Uncover underlying structure.
3. Extract important variables.
4. Detect outliers and anomalies.
5. Test underlying assumptions.
6. Develop parsimonious models.
7. Determine optimal factor settings.

EDA Goals [1]
 The primary goal of EDA is to maximize the analyst's
insight into a data set and into the underlying
structure of a data set.
 To get a "feel" for the data, the analyst also must
know what is not in the data.
 The only way to do that is to draw on our own human
pattern-recognition and comparative abilities in the
context of a series of judicious graphical techniques
applied to the data.

EDA Focus [1]
The EDA approach is precisely that--an approach--not
a set of techniques, but an attitude/philosophy about
how a data analysis should be carried out.

EDA Philosophy [1]
 EDA is not identical to statistical graphics although
the two terms are used almost interchangeably.
 Statistical graphics is a collection of techniques--all
graphically based and all focusing on one data
characterization aspect.
 EDA is an approach to data analysis that postpones
the usual assumptions about what kind of model the
data follow with the more direct approach of
allowing the data itself to reveal its underlying
structure and model.
 EDA is not a mere collection of techniques; EDA is a
philosophy as to how we dissect a data set; what we
look for; how we look; and how we interpret.

History [2]

Data Analysis Approaches [1]
 For classical analysis, the sequence is
 Problem => Data => Model => Analysis => Conclusions
 For EDA, the sequence is
 Problem => Data => Analysis => Model => Conclusions
 For Bayesian, the sequence is
 Problem => Data => Model => Prior Distribution => Analysis
=>Conclusions

EDA Vs Classical [1]
 Models
 Focus
 Techniques
 Rigor
 Data Treatment
 Assumptions

Example [1]
1

Example [1]
2
Criteria DATA SET 1 DATA SET 2 DATA SET 3 DATA SET 4
N 11 11 11 11
Mean of X 9 9 9 9
Mean of Y 7.5 7.5 7.5 7.5
Intercept 3 3 3 3
Slope 0.5 0.5 0.5 0.5
Residual standard deviation 1.237 1.237 1.236 1.236
Correlation 0.816 0.816 0.816 0.817

Example [1]
3
0.00
2.00
4.00
6.00
8.00
10.00
0.00 5.00 10.00 15.00
DATA SET 2
DATA SET 2
0.00
5.00
10.00
15.00
0.00 5.00 10.00 15.00
DATA SET 3
DATA SET 3
0.00
5.00
10.00
15.00
0.00 5.00 10.00 15.00 20.00
DATA SET 4
DATA SET 4
0.00
2.00
4.00
6.00
8.00
10.00
12.00
0.00 5.00 10.00 15.00
DATA SET 1

Data Visualization[5]
Data visualization is the use of tools to represent data in
the form of charts, maps, tag clouds, animations, or any
graphical means that make content easier to
understand.

Data Visualization Steps [6]

Data Visualization Techniques [6]
 Charts: bar or pie.
 Graphs: good for structure, relationships.
 Plots: 1- to n-dimensional.
 Maps: one of most effective.
 Images: use color/intensity instead of distance
(surfaces).
 3-D surfaces and solids.

What Makes a Good Visualization?
 Effective: the viewer gets it (ease of interpretation).
 Accurate: sufficient for correct quantitative
evaluation. Lie factor = size of visual effect/size of
data effect.
 Efficient: minimize data-ink ratio and chart-junk, show
data, maximize data-ink ratio, brase non-data-ink,
brase redundant data-ink.
 Aesthetics: must not offend viewer's senses (e.g.
moire patterns).
 Adaptable: can adjust to serve multiple needs.

Data Visualization Applications [7]
 Marketing managers are viewing multidimensional
demographic analyses to identify demographic
groups and are viewing geospatial maps to identify
where the next group of customers might be located.
 Sales managers are viewing purchase volume,
revenue, and discounting information to quickly
identify high-revenue customers and profit-
maximizing sales representatives.
 Operations managers are using geographic maps to
compare plant production volumes and profitability.
1

Data Visualization Applications [7]
 IT staff are using visualization for application, network,
and security management to rapidly identify root
causes of problems amid millions of log messages
and alarms.
 Telecommunications carriers are viewing usage
patterns and switching traffic to identify fraud and
service theft, such as illegal cellular phone and
calling card usage.
 Insurance and financial service firms are viewing
transactional data patterns and demographic
dimensions to detect fraud.
2

Data Visualization Correction [3]

Simple Data Visualization
Box Plot
Scatter Plot Matrix
Scatter Plot

Google Trends
http://www.google.com/trends

Map of the Market
http://www.smartmoney.com/map-of-the-market/

TouchGraph GoogleBrowser
http://www.touchgraph.com/TGGoogleBrowser.html

Airline Executive Dashboard
http://www.dundas.com/Components/Products/Map/NET/Demos/index.aspx

Boolistic
http://www.boolistic.com/

Conclusion
Modern advances in data visualization have emerged
from scientific research, rooted primarily in studies of
visual perception and human cognition. These studies
have explored the capacities and limitations of both to
produce data visualization methods and applications
that take advantage of our most powerful abilities and
work around many of the limitations that hinder us. As
such, data visualization is well equipped to assume a
central role in business intelligence, for it is intelligence
that it is tailored to foster.

References
1. NIST/SEMATECH e-Handbook of Statistical Methods,
http://www.itl.nist.gov/div898/handbook/, 28/03/2010.
2. STEPHEN FEW, PERCEPTUAL EDGE, “DATA VISUALIZATION PAST,
PRESENT, AND FUTURE” COGNOS INNOVATION CENTER, Wednesday,
January 10, 2007.
3. Stephen Few, Perceptual Edge “Introduction to Geographical Data
Visualization” Visual Business Intelligence Newsletter, March/April
2009.
4. Data Visualization Specialization Overview, Microsoft Products.
5. 7 things you should know about... Data Visualization II,
www.educause.edu/eli, August 2009.
6. David Adams, “Data Visualization”, White Paper.

Thank You..

Exploratory data analysis data visualization

Recommended

More Related Content

What's hot (20)

Similar to Exploratory data analysis data visualization (20)

More from Dr. Hamdan Al-Sabri (20)

Recently uploaded (20)

Exploratory data analysis data visualization