Advanced Data Analysis

and Visualization
Techniques
Contents to Cover

• Time Series Analysis

• Text Data Analysis using NLTK
• Geospatial Data Analysis using GeoPandas
• Interactive Visualisation using Plotly
• Effective Data Storytelling
Time Series Analysis
Analyzing and Visualizing Time
Series Data

What are time series visualization and analytics?

 Time series visualization and analytics empower users to graphically represent time-
based data, enabling the identification of trends and the tracking of changes over different
periods. This data can be presented through various formats, such as line graphs, gauges,
tables, and more.
 The utilization of time series visualization and analytics facilitates the extraction of
insights from data, enabling the generation of forecasts and a comprehensive
understanding of the information at hand. Organizations find substantial value in time
series data as it allows them to analyze both real-time and historical metrics.
Analyzing and Visualizing Time
Series Data

What is Time Series Data?

 Time series data is a sequential arrangement of data points organized in consecutive time
order. Time-series analysis consists of methods for analyzing time-series data to extract
meaningful insights and other valuable characteristics of the data.

Importance of time series analysis

 Time-series data analysis is becoming very important in so many industries, like financial
industries, pharmaceuticals, social media companies, web service providers, research,
and many more. To understand the time-series data, visualization of the data is essential.
In fact, any type of data analysis is not complete without visualizations, because one good
visualization can provide meaningful and interesting insights into the data.
Analyzing and Visualizing Time
Series Data

Types of Time Series Data

Time series data can be broadly classified into two sections:

1. Continuous Time Series Data: Continuous time series data involves measurements or
observations that are recorded at regular intervals, forming a seamless and uninterrupted
sequence. This type of data is characterized by a continuous range of possible values and is
commonly encountered in various domains, including:
• Temperature Data: Continuous recordings of temperature at consistent intervals (e.g., hourly
or daily measurements).
• Stock Market Data: Continuous tracking of stock prices or values throughout trading hours.
• Sensor Data: Continuous measurements from sensors capturing variables like pressure,
humidity, or air quality.
Analyzing and Visualizing Time
Series Data

Types of Time Series Data

2. Discrete Time Series Data: Discrete time series data, on the other hand, consists of
measurements or observations that are limited to specific values or categories. Unlike
continuous data, discrete data does not have a continuous range of possible values but
instead comprises distinct and separate data points. Common examples include:
• Count Data: Tracking the number of occurrences or events within a specific time period.
• Categorical Data: Classifying data into distinct categories or classes (e.g., customer
segments, product types).
• Binary Data: Recording data with only two possible outcomes or states.
Analyzing and Visualizing Time
Series Data

Visualization Approach for Different Data Types:

• Plotting data in a continuous time series can be effectively represented graphically using
line, area, or smooth plots, which offer insights into the dynamic behavior of the trends
being studied.
• To show patterns and distributions within discrete time series data, bar charts,
histograms, and stacked bar plots are frequently utilized. These methods provide insights
into the distribution and frequency of particular occurrences or categories throughout
time.
Analyzing and Visualizing Time
Series Data

Characteristics of Time Series Data:

▪ Trend: Long-term upward or downward movement.
▪ Seasonality: Regular pattern occurring at fixed intervals.
▪ Cyclicity: Patterns occurring over irregular intervals.
▪ Noise: Random variations in the data.

Python Libraries for Time Series Analysis:

▪ pandas: Data manipulation and time-based indexing.
▪ matplotlib & seaborn: Visualization.
▪ statsmodels & scipy: Statistical analysis.
▪ pmdarima & prophet: Advanced forecasting.
Analyzing and Visualizing Time
Series Data

Key Pandas Functions:

•pd.date_range(): Generate a range of dates.
•pd.to_datetime(): Convert strings to datetime objects.
•set_index(): Set datetime as the index.
Analyzing and Visualizing Time
Series Data
Analyzing and Visualizing Time
Series Data

Definition: Time-Series Forecasting

Time-series forecasting, a technique for predicting future values based on historical

data, is essential for tasks like demand forecasting, financial analysis, and operational
planning. By analyzing data that we stored in the past, we can make informed decisions
that can guide our business strategy and help us understand future trends.

The most important property of a time-series algorithm is its ability to extrapolate

patterns outside of the domain of training data, which most machine-learning techniques
cannot do by default. This is where specialized time-series forecasting techniques come
in.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Examples
• Business planning
• Control engineering
• Cryptocurrency trends
• Financial markets
• Modeling disease spreading
• Pattern recognition
• Resources allocation
• Signal processing
• Sports analytics
• Statistics
• Weather forecasting
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

1. Time-Series Decomposition:
Time-series decomposition is a
method for explicitly modeling the
data as a combination
of seasonal, trend,
cycle, and remainder component
s instead of modeling it with
temporal dependencies and
autocorrelations. It can either be
performed as a standalone
method for time-series
forecasting or as the first step in
better understanding your data.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:
Seasonal: Represents predictable fluctuations that occur regularly within a specific time period, like the
increase in ice cream sales during summer months.

Trend: Represents the overall long-term direction of the data, whether it's increasing, decreasing, or
staying relatively stable over time.

Cycle: Represents recurring patterns in the data that are longer than seasonal fluctuations, but not
necessarily tied to a specific calendar period.

Remainder: The "noise" or random variation left over after removing the seasonal, trend, and cycle
components from the data.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

2. Time-Series Regression Models:

Time-series regression is a statistical method
for forecasting future values based on
historical data. The forecast variable is also
called the regressand, dependent, or
explained variable. The predictor variables are
sometimes called the regressors,
independent, or explanatory variables.
Regression algorithms attempt to calculate
the line of best fit for a given dataset. For
example, a linear regression algorithm could
try to minimize the sum of the squares of the
differences between the observed value and
predicted value to find the best fit.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

3. Exponential Smoothing:
When it comes to time-series forecasting, data
smoothing can tremendously improve the
accuracy of our predictions by removing outliers
from a time-series dataset. Smoothing leads to
increased visibility of distinct and repeating
patterns hidden between the noise. Exponential
smoothing is a rule-of-thumb technique for
smoothing time-series data using the exponential
window function. Whereas the simple moving
average method weighs historical data equally to
make predictions for time series, exponential
smoothing uses exponential functions to calculate
decreasing weights over time.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

4. ARIMA Models:
AutoRegressive Integrated Moving
Average, or ARIMA, is a forecasting
method that combines both an
autoregressive model and a moving
average model. Autoregression uses
observations from previous time steps to
predict future values using a regression
equation. An autoregressive model utilizes
a linear combination of past variable
values to make forecasts.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

5. Neural Networks:
Neural networks have rapidly become a go-to
solution for time-series forecasting, offering
powerful tools for classification and prediction in
scenarios with complex data relationships. A
neural network can sufficiently approximate any
continuous functions for time-series forecasting.
Unlike classical models like ARMA or ARIMA,
which rely on linear assumptions between inputs
and outputs, neural networks adapt to nonlinear
patterns, making them suitable for a wider range
of data types. This means they can approximate
any nonlinear function without prior knowledge
about the properties of the data series.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:
6. TBATS:
Time-series data often contains intricate
seasonal patterns that evolve across various time
frames, such as daily, weekly, or yearly trends.
Traditional models like ARIMA and exponential
smoothing usually capture only a single
seasonality, limiting their effectiveness for
complex series. The TBATS model addresses this
limitation by accounting for multiple, non-nested,
and even non-integer seasonal patterns, ideal for
long-term and complex forecasting tasks.
TBATS stands for Trigonometric
seasonality, Box-Cox transformation, ARIMA
errors, Trend, and Seasonal components, each
of which adds a layer of precision to forecasts
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

Trigonometric seasonality: TBATS effectively models complex cyclical patterns with different frequencies,
enabling it to capture and represent overlapping seasonal trends such as daily, weekly, and yearly cycles
that may interact in a time series.

Box-Cox transformation: This transformation stabilizes variance in data, enhancing the model’s
robustness against fluctuations and outliers, which is crucial for achieving reliable predictions in datasets
with varying scales or distributions.
Analyzing and Visualizing Time
Series Data
Time-Series Forecasting Techniques:

ARIMA errors: By integrating ARIMA-like error structures, TBATS improves the handling of residual patterns,
refining prediction accuracy. This combination allows the model to learn from past forecasting errors and
adjust future predictions accordingly.

Trend: TBATS captures both linear and exponential trends, adapting to gradual changes over time for
effective long-term forecasting. This capability is essential for industries where growth rates may change,
such as technology or retail.

Seasonal components: The model accommodates diverse seasonal patterns without strict constraints,
enabling detailed, accurate multi-seasonal forecasts. This flexibility allows TBATS to tackle complex
seasonality that is often present in real-world data, leading to better insights and decision-making.
Text Data Analysis using NLTK
The Natural Language Toolkit
(NLTK) Library
The Natural Language Toolkit (NLTK) is a popular open-source library for natural language
processing (NLP) in Python. It provides an easy-to-use interface for a wide range of tasks,
including tokenization, stemming, lemmatization, parsing, and sentiment analysis.
NLTK is widely used by researchers, developers, and data scientists worldwide to develop
NLP applications and analyze text data.
One of the major advantages of using NLTK is its extensive collection of corpora, which
includes text data from various sources such as books, news articles, and social media
platforms. These corpora provide a rich data source for training and testing NLP models.
What is Sentiment Analysis

Sentiment analysis is a technique used to determine the emotional tone or sentiment

expressed in a text. It involves analyzing the words and phrases used in the text to identify
the underlying sentiment, whether it is positive, negative, or neutral.
Sentiment analysis has a wide range of applications, including social media monitoring,
customer feedback analysis, and market research.
One of the main challenges in sentiment analysis is the inherent complexity of human
language. Text data often contains sarcasm, irony, and other forms of figurative language
that can be difficult to interpret using traditional methods.
Three Methodologies for Sentiment
Analysis
There are several ways to perform sentiment analysis on text data, with varying degrees of
complexity and accuracy. The most common methods include a lexicon-based approach,
a machine learning (ML) based approach, and a pre-trained transformer-based deep
learning approach.

Lexicon-based analysis
This type of analysis, such as the NLTK Vader sentiment analyzer, involves using a set of
predefined rules and heuristics to determine the sentiment of a piece of text. These rules
are typically based on lexical and syntactic features of the text, such as the presence of
positive or negative words and phrases.
While lexicon-based analysis can be relatively simple to implement and interpret, it may
not be as accurate as ML-based or transformed-based approaches, especially when
dealing with complex or ambiguous text data.
Three Methodologies for Sentiment
Analysis
Machine learning (ML)
This approach involves training a model to identify the sentiment of a piece of text based on a set
of labeled training data. These models can be trained using a wide range of ML algorithms,
including decision trees, support vector machines (SVMs), and neural networks.
ML-based approaches can be more accurate than rule-based analysis, especially when dealing
with complex text data, but they require a larger amount of labeled training data and may be
more computationally expensive.
Pre-trained transformer-based deep learning
A deep learning-based approach, as seen with BERT and GPT-4, involve using pre-trained models
trained on massive amounts of text data. These models use complex neural networks to encode
the context and meaning of the text, allowing them to achieve state-of-the-art accuracy on a wide
range of NLP tasks, including sentiment analysis. However, these models require significant
computational resources and may not be practical for all use cases.
Installing NLTK and Setting up
Python Environment
Preprocessing Text
Text preprocessing is a crucial step in performing sentiment analysis, as it helps to clean
and normalize the text data, making it easier to analyze. The preprocessing step involves a
series of techniques that help transform raw text data into a form you can use for analysis.
Some common text preprocessing techniques include tokenization, stop word removal,
stemming, and lemmatization.
Tokenization
Tokenization is a text preprocessing step in sentiment analysis that involves breaking
down the text into individual words or tokens. This is an essential step in analyzing text
data as it helps to separate individual words from the raw text, making it easier to analyze
and understand. Tokenization is typically performed using NLTK's built-
in word_tokenize function, which can split the text into individual words and punctuation
marks.
Stop Words
Stop word removal is a crucial text preprocessing step in sentiment analysis that involves
removing common and irrelevant words that are unlikely to convey much sentiment. Stop
words are words that are very common in a language and do not carry much meaning,
such as "and," "the," "of," and "it." These words can cause noise and skew the analysis if
they are not removed.
By removing stop words, the remaining words in the text are more likely to indicate the
sentiment being expressed. This can help to improve the accuracy of the sentiment
analysis. NLTK provides a built-in list of stop words for several languages, which can be
used to filter out these words from the text data.
Stemming and Lemmatization
Stemming and lemmatization are techniques used to reduce words to their root
forms. Stemming involves removing the suffixes from words, such as "ing" or "ed," to
reduce them to their base form. For example, the word "jumping" would be stemmed
to "jump."
Lemmatization, however, involves reducing words to their base form based on their
part of speech. For example, the word "jumped" would be lemmatized to "jump," but
the word "jumping" would be lemmatized to "jumping" since it is a present participle.
Bag of Words (BoW) Model
The bag of words model is a technique used in natural language processing (NLP) to
represent text data as a set of numerical features. In this model, each document or piece
of text is represented as a "bag" of words, with each word in the text represented by a
separate feature or dimension in the resulting vector. The value of each feature is
determined by the number of times the corresponding word appears in the text.
The bag of words model is useful in NLP because it allows us to analyze text data using
machine learning algorithms, which typically require numerical input. By representing text
data as numerical features, we can train machine learning models to classify text or
analyze sentiments.
Geospatial Data Analysis using
GeoPandas
An Introduction to Geospatial
Analysis
A considerable proportion of the data generated every day is inherently spatial. From Earth
Observation data and GPS data to data included in all kinds of maps, spatial data –also known
sometimes as geospatial data or geographic information– are data for which a specific location
is associated with each record.
Every spatial data point can be located on a map. This can be done using a certain coordinate
reference system, for example, the geographical coordinates, which is based on the usual
latitude-longitude pairs we use to specify where something is on the globe. This allows us to
look at spatial relationships between the data.
Yet, the real power of geospatial data is combining both the data themselves and their location,
unlocking several opportunities for sophisticated analysis. The so-called geospatial data
science is a subfield of data science that focuses on extracting information from geospatial
data by leveraging the power of spatial algorithms and analytical techniques, such as machine
learning and deep learning. In other words, geospatial data science helps us understand where
things happen and why they happen there.
GeoPandas extends the datatypes used by pandas –the standard tool for manipulating
dataframes in Python– to allow spatial operations on geometric types.
Installing Python GeoPandas

To install GeoPandas
>>pip install geopandas

To Import GeoPandas
import geopandas as gpd
Basic Operations with GeoPandas
(1) Reading and writing spatial data
 Just like pandas require input data to convert into a pandas dataframe, GeoPandas reads input
(spatial) data and converts it into the so-called GeoDataFrame.
 Before going into the multiple spatial filetypes GeoPandas can read, it’s important to distinguish the
different types of spatial data. The type of data we are dealing with informs what tools we should
use to analyze and then visualize the data.

Basically, there are two main types of spatial data:

1. Vector data. It describes the features of geographic locations on Earth through the use of discrete
geometries, namely:
• Point. Individual locations, like a building, or a car, with X and Y coordinates.
• Line. A series of connected points describing things like roads or streams.
• Polygon. Formed by a closed line that encircles an area, such as the boundaries of a country. Additionally,
when one feature consists of multiple geometries, we call it MultiPolygon.
Basic Operations with GeoPandas
Reading and writing spatial data

2. Raster data. Encodes the world as a

continuous surface represented by a grid, such
as the pixels of an image. Each piece of the grid
can be either a continuous value (such as an
elevation value) or a categorical classification
(such as land cover classifications). Classic
examples include altitude data or satellite
images.
Basic Operations with GeoPandas
(2) Exploring GeoDataframes
GeoDataFrame is a subclass of [Link]. That means it inherits many of the methods and
attributes of pandas dataframe. What's new in GeoDataFrame is that it can store geometry
columns (also known as GeoSeries) and perform spatial operations.
The geometry column can contain any type of vector data, such as points, lines, and polygons.
Further, it’s important to note that although a GeoDataFrame can have multiple GeoSeries, only
one column is considered the active geometry, meaning that all the spatial operations will be based
on that column.
Basic Operations with GeoPandas
(2) Exploring GeoDataframes
GeoDataFrame is a subclass of [Link]. That means it inherits many of the methods and
attributes of pandas dataframe. What's new in GeoDataFrame is that it can store geometry
columns (also known as GeoSeries) and perform spatial operations.
The geometry column can contain any type of vector data, such as points, lines, and polygons.
Further, it’s important to note that although a GeoDataFrame can have multiple GeoSeries, only
one column is considered the active geometry, meaning that all the spatial operations will be based
on that column.
Plotting with GeoPandas
ax= [Link](figsize=(10,6)) ax= [Link](column=district_name',
figsize=(10,6), edgecolor='black', legend=True)
Plotting with GeoPandas
import contextily
ax= [Link](column='district_name', figsize=(12,6),
alpha=0.5, legend=True)
districts["centroid"].plot(ax=ax, color="green")
sagrada_fam.plot(ax=ax,color='black', marker='+')
contextily.add_basemap(ax, crs=[Link].to_string())
[Link]('A Beautiful Map of Barcelona')
[Link]('off')
[Link]()
Interactive Visualisation using
Plotly
Plotly for Data Visualization
Plotly is an open-source Python library for creating interactive visualizations like line charts,
scatter plots, bar charts and more. In this article, we will explore plotting in Plotly and covers
how to create basic charts and enhance them with interactive features.

Plotly is a Python library that helps you create interactive and visually appealing charts and
graphs. It allows you to display data in a way that’s easy to explore and understand, such as
by zooming in, hovering over data points for more details, and clicking to get deeper insights
Plotly uses JavaScript to handle interactivity, but you don’t need to worry about that when
using it in Python. You simply write Python code to create the charts, and Plotly takes care of
making them interactive.

To install it type the below command in the terminal.

pip install plotly

Plotly for Data Visualization

Output:
Getting Started with Basic Charts in
Plotly
1. Line chart: Plotly line chart is one of the simple plots where a line is drawn to show relation between
the X-axis and Y-axis. It can be created using the [Link]() method with each data position is
represented as a vertex of a polyline mark in 2D space.
Getting Started with Basic Charts in
Plotly
2. Bar Chart: A bar chart is a pictorial representation of data that presents categorical data with
rectangular bars with heights or lengths proportional to the values that they represent. These data sets
contain the numerical values of variables that represent the length or height. It can be created using
the [Link]() method.
Getting Started with Basic Charts in
Plotly
Let’s try to customize this plot. Customizations that we will use –
•color: Used to color the bars.
•facet_row: Divides the graph into rows according to the data passed
•facet_col: Divides the graph into columns according to the data passed
Getting Started with Basic Charts in
Plotly
3. Scatter Plot: A scatter plot is a set of dotted points to represent individual pieces of data in the
horizontal and vertical axis. A graph in which the values of two variables are plotted along X-axis and Y-
axis the pattern of the resulting points reveals a correlation between them and it can be created using
the [Link]() method.
Getting Started with Basic Charts in
Plotly
Let’s see various customizations available for this chart that we will use –
•color: Color the points.
•symbol: Gives a symbol to each point according to the data passed.
•size: The size for each point.
Getting Started with Basic Charts in
Plotly
4. Histogram: A histogram is basically used to represent data in the form of some groups. It is a type of
bar plot where the X-axis represents the bin ranges while the Y-axis gives information about frequency.
It can be created using the [Link]() method.
Getting Started with Basic Charts in
Plotly
Let’s customize the above graph. Customizations that we will be using are –
•color: To color the bars
•nbins: To set the number of bins
•histnorm: Mode through which the bins are represented. Different values that can be passed using this
argument.
•barmode: Can be either ‘group’, ‘overlay’ or ‘relative’.
• group: Bars are stacked above zero for positive values and below zero for negative values
• overlay: Bars are drawn on the top of each other
• group: Bars are placed beside each other.
Getting Started with Basic Charts in
Plotly
5. Pie Chart: A pie chart is a circular statistical graphic, which is divided into slices to illustrate
numerical proportions. It depicts a special chart that uses “pie slices”, where each sector shows the
relative sizes of data. A circular chart cuts in a form of radii into segments to magnitude of different
features. It can be created using the [Link]() method.
Getting Started with Basic Charts in
Plotly
Let’s customize the above graph. Customizations that we will be using are –
•color_discrete_sequence: Strings defining valid CSS colors
•opacity: It determines how transparent or solid the markers (such as points on a scatter plot) appear.
The value should be between 0 and 1
•hole: Creates a hole in between to make it a donut chart. The value should be between 0 and 1
Building Interactive Dashboards
with Plotly Dash
Plotly Dash is a powerful framework for building interactive web applications with Python. It allows you
to create dynamic and visually appealing dashboards that can handle complex interactions and data
visualizations. In this section, we will explore the essentials of building interactive dashboards using
Plotly Dash. For more advanced interactive applications, we can use Plotly Dash to create dashboards.

1. Install necessary packages:

!pip install dash jupyter-dash plotly pandas

Building Interactive Dashboards
with Plotly Dash
Building Interactive Dashboards
with Plotly Dash

Plotly provides a rich set of tools for creating interactive visualizations in Python. From basic charts to
complex interactive dashboards. By customizing our charts and using interactive features we can enhance
your data exploration and communication.
Effective Data Storytelling
Design for an Audience
DESIGNING FOR CLARITY AND IMPACT
Design for an Audience
DESIGNING FOR CLARITY AND IMPACT

At first glance, the plot above seems to be doing its job—it gives us the numbers. But we can already spot a few
opportunities for improvement:

•Thinner bars: We'll reduce the thickness of the bars to create more visual breathing room, making the data feel less
cramped.

•Frame removal: We'll eliminate the borders (spines) around the plot that don't contribute to understanding the data.

•Custom x-ticks: Instead of the default ticks, we'll set specific values to show only the most important points (0;
150,000; 300,000).

•Move x-axis labels: We'll move the x-axis labels to the top of the chart so that the labels are closer to the countries
with higher values.

•Subtle color scheme: We'll change the color of the bars to a deeper red, drawing attention to the data while
maintaining a minimalist design.

•Clean tick marks: We'll remove the distracting tick marks and adjust the remaining ones for a cleaner look.
Design for an Audience
OBJECT ORIENTED INTERFACE
Design for an Audience
OBJECT ORIENTED INTERFACE

As you can see, we've improved upon the previous plot significantly. Here's a breakdown of the adjustments we've
made:

•We reduced the bar thickness, creating more visual breathing room and improving readability.

•The removal of spines (borders) eliminates unnecessary visual noise, keeping the focus squarely on the data.

•We've moved the x-axis labels to the top and styled them in grey, ensuring they're visible but don't distract from the
main data.

•By changing the bar color to a deeper red, we've added subtle emphasis that helps the data stand out while maintaining
a clean design.

Although we've improved the plot's overall appearance and functionality, we can still make a few more tweaks to further
increase clarity and context for our audience.
Design for an Audience
FINAL TOUCHES
Design for an Audience
FINAL TOUCHES

With these final tweaks, we've turned a basic bar chart into a clear, engaging visualization.

The bold title and subtitle deliver the main point at a glance, while the left-aligned country names make it
easier for the eye to follow.

By adding a reference line at 150,000 and simplifying the axis labels, we've helped the viewer quickly spot
meaningful patterns without distractions.

This final version focuses entirely on the data, keeping the design clean and purposeful.

Every element adds value, ensuring the chart is both easy to understand and visually appealing.
Storytelling Data Visualization
KEY ELEMENTS OF A GOOD DATA STORY

When you're crafting a data story, small details can make a big difference. Here are some simple tips that can make
visualizations clearer and more engaging:

[Link] with a clear message: Make sure your entire visualization points toward that insight.

[Link] color intentionally: Keep the same color throughout to keep things visually consistent and easy to follow.

[Link] the eye: Lay out your visuals in a way that naturally leads people from one point to the next. Don't make them
guess what's important.

[Link] context: Add titles, subtitles, and annotations to help your audience understand what they're looking at and
why it matters.

[Link] it simple: Too much information can be overwhelming. If you've got a lot to share, break it up into smaller,
easier-to-digest visualizations.
Thank You !