EMERGING TECHNOLOGIES

CHAPTER two
Data Science

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Outline
 Overview of Data Science
 What are Data and Information?
 Data Types and Their Representation
 Data Value Chain.
 Basic Concepts of Big Data

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 An Overview of Data Science

Data Science is a multi-disciplinary field that uses

scientific methods, processes, algorithms, and systems
to extract knowledge and insights from structured,
semi-structured and unstructured data.
Data science is much more than simply analyzing data.
It offers a range of roles and requires a range of skills.
There are many different pieces of software will be used
to organized, aggregate, visualize and present the data

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Overview of Data Science …

• Professionals use advanced techniques for analyzing

large volumes of data.
• Skills important for data science:
– Statistics
– Linear algebra
– Programming knowledge with focus on data
warehousing, data mining, and data modeling

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data VS Information
• Data: a representation of facts, concepts, or instructions in a formalized
manner, which should be suitable for communication, interpretation, or
processing, by human or electronic machines.
• It can be described as unprocessed facts and figures.
• It is represented groups of non-random symbols in the form of text,
images, voice, videos representing quantities, action and objects.
• It is represented with the help of characters such as alphabets (A-Z, a-z),
digits (0-9) or special characters (+, -, /, *, <,>, =, etc.).
• Information is the processed/interpreted data on which decisions and
actions are based.
• It is interpreted data; created from organized, structured, and processed
data in a particular context.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data VS Information…

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data vs. Information
Examples
• Seeing examples of data and information side-by-side in a chart
can help you better understand the differences between the two
terms.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data Processing Cycle…
• Input − input data is prepared in some convenient
form for processing.
• The form will depend on the processing machine.
For example, when electronic computers are used,
the input data can be recorded on any one of the
several types of input medium, such as magnetic
disks, tapes, and so on.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data Processing Cycle…
• Processing - input data is changed to produce data in a more
useful form.
– For example, pay-checks can be calculated from the time cards,
or a summary of sales for the month can be calculated from
the sales orders.
• Output − the result of the proceeding processing step is collected.
– The particular form of the output data depends on the use of
the data. For example, output data may be pay-checks for
employees.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data Types and
Their Representation
• Data types can be described from diverse perspectives.
1. Computer science and programming perspective:
– A data type is an attribute of data that tells the compiler or interpreter how the
programmer intends to use the data.
– Almost all programming languages explicitly include the notion of data type,
though different languages may use different terminology.
– Common data types include:
• Integers: store integers.
• Booleans: store one of the two values: true or false
• Characters: store a single character (numeric, alphabetic, symbol, …)
• Floating-point numbers: stores real numbers
• Alphanumeric strings: stores a combination of characters and numbers.
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com
 Data Types and
Their Representation …
2. Data types from Data Analytics perspective
• From a data analytics point of view there are three
common types of data types or structures:
– Structured, Semi-structured, and Unstructured
data types, in addition Meta data

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data Types and
Their Representation …

Data types from a data analytics perspective

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data Types and
Their Representation …
• Structured Data: is data that adheres to a pre-defined data model and
is therefore straightforward to analyze.
• Structured data conforms to a tabular format with a relationship
between the different rows and columns.
• Common examples of structured data are Excel files or SQL
databases.
• Each of these has structured rows and columns that can be sorted.
• Structured data is considered the most ‘traditional’ form of data
storage, since the earliest versions of database management systems
(DBMS) were able to store, process and access structured data.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data Types and
Their Representation …
• Semi-structured Data: is a form of structured data that does not conform with the formal
structure of data models associated with relational databases or other forms of data tables.
• But, contain tags or other markers to separate semantic elements and enforce hierarchies of
records and fields within the data.
• Therefore, it is also known as a self-describing structure.
• Examples of semi-structured data include JSON and XML are forms of semi-structured data.
• Unstructured Data: is information that either does not have a predefined data model or is not
organized in a pre-defined manner.
• Unstructured information is typically text-heavy but may contain data such as dates, numbers,
and facts as well.
• This results in irregularities and ambiguities that make it difficult to understand using
traditional programs as compared to data stored in structured databases.
• Common examples of unstructured data include audio, video files or No-SQL databases.
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com
 Data Types and
Their Representation …
• Metadata – Data about Data: A last category of data type is metadata.
• From a technical point of view, this is not a separate data structure, but it is
one of the most important elements for Big Data analysis and big data
solutions.
• Metadata is data about data. It provides additional information about a
specific set of data.
• Example: In a set of photographs, metadata could describe when and where
the photos were taken.
• The metadata then provides fields for dates and locations which, by
themselves, can be considered structured data.
• Because of this reason, metadata is frequently used by Big Data solutions for
initial analysis.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data value Chain
• The Data Value Chain is introduced to describe the information flow within a
big data system as a series of steps needed to generate value and useful
insights from data.
• The Big Data Value Chain identifies the following key high-level activities:

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data value Chain …
• Data Acquisition
• It is the process of gathering, filtering, and cleaning
data before it is put in a data warehouse or any
other storage solution on which data analysis can
be carried out.
• Data acquisition is one of the major big data
challenges in terms of infrastructure requirements.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data value Chain …
• Data Analysis
• It is concerned with making the raw data acquired
amenable to use in decision-making as well as
domain-specific usage.
• Data analysis involves exploring, transforming, and
modelling data with the goal of highlighting relevant data,
synthesizing and extracting useful hidden information
with high potential from a business point of view.
• Related areas include data mining, business intelligence,
and machine learning.
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com
 Data value Chain …
• Data Curation
• It is the active management of data over its life cycle to ensure it meets the
necessary data quality requirements for its effective usage.
• Data curation processes can be categorized into different activities such as
content creation, selection, classification, transformation, validation, and
preservation.
• Data curation is performed by expert curators that are responsible for improving
the accessibility and quality of data.
• Data curators (also known as scientific curators, or data annotators) hold the
responsibility of ensuring that data are trustworthy, discoverable, accessible,
reusable, and fit their purpose.
• A key trend for the curation of big data utilizes community and crowd sourcing
approaches.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data value Chain …
• Data Usage
• It covers the data-driven business activities that need
access to data, its analysis, and the tools needed to
integrate the data analysis within the business activity.
• Data usage in business decision-making can enhance
competitiveness through reduction of costs, increased
added value, or any other parameter that can be
measured against existing performance criteria

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Data value Chain …
• Data Storage
• It is the persistence and management of data in a scalable way that
satisfies the needs of applications that require fast access to the data.
• Relational Database Management Systems (RDBMS) have been the main,
and almost unique, solution to the storage paradigm for nearly 40 years.
• However, the ACID (Atomicity, Consistency, Isolation, and Durability)
properties that guarantee database transactions lack flexibility with regard
to schema changes and the performance and fault tolerance when data
volumes and complexity grow, making them unsuitable for big data
scenarios.
• NoSQL technologies have been designed with the scalability goal in mind
and present a wide range of solutions based on alternative data models.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data: Definition
• Big data is a blanket term for the non-traditional strategies and
technologies needed to gather, organize, process, and gather
insights from large datasets.
• While the problem of working with data that exceeds the computing
power or storage of a single computer is not new, the pervasiveness,
scale, and value of this type of computing has greatly expanded in
recent years.
• What Is Big Data?
• Big data is the term for a collection of data sets so large and
complex that it becomes difficult to process using on-hand
database management tools or traditional data processing
applications.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Characteristics
– The 4Vs
• Big data differs from traditional data in the following
ways:
• Volume: large amounts of data Zeta bytes/Massive
datasets.
• Velocity: Data is live streaming or in motion.
• Variety: data comes in many different forms, quality and
from diverse sources. (Social media, server logs,
sensors, …)
• Veracity: can we trust the data? How accurate is it? Etc...
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com
 Big Data Characteristics
– The 4Vs…

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Characteristics
– The 4Vs…
• Now let’s have a look at the data generated per minute
on internet.
• 2.1M snaps are shard in Snap chat,
• 3.8M search queries are mead in Google,
• 1M people are log in Facebook,
• 4.5M videos are watched in YouTube and
• 188M emails are send.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Clustered Computing
• Individual computers are often inadequate for handling big data at
most stages.
• Clustered computing is used to better address the high storage and
computational needs of big data.
• Clustered computing is a form of computing in which a group of
computers (often called nodes) that are connected through a LAN
(local area network) so that, they behave like a single machine.
• The set of computers is called a cluster.
• The resources from these computers are pooled to appear as one
more powerful computer than the individual computers.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Clustered Computing …
• Big data clustering software combines the resources of many
smaller machines, seeking to provide a number of benefits:
– Resource Pooling: Combining the available storage space, CPU and
memory is extremely important.
– Processing large datasets requires large amounts of all three of these
resources.
– High Availability: Clusters provide varying levels of fault tolerance and
availability guarantees to prevent hardware or software failures from
affecting access to data and processing.
– Increasingly important for real-time analytics of big data.
– Easy Scalability: Clusters make it easy to scale horizontally by adding more
machines to the group. The system can react to changes in resource
requirements without expanding the physical resources on a machine.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Clustered Computing …
• Using clusters requires a solution for managing cluster membership,
coordinating resource sharing, and scheduling actual work on
individual nodes.
• Cluster membership and resource allocation can be handled by
softwares like Hadoop’s YARN (which stands for Yet Another Resource
Negotiator).
• The assembled computing cluster often acts as a foundation that
other software interfaces with to process the data.
• The machines involved in the computing cluster are also typically
involved with the management of a distributed storage system, which
we will talk about when we discuss data persistence.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Hadoop
• Hadoop is an open-source framework intended to make interaction with big data
easier.
• It is a framework that allows for the distributed processing of large datasets
across clusters of computers using simple programming models.
• The four key characteristics of Hadoop are:
• Economical: Its systems are highly economical as ordinary computers can be used
for data processing.
• Reliable: It is reliable as it stores copies of the data on different machines and is
resistant to hardware failure.
• Scalable: It is easily scalable both, horizontally and vertically.
• Flexible: It is flexible and you can store as much structured and unstructured data
as you need.

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Hadoop Ecosystem
• Hadoop Ecosystem is a platform or a suite which provides various
services to solve the big data problems.
• Hadoop has an ecosystem that has evolved from its four core
components: data management, access, processing, and storage.
• It is continuously growing to meet the needs of Big Data.
• It comprises the following components and many others:
• HDFS: Hadoop Distributed File System
• YARN: Yet Another Resource Negotiator
• MapReduce: Programming based Data Processing
• Spark: In-Memory data processing

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Hadoop Ecosystem …
• PIG, HIVE: Query-based processing of data services
• HBase: NoSQL Database
• Mahout, Spark MLLib: Machine Learning algorithm
libraries
• Solar, Lucene: Searching and Indexing
• Zookeeper: Managing cluster
• Oozie: Job Scheduling

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big Data Solutions:
Hadoop Ecosystem …

This presentation uses a free template provided by FPPT.com

www.free-power-point-templates.com
 Big data life cycle
with hadoop
1. Ingesting data into the system
– The first stage of Big Data processing is to Ingest data into the system.
– The data is ingested or transferred to Hadoop from various sources
such as relational databases, systems, or local files.
– Sqoop transfers data from RDBMS to HDFS, whereas Flume transfers
event data.
2. Processing the data in storage.
– The second stage is Processing.
– In this stage, the data is stored and processed.
– The data is stored in the distributed file system, HDFS, and the NoSQL
distributed data, HBase.
– Spark and MapReduce perform data processing.
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com
 Big data life cycle
with Hadoop
3. Computing and analyzing data
– The third stage is to Analyze Data
– Here, the data is analyzed by processing frameworks such as Pig, Hive,
and Impala.
– Pig converts the data using a map and reduce and then analyzes it.
– Hive is also based on the map and reduce programming and is most
suitable for structured data.
4. Visualizing the results
– The fourth stage is access, which is performed by tools such as
Sqoop, Hive, Hue and Cloudera Search.
– In this stage, the analyzed data can be accessed by users.
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com
This presentation uses a free template provided by FPPT.com
www.free-power-point-templates.com