UNIT 2

EMERGING
TECHNOLOGIES
Introduction to Bigdata
 Other well documented use cases of the first forms of data
analysis come from the Roman empire.
 The ancient Roman military utilized very detailed
statistical analysis to ‘predict’ at which border the chance
of an enemy insurgency would be the most prevalent.
Based on these analyses, they were able to deploy their
armies in the most efficient way possible. It is not a far
stretch to consider these calculations one of the earliest
forms of ‘predictive’ data analysis. And again, these
analysis techniques provided the Roman military with an
advantage over other armies.
 The term ‘Big Data’ has been in use since the early 1990s.
Although it is not exactly known who first used the term, most
people credit John R. Mashey (who at the time worked at
Silicon Graphics) for making the term popular.
 The last two decades, the volume and speed with which data is
generated has changed beyond measures of human
comprehension.
 The total amount of data in the world was 4.4 zettabytes in
2013. That is set to rise steeply to 44 zettabytes by 2020. To put
that in perspective, 44 zettabytes is equivalent to
44 trillion gigabytes.
 Even with the most advanced technologies today, it is
impossible to analyze all this data. The need to process these
increasingly larger (and unstructured) data sets is how
traditional data analysis transformed into ‘Big Data’ in the last
decade.
 Definition of Big Data
 Big data is a large data set that is difficult to process and
analyze with traditional methods.
 Most of the time, the term big data means a range
of structured, Unstructured, and Semi-structured data.
 This is largely due to the rise of computers, the Internet, and
technology capable of capturing information from the real
and physical world we live in, and converting it to digital
data that can benefit businesses and organizations.
 Big Data Analysis
 Big data analysis refers to the systematic processing and
analysis of large and complex data sets, known as big data,
to extract valuable insights.
 This process allows organizations to leverage the
exponentially growing data generated from diverse sources,
including IoT sensors, social media, financial transactions,
and smart devices, to derive actionable intelligence through
advanced analytic techniques
Examples of data analysis software
Altair One Amazon QuickSight Cognos
Analytics
Alteryx Platform Domo Power BI
Key Principles and Methods
 Big data analysis involves several key principles and methods to
uncover insights and patterns within an organization's data. These
methods include:
 Descriptive Analytics: This method focuses on summarizing and
describing past data to understand its basic characteristics. It
answers the "what happened" question.
 Diagnostic Analytics: This method delves deep into the data to
identify the root patterns and trends observed in descriptive
analytics. It answers the "why it happened" question.
 Predictive Analytics: This method uses historical data, statistical
modeling, and machine learning to forecast trends. It answers the
"what will happen" question.
 Prescriptive Analytics: This method goes beyond prediction to
provide recommendations for optimizing future actions based on
insights derived from all previous analyses. It answers the "what
to do" question.
 Characteristics of Big data
Big data is characterized by the following five dimensions, which highlight
the core challenges and opportunities inherent in big data analytics:
 Volume: The sheer volume of data generated today presents a significant
challenge. Big data technologies and cloud-based storage solutions enable
organizations to store and manage these vast data sets cost-effectively.
 Velocity: Data is being produced at unprecedented speeds, requiring robust
processing capabilities to capture, process, and deliver accurate analysis in
near real-time.
 Variety: Today's data comes in many formats, from structured to
unstructured text, video, and images. Flexible data management systems
are needed to handle and integrate disparate data types for comprehensive
analysis.
 Veracity: Data reliability and accuracy are critical. Techniques and tools
for data cleaning, validation, and verification are integral to ensuring the
integrity of big data.
 Value: Big data analytics aims to extract actionable insights that offer
tangible value, transforming raw data into strategic assets
 Big data sources
 There are two types of big data sources: internal and external ones. Data
is internal if a company generates, owns and controls it. External data
is public data or the data generated outside the company;
correspondingly, the company neither owns nor controls it.
E.g
 Climate information data collected from IOT based Sensors
 Digital Pictures (Pinterest)
 Digital Audio (Spotify, Pandora, Apple Music, Amazon Music)
 Digital videos (Netflix, HBO Max, Hulu, Disney)
 Social Media (Facebook, Instagram, Pinterest, Snapchat, YouTube,
WhatsApp)
 Retail Stores and E-Commerce Websites (Amazon, Target, Walmart,
Costco, Etsy, Wayfair)
 Financial transaction records (Invoices, E-receipts)
 Cell phone GPS (Global Positioning System) signals data
 Categories of source of Big Data

Multimedia and individuals with smartphones and social

network sites will continue to fuel exponential growth. Big
Data is large pools of data that can be captured, communicated,
aggregated, stored, and analyzed and is now part of every
sector and function of the global economy. Like other essential
factors of production such as hard assets and human capital, it
is increasingly the case that much of modern economic activity,
innovation, and growth couldn’t take place without data.
All of these sources of data can be categorized into three
main sections.
 Machines
People
Organizations
 Benefits of Big data
Big data has many advantages. Some of them are discussed as follow
Cost optimization
One of the most major advantages of Big Data technologies is that
they reduce the cost of storing, processing, and analyzing enormous
volumes of data for enterprises. Not only that, but Big Data
technologies may help find cost-effective and efficient company
practices.
Helps in Understanding the Market Conditions
By examining Big data, it is possible to have a better knowledge of
current market situations. Let's take an example: a corporation can
determine the most popular goods by studying a customer's purchase
behavior. It aids in the analysis of trends and client desires. A
company can use this to get an advantage over its competition.
Improve efficiency
 Big Data techniques can dramatically enhance operational
efficiency. Big Data technologies may collect vast
volumes of usable customer data by connecting with
customers/clients and getting their important input.
Better Decision Making
 The main benefit of using Big Data Analytics is that it has
boosted the decision-making process to a great extent.
Rather than anonymously making decisions, companies
are considering Big Data Analytics before concluding any
decision.
 Product development: it allows you to update existing
products/services while innovating new ones.
 Price optimization: This minimizes the manual work and
reduces the possibility of any man-made errors.
 Targeted advertisements (Ads): Personalization data from
sources such as past purchases, interaction patterns, and
product page viewing histories can help generate
compelling targeted advertising campaigns for users both
on the individual level and on a larger scale
 Application of Big data
 Below are some examples of sectors in which Big data can
contribute by generating value.
 Big Data refers to the large volumes of complex and
unprocessed data that organizations use to make informed
business decisions.
 It is a powerful tool that drives various industries by
enabling data scientists, analytical modelers, and other
professionals to analyze vast amounts of transactional data.
Travel and Tourism
 Big Data helps forecast travel facilities requirements at multiple
locations, improve business through dynamic pricing, and enhance
customer experiences.
Financial and Banking Sector
 Banks and financial institutions use Big Data to analyze customer
behavior based on investment patterns, shopping trends, and financial
backgrounds. This helps in offering personalized services and detecting
fraudulent activities.
Healthcare
 In healthcare, Big Data enables predictive analytics, personalized
healthcare, and efficient patient management. It helps in analyzing
patient data to provide better treatment and preventive care.
Telecommunication and Media
 Telecommunications and multimedia sectors generate zettabytes of data
daily. Big Data technologies help in managing and analyzing this data to
improve services and customer satisfaction.
Social Media
 Social media platforms generate massive amounts of data
daily. Big Data helps in processing and analyzing this data to
understand user behavior, preferences, and trends1.
Government and Military
 Government agencies use Big Data for managing utilities,
traffic control, and crime prevention. The military uses it for
processing vast amounts of data from various sources to
enhance operational efficiency.
E-commerce
 E-commerce platforms like Amazon use Big Data to manage
customer relationships, handle large traffic during sales, and
implement better marketing strategies. It helps in organizing
and analyzing customer data for improved business decisions.
Smart Traffic Systems
 Big Data is used to analyze traffic conditions through data
collected from cameras, GPS devices, and other sources.
This helps in recommending less congested routes and
reducing fuel consumption.
Cyber security
 Big Data analytics are used to detect system vulnerabilities
and cyber threats by analyzing patterns in web traffic and
user behavior. This helps in preventing cyber attacks and
ensuring data security.
Education Sector
 Online educational platforms use Big Data to identify
potential candidates for courses and provide personalized
learning experiences. It helps in analyzing student data to
improve educational outcomes.
Energy Sector
 Smart electric meters use Big Data to analyze power consumption
patterns and recommend optimal usage times to reduce electricity
bills. This helps in efficient energy management.
Media and Entertainment
 Companies like Netflix and Spotify use Big Data to analyze user
preferences and viewing habits. This helps in recommending
content and improving user engagement.
IoT
 Big Data is crucial in the Internet of Things (IoT) for analyzing
data from sensors and devices. It helps in predictive maintenance,
improving operational efficiency, and providing better services.
 Big Data is a versatile technology with applications across
various sectors, driving innovation and efficiency in today's data-
driven world.
 Cloud computing
Cloud computing
 Is a technology that allows users to access and utilize computing resources
over the internet, rather than relying on local hardware or on-premises
servers
 Is the on-demand availability of computing resources (such as storage and
infrastructure), as services over the internet.
Key Principles of Cloud Computing
On-Demand Access: Users can access cloud services and resources whenever
needed, without the need for physical hardware investments.
Pay-As-You-Go: Users only pay for the services they use, which helps in
reducing costs associated with maintaining idle resources.
Scalability: Cloud computing allows users to scale resources up or down based
on demand, ensuring optimal performance and cost efficiency.
Virtualization: This technology enables the creation of virtual instances of
computing resources, providing better management and utilization of hardware.
 Benefit of cloud computing
 Cost Efficiency: Reduces capital expenditures by eliminating the
need for physical infrastructure investments.
 Flexibility and Scalability: Easily scale resources based on
demand, ensuring efficient handling of workloads.
 Collaboration and Accessibility: Provides easy access to data
and applications from anywhere, encouraging collaborative work.
 Automatic Maintenance and Updates: Cloud providers handle
infrastructure management and software updates, ensuring access
to the latest technologies.
 It offers strategic value:- Because cloud providers stay on top of
the latest innovations and offer them as services to customers,
enterprises can get more competitive advantages and a higher
return on investment than if they’d invested in soon-to-be
obsolete technologies.
 Cloud computing has also become indispensable in business
settings, from small startups to global enterprises. Its
many business applications include enabling remote work by
making data and applications accessible from anywhere, creating
the framework for seamless Omni channel customer engagement
and providing the vast computing power and other resources
needed to take advantage of cutting-edge technologies
like generating AI and quantum Computing .
 A cloud services provider (CSP) manages cloud-based technology
services hosted at a remote Datacenter and typically makes these
resources available for a pay-as-you-go or monthly subscription
fee.
 The following are a few of the most integral components of
today’s modern cloud computing architecture.
 Data centers
 Networking capabilities
 Virtualization
 Types of Cloud Computing

 Cloud services can be classified into Four general service

delivery categories:
1. Public cloud 3. Hybrid cloud
2. Private cloud 4.Multi cloud
1.Public cloud:-is a type of cloud computing in which a
cloud service provider makes computing resources available
to users over the public internet.
 Are run by third-party cloud service providers. They offer
compute, storage, and network resources over the internet,
enabling companies to access shared on-demand resources
based on their unique requirements and business goals.
 If you rent 10GB of public cloud Storage.
 These include SaaS applications, individual virtual
machines(VMS), bare metal computing hardware, complete
enterprise-grade infrastructures and development platforms.
 These resources might be accessible for free or according to
subscription-based or pay-per-usage pricing models.
2. Private cloud:-is a cloud environment where all cloud infrastructure
and computing resources are dedicated to one customer only.
 Are built, managed, and owned by a single organization and
privately hosted in their own data centers, commonly known as “on-
premises” or “on-prem.”
 Many companies choose a private cloud over a public cloud
environment to meet their regulatory compliance requirements.
Entities like government agencies, healthcare organizations and
financial institutions often opt for private cloud settings for
workloads that deal with confidential documents, personally
identifiable information (PII), intellectual property, medical records,
financial data or other sensitive data.
3.Hybrid cloud:-: a combination of public cloud, private cloud and
on-premises environments.
 Connects a combination of these three environments into a single,
flexible infrastructure for running the organization’s applications and
workloads.
 Has expanded beyond physical connectivity and cloud migration to
offer a flexible, secure and cost-effective environment that supports
the portability and automated deployment of workloads across
multiple environments.
4. Multi cloud:- uses two or more clouds from two or more different
cloud providers.
 Environment can be as simple as email SaaS from one vendor and
image editing SaaS from another.
 Environment also reduces exposure to licensing, security and
compatibility issues that can result from “Shadow IT“ any software,
hardware or IT resource used on an enterprise network without the IT
department’s approval and often without IT’s knowledge or oversight.
 Cloud Computing Service
 There are several cloud Computing services. The three
major cloud Computing services are:-
1. Software as a service(Saas).
2. Platform as a Service(PaaS).
3. Infrastructure as a Service(IaaS).
1.Software as a service(Saas):-is a distribution model that
delivers software applications over the internet; these
applications are often called web services.
 Also known as cloud-based software or cloud applications,
is application software hosted in the cloud.
 Users access SaaS through a web browser, a dedicated
desktop client or an API that integrates with a desktop or
mobile operating system
 Examples of SaaS - are Google Workspace, Dropbox, Microsoft Office 365,
Slack, Cisco WebEx, Concur, GoToMeeting, and many more
2.Platform as aService(PaaS):-provides software developers with an on
demand platform hardware, complete software stack, infrastructure and
development tools for running, developing and managing applications without the
cost, complexity and inflexibility of maintaining that platform on premises.
 Cloud providers host development tools on their infrastructures.
 Examples of PaaS – are Windows Azure, Heroku, Force.com, Google App Engine,
Apache Stratos, OpenShift, and many more.
3.Infrastructure as a Service(IaaS):-
 Provides on demand access to fundamental computing resources physical and
virtual servers, networking and storage over the internet on a pay-as-you-go
basis.
 Users have an allocated storage capacity and can start, stop, access and
configure the VM and storage as desired.
 Offer small, medium, large, extra-large and memory or compute optimized
instances, in addition to enabling customization of instances for various
workload needs.
 Is closest to a remote data center for business users.
 Examples of IaaS are - Amazon Web Services (AWS), Cisco Metapod, Microsoft Azure, Google
Compute Engine (GCE), and many more
 Fog Computing
 Fog computing is a decentralized infrastructure that places
storage and processing components at the edge of the cloud.
 Is an extension of the cloud.
 Edge computing
 Is a subset of fog computing that involves
processing data right at the point of creation.
 Edge devices include routers, cameras,
switches, embedded servers, sensors, and
controllers. In edge computing, the data
generated by these devices are stored and
computed at the device itself, and the system
doesn’t look at sharing this data with the
cloud.
 Fog Computing Components

1. Physical & virtual nodes (end devices).

2. Fog nodes.
3. Monitoring services.
4. Data processors.
5. Resource manager.
6. Security tools.
7. Applications.
 Internet of Things (IoT)
 Refers to the network of physical devices, vehicles, appliances, and
other objects embedded with sensors, software, and connectivity,
enabling them to collect and exchange data.
 This interconnectedness allows for the automation and optimization
of various processes, transforming industries and daily life.
The history of the Internet of Things
 The origins of the IoT date to the late 1960s.
 It was then that a group of prominent researchers began exploring
ways to connect computers and systems.
 A prime example of this work was ARPANET the network created
by the Advanced Research Projects Agency (ARPA) of the U.S.
Defense Department; this network was a forerunner of
today’s Internet.
IOT(Internet Of Things)
 Major advantages of IoT

1. Easy Access
 You may simply get the necessary information right away,
in real time, from wherever you are.
 All you need is an internet connection and a smart device.
2. Quick operation
 You can finish many tasks very quickly because of the
inflow of data.
3. Monitoring Data
 Keeping track of product expiration dates also increases
your safety.
4.Cost Saving
 It helps people save money and energy.
 Limitations of IOT

1. Dependency on Technology
 The internet connection is the key requirement for IoT.
So, it is useless if the Internet is absent.
2. Security
 A lot of information is available on the Internet as it
connects to all household appliances, industrial
equipment, and public sector services.
Thus, hackers can attack this data to get private
information and the results would be terrible.
3. Inter Compatibility
 Many times, equipment from different manufacturers
connects.
 The solution to this drawback is for the manufacturers to
create a shared standard. But there is a chance that the
technical issues could still exist.
 How Does IOT Work?

IoT works like this:

I. Devices have hardware, like sensors, that
collect data.
II. The data collected by the sensors is then
shared via the cloud and integrated with
software.
III. The software then analyzes and transmits
the data to users via an app or website.
Smart devices connect to an IoT platform,
described by the experts at IoT For All as
“the support software that connects
everything in an IoT system.”
 The Internet of Things is largely made possible by
technologies that connect devices and enable them to
communicate with one another. Connectivity options have a
range of pros and cons, with some more suitable for certain
use cases like smart homes while others may be more
appropriate for IoT applications like industrial automation.
These technologies can be divided into two categories: IoT
data protocols that allow information to be exchanged
between devices even without an internet connection,
and IOT Network protocols that link devices to one another
and to the internet.
 10 common IOT protocols:
1.Wi-Fi 3.Zigbee 5.LoRaWAN 7. MQTT 9. AMQP
2.Bluetooth 4.Cellular 6.Sigfox 8.CoAP 10. HTTP
 Application of Internet of Things

I. Smart Agriculture
II. Smart Home
III. Smart Vehicles
IV. Smart Healthcare
V. Smart Cities
VI. Maintenance management
VII. Smart outlets
Smart Agriculture:- This can be done by first collecting data
for a farm such as soil quality, sunlight levels, seed type, and
rainfall density from various sources like farm sensors,
satellites, local weather stations from various sources like
farm sensors, satellites, local weather stations, etc.
Smart Home:- connecting all the home applications like
lighting, air conditioners, locks, thermostat, etc.
Smart Vehicles:- Smart vehicles or self-driving cars are Iot
applications as they can be called are pretty dependent on IoT.
 These cars have a lot of features that are integrated with
each other and need to communicate such as the sensors that
handle navigation, various antennas, controls for speeding or
slowing down, etc.
 Tesla Motors’ cars use the latest advancements in Artificial
Intelligence and the Internet of Things.
Smart Healthcare:- the Healthcare Industry where doctors can
monitor patients remotely through a web of interconnected devices.
 One of the most common uses of IoT applications in healthcare is
using robots. These include surgical robots that can help doctors in
performing surgeries more efficiently with higher precision and
control.
 There are also disinfectant robots that can clean surfaces quickly
and thoroughly using high-intensity ultraviolet light.
Smart Cities:-Smart City-Traffic Management, Waste management,
and Water Distribution.
 Supply chain:- Real-time goods tracking and getting inventory
information.
 Connect Health:-Patient care, calorie expenditure, heart rate
pattern, temperature, activity levels.
 Connected cars:-Automobile companies can automatically
perform billing, insurance, parking.
 Maintenance management
One of the areas where the application of IoT technology is most
extensive is precisely maintenance management. Through the
combination of sensors and software specialized in CMMS/
EAM maintenance management
The software in charge of processing and arranging
the data collected by the sensors are designed to
specifically address the maintenance management
needs of physical assets, their application is almost
unlimited.
The real-time monitoring of physical assets allows
determining when a measurement is out of range and
it is necessary to perform condition-based
maintenance (CBM), or even applying Artificial
Intelligence (AI) algorithms such as Machine
Learning or Deep Learning to predict the failure
before it happens.
 Smart outlets
 Turning a device on/off remotely, tracking the
energy levels of a device, etc. Generally, IoT
affects many areas of day-to-day life and many
social and technical issues can be solved by
using the applications of IoT.