Introduction to Cloud Storage Models &

Communication APIs
• Cloud Computing is a transformative computing model that involves delivering
applications and services over the internet.

• NIST defines cloud computing-Cloud Computing is a model for enabling ubiquitous,

convenient, on-demand network access to a shared pool of configurable computing
resources(eg., networks, servers, storage, applications and services) that can be rapidly
provisioned and released with minimal management effort or service provider interaction.
 WAMP for IoT

• Web Application Messaging Protocol (WAMP) is a sub-protocol of Websocket

which provides publish-subscribe and remote procedure call (RPC)
messaging patterns.
 WAMP-AutoBahn for IoT
• Web Application Messaging Protocol(WAMP) is a sub-protocol of websocket which provides
publish-subscribe and remote procedure call(RPC) messaging patterns.
• WAMP enables distributed application architectures where the application components are
distributed or multiple nodes and communicate with messaging patterns provided by WAMP.
• The key concepts of WAMP are:
• Transport: Transport is channel that connects two peers.
• Session: Session is a conversation between two peers that runs over a transport.
• Client: Clients are peers that can have one or more roles. In publish-subscribe model client can
have following roles:
– Publisher: Publisher publishes events (including payload) to the topic maintained by the Broker.
– Subscriber: Subscriber subscribes to the topics and receives the events including the payload.
•In RPC model client can have following roles:
– Caller: Caller issues calls to the remote procedures along with call arguments.
– Callee: Callee executes the procedures to which the calls are issued by the caller and
returns the results back to the caller.
• Router: Routers are peers that perform generic call and event routing. In publish-
subscribe model Router has the role of a Broker:
– Broker: Broker acts as a router and routes messages published to a topic to all
subscribers subscribed to the topic.
•In RPC model Router has the role of a Dealer:
– Dealer: Dealer acts a router and routes RPC calls from the Caller to the Callee and
routes results from Callee to Caller.
• Application Code: Application code runs on the Clients (Publisher, Subscriber, Callee or
Caller).
 Xively Cloud for IoT
• Xively is a commercial Platform-as-a-service that can be used for creating solutions for
Internet of Things.
• With Xively cloud, IoT developers can focus on the front-end infrastructure and devices
for IoT(that generate the data), while the backend data collection infrastructure is
managed by Xively.
• Xively platform comprises of a message bus for real-time message movement and
routing, data services for time series archiving, directory services for device provisioning
and management.
• Xively provides an extensive support for various languages and platforms.
• The Xively libraries leverage standards-based API over HTTP, Sockets and MQTT for
connecting IoT devices to the Xively cloud.
• When you create a device, Xively automatically creates a Feed-ID and an API key to connect the
device as shown below screenshot.
• Each device has a unique Feed-ID.
• Feed-ID is a collection of channels or datastreams defined for a device and then associated meta-data.
• API keys are used to provide different levels for permissions.
• The default API key has read, update, create and delete permissions.
• Xively devices have one or more channels. Each channel enables bi-directional communication
between the IoT devices and the Xively Cloud.
• IoT devices can send data to a channel using the Xively APIs.
• For each channel, you can create one or more triggers.
• A trigger specification includes a channel to which the trigger corresponds, trigger condition( eg.,
channel value less than or greater than certain value) and an HTTP POST URL to which the request
is sent when the trigger fires.
• Triggers are used for integration with third-party applications.
 Python Web Application Framework- Django
• To build IoT applications that are backed by Xively cloud or any other data collection
systems, we require some type of web application framework.
• Django is an open source web application framework for developing web applications in
Python.
• A “web application framework” in general is a collection of solutions, packages and best
practices that allows development of web applications and dynamic websites.
• Django is based on the well-known Model-Template-View architecture and provides a
separation of the data model from the business rules and the user interface.
• Django provides a unified API to a database backend.
• Django can work with different databases without requiring any code changes.
• Django is best suited for IoT applications.
• Django concisely stated, consists of an object-relational mapper, a web templating system
and a regular expression-based URL dispatcher.
Django Architecture:
• Django is a Model-Template-View(MTV) framework wherein the roles of model,
template and view.
Model:
• The model acts as a definition of some stored data and handles the interactions with the
database.
• In a web application, the data can be stored in a relational database, non-relational
database and an XML file etc.
• A Django model is a Python class that outlines the variables and methods for a particular
type of data.
Template:
• In a typical Django web application, the template is simply an HTML page with a few
extra placeholders.
• Django’s template language can be used to create various forms of text files(XML, email,
CSS, Javascript, CSV etc).
View:
• The view ties the model to the template.
• The view is where you write the code that actually generates from the web pages.
• View determines what data is to be displayed, retrieves the data from the database and
passes the data to the template.
 Starting Development with Django
Creating a Django Project and App:
• The commands for creating a Django project and an application within a project:
• When you create a new django project a number of files are created as below:
• _init_.py: This file tells python a number of files are created.
• [Link]: This file tells python that this folder is a Python Package.
• [Link]: This file contains the website’s settings
• [Link]: This file contains the URL patterns that map URLs to pages.
• A Django project can have multiple applications. Each project can have mutiple apps and
each app can be part of multiple projects.
• When a new application is created a new directory for the application is also created
Configuring a Database:
• Developers have a wide choice of databases that can be used for web applications
including both relational and non-relational databases.
• Django provides a unified API for database backends thus giving the freedom to choose
the database.
• Django supports various relational database engines including MYSQL, PostgreSQL,
Oracle and SQLite3.
• It also supports for non-relational databases such as MongoDb.
Defining a Model:
• A Model acts as a definition of the data in the database.
• In this we use an example of a weather station application that displays the temperature
data collected by an IoT device.
• The below figure shows an example of a Django model for TemperatureData.
• The TemperatureData table in the database is defined as a Class in he Django model.
• Each class that represents a database table is a subclass of [Link] which
contain functionality that allows the models to interact with the database.
• The TemperatureData class has fields timestamp, temperature, lat and lon all of which are
CharField.
• To sync the models with the database simply run the command:
>python [Link] syncdb
• When the syncdb command is run for the first time, it creates all the tables defined in the
Django model in the configured database.
Django Admin Site:
• Django provides an administrator system that allows you to manage the website without
writing additional code.
• The admin system reads the Django model and provides an interface that can be used to
add content to the site.
Defining a view:
• The view contains the logic the glues the model to the template.
• The view determines the data to be displayed in the template, retrieves the data from the
database and passes it to the template.
• The view also extracts the data posted in a form in the template and inserts in the
database.
• Each page in the website has a separate view, which can be seen in [Link] file.
• Views also performs additional tasks such as authentication, sending emails etc.
Defining a Template:

• A Django template is typically an HTML file(it may be such as XML, email, CSS,
Javascript etc).
• Django allows the separation of the presentation of data from the actual data by
using placeholders and associated logic.
• A template receives a context from the view and presents the data in the context
variables in the placeholders.
Defining the URL Patterns:

• URL patterns are a way of mapping the URLs to the views that should handle the
URL requests.

• The URLs requested by the user are matched with the URL patterns and the view
corresponding to the pattern that matches the URL is used to handle the request.

• URL patterns are constructed using regular expressions.

• The simplest regular expression (r’^$’) corresponds to the root of the website or
home page.
 Designing a RESTful Web API
• With the Django framework already installed, the Django REST framework can
be installed as follows:
• The REST API is allows you to create, view, update and delete a collection of resources
where each resource represents a sensor data reading from a weather monitoring station.
• The figure 8.17 shows the Django model for such a station.
• The station model contains four fields-station name, timestamp, temperature, latitude and
longitude.
• The 8.18 figure shows the Django views for the REST API.
• Views Sets are used for the views that allow you to combine the logic for a set of related
views in a single class.
• The fig 8.19 shows the serializers for the REST API.
• Serializers allow complex data (such as query sets and model instances) to be converted
to native Python datatypes that can then be easily represented into JSON, XML or other
content types.
• Serializers also provide de-serialization, allowing parsed data to be converted
back into complex types, after first validating the incoming data.
• The Fig 8.20 shows the URL patterns for the REST API.
• Since Viewsets are used instead of viws, we can automatically generate the URL
conf for our API, by simply registring the viewsets.
• Routers automatically determining how the URLs for an application should be
mapped to the logic that deals with handling incoming requests.
 Amazon Web Services-Introduction

• Secure cloud service platform

• Flexibility

• Scalability

• Security

• Cost effective

• Case study: Supply chain and irrigation

• AWS IoT Core: This service provides a platform for connecting IoT devices securely to
the cloud. It allows devices to send and receive data through MQTT and HTTP protocols,
and provides features such as device shadowing, which enables offline devices to sync
with the cloud when they come back online.
• AWS IoT Analytics: This service allows you to process and analyze IoT data in the
cloud. It provides tools for data cleansing, filtering, and transformation, and can integrate
with other AWS services like Amazon S3, Amazon Redshift, and Amazon QuickSight.
• AWS IoT Device Management: This service allows you to manage and update your IoT
devices at scale. It provides features such as remote device management, over-the-air
updates, and device grouping, which allows you to manage large fleets of devices
efficiently.
• AWS IoT SiteWise: This service allows you to collect, store, and organize industrial IoT
data from multiple sources. It provides a centralized repository for data, and allows you to
create custom dashboards and visualizations to monitor and analyze your IoT data.
 Elastic Compute Cloud(EC2)
• Provides secure and resizable compute capacity in cloud(developer option)
• Scaling(Instances scaled up and down)
• Integrated with other services(s3,RDS)
• Pay for what you use.
• Instances can be launched in one or more regions and zones
• Support for different OS
• Works with amazon to provide secure network to resources
Example:
• Smart homes(collection of temp)
• Create an instance with power and memory
• EC2 instances for controlling room temperatures
• If room temp is too high, the EC2 instance can send a command to the thermostat to turn
on the air conditioning. Using EC2 in this way allows you to build a scalable, responsive,
and cost-effective IoT application. You can process data in real-time, ensuring that your
application responds quickly to changes in the environment. Additionally, because EC2 is
a managed service, you do not need to worry about managing infrastructure or scaling
issues, allowing you to focus on building your application.
 Autoscaling
• Fault tolerance
• Availability

Minimum 2 Maximum 4
• Cost management
• Manual scaling
• Scheduled scaling
• Dynamic scheduling
• For example, if the incoming data volume increases beyond a certain
threshold, autoscaling can spin up additional instances to handle the
increased workload. Similarly, if the data volume decreases,
autoscaling can terminate unneeded instances to save costs.

• Example: Soil moisture scaling

• Gmail servers
 Simple Storage Service(S3)
• Provides object storage which is built for storing and receiving any amount of data from
anywhere over tha internet.
• Durability:99.99999999999%
• Availability:99.99%
• Cost effective
• Scalability
• Security
• Objects consists of data and metadata
• Data encryption
• Collection of data-central hub-processed and analyzed-make decisions.
• Store the incoming data
• S3 bucket to store data
• Safe and backup in the event failure
• Example, you can use AWS Lambda to trigger functions that analyze the data as it
arrives in the S3 bucket, and take actions based on the insights gained from the
analysis. Alternatively, you can use AWS Glue to transform and clean the data
before storing it in a data warehouse
 RDS

• DB engine: [Link] DB, mysql

• Backup: auto

• manual

• cross region snap

• Security: VPC, Encryption

• e-commerce application
• highly available, scalable, and performant database.
• RDS to set up and manage your database, ensuring that it meets your application's
requirements.
• You can choose the database engine that best suits your needs and create a new
RDS instance using the AWS Management Console or the AWS CLI.
• desired storage, compute capacity, and network settings, and choose the
availability zone where the instance will be deployed.
• Once the instance is set up, you can connect to it from your application using
standard database APIs such as JDBC or ODBC. You can use the database to store
product data, user data, and other application data, and use SQL queries to retrieve
and update the data.
 Dynamo db
• Amazon DynamoDB is a fully managed NoSQL database service that
allows to create database tables that can store and retrieve any amount
of data. It automatically manages the data traffic of tables over
multiple servers and maintains performance.
• It also relieves the customers from the burden of operating and scaling
a distributed database. Hence, hardware provisioning, setup,
configuration, replication, software patching, cluster scaling, etc. is
managed by Amazon.
• Suppose you are building an IoT application that collects data from a large
number of sensors and devices. The data is sent to a central hub, where it is
processed and analyzed to derive insights and make decisions. To store the
incoming data, you can use DynamoDB.

• You can create a new DynamoDB table and define the attributes and primary key
for the table. You can configure the table to store the incoming data from your IoT
devices. Each device can be configured to send its data to a specific table in
DynamoDB, allowing you to organize and manage your data more efficiently.
 SQS
• SQS stands for Simple Queue Service.
• Amazon SQS is a web service that gives you access to a message queue that can be used to
store messages while waiting for a computer to process them.
• Suppose the user wants to look for a package holiday and wants to look at the best possible
flight.
• A User types a query in a browser, it then hits the EC2 instance. An EC2 instance looks
"What the user is looking for?", it then puts the message in a queue to the SQS. An EC2
instance pulls queue. An EC2 instance continuously pulling the queue and looking for the
jobs to do. Once it gets the job, it then processes it. It interrogates the Airline service to get
all the best possible flights. It sends the result to the web server, and the web server sends
back the result to the user. A User then selects the best flight according to his or her budget.
 EMR
• Amazon Elastic Map Reduce (Amazon EMR) is a web service that makes it easy
to process large amounts of data quickly and cost-effectively.
• The central component of Amazon EMR is the cluster. A cluster is a collection of
Amazon Elastic Compute Cloud (Amazon EC2) instances. Each instance in the
cluster is called a node. Each node has a role within the cluster, referred to as the
node type. Amazon EMR also installs different software components on each node
type, giving each node a role in a distributed application like Apache Hadoop.
• Primary node: A node that manages the cluster by running software components to
coordinate the distribution of data and tasks among other nodes for processing.
The primary node tracks the status of tasks and monitors the health of the cluster
• Core node: A node with software components that run tasks and store data in the
Hadoop Distributed File System (HDFS) on your cluster. Multi-node clusters have
at least one core node.
• Task node: A node with software components that only runs tasks and does not
store data in HDFS. Task nodes are optional.
• Machine Learning. EMR's built-in ML tools use the Hadoop framework to build a
variety of algorithms to support decision making, including decision trees, random
forests, support vector machines, and logistic regression.
• Extract, Convert and Load. ETL is the process of moving data from one or more
data stores to another. Data transformation - such as sorting, aggregation and
joining - can be done using EMR.