COMM 527: Distributed Systems

Distributed System Models

 Presentation Outline

 Introduction
 Physical Models:
 Three Generations of DS: Early, Internet-Scale, Contemporary
 Architectural Models
 Software Layers

 System Architectures


Client-Server
 Clients and a Single Server, Multiple Servers, Proxy Servers with Caches, Peer
Model

Alternative Client-Sever models driven by:
 Mobile code, mobile agents, network computers, thin clients, mobile devices,
and spontaneous networking

Design Challenges/Requirements
 Fundamental Models – formal description
 Interaction, failure, and security models.

 Summary

2
 Introduction

 Real world systems should be designed to

function correctly in ALL circumstances/scenarios.
 Distributed system models helps in…

..classifying and understanding different implementations

..identifying their weaknesses and their strengths

..crafting new systems outs of pre-validated building blocks
 We will study distributed system models from
different perspectives

Structure, organization, and placement of components

Interactions

Fundamental properties of systems

3
 Architectural Model

 An Architectural model of a distributed system

is concerned with the placement of its parts
and relationship between them. Examples:
 Client-Server (CS) and peer process models.
 CS can be modified by:

The partitioning of data/replication at cooperative
servers

The caching of data by proxy servers

The use of mobile code and mobile agents

The requirements to add or remove mobile devices.

4
 Fundamental Models

 Fundamental Models are concerned with a formal

description of the properties that are common in all
of the architectural models
 Models addressing time synchronization, message
delays, failures, security issues are addressed as:
 Interaction Model – deals with performance and the
difficulty of setting of time limits in a distributed system.
 Failure Model – specification of the faults that can be
exhibited by processes
 Secure Model – discusses possible threats to processes
and communication channels.

5
 Physical Models

 A representation of the underlying h/w elements of

a DS that abstracts away specific details of the
computer/networking technologies.
 Baseline physical model
 3 Generations of DS:

Early distributed systems [late 70-80s]: LAN-based

Internet-scale distributed systems [early 90-2005]:
Clusters, grids, P2P, Clouds

Contemporary distributed systems: dynamic nodes like
mobile-based services (nodes are very dynamic not
static like other models).
6
 Architectural Models

 The architecture of a system is its structure in terms

of separately specified components.

Its goal is to meet present and likely future demands.

Major concerns are making the system reliable,
manageable, adaptable, and cost-effective.
 Architectural Model:

Simplifies and abstracts the functions of individual
components

The placement of the components across a network of
computers – define patterns for the distribution of data and
workloads

The interrelationship between the components – ie.,
functional roles and the patterns of communication
between them.

7
 Architectural Models

 Architectural Model - simplifies and abstracts

the functions of individual components:
 An initial simplification is achieved by classifying
processes as:

Server processes

Client processes client
client server
server

Peer processes
 Cooperate and communicate in a symmetric manner to
perform a task.

peer
peer

peer
peer 8
 Software Architecture and Layers

 The term software architecture referred:


Originally to the structure of software as layers or modules in a single computer.

More recently in terms of services offered and requested between processes in the
same or different computers.
 Breaking up the complexity of systems by designing them through layers and
services

Layer: a group of related functional components

Service: functionality provided to the next layer.

Layer N

Layer 2

(services offered to above layer)

Layer 1

9
Software and hardware service layers in
distributed systems

Applications, services

Middleware

Operating system
Platform

Computer and network hardware

10
 Platform

 The lowest hardware and software layers are often

referred to as a platform for distributed systems and
applications.
 These low-level layers provide services to the layers
above them, which are implemented independently
in each computer.
 Major Examples

Intel x86/Windows

Intel x86/Linux

Intel x86/Solaris

SPARC/SunOS

PowerPC/MacOS

11
 Middleware

 A layer of software whose purpose is to mask heterogeneity present

in distributed systems and to provide a convenient programming
model to application developers.
 Major Examples:

Microsoft D-COM (Distributed Components Object Model)

Sun Java RMI

Modern Middleware:
 IBM WebSphere
 Microsoft .NET
 Sun J2EE
 Google AppEngine

12
 System Architecture

 The most evident aspect of DS design is the

division of responsibilities between system
components (applications, servers, and other
processes) and the placement of the
components on computers in the network.
 It has major implication for:
 Performance, reliability, and security of the
resulting system.

13
 Client Server Basic Model:
Clients invoke individual servers

Client invocation invocation Server

result result
Server

Client
Key:
Process: Computer:

 Client processes interact with individual server processes in a separate computer

in order to access data or resource. The server in turn may use services of other
servers.
 Example:

A Web Server is often a client of file server.

14
Client-Server Architecture Types


Two-tier model (classic)

client
client server
server


Three-tier (when the server, becomes a client)

client
client Server/client
Server/client server
server


Multi-tier (cascade model)
server
server

client
client Server/client
Server/client Server/client
Server/client
server
server
15
 Clients and Servers

 General interaction between a client and a server.

16
A service provided by multiple servers
Service

Server
Client

Server

Client
Server

 Services may be implemented as several server processes in separate host computers.

 Example: Cluster based Web servers and apps such as Google, parallel databases Oracle

17
 Proxy Servers and Caches

Client Web
server
Proxy
server

Client Web
server

 A cache is a store of recently used data.

18
Peer Processes: A distributed application
based on peer processes
Peer 2

Peer 1
Application

Application

Sharable Peer 3
objects
Application

Peer 4

Application

Peers 5 .... N

 All of the processes play similar roles, interacting cooperatively as peers to

perform distributed activities or computations without distinction between clients
and servers. E.g., music sharing systems Napster, Kaza, etc.
 Distributed “whiteboard” – users on several computers to view and interactively
modify a picture between them.
19
P2P with a Centralized Index Server
(e.g. Napster Architecture)

peer
peer

peer
peer

peer
peer
peer
peer

peer
peer

peer
peer
peer
peer

20
Variants of Client Sever Model: Mobile
code and Web applets
a) client request results in the downloading of applet code

Client Web
Applet code server

b) client interacts with the applet

Web
Client Applet server

 Applets downloaded to clients give good interactive response

 Mobile codes such as Applets are potential security threat, so the
browser gives applets limited access to local resources (e.g. NO
access to local/user file system).
21
 Variants of Client Sever Model: Mobile
Agents

 A running program (code and data) that travels from one computer to
another in a network carrying out an autonomous task, usually on
behalf of some other process
 Its main advantages are flexibility and savings in communications cost
 Potential security threat to the resources in computers they visit. The
environment receiving agent should decide which of the local resource
to allow.
 Agents themselves can be vulnerable – they may not be able to
complete task if they are refused access.

22
 Thin clients and compute servers

Compute server
Network computer or PC

Thin network Application

Client Process

 Network computer: download OS and applications from the

network and run on a desktop at runtime.
 Thin clients: Windows-based UI on the user machine and
application execution on a remote computer. E.g, X-11 system.

23
 Interaction Model

 Computation occurs within processes;

 The processes interact by passing messages,
resulting in:
 Communication (information flow)
 Coordination (synchronization and ordering of activities)
between processes.
 Two significant factors affecting interacting
processes in a distributed system are:
 Communication performance is often a limiting
characteristic.
 It is impossible to maintain a single global notion of time.

24
 Interaction Model:
Performance of Communication Channel

 The communication channel in our model is realised in a variety

of ways in DSs. E.g., by implementation of:

Streams

Simple message passing over a network.
 Communication over a computer network has performance
characteristics:

Latency:

A delay between the start of a message’s transmission from one
process to the beginning of reception by another.

Bandwidth:

the total amount of information that can be transmitted over in a
given time.

Communication channels using the same network, have to share the
available bandwidth.

Jitter

The variation in the time taken to deliver a series of messages. It is
very relevant to multimedia data.

25
 Interaction Model:
Computer clocks and timing events

 Each computer in a DS has its own internal clock, which can be

used by local processes to obtain the value of the current time.
 Therefore, two processes running on different computers can
associate timestamp with their events.
 However, even if two processes read their clocks at the same
time, their local clocks may supply different time.

This is because computer clock drifts from perfect time and their
drift rates differ from one another.
 Even if the clocks on all the computers in a DS are set to the
same time initially, their clocks would eventually vary quite
significantly unless corrections are applied.

There are several techniques to correct time on computer clocks.
For example, computers may use radio receivers to get readings
from GPS (Global Positioning System) with an accuracy about 1
microsecond.

26
 Interaction Model:
Two variants of the interaction model

 In a DS it is hard to set time limits on the time taken for process

execution, message delivery or clock drift.
 Synchronous DS – hard to achieve:
 The time taken to execute a step of a process has known lower

and upper bounds.

 Each message transmitted over a channel is received within a

known bounded time.

 Each process has a local clock whose drift rate from real time

has known bound.

 Asynchronous DS: There is NO bounds on:
 Process execution speeds

 Message transmission delays

 Clock drift rates.

27
 Interaction Model:
Event Ordering

 In many DS applications we are interested in

knowing whether an event occurred before,
after, or concurrently with another event at
other processes.
 The execution of a system can be described in
terms of events and their ordering despite the lack
of accurate clocks.
 Consider a mailing list with:
users X, Y, Z, and A.

28
 Real-time ordering of events

send receive receive

X
1 m1 4
m2
send
receive
Y 2 3 Physical
receive time

send
Z
receive receive

m3 m1 m2
A
receive receive receive
t1 t2 t3

29
 Inbox of User A looks like:

Item From Subject

23 Z Re: Meeting
24 X Meeting
26 Y Re: Meeting

 Due to independent delivery in message delivery, message may

be delivered in different order.
 If messages m1, m2, m3 carry their time t1, t2, t3, then they can
be displayed to users accordingly to their time ordering.
30
 Failure Model

 In a DS, both processes and communication

channels may fail – i.e., they may depart from
what is considered to be correct or desirable
behavior.
 Types of failures:
 Omission Failure
 Arbitrary Failure
 Timing Failure

31
 Processes and Channels

process p process q

send m receive

Communication channel
Outgoing message buffer Incoming message buffer

 Communication channel produces an omission failure if it

does not transport a message from “p”s outgoing
message buffer to “q”’s incoming message buffer. This is
known as “dropping messages” and is generally caused
by a lack of buffer space at the receiver or at gateway or
by a network transmission error.
32
 Omission and Arbitrary Failures

Class of failure
Affects Description
Fail-stop Process
Process halts and remains halted. Other processe
detect this state.
Crash Process
Process halts and remains halted. Other processe
not be able to detect this state.
Omission AChannel
message inserted in an outgoing message buffer
arrives at the other end’s incoming message buff
Send-omission A process completes send,
Process a but the
put in its outgoing message
message buffer.
is not
Receive-omission
Process
A message is put in a process’s incoming messag
buffer, but that process does not receive it.
Arbitrary Process/channel
Process or exhibits arbitrary behaviour: it
(Byzantine)send/transmit
channel arbitrary messages at arbitrary ti
commit omissions; a process may stop or take an
incorrect step.

33
 Timing Failures

Class of Failure
Affects Description
Clock Process
Process’s local clock exceeds the bounds
rate of drift from real time.
Performance Process
Process exceeds the bounds on the interv
between two steps.
Performance Channel
A message’s transmission takes longer th
stated bound.

34
 Masking Failures

 It is possible to construct reliable services from

components that exhibit failures.
 For example, multiple servers that hold replicas of data can
continue to provide a service when one of them crashes.
 A knowledge of failure characteristics of a
component can enable a new service to be designed
to mask the failure of the components on which it
depends:
 Checksums are used to mask corrupted messages.

35
 Security Model

 The security of a DS can be achieved by

securing the processes and the channels
used in their interactions and by protecting
the objects that they encapsulate against
unauthorized access.

36
Protecting Objects: Objects and principals

Access rights Object

invocation

Client
result Server

Principal (user) Network Principal (server)

 Use “access rights” that define who is allowed to perform operation on a

object.
 The server should verify the identity of the principal (user) behind each
operation and checking that they have sufficient access rights to perform
the requested operation on the particular object, rejecting those who do
not.
37
 Potential Threats

Copy of m

The enemy
m’
Process p m Process q
Communication channel

 To model security threats, we postulate an enemy that is capable of

sending any process or reading/copying message between a pair of
processes
 Threats form a potential enemy: threats to processes, threats to
communication channels, and denial of service.
38
 Defeating security threats: Secure
channels

PrincipalA PrincipalB

Process p Secure channel Process q

 Encryption and authentication are used to build secure channels.

 Each of the processes knows the identity of the principal on
whose behalf the other process is executing and can check their
access rights before performing an operation.

39
 Summary

 Most DSs are arranged accordingly to one of a

variety of architectural models:
 Client-Server

Clients and a Single Sever, Multiple Servers, Proxy Servers
with Cache, Peer Model
 Alternative Client-Sever models driven by:

Mobile code, mobile agents, network computers, thin clients,
mobile devices and spontaneous networking
 Fundamental Models – formal description
 Interaction, failure, and security models.

40