Academy of Engineering and
Medical Sciences
Computer Engineering
Sem (10)
Distributed System
Lecture 3 ,4
Dr. Mutaz Hamed
“Latency is not zero”
Two options are made available to a client to retrieve
some information from a remote server:
• The first option allows the client to retrieve a 100MB
file (containing all the relevant information) at once.
• The second option requires the client to make 10
remote requests to the server. Each remote request will
return a file of 1MB (all 10 files are needed by the
client to retrieve the relevant information).
If the network link between the client and the server is
1sec latency and the available bandwidth is 100MB/sec
which option would be faster? What if the latency is
1msec? What do you conclude?
2
Increased Performance
• There are many applications (especially in
science) where good response time is needed.
• How can we increase execution time?
– By using more computers to work in parallel
• This is not as trivial as it sounds and there are
limits (we’ll come back to this)
– Parallel computing
• Examples:
– Weather prediction; Long-running simulations; …
3
How do we speed up things?
Many scientific operations are inherently parallel:
for(i=1;i<100;i++)
A[i]=B[i]+C[i]
for(i=1;i<100;i++)
for(j=1;j<100;j++)
A[i,j]=B[i,j]+C[i,j]
Different machines can operate on different data.
Ideally, this may speedup execution by a factor equal to the
number of processors
(speedup = ts / tp = sequential_time / parallel_time)
27-Aug-22 COMP28112 Lecture 2 4
Parallel Computing vs Distributed Computing
• Parallel Computing:
– Multiple CPUs in the same computer
• Distributed Computing:
– Networked Computers connected by a network.
• In Distributed Computing, we need to deal with:
– Communication (parallel computers may need to send data
to each other)
– Synchronisation (think about the problems with process
synchronisation)
5
How to parallelise an application
1. Try to identify instructions that can be executed in parallel.
These instructions should be independent of each other (the
result of one should not depend on the other)
2. Try to identify instructions that are executed on multiple data;
different machines operate on different data.
(Loops are the biggest source of parallelism)
6
In Summary...
• Several challenges / goals:
– Heterogeneity, openness, security, scalability, failure
handling, concurrency, transparency.
• Parallel computing: not really a topic for this module;
some of the problems are common, but the priorities
are different.
• Next slides: Architectures and Models.
7
➢ ARCHITECTURES OF DISTRIBUTED
SYSTEMS
➢ Fundamental Models
Architectures of Distributed Systems
• Types of Distributed Systems:
1. Tightly coupled
– Highly integrated machines that may look as a single
computer.
2. Loosely coupled (share nothing)
– Client-Server
– Peer-to-Peer
– A Key differentiation based on the Programming Interface:
• Distributed Objects
9
• Web Services
Tightly coupled systems
• Distributed shared memory (DSM)provides the illusion of a
single shared memory: it spares the programmer the concerns of
message passing.
Figure 18.1
The distributed shared memory abstraction
Dis tributed shared memory
DSM appears as
Proc ess
memory in address
ac cess ing DSM
space of proc ess
Physical Physical Physical
memory memory memory
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design
© Addison-Wesley Publishers 2000
Edn. 3 10
Issues with Distributed Shared Memory
• Machines are still connected by a network:
– Minimize network traffic
– Reduce the latency between request and completion
• How to keep track of the location of shared data
• How to overcome delays when accessing remote data
• How to make shared data concurrently accessible
• Replicate shared data on multiple machines:
– Need memory coherence
• Lots of research in the context of building parallel
computers .
11
loosely-coupled systems
Architectural Styles
Based on the logical organization of the components
of distributed systems:
• Layered architectures
• Object-based architectures
• Data-centered architectures
• Event-based architectures
12
The layered architectural style
13
The object-based architectural style
14
The Event-Based architectural style
15
Shared-Data Space Architectural Style
16
System Architectures of Distributed Systems
Figure 2.1
Software and hardware service layers in distributed systems
Applications, services
Middleware
Operating system
Platform
Computer and network hardware
27-Aug-22 COMP28112 Lecture 3
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4
17
© Pearson Education 2005
Middleware
• Middleware: a software layer that provides a
programming abstraction to mask the heterogeneity
of the underlying platforms (networks, languages,
hardware, …)
– E.g., CORBA, Java RMI
• But… the end-to-end argument implies that some
aspects of communication support cannot always be
abstracted away from applications.
18
Client-Server Model in Distributed
Systems
Figure 2.2
Clients invoke individual servers
Client invoc ation Server
invoc ation
result result
Server
Client
Key :
Proc ess : Computer:
27-Aug-22 COMP28112 Lecture 3 19
Client-Server Model in Distributed
Systems
• Characteristics of a server:
– Passive (slave)
– Waits for requests
– Upon receipts of requests, it processes them and sends replies
– Can be stateless (does not keep any information between requests)
or stateful (remembers information between requests)
• Characteristics of a client:
– Active (master)
– Sends requests
– Waits for and receives server replies
20
Peer-to-Peer (P2P) in Distributed
Systems
Figure 2.3
A distributed application based on peer processes
P eer 2
P eer 1
Appl ication
Appl ication
Shara ble P eer 3
objec ts
Appl ication
P eer 4
Appl ication
P eers 5 .... N
27-Aug-22 COMP28112 Lecture 3 21
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4
© Pearson Education 2005
Client-Server vs P2P
• Client-Server • P2P
– Widely Used – Symmetrical, computers have the
– Functional Specialisation same “rights”
– Asymmetrical – Truly Distributed
– Tends to be centralised – Share / exploit resources with a
large number of participants
– Resource discovery is a challenge
22
Consideration of Variations on a
Theme and Application for
Distributed Systems …
• Sometimes we need to consider:
– The use of multiple servers to increase performance
and resilience
– The use of mobile code
– Users’ need for low-cost computers
– Requirements to add and remove mobile devices
23
Multiple Servers in Distributed
Systems
Figure 2.4
A service provided by multiple servers
Service
Server
Client
Server
Client
Server
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4
COMP28112 Lecture 3
© Pearson Education 2005 24
Proxies and Caches Distributed
Systems
Figure 2.5
Web proxy server
Client Web
server
Proxy
server
Client Web
server
Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4
COMP28112 Lecture 3
© Pearson Education 2005 25
Design Requirements in Distributed
Systems
• Performance
– Responsiveness
– Throughput
– Load balancing
• Quality of Service
• Caching and Replication
• Dependability
– Correctness
– Security
– Fault-Tolerance 26
Fundamental Models in Distributed
Systems
• Models include all the essential ingredients that we
need to consider in order to understand and reason
about some aspects of a system’s behaviour.
• Aspects that we need to consider include:
– Process Interaction
– Failure
– Security
Managing concurrency and failure underlies many of the
problems we face!
27
Factors affecting process interaction
• Performance of communication channels
• Computer Clocks and Timing Events
• Two variants of the process interaction model:
– Synchronous systems:
• Process Execution Speeds: has known lower and upper bound
• Message transmitted delay: bounded
• Clock drift rates: drift rate from real time has a known bound
– Asynchronous systems:
• All the above may take an arbitrarily long time.
28
