Chapter 2: System Model
Introduction
Architecture Models
Fundamental Models
Summary
What is a model?
Each model is intended to provide an
abstract, simplified but consistent
description of a relevant aspect of
distributed system design
Architecture model
Architecture model
define the way in which they are mapped onto the
underlying network of computers
determine the distribution of data and
computational tasks amongst the physical nodes of
the system
evaluating the performance, reliability, scalability
and other properties of distributed systems.
Including
Client-server model
Peer process model
Variations of the client-server model
Fundamental model
Are concerned with a more formal description of the
properties that are common in all of the architectural
models
Including
The interaction model: deal with performance and
with the difficulty of setting time limits in a
distributed system
The failure model: give a precise specification of the
faults that can be exhibited by processes and
communication channels
The security model: discuss the possible threats to
processes and communication channels
4
Difficuties for and threats to DTS
Widely varying modes of use
Wide range of system environments
Internal problems
External threats
Chapter 3: System Model
Introduction
Architecture Models
Fundamental Models
Summary
Build architectural models
Simplifies and abstracts the functions of
the individual components
Achieved by classify processes as server,
client and peer processes
Then considers:
The placement of the components
The interrelationships between the
components
Software and hardware service layers in
distributed systems
Applications, services
Middleware
Operating system
Platform
Computer and network hardware
8
Platform
Are the lowest-level hardware and
software layers
e.g.
- Intel x86/Windows
- Intel x86/Linux
- Intel x86/Solaris
- SPARC/SunOS
- PowerPC/MacOS
9
Middleware
Its purpose is to mask heterogeneity and
provide a convenient programming model
e.g. OMGs CORBA, Java RMI, DCOM
Support of abstractions
- Remote method invocation: Sun RPC
- Group communication: Isis
- Notification of events:COBAR
- The replication of shared data
- Transmission of multimedia data
10
Limitation of middleware
some systems require support at the
application level.
E.g. transfer of large electronic mail
the end-to-end argument [1984]
some communication-related functions can be
completely and reliably implemented only with the
knowledge and help of the application standing at
the end points of the communication system
E.g. TCP, DNS and the Web
11
System architectures
The division of responsibilities between
system components (applications, server
and other processes) and the placement of
the components on computers in the
network
12
Arch. 1: Client/Server
Be historically the most important and remain
the most widely employed
Servers may in turn be clients of other servers
Client
invocation
invocation
result
Client
Server
Server
result
Key:
Process:
Computer:
13
Arch. 2: Services provided by multiple servers
Partition service objects on different servers
e.g. Web, CDAL
Maintain replicated service objects on several hosts
hosts
Service
e.g. Sun NIS, realcourse
Server
Client
Server
Client
Server
14
Arch. 3: Proxy servers and caches
Cache
a store of recently used data objects that is closer
than the objects themselves
E.g., web page cache at web browser or web proxy
proxy server
Web
server
Client
Proxy
server
Web
server
Client
15
Arch. 4: Peer to Peer
All processes play similar roles
Interacting cooperatively to perform a distributed
activity
Maintain consistency or synchronize at application level
Example: a peer to peer whiteboard
Application
Coordination
code
Application
Coordination
code
Application
Coordination
code
16
Napster
insert(X,
1.2.3.4)
...
Publish
I have X!
1.2.3.4
17
Napster
4.3.2.1
Fetch
Query
search(A)
-->
4.3.2.1
Reply
Where is file A?
18
Gnutella
I have file A.
I have file A.
Reply
Query
Where is file A?
19
KaZaA
insert(X,
123.2.21.23)
...
Publish
I have X!
123.2.21.23
20
10
KaZaA
search(A)
-->
123.2.22.50
123.2.22.50
Query
search(A)
-->
123.2.0.18
Replies
Where is file A?
123.2.0.18
21
BitTorrent
Tracker
C
22
11
BitTorrent
C
23
Consistent hashing
Node identifier hash = hash(IP address)
Key identifier hash = hash(key)
A key is stored at its successor:
node with next higher ID
In Chord hash function is
Secure Hash SHA-1
24
12
Chord
Key 5
Node 105
K5
K20
N105
Key=Hash(book1)=5
Key=Hash(book2)=20
Key=Hash(book3)=80
Circular ID space
N32
N90
K80
25
Chord
N120
N10
N105
N90 has K80
Where is key 80?
N32
K80 N90
N60
26
13
Key Location
log(N)
log(N)
Notation
Definition
finger[k].start
(n+2k-1)mod 2m, 1=<k=<m
.interval
[finger[k].start,finger[k+1].start)
.node
first node>=n.finger[k].start
successor
the next node on the identifier circle; finger[1].node
predecessor
the previous node on the identifier circle
27
k is the finger table index
Lookup(id)
Finger table for Node 1
Finger tables and key locations with
nodes 0,1,3 and keys 1,2 and 6 28
14
Lookup PseudoCode
To find the successor of an id :
Chord returns the successor of the
closest preceding finger to this id.
29
Finding successor of identifier 1
Lookup cost
The finger pointers at repeatedly doubling
distances around the circle cause each
iteration of the loop in find_predecessor to
halve the distance to the target identifier.
In an N node Network the number of
messages is of
O(Log N)
30
15
ID
Next_hop
file
id
next_hopid
fileid
id
id
next_hop
id
31
query(10)
n2
n1
4 n1 f4
12 n2 f12
5 n3
9 n3 f9
4
4
n4
14 n5 f14
13 n2 f13
3 n6
2
n3
3 n1 f3
14 n4 f14
5 n3
n5
5
4 n1 f4
10 n5 f10
8 n6
small world,
Six Degrees of Separation
32
16
Distributed Hash Table
...
CFS, OceanStore, PAST, ChordDNS
()
put(key, data)
data
get (key)
Hash
(DHash)
lookup(key)
node IP address
(Chord)
node
node
node
33
Variations on the client-server model
Reasons of variation
The use of mobile code and mobile agents
Users need for low-cost computers with
limited hardware resources
The requirement to add and remove mobile
devices in a convenient manner
34
17
Arch. 1.1: Mobile Code
For good interactive response, e.g. applet
a) client request results in the downloading of applet code
Client
Applet code
Web
server
b) client interacts with the applet
Client
Applet
Web
server
35
Arch. 1.2: Mobile Agent
A running program that travels from
one computer to another in a network
Carrying out a task on someones
behalf, e.g. worm[Xerox PARC]
36
18
Arch. 1.3: Network Computer
Download operating system and any
application software from a remote
file server
All the application data and code is
stored by a file server
Users may migrate
37
Arch. 1.4: Thin Client
A GUI on a computer that is local to the user
Execute application programs on a remote
computer
Drawback : high latencies
Implementation: X-11, VNC[AT&T 1998]
Compute server
Network computer or PC
Thin
Client
network
Application
Process
38
19
Arch. 1.5: Spontaneous network
Integrate mobile devices and other devices
into a given network
Key features
Easy connection to a local network
Easy integration with local services
Key design issues
Convenient connection and integration
Limited connectivity
mobile device move around continuously, disconnection
Security and privacy
Discovery Services
registration service, lookup service
39
Spontaneous networking in a hotel
Music
service
gateway
Alarm
service
Internet
Hotel wireless
network
Discovery
service
Camera
TV/PC
Laptop
PDA
Guests
devices
40
20
Interfaces and objects
Interface definitions
-- A set of functions available or invocation
In object-oriented languages
-- Many objects can be encapsulated in
processes
Distribution of responsibilities
-- a static client-server architecture or the
more dynamic object-oriented model
41
Design requirements for distributed
architectures
Resource sharing is taken for granted,
but effective data sharing on a large scale
remains a substantial challenge.
Performance issues
Quality of services
Use of caching and replication
Dependability issues
42
21
Performance issues
Responsiveness. E.g. the performance of webbrowsing clients
the load and performance of the server and
network
delay in the client and server operating systems
communication and middleware services as well
as code of the service
Throughput
the rate at which computational work is done
It is affected by processing speeds and by data
transfer rates
Balancing computational loads
43
E.g. applets, several computers for a service
Quality of service
Reliability, security, performance and
adaptability
The failure model, the security model
and the interaction model
QoS is commandeered to refer to the
ability to meet time-critical data(RSVP)
Qos applies to OSs as well as networks.
44
22
Use of caching and replication
Performance issues are bars to successful
deploy distributed systems
Replication and caching, with a variety
of different cache consistency protocols
E.g. Web-caching
45
Design requirements for distributed
architectures
Dependability issues
Correctness
Fault tolerance
redundancy, e.g. data and processes be
replicated, messages be retransmitted
Security
e.g. locate sensitive data in computers
that can be secured effectively against
attack
46
23
Chapter 3: System Model
Introduction
Architecture Models
Fundamental Models
Summary
47
A system model should address
What are the main entities in the system?
How do they interact?
What are the characteristics that affect
their individual and collective behavior?
48
24
Purpose of a fundamental model
Make explicit all the relevant
assumptions about the system we are
modeling
Make generalizations concerning what
is possible or impossible by logical
analysis and mathematical proof
49
Fundamental models intend to discuss
Interaction model
The processes interact by passing
messages, resulting in communication and
coordination.
delay are often of considerable duration
Its difficult to maintain the same time
notion in distributed system
Accuracy with which of the independent
processes can be coordinated is limited by
the two facts mentioned above
50
25
Fundamental models intend to discuss
Failure model
a fault occurs in computers or network
Failure model can provide the a basis for
the analysis of the potential effect
and for the design of system to be able to
tolerate faults of each type while
continuing to run correctly
51
Fundamental models intend to discuss
Security model
Modular nature of distributed and their
openness exposes them to attack by both
external and internal agent.
Security model provide the basis for
analysis of threats to system and for the
design of system that are able to resist
them
52
26
Interaction model
Examples of interaction in distributed
system
DNS, NIS
multiple server processes cooperate with
one another
P2P voice conference system
with strict real-time constraints
53
Implementation of a interaction model
Distributed algorithm
a definition of the steps to be taken by each
of the processes of which the system is
composed, including the transmission of
messages between them
The proceeding rate and transmission
timing can not be predicted.
difficult to describe all the states, because
of failures of processes and message
transmissions
54
27
Two factors affecting interacting processes
Communication performance is always
a limited characteristic
It is impossible to maintain a single
global notion of time
55
Performance of communication channels
Latency
the time taken for the first string of bits transmitted
through a network to reach its destination
accessing network
OS communication services
Bandwidth
total amount of information that can be transmitted over
computer network in a given time
Jitter
variation in the time taken to deliver a series of messages
E.g. consecutive samples of audio data are played with
56
differing time intervals
28
Computer clocks and timing events
Clock drift rate
the relative amount that a computer clock differs
from a perfect reference clock
Timing event
e.g., GPS, Logical time
57
Two variants of the interaction model
Synchronous distributed system
The time to execute each step of a
process has know lower and upper
bounds
Each message transmitted over a channel
is received within a known bounded time
Each process has a local clock whose
drift rate from real time has a known
bound
58
29
Two variants of the interaction model
Asynchronous distributed system no
bounds on
Process execution speed
e.g. each step may take an arbitrarily long time
Message transmission delay
e.g. a message may be received after an
arbitrarily long time
Clock drift rate
the drift rate of a clock is arbitrary
59
Examples of Syn. DS and Asyn. DS
Asynchronous DS
Email
FTP
Synchronous DS
VOD
Voice Conference System
60
30
Event ordering
send
X
m1
2
receive
receive
receive
send
3
m
2
4
receive
Physical
time
send
receive
A
t1
t2
receive
m
m
m
1
2
3
receive receive receive
t3
Example of disorder of messages
A group including X, Y, Z and A
X send Meeting to all; Y and Z reply Re: Meeting to all
At A, the messages received are Z.Re: Meeting,
61
X.Meeting, Y. Re: Meeting
Failure model
Define the ways in which failure may
occur in order to provide an
understanding of the effects of failures
Taxonomy[Hadzilacos and Toueg, 1994]
Omission failures
Arbitrary failures
Time failures
62
31
1. Omission failures
A process or communication channel fails to
perform actions that it is supposed to do
Process omission failure: Crash
Fail-stop: Crash that can be detected by
other processes certainly, e.g., by timeouts
in synchronous DS
Communication omission failures: dropping
messages
Send omission, receive omission, channel
omission
Benign failures
63
1. Omission failures
p
process
Send m
receive
Communication channel
Outgoing message buffer
Incoming message buffer
64
32
2. Arbitrary (Byzantine) failures
The worst possible failure semantics
Arbitrarily omit intended processing steps
steps or take unintended processing steps.
E.g. return a wrong value in response to an
an invocation
Arbitrary failures in process is hard to be
detected
Arbitrary failures in communication
channel exist but rare.
E.g. checksum, sequence number
65
Class of failure
Affects Description
Fail-stop
Process
Process halts and remains halted. Other
processes may detect this state.
Crash
Process
Process halts and remains halted. Other processes
may not be able to detect this state.
Omission Channel
A message inserted in an outgoing message buffer never
Sendomission
Process
arrives at the other ends incoming message buffer.
A process completes a send, but the message is not
put in its outgoing message buffer.
Receiveomission
Process
A message is put in a processs incoming message
buffer, but that process does not receive it.
Arbitrary Process or Process/channel exhibits arbitrary behaviour: it may
(Byzantine) channel send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.
66
33
3. Timing failures
Applicable in syn. distributed system,
but not in asyn. distributed system
Class of
Failure
Clock
Affects
Description
Process
Processs local clock exceeds the bounds
on its rate of drift from real time.
Performance Process
Process exceeds the bounds on the interval
between two steps.
Performance Channel
A messages transmission takes longer
than the stated bound.
67
Masking failures
Hide
e.g. replicated servers
Convert
e.g. Checksum: arbitrary failure -> omission failure
Reliability of one-to-one communication
Validity
any message in the outgoing message buffer is
eventually delivered to the incoming message
buffer
Integrity
the message received is identical to one sent, and
no messages are delivered twice, against
68
retransmit protocols and spurious messages
34
Security model
The security of a distributed system
The processes
The communication channels
The objects
Protecting the objects
Access rights: who is allowed to perform the
operations of an object
Principal: the authority who has some rights on the
object
69
Security model
Access rights
Object
invocation
Client
Principal (user)
result
Network
Server
Principal (server)
70
35
The enemies
Threats to processes
To servers: invocate with a false identity, e.g. cheating a mail
server
To clients: receive false result, e.g. stealing account password
Threats to communication channels
Copy, alter or inject messages
Save and replay, e.g. retransfer money from one account to
another
Copy of m
The enemy
Process p
Communication channel
Process q
71
The enemies (2)
Denial of service
excessive and pointless invocation on services
or message transmissions in a network
result in overloading of physical resources
(network bandwidth, server processing capacity)
Mobile code
malicious mobile program, e.g. Trojan horse
attachment
72
36
Defeat security threats
Cryptography and shared secret
Identify each other by the shared secrets that are
only known by themselves
Cryptography is the base
Authentication
proving the identities supplied by their senders
73
Secure channels
Each process knows reliably the identities
of the principal on whose behalf the other
process is executing
Ensure the privacy and integrity of the
data transmitted across it
Each message includes physical or logical
time stamp
Principal B
Principal A
Process p
Secure channel
Process q
74
37
Chapter 3: System Model
Introduction
Architecture Models
Fundamental Models
Summary
75
Architecture Models
Client / Server
e.g. Web, FTP, NEWS
Multiple Servers
e.g. DNS
Proxy and Cache
e.g. Web Cache
Peer processes
Variations of C/S
Mobile code, mobile agent, network computer,
thin client, spontaneous networks
76
38
Fundamental Models
Interaction models
synchronous DS and asynchronous DS
Failure models
omission failures
arbitrary failures
timing failures
Security model
the enemies
the approaches of defeating them
77
39
