18CS3040_Distributed Systems

18CS3040
DISTRIBUTED SYSTEMS
Module – II
Dr.A.Kathirvel, Professor, Computer Science and Engg.
Karunya Institute of Technology and Sciences, CBE

18CS3040 Distributed Systems
Module II
Distributed objects and file systems:
Introduction - Communication between distributed objects - Remote
procedure call - Events and notifications - case study - Operating
system support - introduction - operating system layer - protection -
process and threads - communication and invocation - architecture -
Introduction to DFS - File service architecture - Sun network file
system - Andrew file system - Enhancements and future developments.
Dr.A.Kathirvel, Professor,
DCSE, KITS
kathirvel@karunya.edu

Module II
Distributed objects and file systems:
Introduction - Communication between distributed objects - Remote
procedure call - Events and notifications - case study - Operating
system support - introduction - operating system layer - protection -
process and threads - communication and invocation - architecture -
Introduction to DFS - File service architecture - Sun network file
system - Andrew file system - Enhancements and future
developments.
Book Referred: Coulouris, Dollimore, Kindberg and Blair
Distributed Systems:Concepts and Design Edition 5, Addison-
Wesley 2012

Distributed Objects and Remote
Invocation
Including
1. Introduction
2. Communication between distributed objects.
3. Remote procedure call (RPC)
4. Events and Notifications
5. Java RMI case study.

Introduction
5
 Objects that can receive remote method invocations are called
remote objects and they implement a remote interface.
 Programming models for distributed applications are:
 Remote Procedure Call (RPC)
 Client calls a procedure implemented and
executing on a remote computer
 Call as if it was a local procedure
 Remote Method Invocation (RMI)
 Local object invokes methods of an object
residing on a remote computer
 Invocation as if it was a local method call
 Event-based Distributed Programming
 Objects receive asynchronous notifications of
events happening on remote
computers/processes

Introduction
6
 Middleware
 Software that provides a programming model above the
basic building blocks of processes and message passing is
called middleware.
 The middleware layer uses protocols based on messages
between processes to provide its higher-level abstractions
such as remote invocation and events.

Introduction
7
 Transparency Features of Middleware
 Location transparency:
 In RMI and RPCs, the client calls a procedure/method without
knowledge of the location of invoked method/procedure.
 Transport protocol transparency:
 E.g., request/reply protocol used to implement RPC can use either
UDP or TCP.
 Transparency of computer hardware
 They hide differences due to hardware architectures, such as byte
ordering.
 Transparency of operating system
 It provides independency of the underlying operating system.
 Transparency of programming language used
 E.g., by use of programming language independent Interface
Definition Languages (IDL), such as CORBA IDL.

CORBA IDL example
8
// In file Person.idl
struct Person {
string name;
string place;
long year;
} ;
interface PersonList {
readonly attribute string listname;
void addPerson(in Person p) ;
void getPerson(in string name, out Person p);
long number();
};

Introduction
9
 Interfaces for RMI and RPC
 An explicit interface is defined for each module.
 An Interface hides all implementation details.
 Accesses the variables in a module can only occur through
methods specified in interface.
 Interface in distributed system
 No direct access to remote variables is possible
 Using message passing mechanism to transmit data
objects and variables
 Request-reply protocols
 Local parameter passing mechanisms (by value,
 by reference) is not applicable to remote
 invocations
 Specify input, output as attribute to parameters
 Input: transmitted with request message
 Output: transmitted with reply message

10
 Pointers are not valid in remote address spaces
 Cannot be passed as argument along interface
 RPC and RMI interfaces are often seen as a client/server system
 Service interface (in client server model)
 Specification of procedures and methods offered by a server
 Remote interface (in RMI model)
 Specification of methods of an object that can be invoked by
objects in other processes
 Interface Definition Languages (IDL)
 Impossible to specify direct access to variables in remote classes
 Hence, access only through specified interface
 Desirable to have language-independent IDL that compiles into access
methods in application programming language
 Example: CORBA IDL
notation for language independent interfaces
specify type and
kind of parameters
Introduction

Interface Definition Language
examples
CORBA IDL for RMI
Sun XDR for RPC
DCOM IDL
IDL compiler allows interoperability CORBA IDL.

specifies the methods of an object available for remote
invocation
an interface definition language (or IDL) is used to specify
remote interfaces.
Java RMI would have a class for Person, but CORBA has a
struct
Remote Interface

2. Communication between
distributed objects.
including
1. The object model
2. Distributed objects
3. The distributed object model
4. Design issues
5. Implementation
6. Distributed garbage collection

1. The object model
An object-oriented program consists of a collection of
interacting objects.
An object encapsulates both data and methods.
Objects can be accessed via object references.
An interface provides a definition of the signatures of a
set of methods.
Actions are performed by method invocations
The state of receiver may be changed.
Further invocations of methods on other objects may
take place.

Parts of Distributed Object Model
1. Remote Object References
- accessing the remote object
2. Remote Interfaces.
- specifying which methods can be invoked remotely
3. Actions - Objects invoke methods in other objects.
4. Exceptions
– provide a clean way to deal with unexpected events or
errors. Eg: throw, catch
5. Garbage Collection - provide a means of freeing
the space that is no longer needed.

2. Distributed objects
Physical distribution of objects into different processes
or computers in a distributed system.
Object state consists of the values of its instance
variables
Object methods invoked by remote method invocation
(RMI)
Object encapsulation: object state accessed only by the
object methods

3. The distributed object model
Two fundamental concepts:
1. Remote Object Reference
 An object must have the remote object reference of an
object in order to do remote invocation of an object
 Remote object references may be passed as input
arguments or returned as output arguments
2. Remote Interface
 Objects in other processes can invoke only the
methods that belong to its remote interface
 CORBA – uses IDL to specify remote interface
 JAVA – extends interface by the Remote keyword.

interface
remote
m1
m2
m3
m4
m5
m6
Data
implementation
remote object
{ of methods
A remote object and its remote
interface
Each process contains objects, some of which can receive
remote invocations are called remote objects
(B, F), others only local invocations

Remote and local methods invocation
 Objects need to know the remote object reference of an object in
another process in order to invoke its methods, called remote method
invocations
 Every remote object has a remote interface that specifies which of its
methods can be invoked remotely.

4. Design Issues for RMI
Two important issues in making RMI natural
extension of local method: (These problems won‟t
occur in the local invocation.)
Number of times of invocations are invoked in
response to a single remote invocation
Level of location transparency
Exactly once invocation semantics - Every
method is executed exactly once. (Ideal situation)

Choices of invocation semantics
1.Maybe: the method executed once or not at all (no
retry nor retransmit)
Types of failures:
Omission failures
Crash failures
2. At-least-once: the method executed at least once or
an exception
Types of failures : omission failures, Crash failures Arbitrary
failure
3. At-most-once: the method executed exactly once

Error handling for delivery
guarantees
 Retry request message - retransmit the request
message until reply is received.
 Duplicate filtering - when retransmissions are used,
whether to filter out duplicate requests at the server
 Retransmission of results - To keep a history of result
messages to enable lost results to be retransmitted without
re-executing the operations

 request-reply protocol
 single request message and reply message.
Maybe executed
 re-transmit requests until we get a reply
at least once
 save replies for re-transmission
at-most-once
Choices of invocation semantics
Fault tolerance measures Invocation
semantics
Retransmit request
message
Duplicate
filtering
Re-execute procedure
or retransmit reply
No
Yes
Yes
Not applicable
No
Yes
Not applicable
Retransmit reply At-most-once
At-least-once
Maybe

The level of transparency
Make remote invocation as much like local invocation as
possible.
RPC tries to make the remote procedure call look exactly
like a local one.
RPC marshalling and message passing procedures are
hidden.
Issues (Differences from the local invocation)
Syntax may be made identical but behavioral differences
exists. The cause could be failure and latency.
Interface specification
Exceptions and exception handling are needed.
Different invocation semantics can be employed.

Remote Procedure Call (RPC)
28
 A remote procedure call (RPC) is similar to a remote method
invocation (RMI).
 A client program calls a procedure in another program running in a
server process.
 RPC, like RMI, may be implemented to have one of the choices of
invocation semantics - at-least-once, at-most-once are generally
chosen.
 RPC is
generally
implemented
over a
request-reply
protocol.
 The software
that support
RPC is shown
in Fig.

Remote Procedure Call (RPC)
29
 RPC only addresses procedure calls.
 RPC is not concerned with objects and object references.
 A client that accesses a server includes one stub
procedure for each procedure in the service interface.
 1.Like RMI , the client program calls a procedure in
another program running in a server process.
 2.it lacks the ability to create a new instance of objects
and therefore does not support remote object references.
 3.at-least-once and at-most-once invocation semantics
are the most common for RPC.
 4.Generally, RPC is implemented over a request-reply
protocol.

Building blocks
1. Communication module
2. Client stub procedure (as proxy in RMI): marshalling, sending,
unmarshalling
3. Dispatcher: select one of the server stub procedures
4. Server stub procedure (as skeleton in RMI): unmarshalling, calling,
marshalling
 The client that accesses a service includes one stub procedure.
 The stub procedure is similar to RMI proxy.
 Servers implement stub procedures and skeletons.
Implementing RPC

Strength and Weaknesses of RPC
31
RPC is not well suited for adhoc query processing.
Consider the use of RPC to make SQL requests on
servers that return arbitrary size of rows and number of
tuples.
It is not suited for transaction processing without special
modification.
A separate special mode of quering is proposed – Remote
Data Access (RDA).
RDA is specially suited for DBMS.
In a general client_server environment both RPC and
RDA are needed.

Events and Notifications
Idea behind the use of events is that One object
can react to a change occurring in another
object.
For example, when a user interacts with an
application by mouse clicks, these interactions
are seen as events that cause changes in the
objects that compose the whole application,
and may change in the whole state of the
application.

Publish/subscribe paradigm
 It allows multiple objects at different locations to
be notified of events taking place at an object.
 event generator publish the type of events
 event receiver subscribe to the types of events that
are interest to them
 When event occur, notify the receiver
Objects that represent events are called notifications.
Subscribing to a particular type of event is called
registering interest.

Dealer’s computer
Information
provider
Dealer
External
source
External
source
Information
provider
Dealer
Dealer
Dealer
Notification
Notification
Notification
Notification
Notification
Notification
Notification
Notification
Dealer’s computer
Dealer’s computer
Dealer’s computer
Notification
Notification

A dealing room system could be modeled by processes
with two different tasks
An information provider process continuously receives
new trading information from a single external source and
applies to the appropriate stock objects.
A dealer process creates an object to represent each
named stock that the user asks to have displayed.
An event source can generate events of one more different
types. Each event has attributes that specify information
about that event.
Distributed event-based system characteristics
1.Heterogeneous 2. Asynchronous

1.Heterogeneous
 When using event notifications to communicate between objects
in different processes, these objects can be communicate although
they are not designed to interoperate.
 The event-generating objects publish the types of events they
offer, and the other objects subscribe to events and provide an
interface for receiving notifications.
2. Asynchronous
 Notifications are sent asynchronously (without time arrangements)
 There is no need for the server object to still wait for the client to
send it the message, instead the server object can serve other
clients, and when it receives a message, it handles it.

Participants in distributed event
notification
The main component is an event service
event service maintains a database of published events and
of subscriber‟s interests.
1. The object of interest
2. Event
3. Notification
4. Subscriber
5. Observer objects
6. Publisher

Architecture of distributed event
notifications

1. An object of interest inside the event service sends notification
directly to the subscribers.
2. An object of interest inside the event service sends notification via
the observer to the subscribers.
3. The observer queries the object of interest outside the event service
and sends notifications to the subscribers.
 Roles for observers – the task of processing notifications can be divided
among observers:
 Forwarding – Observers simply forward notifications to subscribers.
 Filtering of notifications – Observers address notifications to those
subscribers who find these notifications are useful.
 Patterns of events – Subscribers can specify patterns of events of
interest.
 Notification mailboxes – A subscriber can set up a notification
mailbox which receives the notification on behalf of the subscriber.
notifications

delivery semantics:
 IP multicast protocol – information delivery on the
latest state of a player in an Internet game
 Reliable multicast protocol – information provider /
dealer
 Totally ordered multicast - Computer Supported
Cooperative Working (CSCW) environment
 Real-time – nuclear power station / hospital patient
monitor
notifications

RPC Example 1: Local Program
42
* A first program (hello.c) to test rpc. Use of this program is for testing the logic of
the rpc programs.
#include <stdio.h>
Int main (void) {
static char * result;
static char msg[256];
printf("getting ready to return valuen");
strcpy(msg, "Hello world");
result= msg;
printf("Returning %sn", result);
return (0);
} /

Protocol Definition Program
43
 The name of this program is hello.x
 The number at the end is version number and should be
updated each time the service is updated to make sure the
active old copies is not responding to the client program.
program HELLO {
version ONE{
string PRINT_HELLO() = 1;
} = 1 ;
} = 0x2000059;

Client Program
44
 Now we are ready to use rpcgen (command for
generating the required programs).
 Note that so far we have only hello.c and hello.x
 After running “rpcgen -a -C hello.x” the directory
contain following files:
-rw-rw-r-- 1 aabhari aabhari 131 Oct 5 12:15 hello.c
-rw-rw-r-- 1 aabhari aabhari 688 Oct 5 12:19 hello.h
-rw-rw-r-- 1 aabhari aabhari 90 Oct 5 12:18 hello.x
-rw-rw-r-- 1 aabhari aabhari 776 Oct 5 12:19 hello_client.c
-rw-rw-r-- 1 aabhari aabhari 548 Oct 5 12:19 hello_clnt.c
-rw-rw-r-- 1 aabhari aabhari 316 Oct 5 12:19 hello_server.c
-rw-rw-r-- 1 aabhari aabhari 2076 Oct 5 12:19 hello_svc.c
 The two templates that we should modify for this
example are hello_client.c and hello_server.c.

Template of hello_client Program
45
* This is sample code generated by rpcgen.
* These are only templates and you can use them as a guideline for
developing your own functions.
#include "hello.h"
void hello_1(char *host)
{
CLIENT *clnt;
char * *result_1;
char *print_hello_1_arg;
#ifndef DEBUG
clnt = clnt_create (host, HELLO, ONE, "udp");
if (clnt == NULL) {
clnt_pcreateerror (host);
exit (1); } #endif /* DEBUG */
result_1 = print_hello_1((void*)&print_hello_1_arg, clnt);
if (result_1 == (char **) NULL) {
clnt_perror (clnt, "call failed"); } #ifndef DEBUG
clnt_destroy (clnt); #endif /* DEBUG */
}

Template of hello_client Program
46
Int main (int argc, char *argv[])
{
char *host;
if (argc < 2) {
printf ("usage: %s server_hostn", argv[0]);
exit (1);
}
host = argv[1];
hello_1 (host);
exit (0);
}
 We have to modified hello_client template program.
 The modifications for our first example are very simple.
 Next slides show the modified program of hello_client
that needs only few lines.

Template of hello-server Program
47
#include "hello.h"
char **
print_hello_1_svc(void *argp, struct svc_req *rqstp)
{
/* insert server code here */
return &result;
}
#include "hello.h"
char **
print_hello_1_svc(void *argp, struct svc_req *rqstp)
{
static char msg[256];
printf("getting ready to return valuen");
strcpy(msg, "Hello world");
result= msg;
printf("Returningn");
return &result;
}

Making Client and Server Program
48
 To compile the client
leda% gcc hello_client.c hello_clnt.c -o client -lnsl
 To compile the server
leda% gcc hello_server.c hello_svc.c -o server -lnsl
 To run the server use
leda% ./server
 To run the client use
elara% ./client leda

RPC
49
 rpcgen facilitates the generation of client and server
stubs from the IDL program.
 It even generates client and server template
programs.
 The option -a is passed to rpcgen and also all the
generation of all support files including the make
files.
 The -a option causes the rpcgen to halt with
warnings if template files with default names exist.
 Consider turning a factorial program into a client-
server program using RPC.

RPC Example 2
50
/* A program to calculate factorial numbers. */
#include <stdio.h>
void main(void){
long int f_numb, calc_fact(int);
int number;
printf("Factorial Calculatorn") ;
printf("Enter a positive integer value");
scanf("%d", &number);
if (number < 0)
printf("Positive value only!n");
else if ((f_numb = calc_fact(number)) > 0)
printf("%d! = %dn", number, f_numb);
else
printf("Sorry %d! is out of my range!n", number);
}

RPC Example
51
/* Calculate the factorial number and return the result or return 0 if
the value is out of range. */
long int calc_fact(int n) {
long int total = 1, last = 0;
int idx;
for (idx = n; id<= 1; --idx) {
total *=idx;
if (total < last) /* Have we gone out of range? */
return (0);
last = total;
}
return (total);
}

RPC Example
52
/* The protocol definition file for the factorial program.
the file name is fact.x */
program FACTORIAL {
version ONE{
long int CALC_FAC(int) = 1;
} = 1 ;
} = 0x2000049;
 We may now use rpcgen with the option flags -a -C to generate
header file, the client and the server stub files and in addition,
the client and server template programs.

RPC Example
53
• The content of the fact_client.c program is as shown below.
#include “fact.h”
void factorial_1(char *host)
{ CLIENT *clnt;
long *result_1
int calc_fac_1_arg;
#ifndef DEBUG
clnt = clnt_create(host, FACTORIAL, ONE, “netpath”);
if (clnt == (CLIENT *) NULL) {
clnt_pcreateerror(host);
exit(1);
} #endif /* DEBUG */
result_1 = calc_fac_1(&calc_fac_1_arg, clnt);
if (result_1 == (long *) NULL) {
clnt_perror(clnt, “call failed”);
}

RPC Example
54
#ifndef DEBUG
clnt_destroy(clnt);
#endif /* DEBUG */
}
main(int argc, char *argv[])
{
char *host;
if (arg < 2) {
printf(“usage: %s server_hostn”, argv[0]);
exit(1);
}
host = argv[1];
factorial_1(host);
}

RPC Example
55
The template code for client needs to be modified to conform our original program.
#include <unistd.h> /* added because we will call exit*/
long int factorial_1(int calc_fac_1_arg, char *host)
{ CLIENT *clnt;
long *result_1;
#ifndef DEBUG
clnt = clnt_create(host, FACTORIAL, ONE, “netpath”);
if (clnt == (CLIENT *) NULL) {
clnt_pcreateerror(host);
exit(1);
} #endif /* DEBUG */
result_1 = calc_fac_1(&calc_fac_1_arg, clnt);
if (result_1 == (long *) NULL) {
clnt_perror(clnt, “call failed”);
} #ifndef DEBUG
clnt_destroy(clnt);
#endif /* DEBUG */

RPC Example
56
main(int argc, char *argv[])
{
char *host;
/* Add own declarations here */
long int f_numb;
int number;
if (arg < 2) {
printf(“usage: %s server_hostn”, argv[0]);
exit(1);
}
host = argv[1];

RPC Example
57
/* This is the code from the previous main in program fact.c */
printf(“Factorial Calculationn”);
printf(“Enter a positive integer value”);
scanf(“%d”, &number);
if (number < 0)
printf(“Positive values onlyn”);
else if ((f_numb = factorial_1(number, host)) >0 )
printf(“%d! = %dn”, number, f_numb);
else
printf(“Sorry %d! is out of my range!n”, number);
}

RPC Example
58
• Here is the fact_server.c template generated by rpcgen
/* This is sample code generated by rpcgen.
These are only templates and you can use them
as aguideline for developing your own functions. */
long int * calc_fac_1_srv(int *argp, struct svc_req *rqstp)
{
static long result;
/* insert server code here */
return(&result);
}

RPC Example
59
• Here is the fact_server.c template with modification code
long int * calc_fac_1_srv(int *argp, struct svc_req *rqstp)
{ static long int result; /* insert server code here */
long int total = 1, last = 0;
int idx;
for(idx = *argp; idx – 1; --idx)
total *= idx;
if (total <= last) /* Have we gone out of range? */
{
result = 0;
return (&result);
}
last = total;
}
result = total;
return (&result);
}

RPC Example
60
• Here is a modified makefile.fact
• # This is a template make file generated by
rpcgen
#parameters
#added CC = gcc to use gcc
CC = gcc
CLIENT = fact_client
SERVER = fact_server
SOURCES_CLNT.c =
SOURCES_CLNT.h =
SOURCES_SVC.c =
SOURCES_SVC.h =
SOURCES.x = fact.x
TARGETS_SVC.c = fact_svc.c fact_server.c
TARGETS_CLNT.c = fact_clnt.c fact_client.c
TARGETS = fact.h fact_clnt.c fact_svc.c fact_client.c fact_server.c

RPC Example
61
OBJECT_CLNT = $(SOURCES_CLNT.c:%.c=%.o)
$(TARGETS_CLNT.c:%.c=%.o)
OBJECT_SVC = $(SOURCES_SVC.c:%.c=%.o) $(TARGETS_SVC.c:%.c=%.o)
# Compiler flags
CFLAGS += -g
LDLIBS += -lnsl
# added –C flag to PRCGENFLAGS or add –lm to LDLIBS
RPCGENFLAGS = -C
#Targets
all : $(CLIENT) $(SERVER)
$(TARGETS) : $(SOURCES.x)
rpcgen $(RPCGENFLAGS) $(SOURCES.x)

RPC Example
62
$(OBJECTS_CLNT) : $(SOURCES_CLNT.c) $ (SOURCES_CLNT.h)

$(TARGETS_CLNT.c)
$(OBJECTS_SVC) : $(SOURCES_SVC.c) $ (SOURCES_SVC.h)
$(TARGETS_SVC.c)
$(CLIENT) : $(OBJECTS_CLNT)
$ (LINK.c) –o $(CLIENT) $(OBJECTS_CLNT) $(LDLIBS)
$(SERVER) : $(OBJECTS_SVC)
$ (LINK.c) –o $(SERVER) $(OBJECTS_SVC) $(LDLIBS)
Clean: $(RM) core $(TARGETS) $(OBJECTS_CLNT)
$(OBJECTS_SVC)
$(CLIENT) $(SERVER)

Java RMI
64
 Java RMI extends the Java object model to provide support for
distributed objects in the Java language.
 It allows object to invoke methods on remote objects using the
same syntax as for local invocation.
 It is a single language system – remote interfaces are defined in the
Java language.
 An object making a remote invocation needs to be able to handle
RemoteExceptions that are thrown in the event of communication
subsystem failures
 A remote object must implement Remote interface.
 The next example is a simple Hello world program that is
implemented in Java RMI

Java RMI- Example 1
65
This example is from java.sun.com for more
 explanation see the site:
http://java.sun.com/j2se/1.5.0/docs/guide/rmi/hello/hello-world.html#1
In this example we do the followings:
Define the remote interface
Implement the server
Implement the client
Compile the source files
Start the Java RMI registry, server, and client

Java RMI
66
Hello.java - a remote interface
import java.rmi.Remote;
import java.rmi.RemoteException;
public interface Hello extends Remote {
String sayHello() throws RemoteException;
}
// Implement the server
import java.rmi.registry.LocateRegistry;
import java.rmi.RemoteException;
import java.rmi.server.UnicastRemoteObject;
public class Server implements Hello {
public Server() {}
public String sayHello() { return "Hello, world!"; }

Java RMI
67
public static void main(String args[]) {
try {
Server obj = new Server();
Hello stub = (Hello) UnicastRemoteObject.exportObject(obj, 0);
// Bind the remote object's stub in the registry
Registry registry = LocateRegistry.getRegistry();
registry.bind("Hello", stub);
System.err.println("Server ready");
} catch (Exception e) {
System.err.println("Server exception: " + e.toString());
e.printStackTrace();
}
}}

Java RMI
68
// Implement the client
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
public class Client {
private Client() {}

Java RMI
69
public static void main(String args[]) {
String host = (args.length < 1) ? null : args[0];
try {
Registry registry = LocateRegistry.getRegistry(host);
Hello stub = (Hello) registry.lookup("Hello");
String response = stub.sayHello();
System.out.println("response: " + response);
} catch (Exception e) {
System.err.println("Client exception: " + e.toString());
e.printStackTrace();
}
}
}

Java RMI
70
 The source files for this example can be compiled as
follows:
 javac Hello.java Server.java Client.java
 Start the Java RMI registry :
 rmiregistry
 Start the Server: java Server
 Start the Client: java Client

Java RMI Example 2
71
Shared whiteboard
It allows a group of users to share a common view
of
 a drawing surface containing graphical objects.
The server maintains the current state of a
 drawingClients can poll the server about the latest
 shape of a drawing.
The server attaches version numbers to new arriving
 shapes.
 Remote Interface
 Remote interfaces are defined by extending an interface
called Remote provided in the java.rmi package.
 Figure 4 shows an example of two remote interface called
Shape and ShapeList.

Java RMI
72
import java.rmi.*;
import java.util.Vector;
public interface Shape extends Remote {
int getVersion() throws RemoteException;
GraphicalObject getAllState() throws RemoteException; 1
}
public interface ShapeList extends Remote {
Shape newShape(GraphicalObject g) throws RemoteException; 2
Vector allShapes() throws RemoteException;
}
 In this example, GraphicalObject is a class that holds the
state of a graphical object.
 GraphicalObject must implement the Serializable interface.
 Ordinary and remote objects can appear as input and output
arguments.

Java RMI
73
 Parameter and result passing
 Passing remote objects
 The result is passed by (object) reference.
 In line2, the return value of the method newShape is defined as
shape - a remote interface.
 When a remote object reference is received, an RMI can be
issued on the object refered to by this reference.
 Passing non-remote objects
 The result is passed by value.
 A new object is created locally, with the state differing from the
original object.
 Downloading of Classes
 Classes can be transferred from one Java VM to another.
 Parameters are passed by value or by reference.
 If the recipient does not yet possess the class of an object passed by
value, its code is downloaded automatically.
 If the recipient of a remote object reference does not yet possess the
class for a proxy, its code is downloaded automatically.

Java RMI
74
 RMIregistry
 The RMIregistry is the binder for Java RMI.
 The RMIregistry runs on every server that hosts remote
objects.
 It maps local object names of the form
//computerName:port/objName to object references.
 This service is not a global service.
 Clients need to query a particular host to get reference.
void rebind (String name, Remote obj)
This method is used by a server to register the identifier of a remote object by name,
void bind (String name, Remote obj)
This method can alternatively be used by a server to register a remote object by name, but
if the name is already bound to a remote object reference an exception is thrown.
void unbind (String name, Remote obj) - This method removes a binding.
Remote lookup (String name) - Method is used by clients to look up a remote object by name
String [] list() - Method returns an array of Strings containing the names bound in the registry.

Java RMI
75
import java.rmi.*;
public class ShapeListServer{
public static void main (String args[]){
System.setSecurityManager (new RMISecurityManager());
try{
ShapeList aShapeList = new ShapeListServant(); 1
Naming.rebind(“Shape List”, aShapeList); 2
System.out.println(“ShapeList server ready”);
}catch(Exception e) {
System.out.println(“ShapeList server main” + e.getMessage());}
}}
 Security Manager
 implements various security policies for client accesses
 Main method
 1: create instance of ShapeListServant
 2: binds name "ShapeList" to newly created instance in
RMIRegistry

Java RMI
76
 ShapeListServant implements ShapeList
 gives an outline of the class ShapeListServant.
 1: UnicastRemoteObject - objects that live only as long as
creating process
 2: factory method - client can request creation of a new object
import java.rmi.*;
public class ShapeListServant extends UnicastRemoteObject implements ShapeList{
private Vector theList; // contains the list of Shapes 1
private int version;
public ShapeListServant()throws RemoteException{….}
public Shape newShape(GraphicalObject g) throws RemoteException{ 2
version++
Shape s = new ShapeServant(g, version); 3
theList.addElement(s); return s;
}
public Vector allShapes() throws RemoteException{…}
public int getVersion() throws RemoteException{…}
}

Java RMI
77
 Client program
 A simplified client for the ShapeList sever is illustrated.
polling loop:
 1: look up remote reference
 2: invoke allShapes() in remote object
import java.rmi.*;
import java.rmi.server.*;
public class ShapeListClient{
public static void main(String args[]){
System.setSecurityManager(new RMISecurityManager());
ShapeList aShapeList = null;
try{
aShapeList = (ShapeList) Naming.lookup(“//bruno.ShapeList”); 1
Vector sList = aShapeList.allShapes(); 2
} catch(RemoteException e) {System.out.println(e.getMessage());
} catch(Exception e) {System.out.println(“Client:”+e.getMessage());}
}}

Fig 5.1: Middleware layers
Applications
Middleware
layers
Underlying interprocess communication primitives:
Sockets, message passing, multicast support, overlay networks
UDP and TCP
Remote invocation, indirect communication
This chapter
(and Chapter 6)
Remote invocation

Operations of the request-reply
protocol
public byte[] doOperation (RemoteRef s, int operationId, byte[] arguments)
sends a request message to the remote server and returns the reply.
The arguments specify the remote server, the operation to be invoked and
the
arguments of that operation.
public byte[] getRequest ();
acquires a client request via the server port.
public void sendReply (byte[] reply, InetAddress clientHost, int clientPort);
sends the reply message reply to the client at its Internet address and port.

Request-reply message structure
messageType
requestId
remoteReference
operationId
arguments
int (0=Request, 1= Reply)
int
RemoteRef
int or Operation
array of bytes

RPC exchange protocols
R Request
RR Reply
RRA Acknowledge reply
Request
Request Reply
Client Server Client
Name Messages sent by
HTTP Request/ Reply message
GET //www.dcs.qmw.ac.uk/index.html HTTP/ 1.1
URL or pathname
method HTTP version headers message body
HTTP/1.1 200 OK resource data
HTTP version status code reason headers message body

CORBA IDL example
// In file Person.idl
struct Person {
string name;
string place;
long year;
} ;
Call semantics
interface PersonList {
readonly attribute string listname;
void addPerson(in Person p) ;
void getPerson(in string name, out Person p);
long number();
};
Fault tolerance measures Call
semantics
Retransmit request
message
Duplicate
filtering
or retransmit reply
No
Yes
Yes
Not applicable
No
Yes
Not applicable
Retransmit reply At-most-once
At-least-once
Maybe

Role of client and server stub
procedures in RPC
client
Request
Reply
Communication
Communication
module
module dispatcher
service
client stub server stub
procedure procedure
client process server process
procedure
program

Files interface in Sun XDR
const MAX = 1000;
typedef int FileIdentifier;
typedef int FilePointer;
typedef int Length;
struct Data {
int length;
char buffer[MAX];
};
struct writeargs {
FileIdentifier f;
FilePointer position;
Data data;
};
struct readargs {
FileIdentifier f;
FilePointer position;
Length length;
};
program FILEREADWRITE {
version VERSION {
void WRITE(writeargs)=1; 1
Data READ(readargs)=2; 2
}=2;
} = 9999;

Instantiation of remote objects
The role of proxy and skeleton in
remote method invocation

Java Remote interfaces Shape and
ShapeList
import java.rmi.*;
public interface Shape extends Remote {
GraphicalObject getAllState() throws RemoteException; 1
}
public interface ShapeList extends Remote {
Shape newShape(GraphicalObject g) throws
RemoteException; 2
Vector allShapes() throws RemoteException;
}

Naming class of Java RMIregistry
void rebind (String name, Remote obj)
This method is used by a server to register the identifier of a remote object by
name, as shown in Figure 15.18, line 3.
void bind (String name, Remote obj)
This method can alternatively be used by a server to register a remote object by
name, but if the name is already bound to a remote object reference an
exception is thrown.
void unbind (String name, Remote obj)
This method removes a binding.
Remote lookup(String name)
This method is used by clients to look up a remote object by name, as shown in
Figure 5.20 line 1. A remote object reference is returned.
String [] list()
This method returns an array of Strings containing the names bound in the registry.

Java class ShapeListServer with main
method
import java.rmi.*;
public class ShapeListServer{
try{
ShapeList aShapeList = new ShapeListServant(); 1
Naming.rebind("Shape List", aShapeList ); 2
System.out.println("ShapeList server ready");
}catch(Exception e) {
System.out.println("ShapeList server main " + e.getMessage());
}
}
}

Java class ShapeListServant
implements interface ShapeList
import java.rmi.*;
public class ShapeListServant extends UnicastRemoteObject implements ShapeList {
private Vector theList; // contains the list of Shapes
private int version;
public ShapeListServant()throws RemoteException{...}
public Shape newShape(GraphicalObject g) throws RemoteException { 1
version++;
Shape s = new ShapeServant( g, version); 2
theList.addElement(s);
return s;
}
public Vector allShapes()throws RemoteException{...}
public int getVersion() throws RemoteException { ... }
}

Java client of ShapeList
import java.rmi.*;
import java.rmi.server.*;
public class ShapeListClient{
ShapeList aShapeList = null;
try{
aShapeList = (ShapeList) Naming.lookup("//bruno.ShapeList"); 1
Vector sList = aShapeList.allShapes(); 2
} catch(RemoteException e) {System.out.println(e.getMessage());
}catch(Exception e) {System.out.println("Client: " +
e.getMessage());}
}
}

Classes supporting Java RMI
RemoteServer
UnicastRemoteObject
<servant class>
Activatable
RemoteObject

From Coulouris, Dollimore, Kindberg and
Blair
Distributed Systems:
Concepts and Design
Edition 5, © Addison-Wesley 2012
Slides for Chapter 6:
Indirect Communication

93
Space and time coupling in
distributed systems

Open and closed groups
Closed group Open group

The role of group membership
management
Join
Group
address
expansion
Multicast
communication
Group
send
Fail
Group membership
management
Leave
Process group

96
The architecture of JGroups

97
Java class FireAlarmJG
import org.jgroups.JChannel;
public class FireAlarmJG {
public void raise() {
try {
JChannel channel = new JChannel();
channel.connect("AlarmChannel");
Message msg = new Message(null, null, "Fire!");
channel.send(msg);
}
catch(Exception e) {
}
}

98
Java class FireAlarmConsumerJG
import org.jgroups.JChannel;
public class FireAlarmConsumerJG {
public String await() {
try {
JChannel channel = new JChannel();
channel.connect("AlarmChannel");
Message msg = (Message) channel.receive(0);
return (String) msg.GetObject();
} catch(Exception e) {
return null;
}
}}

Dealing room system
Dealer’s computer
Information
provider
Dealer
External
source
External
source
Information
provider
Dealer
Dealer
Dealer
Notification
Notification
Notification
Notification
Notification
Notification
Notification
Notification
Dealer’s computer
Dealer’s computer
Dealer’s computer
Notification
Notification

100
The publish-subscribe paradigm

102
The architecture of publish-
subscribe systems

103
Filtering-based routing
upon receive publish(event e) from node x 1
matchlist := match(e, subscriptions) 2
send notify(e) to matchlist; 3
fwdlist := match(e, routing); 4
send publish(e) to fwdlist - x;5
upon receive subscribe(subscription s) from node x 6
if x is client then 7
add x to subscriptions; 8
else add(x, s) to routing; 9
send subscribe(s) to neighbours - x;10

104
Rendezvous-based routing
upon receive publish(event e) from node x at node i
rvlist := EN(e);
if i in rvlist then begin
matchlist :=match(e, subscriptions);
send notify(e) to matchlist;
end
send publish(e) to rvlist - i;
upon receive subscribe(subscription s) from node x at node i
rvlist := SN(s);
if i in rvlist then
add s to subscriptions;
else
send subscribe(s) to rvlist - i;

105
Example publish-subscribe system

106
The message queue paradigm

107
A simple networked topology in
WebSphere MQ

108
Programming model offered by JMS

109
Java class FireAlarmJMS
import javax.jms.*;
import javax.naming.*;
public class FireAlarmJMS {
try { 1
Context ctx = new InitialContext(); 2
TopicConnectionFactory topicFactory = 3
(TopicConnectionFactory)ctx.lookup ("TopicConnectionFactory"); 4
Topic topic = (Topic)ctx.lookup("Alarms"); 5
TopicConnection topicConn = 6
topicConnectionFactory.createTopicConnection(); 7
TopicSession topicSess = topicConn.createTopicSession(false, 8
Session.AUTO_ACKNOWLEDGE); 9
TopicPublisher topicPub = topicSess.createPublisher(topic); 10;
TextMessage msg = topicSess.createTextMessage(); 11
msg.setText("Fire!"); 12
topicPub.publish(message);13
} catch (Exception e) { 14
} 15
}

110
Java class FireAlarmConsumerJMS
import javax.jms.*; import javax.naming.*;
public class FireAlarmConsumerJMS
try { 1
Context ctx = new InitialContext(); 2
TopicConnectionFactory topicFactory = 3
(TopicConnectionFactory)ctx.lookup("TopicConnectionFactory"); 4
Topic topic = (Topic)ctx.lookup("Alarms"); 5
TopicConnection topicConn = 6
topicConnectionFactory.createTopicConnection(); 7
TopicSession topicSess = topicConn.createTopicSession(false, 8
Session.AUTO_ACKNOWLEDGE); 9
TopicSubscriber topicSub = topicSess.createSubscriber(topic); 10
topicSub.start(); 11
TextMessage msg = (TextMessage) topicSub.receive(); 12
return msg.getText(); 13
} catch (Exception e) { 14
return null; 15
}16
}

The distributed shared memory
abstraction

112
The tuple space abstraction

113
Replication and the tuple space
operations
write
1. The requesting site multicasts the write request to all members of the view;
2. On receiving this request, members insert the tuple into their replica and
acknowledge this action;
3. Step 1 is repeated until all acknowledgements are received.
read
1. The requesting site multicasts the read request to all members of the view;
2. On receiving this request, a member returns a matching tuple to the requestor;
3. The requestor returns the first matching tuple received as the result of the operation
(ignoring others);
4. Step 1 is repeated until at least one response is received.
continued on next slide
take Phase 1: Selecting the tuple to be removed
1. The requesting site multicasts the take request to all members of the view;
2. On receiving this request, each replica acquires a lock on the associated tuple set and, if
the lock cannot be acquired, the take request is rejected;

114
Replication and the tuple space
operations
Phase 2: Removing the selected tuple
3. All accepting members reply with the set of all matching tuples;
4. Step 1 is repeated until all sites have accepted the request and responded with their set
of tuples and the intersection is non-null;
5. A particular tuple is selected as the result of the operation (selected randomly from the
intersection of all the replies);
6. If only a minority accept the request, this minority are asked to release their locks and
phase 1 repeats.
1. The requesting site multicasts a remove request to all members of the view citing the
tuple to be removed;
2. On receiving this request, members remove the tuple from their replica, send an
acknowledgement and release the lock;
3. Step 1 is repeated until all acknowledgements are received.

115
Partitioning in the York Linda
Kernel

117
Java class AlarmTupleJS
import net.jini.core.entry.*;
public class AlarmTupleJS implements Entry {
public String alarmType;
public AlarmTupleJS() { }
}
public AlarmTupleJS(String alarmType) {
this.alarmType = alarmType;}
}
} Java class FireAlarmJS
import net.jini.space.JavaSpace;
public class FireAlarmJS {
try {
JavaSpace space = SpaceAccessor.findSpace("AlarmSpace");
AlarmTupleJS tuple = new AlarmTupleJS("Fire!");
space.write(tuple, null, 60*60*1000);
catch (Exception e) {
} }}

118
Java class FireAlarmReceiverJS
import net.jini.space.JavaSpace;
public class FireAlarmConsumerJS {
try {
JavaSpace space = SpaceAccessor.findSpace();
AlarmTupleJS template = new AlarmTupleJS("Fire!");
AlarmTupleJS recvd = (AlarmTupleJS) space.read(template,
null,
Long.MAX_VALUE);
return recvd.alarmType;
}
catch (Exception e) {
return null;
}
}}

119
Summary of indirect communication
styles

From Coulouris, Dollimore, Kindberg and
Blair
Distributed Systems:
Concepts and Design
Edition 5, © Addison-Wesley 2012
Slides for Chapter 7:
Operating System support

System layers
Applications, services
Computer &
Platform
Middleware
OS: kernel,
libraries &
servers
network hardware
OS1
Computer &
network hardware
Node 1 Node 2
Processes, threads,
communication, ...
OS2
Processes, threads,
communication, ...

Core OS functionality
Communication
manager
Thread manager Memory manager
Supervisor
Process manager

Address space
St ack
T
ex t
H
e ap
Au xili
a r
y
r
e gions
0
2N

Copy-on-write
a) Before write b) After write
Shared
frame
A's page
table
B's page
table
Process A’s address space Process B’s address space
Kernel
RA RB
RB copied
from RA

Client and server with threads
Server
N threads
Input-output
Client
Thread 2 makes
T1
Thread 1
requests to server
generates
results
Requests
Receipt &
queuing

Alternative server threading
architectures
a. Thread-per-request b. Thread-per-connection c. Thread-per-object
remote
workers
I/O remote
remote I/O
per-connection threads per-object threads
objects objects objects

State associated with execution
environments and threads
Execution environment Thread
Address space tables Saved processor registers
Communication interfaces, open files Priority and execution state (such as
BLOCKED)
Semaphores, other synchronization
objects
Software interrupt handling information
List of thread identifiers Execution environment identifier
Pages of address space resident in memory; hardware cache entries

Java thread constructor and
management methods
Thread(ThreadGroup group, Runnable target, String name)
Creates a new thread in the SUSPENDED state, which will belong to group and
be identified as name; the thread will execute the run() method of target.
setPriority(int newPriority), getPriority() - Set and return the thread‟s priority.
run()
A thread executes the run() method of its target object, if it has one, and otherwise
its own run() method (Thread implements Runnable).
start()
Change the state of the thread from SUSPENDED to RUNNABLE.
sleep(int millisecs)
Cause the thread to enter the SUSPENDED state for the specified time.
yield()
Causes the thread to enter the READY state and invoke the scheduler.
destroy()
Destroy the thread.

Java thread synchronization calls
thread.join(int millisecs)
Blocks the calling thread for up to the specified time until thread has
terminated.
thread.interrupt()
Interrupts thread: causes it to return from a blocking method call
such as sleep().
object.wait(long millisecs, int nanosecs)
Blocks the calling thread until a call made to notify() or notifyAll()
on object wakes the thread, or the thread is interrupted, or the
specified time has elapsed.
object.notify(), object.notifyAll()
Wakes, respectively, one or all of any threads that have called wait()
on object.

Scheduler activations
Process
A
Process
B
Virtual processors Kernel
Process
Kernel
P idle
P needed
P added
SA blocked
SA unblocked
SA preempted
A. Assignment of virtual processors
to processes
B. Events between user-level scheduler & kernel
Key: P = processor; SA = scheduleractivation

Invocations between address spaces
Control t ransfer v
ia
trap inst ruc tion
Us er Kernel
Thread
Us er 1 Us er 2
Control t ransfer v
ia
priv
ileged instruct ions
Thread 1 Thread 2
Prot ect ion domain
boundary
(a) Sys tem c all
(b) RPC/RMI (within one computer)
Kernel
(c ) RPC/RMI (between computers)
Us er 1 Us er 2
Thread 1 Network Thread 2
Kernel 2
Kernel 1

RPC delay against parameter size
1000 2000
RPC delay
Reques ted data
size (bytes)
Packet
size
0

A lightweight remote procedure call
1. Copy args
2. Trap to Kernel
4. Execute procedure
and copy results
Client
User stub
Server
Kernel
stub
3. Upcall 5. Return (trap)
A
A stack

Times for serialized and concurrent
invocations
Client Server
ex ecute request
Send
Receive
unmarshal
mars hal
Receive
unmarshal
proces s results
mars hal
Send
proces s args
mars hal
Send
proces s args
trans miss ion
Receive
unmarshal
proces s results
ex ecute request
Send
Receive
unmarshal
mars hal
mars hal
Send
proces s args
mars hal
Send
proces s args
ex ecute request
Send
Receive
unmarshal
mars hal
ex ecute request
Send
Receive
unmarshal
mars hal
Receive
unmarshal
proces s results
Receive
unmarshal
proces s results
time
Client Server
Serialised invocations Conc urrent invocations

Monolithic kernel and microkernel
Monolithic Kernel Microkernel
Server: Dynamically loaded server program:
Kernel code and data:
.......
.......
Key:
S4
S1 .......
S1 S2 S3
S2 S3 S4

The role of the microkernel
Middleware
Language
support
subsystem
Language
support
subsystem
OS emulation
subsystem
....
Microkernel
Hardware
Themicrokernelsupportsmiddleware via subsystems

Virtualization of memory
management
139

The XenoServer Open Platform
Architecture
142

143
DISTRIBUTED FILE
SYSTEMS
From Chapter 8 of Distributed Systems Concepts and Design,4th Edition,
By G. Coulouris, J. Dollimore and T. Kindberg

DISTRIBUTED FILE SYSTEMS
 Introduction
 File Service Architecture
 DFS: Case Studies
 Case Study: Sun NFS
 Case Study: The Andrew File System

Introduction
 File system were originally developed for centralized computer
systems and desktop computers.
 File system was as an operating system facility providing a
convenient programming interface to disk storage.
 Distributed file systems support the sharing of information in the
form of files and hardware resources.
 With the advent of distributed object systems (CORBA, Java) and the
web, the picture has become more complex.
 Fig 1 provides an overview of types of storage system.
 Fig 2 shows a typical layered module structure for the implementation
of a non-distributed file system in a conventional operating system.
 File systems are responsible for the organization, storage, retrieval,
naming, sharing and protection of files. Files contain both data and
attributes. A typical attribute record structure is illustrated in Fig 3.
 Fig 4 summarizes the main operations on files that are available to
applications in UNIX systems.
145

Introduction
146
Sharing Persis-
tence
Distributed
cache/replicas
Consistency
maintenance
Example
Main memory RAM
File system UNIX file system
Distributed file system Sun NFS
Web Web server
Distributed shared memory Ivy (Ch. 18)
Remote objects (RMI/ORB) CORBA
Persistent object store 1 CORBA Persistent
Object Service
Peer-to-peer storage system OceanStore(Ch. 10)
1
1
1
Figure 1. Storage systems and their properties
Types of consistency between copies: 1 - strict one-copy consistency
√ - approximate consistency
X - no automatic consistency

147
Directory module: relates file names to file IDs
File module: relates file IDs to particular files
Ac ces s control module: checks permis sion for operation requested
File ac cess module: reads or writes file data or attributes
Block module: ac cess es and allocates disk block s
Device module: disk I/O and buffering
Figure 2. File system modules
Introduction

148
File length
Creation timestamp
Read timestamp
Write timestamp
Attribute timestamp
Reference count
Owner
File type
Access control list
Figure 3. File attribute record structure
Introduction

149
filedes = open(name, mode)
filedes = creat(name, mode)
Opens an existing file with the given name.
Creates a new file with the given name.
Both operations deliver a file descriptor referencing the open
file. The mode is read, write or both.
status = close(filedes) Closes the open file filedes.
count = read(filedes, buffer, n)
count = write(filedes, buffer, n)
Transfers n bytes from the file referenced by filedes to buffer.
Transfers n bytes to the file referenced by filedes from buffer.
Both operations deliver the number of bytes actually transferred
and advance the read-write pointer.
pos = lseek(filedes, offset,
whence)
Moves the read-write pointer to offset (relative or absolute,
depending on whence).
status = unlink(name) Removes the file name from the directory structure. If the file
has no other names, it is deleted.
status = link(name1, name2) Adds a new name (name2) for a file (name1).
status = stat(name, buffer) Gets the file attributes for file name into buffer.
Figure 4. UNIX file system operations
Introduction

 Distributed File system requirements
 Related requirements in distributed file systems are:
 Transparency
 Location
 Access
 Performance
 Mobility
 Scaling
 Concurrency
 Replication
 Heterogeneity
 Fault tolerance
 Consistency
 Security
 Efficiency
150
Introduction

 An architecture that offers a clear separation of the main concerns in
providing access to files is obtained by structuring the file service as
three components:
 A flat file service
 A directory service
 A client module.
 The relevant modules and their relationship is shown in Figure 5.
 The Client module implements exported interfaces by flat file and
directory services on server side.
 Responsibilities of various modules can be defined as follows:
 Flat file service:
 Concerned with the implementation of operations on the contents
of file. Unique File Identifiers (UFIDs) are used to refer to files in
all requests for flat file service operations. UFIDs are long
sequences of bits chosen so that each file has a unique among all
of the files in a distributed system.
File Service Architecture
151

152
Client computer Server computer
Application
program
Application
program
Client module
Flat file service
Directory service
Figure 5. File service architecture

 Directory service:
 Provides mapping between text names for the files and their
UFIDs. Clients may obtain the UFID of a file by quoting its text
name to directory service. Directory service supports functions
needed generate directories, to add new files to directories.
 Client module:
 It runs on each computer and provides integrated service (flat
file and directory) as a single API to application programs. For
example, in UNIX hosts, a client module emulates the full set of
Unix file operations.
 It holds information about the network locations of flat-file and
directory server processes; and achieve better performance
through implementation of a cache of recently used file blocks
at the client.
 Flat file service interface:
 Fig 6 contains a definition of the interface to a flat file service.
153

154
Read(FileId, i, n) -> Data if 1≤i≤Length(File): Reads a sequence of up to n items
-throws BadPosition from a file starting at item i and returns it in Data.
Write(FileId, i, Data) if 1≤i≤Length(File)+1: Write a sequence of Data to a
-throws BadPosition file, starting at item i, extending the file if necessary.
Create() -> FileId Creates a new file of length0 and delivers a UFID for it.
Delete(FileId) Removes the file from the file store.
GetAttributes(FileId) -> Attr Returns the file attributes for the file.
SetAttributes(FileId, Attr) Sets the file attributes (only those attributes that are not
shaded in Figure 3.)
Figure 6. Flat file service operations

 Access control
 In distributed implementations, access rights checks have to be
performed at the server because the server RPC interface is an
otherwise unprotected point of access to files.
 Directory service interface
 Fig 7 contains a definition of the RPC interface to a directory service.
155
Lookup(Dir, Name) -> FileId Locates the text name in the directory and
-throws NotFound returns the relevant UFID. If Name is not in dir, throws an exception.
AddName(Dir, Name, File) If Name is not in the directory, adds(Name,File)
-throws NameDuplicate to the directory and updates the file‟s attribute record.
If Name is already in the directory: throws an exception.
UnName(Dir, Name) If Name is in the directory, the entry containing Name
is removed from the directory.
If Name is not in the directory: throws an exception.
GetNames(Dir, Pattern) -> NameSeq Returns all the text names in the directory that match the
regular expression Pattern. Fig 7. Directory service operations

 Hierarchical file system
 A hierarchical file system such as the one that UNIX provides consists
of a number of directories arranged in a tree structure.
 File Group
 A file group is a collection of files that can be located on any server or
moved between servers while maintaining the same names.
 A similar construct is used in a UNIX file system.
 It helps with distributing the load of file serving between several servers.
 File groups have identifiers which are unique throughout the system (and
hence for an open system, they must be globally unique).
 To construct a globally unique ID we use some unique attribute of the machine
on which it is created, e.g. IP number, even though the file group may move
subsequently.
156
IP address date
32 bits 16 bits
File Group ID:

DFS: Case Studies
 NFS (Network File System)
 Developed by Sun Microsystems (in 1985)
 Most popular, open, and widely used.
 NFS protocol standardized through IETF (RFC 1813)
 AFS (Andrew File System)
 Developed by Carnegie Mellon University as part of Andrew
distributed computing environments (in 1986)
 A research project to create campus wide file system.
 Public domain implementation is available on Linux
(LinuxAFS)
 It was adopted as a basis for the DCE/DFS file system in the
Open Software Foundation (OSF, www.opengroup.org) DEC
(Distributed Computing Environment
157

NFS architecture
158
Client computer Server computer
UNIX
file
system
NFS
client
NFS
server
UNIX
file
system
Application
program
Application
program
Virtual file system
Virtual file system
Other
file
system
UNIX kernel
system calls
NFS protocol
(remote operations)
UNIX
Operations
on local files
Operations
on
remote files
*
UNIX kernel
Figure 8. NFS architecture
Case Study: Sun NFS

fh = file handle:
Filesystem identifier i-node number i-node generation
Case Study: Sun NFS
 The file identifiers used in NFS are called file handles.
 A simplified representation of the RPC interface provided by NFS
version 3 servers is shown in Fig 9.
159
 read(fh, offset, count) -> attr, data
 write(fh, offset, count, data) -> attr
 create(dirfh, name, attr) -> newfh, attr
 remove(dirfh, name) status
 getattr(fh) -> attr
 setattr(fh, attr) -> attr
 lookup(dirfh, name) -> fh, attr
 rename(dirfh, name, todirfh, toname)
 link(newdirfh, newname, dirfh, name)
 readdir(dirfh, cookie, count) -> entries
 symlink(newdirfh, newname, string) ->
status
 readlink(fh) -> string
 mkdir(dirfh, name, attr) -> newfh, attr
 rmdir(dirfh, name) -> status
 statfs(fh) -> fsstats
Figure 9. NFS server operations (NFS Version 3 protocol, simplified)

Case Study: Sun NFS
 NFS access control and authentication
 The NFS server is stateless server, so the user's identity and access rights
must be checked by the server on each request.
 In the local file system they are checked only on the file‟s access
permission attribute.
 Every client request is accompanied by the userID and groupID
 It is not shown in the Fig because they are inserted by the RPC
system.
 Kerberos has been integrated with NFS to provide a stronger and more
comprehensive security solution.
 Mount service
 Mount operation: mount(remotehost, remotedirectory, localdirectory)
 Server maintains a table of clients who‟‟s mounted FS at that server.
 Each client maintains a table of mounted file systems holding:
< IP address, port number, file handle>
 Remote file systems may be hard-mounted or soft-mounted in a client
computer. Fig 10 shows a Client with two remotely mounted file stores.
160

Case Study: Sun NFS
jim jane joe
ann
users
students
usr
vmunix
Client Server 2
. . . nfs
Remote
mount
staff
big bob
jon
people
Server 1
export
(root)
Remote
mount
. . .
x
(root) (root)
161
Note: The file system mounted at /usr/students in the client is actually the sub-tree located at /export/people in Server 1;
the file system mounted at /usr/staff in the client is actually the sub-tree located at /nfs/users in Server 2.
Figure 10. Local and remote file systems accessible on an NFS client

Case Study: Sun NFS
 Automounter
 The automounter was added to the UNIX implementation of
NFS in order to mount a remote directory dynamically
whenever an „empty‟ mount point is referenced by a client.
 Automounter has a table of mount points with a reference to one
or more NFS servers listed against each.
 it sends a probe message to each candidate server and then uses
the mount service to mount the filesystem at the first server to
respond.
 Automounter keeps the mount table small.
 Automounter Provides a simple form of replication for
read-only filesystems.
 E.g. if there are several servers with identical copies of /usr/lib
then each server will have a chance of being mounted at some
clients.
162

Case Study: Sun NFS
 Server caching
 Similar to UNIX file caching for local files:
 pages (blocks) from disk are held in a main memory buffer
cache until the space is required for newer pages. Read-ahead
and delayed-write optimizations.
 For local files, writes are deferred to next sync event (30 second
intervals).
 Works well in local context, where files are always accessed
through the local cache, but in the remote case it doesn't offer
necessary synchronization guarantees to clients.
 NFS v3 servers offers two strategies for updating the disk:
 Write-through - altered pages are written to disk as soon as they
are received at the server. When a write() RPC returns, the NFS
client knows that the page is on the disk.
 Delayed commit - pages are held only in the cache until a
commit() call is received for the relevant file. This is the default
mode used by NFS v3 clients. A commit() is issued by the client
whenever a file is closed.
163

Case Study: Sun NFS
 Client caching
 Server caching does nothing to reduce RPC traffic bet. Client/serv
 further optimization is essential to reduce server load in
large networks.
 NFS client module caches the results of read, write, getattr,
lookup and readdir operations
 synchronization of file contents (one-copy semantics) is not
guaranteed when two or more clients are sharing the same
file.
164
t freshness
guarantee
Tc time when
cache entry
was last
validated
Tm time when
block was
last updated
at server
T current time
 Timestamp-based validity check
 It reduces inconsistency, but doesn't eliminate it.
 It is used for validity condition for cache entries at
the client: (T - Tc < t) v (Tmclient = Tmserver)
 t is configurable (per file) but is typically set to 3 seconds for files and 30
secs. for directories.
 difficult to write distributed appl. that share files with NFS.

Case Study: Sun NFS
 Other NFS optimizations
 Sun RPC runs over UDP by default (can use TCP if required).
 Uses UNIX BSD Fast File System with 8-kbyte blocks.
 reads() and writes() can be of any size (negotiated between client and s).
 The guaranteed freshness interval t is set adaptively for individual files to
reduce getattr() calls needed to update Tm.
 File attribute information (including Tm) is piggybacked in replies to all
file requests.
 NFS performance - Early measurements (1987) established that:
 Write() operations are responsible for only 5% of server calls in typical
UNIX environments.
 hence write-through at server is acceptable.
 Lookup() accounts for 50% of operations -due to step-by-step pathname
resolution necessitated by the naming and mounting semantics.
 More recent measurements (1993) show high performance.
 see www.spec.org for more recent measurements.
165

Case Study: Sun NFS
 NFS summary
 NFS is an excellent example of a simple, robust, high-performance dist.service
 Achievement of transparencies are other goals of NFS:
 Access transparency:
 The API is the UNIX system call interface for both local and remote
files.
 Location transparency:
 Naming of filesystems is controlled by client mount operations, but
transparency can be ensured by an appropriate system configuration.
 Mobility transparency:
 Hardly achieved; relocation of files is not possible, relocation of
filesystems is possible, but requires updates to client configurations.
 Scalability transparency:
 File systems (file groups) may be subdivided and allocated to separate
servers.
 Ultimately, the performance limit is determined by the load on the
server holding the most heavily-used filesystem (file group) 166

Case Study: Sun NFS
 Replication transparency: Limited to read-only file systems; for writable
files, the SUN Network Information Service (NIS) runs over NFS and is
used to replicate essential system files.
 Hardware and software operating system heterogeneity:
 NFS has been implemented for almost every known OS and
hardware platform and is supported by a variety of filling systems.
 Fault tolerance: Limited but effective; service is suspended if a server
fails. Recovery from failures is aided by the simple stateless design.
 Consistency: It provides a close approximation to one-copy semantics and
meets the needs of the vast majority of applications.
 But the use of file sharing via NFS for communication or close
coordination between processes on different computers cannot be
recommended.
 Security: Recent developments include the option to use a secure RPC
implementation for authentication and the privacy and security of the data
transmitted with read and write operations.
 Efficiency: NFS protocols can be implemented for use in situations that
generate very heavy loads. 167

Case Study: The Andrew File
System (AFS)
 Like NFS, AFS provides transparent access to remote shared files for
UNIX programs running on workstations.
 AFS is implemented as two software components that exist at UNIX
processes called Vice and Venus. (Figure 11)
 Observations of AFS team
 File size relatively small
 Read operations
 Sequential access
 Mostly only one user writes
 Files are referenced in bursts
 Whole file serving
 Whole file caching - consistency
168

The Andrew File System (AFS)
Venus
Workstations Servers
Venus
Venus
Us er
program
Network
UNIXk ernel
UNIXk ernel
Vic e
Us er
program
Us er
program
Vic e
UNIXk ernel
UNIXk ernel
UNIXk ernel
169
Figure 11. Distribution of processes in the Andrew File System

 The files available to user processes running on workstations are either
local or shared.
 Local files are handled as normal UNIX files.
 They are stored on the workstation‟s disk and are available only to local
user processes.
 Shared files are stored on servers, and copies of them are cached on the
local disks of workstations.
 The name space seen by user processes is illustrated in Fig 12.
 The UNIX kernel in each workstation and server is a modified version
of BSD UNIX.
 The modifications are designed to intercept open, close and some other
file system calls when they refer to files in the shared name space and
pass them to the Venus process in the client computer. (Fig 13)
170

/ (root)
tmp bin cmu
vmunix
. . .
bin
Shared
Local
Sy mbolic
links
171
Figure 12. File name space seen by clients of AFS
Volume number File handle uniquifier
Volume number – file group id (32 bit)
File handle – UFID (32 bit) Uniquifier – File ID are not reused(32 bit)

UNIXfile
system calls
Non-local file
operations
Workstation
Local
disk
User
program
UNIXkernel
Venus
UNIXfile system
Venus
172
Figure 13. System call interception in AFS

 Fig 14 describes the actions taken by Vice, Venus and the UNIX
kernel when a user process issues system calls.
173
User process UNI X kernel Venus Net Vi
c e
open(F i
leNam e,
mode)
If FileName ref ers t o a
file in shar ed file space,
pass t he r equest to
V
enus.
Open the loca l f i
le and
ret urn the file
descript or to the
application.
Check list of files in
loc al cache. If not
present or t he re is no
vali
d call
back prom ise,
send a request f or t he
file to the V
ice ser ver
that is custodian of t he
volume c ontaining t he
file.
Place the copy of the
file in t he local f il
e
system, enter i
t s loc al
name in t he local ca che
list and ret urn the l
ocal
name to UNIX.
T ransf er a copy of the
file anda call
back
promise to the
w
orkst ation. Log the
callback promi
se.
read(FileDesc riptor,
Buffer, l
e ngth)
Per form a normal
UNIX read operation
on t he local copy.
writ
e( FileDescri
ptor,
Buffer, l
e ngth)
Per form a normal
UNIX w
r i
t e operat ion
on t he local copy.
close(F i
leDescript
or) Close the loca l copy
and notif y V
enus t hat
the file has been closed. If the l
ocal copy has
been changed, send a
copy to the V
ice ser ver
that is t he custodian of
the file.
Replace the file
cont ent s and send a
call
back to all other
client s holding callback
promises on the file.
Fig 14. implementation of file system calls in AFS

 Cache consistency
 Callback promise
 valid or cancelled
 Fig 15 shows the RPC calls provided by AFS servers for operations
on files.
 Other Aspects
 UNIX kernel modifications
 Location Database
 Threads - non preemptive
 Bulk Transfers
 Partial File Caching
 Wide-area support 174

175
Fetch(fid) -> attr, data Returns the attributes (status) and, optionally, the contents of file
identified by the fid and records a callback promise on it.
Store(fid, attr, data) Updates the attributes and (optionally) the contents of a specified
file.
Create() -> fid Creates a new file and records a callback promise on it.
Remove(fid) Deletes the specified file.
SetLock(fid, mode) Sets a lock on the specified file or directory. The mode of the
lock may be shared or exclusive. Locks that are not removed
expire after 30 minutes.
ReleaseLock(fid) Unlocks the specified file or directory.
RemoveCallback(fid) Informs server that a Venus process has flushed a file from its
cache.
BreakCallback(fid) This call is made by a Vice server to a Venus process. It cancels
the callback promise on the relevant file.
Figure 15. The main components of the Vice service interface

18CS3040_Distributed Systems

More Related Content

Similar to 18CS3040_Distributed Systems

More from Kathirvel Ayyaswamy

Recently uploaded

18CS3040_Distributed Systems