Module 2

Shared Memory
By Prof. Ankita Mandore
CSE (Data Science)
DSCE
What is a Shared Memory?
 Shared Memory is a concept of Operating Systems and is mainly used in Inter-
process Communication.
 If two processes want to share data with each other, they need some
intermediate medium to do so and this is where shared memory comes into
the picture.
 Every process has a dedicated address space in order to store data.
 If a process wants to share some data with another process, it cannot directly
do so since they have different address spaces.
 In order to share some data, a process takes up some of the address space as
shared memory space.
 This shared memory can be accessed by the other process to read/write the
shared data.
Working of Shared Memory
 Let us consider two processes P1 and P2 that want to perform Inter-process
communication using a shared memory.
 P1 has an address space, let us say A1 and P2 has an address space, let us say
A2. Now, P1 takes up some of the available address space as a shared memory
space, let us say S1. Since P1 has taken up this space, it can decide which
other processes can read and write data from the shared memory space.
 For now, we will assume that P1 has given only reading rights to other
processes with respect to the shared memory. So, the flow of Inter-process
communication will be as follows:
 Process P1 takes up some of the available space as shared memory S1
 Process P1 writes the data to be shared in S1
 Process P2 reads the shared data from S1
Use Cases of Shared Memory
 Inter-Process Communication: Shared memory is primarily used in IPC where
two processes need a shared address space in order to exchange data.
 Parallel Processing: Multiple processes can share and modify data in the
shared address space, thereby speeding up computations.
 Databases: Shared memory is used in database, in the form of cache, so that
reading and writing of data can be much faster
 Graphics and Multimedia Applications: CPU and GPU can access data
concurrently which is helpful in tasks such as video manipulation and
processing.
 Distributed Systems: Two different machines can access data from a shared
space and work as a single system.
 Advantages
 Shared memory is one of the fastest means of IPC since it avoids overheads.
 Easy access to data once set up.
 It is memory efficient as processes do not need to separately store shared data.
 Disadvantages
 Since it is operating system specific, it is difficult to implement
common synchronization and authorization techniques.
 Memory leaks can take place.
 If the processes wait indefinitely for each other in order to release the shared
memory space, a deadlock can occur.
What is Distributed Shared Memory?
 DSM is a type of service that manages the memory across multiple nodes so
that applications that are running will have the illusion that they are running
on shared memory.
 DSM is used to make inter-process communication transparent to end-
users. DSM is supported in hardware and software.
 Distributed Shared Memory(DSM) states that:
 The variable is shared directly.
 The process could cause errors by altering the data.
 Each node in the DSM should have:
 It's own memory.
 Reading and Writing services.
 Consistency protocols.
What is Distributed Shared Memory?
 Distributed shared memory (DSM) is a mechanism that manages memory
across multiple nodes and makes inter-process communications transparent to
end-users.
• The applications will think that they are running on shared memory.
 DSM is a mechanism of allowing user processes to access shared data without
using inter-process communications.
• In DSM every node has its own memory and provides memory read and
write services and it provides consistency protocols.
 The DSM implements the shared memory model in distributed systems but it
doesn't have physical shared memory. All the nodes share the virtual address
space provided by the shared memory model. The Data moves between the
main memories of different nodes.
 Distributed shared memory (DSM) is a form of memory architecture where the
(physically separate) memories can be addressed as one (logically shared)
address space. Here, the term "shared" does not mean that there is a single
centralized memory but "shared" means that the address space is shared.
 The shared memory model provides a virtual address space shared between
all nodes.
 The overcome the high cost of communication in distributed systems, DSM
systems move data to the location of access.
How it works?

 Data moves between main memory and secondary memory (within a node)
and between main memories of different nodes.

 When a process accesses data in the shared address space, the mapping
manager maps shared memory address to physical memory (local or remote).
Types of Distributed shared memory

Types of
DSM

Hardware Software Hybrid

Level Level Level
(HDSM) (SDSM) (HSDSM)
Hardware DSM
1. On-Chip Memory
2. Bus-Based Multiprocessors
3. Ring-Based Multiprocessors
4. Switched Multiprocessors
On-Chip Memory
 The data is present in the CPU portion of the chip.
 Memory is directly connected to address lines.
 On-Chip Memory DSM is expensive and complex.
 Such chips are widely used in appliances cars and even toys.
Bus-Based Multiprocessors
 Connection between CPU and memory is set of parallel wires some holding
address of CPU wants to read or write or for sending or receiving messages
 Network traffic is reduced by using caches with each CPU.
 Algorithms are used to prevent two CPU trying to access same memory
simultaneously.
 Having single bus makes it overloaded.
Ring-Based Multiprocessors
 There is no global centralized memory present in Ring-based DSM.
 All nodes are connected via a token passing ring.
 A single address line is partitioned into a private area and shared area.
 Private area is divided up into regions so each machine has a piece for its
stack
 Shared area is divided into 32 byte blocks.
 All machines are connected via token passing ring. All components are
interconnected via Memnet device.
 No centralized global memory.
Switched Multiprocessors
 Two approaches can be taken to attack the problem of not enough bandwidth.
 Reduce the amount of communication. E.g. Caching.
 Increase the communication capacity. E.g. Changing topology.
 One method is to build the system as a hierarchy. Build the system as multiple
clusters and connect the clusters using an intercluster bus. As long as most
CPUs communicate primarily within their own cluster, there will be relatively
little intercluster traffic. If still more bandwidth is needed, collect a bus,
tree, or grid of clusters together into a supercluster, and break the system
into multiple superclusters.
Switched Multiprocessors
Advantages of Distributed Shared Memory
 Simpler Abstraction: Programmer need not concern about data movement, as the
address space is the same it is easier to implement than RPC.
 Easier Portability: The access protocols used in DSM allow for a natural transition
from sequential to distributed systems. DSM programs are portable as they use a
common programming interface.
 Locality of Data: Data moved in large blocks i.e. data near to the current memory
location that is being fetched, may be needed future so it will be also fetched.
 On-Demand Data Movement: It provided by DSM will eliminate the data exchange
phase.
 Larger Memory Space: It provides large virtual memory space, the total memory
size is the sum of the memory size of all the nodes, paging activities are reduced.
 Better Performance: DSM improve performance and efficiency by speeding up
access to data.
 Flexible Communication Environment: They can join and leave DSM system
without affecting the others as there is no need for sender and receiver to
existing,
 Process Migration Simplified: They all share the address space so one process can
easily be moved to a different machine.
Disadvantages of Distributed Shared
Memory
 Accessibility: The data access is slow in DSM as compare to non-distributed.
 Consistency: When programming is done in DSM systems, programmers need
to maintain consistency.
 Message Passing: DSM use asynchronous message passing and is not efficient
as per other message passing implementation.
 Data Redundancy: DSM allows simultaneous access to data, consistency and
data redundancy is common disadvantage.
 Lower Performance: CPU gets slowed down, even cache memory does not aid
the situation.
Design issues of DSM

 Granularity
 Structure of shared memory space
 Memory coherence and access synchronization
 Data location and access
 Granularity: Granularity refers to the block size of a DSM system. Granularity
refers to the unit of sharing and the unit of data moving across the network
when a network block shortcoming then we can utilize the estimation of the
block size as words/phrases. The block size might be different for the various
networks.
 Structure of shared memory space: Structure refers to the design of the
shared data in the memory. The structure of the shared memory space of a
DSM system is regularly dependent on the sort of applications that the DSM
system is intended to support.
 Memory coherence and access synchronization: In the DSM system the
shared data things ought to be accessible by different nodes simultaneously in
the network. The fundamental issue in this system is data irregularity. The
data irregularity might be raised by the synchronous access. To solve this
problem in the DSM system we need to utilize some synchronization
primitives, semaphores, event count, and so on.
Implementation issues of DSM

 Replacement strategy
 Thrashing
 Heterogeneity
 Data location and access: To share the data in the DSM system it ought to be
possible to locate and retrieve the data as accessed by clients or processors.
Therefore the DSM system must implement some form of data block finding system
to serve network data to meet the requirement of the memory coherence
semantics being utilized.
 Replacement strategy: In the local memory of the node is full, a cache miss at
the node implies not just a get of the gotten to information block from a remote
node but also a replacement. A data block of the local memory should be replaced
by the new data block. Accordingly, a position substitution methodology
 Thrashing: In a DSM system data blocks move between nodes on demand. In this
way on the off chance that 2 nodes compete for write access to the single data
item. The data relating data block might be moved to back and forth at such a
high rate that no genuine work can get gone. The DSM system should utilize an
approach to keep away from a situation generally known as thrashing.
 Heterogeneity: The DSM system worked for homogeneous systems and need not
address the heterogeneity issue. In any case, assuming the underlined system
environment is heterogeneous, the DSM system should be designed to deal with
heterogeneous, so it works appropriately with machines having different
architectures. is additionally vital in the design of a DSM system.
DSM architecture
 High speed communication network is used for connecting nodes.
 A simple message passing system allows on different node to exchange
message with each other.
 Memory mapping manager routine in each node maps local memory onto the
shared virtual memory. For mapping operation, the shared memory space is
partitioned into blocks.
 The main memory of individual nodes is used to cache of shared memory
space.
DSM architecture
 The architecture of a Distributed Shared Memory (DSM) system typically consists of
several key components that work together to provide the illusion of a shared
memory space across distributed nodes. the components of Architecture of
Distributed Shared Memory :
[Link]: Each node in the distributed system consists of one or more CPUs and a
memory unit. These nodes are connected via a high-speed communication network.
[Link] Mapping Manager Unit: The memory mapping manager routine in each
node is responsible for mapping the local memory onto the shared memory space.
This involves dividing the shared memory space into blocks and managing the mapping
of these blocks to the physical memory of the node.
Caching is employed to reduce operation latency. Each node uses its local memory to
cache portions of the shared memory space. The memory mapping manager treats the
local memory as a cache for the shared memory space, with memory blocks as the
basic unit of caching.
[Link] Network Unit: This unit facilitates communication between nodes.
When a process accesses data in the shared address space, the memory mapping
manager maps the shared memory address to physical memory. The communication
network unit handles the communication of data between nodes, ensuring that data
can be accessed remotely when necessary.
DSM architecture
 A layer of code, either implemented in the operating system kernel or as a
runtime routine, is responsible for managing the mapping between shared
memory addresses and physical memory locations.
 Each node’s physical memory holds pages of the shared virtual address space.
Some pages are local to the node, while others are remote and stored in the
memory of other nodes.
 Systems that support DSM, data moves between secondary memory and main
memory as well as between main memories of different nodes.
 When process access data in shared address space, a mapping manager maps
the shared memory address to the physical memory. The mapping manager is
layer of software implemented either in the operating kernel or as runtime
library routine.
Consistency Model in Distributed System
 In distributed systems, consistency models establish criteria for data
synchronization and specify how users and applications should interpret data
changes across several nodes.
 In a distributed system, it specifically controls how data is accessed and
changed across numerous nodes and how clients are informed of these
updates.
 These models range from strict to relaxed approaches.
Types of Consistency Models
1. Strong Consistency
 In a strongly consistent system, all nodes in the system agree on the order in which
operations occurred.
 Reads will always return the most recent version of the data, when an update
occurs on one server, this model makes sure every other server in the system
reflects this change immediately.
 This model provides the highest level of consistency, but it can be slower and
require more resources in a distributed environment since all servers must stay
perfectly in sync.
2. Sequential Consistency Model
 It is a consistency model in distributed systems that ensures all operations across
processes appear in a single, unified order.
 In this model, every read and write operation from any process appears to happen
in sequence, regardless of where it occurs in the system.
 Importantly, all processes observe this same sequence of operations, maintaining a
sense of consistency and order across the system.
3. Causal Consistency Model
 The Causal Consistency Model is a type of consistency in distributed systems that
ensures that related events happen in a logical order.
 In simpler terms, if two operations are causally related (like one action causing
another), the system will make sure they are seen in that order by all users.
 However, if there’s no clear relationship between two operations, the system
doesn’t enforce an order, meaning different users might see the operations in
different sequences.
4. Weak Consistency Model
 A weakly consistent system provides no guarantees about the ordering of operations
or the state of the data at any given time.
 Clients may see different versions of the data depending on which node they connect
to.
 This model provides the highest availability and scalability but at the cost of
consistency.
5. Session Consistency
 Session Consistency guarantees that all of the data and actions a user engages with
within a single session remain consistent.
 Consider it similar to online shopping: session consistency ensures that an item will
always be in your cart until you check out or log out, regardless of how you explore
the page.
 In a distributed system, this means if you’re interacting with different servers or services,
you’ll always get the same view of your data during your session.
 It might not reflect the most recent updates from other users, but for your own actions, it
will remain consistent and reliable.
6. Monotonic Reads and Writes
 Once a piece of data has been read or written, monotonic reads and
writes guarantee that the data will always be viewed in a predictable and
consistent order in subsequent reads or writes.
 Monotonic Reads: If you read a value from a system, the next
time you read it, you will either get the same value or a more
recent one. You won’t get an older value after seeing a newer one.
 Monotonic Writes: This ensures that once a write happens, all
future writes will follow in the correct order. If you update a
record or send a message, the system guarantees that it won’t
reverse the order of your updates.