Scribd
Upload
4 views

Unit 2 - 2.2 (Basic Algorithms)

The document discusses parallel and distributed computing, focusing on the efficiency of parallel algorithms compared to serial approaches. It covers key topics such as parallel matrix multiplication, sorting algorithms, and searching techniques, highlighting their implementation and associated challenges. The document emphasizes the importance of load balancing, minimizing communication overhead, and scalability in parallel computing systems.

Uploaded by

amitnization
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
4 views

Unit 2 - 2.2 (Basic Algorithms)

The document discusses parallel and distributed computing, focusing on the efficiency of parallel algorithms compared to serial approaches. It covers key topics such as parallel matrix multiplication, sorting algorithms, and searching techniques, highlighting their implementation and associated challenges. The document emphasizes the importance of load balancing, minimizing communication overhead, and scalability in parallel computing systems.

Uploaded by

amitnization
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 8

University Institute of Engineering

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

Parallel and Distributed Computing


(22CST-354/21ITH-354)

Prepared by: Er. Anupama Jamwal


(E14665)
Master Subject Coordinator

DISCOVER . LEARN .
EMPOWER
CSE Operating System(21CST-313/21ITH-313) 1
Department of Computer Science and Engineering (CSE)

Content

•Parallel Algorithms

University Institute of Engineering (UIE) 2


Department of Computer Science and Engineering (CSE)

Parallel Computing is defined as the process of distributing a larger task


into a small number of independent tasks and then solving them using
multiple processing elements simultaneously. Parallel computing is more
efficient than the serial approach as it requires less computation time.

Parallel Algorithm Models


The need for a parallel algorithm model arises in order to understand the
strategy that is used for the partitioning of data and the ways in which
these data are being processed. Therefore every model being used
provides proper structuring based on two techniques. They are as follows:

1. Selection of proper partitioning and mapping techniques.

2. Proper use of strategy in order to reduce the interaction.

University Institute of Engineering (UIE)


Department of Computer Science and Engineering (CSE)

1. Matrix Multiplication in Parallel Systems


Matrix multiplication is a fundamental operation in parallel computing due to its importance in
scientific computations and machine learning.
a. Basic Idea
Multiply two matrices and to produce , where . Distribute the workload of computing elements of
among multiple processors.
b. Parallel Algorithms
Row-wise partitioning involves dividing matrix by rows and distributing each row to processors.
Each processor computes its portion of using the entire matrix. Cannon’s Algorithm is designed
for a 2D grid of processors, where each processor computes partial results for using blocks of
and , with an efficient communication pattern to minimize overhead. Fox’s Algorithm is an
alternative for square processor grids, working in stages where rows of and corresponding blocks
of are broadcasted to compute partial products. Strassen’s Algorithm is a divide-and-conquer
approach to reduce computational complexity, which can be parallelized but involves additional
communication overhead.

University Institute of Engineering (UIE)


Department of Computer Science and Engineering (CSE)

Challenges
The primary challenges include load balancing to ensure all processors have an equal amount
of work, minimizing communication overhead to reduce data transfer between processors,
and scalability to maintain performance improvements with an increasing number of
processors.

2. Sorting in Parallel Systems


Sorting large datasets efficiently is a common problem in parallel computing.
a. Parallel Sorting Algorithms
Parallel merge sort is one of the most popular sorting algorithms used in parallel systems due to
its divide-and-conquer nature, which lends itself well to parallelization. The algorithm begins by
dividing the dataset into smaller chunks. Each chunk is assigned to a separate processor and
sorted independently, often using a standard sequential sorting algorithm like quicksort or
heapsort. Once all the individual chunks are sorted, the merging phase begins. During merging,
pairs of sorted chunks are combined into larger sorted arrays. This merging process is repeated
iteratively until the entire dataset is sorted. To facilitate parallelization, the merging steps can
also be distributed among processors, where each processor is responsible for merging a
specific subset of the data. This ensures efficient utilization of resources.

University Institute of Engineering (UIE)


Department of Computer Science and Engineering (CSE)

Another widely used algorithm is the bitonic sort, a specialized sorting network particularly effective for
datasets where the size is a power of two. Bitonic sort works by constructing bitonic sequences, which
are sequences that monotonically increase and then decrease or vice versa. These sequences are then
merged in parallel to produce a fully sorted list. This method is highly structured and predictable, making
it suitable for hardware implementations.
Odd-even transposition sort is a simple algorithm that relies on iterative compare-and-exchange
operations between neighboring elements. In each phase, adjacent elements are compared and
swapped if necessary, ensuring that smaller elements gradually move to the beginning of the array.
Alternate phases compare odd and even indexed pairs, gradually sorting the entire dataset. This
algorithm is easy to implement but can be slower than others for large datasets.

Sample sort is another approach that combines the advantages of partitioning and parallelization. In this
algorithm, a small sample of the data is chosen and sorted to determine pivot points. The dataset is then
divided into partitions based on these pivots, with each partition assigned to a processor. Each processor
sorts its partition independently, and the sorted partitions are combined to produce the final sorted
array. Sample sort is particularly effective for unevenly distributed data as it minimizes load imbalance
among processors.

University Institute of Engineering (UIE)


Department of Computer Science and Engineering (CSE)

b. Challenges
The main challenges in parallel sorting include communication overhead during partitioning
and merging, as processors must exchange data to ensure global order. Additionally,
maintaining global ordering while allowing local independence for individual processors can
be complex. Efficient parallel sorting algorithms must address these issues while ensuring
scalability and minimizing synchronization costs.
3. Searching in Parallel Systems
Efficient data retrieval is essential for large-scale systems.
a. Parallel Searching Algorithms
Binary search is implemented in parallel by distributing the array among processors. Each
processor performs a binary search on its portion of the array, and the results are combined to
determine the target’s location. Hash-based searching distributes data across processors using a
hash function, allowing search queries to be routed to the appropriate processor. Breadth-first
search (BFS) is used for graph traversals in parallel, where nodes at the current level are
processed in parallel, generating the next level’s nodes. Depth-first search (DFS), although more
challenging to parallelize, can be implemented using task-based frameworks.

University Institute of Engineering (UIE)


Department of Computer Science and Engineering (CSE)

b. Challenges
Challenges include balancing workload, particularly for unstructured data, synchronization to
ensure correct results, and efficient memory access patterns to optimize performance.

University Institute of Engineering (UIE)

You might also like