PARALLEL PROGRAM
CHALLENGES
Synchronization primitives
(semaphores, barriers)
S.SOUNDHARYA,AP/CSE 1
Semaphore
• Synchronization tool that provides more sophisticated ways (than Mutex locks) for
process to synchronize their activities.
• Semaphore S – integer variable
• Can only be accessed via two indivisible (atomic) operations
– wait() and signal()
• Originally called P() and V()
• Definition of the wait() operation
wait(S) {
while (S <= 0)
; // busy wait
S--;
}
• Definition of the signal() operation
signal(S) {
S++;
}
S.SOUNDHARYA,AP/CSE 2
Semaphore Usage
• Counting semaphore – integer value can range over an unrestricted domain
• Binary semaphore – integer value can range only between 0 and 1
– Same as a mutex lock
• Can solve various synchronization problems
• Consider P1 and P2 that require S1 to happen before S2
Create a semaphore “synch” initialized to 0
P1:
S1;
signal(synch);
P2:
wait(synch);
S2;
• Can implement a counting semaphore S as a binary semaphore
S.SOUNDHARYA,AP/CSE 3
Semaphore Implementation
• Must guarantee that no two processes can execute the
wait() and signal() on the same semaphore at the same
time
• Thus, the implementation becomes the critical section
problem where the wait and signal code are placed in the
critical section
– Could now have busy waiting in critical section
implementation
• But implementation code is short
• Little busy waiting if critical section rarely occupied
• Note that applications may spend lots of time in critical
sections and therefore this is not a good solution
S.SOUNDHARYA,AP/CSE 4
Semaphore Implementation with no Busy waiting
• With each semaphore there is an associated waiting
queue
• Each entry in a waiting queue has two data items:
– value (of type integer)
– pointer to next record in the list
• Two operations:
– block – place the process invoking the operation on the
appropriate waiting queue
– wakeup – remove one of processes in the waiting queue
and place it in the ready queue
• typedef struct{
int value;
struct process *list;
} semaphore;
S.SOUNDHARYA,AP/CSE 5
Implementation with no Busy waiting (Cont.)
wait(semaphore *S) {
S->value--;
if (S->value < 0) {
add this process to S->list;
block();
}
}
signal(semaphore *S) {
S->value++;
if (S->value <= 0) {
remove a process P from S->list;
wakeup(P);
}
}
S.SOUNDHARYA,AP/CSE 6
Barriers
• Synchronizing the threads to make sure that
they all are at the same point in a program is
called a barrier.
• No thread can cross the barrier until all the
threads have reached it.
S.SOUNDHARYA,AP/CSE 7
Using barriers to time the slowest thread
S.SOUNDHARYA,AP/CSE 8
Using barriers for debugging
S.SOUNDHARYA,AP/CSE 9
Busy-waiting and a Mutex
• Implementing a barrier using busy-waiting and
a mutex is straightforward.
• We use a shared counter protected by the
mutex.
• When the counter indicates that every thread
has entered the critical section, threads can
leave the critical section.
S.SOUNDHARYA,AP/CSE 10
Busy-waiting and a Mutex
We need one counter
variable for each
instance of the barrier,
otherwise problems
are likely to occur.
S.SOUNDHARYA,AP/CSE 11
Implementing a barrier with semaphores
S.SOUNDHARYA,AP/CSE 12
