1

Sequential Consistency

and

Cache Coherence Protocols

Arvind

Computer Science and Artificial Intelligence Lab

M.I.T.

Based on the material prepared by

Arvind and Krste Asanovic

 6.823 L17- 2
Arvind

Memory Consistency in SMPs

CPU-1 CPU-2

A 100 cache-1 A 100 cache-2

CPU-Memory bus

A 100 memory

Suppose CPU-1 updates A to 200.

write-back: memory and cache-2 have stale values
write-through: cache-2 has a stale value

Do these stale values matter?

What is the view of shared memory for programming?
November 9, 2005
 6.823 L17- 3
Arvind

Write-back Caches & SC

prog T1 cache-1 memory cache-2 prog T2

ST X, 1 X= 1 X=0 Y= LD Y, R1
ST Y,11 Y=11 Y =10 Y’= ST Y’, R1
• T1 is executed X’= X= LD X, R2
Y’= X’= ST X’,R2
X= 1 X=0 Y=
• cache-1 writes back Y
Y=11 Y =11 Y’=
X’= X=
Y’= X’=
X= 1 X=0 Y = 11
Y=11 Y =11 Y’= 11
• T2 executed X’= X=0
Y’= X’= 0
X= 1 X=1 Y = 11
• cache-1 writes back X
Y=11 Y =11 Y’= 11

t
X’= X=0 n
Y’= X’= 0
r e

he
X= 1 X=1 Y =11 co
• cache-2 writes back
Y=11 Y =11 Y’=11 in
X’= 0 X=0
X’ & Y’
Y’=11 X’= 0
November 9, 2005
 6.823 L17- 4
Arvind

Write-through Caches & SC

cache-1 memory cache-2 prog T2

prog T1
X= 0 X=0 Y= LD Y, R1
ST X, 1
Y=10 Y =10 Y’= ST Y’, R1
ST Y,11
X’= X=0 LD X, R2
Y’= X’= ST X’,R2

X= 1 X=1 Y=
Y=11 Y =11 Y’=
• T1 executed
X’= X=0
Y’= X’=

X= 1 X=1 Y = 11
• T2 executed
Y=11 Y =11 Y’= 11
X’= 0 X=0
Y’=11 X’= 0

Write-through caches don’t preserve

sequential consistency either
November 9, 2005
 6.823 L17- 5
Arvind

Maintaining Sequential Consistency

SC is sufficient for correct producer-consumer

and mutual exclusion code (e.g., Dekker)

Multiple copies of a location in various caches

can cause SC to break down.

Hardware support is required such that

• only one processor at a time has write

permission for a location

• no processor can load a stale copy of
the location after a write

⇒ cache coherence protocols

November 9, 2005
 6.823 L17- 6
Arvind

A System with Multiple Caches

P P P P
L1 L1 L1 L1
P
P L1
L1 L2 L2

Interconnect
M

• Modern systems often have hierarchical caches

• Each cache has exactly one parent but can have zero
or more children
• Only a parent and its children can communicate
directly
• Inclusion property is maintained between a parent
and its children, i.e.,

a ∈ Li ⇒ a ∈ Li+1

November 9, 2005
 6.823 L17- 7
Arvind

Cache Coherence Protocols for SC

write request:
the address is invalidated (updated) in all other
caches before (after) the write is performed

read request:

if a dirty copy is found in some cache, a write-

back is performed before the memory is read

We will focus on Invalidation protocols

as opposed to Update protocols

November 9, 2005
 6.823 L17- 8
Arvind

Warmup: Parallel I/O

Memory Physical
Address (A) Bus
Memory
Proc. Data (D) Cache

R/W
Page transfers
occur while the
Processor is running
A
Either Cache or DMA can D
DMA
be the Bus Master and DISK
R/W
effect transfers

DMA stands for Direct Memory Access

November 9, 2005
 6.823 L17- 9
Arvind

Problems with Parallel I/O

Cached portions
of page Physical
Memory Memory
Bus
Proc.
Cache
DMA transfers

DMA
DISK
Memory Disk: Physical memory may be
stale if Cache copy is dirty

Disk Memory: Cache may have data

corresponding to the memory

November 9, 2005
 6.823 L17- 10
Arvind

Snoopy Cache Goodman 1983

• Idea: Have cache watch (or snoop upon)

DMA transfers, and then “do the right
thing”
• Snoopy cache tags are dual-ported

Used to drive Memory Bus

when Cache is Bus Master

A A
Tags and Snoopy read port
State attached to Memory
Proc. R/W R/W
Bus
Data
D (lines)

Cache

November 9, 2005
 6.823 L17- 11
Arvind

Snoopy Cache Actions

Observed Bus
Cycle Cache State Cache Action

Address not cached No action

Read Cycle Cached, unmodified No action

Memory Disk Cached, modified Cache intervenes

Address not cached No action

Write Cycle Cached, unmodified Cache purges its copy
Disk Memory Cached, modified ???

November 9, 2005
 6.823 L17- 12
Arvind

Shared Memory Multiprocessor

Memory

Bus

Snoopy
M1 Cache Physical
Memory
Snoopy
M2 Cache

M3 Snoopy DMA DISKS

Cache

Use snoopy mechanism to keep all

processors’ view of memory coherent

November 9, 2005
 6.823 L17- 13
Arvind

Cache State Transition Diagram

The MSI protocol

Each cache line has a tag M: Modified

S: Shared
Address tag I: Invalid
state
bits
P1 reads
Other processor reads M or writes
P1 writes back Write miss

rite
w Other processor
to
n ts intents to write
t e
in
Read
P1
miss

S I
Read by any
Other processor
processor
intents to write
Cache state in
processor P1
November 9, 2005
 6.823 L17- 14
Arvind

2 Processor Example
P1 reads
P1 reads P1 P2 reads, or writes
P1 writes P1 writes back
M
P2 reads Write miss
rite
P2 writes
tow P2 intent to write
P1 reads t
in ten
P1 writes Read P1
miss
P2 writes S I
P1 writes P2 intent to write

P2 P1 reads,
P2 reads
or writes
P2 writes back M
Write miss
e
writ
t to P1 intent to write
ten
Read in
P2
miss
S I
P1 intent to write
November 9, 2005
 6.823 L17- 15
Arvind

Observation
P1 reads

Other processor reads M or writes

P1 writes back Write miss

rite
w Other processor
to
n ts intents to write
t e
in
Read
P1
miss

S I
Read by any Other processor
processor intents to write

• If a line is in the M state then no other

cache can have a copy of the line!
– Memory stays coherent, multiple differing copies
cannot exist
November 9, 2005
 6.823 L17- 16
Arvind

MESI: An Enhanced MSI protocol

Each cache line has a tag M: Modified Exclusive

E: Exclusive, unmodified
Address tag S: Shared
state
I: Invalid
bits

P1 write P1 read
P1 write M E
or read
Write miss
it e
Other processor reads w r Other processor
P1 writes back t to intent to write
n
Read miss, te
in
shared P1
S
I
Read by any Other processor

processor intent to write

Cache state in
processor P1
November 9, 2005
 17

Five-minute break to stretch your legs

 6.823 L17- 18
Arvind

Cache Coherence State Encoding

block Address

tag indexm offset tag V M data block

Valid and dirty bits can be used

to encode S, I, and (E, M) states
V=0, D=x ⇒ Invalid Hit? word
V=1, D=0 ⇒ Shared (not dirty)
V=1, D=1 ⇒ Exclusive (dirty)

November 9, 2005
 6.823 L17- 19
Arvind

2-Level Caches
CPU CPU CPU CPU

L1 $ L1 $ L1 $ L1 $

L2 $ L2 $ L2 $ L2 $

Snooper Snooper Snooper Snooper

• Processors often have two-level caches

• Small L1 on chip, large L2 off chip

• Inclusion property: entries in L1 must be in L2
invalidation in L2 ⇒ invalidation in L1
• Snooping on L2 does not affect CPU-L1 bandwidth
What problem could occur?
November 9, 2005
 6.823 L17- 20
Arvind

Intervention

CPU-1 CPU-2

A 200 cache-1 cache-2

CPU-Memory bus

A 100 memory (stale data)

When a read-miss for A occurs in cache-2,

a read request for A is placed on the bus
• Cache-1 needs to supply & change its state to shared
• The memory may respond to the request also!
Does memory know it has stale data?
Cache-1 needs to intervene through memory
controller to supply correct data to cache-2
November 9, 2005
 6.823 L17- 21
Arvind

False Sharing

state blk addr data0 data1 ... dataN

A cache block contains more than one word

Cache-coherence is done at the block-level and

not word-level

Suppose M1 writes wordi and M2 writes wordk and

both words have the same block address.

What can happen?

November 9, 2005
 6.823 L17- 22
Arvind

Synchronization and Caches:

Performance Issues
Processor 1 Processor 2 Processor 3

R←1 R←1 R←1

L: swap(mutex, R); L: swap(mutex, R); L: swap(mutex, R);
if <R> then goto L; if <R> then goto L; if <R> then goto L;
<critical section> <critical section> <critical section>
M[mutex] ← 0; M[mutex] ← 0; M[mutex] ← 0;

cache mutex=1
cache cache

CPU-Memory Bus

Cache-coherence protocols will cause mutex to ping-pong

between P1’s and P2’s caches.

Ping-ponging can be reduced by first reading the mutex

location (non-atomically) and executing a swap only if it is
found to be zero.

November 9, 2005
 6.823 L17- 23

Performance Related to Bus

Arvind

occupancy
In general, a read-modify-write instruction

requires two memory (bus) operations without

intervening memory operations by other

processors

In a multiprocessor setting, bus needs to be

locked for the entire duration of the atomic read

and write operation

⇒ expensive for simple buses

⇒ very expensive for split-transaction buses

modern processors use

load-reserve
store-conditional
November 9, 2005
 6.823 L17- 24
Arvind

Load-reserve & Store-conditional

Special register(s) to hold reservation flag and

address, and the outcome of store-conditional

Load-reserve(R, a): Store-conditional(a, R):

<flag, adr> ← <1, a>; if <flag, adr> == <1, a>
R ← M[a]; then cancel other procs’
reservation on a;
M[a] ← <R>;
status ← succeed;
else status ← fail;
If the snooper sees a store transaction to the address
in the reserve register, the reserve bit is set to 0
• Several processors may reserve ‘a’ simultaneously
• These instructions are like ordinary loads and stores
with respect to the bus traffic

November 9, 2005
 6.823 L17- 25
Arvind

Performance:
Load-reserve & Store-conditional

The total number of memory (bus) transactions

is not necessarily reduced, but splitting an
atomic instruction into load-reserve & store-
conditional:

• increases bus utilization (and reduces

processor stall time), especially in split-
transaction buses

• reduces cache ping-pong effect because

processors trying to acquire a semaphore do
not have to perform a store each time

November 9, 2005
 6.823 L17- 26
Arvind

Out-of-Order Loads/Stores & CC

snooper
Wb-req, Inv-req, Inv-rep

load/store

buffers
pushout (Wb-rep) Memory
Cache
CPU

(I/S/E) (S-rep, E-rep)

(S-req, E-req)
Blocking caches CPU/Memory
One request at a time + CC ⇒ SC Interface
Non-blocking caches
Multiple requests (different addresses) concurrently + CC
⇒ Relaxed memory models
CC ensures that all processors observe the same
order of loads and stores to an address
November 9, 2005
 6.823 L17- 27
Arvind

next time

Designing a Cache Coherence

Protocol

November 9, 2005
 28

Thank you !

 6.823 L17- 29
Arvind

2 Processor Example

P1 write P1 read
Block b P1 write
or read
M E
Write miss
P2 reads, rite
ow
P1
P1 writes back
ten
t t P2 intent to write
Read
in
P1
miss

S I
P2 intent to write

P2 write P2 read
P2 write
Block b or read
M E
Write miss
e
P1 reads, writ
P2
P2 writes back t to P1 intent to write
ten
Read
in
P2
miss

S I
P1 intent to write
November 9, 2005