Pipelining and Vector Processing 1
PIPELINING
•
• Pipelining
• Instruction Pipeline
•Hazards
• RISC Pipeline
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 2 Pipelining
PIPELINING
A technique of decomposing a sequential process
into suboperations, with each subprocess being
executed in a partial dedicated segment that
operates concurrently with all other segments.
Ai * Bi + Ci for i = 1, 2, 3, ... , 7
Ai Bi Memory Ci
Segment 1
R1 R2
Multiplier
Segment 2
R3 R4
Adder
Segment 3
R5
R1 Ai, R2 Bi Load Ai and Bi
R3 R1 * R2, R4 Ci Multiply and load Ci
R5 R3 + R4 Add
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 3 Pipelining
OPERATIONS IN EACH PIPELINE STAGE
Clock Segment 1 Segment 2 Segment 3
Pulse
Number R1 R2 R3 R4 R5
1 A1 B1
2 A2 B2 A1 * B1 C1
3 A3 B3 A2 * B2 C2 A1 * B1 + C1
4 A4 B4 A3 * B3 C3 A2 * B2 + C2
5 A5 B5 A4 * B4 C4 A3 * B3 + C3
6 A6 B6 A5 * B5 C5 A4 * B4 + C4
7 A7 B7 A6 * B6 C6 A5 * B5 + C5
8 A7 * B7 C7 A6 * B6 + C6
9 A7 * B7 + C7
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 4 Pipelining
GENERAL PIPELINE
General Structure of a 4-Segment Pipeline
Clock
Input S1 R1 S2 R2 S3 R3 S4 R4
Space-Time Diagram
1 2 3 4 5 6 7 8 9 Clock cycles
Segment 1 T1 T2 T3 T4 T5 T6
2 T1 T2 T3 T4 T5 T6
3 T1 T2 T3 T4 T5 T6
4 T1 T2 T3 T4 T5 T6
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 5 Pipelining
PIPELINE SPEEDUP
n: Number of tasks to be performed
Conventional Machine (Non-Pipelined)
tn: Clock cycle
: Time required to complete the n tasks
= n * t n
Pipelined Machine (k stages)
tp: Clock cycle (time to complete each suboperation)
: Time required to complete the n tasks
= (k + n - 1) * tp
Speedup
Sk: Speedup
Sk = n*tn / (k + n - 1)*tp
tn
lim Sk = ( = k, if tn = k * tp )
n tp
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 6 Pipelining
PIPELINE AND MULTIPLE FUNCTION UNITS
Example
- 4-stage pipeline
- subopertion in each stage; tp = 20nS
- 100 tasks to be executed
- 1 task in non-pipelined system; 20*4 = 80nS
Pipelined System
(k + n - 1)*tp = (4 + 99) * 20 = 2060nS
Non-Pipelined System
n*k*tp = 100 * 80 = 8000nS
Speedup
Sk = 8000 / 2060 = 3.88
4-Stage Pipeline is basically identical to the system
Ii Ii+1 I i+2 I i+3
with 4 identical function units
Multiple Functional Units P1 P2 P3 P4
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 7 Instruction Pipeline
INSTRUCTION CYCLE
Six Phases* in an Instruction Cycle
[1] Fetch an instruction from memory
[2] Decode the instruction
[3] Calculate the effective address of the operand
[4] Fetch the operands from memory
[5] Execute the operation
[6] Store the result in the proper place
* Some instructions skip some phases
* Effective address calculation can be done in
the part of the decoding phase
* Storage of the operation result into a register
is done automatically in the execution phase
==> 4-Stage Pipeline
[1] FI: Fetch an instruction from memory
[2] DA: Decode the instruction and calculate
the effective address of the operand
[3] FO: Fetch the operand
[4] EX: Execute the operation
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 8 Instruction Pipeline
INSTRUCTION PIPELINE
Execution of Three Instructions in a 4-Stage Pipeline
Conventional
i FI DA FO EX
i+1 FI DA FO EX
i+2 FI DA FO EX
Pipelined
i FI DA FO EX
i+1 FI DA FO EX
i+2 FI DA FO EX
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 9 Instruction Pipeline
INSTRUCTION EXECUTION IN A 4-STAGE PIPELINE
Segment1: Fetch instruction
from memory
Decode instruction
Segment2: and calculate
effective address
yes Branch?
no
Segment3: Fetch operand
from memory
Segment4: Execute instruction
Interrupt yes
Interrupt?
handling
no
Update PC
Empty pipe
Step: 1 2 3 4 5 6 7 8 9 10 11 12 13
Instruction 1 FI DA FO EX
2 FI DA FO EX
(Branch) 3 FI DA FO EX
4 FI FI DA FO EX
5 FI DA FO EX
6 FI DA FO EX
7 FI DA FO EX
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 10 Instruction Pipeline
MAJOR HAZARDS IN PIPELINED EXECUTION
Structural hazards(Resource Conflicts)
Hardware Resources required by the instructions in
simultaneous overlapped execution cannot be met
Data hazards (Data Dependency Conflicts)
An instruction scheduled to be executed in the pipeline requires the
result of a previous instruction, which is not yet available
R1 <- B + C ADD DA B,C + Data dependency
R1 <- R1 + 1
INC DA bubble R1 +1
Control hazards
Branches and other instructions that change the PC
make the fetch of the next instruction to be delayed
JMP ID PC + PC Branch address dependency
bubble IF ID OF OE OS
Hazards in pipelines may make it Pipeline Interlock:
necessary to stall the pipeline Detect Hazards Stall until it is cleared
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 11 Instruction Pipeline
STRUCTURAL HAZARDS
Structural Hazards
Occur when some resource has not been
duplicated enough to allow all combinations
of instructions in the pipeline to execute
Example: With one memory-port, a data and an instruction fetch
cannot be initiated in the same clock
i FI DA FO EX
i+1 FI DA FO EX
i+2 stall stall FI DA FO EX
The Pipeline is stalled for a structural hazard
<- Two Loads with one port memory
-> Two-port memory will serve without stall
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 12
1. Structural Hazards:
= pipeline hazards due to hardware resource conflicts
Example Instruction fetch and data access
A processor instruction = [operator] + [parameters]
instruction fetch
processor instruction: sw $t1, 0($t2) // $t1 Mem[$t2+0]
save register to memory
Execution of this instruction requires two memory accesses
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 13
1. Structural Hazards (continued)
Unified Cache Architecture Split Cache Architecture
Main Memory CPU Main Memory CPU
Instruction
Cache Cache
Memory
CPU Core CPU Core
Data
Cache
Instruction code and data Instruction code and data
need to take the same path take its own path
(i486) (Pentium and later)
Instruction code
Data
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 14
Unified Cache Architecture
Main Memory CPU
Cache Instruction code
Memory
CPU Core Datais accessed
Memory address
and its content is loaded to
this processor
lw $t1, 0($t2) IF ID EX ME WB // Mem[$t2+0] $t1
instruction1 IF ID EX ME WB
instruction2 IF ID EX ME WB
instruction3 IF ID EX ME WB
These two memory-access IF ID EX ME WB
operations can not happen
at the same time Instruction3 is now
“fetched” from memory
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 15 Instruction Pipeline
DATA HAZARDS
Data Hazards
Occurs when the execution of an instruction
depends on the results of a previous instruction
ADD R1, R2, R3
SUB R4, R1, R5
Data hazard can be dealt with either hardware
techniques or software technique
Hardware Technique
Interlock
- hardware detects the data dependencies and delays the scheduling
of the dependent instruction by stalling enough clock cycles
Forwarding (bypassing, short-circuiting)
- Accomplished by a data path that routes a value from a source
(usually an ALU) to a user, bypassing a designated register. This
allows the value to be produced to be used at an earlier stage in the
pipeline than would otherwise be possible
Software Technique
Instruction Scheduling(compiler) for delayed load
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 16
2. Data Hazards:
= pipeline hazards due data-dependency between instructions
Example The same registers are used in more than
one instruction
move $t1, $t2 // t1 t2
add $t3, $t1, 5 // t3 t1 + 5
Data Dependency
This can not be done
move $t1, $t2: IF ID EX ME WB
add $t3, $t1, 5: IF ID EX ME WB
Time
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 17
Data Dependency
move $t1, $t2: IF ID EX ME WB
add $t3, $t1, 5: IF ID EX ME WB
Resolve data dependency by stalls
move $t1, $t2: IF ID EX ME WB
add $t3, $t1, 5: IF ID EX ME WB
Pipeline stalls
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 18
Data Hazards
Execution Order is:
InstrI Read After Write (RAW)
InstrJ
InstrJ tries to read operand before InstrI writes it
I: add r1,r2,r3
J: sub r4,r1,r3
• Known as true dependency.
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 19
Data Hazards
Write After Read (WAR)
• i1. R4 <- R1 + R5
i2. R5 <- R1 + R2
• In any situation with a chance that i2 may finish before
i1 (i.e., with concurrent execution), it must be ensured
that the result of register R5 is not stored before i1 has
had a chance to fetch the operands.
– Called an “anti-dependence”.
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 20
Data Hazards
Execution Order is: Write After Write (WAW)
InstrI
InstrJ
InstrJ tries to write operand before InstrI
writes it
– Leaves wrong result ( InstrI not InstrJ )
I: sub r1,r4,r3
J: add r1,r2,r3
K: mul r6,r1,r7
• Called an “output dependence”.
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 21 Instruction Pipeline
FORWARDING HARDWARE
Example: Register
file
ADD R1, R2, R3
SUB R4, R1, R5
3-stage Pipeline MUX MUX Bypass
path
Result
I: Instruction Fetch write bus
A: Decode, Read Registers, ALU
ALU Operations
E: Write the result to the
destination register R4
ALU result buffer
ADD I A E
SUB I A E Without Bypassing
SUB I A E With Bypassing
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 22 Instruction Pipeline
INSTRUCTION SCHEDULING
a = b + c;
d = e - f;
Unscheduled code: Scheduled Code:
LW Rb, b LW Rb, b
LW Rc, c LW Rc, c
ADD Ra, Rb, Rc LW Re, e
SW a, Ra ADD Ra, Rb, Rc
LW Re, e LW Rf, f
LW Rf, f SW a, Ra
SUB Rd, Re, Rf SUB Rd, Re, Rf
SW d, Rd SW d, Rd
Delayed Load
A load requiring that the following instruction not use its result
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 23 Instruction Pipeline
CONTROL HAZARDS
Branch Instructions
- Branch target address is not known until
the branch instruction is completed
Branch
FI DA FO EX
Instruction
Next FI DA FO EX
Instruction
Target address available
- Stall -> waste of cycle times
Dealing with Control Hazards
* Prefetch Target Instruction
* Branch Target Buffer
* Loop Buffer
* Branch Prediction
* Delayed Branch
Computer Organization Computer Architectures Lab
Pipelining and Vector Processing 24 Instruction Pipeline
CONTROL HAZARDS
Prefetch Target Instruction
– Fetch instructions in both streams, branch not taken and branch taken
– Both are saved until branch branch is executed. Then, select the right
instruction stream and discard the wrong stream
Branch Target Buffer(BTB; Associative Memory)
– Entry: Addr of previously executed branches; Target instruction
and the next few instructions
– When fetching an instruction, search BTB.
– If found, fetch the instruction stream in BTB;
– If not, new stream is fetched and update BTB
Loop Buffer(High Speed Register file)
– Storage of entire loop that allows to execute a loop without accessing memory
Branch Prediction
– Guessing the branch condition, and fetch an instruction stream based on
the guess. Correct guess eliminates the branch penalty
Delayed Branch
– Compiler detects the branch and rearranges the instruction sequence
by inserting useful instructions that keep the pipeline busy
in the presence of a branch instruction
Computer Organization Computer Architectures Lab
