Parallel Computing and Programming.
Lecture 1: Von Neumann vs Dataflow Models
Dr. Rony Kassam
IEF Tishreen Uni
S1 - 2021
Index
n Von Neumann vs Dataflow Models.
n ISA vs Microarchitecture.
n Single-cycle vs Multi-cycle Microarchitectures.
n Instruction Level Parallelism: Pipelining Intro.
n Instruction Level Parallelism: Issues in Pipeline Design.
n Thread Level Parallelism: Data Dependence Solutions.
n Thread Level Parallelism: Shared Memory and OpenMP.
2
Recall: The Von Neumann Model
MEMORY
Mem Addr Reg
Mem Data Reg
INPUT PROCESSING UNIT OUTPUT
Keyboard, Monitor,
Mouse, ALU TEMP Printer,
Disk… Disk…
CONTROL UNIT
PC or IP Inst Register
3
Recall: The Instruction Cycle
q FETCH
q DECODE
q EVALUATE ADDRESS
q FETCH OPERANDS
q EXECUTE
q STORE RESULT
4
Recall: The Instruction Set Architecture
n The ISA is the interface between what the software commands
and what the hardware carries out
Problem
n The ISA specifies
q The memory organization Algorithm
n Address space (LC-3: 216, MIPS: 232) Program
n Addressability (LC-3: 16 bits, MIPS: 32 bits)
ISA
n Word- or Byte-addressable
Microarchitecture
q The register set Circuits
n R0 to R7 in LC-3 Electrons
n 32 registers in MIPS
q The instruction set
n Opcodes
n Data types
n Addressing modes
n Semantics of instructions
5
Microarchitecture
n An implementation of the ISA
6
Microarchitecture
n An implementation of the ISA
n How do we implement the ISA?
6
Microarchitecture
n An implementation of the ISA
n How do we implement the ISA?
n There can be many implementations of the same ISA
6
Microarchitecture
n An implementation of the ISA
n How do we implement the ISA?
n There can be many implementations of the same ISA
q MIPS R2000, R10000, …
6
Microarchitecture
n An implementation of the ISA
n How do we implement the ISA?
n There can be many implementations of the same ISA
q MIPS R2000, R10000, …
q Intel 80486, Pentium, Pentium Pro, Pentium 4, Kaby Lake,
Coffee Lake, … AMD K5, K7, K9, Bulldozer, BobCat, …
6
The Von Neumann Model/Architecture
n Von Neumann model is also called stored program computer
(instructions in memory). It has two key properties:
7
The Von Neumann Model/Architecture
n Von Neumann model is also called stored program computer
(instructions in memory). It has two key properties:
n Stored program
n Sequential instruction processing
7
The Von Neumann Model/Architecture
n Von Neumann model is also called stored program computer
(instructions in memory). It has two key properties:
n Stored program
q Instructions stored in a linear memory array
q Memory is unified between instructions and data
n The interpretation of a stored value depends on the control signals
n Sequential instruction processing
7
The Von Neumann Model/Architecture
n Von Neumann model is also called stored program computer
(instructions in memory). It has two key properties:
n Stored program
q Instructions stored in a linear memory array
q Memory is unified between instructions and data
n The interpretation of a stored value depends on the control signals
When is a value interpreted as an instruction?
n Sequential instruction processing
7
The Von Neumann Model/Architecture
n Von Neumann model is also called stored program computer
(instructions in memory). It has two key properties:
n Stored program
q Instructions stored in a linear memory array
q Memory is unified between instructions and data
n The interpretation of a stored value depends on the control signals
When is a value interpreted as an instruction?
n Sequential instruction processing
q One instruction processed (fetched, executed, completed) at a time
q Program counter (instruction pointer) identifies the current instruction
q Program counter is advanced sequentially except for control transfer
instructions
7
The Von Neumann Model (of a Computer)
MEMORY
Mem Addr Reg
Mem Data Reg
PROCESSING UNIT
INPUT OUTPUT
ALU TEMP
CONTROL UNIT
IP Inst Register
8
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
n Dataflow model: An instruction is fetched and executed in
data flow order
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
n Dataflow model: An instruction is fetched and executed in
data flow order
q i.e., when its operands are ready
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
n Dataflow model: An instruction is fetched and executed in
data flow order
q i.e., when its operands are ready
q i.e., there is no instruction pointer
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
n Dataflow model: An instruction is fetched and executed in
data flow order
q i.e., when its operands are ready
q i.e., there is no instruction pointer
q Instruction ordering specified by data flow dependence
n Each instruction specifies “who” should receive the result
n An instruction can “fire” whenever all operands are received
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
n Dataflow model: An instruction is fetched and executed in
data flow order
q i.e., when its operands are ready
q i.e., there is no instruction pointer
q Instruction ordering specified by data flow dependence
n Each instruction specifies “who” should receive the result
n An instruction can “fire” whenever all operands are received
q Potentially many instructions can execute at the same time
9
The Dataflow Model (of a Computer)
n Von Neumann model: An instruction is fetched and
executed in control flow order
q As specified by the instruction pointer
q Sequential unless explicit control flow instruction
n Dataflow model: An instruction is fetched and executed in
data flow order
q i.e., when its operands are ready
q i.e., there is no instruction pointer
q Instruction ordering specified by data flow dependence
n Each instruction specifies “who” should receive the result
n An instruction can “fire” whenever all operands are received
q Potentially many instructions can execute at the same time
n Inherently more parallel
9
Von Neumann vs Dataflow
n Consider a Von Neumann program
10
Von Neumann vs Dataflow
n Consider a Von Neumann program
v <= a + b;
w <= b * 2;
x <= v - w
y <= v + w
z <= x * y
Sequential
10
Von Neumann vs Dataflow
n Consider a Von Neumann program
q What is the significance of the program order?
v <= a + b;
w <= b * 2;
x <= v - w
y <= v + w
z <= x * y
Sequential
10
Von Neumann vs Dataflow
n Consider a Von Neumann program
q What is the significance of the program order?
q What is the significance of the storage locations?
v <= a + b;
w <= b * 2;
x <= v - w
y <= v + w
z <= x * y
Sequential
10
Von Neumann vs Dataflow
n Consider a Von Neumann program
q What is the significance of the program order?
q What is the significance of the storage locations?
a b
v <= a + b;
w <= b * 2;
x <= v - w + *2
y <= v + w
z <= x * y
- +
Sequential
*
Dataflow
10
Von Neumann vs Dataflow
n Consider a Von Neumann program
q What is the significance of the program order?
q What is the significance of the storage locations?
a b
v <= a + b;
w <= b * 2;
x <= v - w + *2
y <= v + w
z <= x * y
- +
Sequential
*
Dataflow
z
n Which model is more natural to you as a programmer?
10
More on Data Flow
n In a data flow machine, a program consists of data flow
nodes
q A data flow node fires (fetched and executed) when all it
inputs are ready
n i.e. when all inputs have tokens
n Data flow node and its ISA representation
11
Data Flow Nodes
12
An Example Data Flow Program
13
ISA-level Tradeoff: Instruction Pointer
n Do we need an instruction pointer in the ISA?
14
ISA-level Tradeoff: Instruction Pointer
n Do we need an instruction pointer in the ISA?
q Yes: Control-driven, sequential execution
n An instruction is executed when the IP points to it
n IP automatically changes sequentially (except for control flow
instructions)
14
ISA-level Tradeoff: Instruction Pointer
n Do we need an instruction pointer in the ISA?
q Yes: Control-driven, sequential execution
n An instruction is executed when the IP points to it
n IP automatically changes sequentially (except for control flow
instructions)
q No: Data-driven, parallel execution
n An instruction is executed when all its operand values are
available (data flow)
14
ISA-level Tradeoff: Instruction Pointer
n Do we need an instruction pointer in the ISA?
q Yes: Control-driven, sequential execution
n An instruction is executed when the IP points to it
n IP automatically changes sequentially (except for control flow
instructions)
q No: Data-driven, parallel execution
n An instruction is executed when all its operand values are
available (data flow)
n Tradeoffs: MANY high-level ones
14
ISA-level Tradeoff: Instruction Pointer
n Do we need an instruction pointer in the ISA?
q Yes: Control-driven, sequential execution
n An instruction is executed when the IP points to it
n IP automatically changes sequentially (except for control flow
instructions)
q No: Data-driven, parallel execution
n An instruction is executed when all its operand values are
available (data flow)
n Tradeoffs: MANY high-level ones
q Ease of programming (for average programmers)?
q Ease of compilation?
q Performance: Extraction of parallelism?
q Hardware complexity?
14
ISA vs. Microarchitecture Level Tradeoff
n A similar tradeoff (control vs. data-driven execution) can be
made at the microarchitecture level
15
ISA vs. Microarchitecture Level Tradeoff
n A similar tradeoff (control vs. data-driven execution) can be
made at the microarchitecture level
n ISA: Specifies how the programmer sees the instructions to
be executed
15
ISA vs. Microarchitecture Level Tradeoff
n A similar tradeoff (control vs. data-driven execution) can be
made at the microarchitecture level
n ISA: Specifies how the programmer sees the instructions to
be executed
n Microarchitecture: How the underlying implementation
actually executes instructions
15
ISA vs. Microarchitecture Level Tradeoff
n A similar tradeoff (control vs. data-driven execution) can be
made at the microarchitecture level
n ISA: Specifies how the programmer sees the instructions to
be executed
q Programmer sees a sequential, control-flow execution order vs.
q Programmer sees a data-flow execution order
n Microarchitecture: How the underlying implementation
actually executes instructions
15
ISA vs. Microarchitecture Level Tradeoff
n A similar tradeoff (control vs. data-driven execution) can be
made at the microarchitecture level
n ISA: Specifies how the programmer sees the instructions to
be executed
q Programmer sees a sequential, control-flow execution order vs.
q Programmer sees a data-flow execution order
n Microarchitecture: How the underlying implementation
actually executes instructions
q Microarchitecture can execute instructions in any order as long
as it obeys the semantics specified by the ISA when making the
instruction results visible to software
n Programmer should see the order specified by the ISA
15
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
q Pipelined instruction execution: Intel 80486 uarch
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
q Pipelined instruction execution: Intel 80486 uarch
q Multiple instructions at a time: Intel Pentium uarch
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
q Pipelined instruction execution: Intel 80486 uarch
q Multiple instructions at a time: Intel Pentium uarch
q Out-of-order execution: Intel Pentium Pro uarch
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
q Pipelined instruction execution: Intel 80486 uarch
q Multiple instructions at a time: Intel Pentium uarch
q Out-of-order execution: Intel Pentium Pro uarch
q Separate instruction and data caches
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
q Pipelined instruction execution: Intel 80486 uarch
q Multiple instructions at a time: Intel Pentium uarch
q Out-of-order execution: Intel Pentium Pro uarch
q Separate instruction and data caches
n But, what happens underneath that is not consistent with
the von Neumann model is not exposed to software
16
The Von-Neumann Model
n All major instruction set architectures today use this model
q x86, ARM, MIPS, SPARC, Alpha, POWER, RISC-V, …
n Underneath (at the microarchitecture level), the execution
model of almost all implementations (or, microarchitectures)
is very different
q Pipelined instruction execution: Intel 80486 uarch
q Multiple instructions at a time: Intel Pentium uarch
q Out-of-order execution: Intel Pentium Pro uarch
q Separate instruction and data caches
n But, what happens underneath that is not consistent with
the von Neumann model is not exposed to software
q Difference between ISA and microarchitecture
16
