Congestion Control
and Traffic Management
in High Speed Networks
Carey Williamson
University of Calgary
1
Introduction
The goal of congestion control is to
regulate traffic flow in the network in
order to avoid saturating or
overloading intermediate nodes in the
network
2
Congestion: Effects
Congestion is undesirable because it
can cause:
Increased delay, due to queueing
within the network
Packet loss, due to buffer overflow
Reduced throughput, due to packet
loss and retransmission
Analogy: “rush hour” traffic
3
Congestion: Causes
The basic cause of congestion is that the
input traffic demands exceed the
capacity of the network
In typical packet switching networks,
this can occur quite easily when:
- output links are slower than inputs
- multiple traffic sources competing for
same output link at the same time
4
Buffering: A Solution?
Buffering in switches can help alleviate
short term or transient congestion
problems, but...
Under sustained overload, buffers will
still fill up, and packets will be lost
– only defers the congestion problem
More buffers means more queuing delay
– beyond a certain point, more buffering makes
the congestion problem worse, because of
increased delay and retransmission
5
Motivation
The congestion control problem is even
more acute in high speed networks
Faster link speeds mean that congestion
can happen faster than before
e.g., 64 kilobyte buffer
@ 64 kbps: 8.2 seconds
@ 10 Mbps: 52 milliseconds
@ 1 Gbps: 0.52 milliseconds
6
Motivation (Cont’d)
Buffer requirements increase with
link speeds
e.g., to store 1 second worth of traffic
@ 64 kbps: 8 kilobytes
@ 10 Mbps: 1.25 Mbytes
@ 1 Gbps: 125 Mbytes
7
Motivation (Cont’d)
Heterogeneity of link speeds
- just because you add new high speed
links to a network doesn’t mean that
the old low speed links go away
- interconnecting high speed and lower
speed networks creates congestion
problems at the point of interconnect
8
Motivation (Cont’d)
Traffic is bursty
- high peak-to-mean ratio, peak rates
- e.g., data traffic: 10-to-1, 1-10 Mbps
- e.g., video traffic: 20-to-1, 5-100 Mbps
- can statistically multiplex several
channels, but if too many are active at
the same time, congestion is inevitable
9
Motivation (Cont’d)
Reaction time is bounded by the
propagation delay
- in a high-speed wide-area network, the
delay x bandwidth product is HUGE!!!
- d x b tells you how many bits fit in the
“pipe” between you and the receiver
- by the time you realize that network is
congested, you may have already sent
another Mbit or more of data!!!
10
Reactive versus Preventive
There are two fundamental approaches
to congestion control: reactive
approaches and preventive approaches
Reactive: feedback-based
– attempt to detect congestion, or the onset
of congestion, and take action to resolve
the problem before things get worse
Preventive: reservation-based
– prevent congestion from ever happening in
the first place, by reserving resources
11
Reactive versus Preventive (Cont’d)
Most of the Internet approaches are
reactive schemes
– TCP Slow Start
– Random-Early-Detection (RED) Gateways
– Source Quench
The large d x b product means that
many of these approaches are not
applicable to high speed networks
Most ATM congestion control strategies
are preventive, reservation-based
12
Congestion Control in ATM
When people discuss congestion control
in the context of high speed ATM
networks, they usually distinguish
between call-level controls and
cell-level controls
13
Call-Level Control
An example of the call-level approach
to congestion control is call admission
control (to be discussed later this
semester)
Tries to prevent congestion by not
allowing new calls or connections into
the network unless the network has
sufficient capacity to support them
14
Call-Level Control (Cont’d)
At time of call setup (connection
establishment) you request the
resources that you need for the duration
of the call (e.g., bandwidth, buffers)
If available, your call proceeds
If not, your call is blocked
E.g., telephone network, busy signal
15
Call-Level Control (Cont’d)
Tradeoff: aggressive vs conservative
Want to accept enough calls to have
reasonably high network utilization,
but don’t want to accept so many calls
that you have a high probability of
network congestion (which might
compromise the QOS requirements
that you are trying to meet)
16
Call-Level Control (Cont’d)
Problems:
Can be unfair
- denial of service, long access delay
Hard to specify resource requirements
and QOS parameters precisely
- may not know, or may cheat
- congestion can still occur
17
Cell-Level Control
Also called input rate control
Control the input rate of traffic sources
to prevent, reduce, or control the level of
congestion
Many possible mechanisms:
Traffic shaping, traffic policing, UPC
Leaky bucket (token bucket)
Cell tagging (colouring), cell discarding
Cell scheduling disciplines
18
Congestion Control in ATM
There is actually a complete spectrum
of traffic control functions, ranging
from the very short-term (e.g., traffic
shaping, cell discarding) to the very
long-term (e.g., network provisioning)
See [Gilbert et al 1991]
19
ATM Traffic Control Schemes
Time Scale
20
ATM Traffic Control Schemes
Short
Term
usec
Time Scale
21
ATM Traffic Control Schemes
Long
Term
Months, years
Short
Term
usec
Time Scale
22
ATM Traffic Control Schemes
Long Resource Provisioning
Term
Admission Control
Call Routing, Load Balancing
Duration
Explicit Congestion Notification
Fast Reservation Protocol
Propagation
Node to Node Flow Control
Delay Time
Usage Parameter Control
Priority Control
Cell Time
Traffic Shaping
Time Scale Cell Discarding
23
ATM Traffic Control Schemes
Usage Parameter Control
Priority Control
Cell Time
Traffic Shaping
Time Scale Cell Discarding
24
ATM Traffic Control Schemes
Explicit Congestion Notification
Fast Reservation Protocol
Propagation
Node to Node Flow Control
Delay Time
Time Scale
25
ATM Traffic Control Schemes
Admission Control
Call Routing, Load Balancing
Duration
Time Scale
26
ATM Traffic Control Schemes
Long Resource Provisioning
Term
Time Scale
27
ATM Traffic Control Schemes
Long Resource Provisioning
Term
Admission Control
Call Routing, Load Balancing
Duration
Explicit Congestion Notification
Fast Reservation Protocol
Propagation
Node to Node Flow Control
Delay Time
Usage Parameter Control
Priority Control
Cell Time
Traffic Shaping
Time Scale Cell Discarding
28
ATM Traffic Control Schemes
Preventive controls:
Resource provisioning
Connection admission control
Call routing and load balancing
Usage parameter control
Priority control
Traffic shaping
Fast reservation protocol
29
ATM Traffic Control Schemes
Reactive controls:
Adaptive admission control
Call routing and load balancing
Adaptive usage parameter control
Explicit congestion notification
(forward or backward)
Node to node flow control
Selective cell discarding
30
Leaky Bucket
One of the cell-level control
mechanisms that has been proposed is
the leaky bucket (a.k.a. token bucket)
Has been proposed as a traffic policing
mechanism for Usage Parameter
Control (UPC), to check conformance of
a source to its traffic descriptor
Can also be used as a traffic shaper
31
Leaky Bucket (Cont’d)
Think of a bucket (pail) with a small
hole in the bottom
You fill the bucket with water
Water drips out the bottom at a nice
constant rate: drip, drip, drip...
32
Leaky Bucket (Cont’d)
33
Leaky Bucket (Cont’d)
Bucket
34
Leaky Bucket (Cont’d)
Empty
Bucket
35
Leaky Bucket (Cont’d)
Bucket
Hole
36
Leaky Bucket (Cont’d)
Water
Bucket
Hole
37
Leaky Bucket (Cont’d)
38
Leaky Bucket (Cont’d)
Drip
39
Leaky Bucket (Cont’d)
40
Leaky Bucket (Cont’d)
41
Leaky Bucket (Cont’d)
42
Leaky Bucket (Cont’d)
Constant
rate stream of
drips, all nicely
spaced, periodic
43
Leaky Bucket (Cont’d)
Storage area
for drips
waiting to go
Constant
rate stream of
drips, all nicely
spaced, periodic
44
Leaky Bucket (Cont’d)
A leaky bucket flow control mechanism
is then a software realization of this
very simple idea
Packets (cells) waiting for transmission
arrive according to some (perhaps
unknown) arrival distribution
Tokens arrive periodically
(deterministically)
Cell must have a token to enter network
45
Leaky Bucket (Cont’d)
Incoming Tokens
at rate r tokens/sec
Incoming Cells
(generated by
To
traffic source
with rate X)
+ Network
46
Leaky Bucket (Cont’d)
Incoming Tokens
at rate r tokens/sec
Incoming Cells
To
5 1
+ Network
4 32
47
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
To
5 4 32 1
+ Network
48
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
To
5 43 2
+ 1 Network
49
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
To
5 4 3
+ 2 1 Network
50
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
To
5 4
+ 3 2 1 Network
51
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
5
X
4
+ 3 2 1
To
Network
52
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
XX + 3 2 1
To
Network
5 4
53
Leaky Bucket (Cont’d)
Incoming Tokens
Incoming Cells
XX + 3 2 1
To
Network
54
Leaky Bucket (Cont’d)
Cells must obtain tokens in order to
proceed into the network
If no token available, then cell is
discarded
Constrains the rate at which cells enter
the network to be the rate negotiated at
the time of call setup
Shapes traffic, reduces burstiness
55
Buffered Leaky Bucket
Arriving cells that find a token waiting
can proceed directly into the network
Arriving cells that find no token ready
must wait in queue for a token
Cells that arrive to a full queue are lost
Tokens that arrive to a full token pool
are simply discarded
56
Buffered Leaky Bucket
Incoming Tokens
at rate r tokens/sec
Pool of at most M
waiting tokens
Incoming Cells
To
+ Network
Queue of
at most B
waiting cells
57
Buffered Leaky Bucket (Cont’d)
Incoming cell rate: X
Token rate: r
If X > r, then cells wait in buffer until
tokens are available
Output traffic is r cells/sec, nicely
paced
If X < r, then tokens always ready
Output traffic is X (< r)
Use for traffic shaping or UPC
58
Buffered Leaky Bucket (Cont’d)
A station can “save up” at most M
tokens
Limits the maximum burst size in the
network
Can send at most M cells back to back
B can be set to balance the tradeoff
between cell loss and cell delay
59
Leaky Bucket UPC
The token rate r is set based on the rate
declared at the time of call setup
Makes sure that each source obeys rate
that was used when the call admission
decision was made (i.e., descriptor)
Can use “single leaky bucket” to police
just the peak cell rate (PCR)
Can use “dual leaky bucket” to police
both PCR and SCR
60
Variations
There are several different variations of
the basic leaky bucket concept
described in the literature, such as the
virtual leaky bucket, spacer, others
Basic idea: rather than strictly enforcing
rates, allow senders to occasionally
exceed their prescribed rate, as long as
they mark or tag their extra cells
61
Cell Marking Scheme
Uses leaky bucket to regulate cell
transmissions as before, but rather than
having cells wait for tokens when there
are no tokens ready, the station can
transmit the cell and mark it as a
violation cell (i.e., cell colouring)
Green (CLP = 0) for cells that obey rate
Red (CLP = 1) for cells that don’t
62
Cell Colouring (Cont’d)
If the network detects congestion at any
point, then it does not hesitate to throw
away red cells (CLP = 1), but always
gives preference to green cells
Green cells must get through
Red cells get through only if there is
spare capacity in the network
“No harm in trying!” principle
63
Selective Cell Discard (SCD)
A cell-level control mechanism in ATM
switches called selective cell discard
can be implemented quite easily using
a CLP threshold on each queue/buffer
Below the threshold, can accept both
green and red cells
Beyond the threshold, can only accept
green cells
64
Selective Cell Discard (Cont’d)
Buffer in an ATM switch
65
Selective Cell Discard (Cont’d)
Buffer in an ATM switch
66
Selective Cell Discard (Cont’d)
Some cells
waiting to go
Buffer in an ATM switch
67
Selective Cell Discard (Cont’d)
CLP
Threshold
Buffer in an ATM switch
68
Selective Cell Discard (Cont’d)
CLP
Threshold
69
Selective Cell Discard (Cont’d)
CLP
Threshold
70
Selective Cell Discard (Cont’d)
CLP
Threshold
71
Selective Cell Discard (Cont’d)
CLP
Threshold
72
Selective Cell Discard (Cont’d)
CLP
Threshold
73
Selective Cell Discard (Cont’d)
CLP
Threshold
74
Selective Cell Discard (Cont’d)
CLP
Threshold
75
Selective Cell Discard (Cont’d)
CLP
Threshold
76
Selective Cell Discard (Cont’d)
CLP
Threshold
77
Selective Cell Discard (Cont’d)
CLP
Threshold
78
Selective Cell Discard (Cont’d)
CLP
Threshold
79
Selective Cell Discard (Cont’d)
CLP
Threshold
80
Selective Cell Discard (Cont’d)
CLP
Threshold
81
Selective Cell Discard (Cont’d)
CLP
Threshold
82
Selective Cell Discard (Cont’d)
CLP
Threshold
83
Selective Cell Discard (Cont’d)
CLP
Threshold
84
Selective Cell Discard (Cont’d)
CLP
Threshold
85
Selective Cell Discard (Cont’d)
CLP
Threshold
86
Selective Cell Discard (Cont’d)
CLP
Threshold
87
Selective Cell Discard (Cont’d)
CLP
Threshold
88
Explicit Congestion Notification
There are some proposals to use
reactive congestion control approaches
for end-to-end flow control in ATM
One of the mechanisms proposed is
called Explicit Forward Congestion
Notification (EFCN) (or EFCI, for
Explicit Forward Congestion Indication)
89
EFCI: Basic Operation
Switches can detect the onset of
congestion (e.g., buffers filling up)
Switches set a control bit in cell headers
to indicate this congestion condition
Sources react by reducing the volume
of traffic that they are sending through
that switch
Suitable for VBR or ABR traffic
90
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
91
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
Buffer
92
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
Unoccupied Occupied
93
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
94
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
Data
Cell
EFCI Threshold
95
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
96
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
97
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
98
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
99
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
100
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
101
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
102
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
!!!
EFCI Threshold
103
EFCI: Basic Operation (Cont’d)
Ack
Cell
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
104
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
105
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
106
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
107
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
!!!
EFCI Threshold
108
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
109
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
110
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
111
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
112
EFCI: Basic Operation (Cont’d)
Traffic Traffic
Source Sink
Switch Switch Switch
EFCI Threshold
113
EFCI Issues
How to set EFCI threshold
What should sources do when EFCI
signal is seen
What should sources do when no EFCI
signal is seen
Forward versus backward notification
Effect of feedback delay
Delay x bandwidth product
114
Summary
Congestion control in high speed ATM
networks is a difficult problem
Lots of good ideas of how to do it, but
no real standard (yet?)
Will likely require a combination of
schemes at different time scales and for
different classes of traffic
Lots more remains to be done
115