CSCI 4470

Algorithms

Chapter 22

Lecture slides are adapted from the CSCI

4470 course taught by Dr Shelby Funk and
the slides provided by the publisher
 SINGLE SOURCE
SHORTEST PATH
Given a directed graph G=(V,E) with edge weights
w(u,v), and some vertex s in V, we want to find a
subset G’=(V’,E’) such that

V’ is the set of vertices reachable from s

For all v in V’, the unique simple path from s to
v is a minimum weight path from s to v in G

We call a minimum weight path a shortest
path
If all edge weights are 1, this problem is solved by
BFS
 EXAMPLE
Find a shortest path graph starting at s
t x
7

3
4
3 1 2 7
s
3
5
4
y z
 SOME NOTATION

The graph G’ has

Let p = <v0,v1,
Let u and v be any w(p) = δ(s,v) for
…,vk> be any path two vertices in G every path p from
in G s to v in V’

Denoted v0 ↝p vk 
If there exists a 
G’ will be a tree

w(p) is the path from u to v
weight of p in G

Σi=1..kw(vi-1,vi)

δ(u,v) =
min{w(p)|
u↝pv}

Otherwise

δ(u,v) = ∞
OPTIMAL SUBSTRUCTURE
Lemma: Proof:
Given a weighted Decompose p into p0i, pij,
directed graph G = (V,E). pjk.
Let p = <v0,v2,…,vk> be a
shortest path from v0 to vk and Assume pij is not a shortest
for any i,j such that 1 ≤ i ≤ j path and let pij’ be a shorter
≤ k. Let pij be the subpath of path.
p from vi to vj.
Derive a contradiction.
Then pij is a shortest path
from vi to vj.
NEGATIVE WEIGHT EDGES AND
CYCLES

G can have negative weight edges


If G has negative weight edges but no
negative weight cycles, the shortest
path is well defined


If G has any negative weight cycles, the
shortes path is no longer well defined
 ALGORITHMS
Bellman-Ford SSSP in DAGs Dijkstra

Allows negative 
Applies only to 
No negative
edge weights directed acyclic weight edges

θ(V + E) time

O(VE) running graphs 
O(V2) running
time 
time (can be

Reports error if 
Negative edge improved for
G contains any weights allowed sparse graphs)
negative weight 
Application:
cycles Difference
constraints
All these algorithms share two subroutines and
rely on several properties.
 INITIALIZE-SINGLE-
SOURCE(G,S)
1. //initialize .d and .π
2. for each vertex v in V
3. v.d = ∞
4. v.π = NIL
5. s.d = 0
 RELAX(U,V,W)
1. //update v.d and v.π if shorter path exists
2. if v.d > u.d + w(u,v)
3. v.d = u.d + w(u,v)
4. u v. π = u u
v v
5 5
3 12 3 4

u v u v
5 8
5 4
3 3
 PROPERTIES OF SHORTEST PATHS
AND RELAXATION

Triangle 
For any edge (u,v) in E, δ(s,v) ≤ δ(s,u)+w(u,v)
inequality

Upper 
Let v.d be the shortest path estimate during
computation
bound 


v.d ≥ δ(s,v) always, and
Once v.d = δ(s,v), the value of v.d is never
property changed

These properties are not proven in slides – see text Section 24.5
Most follow pretty directly from the Relax algorithm and from other properties
 PROPERTIES OF SHORTEST PATHS
AND RELAXATION (CONT.)

No-path 
If there is no path from s to v then v.d
= δ(s,v) = ∞ always
property


If s ↝u → v is a shortest path for some
Convergenc u, v in V and if u.d = δ(s,u) at any time
prior to relaxing edge (u,v), then v.d =
e property δ(s,v) at all times afterward

These properties are not proven in slides – see text Section 24.5
Most follow pretty directly from the Relax algorithm and from other properties
 PROPERTIES (CONT.)
Path 
If p = <v0,v1,…,vk> is a shortest path from v0 to vk
and the edges of p are relaxed in the order (v0,v1),

relaxation (v1,v2),…,(vk-1,vk), then vk.d = δ(s,vk)


This property holds regardless of any other
relaxation steps that occur even if they are
property intermixed with relaxations of the edges of p

Predecess 
Once v.d = δ(s,v) for all v in V, the predecessor
or subgraph is a shortest-paths tree rooted at s

Because of this property, correctness proofs in
subgraph slides focus on proving that v.d = δ(s,v) ∀ v ∈ V

Predecessor subgraph follows automatically

property
 THE BELLMAN-FORD
ALGORITHM

Returns TRUE if no
Allows negative negative-weight
weight edges. cycles, FALSE
otherwise.
 BELLMAN-FORD(G,W,S)
1. INITIALIZE-SINGLE-SOURCE(G,s)
2. for i = 1 to |G.V| - 1
3. for each edge (u,v) ∈ G.E
4. RELAX(u,v,w)
5. for each edge (u,v) ∈ G.E
6. if v.d > u.d + w(u,v)
7. // G contains a negative weight cycle
8. return FALSE Running time:
9. return TRUE
INITIALIZE-SINGLE- v.d, Rounds 0-4
SOURCE(G,s)
for i = 1 to |G.V| - 1

EXAMPLE (SOURCE:A)
v 0 1 2 3 4
for each edge
(u,v) ∈ G.E
a 0
RELAX(u,v,w)
for each edge (u,v)
∈ G.E c b b ∞
5
if v.d > u.d + ∞ h
w(u,v) ∞ c ∞
return FALSE ∞
return TRUE -8 d ∞
12 2 -1 9
a e ∞
3
d ∞ 0 ∞
20 f ∞
g
4 -6 g ∞
5
∞ h ∞
∞
f
e
 CORRECTNESS OF
BELLMAN-FORD (3
STEPS)
v.d = δ(s,v) for all v ∈ V


1

If Bellman-Ford there is no negative-
weight cycle, the algorithm returns
2 TRUE

If there is a negative-weight cycle,
3 the algorithm returns false
DAG SHORTEST PATHS
ALGORITHM
Applies only
to directed 
Hence, no
acyclic negative
graphs weight cycles
(DAGs) 
Relaxes edges
Built upon (u,v),
topological considering
sort starting points
algorithm in topological
order
 DAG-SHORTEST-
PATHS(G,W,S)
1. Topologically sort the vertices of G
2. INITIALIZE-SINGLE-SOURCE(G,s)
3. for each vertex u, taken in topologically
sorted order
4. for each v ∈ G.Adj[u]
5. RELAX(u,v,w)

• Runtime?
 EXAMPLE DAG-
SHORTEST-PATHS(G,W,A)
b 4
• First perform DFS to find
c
the topological order (use
2 -1 reverse alpha order
v v.d v.f
a 7 -2 e
d
6 2 c
b
1 a
e d • Then initialize nodes
• Then relax the nodes in
topological sorting order
 CORRECTNESS OF DAG-SHORTEST
PATHS ALGORITHM

Consider any What property What about

path <v0v1…vk> can we use to vertices
in G, where v0 = prove δ(s,vk) = unreachable
s vk.d from s?

The edges in
any path
must be
relaxed in
order
 APPLICATION OF DAG-
SHORTEST-PATHS
Edges
Nodes
represent
PERT chart represent Goal
precedence
jobs
constraints
E.g., could be
installing Find the
some engine
part in an Edges that critical
Programmin automotive don’t path in G
assembly line
g evaluation share a
Edge weight
and review (u,v) node can This is the
technique represents be done in longest
time to parallel path from
complete the
job the root to
represented a leaf
by (u,v)
EXAMPLE PERT CHART

https://www.investopedia.com/terms/p/pert-chart.asp
 USING DAG-SHORTEST-PATHS
ALGORITHM FOR PERT CHARTS
Problem

Rather than looking for the shortest path, we want
to know the longest path
Two strategies

Replace all weights w(u,v) with –w(u,v)

Initialize weights to -∞ and relax to increase v.d
instead of decreasing
 DIJKSTRA’S ALGORITHM
Assumptions Strategy

Directed weighted 
Use a greedy BFS-
graph style approach

No negative 
Modified for
weight edges weighted edges

Similar to Prim’s

Uses path weight
instead of final
edge weight
 Dijkstra(G,w,s)

DIJKSTRA(G, w, s)
1. INITIALIZE-SINGLE-SOURCE(G,
s)
2. S = O
3. Q = O
4. for each vertex u ∈ G.V
5. INSERT(Q, u)
6. while Q ≠ O
7. u = EXTRACT-MIN(Q)
8. S = S ∪ {u}
9. for each vertex v in G.Adj[u]
10. RELAX(u, v, w)
11. if the call of RELAX
decreased v.d
12. DECREASE-KEY(Q, v, v.d)
 DIJKISTRA ALGORITHM: TRACING
WHAT IS THE SHORTEST PATH FROM S TO X?

t x K dv pv
1
  s 0 -
10 9
t 
2 3 4 6
s 0 
x
5 7
Y 
 
2 
y z
z
 SOURCE(G,s)
S = Ø; Q = G.V

RUNTIME FOR 2 while Q ≠ Ø

u = EXTRACT-
MIN(Q)

IMPLEMENTATIONS
• Array, where vertex v is in i entry
i
th
S = S ∪ {u}
for each v ∈
G.Adj[u]
RELAX(u,v,w)
– Cost of creating the queue?
– Each Extract-Min(Q)?
• How many are there total?
– Each Relax(u,v,w)?
• How many are there total?
– Overall Cost:
• Min Heap (Recommended if E = o(V2/lg V))
– Cost of creating the queue?
– Each Extract-Min(Q)?
• How many are there total?
– Each Relax(u,v,w)?
• How many are there total?
– Overall Cost:
 SOURCE(G,s)
S = Ø; Q = G.V
while Q ≠ Ø

CORRECTNESS
u = EXTRACT-
MIN(Q)
S = S ∪ {u}
for each v ∈
G.Adj[u]

Dijkstra’s algorithm, run on RELAX(u,v,w)
a
Theore weighted directed graph G=(V,E)
with non-negative edge weights,
m: terminates with v.d = δ(s,v) for all
v in V.


Use the following loop invariant
At the start of each iteration of
Proof:


the while loop (lines 3 – 7), v.d =

δ(s,v) for all v in S
SOURCE(G,s)
S = Ø; Q = G.V
At the start of each
while Q ≠ Ø iteration of the
u = EXTRACT-
MIN(Q)
S = S ∪ {u}
LOOP INVARIANT
while loop (lines 4
– 8), v.d = δ(s,v)
for each v ∈ for all v in S
G.Adj[u]
RELAX(u,v,w)

Initializatio
n

Terminatio
n
SOURCE(G,s)
S = Ø; Q = G.V At the start of each
while Q ≠ Ø iteration of the
u = EXTRACT-
MIN(Q)
S = S ∪ {u}
LOOP INVARIANT while loop (lines 4 –
8), v.d = δ(s,v) for
for each v ∈ all v in S
G.Adj[u]
RELAX(u,v,w)

On each loop iteration, one
vertex u is added to S

Need to show u.d = δ(s,u)
Maintenan when it is added (i.e., when it
has min .d value in Q)
ce 
Proof by contradiction

Let u be the first vertex
added to S with u.d ≠ δ(s,u)
SOURCE(G,s)
S = Ø; Q = G.V u is the first
while Q ≠ Ø
vertex added to
u = EXTRACT-
MIN(Q)
S = S ∪ {u}
MAINTENANCE S with u.d ≠
δ(s,u)
for each v ∈
G.Adj[u]
RELAX(u,v,w)
Note u ≠ s and δ(s,u) < ∞


Therefore S ≠ Ø just before u is added to S



Let p be a shortest path from s to u


We know p exists because

Prior to adding u to s, p connects a vertex in S to a

vertex in V-S

SOURCE(G,s)
S = Ø; Q = G.V • u is first vertex
while Q ≠ Ø added to S with
MIN(Q) MAINTENANCE (CONT.)
u = EXTRACT-

S = S ∪ {u}
u.d ≠ δ(s,u)
• p is a shortest
for each v ∈ path from s to u
G.Adj[u]
RELAX(u,v,w)
Let y be the first vertex in p that is in V-S

Let x be y’s predecessor (x is in S)

When x is added to S, x.d =

Decompose p into s ↝p1 x → y ↝p2 u


Either p1 or p2 may have no edges

I.e., may have s = x or y = u
SOURCE(G,s) At the start of
ILLUSTRATION OF P1 AND
S = Ø; Q = G.V
while Q ≠ Ø
u = EXTRACT-
each iteration of
the while loop,
MIN(Q) v.d = δ(s,v) for
S = S ∪ {u}
for each v ∈
G.Adj[u]
P2 all v in S

RELAX(u,v,w)
S u
• u is first vertex added to S
p2
s with u.d ≠ δ(s,u)
s • p is a shortest path from s to
u
p • p = p1 (x,y) p2
x
1 x • S is the set of vertices
already processed by
Dijkstra’s algorithm when u
y is chosen
• y is the first vertex in p that
is ∉ S when u is chosen
• x ∈ X is y’s predecessor in p
• x & y are distinct, but we
might have x = s or y = u
• Also, p2 may or may not
reenter S
SOURCE(G,s)

CORRECTNESS OF Y.D
S = Ø; Q = G.V
while Q ≠ Ø
u = EXTRACT-
MIN(Q)

(MAINT. PROOF CONT.)

S = S ∪ {u}
for each v ∈
G.Adj[u]
Claim: y.d
RELAX(u,v,w)= δ(s,y) when u Use this claim to reach a
is added to S contradiction

x is added to S before u 
How are u.d, y.d, δ(s,u)
is (by assumption) and δ(s,y) related?

x.d =

Edge (x,y) was relaxed 
Given u was added to S
when x was added to S before y, what can we

So, y.d = conclude about u.d and
y.d?


Show u.d = δ(s,u).
 APPLICATION:
DIFFERENCE
CONSTRAINTS
Given a set of inequalities of the form x – x ≤ b
i j k


x1 – x2 ≤ 5

x1 – x3 ≤ 6

x2 – x4 ≤ -1

x3 – x4 ≤ -2

x4 – x1 ≤ -3
Find x values that satisfy all inequalities

What values satisfy the inequalities above?
 LEMMA
If x is a feasible solution for a set of
inequalities, then so is x + d for any
constant d.

Proof
GRAPH REPRESENTATION OF DIFF
CONSTR PROBLEM
Create Graph Assign weight
G=(V,E) function w
1 vertex per
variable w(v0,vj) = 0 for all j
+ v0
1 edge per
constraint + w(vj,vi) = bk if
1 edge from v0 to xi – xj ≤ bk
every other vertex
• x1 – x2 ≤ 5 •
w(v0,vi) = 0 for all i>0
•
•
x1 – x3 ≤ 6
x2 – x4 ≤ -1
EXAMPLE
•
w(vj,vi) = bk if xi – xj ≤
bk
• x3 – x4 ≤ -2
• x4 – x1 ≤ -3 v1 v2

v0

v4 v3
 THEOREM

Given a system of difference
constraints, let G=(V,E) be the
corresponding graph.

If G has no negative-weight cycles,
then x=(δ(v0,v1),…,δ(v0,vn)) is a
1 feasible solution.

If G has a negative-weight cycle, then
2 there is no feasible solution.
 PROOF OF 1.
G has no negative-weight cycles ⇒ xi=δ(v0,vi) (i=1,…,n) is a feasible
solution.
Consider any constraint xi-xj ≤ bk
What values are assigned to xi and xj?

What value is associated with bk in G?

Use the triangle inequality

 PROOF OF 2.
If G has a negative-weight cycle, then there is no feasible
solution.
Let c = <v0,v1,…,vk,v0=vk+1> be a negative-
weight cycle
What constraints are associate with c?

Assume the constraints are satisfied & add them

up

Argue there’s a contradiction

 SOLVING DIFF
CONSTRAINT PROBLEM
• USING
Given m constraints over n SSSP
variables
1. Form the corresponding constraint graph G=(V,E)
– |V| =
– |E| =
2. Run Bellman-Ford(G,w,v0)
Runtime
3. If return value is TRUE
4. xi = δ(v0,vi) for all i
5. Else
6. No feasible solution