Design and Analysis of Algorithms

CSE 5311
Lecture 21 Single-Source Shortest Paths

Junzhou Huang, Ph.D.

Department of Computer Science and Engineering

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 1

 Single-Source Shortest Paths

• Given: A single source vertex in a weighted, directed

graph.
• Want to compute a shortest path for each possible
destination.
– Similar to BFS.
• We will assume either
– no negative-weight edges, or
– no reachable negative-weight cycles.
• Algorithm will compute a shortest-path tree.
– Similar to BFS tree.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 2

 Outline

• General Lemmas and Theorems.

• Bellman-Ford algorithm.
• DAG algorithm.
• Dijkstra’s algorithm.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 3

 General Results (Relaxation)
Lemma 24.1: Let p = ‹v1, v2, …, vk› be a SP from v1 to vk. Then,
pij = ‹vi, vi+1, …, vj› is a SP from vi to vj, where 1 ≤ i ≤ j ≤ k.

So, we have the optimal-substructure property.

Bellman-Ford’s algorithm uses dynamic programming.
Dijkstra’s algorithm uses the greedy approach.
Let δ(u, v) = weight of SP from u to v.
p'
Corollary: Let p = SP from s to v, where p = s

→ u →v. Then,
δ(s, v) = δ(s, u) + w(u, v).

Lemma 24.10: Let s ∈ V. For all edges (u,v) ∈ E, we have

δ(s, v) ≤ δ(s, u) + w(u,v).
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 4
 • Lemma 24.1 holds because one edge gives the shortest
path, so the other edges must give sums that are at least as
large.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 5

 Relaxation
Algorithms keep track of d[v], π[v]. Initialized as follows:

Initialize(G, s)
for each v ∈ V[G] do
d[v] := ∞;
π[v] := NIL
end;
d[s] := 0
These values are changed when an edge (u, v) is relaxed:
Relax(u, v, w)
if d[v] > d[u] + w(u, v) then
d[v] := d[u] + w(u, v);
π[v] := u
end
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 6
 Properties of Relaxation

• d[v], if not ∞, is the length of some path from s to v.

• d[v] either stays the same or decreases with time
• Therefore, if d[v] = δ(s, v) at any time, this holds thereafter
• Note that d[v] ≥ δ(s, v) always
• After i iterations of relaxing on all (u,v), if the shortest path
to v has i edges, then d[v] = δ(s, v).

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 7

 Properties of Relaxation
Consider any algorithm in which d[v], and π[v] are first initialized by
calling Initialize(G, s) [s is the source], and are only changed by calling
Relax. We have:
Lemma 24.11: (∀ v:: d[v] ≥ δ(s, v)) is an invariant.
Implies d[v] doesn’t change once d[v] = δ(s, v).
Proof:
Initialize(G, s) establishes invariant. If call to Relax(u, v, w)
changes d[v], then it establishes:
d[v] = d[u] + w(u, v)
≥ δ(s, u) + w(u, v) , invariant holds before call.
≥ δ(s, v) , by Lemma 24.10.
Corollary 24.12: If there is no path from s to v, then
d[v] = δ(s, v) = ∞ is an invariant.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 8
 • For lemma 24.11, note that initialization makes the
invariant true at the beginning.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 9

 More Properties
Lemma 24.13: Immediately after relaxing edge (u, v) by calling
Relax(u, v, w), we have d[v] ≤ d[u] + w(u, v).
p'
Lemma 24.14: Let p = SP from s to v, where p = s

→ u →v.
If d[u] = δ(s, u) holds at any time prior to calling Relax(u, v, w),
then d[v] = δ(s, v) holds at all times after the call.
Proof:
After the call we have:
d[v] ≤ d[u] + w(u, v) , by Lemma 24.13.
= δ(s, u) + w(u, v) , d[u] = δ(s, u) holds.
= δ(s, v) , by corollary to Lemma 24.1.
By Lemma 24.11, d[v] ≥ δ(s, v), so d[v] = δ(s, v).
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 10
 • Lemma 24.13 follows simply from the structure of Relax.
• Lemma 24.14 shows that the shortest path will be found
one vertex at a time, if not faster. Thus after a number of
iterations of Relax equal to V(G) - 1, all shortest paths
will be found.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 11

 • Bellman-Ford returns a compact representation of the set
of shortest paths from s to all other vertices in the graph
reachable from s. This is contained in the predecessor
subgraph.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 12

 Predecessor Subgraph
Lemma 24.16: Assume given graph G has no negative-weight cycles
reachable from s. Let Gπ = predecessor subgraph. Gπ is always a
tree with root s (i.e., this property is an invariant).

Proof:
Two proof obligations:
(1) Gπ is acyclic.
(2) There exists a unique path from source s to each vertex in Vπ.
Proof of (1):
Suppose there exists a cycle c = ‹v0, v1, …, vk›, where v0 = vk.
We have π[vi] = vi-1 for i = 1, 2, …, k.
Assume relaxation of (vk-1, vk) created the cycle.
We show cycle has a negative weight.
Note: Cycle must be reachable from s.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 13
 Proof of (1) (Continued)
Before call to Relax(vk-1, vk, w):
π[vi] = vi-1 for i = 1, …, k–1.
Implies d[vi] was last updated by “d[vi] := d[vi-1] + w(vi-1, vi)”
for i = 1, …, k–1. [Because Relax updates π.]
Implies d[vi] ≥ d[vi-1] + w(vi-1, vi) for i = 1, …, k–1. [Lemma 24.13]
Because π[vk] is changed by call, d[vk] > d[vk-1] + w(vk-1, vk). Thus,
k k

∑ d[v ] > ∑ (d[v

i =1
i
i =1
i −1 ] + w(vi −1 , v i ))
k k
= ∑ d[vi −1 ] + ∑ w(vi −1 , v i )
i =1 i =1
k k k
Because ∑ d[vi ] = ∑ d[vi −1 ], ∑ w(v i −1 , v i ) < 0, i.e., neg. - weight cycle!
i =1 i =1 i =1

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 14

 Comment on Proof

• d[vi] ≥ d[vi-1] + w(vi-1, vi) for i = 1, …, k–1 because when

Relax(vi-1 , vi , w) was called, there was an equality, and
d[vi-1] may have gotten smaller by further calls to Relax.
• d[vk] > d[vk-1] + w(vk-1, vk) before the last call to Relax
because that last call changed d[vk].

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 15

 Lemma 24.17
Lemma 24.17: Same conditions as before. Call Initialize & repeatedly
call Relax until d[v] = δ(s, v) for all v in V. Then, Gπ is a shortest-path
tree rooted at s.

Proof:
Key Proof Obligation: For all v in Vπ, the unique simple path p from
s to v in Gπ (path exists by Lemma 24.16) is a shortest path from s to v
in G.
Let p = ‹v0, v1, …, vk›, where v0 = s and vk = v.
We have d[vi] = δ(s, vi)
d[vi] ≥ d[vi-1] + w(vi-1, vi) (reasoning as before)
Implies w(vi-1, vi) ≤ δ(s, vi) – δ(s, vi-1).
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 16
 Proof (Continued)
w(p)
k
= ∑ w(vi −1 , v i )
i =1
k
≤ ∑ (δ(s , v i ) − δ(s , v i-1 ))
i =1

= δ(s , v k ) − δ(s , v 0 )
= δ(s , v k )

So, equality holds and p is a shortest path.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 17

 • And note that this shortest path tree will be found after
V(G) - 1 iterations of Relax.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 18

 Bellman-Ford Algorithm
Can have negative-weight edges. Will “detect” reachable negative-
weight cycles.
Initialize(G, s);
for i := 1 to |V[G]| –1 do
for each (u, v) in E[G] do
Relax(u, v, w) Time
od Complexity
od; is O(VE).
for each (u, v) in E[G] do
if d[v] > d[u] + w(u, v) then
return false
fi
od;
return true
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 19
 • So if Bellman-Ford has not converged after V(G) - 1
iterations, then there cannot be a shortest path tree, so
there must be a negative weight cycle.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 20

 Example

u v
5
∞ ∞
–2
6 –3
8
z 0 7
–4
7 2

∞ ∞
9
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 21

 Example

u v
5
6 ∞
–2
6 –3
8
z 0 7
–4
7 2

7 ∞
9
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 22

 Example

u v
5
6 4
–2
6 –3
8
z 0 7
–4
7 2

7 2
9
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 23

 Example

u v
5
2 4
–2
6 –3
8
z 0 7
–4
7 2

7 2
9
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 24

 Example

u v
5
2 4
–2
6 –3
8
z 0 7
–4
7 2

7 -2
9
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 25

 Another Look
Note: This is essentially dynamic programming.

Let d(i, j) = cost of the shortest path from s to i that is at most j hops.
0 if i = s ∧ j = 0
∞ if i ≠ s ∧ j = 0
d(i, j) =
min({d(k, j–1) + w(k, i): i ∈ Adj(k)}
∪ {d(i, j–1)}) if j > 0
i
z u v x y
1 2 3 4 5
j 0 0 ∞ ∞ ∞ ∞
1 0 6 ∞ 7 ∞
2 0 6 4 7 2
3 0 2 4 7 2
4 0 2 4 7 –2
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 26
 Lemma 24.2
Lemma 24.2: Assuming no negative-weight cycles reachable from
s, d[v] = δ(s, v) holds upon termination for all vertices v reachable
from s.

Proof:
Consider a SP p, where p = ‹v0, v1, …, vk›, where v0 = s and vk = v.

Assume k ≤ |V| – 1, otherwise p has a cycle.

Claim: d[vi] = δ(s, vi) holds after the ith pass over edges.
Proof follows by induction on i.

By Lemma 24.11, once d[vi] = δ(s, vi) holds, it continues to hold.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 27
 Correctness

Claim: Algorithm returns the correct value.

(Part of Theorem 24.4. Other parts of the theorem follow easily from earlier results.)

Case 1: There is no reachable negative-weight cycle.

Upon termination, we have for all (u, v):

d[v] = δ(s, v) , by lemma 24.2 (last slide) if v is reachable;
d[v] = δ(s, v) = ∞ otherwise.
≤ δ(s, u) + w(u, v) , by Lemma 24.10.
= d[u] + w(u, v)

So, algorithm returns true.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 28

 Case 2
Case 2: There exists a reachable negative-weight cycle
c = ‹v0, v1, …, vk›, where v0 = vk.

We have Σi = 1, …, k w(vi-1, vi) < 0. (*)

Suppose algorithm returns true. Then, d[vi] ≤ d[vi-1] + w(vi-1, vi) for
i = 1, …, k. (because Relax didn’t change any d[vi] ). Thus,

Σi = 1, …, k d[vi] ≤ Σi = 1, …, k d[vi-1] + Σi = 1, …, k w(vi-1, vi)

But, Σi = 1, …, k d[vi] = Σi = 1, …, k d[vi-1].
Can show no d[vi] is infinite. Hence, 0 ≤ Σi = 1, …, k w(vi-1, vi).
Contradicts (*). Thus, algorithm returns false.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 29
 Shortest Paths in DAGs

Topologically sort vertices in G;

Initialize(G, s);
for each u in V[G] (in order) do
for each v in Adj[u] do
Relax(u, v, w)
od
od

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 30

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 ∞ ∞ ∞ ∞

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 31

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 ∞ ∞ ∞ ∞

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 32

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 2 6 ∞ ∞

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 33

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 2 6 6 4

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 34

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 2 6 5 4

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 35

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 2 6 5 3

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 36

 Example

6 1
r s t u v w
5 2 7 –1 –2
∞ 0 2 6 5 3

4
3
2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 37

 Dijkstra’s Algorithm
Assumes no negative-weight edges.
Maintains a set S of vertices whose SP from s has been determined.
Repeatedly selects u in V–S with minimum SP estimate (greedy choice).
Store V–S in priority queue Q.
Initialize(G, s);
S := ∅;
Q := V[G];
while Q ≠ ∅ do
u := Extract-Min(Q);
S := S ∪ {u};
for each v ∈ Adj[u] do
Relax(u, v, w)
od
od
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 38
 Example

u v
1
∞ ∞

10
9
2 3
s 0 4 6

5 7

∞ ∞
2
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 39

 Example

u v
1
10 ∞

10
9
2 3
s 0 4 6

5 7

5 ∞
2
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 40

 Example

u v
1
8 14

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 41

 Example

u v
1
8 13

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 42

 Example

u v
1
8 9

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 43

 Example

u v
1
8 9

10
9
2 3
s 0 4 6

5 7

5 7
2
x y

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 44

 Correctness

Theorem 24.6: Upon termination, d[u] = δ(s, u) for all u in V

(assuming non-negative weights).

Proof:
By Lemma 24.11, once d[u] = δ(s, u) holds, it continues to hold.
We prove: For each u in V, d[u] = δ(s, u) when u is inserted in S.
Suppose not. Let u be the first vertex such that d[u] ≠ δ(s, u) when
inserted in S.
Note that d[s] = δ(s, s) = 0 when s is inserted, so u ≠ s.
⇒ S ≠ ∅ just before u is inserted (in fact, s ∈ S).

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 45

 Proof (Continued)
Note that there exists a path from s to u, for otherwise
d[u] = δ(s, u) = ∞ by Corollary 24.12.
⇒ there exists a SP from s to u. SP looks like this:
p2

u
s
p1

x y
S

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 46

 Proof (Continued)
Claim: d[y] = δ(s, y) when u is inserted into S.
We had d[x] = δ(s, x) when x was inserted into S.
Edge (x, y) was relaxed at that time.
By Lemma 24.14, this implies the claim.

Now, we have: d[y] = δ(s, y) , by Claim.

≤ δ(s, u) , nonnegative edge weights.
≤ d[u] , by Lemma 24.11.

Because u was added to S before y, d[u] ≤ d[y].

Thus, d[y] = δ(s, y) = δ(s, u) = d[u].
Contradiction.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 47
 Complexity

Running time is
O(V2) using linear array for priority queue.
O((V + E) lg V) using binary heap.
O(V lg V + E) using Fibonacci heap.

(See book.)

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 48