Graph Algorithms

Discrete Mathematics and Optimization

Bioinformatics

(ESCI) Graph Algorithms 1 / 10

Outline

Depth First Search

Breadth First Search
Application: Check bipartiteness
Minimum Spanning Tree: Kruskal algorithm
Minimum Spanning Tree. Prim algorithm
Application: Clustering

(ESCI) Graph Algorithms 2 / 10

Depth First Search
Exploring graphs

DFS Algorithm
Input: A graph G (by adjacency lists) and a root vertex r
Output: An ordering of the vertices in the connected component of G
containing r .
DFS(Ex, r ): Explore first indepth before backtracking
1 1 1 1
2 2 2
3 3 4

1 1 1
1
2 2 7 2 7
2
3 4 5 3 4 5 3 4 5
3 4 5
6 6 6 8

(ESCI) Graph Algorithms 3 / 10

Breadth First Search
Exploring graphs

BFS Algorithm
Input: A graph G (by adjacency lists) and a root vertex r
Output: An ordering of the vertices in the connected component of G
containing r .
▶ Start at initial vertex (root) r .
▶ Visit all vertices adjacent to the root r (put them on a queue).
▶ Repeatedly visit the neighbours of visited vertices.
▶ Stop when there are no new vertices to visit.
1 1 1 1
2 3 2 3 2 3
4 5 4 5 6

1
2 3
4 5 6
7 8
(ESCI) Graph Algorithms 4 / 10
Breadth First Search
Exploring graphs

BFS Algorithm
Input: A graph G (by adjacency lists) and a root vertex r
Output: An ordering of the vertices in the connected component of G
containing r .
▶ Start at initial vertex (root) r .
▶ Visit all vertices adjacent to the root r (put them on a queue).
▶ Repeatedly visit the neighbours of visited vertices.
▶ Stop when there are no new vertices to visit.

Set Explored[r] = true and Explored[v] = false for v ̸= r

L[0] = s (vertices at distance 0 from r )
i = 0 (layer counter)
Set T = ∅ (initial tree)
While L[i] is not empty
Initialize an empty list L[i + 1]
For each vertex u in L[i] Consider each edge (u, v )
If Explored[v ] = false then
Set Explored[v ] = true Add (u, v ) to T Add v to L[i + 1]
Endif
Endfor
(ESCI) Graph Algorithms 4 / 10
Breadth First Search
Exploring graphs

BFS Algorithm
Input: A graph G (by adjacency lists) and a root vertex r
Output: An ordering of the vertices in the connected component of G
containing r .
▶ Start at initial vertex (root) r .
▶ Visit all vertices adjacent to the root r (put them on a queue).
▶ Repeatedly visit the neighbours of visited vertices.
▶ Stop when there are no new vertices to visit.

Time complexity O(|V | + |E |).

It provides a Layer decomposition V = L0 ∪ L1 ∪ · · · ∪ Lt of the connected
component containing r .
It can provide the (number of) connected components of G : restart the
algorithm with a new root if not all vertices have been explored.
Can be applied to find the shortest path distance from the root to every
other node in the graph.

(ESCI) Graph Algorithms 4 / 10

Breadth First Search

Check if a graph is bipartite

A graph is bipartite if there is a bipartition V = A ∪ B such that every edge has

precisely one end point in A and one endpoint in B.

Input: A connected graph G

Output: Decide if G is bipartite (decision algorithm)
▶ Choose a vertex r .
▶ Run BFS algorithm from r .
▶ Build A = L0 ∪ L2 ∪ · · · and B = L1 ∪ L3 ∪ · · · .
▶ Check if there are edges in A or in B. If not, output G is bipartite.

Theorem
G is a bipartite graph if and only if it contains no odd cycles.

(ESCI) Graph Algorithms 5 / 10

Breadth First Search

Check if a graph is bipartite

A graph is bipartite if there is a bipartition V = A ∪ B such that every edge has

precisely one end point in A and one endpoint in B.

Theorem
G is a bipartite graph if and only if it contains no odd cycles.

If G is bipartite then all of its subgraphs are bipartite

An odd cycle is nonbipartite: a bipartite graph can not contain odd cycles
If G does not contain odd cycles then A = L0 ∪ L2 ∪ · · · and
B = L1 ∪ L3 ∪ · · · form a bipartition of G .

(ESCI) Graph Algorithms 6 / 10

Minimum Spanning Tree
Weighted Graph G = (V , E )
V set of nodes or vertices
E set of edges (pairs of nodes) (can be directed, multiple, loops,...)
w : E → R+ : each edge has a weight.
The weight of a subgraph H ⊂ G is
X
w (H) = w (e).
e∈H

2 2
3 7 73
2 2
9 9
1 5 3 1 5 3 w (H) = 24
1 6 8 1 6 8
4 4

MST Problem
Given an edge–weighted graph G = (V , E , w ) find a spanning tree of G with
minimum weight.
(ESCI) Graph Algorithms 7 / 10
Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .

Correctedness of the algorithm

Complexity of the algorithm
Implementation

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .
Correctedness:
The output is a forest:
▶ acyclic by construction
▶ The output is a minimum weight spanning forest: E (Fi ) contains the edges of
an MST T for each i.

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .
Complexity:
O(m log m) to sort the edges.
Check acyclicity at each step: keep track of the connected components (n at
the begining). Each added edge has one only vertex in an existing connected
component: check in constant time: O(m).
Complexity is O(m log m) = O(m log n).

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .
Implementation

Disjoint Sets data structure

makeset(x): creates a singleton set containing just x
find(x): returns the identifier of the set containing x
union(x,y): merges the sets containing x and y .
See kruskal.py

(ESCI) Graph Algorithms 8 / 10

Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .

Kruskal MST algorithm is an example of a Greedy algorithm: at each step

optimize locally your next move.

Greedy algorithms not always give the desired result.

Hamiltonian Path with minimum weight (TSP)

103 103
1 1
1 1
1 1
103 1 103 1
1 1
1 1
103 103
3 3
10 10
(ESCI) Graph Algorithms 8 / 10
Kruskal Algorithm
Input: An edge–weighted graph G .
Ouput: A forest of minimum weight (in terms of its edges)
▶ Sort the edges in non–decreasing weights {e1 , . . . , em }
▶ Initialize F0 = {∅}.
▶ for i from 1 to n do
if Fi−1 ∪ ei is acyclic then Fi = Fi−1 ∪ ei else Fi = Fi−1 .
▶ Return Fm .

Joseph Kruskal (1928-2010)

American mathematician, statistician and computer scientist

Worked in Bell Labs (1959–1995)

(ESCI) Graph Algorithms 8 / 10

Application of Kruskal algorithm to Clustering
Input: A set of n items and the distance (u, v ) between each pair.
Output: Divide the items into k groups so that the minimum distance
between the groups is maximized

(ESCI) Graph Algorithms 9 / 10

Application of Kruskal algorithm to Clustering
Input: A set of n items and the distance (u, v ) between each pair.
Output: Divide the items into k groups so that the minimum distance
between the groups is maximized

(ESCI) Graph Algorithms 9 / 10

Application of Kruskal algorithm to Clustering
Input: A set of n items and the distance (u, v ) between each pair.
Output: Divide the items into k groups so that the minimum distance
between the groups is maximized

Build the weighted graph with distances as weights

Run Kruskal algorithm keeping track of the number of connected components
at each step.
Stop when the number of connected components is k
Return the connected components

(ESCI) Graph Algorithms 9 / 10

Prim Algorithm
Input: An edge–weighted connected graph G .
Ouput: A tree of minimum weight
▶ Start at a vertex x0 (root)
▶ Initialize F = ({x0 }, ∅).
▶ While F ̸= V (G )
⋆ Find the edge e = xy with minimum weight with x ∈ F and y ̸∈ F
⋆ F = (V (F ) ∪ {y }, E (F ) ∪ {xy })
▶ Return F .

(ESCI) Graph Algorithms 10 / 10

Prim Algorithm
Input: An edge–weighted connected graph G .
Ouput: A tree of minimum weight
▶ Start at a vertex x0 (root)
▶ Initialize F = ({x0 }, ∅).
▶ While F ̸= V (G )
⋆ Find the edge e = xy with minimum weight with x ∈ F and y ̸∈ F
⋆ F = (V (F ) ∪ {y }, E (F ) ∪ {xy })
▶ Return F .

Correctedness of the algorithm: similar to Kruskal

Complexity of the algorithm:O(n log n)
Implementation: somewhat simpler

(ESCI) Graph Algorithms 10 / 10

Prim Algorithm
Input: An edge–weighted connected graph G .
Ouput: A tree of minimum weight
▶ Start at a vertex x0 (root)
▶ Initialize F = ({x0 }, ∅).
▶ While F ̸= V (G )
⋆ Find the edge e = xy with minimum weight with x ∈ F and y ̸∈ F
⋆ F = (V (F ) ∪ {y }, E (F ) ∪ {xy })
▶ Return F .

a
7
8
5 b c
7
9 15 5
d e
6 9
8 11
f g

(ESCI) Graph Algorithms 10 / 10