DESIGN AND ANALYSIS OF ALGORITHMS

Dr. Kashif Ayyub

1
Binary Search Tree (BST)
• A Binary Search Tree (BST) is a type of Binary Tree data structure, where the
following properties must be true for any node "X" in the tree:
• The X node's left child and all its descendants (children, children's children, and so on)
have lower values than X's value.
• The right child, and all its descendants have higher values than X's value.
• Left and right subtrees must also be Binary Search Trees.

2
Balanced Binary Tree
• A balanced binary tree, also referred to as a height-balanced binary tree, is
defined as a binary tree in which the height of the left and right subtree of
any node differ by not more than 1.

3
Balanced Binary Search Tree
• If we have keys: 15, 25, 35, then see how many BST can be
formed, which grows exponentially, brute force is hopeless.
• In the third case, the total number of comparisons is less as
compared to others because the height of the tree is less. So,
we can say that it’s a balanced binary search tree.

4
Avg. No. of Comparisons
Optimal Binary Search Tree (OBST)
• Now assume there is an associated cost to search a value in the tree, now we
have to find out which balanced binary search tree is optimal
Keys 15 25 35
Associated Cost (AC) 3 2 5

1x3=3

2x2=4

3x5=15

Keys 15 25 35 Keys 15 25 35 Keys 15 25 35 Keys 15 25 35 Keys 15 25 35

Cost 1x3 2x2 3x5 Cost 1x3 3x2 2x5 Cost 2x3 1x2 2x5 Cost 2x3 3x2 1x5 Cost 3x3 2x2 1x5
Total Cost = 22 Total Cost = 19 Total Cost = 18 Total Cost = 17 Total Cost = 18
5
Optimal Binary Search Tree (OBST)
• Height balanced tree might not be the best option when there is associated cost.
• OBST is a binary search tree that provides the shortest possible search time or expected search time.
• We can find out the OBST by using dynamic programming approach.

Keys 15 25 35
Cost 2x3 3x2 1x5
Total Cost = 17

Height Balanced BST Optimal BST

6
Optimal Binary Search Trees
• Let 𝑇(𝑛) be the time to solve the OBST for n keys.
• For each of the 𝑛 keys, the algorithm recursively solves
for left and right subtrees.

one on size 𝑖−1 and one on size 𝑛−𝑖.

• So, for each root candidate, it makes two recursive calls,

• This recurrence grows faster than exponential.

7
Optimal Binary Search Trees
• Key Table
• Keys (in order) + frequency

A B C D E F G H I J K L M N O P …
23 10 8 12 30 5 14 18 20 2 4 11 7 22 22 10 …

• Key Problem
• Which key should be placed at the root?
• If we can determine this, we can ask the same question for the left and right subtrees.

8
Optimal Binary Search Tree
• Divide and conquer?
• Choose a key for the root n choices
• Repeat the process for the sub-trees O(2n)

9
Optimal Binary Search Tree
• Start with the small problems

A B C D E F G H I J K L M N O P …
23 10 8 12 30 5 14 18 20 2 4 11 7 22 22 10 …

• Look at pairs of adjacent keys C D

• Two possible D C
Min

arrangements
Cost 8x1 + 12x2 = 32 8x2 + 12x1 = 28

10
Optimal Binary Search Tree - Cost Matrix
• Initialise
Cjj
• Diagonal
• C[j,j]
• Costs of
one-element ‘trees’ Zer
o
• Below diagonal
• C[j,k] x
• Costs of best tree
j to k
Cost of best tree C-G

11
Optimal Binary Search Tree - Cost matrix

• Store the costs of the best two element trees

Cj-1,j
• Diagonal
• C[j,j]
• Costs of
one-element ‘trees’
• Below diagonal
• C[j-1,j]
• Costs of best
2-element trees
j-1 to j

12
Optimal Binary Search Tree - Root matrix

• Store the roots of the best two element trees

• Diagonal
• Roots of 1-element trees
• Below diagonal
• best[j-1,j]
• Root of best
2-element trees
j-1 to j

13
Optimal Binary Search Tree - 3-element trees

• Now examine the 3-element trees

A B C D E F G H I J K L M N O P …
23 10 8 12 30 5 14 18 20 2 4 11 7 22 22 10 …

• Choose each in turn as the root

• B with (C,D) to the right Next slide

• C with B and D as children

• D with (B,C) to the left
• Find best, store cost in C[B,D]
• Store root in best[B,D]
14
Optimal Binary Search Tree - 3-element trees

• 3-element trees
 Root =  Root =  Root =
B C D
B We already D
know this C
D is B
best for C,D B D
C and stored
C
its cost

Best
B,C
• Find best, store cost in C[B,D]
• Store root in best[B,D]
15
Optimal Binary Search Tree - 3-element trees

• Similarly, update all C[j-2,j] and best[j-2,j]

Costs Roots

16
Optimal Binary Search Trees - 4-trees
• Now the 4-element trees
eg A-D

Use 0 for left

• Choose A as root Best B-D is known
A-A is in C[0,0]
• Choose B as root Best C-D is known
A-B is in C[0,1]
• Choose C as root D is in C[3,3]
A-C is in C[0,2]
• Choose D as root Use 0 in C[4,3] for right

17
Optimal Binary Search Trees
• Final cost will be in C[0,n-1]

Final
cost

18
Optimal Binary Search Trees
• Construct the search tree
• Root will be in
best[0,n-1]
• If r0 = best[0,n-1],
• Left subtree root is
best[0,r0-1],
Right subtree root is
best[r0+1,n-1]
Root = ‘E’

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 …

A B C D E F G H I J K L M N O P …
23 10 8 12 30 5 14 18 20 2 4 11 7 22 22 10 … 19
Optimal Binary Search Trees
• Construct the search tree

B H

A D G I

C F J

20
Optimal Binary Search Trees - Analysis

21
Optimal Binary Search Trees - Analysis

• Let C(i, j) denote the cost of an optimal binary search

tree containing ai,…,aj .
• The cost of the optimal binary search tree with ak as its
root :
  k 1
  n

       
C(1, n) min Pk   Q 0   Pi  Q i  C 1, k  1    Q k   Pi  Q i  C k  1, n  
1k n
  i 1   i k 1 

ak
P 1 ...P k-1 P k+1 ...P n
Q 0 ...Q k-1 Q k ...Q n

a1...ak-1 ak+1...an
8-
C(1,k-1) C(k+1,n) 22
Optimal Binary Search Trees - Analysis
  k 1

C(i, j) min  Pk   Qi-1   Pm  Q m  Ci, k  1
i k j
  m i 
 j

  Q k   Pm  Q m  Ck  1, j 
 m k 1 
 j

min Ci, k  1 Ck  1, j Qi-1   Pm  Q m 
i k j
 m i 
ak
P 1 ...P k-1 P k+1 ...P n
Q 0 ...Q k-1 Q k ...Q n

a1...ak-1 ak+1...an
8-
C(1,k-1) C(k+1,n) 23
Optimal Binary Search Trees - Analysis
• k -element trees require k operations
• One for each candidate root
• There are k of them O(k2) n
• There are n levels  k 2
= O(n3)
k =1

24