Algorithmic Analysis Overview:

Running Time Analysis,

Correctness, Exhaustive Search
CMPSC 463 Design and Analysis of Algorithms
Dr. Truong X. Tran (Ph.D.)
Topics
• Reasoning about Algorithms
• Time and Space Complexity
• Exhaustive Search
What is an Algorithm
• Definition: An algorithm is a finite set of precise instructions for performing a
computation or for solving a problem.
• Take some value as input and produces some values as output.
• Input: An algorithm has input values from a specified set.
• Output: From the input values, the algorithm produces the output values from a
specified set. The output values are the solution.
• Correctness: An algorithm should produce the correct output values for each set
of input values.
• Finiteness: An algorithm should produce the output after a finite number of steps
for any input.
• Effectiveness: It must be possible to perform each step of the algorithm correctly
and in a finite amount of time.
• Generality: The algorithm should work for all problems of the desired form.
Designing and Analyzing Algorithms
• Designing: Multiple techniques to design an algorithm:
• Increment (induction)
• Divide-and-Conquer (recursion)
• Analyzing: resources that the algorithm requires
• Properties of algorithms
• Time and space complexities.
Data Structures: a Review
All running times are worst case (WCRT), unless otherwise specified
Linked List
• Prepend: O(1)
• Search: O(n)
• Access the kth element of the list: O(k)
Array
• Search: O(n)
• Access the kth element of the array: O(1)
Sorted Array
• Search: O(log n)
Dynamic array: where we double the size of the array every time it fills up
• Append: O(n) WCRT, O(1) amortized running time
Data Structures: a Review
Binary Search Tree
• Search: O(n) WCRT, O(log n) average case running time (assuming that items are inserted in a random order)
• Insert: same as search
• Delete: same as search
Balanced Search Tree (e.g. 2-3 tree)
• Search, Add, Delete: O(log n)
Min-Heap
• Min: O(1)
• Add: O(log n), Delete: O(log n)
• MakeHeap (Create a heap from an array of n elements): O(n)
Hash Table
• Search: O(1) average case running time, if the load factor is bounded above by a constant, O(n) WCRT
• Add: O(1), Delete: same as search
Space and Time Complexity
Performance measure of algorithms
• Space: Memory requirement
• Time: Running Time
This course focus mainly on: Running time complexity
• Worse case running time
• Average case running time
• Best case running time
All running times are worst case (WCRT), unless otherwise specified
Analyze time complexity:
• Determine the number of operations with respected to the input’s size
• Number of the operations as a function, f(n), of the size,n, of the input
• How fast f(n) grows as the size of input grows
• To compare the efficiency of two algorithms when solving the same problem.
• The faster grows of f(n), the slower the algorithm can solve a problem
Growth of Functions: O-notation

8
Growth of Functions: ꭥ-notation

9
Growth of Functions: Θ-notation

10
Display of Growth of Functions

Note the difference in behavior of functions as n gets

larger
Useful Big-O Estimates Involving Logarithms, Powers, and
Exponents
If d > c > 1, then
nc is O(nd), but nd is not O(nc).
If b > 1 and c and d are positive, then
(logb n)c is O(nd), but nd is not O((logb n)c).
If b > 1 and d is positive, then
nd is O(bn), but bn is not O(nd).
If c > b > 1, then
bn is O(cn), but cn is not O(bn).
Exhaustive Search
• A brute-force algorithm
• Examining all possible paths for a valid solution.
• Suitable for small and well-defined search space.
Problem: Traveling salesman
• Given n cities with known distances between each pair, find the
shortest tour that passes through all the cities exactly once before
returning to the starting city.
• Example:
Exhaustive Search Model
• Choosing
• We generally iterate over decisions. What are we iterating over here? The iteration
will be done by recursion.
• What are the choices for each decision? Do we need a for loop?
• Exploring
• How can we represent that choice? How should we modify the parameters?
• Do we need to use a wrapper due to extra parameters?
• Un-Choosing
• How do we un-modify the parameters from step 3? Do we need to explicitly un-
modify, or are they copied? Are they un-modified at the same level as they were
modified?
• Base Case
• What should we do in the base case when we're out of decisions?
Problem: print all binary
• Input: A positive integer n.
• Return: print all binary number of n bits
• Exhaustive Search: Examining all binary of n bits
void printBinary(int n) {
helper(n, " ");
}
void helper(int n, string bits) {
if (n == 0) {
cout << bits<< endl;
} else {
helper(digits - 1, bits + "0");
helper(digits - 1, bits + "1");
}
}
printBinary(2)
Problem: Subset Sum
• Input: An array with n integers and k.
• Return: whether there is a subset of integers that adds up to k
• Exhaustive Search: Examining all Subsets
static boolean subsetSum(int[] values, int k) {
return helper(values, 0, k);
}

static boolean helper(int[] values, int nextNdx, int sum) {

if (nextNdx == values.length) {
return (sum == 0);
}
return helper(values, nextNdx + 1, sum - values[nextNdx]) ||
helper(values, nextNdx + 1, sum);
}
Subset sum example
 Output all subsets with sum k
static void subsetSum(int[] values, int k) {
boolean[] included = new boolean[values.length];
helper(values, 0, subset, k);}
static void helper(int[] values, int nextNdx, boolean[] included, int sum) {
if (nextNdx == values.length) {
if (sum == 0) {
for (int i = 0; i < values.length; i++) {
if (included[i])
System.out.print(" " + values[i]);
}
System.out.println();
}
return;
}
included[nextNdx] = true;
helper(values, nextNdx + 1, included, sum - values[nextNdx]);
included[nextNdx] = false;
helper(values, nextNdx + 1, included, sum);}
 Problem: Generating all Strings
• Input: A string containing an alphabet, and an integer n
• Return: A dynamic array with all of the strings of length n using symbols from the alphabet
static ArrayList<String> generateStrings(String alphabet, int n) {
ArrayList<String> result = new ArrayList<>();
helper(result, alphabet, "", n);
return result;}
static void helper(ArrayList<String> strings, String alphabet, String prefix,
int charsRemaining) {
if (charsRemaining == 0) {
strings.add(prefix);
return; }
for (int i = 0; i < alphabet.length(); i++) {
helper(strings, alphabet, prefix + alphabet.charAt(i), charsRemaining - 1);
// charsRemaining--?
}
}
Computational Intractability
Problems that cannot be solved in polynomial time I.e. not constant k
for which the running time of the solution is in O(n^k)
• Can only solve small instances of the problem
• Seen exhaustive search approaches for this
• ONE OF CENTRAL IDEAS IS HOW TO IMPROVE ON EXHAUSTIVE
SEARCH…
• Find polynomial time algorithms
• If not, squeeze performance out of super-polynomial algorithms
• Accept “good enough” solutions to problems