DESIGN AND ANALYSIS OF ALGORITHMS K.PALRAJ M.E.,(Ph.d) Assistant Professor Vtu networkcs6402 DESIGN AND ANALYSIS OF ALGORITHMS LTPC 3003 OBJECTIVES: The student should be made to: * Leam the algorithm analysis techniques. «Become familiar with the different algorithm design techniques. + Understand the limitations of Algorithm power. UNIT INTRODUCTION 3 Notion of an Algorithm - Fundamentals of Algorithmic Problem Solving — Important Problem Types — Fundamentals of the Analysis of Algorithm Efficiency — Analysis Framework - Asymptotic Notations and its properties - Mathematical analysis for Recursive and Non-recursive algorithms. UNIT Il BRUTE FORCE AND DIVIDE-AND-CONQUER 9 Brute Force - Closest-Pair and Convex-Hull Problems-Exhaustive Search - Traveling Salesman Problem - Knapsack Problem - Assignment problem. Divide and conquer methodology ~ Merge sort - Quick sort - Binary search — Multiplication of Large Integers — Strassen’s Matrix Multiplication-Closest-Pair and Convex-Hull Problems. UNIT Ill DYNAMIC PROGRAMMING AND GREEDY TECHNIQUE 9 Computing a Binomial Coefficient - Warshall’s and Floyd’ algorithm - Optimal Binary Search Trees — Knapsack Problem and Memory functions. Greedy Technique— Prim’s algorithm. Kruskal’s Algorithm-Dijkstra’s Algorithm-Huffman Trees. UNIT IV ITERATIVE IMPROVEMENT 9 The Simplex Method-The Maximum-Flow Problem ~ Maximmm Matching in Bipartite Graphs: The Stable marriage Problem. UNIT V COPING WITH THE LIMITATIONS OF ALGORITHM POWER 9 Limitations of Algorithm Power-Lower-Bound Arguments-Decision Trees-P, NP and NP- Complete Problems-Coping with the Limitations - Backtracking — n-Queens problem - Hamiltonian Circuit Problem — Subset Sum Problem-Branch and Bound — Assignment problem - Knapsack Problem ~ Traveling Salesman Problem- Approximation Algorithms for NP — Hard Problems — Traveling Salesman problem — Knapsack problem. TOTAL: 45 PERIODS OUTCOMES: At the end of the course, the student should be able to: * Design algorithms for various computing problems. + Analyze the time and space complexity of algorithms. ‘Critically analyze the different algorithm design techniques for a given problem. ‘+ Modify existing algorithms to improve efficiency. TEXT BOOK: 4. Anany Levitin, “Introduction to the Design and Analysis of Algorithms’, Third Edition, Pearson Education, 2012. REFERENCES: 4, Thomas H.Cormen, Charles E.Leiserson, Ronald L. Rivest and Clifford Stein, “Introduction to Algorithms”, Third Edition, PHI Learning Private Limited, 2012, 2. Alfred V. Aho, John E. Hopcroft and Jeffrey D. Ullman, “Data Structures and Algorithms’, Pearson Education, Reprint 2006. 3. Donald £. Knuth, “The Art of Computer Programmin Education, 2009. 4. Steven S. Skiena, “The Algorithm Design Manual”, Second Edition, Springer, 2008. 5. http:/inptel.ac:i Volumes 1& 3 PearsonTABLE OF CONTENTS UNIT-I INTRODUCTION 1.1 Notion of an Algorithm LL 1.2 Fundamentals of Algorithmic Problem Solving 13 1.3. Important Problem Types 16 1.4 Fundamentals of the Analysis of Algorithm Efficiency 18. 1.5 Analysis Framework 18 1.6 Asymptotic Notations and its properties 1.10 1.7. Mathematical analysis for Recursive algorithms 14 1.8 Mathematical analysis for Non-recursive algorithms 17 UNIT Il BRUTE FORCE AND DIVIDE-AND-CONQUER 2.1 Brute Force 21 2.2. Closest-Pair and Convex-Hull Problems 23 2.3 Exhaustive Search 26 2.4. ‘Traveling Salesman Problem 26 2.5. Knapsack Problem 27 2.6 Assignment problem 2.9 2.7. Divide and conquer methodology 2.10 2.8 Merge sort 2.10 2.9 Quick sort 212 2.10 Binary search 2.14 2.11 Multiplication of Large Integers 2.16 2.42. Strassen’s Matrix Multiplication 218 2.13 Closest-Pair and Convex-Hull Problems 2.20 UNIT III DYNAMIC PROGRAMMING AND GREEDY TECHNIQUE 3.1 Computing a Binomial Coefficient 3.2 Warshall’s and Floyd? algorithm 3.3 Optimal Binary Search Trees 37 3.4. Knapsack Problem and Memory functions 31 3.5. Greedy Technique 3.14 3.6 Prim’s algorithm 3.15 3.7. Kruskal's Algorithm 318 3.8. Dijkstra's Algorithm 3.20 3.9 Huffman Trees 3.21UNIT IV ITERATIVE IMPROVEMENT 4.1. The Simplex Method 4.2. The Maximum-Flow Problem 4.3. Maximum Matching in Bipartite Graphs 4.4 The Stable marriage Problem. 4. 4.10 4.17 4.19 UNIT V COPING WITH THE LIMITATIONS OF ALGORITHM POWER 5.1 Limitations of Algorithm Power 5.2. Lower-Bound Arguments 5.3 Decision Trees 5.4 P, NP and NP-Complete Problems 5.5 Coping with the Limitations 5.6 Backtracking 5.7 N-Queens problem 5.8 Hamiltonian Circuit Problem 5.9 Subset Sum Problem 5.10 Branch and Bound 5.11 Assignment problem 5.12, Knapsack Problem 5.13, Traveling Salesman Problem 5.14 Approximation Algorithms for NP —Hard Problems 5.15 Traveling Salesman problem 5.16 Knapsack problem. 5. Sl 353 5.6 5.10 5.1L 5.12 5.15, 5.16 5.18 5.19 5.21 5.22 5.24 5.25 5.27Design and Analysis of Algorithms Unit | An algorithm is a sequence of unambiguous instructions for solving a problem, ie., for obtaining a required output for any legitimate input in a finite amount of time. Problem to be solved a Algorithm, Computer Program FIGURE 1.1 The notion of the algorithm. Ivis a step by step procedure with the input to solve the problem in a finite amount of time to obtain the required output. The notion of the algorithm illustrates some important points: The non-ambiguity requirement for each step of an algorithm cannot be compromised. ‘The range of inputs for which an algorithm works has to be specified carefully. The same algorithm can be represented in several different ways. There may exist several algorithms for solving the same problem. Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds. Characteristics of an algorithm: Input: Zero / more quantities are externally supplied. Outpu At least one quantity is produced. Definiteness: Each instruction is clear and unambiguous, Finiteness: If the instructions of an algorithm is traced then for all cases the algorithm must terminates after a finite number of steps. Efficiency: Every instruction must be very basic and runs in short time,36402 __Design and Analysis of Algorithms Unit | Steps for writing an algorithm: 1. An algorithm is a procedure. It has two parts; the first part is head and the second part is body. . The Head section consists of keyword Algorithm and Name of the algorithm with parameter list. E.g. Algorithm namel(pl, p2,....p3) ‘The head section also has the following: Problem Descript /nput: HOutput: In the body of an algorithm various programming constructs like if, for, while and some statements like assignments are used. |. The compound statements may be enclosed with { and } brackets. if, for, while can be closed by endif, endfor, endwhile respectively. Proper indention is must for block. Comments are written using // at the beginning. . The identifier should begin by a letter and not by digit. It contains alpha numeric letters after first letter. No need to mention data types. The left arrow “<—" used as assignment operator. E.g. v—10 Boolean operators (TRUE, FALSE), Logical operators (AND, OR, NOT) and Relational operators (<<=, >, >= =, 4, <>) are also used. Input and Output can be done using read and write. 10, Array[], if then else condition, branch and loop can be also used in algorithm. Example: The greatest common divisor(GCD) of two nonnegative integers m and n (not-both-zero), denoted ged(m, n), is defined as the largest integer that divides both m and n evenly, ie., with a remainder of zero. Euclid’s algorithm is based on applying repeatedly the equality ged(m, n) = ged(n, m mod n), where m mod nis the remainder of the division of m by n, until m mod n is equal to 0. Since ged(m, 0) =m, the last value of m is also the greatest common divisor of the initial m and n. ged(60, 24) can be computed as follows:gcd(60, 24) = ged(24, 12) = ged(12, 0) = 12. Euclid’s algorithm for computing ged(m, n) in simple steps Step 1 If 7 = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2. Step 2 Divide m by n and assign the value of the remainder to r. Step 3 Assign the value of 7 to m and the value of r to n. Go to Step 1. Euclid’s algorithm for computing ged(m, n) expressed in pseudocode ALGORITHM Euclid_ged(m, n) 1/Computes ged(m, n) by Euclid’s algorithm. /MInput: Two nonnegative, not-both-zero integers m and n 1/Output: Greatest common divisor of m and n while n 4 0-do rem mod n mon ner return m(56402 Design and Analysis of Algorithms Unit | 1.2 FUNDAMENTALS OF ALGORITHMIC PROBLEM SOLVING ‘A sequence of steps involved in designing and analyzing an algorithm is shown in the figure below. Decide on ‘algorithm design technique Design an FIGURE 1.2 Algorithm design and analysis process. @ Understanding the Problem This is the first step in designing of algorithm, Read the problem’s description carefully to understand the problem statement completely. Ask questions for clarifying the doubts about the problem: Identify the problem types and use existing algorithm to find solution, Input (instance) to the problem and range of the input get fixed. Gi) Decision making jon making is done on the following: (a) Ascertaining the Capabilities of the Computational Device © In random-access machine (RAM), instructions are executed one after another (The central assumption is that one operation ata time). Accordingly, algorithms designed to be executed on such machines are called sequential algorithms. In some newer computers, operations are executed concurrently, i.e., in parallel, Algorithms that take advantage of this capability are called parallel algorithms. Choice of computational devices like Processor and memory is mainly based on space and time efficiency (b) Choosing between Exact and Approximate Problem Solving © The next principal decision is to choose between solving the problem exactly or solving it approximately. An algorithm used to solve the problem exactly and produce correct result is called an exact algorithm. If the problem is so complex and not able to get exact solution, then we have to choose an algorithm called an approximation algorithm. ie., produces anDesign and Analysis of Algorithms Unit | 14 approximate answer. E.g., extracting square roots, solving nonlinear equations, and evaluating definite integrals. (©) Algorithm Design Techniques © An algorithm design technique (or “strategy” or “paradigm”) is a general approach to solving problems algorithmically that is applicable to a variety of problems trom different areas of computing. [Algorithms+ Data Structures = Programs Though Algorithms and Data Structures are independent, but they are combined together to develop program. Hence the choice of proper data structure is required before designing the algorithm. Implementation of algorithm is possible only with the help of Algorithms and Data Structures Algorithmic strategy / technique / paradigm are a general approach by which many problems can be solved algorithmically. E.g., Brute Force, Divide and Conquer, Dynamic Programming, Greedy Technique and so on. ii) Methods of Specifying an Algorithm ‘There are three ways to specify an algorithm. They are: a, Natural language b, Pseudocode ¢. Flowchart Algorithm Specification ko NI FIGURE 1.3 Algorithm Specifications Pseudocode and flowchart are the two options that are most widely used nowadays for specifying algorithms. a. Natural Language It is very simple and easy to specify an algorithm using natural language. But many times specification of algorithm by using natural language is not clear and thereby we get brief Example: An algorithm to perform addition of two numbers. Step 1: Read the first number, say a. Step 2: Read the first number, say b. Step 3: Add the above two numbers and store the result in c. Step 4: Display the result from c. Such a specification creates difficulty while actually implementing it. Hence many programmers prefer to have specification of algorithm by means of Pseudocode.36402 __Design and Analysis of Algorithms Unit | b. Pseudocode * Pseudocode is a mixture of a natural language and programming language constructs. Pseudocode is usually more precise than natural language. For Assignment operation left arrow “+”, for comments two slashes “//”,if condition, for, while loops are used ALGORITHM Swn(a.b) //Problem Description: This algorithm performs addition of two numbers //input: Two integers a and b //Output: Addition of two integers coat return ¢ This specification is more useful for implementation of any language. ¢. Flowchart In the earlier days of computing, the dominant method for specifying algorithms was a flowchart, this representation technique has proved to be inconvenient. Flowchart is a graphical representation of an algorithm. It is a a method of expressing an algorithm by a collection of connected geometric shapes containing descriptions of the algorithm’s steps ‘Symbols Example; Addition of a an¢ Start state Transition / Assignment Input the value of a Processing / Input read Input the value of b Input and Output deer ee Display the value of c Flow connectivity Stop stop Stop state FIGURE 1.4 Flowchart symbols and Example for two integer addition. (iv) Proving an Algorithm’s Correctness * Once an algorithm has been specified then its correctness must be proved. + An algorithm must yields a required result for every legitimate input in a finite amount of time.Design and Analysis of Algorithms Unit | 16 For example, the correctness of Euclid’s algorithm for computing the greatest common divisor stems from the correctness of the equality ged(m, n) = gcd(n, m mod 1). A common technique for proving correctness is to use mathematical induetion because an algorithm’s iterations provide a natural sequence of steps needed for such proof’. ‘The notion of correctness for approximation algorithms is less straightforward than it is for exact algorithms. The error produced by the algorithm should not exceed a predefined limit. (v) Analyzing an Algorithm For an algorithm the most important is efficiency. In fact, there are two kinds of algorithm efficiency. They are: Time efficiency, indicating how fast the algorithm runs, and Space efficiency, indicating how much extra memory it uses. ‘The efficiency of an algorithm is determined by measuring both time efficiency and space efficiency. So factors to analyze an algorithm are: Time efficiency of an algorithm Space efficiency of an algorithm Simplicity of an algorithm Generality of an algorithm (vi) Coding an Algorithm The coding / implementation of an algorithm is done by a suitable programming language like C, C++, JAVA. The transition from an algorithm to a program can be done either incorrectly or very inefficiently. Implementing an algorithm correctly is necessary. The Algorithm power should not reduced by inefficient implementation, Standard tricks like computing a loop’s invariant (an expression that does not change its value) outside the loop, collecting common subexpressions, replacing expensive operations by cheap ones, selection of programming language and so on should be known to the programmer. Typically, such improvements can speed up a program only by a constant factor, whereas a better algorithm can make a difference in running time by orders of magnitude. But once an algorithm is selected, a 10-50% speedup may be worth an effort. It is very essential to write an optimized code (efficient code) to reduce the burden of compiler, 1.3 IMPORTANT PROBLEM TYPES The most important problem types are: (@). Sorting Gi). Searching Gi), String processing (iv). Graph problems (¥). Combinatorial problems (vi). Geometric problems (vii). Numerical problemsDesign and Analysis of Algorithms Unit | () Sorting ‘The sorting problem is to rearrange the items of a given list in nondecreasing (ascending) order Sorting can be done on numbers, characters, strings or records, To sort student records in alphabetical order of names or by student number or by student grade-point average. Such a specially chosen piece of information is called a key. An algorithm is said to be in-place if it does not require extra memory, E.g., Quick sort A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input. (ii) Searching ‘© The searching problem deals with finding a given value, called a search key, in a given set. * Eg,, Ordinary Linear search and fast binary search. String processing A string is a sequence of characters from an alphabet. Strings comprise letters, numbers, and special characters; bit strings, which comprise zeros and ones; and gene sequences, which can be modeled by strings of characters from the four- character alphabet (A, C, G, T}. Itis very useful in bioinformatics. Searching for a given word in a text is called string matching (iv) Graph problems © A graph is a collection of points called vertices, some of which are connected by line segments called edges. © Some of the graph problems are graph traversal, shortest path algorithm, topological sort, traveling salesman problem and the graph-coloring problem and so on. (v) Combinatorial problems ‘These are problems that ask, explicitly or implicitly, to find a combinatorial object such as a permutation, a combination, or a subset that satisfies certain constraints. ‘A desired combinatorial object may also be required to have some additional property such sa maximum value or a minimum cost. In practical, the combinatorial problems are the most difficult problems in computing. ‘The traveling salesman problem and the graph coloring problem are examples of combinatorial problems (vi) Geometric problems © Geometric algorithms deal with geometric objects such as points, lines, and polygon * Geometric algorithms are used in computer graphics, robotics, and tomography. The closest-pair problem and the convex-hull problem are comes under this category. Numerical problems ate problems that involve mathematical equations, systems of equations, computing definite integrals, evaluating functions, and so on. ‘The majority of such mathematical problems can be solved only approximately.56404 _Design and Analysis of Algorithms Unit | 1.4 FUNDAMENTALS OF THE ANALYSIS OF ALGORITHM EFFICIENCY The efficiency of an algorithm can be in terms of time and space. The algorithm efficiency can be analyzed by the following ways. a. Analysis Framework. b. Asymptotic Notations and its properties. c. Mathematical analysis for Recursive algorithms 4d. Mathematical analysis for Non-recursive algorithms. 1.5 Analysis Framework ‘There are two kinds of efficiencies to analyze the efficiency of any algorithm. They are: © Time efficiency, indicating how fast the algorithm runs, and * Space efficiency, indicating how much extra memory it uses. ‘The algorithm analysis framework consists of the following: © Measuring an Input’s Size Units for Measuring Running Time Orders of Growth Worst-Case, Best-Case, and Average-Case Efficiencies (i) Measuring an Input’s Size * An algorithm’s efficiency is defined as a function of some parameter n indicating the algorithm's input size. In most cases, selecting such a parameter is quite straightforward. For example, it will be the size of the list for problems of sorting, searching, For the problem of evaluating a polynomial p(x) = ay" +. . . + do of degree n, the size of the parameter will be the polynomial’s degree or the number of its coefficients, which is larger by 1 than its degree. In computing the product of two nx 1 matrices, the choice of a parameter indicating an input size does matter. Consider a spell-checking algorithm. If the algorithm examines individual characters of its input, then the size is measured by the number of characters. In measuring input size for algorithms solving problems such as checking primality of a positive integer n. the input is just one number, ‘The input size by the number b of bits in the n’s binary representation is b=(log: n)+1 )) Units for Measuring Running Time Some standard unit of time measurement such as a second, or millisecond, and so on can be used to measure the running time of a program after implementing the algorithm. Drawbacks Dependence on the speed of a particular computer. Dependence on the quality of a program implementing the algorithm. «The compiler used in generating the machine code. + The difficulty of clocking the actual running time of the program. So, we need metric to measure an algorithm’s efficiency that does not depend on these extraneous factors. One possible approach is to count the number of times each of the algorithm's operations is executed. This approach is excessively difficult. The most important operation (+, -, *, /) of the algorithm, called the basic operation. Computing the number of times the basic operation is executed is easy. The total running time is determined by basic operations count.Design and Analysis of Algorithms Unit | (ii) Orders of Growth * A difference in running times on small inputs is not what really distinguishes efficient algorithms from inefficient ones. For example, the greatest common divisor of two small numbers, it is not immediately clear how much more efficient Euclid’s algorithm is compared to the other algorithms, the difference in algorithm efficiencies becomes clear for larger numbers only. For large values of n, itis the function’s order of growth that counts just like the Table 1.1, which contains values of a few functions particularly important for analysis of algorithms. TABLE 1.1 Values (approximate) of several functions important for analysis of algorithms logan n logon n 0 0 1 1 2 2 8 16 64 3 2.4610" | 64 33 3.310" | 10° 10° 4 6.4010" | 2.6610? | 4.110% 10° 6.6 6.610" | 10° 10° 10° 31 10 1.0+10° | __ 10° 10° 10" 10" 13 13+10°| 108 10" 10° [326107 | 17 17-10°| 10° 10 computation 10° 10° 20 20-10" [ 10 (iv) Worst-Case, Best-Case, and Average-Case Efficiencies Consider Sequential Search algorithm some search key K ALGORITHM SequentialSearch(Al0..n- 1}, K) Searches for a given value in a given array by sequential search ‘input: An array A[0..n- 1} and a search key K //Output: The index of the first element in A that matches K or -1 if there are no i matching elements i-0 while i no. Where t(n) and g(n) are nonnegative functions defined on the set of natural numbers. O = Asymptotic upper bound = Useful for worst case analysis = Loose bound gin) tin) FIGURE L.5 Big-oh notation: t (n) € O(g(n)). Example 2: Prove the assertions 100n + 5 € O(n’). Proof: 100n + 5 < 1001 + n (for all n> 5) Oln 101n? (on < n°) Since, the definition gives us a lot of freedom in choosing specific values for constants € and np. We have c=101 and n, Example 3: Prove the assertions 100n + 5 € O(n) Proof: 100n +5 < 1001+ Sn (for all n= 1) 051 100n +5 105n ie, t(n) Scg(n) <100n + 5 € O(n) with c=105 and ng=1 (ii) Q - Big omega notation A funetion t(n) is said to be in Q/g(n)), denoted t(n) € O(g(n)). if t(n) is bounded below by some positive constant multiple of g(n) for all large n, i.e, if there exist some positive constant ¢ and some nonnegative integer np such that t(n) = eg(n) for all n> no. Where (1) and g(n) are nonnegative functions defined on the set of natural numbers. Q= Asymptotic lower bound = Useful for best case analysis = Loose boundDesign and Analysis of Algorithms Unit | doesn't matter 7 FIGURE 1.6 Big-omega notation: 1 (n) € © (g(1). Example 4: Prove the assertions n'+10n"+4n+2 € Q (n°). Proof: n'+10n?+4n+2 > n° (for all n> 0) ice., by definition t(n) > cg(n), where e=1 and no=0 (iii) © - Big theta notation A function t(n) is said to be in ©(¢(n)), denoted t(n) € ©(g(n)), if t(n) is bounded both above and below by some positive constant multiples of g(n) for all large n, i.c., if there exist some positive constants cj and c; and some nonnegative integer no such that e2g(n) St (n) S cig(n) for all n> no. Where (1) and g(n) are nonnegative functions defined on the set of natural numbers. ©= Asymptotic tight bound = Useful for average case analysii igin) tin) a(n) doesn't matter [— FIGURE 1.7 Big-theta notation: t (n) € ©(g(n). Example 5: Prove the assertions $n (n= 1) € O(n’). Proof: First prove the right inequality (the upper bound) tee ye ee gn (n= 1) = 3? In < $n? forall n20 Second, we prove the left inequality (the lower bound): dn (n-1) = 3n?- En > En? - En] En) fora neDesign and Analysis of Algorithms Unit | na & in? tn (n—1) 2 3h i dee xx 3, S ip gms gn) < pn Hence, $n (n=1) € ©(n*), where c2= +, cl Note: asymptotic notation can be thought of as\“relational operators" for functions similar to the corresponding relational operators for values == 00, <= 00, =3.20, <= 00, >300) Useful Property Involving the Asymptotic Notations The following property, in particular, is useful in analyzing algorithms that comprise two consecutively executed parts. THEOREM: If ti(n) € O(g(n)) and t(n) € O(g2(n)), then ti(n) + H(n) € OCmax( gin), go(n)}). (The analogous assertions are true for the O and © notations as well.) PROOF: The proof extends to orders of growth the following simple fact about four arbitrary real numbers a), bj, a2, bo: if a; < by and a; ny. Similarly, since t(n) € O(g:(n)), {(n) < exg2(n) for all n> na, Let us denote cs = max{¢), ca} and consider n> max{n), m) so that we can use both inequalities. Adding them yields the following: tn) + 01m) < cigi(n) Feago(n) csgi(n) + esgx(n) cslgi(n) + go(n)] Ses2 max{gi(n), go(n)} Hence, ty(n) + t(n) € O(max{gi(n), go(n)}), with the constants ¢ and no required by the definition O being 2c; = 2 max{ cy, c2) and max{n;, n2}, respectively. ‘The property implies that the algorithm’s overall efficiency will be determined by the part with a higher order of growth, ie., its least efficient part. *. (a) € O(gi(n)) and b(n) € O(ex(n)), then ti(n) + t(n) € O(max{ gin), go(n)}). Basic rules of sum manipulation ‘Summation formulas DY l=4=14 1 whefe 1 | and O!= | by definition, we can compute F(n) = F(n ~ 1) + n with the following recursive algorithm. (ND 2015) ALGORITHM F(n) Computes n! recursively //nput: A nonnegative integer 1/Qutput: The value of n! ifn =0 return 1 else return F(n~ 1) * Algorithm analysis * For simplicity, we consider » itself'as an indicator of this algorithm’s input size. ie. 1. * The basic operation of the algorithm is multiplication, whose number of executions we denote M(n). Since the function F(1) is computed according to the formula F(n) = Fin —1)en for n >0. + The number of multiplications M(n) needed to compute it must satisfy the equality Min) = Mo) + 1 forn>0 To compute To multiply F(n-1)—F(n-a) byn M(n~ 1) multiplications are spent to compute F(n ~ 1), and one more multiplication is needed to multiply the result by 1. Recurrence relations The last equation defines the sequence M(n) that we need to find, This equation defines M(n) not explicitly, ie., as a function of n, but implicitly as a function of its value at another point, namely n~ I, Such equations are called recurrence relations ot recurrences. Solve the recurrence relation M(n) = M(n — 1) + 1, ie., to find an explicit formula for ‘M(n) in terms of n only. To determine a solution uniquely, we need an initial condition that tells us the value with which the sequence starts. We can obtain this value by inspecting the condition that makes the algorithm stop its recursive calls if n= 0 return 1. This tells us two things. First, since the calls stop when n = 0, the smallest value of n for which this algorithm is executed and hence M(n) defined is 0. Second, by inspecting. the pseudocode’s exiting line, we can see that when n =0, the algorithm performs no multiplications.Design and Analysis of Algorithms Unit | MO) = 0. the cats stop wnen n=0———f Lo mutations when n= 0 ‘Thus, the recurrence relation and initial condition for the algorithm’s number of multiplications M(n): M(n) =M(n- 1) + 1 for n>0, M@)=0 forn=0. Method of backward substitutions Min) =M(n—1) +1 substitute M(n~ 1) = M(n 2) +1 Min 2)+ 141 =M(n~2) +2 substitute Min ~ 2) = M(n 3) +1 M(n~3) + 1}+2 =M(n-3)+3 =M(n=i) +i =M(n-n)+n ‘Therefore M(n)=n EXAMPLE 2: consider educational workhorse of recursive algorithms: the Tower of Hanoi puzzle. We have n disks of different sizes that can slide onto any of three pegs. Consider A (source), B (auxiliary), and C (Destination). Initially, all the disks are on the first peg in order of size, the largest on the bottom and the smallest on top. The goal is to move all the disks to the third peg, using the second one as an auxiliary. FIGURE 1.8 Recursive solution to the Tower of Hanoi puzzle.56404 _Design and Analysis of Algorithms Unit | ALGORITHM TOH(, A, C. B) //Move disks from source to destination recursively iMnput: n disks and 3 pegs A,B, and C //Output: Disks moved to destination as in the source order. iftn=1 Move disk from A to C else Move top n-1 disks from A to B using C TOH(n - 1, A, B,C) Move top n-1 disks from B to C using A TOH(n - 1,B,C, A) Algorithm analysis ‘The number of moves M(n) depends on n only, and we get the following recurrence equation for it: M(n) = M(a~ 1) + 1+ M(n~ 1) forn > 1. With the obvious initial condition M(1) = 1, we have the following recurrence relation for the number of moves M(n}: M(n) =2M(n— 1) +1 forn > 1, M()=1 We solve this recurrence by the same method of backward substitutions Min) =2M(n=1)+1 sub, M(n=1)=2M(n~2) +1 [2M(n~2) + 1}+ 1 M(n—2)+241 sub, M(n ~ 2)=2M(n-3)+1 2M(n~ 3) + 1}+2+1 M(n- 3)+2?+2+1 sub, M(n — 2M(n—4) +1 ‘Min-4)+2424241 2M(n =i) + 2°" +2. +24 1=2M(n~i)+2'~1 Since the initial condition is specified for n = 1, which is achieved for i= n~ 1 M(n) = 2" 'M(n~ (n= 1)) +2" "= 1 = 2" 'MC) +2" ‘Thus, we have an exponential time algorithm EXAMPLE 3: An investigation of a recursive version of the algorithm which finds the number of binary digits in the binary representation of a positive decimal integer. ALGORITHM BinRec(n) ‘(Minput: A positive decimal integer n //Output: The number of binary digits in n’s binary representation ifn= 1 return 1 else return BinRec({n/2|)+ 1 Algorithm analysis The number of additions made in computing BinRec({n/2)) is A({n/2)), plus one more addition is made by the algorithm to increase the returned value by I. This leads to the recurrence A(n) = A(|n/2) +1 forn > 1 Since the recursive calls end when 7 is equal to 1 and there are no additions made56404 _Design and Analysis of Algorithms Unit | then, the initial condition is A(1) = 0. The standard approach to solving such a recurrence is to solve it only for n= 2" AQ = AQ) +1 fork>0, AQ) backward substitutions AG=AQTH +1 substitute Ak!) = AGE) +1 =[AQ*) + 1}4 12 AQ) +2 substitute A(2! AQ?) +1 = (AQ) + 12 = AQ) 43 =A) 4i =a) +k. ‘Thus, we end up with A(2‘) = A(I) + k =k, or, after returning to the original variable n = 2* and hence k = log. n, A(n) = log: n€ © (log> n). 1.8 MATHEMATICAL ANALYSIS FOR NON-RECURSIVE ALGORITHMS General Plan for Analyzing the Time Efficiency of Nonrecursive Algorithms 1. Decide on a parameter (or parameters) indicating an input's size. Identify the algorithm’s basic operation (in the innermost loop) Check whether the number of times the basic operation is executed depends only on the size ‘of an input. If it also depends on some additional property, the worst-case, average-case, and, if necessary, best-case efficiencies have to be investigated separately Set up a sum expressing the number of times the algorithm’s basic operation is executed. Using standard formulas and rules of sum manipulation either find a closed form formula for the count or at the least, establish its order of growth. EXAMPLE 1: Consider the problem of finding the value of the largest element in a list of n numbers. Assume that the list is implemented as an array for simplic ALGORITHM MaxElement(A[0..n~ 1]) Determines the value of the largest element in a given array ‘iMmput: An array A(0..n~ 1] of real numbers //Output: The value of the largest element in A maxval —A[0] fori 1 ton—1do if A{iJ>maxval maxval—A(i] return maxval Algorithm analysis * The measure of an input’s size here is the number of elements in the array # There are two operations in the for loop’s body ‘© The comparison Afi]> maxval and 0 The assignment maxval—A[ii]Design and Analysis of Algorithms Unit | 119 The comparison operation is considered as the algorithm's basic operation, because the comparison is executed on each repetition of the loop and not the assignment. ‘The number of comparisons will be the same for all arrays of size n; therefore, there is no need to distinguish among the worst, average, and best cases here. Let C(n) denotes the number of times this comparison is executed. The algorithm makes one comparison on each execution of the loop, which is repeated for each value of the loop’s variable i within the bounds 1 and n~ 1, inclusive. Therefore, the sum for C(n) is calculated as follows: mt em = 1 i.e., Sum up 1 in repeated n-1 times ni em) = 4 -1€@(n) & EXAMPLE 2: Consider the element uniqueness problem: check whether all the Elements in a given array of n elements are distinct. ALGORITHM UniqueElements(A[0..n~ 1]) Determines whether all the elements in a given array are distinet ‘/Mmput: An array A[0..n~ 1] //Output: Returns “true” if all the elements in A are distinet and “false” otherwise for iO ton-2do for | i+ 1 ton—1do if A{iJ= Aj | return false return true Algorithm analysis + The natural measure of the input’s size here is again n (the number of elements in the array) * Since the innermost loop contains a single operation (the comparison of two elements), we should consider it as the algorithm’s basic operation. * The number of element comparisons depends not only on n but also on whether there are ‘equal elements in the array and, if there are, which array positions they occupy. We will limit our investigation to the worst case only. * One comparison is made for each repetition of the innermost loop, ie, for each value of the loop variable j between its limits i+ 1 and n ~ 1; this is repeated for each value of the outer loop, ie, for each value of the loop variable i between its limits 0 and n—2 a2 nd 2 2 Coors) => > 1= Pla -)- G++ 1] = -1-4 = = inn 15 =a-1- = Oo Tt Onn). EXAMPLE 3: Consider matrix multiplication. Given two n x n matrices A and B, find the time efficiency of the definition-based algorithm for computing their product C = AB. By definition, C56404 _Design and Analysis of Algorithms Unit | 1.20 is an n xn matrix whose elements are computed as the scalar (dot) products of the rows of matrix A and the columns of matrix B A B rowi (CTT TTI col. j where Cli, j = Ali, OJB[O, j}+... +Afi, KIBIk, j}+ .. . + Afi, n- 1]B[n — 1, j] for every pair of indices 0 I do count count + 1 ne-{n/2) return count Algorithm analysis An input’s size is n ‘The loop variable takes on only a few values between its lower and upper limits, Since the value of n is about halved on each repetition of the loop, the answer should be about log: n. ‘The exact formula for the number of times. ‘The comparison n > 1 will be executed is actually [log» n] + 1