Analysis and Design
DESIGN AND ANALYSIS OF COMPUTER
ALGORITHMS
Algorithm
An algorithm is a finite set of instructions that, if
followed, accomplishes a particular task. In addition, all
algorithms must satisfy the following criteria
1. Input: - Zero or more quantities are externally
supplied.
2. Output: - At least one quantity is produced.
3. Definiteness: - Each instruction is clear and
unambiguous.
4. Finitess: - If we trace out the instructions of an
algorithm, then for all cases, the algorithm terminates after a
finite number of steps.
5. Effectiveness: - It must be possible to carry out
every step of the algorithm.
THE BASIC STEPS IN THE COMPLETE
DEVELOPMENT OF AN ALGORITHM
The development of an algorithm involves the following
steps.
1. Statement of the problem
For writing algorithm for the problem, we must be
able to give it precise statement.
1
Analysis and Design
2. Development of a model
Once a problem has been clearly stated, it is time to
formulate it as a mathematical model. This is a very
important step in overall solution process. The choice of a
model has substantial influence on the remainder of the
solution process.
3. Design of an algorithm
Once a problem has been clearly stated and a model
has been developed, we must design an algorithm for
solving the problem, the choice of design technique, which
is highly dependent on the choice of the model, can greatly
influence the effectiveness of an algorithm, two different
algorithms may be correct, but may tremendously in their
effectiveness.
4. Correctness of the algorithm
The most common procedure followed to assert that a
program is correct is to run it on variety of test cases. If the
answers produced by the program can be verified against
hand calculations or known values.
6. Implementation
Once an algorithm has been stated, in terms of a sequence
of steps, and one is convinced that it is correct, it is time to
implement the algorithm. That is to code it into a computer
program.
2
Analysis and Design
This step can be quite hard. One reason for this
difficulty is that after a particular step of an algorithm will
be stated in a form that is not directly translatable into code.
Another reason is that before we can begin to write
code, we must design an entire system of computer data
structures to represent important aspects of the model being
used. To do this we must answer such questions as the
following.
1. What are the variables?
2. What are their types?
3. How many arrays, and of what size are needed.
4. What subroutines are needed?
5. What programming languages should be used?
Etc……
The particular implementation used can significantly affect
both the memory requirements and the speed of the
algorithm.
5 Analysis and complexity of the algorithm
Practical reason for analyzing algorithms is that we
need to obtain estimates or bounds on the storage or rum
time, which our algorithm will need to successfully process
a particular input. Our objective is to produce an algorithm,
which requires minimum computer time and memory.
There are also important theoretical reasons for
analyzing algorithm. One would like to have some
quantitative standard for comparing two algorithms, which
claim to solve the same problem. The weaker algorithm
should be improved or discarded.
3
Analysis and Design
6. Program testing
Testing and debugging refer to the tasks of detecting
and removing errors in a program, so that the program
produces the desired result in all occasions.
7. Documentation
The main reason for documentation is to enable
individuals to understand programs which they don’t write.
The best way to do this is to write program that is so clear
and self-documenting.
***********************************
****************
The study of algorithms includes many important and
active areas of research. There are four distinct areas of
study one can identify.
1. How to devise algorithms
Creating an algorithm is an art, which may never
be fully automated. There are various design techniques
that have proves to be useful in that they have after prod
ceded good algorithms. By mastering their design
strategies, it will become easier to devise new and useful
algorithms.
4
Analysis and Design
2. How to validate algorithms
Once an algorithm is devised it is necessary to
show that it compute the correct answer for all possible
inputs. We refer to this process as algorithm validation.
Once the validity of the method has been shown, a
program can be written and a second phase begins. This
phase is referred to as program proving or sometimes as
program verification.
3. How to analyze algorithm
As an algorithm is executed, it uses the
computers CPU. Analysis of algorithms or performance
analysis refers to the task of determining how much
computing time and storage an algorithm requires.
4. How to test programs
Testing a program consist if two phases,
debugging and profiling (or performance measurement).
Debugging is the process of executing programs as
sample data sets to determine whether faulty results
occur and, if so correct them.
Performance analysis
Performance evaluation can be totally divided into two
major phases
1. Performance analysis.
2. Performance measurement
5
Analysis and Design
SPACE COMPLEXITY
The space complexity of an algorithm is the
amount of memory it needs to run to completion.
The space needed by each of these algorithms is the
sum of the following components.
1. A fixed part that is independent of two characteristics
(eg number, size)of the inputs and outputs. This part
includes the instruction space(ie the space for the
code), space for simple variables,space for constants,
and so on.
2. A variable part that consists of the space needed by
component variables whose size is dependent on the
particular problem instance being solved, the space
needed by referenced variables (to the extent that this
depends on instance characteristics), and the recursion
stack space (in so far as this space depends on the
instance characteristics)
The space requirement S(P) of any algorithm P
may therefore be written as S(P)=C + Sp(instance
characteristics), where C is a constant.
When analyzing the space complexity of an
algorithm, we concentrate on estimating Sp
Example 1
1 Algorithm abc(a,b,c)
6
Analysis and Design
2 {
3 return a+b+b*c+(a+b-c)/(a+b)+40;
4 }
Variables are a,b,c making the assumption that one
word is adequate to store the values of each of a,b,c. the
apace needed by a,b,c is independent of the instance
characteristics.
Therefore Sp=0
Example 2
7
Analysis and Design
Algorithm sum(a,n)
{
s:=0;
for I:=1 to n do
{
s:=s+a[I];
}
return s;
}
The problem instance of this algorithm is characterized by
n, the number of elements to be summed. The space needed
by n is one word. The space needed by is the space needed
by variables of type array of floating point numbers. This is
at least n word. So we obtain Ssum(n) >= (n+3) ( n for a[],
one each for n,i, and s).
Example 3
Algorithm Rsum(a,n)
8
Analysis and Design
{
if(n<=0)then return 0.0;
else
return Rsum(a,n-1)+a[n];
}
Recursive function for sum
In this algorithm instance are characterized by n. The
recursion stack space includes space for formal parameters,
the local variables, and the return address. Assume that the
return address requires only one word of memory. Each call
to Rsum requires at least three words (including space for
the values of n, the return address, and a pointer to a[]).
Since the depth of recursion is n+1, the recursion stack
space needed is>3(n+1).
TIME COMPLEXITY
The time complexity of an algorithm is the amount of
computer time it needs to run to completion.
The time T (p) taken by a program P is the sum to
compile time and the run time. The compile time does not
depend on the instance characteristics. Also we may
assume that a compiled program will be run several times
without recompilation. So consequently we concern
ourselves with just the run time of a program. This run time
is denoted by tp(instance characteristic).
9
Analysis and Design
We can determine the number of steps needed by a
program to solve a particular program instance in one of
two ways.
First Method
In the first method, we introduce a new variable, count
, into the program. This is a global variable with initial
value 0.statements to increment count by the appropriate
amount as introduced in to the program. This is done so
that each line a statement in the original program is
executed, count is incremented by the step count of that
statement.
Example;
Algorithm sum (a,n)
{
S: =0.0;
Count: =count+1;
For i: =1 to n do
{
Count: =count+1;
S: =s+a[i];
Count: =count+1;
}
Count: =count+1;
Count: =count+1;
Return s;
}
10
Analysis and Design
Algorithm sum(a,n)
{
For i: =1 to n do
{
Count: =count+2;
}
Count: =count+3;
}
In for loop the value of count will increase by a total
of 2n.If initial value of count is zero to start with,
and then it will be 2n+3 on terminations. So each
invocation of sum executes a total of 2n+3 steps.
11
Analysis and Design
Example 2:
Algorithm Rsum(a,n)
{
Count: =count+1;
If (n<=0) then
{
Count: =count+1;
Return 0.0;
}
Else
{
Count: =count+1;
Return Rsum(a,n-1)+a[n];
}
}
In example 2 tRsum(0)=2. When n>0 count increases by 2
plus whatever increase results from the invocation of
tRsum from within the else clause. From the definition of
tRsum, it follows that this additional increase is tRsum(n-
1). So if the value of count is zero initially, its value at the
time of termination is 2 + tRsum(n-1), n>0.
When analyzing a recursive program for its step count. We
often obtain a recursive formula for the step count, for e.g.
tRsum(n) = { 2 if n=0
2 + tRsum if n>0
The recursive formula is referred to as recurrence relations.
One way of solving any such recurrence relation is to make
repeated substitution for each occurrence of the function
12
Analysis and Design
tRsum on the right-hand side until all such occurrence
disappear.
tRsum(n) = 2 + tRsum(n-1)
= 2 + 2 + tRsum(n-2)
= 2(2)+ + tRsum(n-2)
.
.
.
= n(2) + tRsum(0)
= 2n + 2 n>=0
So the step count for tRsum is 2n + 2.
The step count is useful in that it tells us hoe the run time of
a function changes with the change in the instance
characteristics. From the step count for sum, we see that if
n is doubled, the runtime also doubles (approximately). If n
increases by a factor of 10, the run time increases by a
factor of 10 and so on. So the runtime grows linearly in n.
We say that sum is a linear line algorithm.
Second method
The second method to determine the step count of an
algorithm is to build a table in which we list the total
number of steps contributed by each statement. For finding
this first determining the number of steps p<r execution
(s/c) of the statement and the total number of times (i.e
frequency) each statement is executed. The s/c of a
statement is the amount by which the count changes as a
13
Analysis and Design
result of the execution of that statement. By combining
these two quantities, the total contribution of each
statement is obtained. By adding the contribution of all
statements, the step count of the entire algorithm is
obtained.
Example
Statement s/e Frequency Total
steps
Algorithm sum(a,n) 0 _______ 0
{ 0 _______ 0
s:=0.0; 1 1 1
for(i=1 to n) do 1 n+1 n+1
a:=s+a[i]; 1 n n
return s; 1 1 1
} 0 ______ 0
Total 2n+3
Statement s/e Frequency Total steps
14
Analysis and Design
n=0 n>0 n=0 n>0
Algorithm Rsum(a,n) 0
{
if(n<=0) 1 1 1 1 1
return 0.0; 1 1 0 1 0
else
returnRsum(a,n-+a[n]); 1+x 0 1 0 1+x
0 0 0
}
Total 2 2+x
X=trsum(n-1)
Best case, Worst case, Average case
There are three kinds of step counts. Best case, worst case
and average.
The best-case step count is the minimum number of steps
that can be executed for the given parameters.
The worst-case step count is the maximum number of
steps that can be executed for the given parameters.
The average case step count is the average number of
steps executed on instance with the given parameters.
15
Analysis and Design
16
