Algorithm Design and Analysis
Faculty of Information Technology - Computer Science Department 1
Algorithm Design and Analysis
Faculty of Information Technology - Computer Science Department 2
Chapter 1
Introduction to Algorithm
Faculty of Information Technology - Computer Science Department 3
Objectives
• The objectives of this chapter are:
Understand what do we mean by algorithms.
Understand the importance of Algorithms in problem-solving process.
Understand the correctness and efficiency of algorithms.
Study some simple computational algorithms
Faculty of Information Technology - Computer Science Department 4
Outline
• Introduction to Data Structures and Algorithms
• Why Study Algorithms?
• The Problem-solving Process
• Applications
• Computational problems
• Writing Algorithms
• Algorithms correctness and efficiency
• Analyzing Algorithms
Faculty of Information Technology - Computer Science Department 5
Introduction to Data Structures and Algorithms
• An algorithm is any well-defined computational procedure that takes some
value, or set of values, as input and produces some value, or set of values,
as output.
• An algorithm is a tool for solving a well-specified computational problem.
Faculty of Information Technology - Computer Science Department 6
Why Study Algorithms?
• Necessary in any computer programming problem
Improve algorithm efficiency: run faster, process more data, do something
that would otherwise be impossible
Solve problems of significantly large size
Technology only improves things by a constant factor
• Compare algorithms
• Algorithms as a field of study
Learn about a standard set of algorithms
Discoveries arise
Numerous application areas
• Learn techniques of algorithm design and analysis
Faculty of Information Technology - Computer Science Department 7
Why Study Algorithms?
Algorithm
Design Analysis
Methods/ ideas/ tricks Abstract/ mathematical computation
for developing (fast !) of algorithms
algorithms. (without actually implementing them)
• Mathematical are needed:
• To formally specify the problem
• Analysis of correctness
• Analysis of efficiency (time, memory use,…)
Faculty of Information Technology - Computer Science Department 8
The Problem-solving Process
Faculty of Information Technology - Computer Science Department 9
Applications
• Multimedia
CD player, DVD, MP3, JPG, DivX, HDTV
• Internet
Packet routing, data retrieval (Google)
• Communication
Cell-phones, e-commerce
• Computers
Circuit layout, file systems
• Science
Human genome
• Transportation
Airline crew scheduling, ARAMEX and DHL deliveries
Faculty of Information Technology - Computer Science Department 10
Computational problems
• The statement of the problem specifies in general terms the desired input/output
relationship.
• The algorithm describes a specific computational procedure for achieving that
input/output relationship.
• An instance of a problem consists of the input (satisfying whatever constraints are
imposed in the problem statement) needed to compute a solution to the problem.
• There might be different ways (algorithms) to solve a given problem!
• Usually, we use “pseudo-code” to describe algorithms
• Algorithms can be implemented in any programming language
Faculty of Information Technology - Computer Science Department 11
Roadmap
• Different problems • Different design paradigms
• Searching – Incremental
• Sorting – Divide-and-conquer
• Graph problems – Greedy algorithms
Faculty of Information Technology - Computer Science Department 12
Writing Algorithms
• Any algorithm should be written as:
Name: give an appropriate name
Input: value or set of values
Output: value or set of values
Algorithm:
• Step (usually steps) that can take you from the input to the output
• …
• ..
Faculty of Information Technology - Computer Science Department 13
Algorithm Example
• Example 1: Determining if a number is prime or not.
• Name: Prime
• Input: n (an integer number)
• Output: Prime or not prime
• Algorithm:
• Think of such an algorithm here.
• 2, 5, 11, and 17 are examples of prime numbers
• 4, 9, 15, and 22 are examples of non-prime numbers
Faculty of Information Technology - Computer Science Department 14
Algorithm Example
• Example 2:
• Name: ListSort
• Input: L (list of names of people)
• Output: L (same list sorted alphabetically). .
• Algorithm:
• Steps …
• We will see several algorithms in later chapters
Faculty of Information Technology - Computer Science Department 15
Algorithm Example
• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
initially sum is zero
Loop from 1 to n
Add current loop counter to the sum
output sum
Faculty of Information Technology - Computer Science Department 16
Algorithm Example
• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
sum = 0
for count =1 to n
{
sum = sum + count
}
output sum
Faculty of Information Technology - Computer Science Department 17
Algorithm Example
• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
count = 1
sum = 0
while (count <= n)
{
sum = sum + count
count = count + 1
}
output sum
Faculty of Information Technology - Computer Science Department 18
Algorithm Example
• Example 3:
• Name: SumOfNumbers
• Input: n (positive integer)
• Output: sum( sum of all positive integers from 1 to n)
• Algorithm:
count = 1
sum = 0
while (count <= n)
{
sum = sum + count
count = count + 1
}
output sum
Faculty of Information Technology - Computer Science Department 19
Algorithms correctness and efficiency
• An algorithm is said to be correct if, for every input instance, it halts with the proper
output
• An incorrect algorithm might not halt at all on some input instances, or it might halt
with an answer other than the desired one.
• Algorithms must be:
• Correct: For each input produce an appropriate output
• Efficient: run as quickly as possible, and use as little memory as possible – more
about this later
Faculty of Information Technology - Computer Science Department 20
Analyzing Algorithms
• Predict the amount of resources required:
• memory: how much space is needed?
• computational time: how fast the algorithm runs?
• FACT: running time grows with the size of the input
• Input size (number of elements in the input)
• Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary
representation of the input, vertices and edges in a graph
• Def: Running time = the number of primitive operations (steps) executed before termination
• Arithmetic operations (+, -, *), data movement, control, decision making (if, while),
comparison
Faculty of Information Technology - Computer Science Department 21
Algorithm Example
• Example 4: We need to solve a computational problem
“Convert a weight in pounds to Kg”
• Name: ConvertPoundToKg
• Input: weight-in-pounds (weight in pound)
• Output: weight-in-Kg (weight in Kg )
• Algorithm:
1. Read weight-in-pounds
2. Calculate weight-in-Kg = weight-in-pounds * 0.454
3. Print weight-in-Kg
Faculty of Information Technology - Computer Science Department 22
Algorithm Example (cont.)
OR
• Name: ConvertPoundToKg
• Input: x (weight in pound)
• Output: y (weight in Kg)
• Algorithm:
1. Read x
2. Calculate y = x * 0.454
3. Print y
Faculty of Information Technology - Computer Science Department 23
Incorrect Algorithm example
• Name: ConvertPoundToKg
• Input: weight-in-pounds (weight in pound)
• Output: weight-in-Kg (weight in Kg )
• Algorithm:
1. Read weight-in-pounds
2. Calculate weight-in-Kg = weight-in-pounds / 0.454
3. Print weight-in-Kg
Faculty of Information Technology - Computer Science Department 24
Inefficient Algorithm example
• Name: ConvertPoundToKg
• Input: weight-in-pounds (weight in pound)
• Output: weight-in-Kg (weight in Kg )
• Algorithm:
1. Read weight-in-pounds
2. weight-in-Kg = 0
3. Loop until weight-in-pounds =0
4. if weight-in-pounds > 1
5. weight-in-Kg = weight-in-Kg + 0.454
6. weight-in-pounds = weight-in-pounds -1
7. else
8. weight-in-Kg = weight-in-Kg + weight-in-pounds *0.454
9. weight-in-pounds =0
10. Print weight-in-Kg
Faculty of Information Technology - Computer Science Department 25
Another Simple Algorithm
Example 5:
• Name: FindMinInArray
• INPUT: T (a sequence of n numbers T[1], T[2], …, T[n])
• OUTPUT: min (the smallest number among them)
• Algorithm:
Faculty of Information Technology - Computer Science Department 26
Algorithm Example
• Example 1 again
Name: Prime
Input: n (an integer number)
Output: Prime or not prime
Algorithm:
Faculty of Information Technology - Computer Science Department 27
Greatest Common Divisor
• Definition: The GCD of two natural numbers x, and y is the largest integer j that divides both (without
remainder).
i.e. mod(x, j)=0, mod(y, j)=0, and j is the largest integer with this property.
• mod(a, b) gives the remainder of the division of a over b
• Examples:
GCD(8,12) = 4
GCD(12,8) = 4
GCD(10, 25) = 5
GCD(23, 19) = 1
Faculty of Information Technology - Computer Science Department 28
Greatest Common Divisor
Example 6: GCD Simple algorithm:
• Name: GreatestCommonDivisor1 (or GCD1)
• Input: x, y (two natural numbers)
• Output: x (Greatest Common Divisor of x and y)
• Algorithm:
Faculty of Information Technology - Computer Science Department 29
Euclid’s GCD Algorithm
Example 7: GCD Euclid’s algorithm
Name: GreatestCommonDivisor2 (or GCD2)
Input: x, y (two natural numbers)
Output: x (Greatest Common Divisor of x and y)
Algorithm:
while (y != 0)
{
t = mod(x, y)
x=y
y=t
}
Output x
Faculty of Information Technology - Computer Science Department 30
Euclid’s GCD Algorithm
OR
Name: GreatestCommonDivisor2 (or GCD2)
Input: x, y (two natural numbers)
Output: x (Greatest Common Divisor of x and y)
Algorithm:
GCD(x, y)
{
while (y != 0)
{
t = mod(x, y)
x=y
y=t
}
Output x
Faculty of Information Technology - Computer Science Department 31
}
Euclid’s GCD Algorithm
OR
Name: GreatestCommonDivisor2 (or GCD2)
Input: x, y (two natural numbers)
Output: x (Greatest Common Divisor of x and y)
Algorithm:
GCD(x, y)
{
if y = 0 return x
else return GCD( y, x mod y)
}
Faculty of Information Technology - Computer Science Department 32
