Center of Biomedical Engineering

Data Structures and Algorithms

(BMED – 3112)
Contents
•Algorithm and its analysis
•Diffrenet types of algorithms
•Big O-notation
•Complexity
•Growth of fucntions
•Halting problem

2
 Algorithms
• What is an algorithm?
• An algorithm is a finite set of precise instructions for performing a
computation or for solving a problem.

3
Properties of algorithms
• Input from a specified set,
• Output from a specified set (solution),
• Definiteness of every step in the computation,
• Correctness of output for every possible input,
• Finiteness of the number of calculation steps,
• Effectiveness of each calculation step and
• Generality for a class of problems.

4
 Types of Algorithms
• Brute-force Algorithm
• It is the approach that comes first to mind when solving a problem.
• It is the most basic approach to solving a problem.
• The brute force solution is simply to calculate the total distance for
every possible route and then select the shortest one

• Searching Algorithm
• The searching algorithm searches for a particular element or
a set of elements in a data structure.
• It can be linear search, binary search or any other search algorithm.

• Sorting Algorithm
• Sorting algorithms sort a data structure according to a set of rules.
• We use them to sort elements in increasing or decreasing order
• These include bubble sort, insertion sort, and more.

5
 Cont....
• Recursive Algorithm
• The recursive algorithm uses the concept of recursion.
• It breaks a problem into more minor problems and calls the same function repeatedly for those
sub-problems.
• It continues to do so until it reaches a base case that terminates the function provided the
given conditions are met.

6
 Cont...
• Backtracking Algorithm the time complexity of O(n factorial)
• Backtracking algorithm searches for all the possible solutions.
• For each path, we backtrack to the first failure point of that path and
then search again for the possible solution in the next path until we
find the desired solution.
• The algorithm takes more amount of time.
• Divide and Conquer Algorithm
• Divide and Conquer algorithms divide a problem set into smaller subsets, solve each separately, and
then combine them for the final solution of the main problem. These include merge sort, quick sort,
and more.

7
 Cont...
• Greedy Algorithm
• Greedy algorithms look for the best possible solution at each step and continue with that to
obtain the final solution.
• The solutions obtained by this algorithm may or may not be optimal, but it computes
solutions relatively quickly. These include algorithms like topological sort, selection sort, and
more.

8
Cont....
• Dynamic Programming Algorithm
• A Dynamic Programming algorithm uses the already computed results to avoid the repetition
of the same calculations. This algorithm guarantees an optimal solution to the problem.
• In the same example as discussed above, a solution following a dynamic programming
algorithm will give the right path with cost minimization. The answer will be the path with the
cost of 12.
• Example: Consider the problem of finding the Fibonacci sequence:
• Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, …

9
Cont...
• Randomized Algorithm
• A randomized algorithm employs random integers to choose what action to take at any point
in its logic. Due to the added randomness, the output can be different for the same inputs on
different runs.
• It is commonly used in cryptographic algorithms to ensure safety. It also helps you avoid
worst-case scenarios that a deterministic algorithm may run into. For example, randomized
quick sort performs better than conventional quick sort in the worst-case scenario.

10
 Algorithm Examples
•We will use a pseudocode to specify algorithms, which slightly
reminds us of basic and pascal.
•Example: an algorithm that finds the maximum element in a
finite sequence

•Procedure:
max(a1, a2, …, an: integers)
max := a1
for i := 2 to n
if max < ai then max := ai
{max is the largest element}

11
 Cont...
•Another example:
• A linear search algorithm, that is an algorithm that linearly
searches a sequence for a particular element.
• Procedure:
• linear_search(x: integer; a1, a2, …, an: integers)
• i := 1
• while (i  n and x  ai)
• i := i + 1
• if i  n then location := i
• else location := 0
• {location is the subscript of the term that equals x, or is zero if x is
not found}

12
 Cont....
• If the terms in a sequence are ordered, a binary search algorithm is
more efficient than linear search.
• The binary search algorithm iteratively restricts the relevant search
interval until it closes in on the position of the element to be located.

13
Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

14
Algorithm Examples

15
Algorithm Examples

16
Algorithm Examples

17
Algorithm Examples

18
cont...
•Procedure binary_search(x: integer; a1, a2, …, an:
integers)
•i := 1 { i is left endpoint of search interval }
•j := n { j is right endpoint of search interval }
•while (i < j)
•begin
• m := (i + j)/2
• if x > am then i := m + 1
• else j := m
•end
•if x = ai then location := i
•else location := 0
•{location is the subscript of the term that equals x, or is
zero if x is not found}
19
 Advantages of Algorithm
• Efficiency: Algorithms provide systematic and optimized approaches to solving
problems, ensuring that solutions are obtained with minimal time and resource
usage.
• Reproducibility: Algorithms are well-defined and deterministic, producing the
same output for a given set of inputs consistently. This reproducibility ensures
reliable results and easy verification.
• Scalability: Well-designed algorithms can handle large problem sizes, making
them suitable for various applications, including big data processing and complex
computations.
• Abstraction: Algorithms provide an abstraction over complicated processes by
describing them with a set of clear and concise instructions, which makes it easy
for you to understand them.

20
Disadvantages of Algorithm
• Complexity: Some algorithms can be complex and difficult to understand or implement.
• Time-Consuming: Developing and optimizing algorithms can be time-intensive.
• Space Consumption: Certain algorithms require significant memory, affecting
performance.
• Problem-Specific: Algorithms are often tailored to specific problems and may not be
easily adaptable.
• Maintenance: Algorithms may require frequent updates and maintenance as
requirements change.
• Efficiency: Not all algorithms are efficient; some may have high time or space
complexity.
• Scalability: Some algorithms may not scale well with increased data or input size.

21
 Algorithm Complexity
• The characteristics of an algorithm help it achieve desired results, but the performance of an
algorithm is determined by its complexities. The complexities are namely time complexity and
space complexity.
• Time Complexity
• It is the amount of time an algorithm needs to run entirely, depending on the size of the input
given
• There are asymptotic notations used for determining the time complexity of an algorithm.
Generally, the time complexity of an algorithm is determined using big O notation.

• Space Complexity
• It is the measure of the memory required by an algorithm to run entirely, depending on the
size of the input given
• It is also determined using big O notation.

22
 Cont...
•In general, we are not so much interested in the time and
space complexity for small inputs.

•For example, while the difference in time complexity between

linear and binary search is meaningless for a sequence with n =
10, it is gigantic for n = 230.

23
Complexity
• For example, let us assume two algorithms A and B that
solve the same class of problems.
• The time complexity of A is 5,000n, the one for B is
1.1n for an input with n elements.
• For n = 10, A requires 50,000 steps, but B only 3, so B
seems to be superior to A.
• For n = 1000, however, A requires 5,000,000 steps,
while B requires 2.5 x 1041 steps.
• This means that algorithm B cannot be used for large
inputs, while algorithm A is still feasible.

24
 Complexity
• So, what is important is the growth of the complexity
functions.

• The growth of time and space complexity with

increasing input size n is a suitable measure for the
comparison of algorithms.

to compare aligorthim faasible increasing input size

25
• Comparison: time complexity of algorithms A and B.

Input Size Algorithm A Algorithm B

n 5,000n 1.1n

10 50,000 3

100 500,000 13,781

1,000 5,000,000 2.5 x 1041

1,000,000 5x109 4.8 x 1041392

26
 The Growth of Functions
• The growth of functions is usually described using the big-
O notation.

• Definition: Let f and g be functions from the integers or

the real numbers to the real numbers.
• We say that f(x) is O(g(x)) if there are constants C and k
such that
•|f(x)|  C|g(x)|
•whenever x > k.

27
 Cont...
• When we analyze the growth of , f(x)
and g(x) are always positive.
• Therefore, we can simplify the big-O requirement to
• f(x)  Cg(x) whenever x > k.

• If we want to show that f(x) is O(g(x)), we only need to find

pair (C, k) (which is never unique).

28
 Cont....
•The idea behind the big-O notation is to establish an upper
boundary for the growth of a function f(x) for large x.
•This boundary is specified by a function g(x) that is usually
much simpler than f(x).
•We accept the constant C in the requirement
•f(x)  Cg(x) whenever x > k,
•because C does not grow with x.
•We are only interested in large x, so it is OK if
f(x) > Cg(x) for x  k.

29
 Cont....
Example:
Show that f(x) = x2 + 2x + 1 is O(x2).

• For x > 1 we have:

• x2 + 2x + 1  x2 + 2x2 + x2
 x2 + 2x + 1  4x2
Therefore, for C = 4 and k = 1:
• f(x)  Cx2 whenever x > k.

 f(x) is O(x2).

30
 Cont...

• Question: If f(x) is O(x2), is it also O(x3)?

• Yes. x3 grows faster than x2, so x3 grows

also faster than f(x).

• Therefore, we always have to find the

smallest simple function g(x) for which f(x)
is O(g(x)).

31
 The Growth of Functions
• “Popular” functions g(n) are nlog n, 1, 2n, n2, n!, n, n3, log n

•Listed from slowest to fastest growth:

• 1
• log n
• n
• nlogn
• n2
• n3
• 2n
• n!
32
 The Growth of Functions

•A problem that can be solved with polynomial worst-case

complexity is called tractable.

•Problems of higher complexity are called intractable.

• These problems are theoretically impossible to solve in a reasonable time — i.e.,
there are known algorithmic solutions, but the algorithms are too inefficient/slow to
solve the problem when the number of inputs grows large.
• These problems may be used to protect us: In cryptography, password protection

• Problems that no algorithm can solve are called unsolvable.

33
34
 Useful Rules for Big-O
• For any polynomial f(x) = anxn + an-1xn-1 + … + a0, where a0, a1, …,
an are real numbers,
• f(x) is O(xn).

• If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then

(f1 + f2)(x) is O(max(g1(x), g2(x)))

• If f1(x) is O(g(x)) and f2(x) is O(g(x)), then

(f1 + f2)(x) is O(g(x)).

• If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then

(f1f2)(x) is O(g1(x) g2(x)).
35
 Complexity Examples
•What does the following algorithm compute?
•procedure who_knows(a1, a2, …, an: integers)
• m := 0
• for i := 1 to n-1
• for j := i + 1 to n
• if |ai – aj| > m then m := |ai – aj|
• {m is the maximum difference between any two numbers in
the input sequence}
• Comparisons: n-1 + n-2 + n-3 + … + 1
= (n – 1)n/2 = 0.5n2 – 0.5n

•Time complexity is O(n2).

36
 Complexity Examples
•Another algorithm solving the same problem:
•procedure max_diff(a1, a2, …, an: integers)
•min := a1
•max := a1
•for i := 2 to n
• if ai < min then min := ai
• else if ai > max then max := ai
•m := max - min
•Comparisons: 2n - 2
•Time complexity is O(n).

37
 Thank you!!

Next Session:
Chapter 5: Sorting and Searching Algorithms