03/09/2020

Searching Elements in an Array:

Linear and Binary Search

Searching

• Check if a given element (called key) occurs in

the array.
– Example: array of student records; rollno can be the
key.
• Two methods to be discussed:
– If the array elements are unsorted.
• Linear search
– If the array elements are sorted.
• Binary search

1
 03/09/2020

Linear Search

Basic Concept

• Basic idea:
– Start at the beginning of the array.
– Inspect elements one by one to see if it matches the key.
• Time complexity:
– A measure of how long an algorithm takes to run.
– If there are n elements in the array:
• Best case: match found in first element (1 search operation)
• Worst case: no match found, or match found in the last element
(n search operations)
• Average case: (n + 1) / 2 search operations

2
 03/09/2020

#include <stdio.h>

int linear_search (int a[], int size, int key)

{
for (int i=0; i<size; i++)
if (a[i] == key) return i;
return -1;
}

int main()
{
int x[]={12,-3,78,67,6,50,19,10}, val;
printf (”\nEnter number to search: ”);
scanf (”%d”, &val);
printf (”\nValue returned: %d \n”, linear_search (x,8,val);
}
5

• What does the function linear_search do?

– It searches the array for the number to be searched
element by element.
– If a match is found, it returns the array index.
– If not found, it returns -1.

3
 03/09/2020

Contd.

int x[]= {12, -3, 78, 67, 6, 50, 19, 10};

• Trace the following calls :

search (x, 8, 6) ; Returns 4
search (x, 8, 5) ;

Returns -1

Binary Search

4
 03/09/2020

Basic Concept

• Binary search works if the array is sorted.

– Look for the target in the middle.
– If you don’t find it, you can ignore half of the array,
and repeat the process with the other half.
• In every step, we reduce the number of
elements to search by half.

The Basic Strategy

• What we want?
– Find split between values larger and smaller than key:
0 n-1
x: <=key >key
L R

– Situation while searching:

• Initially L and R contains the indices of first and last elements.
– Look at the element at index [(L+R)/2].
• Move L or R to the middle depending on the outcome of test.

10

5
 03/09/2020

Iterative Version
#include <stdio.h>

int bin_search (int a[], int size, int key)

{
int L, R, mid;
L = 0; R = size – 1;

while (L <= R) {
mid = (L + R) / 2;
if (a[mid] < key) L = mid + 1;
else if (a[mid] > key) R = mid -1;
else return mid; /* FOUND AT INDEX mid */
}

return -1; /* NOT FOUND */

}

11

int main()
{
int x[]={10,20,30,40,50,60,70,80}, val;

printf (”\nEnter number to search: ”);

scanf (”%d”, &val);

printf (”\nValue returned: %d \n”, bin_search (x,8,val);

}

12

6
 03/09/2020

Recursive Version
#include <stdio.h>

int bin_search (int a[], int L, int R, int key)

{
int mid;

if (R < L) return -1; /* NOT FOUND */

mid = (L + R) / 2;
if (a[mid] < key) return (bin_search(a,mid+1,R,key));
else if (a[mid] > key) return (bin_search(a,L,mid-1,key));
else return mid; /* FOUND AT INDEX mid */
}

13

int main()
{
int x[]={10,20,30,40,50,60,70,80}, val;

printf (”\nEnter number to search: ”);

scanf (”%d”, &val);

printf (”\nValue returned: %d \n”, bin_search (x,0,7,val);

}

14

7
 03/09/2020

Is it worth the trouble ?

• Suppose that the array x has 1000 elements.

• Ordinary search
– If key is present in x, it would require 500 comparisons
on the average.
• Binary search
– After 1st compare, left with 500 elements.
– After 2nd compare, left with 250 elements.
– After 3rd compare, left with 125 elements.
– After at most 10 steps, you are done.

15

Time Complexity

• If there are n elements in the array.

– Number of searches required in the worst case: log2n
• For n = 64 (say).
– Initially, list size = 64.
2k = n, where k is the
– After first compare, list size = 32. number of steps.
– After second compare, list size = 16. k = log2n
– After third compare, list size = 8.
– ……. log264 = 6
– After sixth compare, list size = 1. log21024 = 10

16

8
 03/09/2020

Sorting

17

The Basic Problem

• Given an array
x[0], x[1], ... , x[size-1]
• reorder entries so that
x[0] ≤ x[1] ≤ ... ≤ x[size-1]

• List is in non-decreasing order.

• We can also sort a list of elements in non-

increasing order.

18

9
 03/09/2020

Example

• Original list:
10, 30, 20, 80, 70, 10, 60, 40, 70

• Sorted in non-decreasing order:

10, 10, 20, 30, 40, 60, 70, 70, 80

• Sorted in non-increasing order:

80, 70, 70, 60, 40, 30, 20, 10, 10

19

Sorting Problem

• What we want ?
– Sort the given data sorted in the specified order

0 size-1

x: Unsorted list

Sorted list

20

10
 03/09/2020

Selection Sort

21

How it works?

• General situation :
0 k size-1
x: smallest elements, sorted remainder, unsorted

• Steps :
• Find smallest element, mval, in x[k..size-1]
• Swap smallest element with x[k], then increase k by 1

0 k mval size-1

swap
22

11
 03/09/2020

An example worked out

PASS 1:
10 5 17 11 -3 12 Find the minimum
10 5 17 11 -3 12 Exchange with 0th
-3 5 17 11 10 12 element
PASS 2:
-3 5 17 11 10 12 Find the minumum
-3 5 17 11 10 12 Exchange with 1st
-3 5 17 11 10 12 element
PASS 3:
-3 5 17 11 10 12 Find the minumum
-3 5 17 11 10 12 Exchange with 2nd
-3 5 10 11 17 12 element

23

PASS 4:
-3 5 10 11 17 12 Find the minumum
-3 5 10 11 17 12 Exchange with 3rd
-3 5 10 11 17 12 element
PASS 5:
-3 5 10 11 17 12 Find the minumum
-3 5 10 11 17 12 Exchange with 4th
-3 5 10 11 12 17 element

24

12
 03/09/2020

Subproblem

/* Yield index of smallest element in x[k..size-1];*/

int min_loc (int x[], int k, int size)

{
int j, pos;

pos = k;
for (j=k+1; j<size; j++)
if (x[j] < x[pos])
pos = j;
return pos;
}

25

The main sorting function

/* Sort x[0..size-1] in non-decreasing order */

int sel_sort (int x[], int size)

{ int k, m;

for (k=0; k<size-1; k++)

{
m = min_loc (x, k, size);
temp = a[k];
a[k] = a[m]; SWAP
a[m] = temp;
}
}

26

13
 03/09/2020

int main()
{
int x[ ]={-45,89,-65,87,0,3,-23,19,56,21,76,-50};
int i;
for(i=0;i<12;i++)
printf("%d ",x[i]);
printf("\n");
-45 89 -65 87 0 3 -23 19 56 21 76 -50
sel_sort(x,12); -65 -50 -45 -23 0 3 19 21 56 76 87 89
for(i=0;i<12;i++)
printf("%d ",x[i]);
printf("\n");
}

27

Analysis

• How many steps are needed to sort n items ?

– Total number of steps proportional to n2.
– No. of comparisons?
(n-1) + (n-2) + … + 1 = n(n-1)/2

Of the order of n2

– Worst Case? Best Case? Average Case?

28

14
 03/09/2020

Insertion Sort

29

Basic Idea

• Insert elements one at a time, and create a partial sorted

list.
– Sorted list of 2 elements, 3 elements, 4 elements, and
so on.
• In general, in every iteration an element is compared
with all the elements before it.
• After finding the position of insertion, space is created
for it by shifting the other elements and the desired
element is then inserted at the suitable position.
• This procedure is repeated for all the elements in the list.

30

15
 03/09/2020

An example worked out

PASS 1:
10 5 17 11 -3 12 item = 5
? 10 17 11 -3 12 compare 5 and 10
5 10 17 11 -3 12 insert 5
PASS 2:
5 10 17 11 -3 12 item = 17
5 10 17 11 -3 12 compare 17 and 10

PASS 3:
5 10 17 11 -3 12 item = 11
5 10 ? 17 -3 12 compare 11 and 17
5 10 ? 17 -3 12 compare 11 and 10
5 10 11 17 -3 12 insert 11

31

PASS 4:
5 10 11 17 -3 12 item = -3
5 10 11 ? 17 12 compare -3 and 17
5 10 ? 11 17 12 compare -3 and 11
5 ? 10 11 17 12 compare -3 and 10
? 5 10 11 17 12 compare -3 and 5
-3 5 10 11 17 12 insert -3
PASS 5:
-3 5 10 11 17 12 item = 12
-3 5 10 11 ? 17 compare 12 and 17
-3 5 10 11 ? 17 compare 12 and 11
-3 5 10 11 12 17 insert 12

32

16
 03/09/2020

Insertion Sort
void insert_sort (int list[], int size)
{
int i,j,item;

for (i=1; i<size; i++)

{
item = list[i] ;
j = i – 1;
while ((item < list[j]) && (j >= 0))
{
list[j+1] = list[j];
j--;
}
list[j+1] = item ;
}
}
33

int main()
{
int x[ ]={-45,89,-65,87,0,3,-23,19,56,21,76,-50};
int i;
for(i=0;i<12;i++)
printf("%d ",x[i]);
printf("\n");
-45 89 -65 87 0 3 -23 19 56 21 76 -50
insert_sort (x,12); -65 -50 -45 -23 0 3 19 21 56 76 87 89
for(i=0;i<12;i++)
printf("%d ",x[i]);
printf("\n");
}

34

17
 03/09/2020

Time Complexity

• Number of comparisons and shifting:

– Worst case?
1 + 2 + 3 + …… + (n-1) = n(n-1)/2

– Best case?
1 + 1 + …… to (n-1) terms = (n-1)

35

Bubble Sort

36

18
 03/09/2020

How it works?

• The sorting process proceeds in several passes.

– In every pass we go on comparing neighboring pairs, and
swap them if out of order.
– In every pass, the largest of the elements under
considering will bubble to the top (i.e., the right).
10 5 17 11 -3 12
5 10 17 11 -3 12
5 10 17 11 -3 12
5 10 11 17 -3 12
5 10 11 -3 17 12 Largest
5 10 11 -3 12 17

37

An example worked out

PASS 1:
10 5 17 11 -3 12
5 10 17 11 -3 12
5 10 17 11 -3 12
5 10 11 17 -3 12
5 10 11 -3 17 12
5 10 11 -3 12 17
PASS 2:
5 10 11 -3 12 17
5 10 11 -3 12 17
5 10 11 -3 12 17
5 10 -3 11 12 17
5 10 -3 11 12 17

38

19
 03/09/2020

PASS 3:
5 10 -3 11 12 17
5 10 -3 11 12 17
5 -3 10 11 12 17
5 -3 10 11 12 17
PASS 4:
5 -3 10 11 12 17
-3 5 10 11 12 17
-3 5 10 11 12 17
PASS 5:
-3 5 10 11 12 17 Sorted
-3 5 10 11 12 17

39

• How the passes proceed?

– In pass 1, we consider index 0 to n-1.
– In pass 2, we consider index 0 to n-2.
– In pass 3, we consider index 0 to n-3.
– ……
– ……
– In pass n-1, we consider index 0 to 1.

40

20
 03/09/2020

Bubble Sort
void swap(int *x, int *y) void bubble_sort
{ (int x[], int n)
int tmp = *x; {
*x = *y; int i,j;
*y = tmp;
} for (i=n-1; i>0; i--)
for (j=0; j<i; j++)
if (x[j] > x[j+1])
swap(&x[j],&x[j+1]);
}

41

int main()
{
int x[ ]={-45,89,-65,87,0,3,-23,19,56,21,76,-50};
int i;
for(i=0;i<12;i++)
printf("%d ",x[i]);
printf("\n");
-45 89 -65 87 0 3 -23 19 56 21 76 -50
bubble_sort (x,12); -65 -50 -45 -23 0 3 19 21 56 76 87 89
for(i=0;i<12;i++)
printf("%d ",x[i]);
printf("\n");
}

42

21
 03/09/2020

Time Complexity

• Number of comparisons :
– Worst case?
1 + 2 + 3 + …… + (n-1) = n(n-1)/2
– Best case?
• Same

43

• How do you make best case with (n-1)

comparisons only?
– By maintaining a variable flag, to check if there has
been any swaps in a given pass.
– If no swaps during a pass, the array is already sorted.

44

22
 03/09/2020

void bubble_sort (int x[], int n)

{
int i, j, flag;

for (i=n-1; i>0; i--)

{
flag = 0;
for (j=0; j<i; j++)
if (x[j] > x[j+1])
{
swap(&x[j],&x[j+1]);
flag = 1;
}
if (flag == 0) return;
}

45

Some Efficient Sorting Algorithms

46

23
 03/09/2020

• Two of the most popular sorting algorithms are based on

divide-and-conquer approach.
– Quick sort
– Merge sort
• Basic concept (divide-and-conquer method):
sort (list)
{
if the list has greater than 1 elements
{
Partition the list into lowlist and highlist;
sort (lowlist);
sort (highlist);
combine (lowlist, highlist);
}
}
47

Quick Sort

48

24
 03/09/2020

How it works?

• At every step, we select a pivot element in the

list (usually the first element).
– We put the pivot element in the final position of the
sorted list.
– All the elements less than or equal to the pivot
element are to the left.
– All the elements greater than the pivot element are
to the right.

49

Partitioning

0 size-1
x:
pivot

Values smaller Values greater

Perform Perform
partitioning partitioning

50

25
 03/09/2020

void print (int x[], int low, int high)

{
int i;
for(i=low; i<=high; i++)
printf(" %d", x[i]);
printf("\n");
}

void swap (int *a, int *b)

{
int tmp=*a;
*a=*b;
*b=tmp;
}

52

void partition (int x[], int low, int high)

{
int i = low+1, j = high;
int pivot = x[low];
if (low >= high) return;
while (i<j) {
while ((x[i]<pivot) && (i<high)) i++;
while ((x[j]>=pivot) && (j>low)) j--;
if (i<j) swap (&x[i], &x[j]);
}
if (j==high) {
swap (&x[j], &x[low]);
partition (x, low, high-1);
}
else
if (i==low+1)
partition (x, low+1, high);
else {
swap (&x[j], &x[low]);
partition (x, low, j-1);
partition (x, j+1, high);
}
}
53

26
 03/09/2020

int main (int argc, char *argv[])

{
int x[] = {-56,23,43,-5,-3,0,123,-35,87,56,75,80};
int i=0;
int num;
num = 12; /* Number of elements */

partition(x,0,num-1);
printf("Sorted list: ");
print (x,0,num-1);
}

54

Trace of Partitioning: an example

45 -56 78 90 -3 -6 123 0 -3 45 69 68

45 -56 78 90 -3 -6 123 0 -3 45 69 68

-6 -56 -3 0 -3 45 123 90 78 45 69 68

-56 -6 -3 0 -3 68 90 78 45 69 123

-3 0 -3 45 68 78 90 69

-3 0 69 78 90

Sorted list: -56 -6 -3 -3 0 45 45 68 69 78 90 123

55

27
 03/09/2020

Time Complexity

• Worst case:
n2 ==> list is already sorted

• Average case:
n log2n

• Statistically, quick sort has been found to be one

of the fastest algorithms.

56

Merge Sort

57

28
 03/09/2020

Merge Sort

Input Array

Part-I Part-II

Part-I Part-II Part-I Part-II

Merge Split
Sorted Arrays

58

Merging two sorted arrays

pa pb

a Sorted Array b Sorted Array

0 m 0 n

c Merged sorted array

0 m+n-1

Move and copy elements pointed by pa if its value is smaller

than the element pointed by pb in (m+n-1) operations; otherwise, copy
elements pointed by pb .

59

29
 03/09/2020

Example

• Initial array A contains 14 elements:

– 66, 33, 40, 22, 55, 88, 60, 11, 80, 20, 50, 44, 77, 30

• Pass 1 :: Merge each pair of elements

– (33, 66) (22, 40) (55, 88) (11, 60) (20, 80) (44, 50) (30, 70)

• Pass 2 :: Merge each pair of pairs

– (22, 33, 40, 66) (11, 55, 60, 88) (20, 44, 50, 80) (30, 77)

• Pass 3 :: Merge each pair of sorted quadruplets

– (11, 22, 33, 40, 55, 60, 66, 88) (20, 30, 44, 50, 77, 80)

• Pass 4 :: Merge the two sorted subarrays to get the final list
– (11, 20, 22, 30, 33, 40, 44, 50, 55, 60, 66, 77, 80, 88)

60

void merge_sort (int *a, int n)

{
int i, j, k, m;
int *b, *c;

if (n>1) {
k = n/2;
m = n-k;
b = (int *) malloc(k*sizeof(int));
c = (int *) malloc(m*sizeof(int));

for (i=0; i<k; i++)

b[i]=a[i];
for (j=k; j<n; j++)
c[j-k]=a[j];

merge_sort (b, k);

merge_sort (c, m);
merge (b, c, a, k, m);
free(b); free(c);
}
}

61

30
 03/09/2020

void merge (int *a, int *b, int *c, int m, int n)
{
int i, j, k, p;
i = j = k = 0;
do {
if (a[i] < b[j]) {
c[k]=a[i]; i++;
}
else {
c[k]=b[j]; j++;
}
k++;
} while ((i<m) && (j<n));
if (i == m) {
for (p=j; p<n; p++) { c[k]=b[p]; k++; }
}
else {
for (p=i; p<m; p++) { c[k]=a[p]; k++; }
}
}

62

main()
{
int i, num;
int a[ ] = {-56,23,43,-5,-3,0,123,-35,87,56,75,80};

num = 12;

printf ("\n Original list: ");

print (a, 0, num-1);

merge_sort (a, 12);

printf ("\n Sorted list: ");

print (a, 0, num-1);
}

63

31
 03/09/2020

Time Complexity

• Best/Worst/Average case:
n log2n

• Drawback:
– Needs double amount of space for storage.
– For sorting n elements, requires another array of size
n to carry out merge.

64

Example :: sorting arrays of structures (bubble

sort)
#include <stdio.h> for (k=0; k<n; k++)
struct stud scanf (”%d %s %f”, &class[k].roll,
{ class[k].dept_code,
int roll; &class[k].cgpa);
char dept_code[25]; for (j=0; j<n-1; j++)
float cgpa; for (k=1; k<n-j; k++)
}; {
if (class[k-1].roll >
main() class[k].roll)
{ {
struc stud class[100], t; t = class[k-1];
int j, k, n; class[k-1] = class[k];
class[k] = t;
scanf (”%d”, &n); }
/* no. of students */ }
<<<< PRINT THE RECORDS >>>>
}

65

32
 03/09/2020

Example :: sorting arrays of structures

(selection sort)
int min_loc (struct stud x[],
int k, int size) int selsort (struct stud x[],int n)
int j, pos;
{
{
int k, m;
pos = k;
for (k=0; k<n-1; k++)
for (j=k+1; j<size; j++)
{
if (x[j] < x[pos])
m = min_loc (x, k, n);
pos = j;
temp = a[k];
return pos;
a[k] = a[m];
}
a[m] = temp;
}
main() }
{
struc stud class[100];
int n;
…
selsort (class, n);
…
66

Algorithm Analysis

67

33
 03/09/2020

Analysis of Algorithms

• How much resource is required ?

• Measures for efficiency
– Execution time  time complexity
– Memory space  space complexity

• Observation :
– The larger amount of input data an algorithm has, the
larger amount of resource it requires.
– Complexities are functions of the amount of input
data (input size).

68

What do we use for a yardstick?

• The same algorithm will run at different speeds

and will require different amounts of space.
– When run on different computers, different
programming languages, different compilers.
• But algorithms usually consume resources in some
fashion that depends on the size of the problem
they solve.
– Some parameter n (for example, number of elements
to sort).

69

34
 03/09/2020

An example of a sorting algorithm

• We run this sorting Array Size Home Desktop

n Computer Computer
algorithm on two
125 12.5 2.8
different computers,
250 49.3 11.0
and note the time (in 500 195.8 43.4
milliseconds) for 1000 780.3 72.9
different sizes of 2000 3114.9 690.5
input.

70

3500

3000

2500

2000
Home
Desktop
1500

1000

500

0
125 250 500 1000 2000

71

35
 03/09/2020

Contd.

• Home Computer :
f1(n) = 0.0007772 n2 + 0.00305 n + 0.001

• Desktop Computer :
f2(n) = 0.0001724 n2 + 0.00040 n + 0.100

– Both are quadratic function of n.

– The shape of the curve that expresses the running
time as a function of the problem size stays the same.

72

Complexity Classes

• The running time for different algorithms fall

into different complexity classes.
– Each complexity class is characterized by a different
family of curves.
– All curves in a given complexity class share the same
basic shape.

• The O-notation is used for talking about the

complexity classes of algorithms.

73

36
 03/09/2020

Running time of algorithms

Assume speed is 107 instructions per second.

size n 10 20 30 50 100 1000 10000

n .001 ms .002 ms .003 ms .005 ms .01 ms .1 ms 1 ms

nlogn .003 ms .008 ms .015 ms .03 ms .07 ms 1 ms 13 ms

n2 .01 ms .04 ms .09 ms .25 ms 1 ms 100 ms 10 s

n3 .1 ms .8 ms 2.7 ms 12.5 ms 100 ms 100 s 28 h

n 13
2 .1 ms .1 s 100 s 3y 3x10 c inf inf

74

• The complexity classes:

log2n
n
n log2n
Complexity
n2 increases
n3
2n
n!

75

37
 03/09/2020

Introducing the language of O-notation

• Definition:
f(n) = O(g(n)) if there exists positive constants c
and n0 such that f(n) c.g(n) when n  n0.

• The big-Oh notation is used to categorize the

complexity class of algorithms.
– It gives an upper bound.
– Other measures also exist, like small-Oh, Omega, Theta,
etc.

76

Examples
• f(n) = 2n2+4n+5 is O(n2).
– One possibility: c=11, and no=1.
• f(n) = 2n2+4n+5 is also O(n3),O(n4),etc.
– One possibility: c=11, and no=1.
• f(n) = n(n-1)/2 is O(n2).
– One possibility: c=1/2, and no=1.
• f(n) = 5n4+log2n is O(n4).
– One possibility: c=6, and no=1.
• f(n) = 75 is O(1).
– One possibility: c=75, and no=1.
77

38
 03/09/2020

Complexities of Known Algorithms

Algorithm Best-case Average-case Worst-case

Selection sort O(n2) O(n2) O(n2)
Insertion sort O(n) O(n2) O(n2)
Bubble sort O(n) O(n2) O(n2)
Quick sort O(n log2n) O(n log2n) O(n2)
Merge sort O(n log2n) O(n log2n) O(n log2n)
Linear search O(1) O(n) O(n)
Binary search O(1) O(log2n) O(log2n)

78

Observations

• There is a big difference between polynomial time

complexity and exponential time complexity.
• Hardware advances affect only efficient algorithms
and do not help inefficient algorithms.

79

39