Design and Analysis of Algorithms
By: Dr. Kashif Ayyub
Lecture - 01
Introduction to
Algorithms
Department of Computer Science, COMSATS University Islamabad, Wah Campus 1
What are Algorithms
• An algorithm is any well-defined computational procedure that takes some
value, or set of values, as input and produces some value, or set of values, as
output in a finite amount of time. An algorithm is thus a sequence of
computational steps that transform the input into the output.
• As an example, suppose that you need to sort a sequence of numbers into
monotonically increasing order.
• Input: A sequence of n numbers 𝑎1 , 𝑎2 , … , 𝑎𝑛
• Output: A permutation (reordering) 𝑎1′ , 𝑎2′ , … , 𝑎𝑛′ of the input sequence such that
𝑎1′ ≤ 𝑎2′ ≤ ⋯ ≤ 𝑎𝑛′
• Thus, given the input sequence 31, 41, 59, 26, 41, 58 , a correct sorting algorithm
returns as output the sequence 26, 31, 41, 41, 58, 59 .
• Such an input sequence is called an instance of the sorting problem. In general, an
instance of a problem consists of the input (satisfying whatever constraints are
imposed in the problem statement) needed to compute a solution to the problem.
2
What are Algorithms
• An algorithm for a computational problem is correct if, for every problem
instance provided as input, it halts (finishes) its computing in finite time and
outputs the correct solution to the problem instance.
• A correct algorithm solves the given computational problem.
• An incorrect algorithm might not halt at all on some input instances, or it
might halt with an incorrect answer.
3
Why Do You Need to Study Algorithms
• Foundation for Programming: Algorithms are fundamental to computer
science and programming, providing a basis for understanding how software
operates and how to build efficient systems.
• Problem-Solving: Algorithms provide a structured approach to solving
complex problems efficiently and effectively.
• Efficiency: Understanding algorithms helps in designing solutions that are
both time and resource-efficient, leading to faster and more reliable software.
• Optimization: Knowledge of algorithms allows for optimizing code and
improving the performance of applications by selecting the most appropriate
algorithm for a given task.
4
Why Do You Need to Study Algorithms
• Data Handling: Algorithms are crucial for managing and processing large
amounts of data, including sorting, searching, and manipulating data
structures.
• Innovation: A deep understanding of algorithms drives innovation in
technology, leading to the development of new methods and solutions in
fields such as artificial intelligence, machine learning, and data analysis.
• Career Opportunities: Proficiency in algorithms is highly valued in the tech
industry, as it demonstrates problem-solving skills and the ability to handle
complex computational tasks.
• Understanding Complexity: Studying algorithms helps in understanding
computational complexity, which is vital for evaluating the feasibility and
scalability of software solutions.
5
What kinds of problems are solved by algorithms?
• Search Problems: Algorithms help in finding specific data within large datasets,
such as searching for a particular element in a list or database.
• Sorting Problems: Algorithms arrange data in a particular order, such as sorting
numbers in ascending order or arranging names alphabetically.
• Optimization Problems: Algorithms are used to find the best solution among many
possible options, such as minimizing costs, maximizing profits, or optimizing
resource allocation.
• Graph Problems: Algorithms solve problems related to networks and connections,
such as finding the shortest path between two points, detecting cycles, or
identifying connected components in a graph.
• Data Processing: Algorithms handle tasks like filtering, aggregating, and
transforming data, often used in data analysis, machine learning, and big data
applications.
6
What kinds of problems are solved by algorithms?
• Cryptography: Algorithms are essential for securing data, encrypting and decrypting
information to ensure privacy and security in communications.
• Mathematical Problems: Algorithms perform complex mathematical computations,
solve equations, and carry out tasks like prime factorization, matrix multiplication,
and numerical integration.
• Artificial Intelligence: Algorithms enable machines to learn from data, make
decisions, and solve problems autonomously in areas like machine learning, natural
language processing, and computer vision.
• Simulation and Modeling: Algorithms are used to model real-world systems and
simulate their behavior, such as in climate modeling, financial forecasting, and
engineering simulations.
• Automated Decision Making: Algorithms drive decision-making processes in
automated systems, such as in recommendation engines, self-driving cars, and
automated trading systems.
7
Algorithms as Technology
• If computers were infinitely fast and computer memory were free, would you have any reason to study algorithms? The
answer is yes, Of course, computers may be fast, but they are not infinitely fast. Computing time is therefore a bounded
resource, which makes it precious.
• Problem: Sorting n values
• Input Size: 107 numbers (10 million)
• Computer – A is 1000 times faster than Computer - B
Computer – A Computer – B
Speed 1010 inst/sec 107 inst/sec
Algorithm Bubble Sort Merge Sort
Complexity 2n2 50nlgn
Programmer World’s Best Average
Language Machine High-Level
2 x (107)2 inst 50 x 107 x lg107 inst
1010 inst/sec 107 inst/sec
= 20,000 seconds = 1,163 seconds
8
= more than 5.5 hours = less than 20 minutes
Representation of an Algorithm
• An algorithm can be represented in many ways, for example, a GCD aglorithm
