Computability of Algorithms

A given computational problem is said to be computable if there exists an algorithm that

can solve a problem in an effective manner. Many problems (or functions) that are computable (or
decidable), meaning there exists some algorithm that computes an answer (or output) to any
instance of the problem (or for any input to the function) in a finite number of simple steps. A
simple example is the integer increment operation:

f(x) = x + 1
It should be intuitive that given any integer x, we can compute x + 1 in a finite number of steps.
Since x is finite, it may be represented by a finite string of digits.

Yet there are also problems and functions that that are non-computable (or undecidable or
uncomputable), meaning that there exists no algorithm that can compute an answer or output for
all inputs in a finite number of simple steps. (Undecidable simply means non-computable in the
context of a decision problem, whose answer (or output) is either “true” or “false”).
Non-Computable Problems –
A non-computable is a problem for which there is no algorithm that can be used to solve it. Most
famous example of a non-computablity (or undecidability) is the Halting Problem. Given a
description of a Turing machine and its initial input, determine whether the program, when
executed on this input, ever halts (completes).
The alternative is that it runs forever without halting. The halting problem is about seeing if a
machine will ever come to a halt when a certain input is given to it or if it will finish running.
This input itself can be something that keeps calling itself forever which means that it will cause
the program to run forever.

Other example of an uncomputable problem is: determining whether a computer program loops
forever on some input.

Some Examples On Computable Problems –

These are simple examples of computable problem:
1. Computing the greatest common divisor of a pair of integers.
2. Computing the least common multiple of a pair of integers.
3. Finding the shortest path between a pair of nodes in a finite graph.

Tractable Problem: A problem that is solvable by a polynomial-time algorithm.

The upper bound is polynomial.
Here are examples of tractable problems (ones with known polynomial-time algorithms):
– Searching an unordered list
– Searching an ordered list
– Sorting a list
– Multiplication of integers (even though there’s a gap)
– Finding a minimum spanning tree in a graph (even though there’s a gap)

Intractable Problem: a problem that cannot be solved by a polynomial-time algorithm.

The lower bound is exponential.
From a computational complexity stance, intractable problems are problems for which there
exist no efficient algorithms to solve them.
Most intractable problems have an algorithm that provides a solution, and that algorithm is
the brute-force search.
This algorithm, however, does not provide an efficient solution and is, therefore, not feasible
for computation with anything more than the smallest input.

Examples
Towers of Hanoi: we can prove that any algorithm that solves this problem must have a
worst-case running time that is at least 2n − 1.
* List all permutations (all possible orderings) of n numbers.

Complexity Classes
Definition of NP class Problem: - The set of all decision-based problems came into
the division of NP Problems who can't be solved or produced an output within
polynomial time but verified in the polynomial time. NP class contains P class as a
subset. NP problems being hard to solve.

Note: - The term "NP" does not mean "not polynomial." Originally, the term meant "non-
deterministic polynomial. It means according to the one input number of output will be
produced.

Definition of P class Problem: - The set of decision-based problems come into the
division of P Problems who can be solved or produced an output within polynomial time.
P problems being easy to solve

Definition of Polynomial time: - If we produce an output according to the given input

within a specific amount of time such as within a minute, hours. This is known as
Polynomial time.

Definition of Non-Polynomial time: - If we produce an output according to the given

input but there are no time constraints is known as Non-Polynomial time. But yes output
will produce but time is not fixed yet.

Definition of Decision Based Problem: - A problem is called a decision problem if its

output is a simple "yes" or "no" (or you may need this of this as true/false, 0/1,
accept/reject.) We will phrase many optimization problems as decision problems. For
example, Greedy method, D.P., given a graph G= (V, E) if there exists any Hamiltonian
cycle.

Definition of NP-hard class: - Here you to satisfy the following points to come into
the division of NP-hard

1. If we can solve this problem in polynomial time, then we can solve all NP
problems in polynomial time

2. If you convert the issue into one form to another form within the polynomial time

Definition of NP-complete class: - A problem is in NP-complete, if

 1. It is in NP

2. It is NP-hard

Pictorial representation of all NP classes which includes NP, NP-hard, and NP-
complete

In Computer Science, many problems are solved where the objective is to maximize or minimize some
values, whereas in other problems we try to find whether there is a solution or not. Hence, the problems
can be categorized as follows −

Optimization Problem
Optimization problems are those for which the objective is to maximize or minimize some values. For
example,
 Finding the minimum number of colors needed to color a given graph.
 Finding the shortest path between two vertices in a graph.

Decision Problem
There are many problems for which the answer is a Yes or a No. These types of problems are known
as decision problems. For example,
 Whether a given graph can be colored by only 4-colors.
 Finding Hamiltonian cycle in a graph is not a decision problem, whereas checking a graph is
Hamiltonian or not is a decision problem.

What is Language?
Every decision problem can have only two answers, yes or no. Hence, a decision problem may belong to
a language if it provides an answer ‘yes’ for a specific input. A language is the totality of inputs for which
the answer is Yes. Most of the algorithms discussed in the previous chapters are polynomial time
algorithms.
For input size n, if worst-case time complexity of an algorithm is O(nk), where k is a constant, the
algorithm is a polynomial time algorithm.
Algorithms such as Matrix Chain Multiplication, Single Source Shortest Path, All Pair Shortest Path,
Minimum Spanning Tree, etc. run in polynomial time. However there are many problems, such as
traveling salesperson, optimal graph coloring, Hamiltonian cycles, finding the longest path in a graph,
and satisfying a Boolean formula, for which no polynomial time algorithms is known. These problems
belong to an interesting class of problems, called the NP-Complete problems, whose status is unknown.
In this context, we can categorize the problems as follows −

P-Class
The class P consists of those problems that are solvable in polynomial time, i.e. these problems can be
solved in time O(nk) in worst-case, where k is constant.
These problems are called tractable, while others are called intractable or superpolynomial.
Formally, an algorithm is polynomial time algorithm, if there exists a polynomial p(n) such that the
algorithm can solve any instance of size n in a time O(p(n)).
Problem requiring Ω(n50) time to solve are essentially intractable for large n. Most known polynomial time
algorithm run in time O(nk) for fairly low value of k.
The advantages in considering the class of polynomial-time algorithms is that all
reasonable deterministic single processor model of computation can be simulated on each other
with at most a polynomial slow-d

NP-Class
The class NP consists of those problems that are verifiable in polynomial time. NP is the class of
decision problems for which it is easy to check the correctness of a claimed answer, with the aid of a
little extra information. Hence, we aren’t asking for a way to find a solution, but only to verify that an
alleged solution really is correct.
Every problem in this class can be solved in exponential time using exhaustive search.

P versus NP
Every decision problem that is solvable by a deterministic polynomial time algorithm is also solvable by a
polynomial time non-deterministic algorithm.
All problems in P can be solved with polynomial time algorithms, whereas all problems in NP - P are
intractable.
It is not known whether P = NP. However, many problems are known in NP with the property that if they
belong to P, then it can be proved that P = NP.
If P ≠ NP, there are problems in NP that are neither in P nor in NP-Complete.
The problem belongs to class P if it’s easy to find a solution for the problem. The problem belongs to NP,
if it’s easy to check a solution that may have been very tedious to find.

Design and Analysis Cook’s Theorem

Advertisements

Previous Page
Next Page
Stephen Cook presented four theorems in his paper “The Complexity of Theorem Proving Procedures”.
These theorems are stated below. We do understand that many unknown terms are being used in this
chapter, but we don’t have any scope to discuss everything in detail.
Following are the four theorems by Stephen Cook −

Theorem-1
If a set S of strings is accepted by some non-deterministic Turing machine within polynomial time,
then S is P-reducible to {DNF tautologies}.

Theorem-2
The following sets are P-reducible to each other in pairs (and hence each has the same polynomial
degree of difficulty): {tautologies}, {DNF tautologies}, D3, {sub-graph pairs}.
Theorem-3
 For any TQ(k) of type Q, TQ(k)k√(logk)2TQ(k)k(logk)2 is unbounded
 There is a TQ(k) of type Q such that TQ(k)⩽2k(logk)2TQ(k)⩽2k(logk)2

Theorem-4
If the set S of strings is accepted by a non-deterministic machine within time T(n) = 2n, and if TQ(k) is an
honest (i.e. real-time countable) function of type Q, then there is a constant K, so S can be recognized
by a deterministic machine within time TQ(K8n).
 First, he emphasized the significance of polynomial time reducibility. It means that if we have a
polynomial time reduction from one problem to another, this ensures that any polynomial time
algorithm from the second problem can be converted into a corresponding polynomial time
algorithm for the first problem.
 Second, he focused attention on the class NP of decision problems that can be solved in
polynomial time by a non-deterministic computer. Most of the intractable problems belong to this
class, NP.
 Third, he proved that one particular problem in NP has the property that every other problem in
NP can be polynomially reduced to it. If the satisfiability problem can be solved with a polynomial
time algorithm, then every problem in NP can also be solved in polynomial time. If any problem
in NP is intractable, then satisfiability problem must be intractable. Thus, satisfiability problem is
the hardest problem in NP.
 Fourth, Cook suggested that other problems in NP might share with the satisfiability problem this
property of being the hardest member of NP.

NP Hard and NP-Complete Classes

Advertisements

Previous Page
Next Page
A problem is in the class NPC if it is in NP and is as hard as any problem in NP. A problem is NP-hard if
all problems in NP are polynomial time reducible to it, even though it may not be in NP itself.

If a polynomial time algorithm exists for any of these problems, all problems in NP would be polynomial
time solvable. These problems are called NP-complete. The phenomenon of NP-completeness is
important for both theoretical and practical reasons.

Definition of NP-Completeness
A language B is NP-complete if it satisfies two conditions
  B is in NP
 Every A in NP is polynomial time reducible to B.
If a language satisfies the second property, but not necessarily the first one, the language B is known
as NP-Hard. Informally, a search problem B is NP-Hard if there exists some NP-
Complete problem A that Turing reduces to B.
The problem in NP-Hard cannot be solved in polynomial time, until P = NP. If a problem is proved to be
NPC, there is no need to waste time on trying to find an efficient algorithm for it. Instead, we can focus
on design approximation algorithm.

NP-Complete Problems
Following are some NP-Complete problems, for which no polynomial time algorithm is known.

 Determining whether a graph has a Hamiltonian cycle

 Determining whether a Boolean formula is satisfiable, etc.

NP-Hard Problems
The following problems are NP-Hard

 The circuit-satisfiability problem

 Set Cover
 Vertex Cover
 Travelling Salesman Problem
In this context, now we will discuss TSP is NP-Complete

TSP is NP-Complete
The traveling salesman problem consists of a salesman and a set of cities. The salesman has to visit
each one of the cities starting from a certain one and returning to the same city. The challenge of the
problem is that the traveling salesman wants to minimize the total length of the trip

Proof
To prove TSP is NP-Complete, first we have to prove that TSP belongs to NP. In TSP, we find a tour
and check that the tour contains each vertex once. Then the total cost of the edges of the tour is
calculated. Finally, we check if the cost is minimum. This can be completed in polynomial time.
Thus TSP belongs to NP.
Secondly, we have to prove that TSP is NP-hard. To prove this, one way is to show that Hamiltonian
cycle ≤p TSP (as we know that the Hamiltonian cycle problem is NPcomplete).
Assume G = (V, E) to be an instance of Hamiltonian cycle.
Hence, an instance of TSP is constructed. We create the complete graph G' = (V, E'), where

E′={(i,j):i,j∈Vandi≠jE′={(i,j):i,j∈Vandi≠j

Thus, the cost function is defined as follows −

t(i,j)={01if(i,j)∈Eotherwiset(i,j)={0if(i,j)∈E1otherwise
Now, suppose that a Hamiltonian cycle h exists in G. It is clear that the cost of each edge
in h is 0 in G' as each edge belongs to E. Therefore, h has a cost of 0 in G'. Thus, if graph G has a
Hamiltonian cycle, then graph G' has a tour of 0 cost.
Conversely, we assume that G' has a tour h' of cost at most 0. The cost of edges in E' are 0 and 1 by
definition. Hence, each edge must have a cost of 0 as the cost of h' is 0. We therefore conclude
that h' contains only edges in E.
We have thus proven that G has a Hamiltonian cycle, if and only if G' has a tour of cost at most 0. TSP is
NP-complete.
There are computational problems that can not be solved by algorithms even with unlimited time.
For example Turing Halting problem (Given a program and an input, whether the program will
eventually halt when run with that input, or will run forever). Alan Turing proved that general
algorithm to solve the halting problem for all possible program-input pairs cannot exist. A key part
of the proof is, Turing machine was used as a mathematical definition of a computer and program
(Source Halting Problem).
Status of NP Complete problems is another failure story, NP complete problems are problems
whose status is unknown. No polynomial time algorithm has yet been discovered for any NP
complete problem, nor has anybody yet been able to prove that no polynomial-time algorithm exist
for any of them. The interesting part is, if any one of the NP complete problems can be solved in
polynomial time, then all of them can be solved.

What are NP, P, NP-complete and NP-Hard problems?

P is set of problems that can be solved by a deterministic Turing machine in Polynomial time.
NP is set of decision problems that can be solved by a Non-deterministic Turing Machine
in Polynomial time. P is subset of NP (any problem that can be solved by deterministic machine in
polynomial time can also be solved by non-deterministic machine in polynomial time).
Informally, NP is set of decision problems which can be solved by a polynomial time via a “Lucky
Algorithm”, a magical algorithm that always makes a right guess among the given set of choices
(Source Ref 1).
NP-complete problems are the hardest problems in NP set. A decision problem L is NP-complete if:
1) L is in NP (Any given solution for NP-complete problems can be verified quickly, but there is no
efficient known solution).
2) Every problem in NP is reducible to L in polynomial time (Reduction is defined below).
A problem is NP-Hard if it follows property 2 mentioned above, doesn’t need to follow property 1.
Therefore, NP-Complete set is also a subset of NP-Hard set.
Fig: Complexity Classes

The Complexity Class P • Definition: The complexity class P is the set of all decision problems that can
be solved with worst-case polynomial time-complexity. • In other words, a problem is in the class P if
it is a decision problem and there exists an algorithm that solves any instance of size n in O(n k ) time,
for some integer k. (Strictly, n must be the number of bits needed for a ‘reasonable’ encoding of the
input. But we won’t get bogged down in such fine details.) • So P is just the set of tractable decision
problems: the decision problems for which we have polynomial-time algorithms.

What Are P And NP Problem?

Computer algorithms take a certain amount of time to solve the problem they are
tasked with. Generally, you can roughly estimate how much time an algorithm will take
using the number of elements they need to handle. Computer scientists call the number
of elements N.

Because some algorithms are more or less efficient than others, the time they take to
complete could be related to N, N2, N3, and so on. The important thing, though, is that
the exponent is a constant—it’s 1, or 2, etc. When this is the case, an algorithm is said
to complete in polynomial time, or P.

Unfortunately, not all problems work this way. Solving some problems can take an
algorithm an amount of time proportional to 2 N, 3N, and so on. In this case, N is the
exponent, meaning that every element the algorithm has to deal with increases its
complexity exponentially. In this case, the algorithm can be completed in exponential
time, or NP (which really stands for nondeterministic polynomial time).

The difference between these two can be huge. If a P algorithm has 100 elements, and
its time to complete working is proportional to N3, then it will solve its problem in about
3 hours. If it’s an NP algorithm, however, and its completion time is proportional to 2 N,
then it will take roughly 300 quintillion years.

Therefore, A problem is called NP if its solution can be guessed and verified in

polynomial time, and nondeterministic means that no particular rule is followed to make
the guess. On the other hand, a P problem is one that can be solved in polynomial time
by deterministic algorithms.

Facts About P Problems

 P problems are set of problems which can be solved in polynomial time by
deterministic algorithms.
 The problem belongs to class P if it’s easy to find a solution for the problem.
 P Problems can be solved and verified in polynomial time.
 P problems are subset of NP problems.
 It is not known whether P=NP. However, many problems are known in NP with
the property that if they belong to P, then it can be proved that P=NP.
 All P problems are deterministic in nature.
 Example: Selection sort, linear search

Facts About NP-Class Problems

 NP problems are the problems which can be solved in non-deterministic
polynomial time.
  The problem belongs to NP, if it’s easy to check a solution that may have been
very tedious to find.
 Solution to NP problems cannot be obtained in polynomial time, but if the
solution is given, it can be verified in polynomial time.
 If P≠NP, there are problems in NP that are neither in P nor in NP-Complete.
 NP problems are superset of P problems.
 All the NP problems are non-deterministic in nature.
 Example TSP, Knapsack problem.
Also Read: Difference Between Prim’s And Kruskal’s Algorithm

Difference Between P Problems And NP

Problems In Tabular Form
P PROBLEMS NP PROBLEMS

P problems are set of problems which can be NP problems are the problems which
solved in polynomial time by deterministic can be solved in non-deterministic
algorithms. polynomial time.

The problem belongs to NP, if it’s easy

The problem belongs to class P if it’s easy to
to check a solution that may have been
find a solution for the problem.
very tedious to find.

Solution to NP problems cannot be

P Problems can be solved and verified in obtained in polynomial time, but if the
polynomial time. solution is given, it can be verified in
polynomial time.

P problems are subset of NP problems. NP problems are superset of P problems.

It is not known whether P=NP. However,

many problems are known in NP with the If P≠NP, there are problems in NP that
property that if they belong to P, then it can are neither in P nor in NP-Complete.
be proved that P=NP.

All the NP problems are non-

All P problems are deterministic in nature.
deterministic in nature.

Selection sort, linear search TSP, Knapsack problem.

 The list below contains some well-known problems that are NP-complete when expressed as
decision problems.

 Boolean satisfiability problem (SAT)

 Knapsack problem.
 Hamiltonian path problem.
 Travelling salesman problem (decision version)
 Subgraph isomorphism problem.
 Subset sum problem.
 Clique problem.
 Vertex cover problem.

What is Reduction?
Let L1 and L2 be two decision problems. Suppose algorithm A 2 solves L2. That is, if y is an input for
L2 then algorithm A2 will answer Yes or No depending upon whether y belongs to L 2 or not.
The idea is to find a transformation from L 1 to L2 so that the algorithm A 2 can be part of an
algorithm A1 to solve L1.

Learning reduction in general is very important. For example, if we have library functions to solve
certain problem and if we can reduce a new problem to one of the solved problems, we save a lot
of time. Consider the example of a problem where we have to find minimum product path in a
given directed graph where product of path is multiplication of weights of edges along the path. If
we have code for Dijkstra’s algorithm to find shortest path, we can take log of all weights and use
Dijkstra’s algorithm to find the minimum product path rather than writing a fresh code for this new
problem.
How to prove that a given problem is NP complete?
From the definition of NP-complete, it appears impossible to prove that a problem L is NP-
Complete. By definition, it requires us to that show every problem in NP is polynomial time
reducible to L. Fortunately, there is an alternate way to prove it. The idea is to take a known NP-
Complete problem and reduce it to L. If polynomial time reduction is possible, we can prove that L
is NP-Complete by transitivity of reduction (If a NP-Complete problem is reducible to L in
polynomial time, then all problems are reducible to L in polynomial time).