Theory of Computation

Lecture #7

Sarmad Abbasi

Virtual University

Sarmad Abbasi (Virtual University) Theory of Computation 1 / 39

Lecture 7: Overview

Decidable Languages
Decidable Problems concerning Regular Languages
Decidable Problems concerning Context Free Languages
The Halting Problem
Undecidability of the Halting Problem

Sarmad Abbasi (Virtual University) Theory of Computation 2 / 39

Decidable Languages

Let us recall that a language A is called decidable if there exists a TM,

M, that halts on all inputs and L(M) = A.
1 If x ∈ A then M accepts x
2 If x 6∈ A then M rejects x
Intuitively M is just an algorithm that solves the yes/no problem A.

Sarmad Abbasi (Virtual University) Theory of Computation 3 / 39

The Acceptance Problem for DFAs

You must have studied DFAs in your previous automata course. DFAs
are simple machines with a readonly head. Let us look at the following
problem:

ADFA = {hB, wi : B is a DFA that accepts the input string w}.

Sarmad Abbasi (Virtual University) Theory of Computation 4 / 39

The Acceptance Problem for DFAs

The input to this problem consist of two parts. The first part is going to
be a description of a deterministic finite automata B. The second part
is going to be a string w. We have to decide if the DFA B accepts the
string w.

Sarmad Abbasi (Virtual University) Theory of Computation 5 / 39

The Acceptance Problem for DFAs

Note that the input will not be a DFA B but a suitable encoding of B.
Let us look at the following picture which shows two DFAs:

Sarmad Abbasi (Virtual University) Theory of Computation 6 / 39

Acceptance Problem for DFAs

In this case:
1 hB1 , 1i ∈ ADFA .
2 hB1 , 000i 6∈ ADFA .
3 hB2 , 1i 6∈ ADFA .
4 hB2 , 000i ∈ ADFA .
We want to show ADFA is decidable.

Sarmad Abbasi (Virtual University) Theory of Computation 7 / 39

Acceptance Problem for DFAs

The question that we want to ask is:

Question
Is ADFA decidable?

Lets ponder on this question for a bit.

Sarmad Abbasi (Virtual University) Theory of Computation 8 / 39

Acceptance Problem for DFAs

We are asking if we can make an algorithm (Turing machine) that will

take as input a description of a DFA B and a string w ∈ {0, 1}∗ and tell
us if B accepts w?

Sarmad Abbasi (Virtual University) Theory of Computation 9 / 39

Acceptance Problem for DFAs

The answer is yes. We can devise such an Turing machine. Since, a

Turing machine can “simulate the behavior of a DFA.”
Here is a high-level description of such a Turing machine MD .
1 On input hB, wi
2 Simulate B on w.
3 If B reaches an accepting state then accept. If it reaches a
rejecting state then reject.

Sarmad Abbasi (Virtual University) Theory of Computation 10 / 39

Acceptance Problem for DFAs

We can give a more detailed level description of this Turing machine.

An implementation level description.
1 On input hB, wi
2 Copy the transition function of B on your second tape.
3 Write the current state on the third tape.
4 Write the w on the first tape.
5 For i = 1, . . . , |w|
6 Simulate a single step of the DFA by consulting the the
transition function on the first tape and the current state on the
third tape.
7 If the state on the third tape is a final state accept. Else reject.

Sarmad Abbasi (Virtual University) Theory of Computation 11 / 39

Acceptance Problem for NFAs

We can also look at the acceptance problem for NFAs.

ANFA = {hB, wi : B is a NFA that accepts the input string w}

Is ANFA decidable?

Once again think of the question intuitively.

Sarmad Abbasi (Virtual University) Theory of Computation 12 / 39

Acceptance Problem for NFAs

We are asking if we can make an algorithm (Turing machine) that will

take as input a description of a NFA B and a string w ∈ {0, 1}∗ and tell
us if B accepts w?

Sarmad Abbasi (Virtual University) Theory of Computation 13 / 39

Acceptance Problem for NFAs

There are two ways we can approach this problem. The first one is
very similar to the previous one. We can show that a Turing machine
can simply “simulate” an NFA. All it has to do is to keep track of the set
of states that the NFA is in.

Sarmad Abbasi (Virtual University) Theory of Computation 14 / 39

Acceptance Problem for NFAs

However, there is another approach. We know that every NFA can be

converted to an equivalent DFA. In fact, there is an algorithm that can
convert any (description of an) NFA to a (description of an) equivalent
DFA. So we have the following solution also:
1 On input hB, wi where B is a NFA and w is a string.
2 Convert B to an equivalent DFA C.
3 Run the TM MD on input hC, wi
4 If MD accepts accept. If it rejects reject.

Sarmad Abbasi (Virtual University) Theory of Computation 15 / 39

Regular expressions

We can also look another problem:

AREX = {hR, wi : B is a regular expression that generates string w}

1 h0∗ 1∗ , 0011i ∈ AREX

2 h0∗ 1∗ , 10011i 6∈ AREX
3 h1(0 + 1)∗ , 10i ∈ AREX
4 h1(0 + 1)∗ , 0011i 6∈ AREX

Sarmad Abbasi (Virtual University) Theory of Computation 16 / 39

regular expressions

Is AREX decidable?

Once again think of the question intuitively.

We are asking if we can make an algorithm (Turing machine) that will
take as input a description of a regular expression R and a string
w ∈ {0, 1}∗ and tell us if R generates w?

Sarmad Abbasi (Virtual University) Theory of Computation 17 / 39

regular expressions

Once again, there is another approach. We know that every REX can
be converted to an equivalent DFA. In fact, there is an algorithm that
can convert any (suitable description of an) regular expression to a
(description of an) equivalent DFA.

Sarmad Abbasi (Virtual University) Theory of Computation 18 / 39

regular expressions

So we have the following solution also.

1 On input hR, wi where R is a regular expression and w is a string.
2 Convert R to an equivalent DFA C.
3 Run the TM MD on input hC, wi
4 If MD accepts accept. If it rejects reject.

Sarmad Abbasi (Virtual University) Theory of Computation 19 / 39

The Emptiness Problem for DFAs

Lets look at another problem which is more interesting.

EDFA = {hAi : A is a DFA and L(A) = ∅}.

The input to this problem is a description of a deterministic finite
automata A. We have to decide if the L(B) = ∅.

Sarmad Abbasi (Virtual University) Theory of Computation 20 / 39

The Emptiness Problem for DFAs

The problem on the surface looks very difficult. We have to make sure
that A does not accept any strings. However, the number of potential
strings is infinite. Thus it would be futile to simulate A on all the strings.
The process will never end.

Sarmad Abbasi (Virtual University) Theory of Computation 21 / 39

The Emptiness Problem for DFAs

So we may start suspecting that this is perhaps a difficult question and

designing a TM that tells us in finite time if L(A) = ∅ may not be
possible.
However, a little thought shows that it is possible to find a TM that
decides this problem.

Sarmad Abbasi (Virtual University) Theory of Computation 22 / 39

The Emptiness Problem for DFAs

The idea is that if we want to tell if the language of a DFA is non-empty.

All we are trying to see is if it is possible to reach some accepting
state. So, we can use the following method:
1 On input hAi
2 Mark the state state of A.
3 For i = 1, 2, . . . , n where n is the number of states.
4 Mark any state that has a transition coming into it from any
state that is already marked.
5 If any accept state gets marked reject. Otherwise, accept the
input.

Sarmad Abbasi (Virtual University) Theory of Computation 23 / 39

The Emptiness Problem for DFAs

This method can be explained in one line as follows:

1 On input hAi
2 Perform a DFS on the transition diagram of the A starting from the
start state.
3 If any final state gets marked reject. Otherwise accept.
Notice that we managed to avoid a potential infinite computation by a
clever method.
If the problem looks difficult think again. Think of ways to avoid
potential infinite computations.

Sarmad Abbasi (Virtual University) Theory of Computation 24 / 39

The equivalence Problem for DFAs

Lets look at another problem which is more interesting.

EQDFA = {hA, Bi : A and B are DFAs and L(A) = L(B)}.

The input to this problem is a description of two deterministic finite
automata A and B. We have to decide if the L(A) = L(B). That is if
they accept exactly the same language.

Sarmad Abbasi (Virtual University) Theory of Computation 25 / 39

The equivalence Problem for DFAs

Note the problem on the surface again looks very difficult. We have to
make sure that A and B accept exactly the same strings and reject
exactly the same strings. However, the number of potential strings that
we can as input is infinite. Thus it would be futile to simulate A and B
on all the strings. The process will never end.

Sarmad Abbasi (Virtual University) Theory of Computation 26 / 39

The Equivalence Problem for DFAs

So we may start suspecting that this is perhaps a difficult question. We

have seen an example like this already.
Note that L(A) 6= L(B) if either
1 there is an x ∈ L(A) ∩ L(B)
2 there is an x ∈ L(B) ∩ L(A)
In short
if
(L(A) ∩ L(B)) ∪ (L(B) ∩ L(A)) = ∅
then

L(A) = L(B)

Sarmad Abbasi (Virtual University) Theory of Computation 27 / 39

The Equivalence Problem for DFAs

Now, we know that the following properties of regular languages.

1 If P is regular and Q is regular then P ∪ Q is regular. Furthermore,
given two DFAs D1 and D2 we can construct a DFA such that D
such that
L(D) = L(D1 ) ∪ L(D2 ).
2 If P is regular then P is regular. Furthermore, given a DFA D1 we
can construct a DFA D 0 such that

L(D 0 ) = L(D).

Sarmad Abbasi (Virtual University) Theory of Computation 28 / 39

The Equivalence Problem for DFAs

Hence we can make a DFA C such that:

L(C) = (L(A) ∩ L(B)) ∪ (L(B) ∩ L(A)) = ∅

1 On input hA, Bi
2 Construct the DFA C described above.
3 If L(C) = ∅ accept. Else reject.

Sarmad Abbasi (Virtual University) Theory of Computation 29 / 39

The Acceptance Problem for CFGs

You have studied CFG in your previous automata course. Let us look
at the following problem:

ACFG = {hG, wi : B is a CFG that generates w}.

Sarmad Abbasi (Virtual University) Theory of Computation 30 / 39

The Acceptance Problem for DFAs

The input to this problem consist of two parts. The first part is going to
be a CFG G. The second part is going to be a string w. We have to
decide if the G generates w.
Note that the input will not be a CFG G but a suitable encoding of G.

Sarmad Abbasi (Virtual University) Theory of Computation 31 / 39

Acceptance Problem for DFAs

The question that we want to ask is:

Question
Is ACFG decidable?

Lets ponder on this question for a bit.

Sarmad Abbasi (Virtual University) Theory of Computation 32 / 39

Acceptance Problem for CFGs

We are asking if we can make an algorithm (Turing machine) that will

take as input a CFG G and a string w ∈ {0, 1}∗ and tell us if G
generates w?

Sarmad Abbasi (Virtual University) Theory of Computation 33 / 39

Acceptance Problem for CFGs

One idea is to try all possible derivation and check if we ever get the
string w. However, we do not know how many derivations to try and
this can be potentially infinite.
But we know how to convert a grammar into Chomsky Normal Form.
In chomsky normal form every string of length n has a derivation of
length at most 2n − 1.

Sarmad Abbasi (Virtual University) Theory of Computation 34 / 39

Acceptance Problem for CFGs

1 On input hG, wi
2 Convert G into Chomsky normal form G’.
3 List all derivations with 2n − 1 steps.
4 If one of them generates w accept. Otherwise reject.

Sarmad Abbasi (Virtual University) Theory of Computation 35 / 39

Acceptance Problem for NFAs

Note the above method is not the most efficient. You have studied
more efficient algorithms for this problem. You can recall the CYK
algorithm.

Sarmad Abbasi (Virtual University) Theory of Computation 36 / 39

The Emptiness Problem for CFGs

Lets look at another problem which is more interesting.

ECFG = {hGi : A is a CFG and L(A) = ∅}.

Note the problem on the surface looks very difficult. We have to make
sure that G does not generate any strings. In other words, we would
like to show that starting from the start symbol of G we can never get
to a string which consists only of terminals. Since, there are potentially
infinite number of derivations the problem looks difficult.

Sarmad Abbasi (Virtual University) Theory of Computation 37 / 39

The Emptiness Problem for CFGs

However, a little thought shows that it is possible to find a TM that

decides this problem.
The idea is that if we want to tell if the language of a CFG is
non-empty. All we are trying to see is if it is possible to generate all
terminals from the start symbol. So, we can use the following method:
1 On input hGi
2 Mark all the terminals symbols of G.
3 For i = 1, 2, . . . , n where n is the number of variables.
4 If U → U1 . . . Uk is a rule and U1 , . . . , Uk are all marked then mark
U
5 If the start symbol gets marked reject. Otherwise, accept.

Sarmad Abbasi (Virtual University) Theory of Computation 38 / 39

The equivalence Problem for CFGs

Lets look at another problem which is more interesting.

EQCFG = {hG1 , G2 i : G1 and G2 are CFGs and L(G1 ) = L(G2 )}.

Note the problem again looks very difficult. We have to make sure that
G1 and G2 generate exactly the same strings.

Question
Is EQCFG decidable?

We will find out in the next lecture.

Sarmad Abbasi (Virtual University) Theory of Computation 39 / 39