Computing is recognizing/describing/generating/accepting a language.

1 Context-free grammars: introduction

Two functions associated with languages:

reading: for language recognizers; writing: for language generators

One way to generate a language is to use “expressions,” for example, regular expressions. A different
way is to use “substitution rules.”

Generating L = {a}∗ :

rules application
R1: S → aS
R2: S→e
rule #1 S ⇒ aS
rule #1 ⇒ a2 S
rule #1 ⇒ a3 S
rule #1 ⇒ a4 S
rule #2 ⇒ a6

Generating L = {a}∗ ∪ {b}∗

rules application
R1: S→M
R2: S→N
R3: M → aM
R4: N → bN
R5: M →e
R6: N →e
rule #1 S⇒M
rule #3 ⇒ aM
... .........
rule #5 ⇒ am
rule #2 S⇒N
... .........
rule #6 ⇒ bn

Problem 1 What does this grammar generate?

1
 rules application
R1: S → aSb
R2: S→e
rule #1 S ⇒ aSb
rule #1 ⇒ aaSbb
rule #1 ⇒ aaaSbbb
rule #2 ⇒ aaabbb

Problem 2 Consider the grammar given in the table below. Show the derivation of the string

aboywithaflowerseesthegirl

How to improve the grammar so that it would generate words separated by blanks?

1 hSEN T EN CEi → hN OU N -P HRASEihV ERB-P HRASEi

2 hN OU N -P HRASEi → hCM P LX-N OU N i |

3 hCM P LX-N OU N ihP REP -P HRASEi

4 hV ERB-P HRASEi → hCM P LX-V ERBi |

5 hCM P LX-V ERBihP REP -P HRASEi

6 hP REP -P HRASEi → hP REP ihCM P LX-N OU N i

7 hCM P LX-N OU N i → hART ICLEihN OU N i

8/9 hCM P LX-V ERBi → hV ERBi |hV ERBi hN OU N i-hP HRASEi

10/11 hART ICLEi → a|the

12/13/14 hN OU N i → boy|girl|flower

15/16 hV ERBi → likes|sees

17 hP REP i → with

Definition 3 A context-free grammar G is a quadruple (V, Σ, R, S) where

V set of variables (nonterminals)

Σ set of terminals; Σ ∩ V = ∅
R set of rules; a finite subset of V × (V ∪ Σ)∗
S start nonterminal.

2
 Notations:
A →G u or A → u: (A, u) ∈ R
u ⇒ v: u yields v; ∃x, y, w ∈ (V ∪ Σ)∗ , A ∈ V such that
u = xAy; v = xwy; A → w
⇒∗G : the reflexive, transitive closure of ⇒G
L(G): the language generated by G;
L(G) = {w : w ∈ Σ∗ &S ⇒∗G w}

derivation v0 ⇒G v1 ⇒G v2 . . . ⇒G vp
p the length of the derivation

Problem 4 Consider the grammar G = (V, Σ, R, S) where V = {S, A}; Σ = {a, b}; and

R = {S → AA; A → AAA | bA | Ab | a}.

1. Which strings of L(G) can be produced by derivations of four or fewer steps?

2. Give at least three distinct derivations for the string babbab;

3. For any m, n, p ≥ 0, describe a derivation in G of the string bm abn abp ;

4. Prove that L(G) comprises the set of all strings in Σ∗ that contain a positive even number of
a’s.

Definition 5 A parse (derivation) tree of a grammar G = (V, Σ, R, S) is a labeled rooted tree

obtained through the use of the following operations:

• A one node tree labeled A where A ∈ V ; the single node of this tree is the root and the leaf;
the yield is A.

• If A → w is a rule in G, then the root (resp. leaf ) of this tree is the node labeled A (resp.
w); the yield is w.

• If T1 . . . , Tn are parse trees with roots labeled A1 , . . . , An and with yields w1 , . . . , wn respec-
tively and A → A1 . . . An is a rule in R, then the tree T with a root label A whose n subtrees
are T1 , . . . , Tn is also a parse tree. The leaves of T are the leaves of all Ti ’s and the yield of
T is w1 . . . wn .

3
 A1 A2 An

w w w
1 2 n

A1 A2 An

w w wn
1 2

w = w w ... w new yield

1 2 n

2 Leftmost derivations

Definition 6 A derivation is called leftmost(resp. rightmost) if the leftmost (resp. rightmost)

nonterminal is always expanded.

Proposition 7 For every for w ∈ L(G), where G is a context-free grammar, there is a leftmost
derivation of w.

Proof: Induction on the location of a non-leftmost step in a given derivation.

Given a non-leftmost derivation of a word w, we construct a new derivation of w for which a non-
leftmost step, if any, is farther to the right than it was in the original derivation. We do it without
increasing the total length of the derivation.

We start by locating the 1st occurrence of a non-leftmost derivation.

S = w0 ⇒ w1 ⇒ ... wk ⇒ wk+1 ⇒ ... wn ∈ Σ∗

⇑

the first occurrence of a non-leftmost derivation

Initial derivation:

. . . ⇒ wk = αAβBγ ⇒ αAβδγ ⇒ ... ⇒ αAǫ ⇒ αζǫ ⇒ . . .

B→δ βδγ ⇒ ǫ A→ζ

4
New derivation:

. . . ⇒ wk = αAβBγ ⇒ αζβBγ ⇒ αζβδγ ⇒ ... ⇒ αζǫ ⇒ . . .

A→ζ B→δ βδγ ⇒ ǫ

To construct a leftmost derivation, we repeat the transformation above as long as there are occur-
rences of non-leftmost derivations.

Problem 8 Given a grammar

G : S → SS|bA|Ba|e; A → aa; B → bb

and a derivation

S ⇒ SS ⇒ SBa ⇒ SSBa ⇒ SSbba ⇒ SbAbba ⇒ Sbaabba ⇒ baabba,

construct the leftmost derivation of baabba.

Solution. The 1st occurrence of the non-leftmost step is in step 2:

. . . ⇒ SS ⇒ SBa ⇒ . . . .

Instead of expanding the 1st S we expanded the 2nd one. According to the algorithm from the
proof, we find the step which expands the 1st S. In our example, this is step #3. Thus, by the
algorithm, we switch these two steps (#2 and #3). Remember, if the steps were not neighbors, but
were separated by other steps, we would in addition, have placed the separating steps after these
two. The resulting derivation is as follows:

S ⇒ SS ⇒ SSS ⇒ SSBa ⇒ SSbba ⇒ SbAbba ⇒ Sbaabba ⇒ baabba.

The new derivation is not leftmost, but the first occurrence of a non-leftmost step is now farther
from the beginning although the total number of steps didn’t change. The 1st non-leftmost step
is now #3: . . . ⇒ SSS ⇒ SSBa ⇒ . . .. Again, we find the step which expands the 1st S (as you
can see, the nonterminal which was supposed to be expanded, but wasn’t, happened to be S again).
This time, the step dealing with the missed non-terminal is #7; it uses the rule S → e. Thus,
according to the algorithm, in the new derivation, we preserve the first two steps, then execute step
#7, followed by step #3, followed by the steps #4, 5, and 6. The result is:

S ⇒ SS ⇒ SSS ⇒ SS ⇒ SBa ⇒ Sbba ⇒ bAbba ⇒ baabba.

One can continue applying the algorithm pushing the non-leftmost steps to the right, until all of
them are removed. On a side, one can also see that the final derivation can be simplified, giving
rise to a shorter left-most derivation.

5
3 Transforming CFG’s.

Theorem 9
Let a context-free grammar G(V, Σ, R, S) contain a production of the form

A → uBv, where A, B ∈ V and u, v ∈ (V ∪ Σ)∗

and let the set of all productions for substituting B 6= A is

B → w1 |w2 | . . . |wn .

Construct a new grammar H by removing A → uBv and adding A → uwi v (i = 1, . . . , n). Then

L(G) = L(H).

Definition 10
A variable A is useful for a grammar G, if at least one derivation of a string in L(G) contains A.
Else, A is called useless. A production is useless if it contains any useless variable.

Theorem 11
For every grammar G, there is an equivalent grammar H (L(H) = L(G)), without useless variables
and useless productions.

Step 1.

Procedure Eliminate (G(V, Σ, R, S))

V1 = ∅;
while (not done)
V1 = V1 ∪ {all productions in G of the form A → w where w ∈ (V1 ∪ Σ)∗ }

Step 2.

Based on G1 , construct a dependency graph, defined as follows:

the set of vertices are variables of the grammar;

there is a directed edge (A, B) iff there is a production A → uBv.

Remove all variables that cannot be reached from S. Remove all productions containing removed
variables. The resulting grammar H has no useless variables or productions.

Definition 12
A production of the form A → ǫ is called a null-production (λ-production). A variable A for which
the derivation A ⇒∗ ǫ, is called nullable.

Theorem 13

6
If ǫ 6∈ L(G), then there is a grammar H such that

(1) L(G) = L(H), and

(2) H has no null productions.

Definition 14
A production of the form A → B, where A, B ∈ V , is called a unit-production.

Theorem 15
If ǫ 6∈ L(G), then there is a grammar H such that

(1) L(G) = L(H), and

(2) H has no unit productions.

Proof (idea). Find, for each variable A, all variables B 6= A, such that A ⇒∗ B. Include in H all
non-unit productions. Then for every pair A, B with A ⇒∗ B, add to H all rules A → w, for every
rule B → w currently in H.

4 Regular CFG’s.

Definition 16 A context free grammar is called regular if for every production T → w of G, all
letters of w, with a possible exception for the last one, are terminals.

R ⊆ V × Σ∗ (V ∪ {e}).

Example:
V = {S, T }; Σ = {a, b}
S → abaT
T → bbaS|aa

Theorem 17 A language is regular iff it is generated by a regular grammar.

Proof. We prove the statement by presenting two constructions. The first one constructs, for
a given DFA, an equivalent regular grammar. The second one constructs, for a given regular
grammar, an equivalent NFA (non-deterministic finite automaton).

⇒ Let M = (Q, Σ, δ, q0 , F ) be a DFA, then the regular grammar G = (V, Σ, R, S) is defined as

follows:

• set V of variables is the set Q of states in M ;

7
 • alphabet Σ is the same as for the DFA;

• the start variable S of G is q0 ;

• for every transaction q, p labeled with a ∈ Σ, i.e. q = δ(p, a), there is a rule p → aq in the
grammar;

• for every final state f ∈ F , there is a rule f → ǫ.

One can easily prove that the grammar generates the language L(M ) which is accepted by M . End
of ⇒

⇐ Let G = (V, Σ, R, S) be a regular grammar. Before constructing an equivalent finite automaton,

modify G as follows:

• replace every rule A → a1 a2 · · · ak B, where k ≥ 2 and B is a variable, with the rules A → a1 A1 ,

A1 → a2 A2 , . . . , Ak−1 → ak B. Here A1 , . . . , Ak are new variables.

• replace every rule A → a1 a2 · · · ak , where k ≥ 2, with the rules A → a1 A1 , A1 → a2 A2 , . . . ,

Ak−1 → ak . Here A1 , . . . , Ak−1 are new variables.

Obviously, the new grammar is equivalent to the initial one. All rules in the new grammar G′ are
of the form
A → aB or A → a,
where a is a terminal or the empty string ǫ, and B is a variable.

To construct an equivalent NFA,

• define every variable in V as a state of the new NFA;

• for every rule A → aB, introduce a transaction (A, B) labeled with a letter a;

• for every rule A → a (here a is either a letter or ǫ, the empty string), introduce a new state
F and a transaction (A, F ) labeled a; make F a final state;

• define the start variable to be the start state of the automaton.

One easily proves that the new NFA accepts the language generated by the grammar. End of ⇐

5 Chomsky Normal Form.

Definition 18 A context-free grammar is in Chomsky normal form if every rule is of the form

8
 A → BC A, B, C are variables; B, C 6= S
A → a a is a terminal;
S → ǫ S is the start variable.

Theorem 19 If L is context-free, then there is a context-free grammar in Chomsky normal form

H which generates this language.

Step 1. Add a new start variable S0 and a new rule S0 → S, where S is the old start variable.

Step 2. For every A → ǫ, where A 6= S,

• remove it;
• ∀B → uAv, replace it with B → uv;
/*If B → uAvAw, then we first add B → uvAw, then B → uAvw and then B → uvw.
*/
• If A → B, add B → ǫ, unless this rule was previously removed.

Repeat step 2 until all ǫ-rules not involving S are eliminated.

Step 3. For every rule A → B and for every rule B → w,

• remove A → B;
• add A → w, unless that was previously removed.

Repeat step 3 until all unit rules are eliminated.

Step 4. For every rule A → u1 u2 · · · uk , where k ≥ 3, replace it with the rules

A → u1 A1 ; A1 → u2 A2 ; . . . , Ak−2 → uk−1 uk ,

where Ai ’s are new variables.

For k > 1, replace any terminal ui in the preceding rules with the new variable Ui and add
the rule Ui → ui .

6 Some problems for CFG’s and some solutions.

Problem 20 Construct four parse trees for the four derivations of the string babbab in the grammar
G of problem 4 (section 1).

Problem 21 Consider the grammar G = (V, Σ, R, S) where

V = {S, A}; Σ = {a, b}; and

R= S → aAa | bAb | ǫ
A → SS.

9
 1. Which strings of L(G) can be produced by derivations of four or fewer steps?

2. Give a derivation of baabbb in G;

3. Construct the parse tree of the derivation from #2;

4. Describe L(G) in English?

Problem 22 Construct context-free grammars that generate each of the following languages:

1. L = {wcwR : w ∈ (a ∪ b)∗ };

2. L = {wwR : w ∈ (a ∪ b)∗ };

3. L = {w ∈ (a ∪ b)∗ } : w = wR };

4. L = {am bn : m ≥ n};

5. L = {am bn cp dq : m + n = p + q};

6. L = {w ∈ {a, b}∗ : na (w) = nb (w)};

(na (w) denotes the number of occurrences of letter a in the string w)

7. L = {w ∈ {a, b}∗ : na (w) = 2nb (w)};

8. L = {uawb : u, w ∈ {a, b}∗ &|w| = |u|};

Solutions to some problems:

• Construct a CFG that generates

L = {w ∈ {a, b}∗ : na (w) = nb (w)}.

Solution. The grammar which generates the language is as follows:

G: S → aSb|bSa|SS|e

Call property A of a string w to have na (w) = nb (w). It is easy to see that every string w ∈ L(G)
has property A. The reverse statement—every string with property A is in L—is also true but the
proof is not evident.

We prove it by induction on the length of |w|. Obviously, every string with property A must have
an even length. Thus, the base of the induction is two strings of length 2. It is easy to see that
they are in L.

Assume that the reverse statement is true for all strings of length n ≥ 2, and let |w| > n. If the first
and the last letters of w are distinct, w = σ1 w1 σ2 , σ1 6= σ2 , then using the appropriate production,
we reduce the statement to that on w1 which is correct by the induction hypothesis.

10
Let us now consider the case of the first and the last letters of w being equal, say w = aw1 a.
Moving from left to right, we add 1 for every a and subtract 1 for every b. After the first a, the
counter is 1; since at the end it must be 0, its value just before the last a must be -1.
Since every step changes the counter by 1, there must be a moment at which the value of the
counter is 0. Therefore, w can be presented as w1 w2 so that both w1 w2 where w1 and w2 have
property A.
To generate w using G, we use S → SS and derivations of w1 and w2 (the latter exist by induction
hypothesis).

• Construct a CFG that generates

L = {w ∈ {a, b}∗ : na (w) = 2nb (w)}.

Solution. The grammar which generates the language is as follows:

G: S → aaSb|aSab|abSa|aSba|bSaa|baSa|SS|aSbSa|e

Call B the property wa = 2wb of a word w. We use again induction on |w|. It is easy see that the
only case that is not trivial is the case of w of the type aaw1 aa or bbw1 bb. Consider the former
one. Starting with a 0, move from left to right adding 1 for each a and subtracting 2 after each b.
After the second letter, the counter equals 2, and before the last two a’s it is -2.
If for some intermediate position, the counter = 0, we follow the proof to the previous problem.
Since for our case the counter changes by 1 and 2, if it is never 0, we prove (make it exact !) that
two consecutive values of the counter are 1 and -1, implying that w = aw1 bw1 a where both w1 and
w2 have property B. Then we use S → aSbSa to finish the proof.

7 Parsing: algorithmic aspects

Two algorithmic problems related to representing a language by a grammar:

Problem #1: (the parsing problem):

given a grammar G and a string w, is w ∈ L(G)?

Problem #2: given a grammar G, is L(G) = ∅?

Parsing is the process of determining if a string of terminals (in the Compiler Theory, they are
called tokens) can be generated by a given grammar. Most parsing methods fall into one of the two
classes: top-down and down-up methods. Both construct the parse tree of the input string, but
do it in the opposite directions. The former starts at the root and proceeds towards leaves, while
the latter starts at the leaves and proceeds to the root.

An exhaustive top-down parsing: systematically construct all possible leftmost derivations

and see if any of them match w (the breadth-first search algorithm).

Example 23
11
Grammar G : S → SS|aSb|bSa|ǫ; string w = aabb

Round 1: S ⇒ SS; S ⇒ aSb; S ⇒ bSa; S ⇒ ǫ

the last two productions can be removed

Round 2: S ⇒ SS ⇒ SSS; S ⇒ SS ⇒ aSbS;

S ⇒ SS ⇒ bSaS; S ⇒ SS ⇒ S;
S ⇒ aSb ⇒ aSSb; S ⇒ aSb ⇒ aaSbb;
S ⇒ aSb ⇒ abSab; S ⇒ aSb ⇒ ab;

·······································

Round 3: S ⇒ aSb ⇒ aaSbb; S ⇒ aabb.

Idea for parsing: list out all derivations (= parse trees) that produce strings of length 1, 2, . . . , |w|.

Caveat: some of the rules are of the form A → α, where |α| ≤ 1, which may create infinite number
of derivations even though they produce a finite number of strings; thus, the algorithm must
understand how to eliminate repetitions.

New idea: modify G so that the resulting grammar G′ generates the same language but does not
have rules of the type A → ǫ and A → B.

Caveat: It is not always possible, since if ǫ ∈ L(G), then G must have a rule A → ǫ for some
A∈V.

Final idea: construct a new grammar G′ such that

• L(G) = L(G′ );
• G′ does not have any rule A → ǫ and A → B, where A, B ∈ V ;
• if ǫ ∈ L(G) then S → ǫ is allowed.

Question: why does the final idea work?

Answer: Let every rule of grammar L be of the form A → u where u is either a terminal symbol,
or |u| > 1, with only exception for S → ǫ. Then after each derivation step, different from
S → ǫ, either the derived string becomes longer, or the number of its terminals becomes
bigger. Thus, any derivation of w 6= ǫ has at most 2|w| − 1 steps. Consequently, we only
need to examine parse trees whose size is bounded by 2|w| − 1. Therefore, for every w, the
algorithm determines in a finite number of steps whether w ∈ L(G).

The implementation of the idea for parsing.

Eliminate all rules A → ǫ except for S → ǫ.

For every rule A → ǫ, if there is a rule B → uAv, add B → uv to the set of rules. Repeat
until no new rule can be added. Then remove all rules of the form A → ǫ, except S → ǫ, if it
is in the grammar.

12
 For the resulting grammar G′ , L(G) = L(G′ ). (Why ?)

Eliminate all rules A → B.

• For every two variables A and B, determine if A ⇒∗ B; (How?).

• Whenever B → u is a rule in G and A ⇒∗ B, add A → u. Repeat until no new rule can
be added. Eliminate all rules A → B.
For the resulting grammar G′′ , L(G) = L(G′′ ). (Why ?)

Problem #2: Given a grammar G, is L(G) = ∅?

Answer. If grammar G generates a nonempty language, then there is a parse tree whose yield is a
terminal string and whose height ≤ |V |. This is because every parse tree of height > |V | has a path
with two nodes labeled with the same variable. Then, a smaller parse tree can be constructed by
excising the part of the tree between the nodes with the same label. The process can be repeated
until a parse tree of height ≤ |V | is constructed with the terminal yield.

Thus, to answer the question, it is sufficient to search through all parse trees of height ≤ |V |; if
none of them yields a terminal string, the language is empty.

S S

A
11111111111
00000000000
A
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
A
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111
00000000000
11111111111

The solutions above to both problems are only of a theoretical value; a practical solution (when it
is available) is based on the notion of a pushdown automaton.

13
8 Ambiguity

Steve saw a man with a telescope

Two ways to parse (to understand) the sentence:

Steve used a telescope to see the man The man was with the telescope

A grammar is said to be ambiguous if two or more parse (derivation) trees exist for the same
string. A grammar which is not ambiguous is called unambiguous.

Example 24 S → SbS | ScS | a; S ⇒∗G abaca.

derivation #1: S ⇒ SbS ⇒ abS ⇒ abScS ⇒ abacS ⇒ abaca

derivation #2: S ⇒ ScS ⇒ SbScS ⇒ abScS ⇒ abacS ⇒ abaca

S S

S b S S c S

a S c S S b S a

a a a a

Example 25 S → aSb | SS | ǫ; S ⇒∗G abaca.

An unambiguous equivalent of the grammar: S → aT b | SS; T → S | ǫ;

Example 26 The grammar below describes a fragment of a programming language concerned with
arithmetic expressions.

A → I := E; I → a|b|c; E → E + E|E ∗ E|( E )|I.

A ⇒G a := b + c ∗ a

14
 A A

I := E I := E

E E *
E
a E + a

I E * E E + E I

b I I I I a

c a b c

An unbiguous version of the grammar for arithmetic expressions.

A → I := E; I → a|b|c; E → T | E + T ; T → F | T ∗ F ; F → I | ( E ).

I := E

a E + T

I T * F

b F I

I a

Example 27
S → if C then S | if C then S else S | statement; C → condition

The grammar is ambiguous; consider the string

if C1 then if C2 then S1 else S2 .

(if a = b then if b = c then x = 1 else x = 2)

15
 S

if C

C S
S

condition then if condition then statement else statement

S S
if C

C S

condition then if condition then statement else statement

An unambiguous equivalent grammar (idea):

S → M |U
M → if C then M else M | other
U → if C then S | if C then M else U

9 Pushdown Automata: finite automata + memory

Definition 28 A pushdown automaton M is a 6-tuple (Q, Σ, Γ, δ, q0 , F ), where

16
 Q: a finite set of states;

Σ: a finite alphabet of input symbols;

Γ: a finite alphabet of stack symbols;

q0 ∈ Q : the initial state;

F ⊆Q: a set of final states;

δ: the transition function;

Q × (Σ ∪ ǫ) × (Γ ∪ ǫ) → P(Q × (Γ ∪ ǫ)).

An input string w to a PDA M is considered to be written on the input tape as

w = w1 w2 · · · wm ,

where wi ∈ Σ ∪ ǫ ≡ Σǫ .

Given the current state q ∈ Q, the current letter u ∈ Σ ∪ ǫ of the input, and the current letter
a ∈ Γ ∪ ǫ ≡ Γǫ at the top of the stack, if

(q ′ , b) ∈ δ(q, u, a),

then M may select q as its next state and replace the top letter in the stack with b. If δ(q, u, a) = ∅,
no action of M is possible.

A computation by M begins in the start state with an empty stack.

If the input string w is written as

w = w1 w2 · · · wm ,
then for every i ∈ [1, m − 1],
(ri+1 , b) ∈ δ(ri , wi+1 , a),
where {ri } is the sequences of states; a, b are stack symbols.

A string w is accepted by M if at the end of w M enters a final state.

The definition of a PDA does not include a requirement of an empty stack at the end of computation
neither does it contain a formal mechanism to allow the PDA to test for an empty stack. However,
it is always possible to achieve that the stack is empty at the end by introducing a special symbol
$ which is initially placed on the stack. If the PDA ever sees the $ again, it “knows” that the stack
is effectively empty. Thus, we will always design pushdown automata that empty the stack by the
time they accept the string.

17
Definition 29 The language L(M ) accepted by M is the set of all strings accepted by M .

If (q, b) ∈ δ(p, u, a), we say that (p, u, a), (q, b) is a transition of M . A transition (p, u, a), (q, b) is
represented by a diagram composed of two nodes labeled p and q, respectively, and an arrow (p, q)
labeled with u, a → b.

u,a b
p q

Since δ(q, u, a) ⊆ Q × Γ ∪ ǫ, at every particular step, there can be more than one applicable
transitions; if δ(q, u, a) = ∅, then no transition is applicable.

The transition function δ is completely described by the set ∆ of its applicable transitions.

If M performs transition (q, u, a)(p, ǫ), we say that M pops a from the stack. The transition is
called a pop. If M performs transition (q, u, ǫ)(q, b), we say that M pushes b onto the stack; the
transition is called a push.

Example 30 G = (Q, Σ, Γ, δ, q1 , F ), where Q = {q1 , q2 , q3 , q4 }; Σ = {0, 1}; Γ = {0, $}; and F =

{q1 , q4 }. What is the language recognized by this pushdown automaton?

0,e 0 1,0 e

1,0 e
e,e $ e,$ e
q q q3 q4
1 2

Example 31 What is the language recognized by the pushdown automaton described below?

Q = {q0 , q1 , q2 , f }
Σ = {a, b, c} The transition function δ is represented by the
Γ = {a, b, $} diagram below. For all triples (q, u, a) that are not
F = {f } shown on the diagram, δ(q, u, a) = ∅.

10 Pushdown Automata and Context-free Grammars

Theorem 32 The class of languages accepted by pushdown automata is exactly the class of context-
free languages.

18
 =⇒: Each context-free language is accepted by a PDA.
⇐=: Each language accepted by a PDA is context-free.

Proving =⇒:
∀G = (V, Σ, R, S), ∆: (p, ǫ, ǫ)(q, S)
∃M = ({p, q, . . .}, Σ, V ∪ Σ, ∆, p, {q}) (q, ǫ, A)(q, w), ∀A → w
such that L(G) = L(M ). (q, a, a)(q, ǫ), ∀a ∈ Σ.

How to interpret a transition (q, u, a)(q, w), where w is a string?

e,e b

e,e a
u,a c
u,a abc q
p q p Interpretation

For every string that can be derived by a grammar G, the new PDA accepts the string by “mim-
icking” its leftmost derivation. The end of the derivation corresponds to the moment the PDA
finished reading the input-string and its stack is empty.

Example 33
G : V = {S, T }; Σ = {a, b}
R : {S → abT a; T → ba : . . .}
Derivations in G Transitions in M
S ⇒ abT a (p, ǫ, ǫ)(q, S); (q, ǫ, S)(q, abT a); Proving ⇐=:
(q, a, a)(q, ǫ); (q, b, b)(q, ǫ);
⇒ abbaa (q, ǫ, T )(q, ba);
(q, b, b)(q, ǫ); (q, a, a)(q, ǫ); (q, a, a)(q, ǫ).

Given PDA M , construct a grammar G, such that L(G) = L(M ).

First, we modify M so that it satisfies the following conditions:

1. the final state of M is unique; (easy)

2. when M accepts a string, its stack is empty; (easy)

3. each transition is either a push or a pop.

Let (p, u, a)(q, b) be a transition with a 6= ǫ and b 6= ǫ. Replace it with (p, u, a)(qnew , ǫ)&(qnew , ǫ, ǫ)(q, b),
where qnew is a new state.

Design G = (V, Σ, R, q0 ) by:

19
 V = {Ap,q } (∀p, q ∈ Q); Apq → aArs b ∀(p, a, ǫ)(r, t) & (s, b, t)(q, ǫ)
Σ set of terminals; Apq → Apr Arq , ∀p, q, r ∈ Q
Aq0 ,f the start variable; App → ǫ. ∀p ∈ Q.

Proof. We prove a more general statement:

∀p, q ∈ Q(M ), the set L(Ap,q ) of strings generated from Apq is the set Σpq of strings
that take M from state p to state q, starting and ending with the empty stack.

Claim 1. We prove that every w ∈ L(Ap,q ) is also a string in Σpq by induction on the length of
the derivation of w.

If the number of steps in the derivation of w is 1, then the rule used can only be App → ǫ. Obviously,
ǫ ∈ Σp,p, since it takes M from p to p with the stack empty at the beginning and the end.

Assuming the statement is correct for all derivations with the number of steps ≤ k ≥ 0, and let the
number of steps in the derivation of wnew is k + 1.

Case 1: the first step in the derivation of wnew is Apq ⇒ aAr,s b. Then wnew = aw′ b and the
following two transitions must exist in M : (p, a, ǫ)(r, t) and (s, b, t)(q, ǫ).

Case 2: the first step in the derivation of wnew is Apq ⇒ Apr Arq . Then wnew = w′ w′′ , where w′
and w′′ are two strings generated in G from Apr and Ar, q, respectively.

In both case, the statement about wnew follows from that for w′ (case 1) and for w′ &w′′ (Case 2).

Claim 2. We prove that Σpq ⊆ L(Apq ) by induction on the length of w.

If w ∈ Σpq , that is w can bring M from p to q with the empty stack at the beginning and the end,
then the first (resp. last) transition of w is a push (resp. pop).

Let |w| = 0. Then w = ǫ and the path starts and ends at p. The derivation of w is App → ǫ.

Let our statement be correct for strings of length k ≥ 0 and let string w of length k + 1 can bring M
from p to q with the empty stack at the beginning and the end. If moving M from p to q contains
a moment at which the stack is empty, the first step of a derivation of w in G is Apq → Apr Ar,q .

Else, let (p, a, ǫ)(r, t) and (q, b, s)(q, ǫ) be the first and the last steps, respectively, in the path from
p to q according to w. Since during no intermediate step the stack becomes empty, the symbol t
pushed into the stack the first step is the symbol which is popped during the last step. Thus, s = t,
and wnew = aw′ b. Furthermore, w′ takes M from r to s with the stack being empty initially and
at the end. Since |w′ | = k − 1, it can be derived from Lr,s . Then the first step of the derivation for
w is
Apq ⇒ aAr,s b.
The derivations that follow are those used for w′ .

20
11 Languages that are not context-free.

The pumping lemma for context-free languages is a technique for proving that certain languages
are not context-free. It states that every context-free language has a special value called pumping
length such that all sufficiently long strings in the languages can by “pumped”. The pumping
theorem is typically used to prove that some languages are not CFL: for this purpose, one proves
that a given language has no finite pumping length.

Theorem 34 (The Pumping Theorem) Let G be a context-free grammar which describes an infinite
language. Then there is a number p depending on G only such that any string w ∈ L(G) of length
> p can be rewritten as uvxyz so that

• |v| + |y| > 0;

• |vxy| ≤ p;

• uv i xy i z ∈ L(G) ∀i ≥ 0.

Important comment: the theorem asserts the existence of a sub-division into five substrings
that satisfy the three conditions above; it does not indicate what they are, or how many exist. Not
every subdivision would satisfy the three conditions.

Proof. Let an infinite CFL L be generated by a CFG G(V, Σ, R, S) and let b be the maximum
length of the string in the right-hand side of any rule. Since L is infinite, b ≥ 2.

Claim. In any parse tree of a string w ∈ L, the number of nodes on the k-th level
(k ≥ 0) of the tree does not exceed bk .
This claim is easily proved by induction of k. Indeed, for k = 0, it is trivially correct.
If the claim is correct for some level k, then the size of the level k + 1 can increase by at
most b times, since every node may have at most b children (for some nodes the number
of children is smaller than b, if the the corresponding rule has fewer than b letters in its
right-hand side.).

Set p = b|V |+2 and let w ∈ L with |w| > p. Take the parse tree for w with the smallest number
of nodes. The height of the tree is at least |V | + 2, and so is the length of the longest path of
this tree. The labels of the nodes of this path are variables, except for the leaf which is labeled
with a terminal. Since there are |V | + 2 labels of non-leaf nodes and V variables, by pigeonhole
principle, some variable A appear more than once on the path. Select A to be the variable that
repeats among the lowest |V | + 1 variables on the path. Split w into five parts u, v, x, y, and z
according to the figure below:

21
 S

w = u v x y z

|V|+2
| w | > b

u v x y z

Both subtrees vxy and x are generated by the some variable A, so we may substitute one for the
other and still obtain a valid tree. Repeating the substitution i times, we get i “pumped” parsed
trees of strings in L. This satisfies the first condition of the lemma.

u v y z
A

v x y

To satisfy the second condition, we must check, that v and y are not both empty strings. Indeed,
if they were, the subtree generated by the upper A would be a path and a tree generated from the
last vertex of the path; the labels of the first and the last vertices of the path would be A. But
then, we would be able to construct a smaller parse tree for w by simply removing the edges of the
path. This would contradict the minimality of the initial parse tree for w.

To show that the third condition of the theorem holds, notice that in the parse tree, the upper
occurrence of A generates vxy. By our choice, both occurrences of A are among the bottom |V | + 2
nodes of the longest path of the tree. So the height of the subtree generated by A is at most |V | + 2.
This implies that the length of the yield of this subtree is at most b|V |+2 = p. This completes the
proof of the pumping theorem.

Problem 35

22
 • Prove that L = {an bn cn : n ≥ 0} is not context-free.

• Prove that L = {ww : w ∈ (a ∪ b)∗ } is not context-free.

12 Stronger implementations of the pumping idea

Here we consider strings in a CFL language L and mark some letters of the string. We show that
there is a positive integer p such that if w ∈ L and p of its characters are marked, then w is
“pumpable” in such a way that one of the pumped strings contains a marked character.

Definition 36 If some letters of a string w are marked and u is a substring of w, then the weight
of u (w.r.t. the marking of w) is the number of marked symbols in u. The weight is denoted
weight(u).

Lemma 37 (Ogden’s Derivation-Pumping Lemma.) Let G be a context-free grammar. Then there

is a positive integer p such that for any w ∈ L(G), if at least p characters of w are marked, then
there is a variable A and splitting w = uvxyz of w such that

1. weight(v) + weight(y) ≥ 1;

2. weight(vxy) ≤ p;

3. S ⇒∗ uAz;

4. A ⇒∗ vAy;

5. A ⇒∗ x;

6. ∀i ≥ 0, A ⇒∗ v i xy i .

Theorem 38 (Ogden’s Pumping Theorem.) Let L be a CFL. Then there is a positive integer p
(depending on L) such that for any w ∈ L, if at least p characters of w are marked, then there exist
splitting w = uvxyz such that

1. weight(v) + weight(y) ≥ 1;

2. weight(vxy) ≤ p;

3. ∀i ≥ 0, uv i xy i z ∈ L.

Example 39 Let L = {ai bj ck : i 6= j and j = k }. Then L is not context-free.

23
Solution. Assume that L is a CFL and let p be the pumping number as in Ogden’s theorem.
Consider
w = ap!+p bp cp ,
where p! = 1 × 2 × . . . × p. Mark all b’s. Then w = uvxyz for some u, v, x, y, z satisfying Ogden’s
theorem, in particular, one of v or y overlaps with the b-substring of w. (Notice, that this overlap
is not guaranteed by the original pumping theorem.)

Claim: For some m ≤ p, v = bm and y = cm . Indeed

1. v and y must belong to either a∗ , or b∗ , or c∗ , since otherwise pumping would create strings
not even in a∗ b∗ c∗ ;

2. if v = ar , for some r, then y would be the one to contain marked symbols (they are b’s only),
and pumping would increase the number of b’s without increasing the number of c’s; this
would yield a string not in L;

3. if v = cr , then y would contain only c’s and pumping would increase the number of c’s without
increasing the number of b’s;

4. the remaining option for v is to be bm ; in this case y must be cm so as to guarantee that the
increase of b’s equals that of c’s.

Now let us pump i times for i = p!/m + 1. Then

uv i xy i z = uvv p!/m xy p!/m yz = uvbp! xcp! yz = ap+p!bp+p! cp+p! .

The resulting string is not in L, thus the initial assumption that L is a CFL is incorrect.

Theorem 40 There exists an inherently ambiguous context-free language.

Proof. We will show that for any grammar generating L = {ai bj ck : i = j or j = k}, there is a
string of the form am bm cm that can be parsed in two different ways.

Let G be a grammar for L and let p > 1 be the pumping number for G as in Ogden’s lemma.
Let w0 = ap bp cp+p! . Mark all b’s. By Ogden’s lemma, G contains a variable A and a splitting
w0 = uvxyz such that

1. S ⇒∗ uAz; and

2. ∀i ≥ 0, A ⇒∗ v i xy i .

The only way to pump without getting strings out of L is if v and y are of the form v = ak and
y = bk for 1 ≤ k ≤ p (if v or y contained more than one distinct character, pumping would yield a
string not even of the form a∗ b∗ c∗ ; furthermore, |v| and |y| must be equal, for otherwise pumping
down produces a string with all different numbers of a’s, b’s, and c’s).

24
Letting i = p!/k + 1, we see that A ⇒ ap!+k xy p!+k , so

S ⇒∗ u A z ⇒∗ u ap!+k x bp!+k z ⇒∗ u ap! v x bp! y z ⇒∗ ap+p! bp+p! cp+p!

Thus, there is a derivation of ap+p! bp+p!cp+p! in which ap!+k xbp!+k is a phrase; (a phrase is defined
to be the yield of a subtree of a parse tree provided the subtree’s root is a variable.)

If we apply the same argument to w′ = ap+p! bp cp to derive ap+p!bp+p! cp+p! , we discover that this
derivation contains a phrase bp!+l x′ cp!+l for some l ≥ 0.

Each of the two phrases, ap!+k xbp!+k and bp!+l x′ cp!+l , contains at least p! + 1 b’s, and the strings
derived contain just p! + p b’s; thus each of the phrases contain more than a half of the b’s that are
derived. Hence, the two phrases overlap. The 1st string contain some a’s but no c’s; the 2nd string
contains some c’s but no a’s. Therefore neither phrase is a substring of the other, Thus, the parse
trees are different, and the grammar is ambiguous.

Problem 41 Prove that the following languages are not CFLs:

1. {ai bj ck : i < j < k}; ({ai bj ck : j < i = k});

2
2. {ai };

3. {ai : i is composite};
Hint: let q be the least prime greater than (p!+1)!+1, and show that in fact q > (p!+1)!+p!+1.
Then let i = q − p! and apply the pumping theorem.

4. L = {ai bj ci dj : i, j ≥ 0}.

Problem 42 Prove that L = {ai bj ck : i 6= j and j = k} satisfies the conclusions of the original
pumping theorem (we already know that L is not CFL which was proved using Ogden’s pumping
theorem.)

25