Theory of Computation

(Finite Automata)

Pramod Ganapathi
Department of Computer Science
State University of New York at Stony Brook

January 24, 2021

Contents

Contents
Deterministic Finite Automata (DFA)
Regular Languages
Regular Expressions
Nondeterministic Finite Automata (NFA)
Transformations
Non-Regular Languages
Deterministic Finite Automata (DFA)
Electric bulb

Problem
Design the logic behind an electric bulb.
Electric bulb

Problem
Design the logic behind an electric bulb.
Solution
Diagram.

Analysis.
States = {nolight, light}, Input = {off, on}
Finite Automaton.
on

off
Multispeed fan

Problem
Design the logic behind a multispeed fan.
Multispeed fan

Problem
Design the logic behind a multispeed fan.
Solution
Diagram. Finite Automaton.
0
Off
1 3
2
1 3
Analysis.
States = {0, 1, 2, 3}
Input = {, } 2
Automatic doors

Problem
Design the logic behind automatic doors in Walmart.
Automatic doors
Solution
Diagram.

Analysis.
States = {close, open}, Input = {left, right, neither}
Finite Automaton.
neither left, right
left, right

close open

neither
Basic features of finite automata

A finite automaton is a simple computer with extremely limited

memory
A finite automaton has a finite set of states
Current state of a finite automaton changes when it reads an
input symbol
A finite automaton acts as a language acceptor i.e., outputs
“yes” or “no”
Why should you care?

Deterministic Finite Automata (DFA) are everywhere.

ATMs
Ticket machines
Vending machines
Traffic signal systems
Calculators
Digital watches
Automatic doors
Elevators
Washing machines
Dishwashing machines
Thermostats
Train switches
(CS) Compilers
(CS) Search engines
(CS) Regular expressions
Why should you care?

Probabilistic Finite Automata (PFA) are everywhere, too.

Speech recognition
Optical character recognition
Thermodynamics
Statistical mechanics
Chemical reactions
Information theory
Queueing theory
PageRank algorithm
Statistics
Reinforcement learning
Price changes in finance
Genetics
Algorithmic music composition
Bioinformatics
Probabilistic forecasting
What is a decision problem?

Definition
A decision problem is a computational problem with a ‘yes’ or
‘no’ answer.
A computer that solves a decision problem is a decider.
Input to a decider: A string w
Output of a decider: Accept (w is in the language) or Reject
(w is not in the language)
w Decider yes/no
What is a decision problem?

English word Accept

Other word Reject

Language = English language = {milk, food, sleep, . . .} B Accept

Not in language = {zffgb, cdcapqw, . . .} B Reject
What is a decision problem?

Some strings Accept

Other strings Reject

How does a DFA work?
Problem
Does the DFA accept the string bbab?
a b
a
b
start q0 q1 q2
a, b
How does a DFA work?
Problem
Does the DFA accept the string bbab?
a b
a
b
start q0 q1 q2
a, b
Solution
The DFA accepts the string bbab. The computation is:
1. Start in state q0
2. Read b, follow transition from q0 to q1 .
3. Read b, follow transition from q1 to q1 .
4. Read a, follow transition from q1 to q2 .
5. Read b, follow transition from q2 to q1 .
6. Accept because the DFA is in an accept state q1 at the end of
the input.
How does a DFA work?
Problem
Does the DFA accept the string aaba?
a b
a
b
start q0 q1 q2
a, b
How does a DFA work?
Problem
Does the DFA accept the string aaba?
a b
a
b
start q0 q1 q2
a, b
Solution
The DFA rejects the string aaba. The computation is:
1. Start in state q0
2. Read a, follow transition from q0 to q0 .
3. Read a, follow transition from q0 to q0 .
4. Read b, follow transition from q0 to q1 .
5. Read a, follow transition from q1 to q2 .
6. Reject because the DFA is in a reject state q2 at the end of
the input.
How does a DFA work?

a b
a
b
bbab q0 q1 q2 Accept

a, b

a b
a
b
aaba q0 q1 q2 Reject

a, b
How does a DFA work?

Problem
What language does the DFA accept?
a b
a
b
start q0 q1 q2
a, b
How does a DFA work?

Problem
What language does the DFA accept?
a b
a
b
start q0 q1 q2
a, b
Examples
The DFA accepts the following strings:
b, ab, bb, aabbbb, ababababab, . . . B ends with b
baa, abaa, ababaaaaaa, . . . B ends with b followed by even a’s
The DFA rejects the following strings:
a, ba, babaaa, . . .
What language does the DFA accept?
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {, a, aa, aaa, aaaa, . . .}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {, a, aa, aaa, aaaa, . . .}
Solution
Language L: Σ∗ = {, a, aa, aaa, aaaa, . . .}
Expression: a∗
Deterministic Finite Automaton (DFA) M :
a

start q0
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {}
Solution
Language L: φ = {} B Empty language
Expression: φ
DFA M :
a

start q0
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {a, aa, aaa, aaaa, . . .}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {a, aa, aaa, aaaa, . . .}
Solution
Language L: Σ∗ − {} = {a, aa, aaa, aaaa, . . .}
Expression: a+
DFA M :
a
a
start q0 q1
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {}
Solution
Language L: = {}
Expression: 
DFA M :
a
a
start q0 q1
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {aaa}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {aaa}
Solution
Language L: {aaa}
Expression: aaa
DFA M :
a
a a a a
start q0 q1 q2 q3 q4
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with even size}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with even size}
Solution
Language L: {, aa, aaaa, aaaaaa, . . .}
Expression: (aa)∗
DFA M :
a
start q0 q1
a
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with odd size}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with odd size}
Solution
Language L: {a, aaa, aaaaa, . . .}
Expression: a(aa)∗
DFA M :
a
start q0 q1
a
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size divisible by 3}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size divisible by 3}
Solution
Language L: {, aaa, aaaaaa, aaaaaaaaa, . . .}
Expression: (aaa)∗
DFA M :
a a
start q0 q1 q2

a
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size not divisible by 3}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size not divisible by 3}
Solution
Language L: {a, aa, aaaa, aaaaa, . . .}
Expression: (a ∪ aa)(aaa)∗
DFA M :
a a
start q0 q1 q2

a
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size divisible by 6}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size divisible by 6}
Solution
Language L: {, aaaaaa, aaaaaaaaaaaa, . . .}
Expression: (aaaaaa)∗
DFA M :
a a a a a
start q0 q1 q2 q3 q4 q5

a
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size divisible by 6}
Solution
Language L: {, aaaaaa, aaaaaaaaaaaa, . . .}
Expression: (aaaaaa)∗
DFA M :
a a a a a
start q0 q1 q2 q3 q4 q5

a
Can you think of another approach?
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings of size divisible by 6}
Solution
Let n = string size
Observation
n mod 6 = 0 ⇐⇒ n mod 2 = 0 and n mod 3 = 0
Idea
Build DFA M1 for n mod 2 = 0.
Build DFA M2 for n mod 3 = 0.
Run M1 and M2 in parallel.
Accept a string if both DFAs M1 and M2 accept the string.
Reject a string if at least one of the DFAs M1 and M2 reject
the string.
It is possible to build complicated DFAs from simpler DFAs
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with size n where n mod 4 = 2}
Construct DFA for Σ = {a}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with size n where n mod 4 = 2}
Solution
Language L: {aaa, aaaaaaa, aaaaaaaaaaa, . . .}
Expression: aa(aaaa)∗
DFA M :
a a a
start rem 0 rem 1 rem 2 rem 3

a
What about strings with size n where n mod k = i?
Construct DFA for Σ = {a}

More Problems
Construct a DFA that accepts all strings from the language L =
{strings with size n} such that
n2 − 5n + 6 = 0
n ∈ [4, 37]
n is a perfect cube
n is a prime number
n satisfies a mathematical function f (n)
Specifying a DFA

The specification of DFA consists of:

A (finite) alphabet
A (finite) set of states
Which state is the start state?
Which states are the final states?
What is the transition from each state, on each input character?
What is a deterministic finite automaton (DFA)?

Deterministic = Events can be determined precisely

Finite = Finite and small amount of space used
Automaton = Computing machine
What is a deterministic finite automaton (DFA)?

Deterministic = Events can be determined precisely

Finite = Finite and small amount of space used
Automaton = Computing machine

Definition
A deterministic finite automaton (DFA) M is a 5-tuple
M = (Q, Σ, δ, q0 , F ), where,
1. Q: A finite set (set of states). B Space (computer memory)
2. Σ: A finite set (alphabet).
3. δ : Q × Σ → Q is the transition function.
B Time (computation)
4. q0 : The start state (belongs to Q).
5. F : The set of accepting/final states, where F ∈ Q.
Acceptance and rejection of strings

Definition
A DFA accepts a string w = w1 w2 . . . wk iff there exists a
sequence of states r0 , r1 , . . . , rk such that the current state
starts from the start state and ends at a final state when all
the symbols of w have been read.
A DFA rejects a string iff it does not accept it.
What is a regular language?

Definition
We say that a DFA M accepts a language L if
L = {w | M accepts w}.
A language is called a regular language if some DFA accepts or
recognizes it.
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with odd number of b’s}
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with odd number of b’s}
Solution
States
qodd : DFA is in this state if it has read odd b’s.
qeven : DFA is in this state if it has read even b’s.
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with odd number of b’s}
Solution
Language L: {strings with odd number of b’s}
Expression: a∗ b(a ∪ ba∗ b)∗ or a∗ ba∗ (ba∗ ba∗ )∗
DFA M :
a a
b

start qeven qodd

b
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings with odd number of b’s}
Solution (continued)
DFA M is specified as
Set of states is Q = {qeven , qodd }
Set of symbols is Σ = {a, b}
Start state is qeven
Set of accept states is F = {qodd }
Transition function δ is:
δ a b
qeven qeven qodd
qodd qodd qeven
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings containing bab}
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings containing bab}
Solution
States
qb : DFA is in this state if the last symbol read was b, but the
substring bab has not been read.
qba : DFA is in this state if the last two symbols read were ba,
but the substring bab has not been read.
qbab : DFA is in this state if the substring bab has been read in
the input string.
q: In all other cases, the DFA is in this state.
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings containing bab}
Solution (continued)
Language L: {strings containing bab}
Expression: (a∗ b+ aa)∗ bab(a ∪ b)∗
DFA M :
a, b
a b
b a b
start q qb qba qbab

a
Construct DFA for Σ = {a, b}

Problem
Construct a DFA that accepts all strings from the language
L = {strings containing bab}
Solution (continued)
DFA M is specified as
Set of states is Q = {q, qb , qba , qbab }
Set of symbols is Σ = {a, b}
Start state is q
Set of accept states is F = {qbab }
Transition function δ is:
δ a b
q q qb
qb qba qb
qba q qbab
qbab qbab qbab
Closure properties of regular languages

Properties
Let L1 and L2 be regular languages.
Then, the following languages are regular.
Complement. L1 = {x | x ∈ Σ∗ and x 6∈ L1 }.
Union. L1 ∪ L2 = {x | x ∈ L1 or x ∈ L2 }.
Intersection. L1 ∩ L2 = {x | x ∈ L1 and x ∈ L2 }.
Concatenation. L1 · L2 = {xy | x ∈ L1 and y ∈ L2 }.
Star. L∗1 = {x1 x2 . . . xk | k ≥ 0 and each xi ∈ L1 }.
Closure properties for languages

Operation
Language L1 ∪ L2 L1 ∩ L2 L0 L1 L2 L∗ LR LT
Regular 3 3 3 3 3 3 3
DCFL 7 7 3 7 7 7 7
CFL 3 7 7 3 3 3 3
Recursive 3 3 3 3 3 3 7
R.E. 3 3 7 3 3 3 3

L1 ∪ L2 = Union of L1 and L2
L1 ∩ L2 = Intersection of L1 and L2
L0 = Complement of L
L1 L2 = Concatenation of L1 and L2
L∗ = Powers of L
LR = Reverse of L
LT = Finite transduction of L (may include:
intersection/shuffle/perfect-shuffle/quotient with arbitrary regular languages)
Construct DFA for L1 ∪ L2

Problem
Construct a DFA that accepts all strings from the language
L = {strings with size multiples of 2 or 3} where Σ = {a}
Solution
Language L1 = {strings with size multiples of 2}
Language L2 = {strings with size multiples of 3}
a
start p0 p1
a
a a
start q0 q1 q2

a
Construct DFA for L1 ∪ L2

Solution (continued)
Language L1 ∪ L2 = {strings with size multiples of 2 and 3}
a a a
start q0 q1 q2
start p0 p1
a a

start (p0 , q0 ) (p0 , q1 ) (p0 , q2 )

a a a a
a a

(p1 , q0 ) (p1 , q1 ) (p1 , q2 )

Construct DFA for L1 ∪ L2

Union
Let M1 accept L1 , where M1 = (Q1 , Σ, δ1 , q1 , F1 )
Let M2 accept L2 , where M2 = (Q2 , Σ, δ2 , q2 , F2 )
Let M accept L1 ∪ L2 , where M = (Q, Σ, δ, q, F ). Then
Q = {(r1 , r2 ) | r1 ∈ Q1 and r2 ∈ Q2 } B Cartesian product
δ((r1 , r2 ), a) = (δ1 (r1 , a), δ2 (r2 , a)) ∀(r1 , r2 ) ∈ Q, a ∈ Σ
q0 = (q1 , q2 )
F = {(r1 , r2 ) | r1 ∈ F1 or r2 ∈ F2 }
Construct DFA for L1 ∩ L2

Problem
Construct a DFA that accepts all strings from the language
L = {strings with size multiples of 2 and 3} where Σ = {a}
Solution
Language L1 = {strings with size multiples of 2}
Language L2 = {strings with size multiples of 3}
a
start p0 p1
a
a a
start q0 q1 q2

a
Construct DFA for L1 ∩ L2

Solution (continued)
Language L1 ∩ L2 = {strings with size multiples of 2 and 3}
a a a
start q0 q1 q2
start p0 p1
a a

start (p0 , q0 ) (p0 , q1 ) (p0 , q2 )

a a a a
a a

(p1 , q0 ) (p1 , q1 ) (p1 , q2 )

Construct DFA for L1 ∩ L2

Intersection
Let M1 accept L1 , where M1 = (Q1 , Σ, δ1 , q1 , F1 )
Let M2 accept L2 , where M2 = (Q2 , Σ, δ2 , q2 , F2 )
Let M accept L1 ∩ L2 , where M = (Q, Σ, δ, q, F ). Then
Q = {(r1 , r2 ) | r1 ∈ Q1 and r2 ∈ Q2 } B Cartesian product
δ((r1 , r2 ), a) = (δ1 (r1 , a), δ2 (r2 , a)) ∀(r1 , r2 ) ∈ Q, a ∈ Σ
q0 = (q1 , q2 )
F = {(r1 , r2 ) | r1 ∈ F1 and r2 ∈ F2 }
Problems for practice

Problems
Assume Σ = {a, b} unless otherwise mentioned.
Construct DFA’s for the following languages and generalize:
L = {w | |w| = 2}
L = {w | |w| ≤ 2}
L = {w | |w| ≥ 2}
L = {w | na (w) = 2}
L = {w | na (w) ≤ 2}
L = {w | na (w) ≥ 2}
L = {w | na (w) mod 3 = 1}
L = {w | na (w) mod 2 = 0 and nb (w) mod 2 = 0}
L = {w | na (w) mod 3 = 2 and nb (w) mod 2 = 1}
L = {w | na (w) mod 5 = 3, nb (w) mod 3 = 2, and
nc (w) mod 2 = 1} for Σ = {a, b, c}
L = {w | na (w) mod 3 ≥ nb (w) mod 2}
Problems for practice

Problems (continued)
L = {b | binary number b mod 3 = 1} for Σ = {0, 1}
L = {t | ternary number t mod 4 = 3} for Σ = {0, 1, 2}
L = {w | w starts with a}
L = {w | w contains a}
L = {w | w ends with a}
L = {w | w starts with ab}
L = {w | w contains ab}
L = {w | w ends with ab}
L = {w | w starts with a and ends with b}
L = {w | w starts and ends with different symbols}
L = {w | w starts and ends with the same symbol}
L = {w | every a in w is followed by a b}
L = {w | every a in w is never followed by a b}
Problems for practice

Problems (continued)
L = {w | every a in w is followed by bb}
L = {w | every a in w is never followed by bb}
L = {w | w = am bn for m, n ≥ 1}
L = {w | w = am bn for m, n ≥ 0}
L = {w | w = am bn c` for m, n, ` ≥ 1} for Σ = {a, b, c}
L = {w | w = am bn c` for m, n, ` ≥ 0} for Σ = {a, b, c}
L = {w | second symbol from left end of w is a}
L = {w | second symbol from right end of w is a}
L = {w | w = a3 bxa3 such that x ∈ {a, b}∗ }
Equivalence of different computation models

Two machines or computational models are computationally

equivalent if they accept/recognize the same language.
The following models are computationally equivalent:
DFA, regular expressions, NFA, and regular grammars.

DFA

Regular Regular Regular

expressions Languages grammar

NFA
Closure properties for languages

Operation
Language L1 ∪ L2 L1 ∩ L2 L̄ L1 ◦ L2 L∗
DFA Easy Easy Easy Hard Hard
Regex Easy Hard Hard Easy Easy
NFA Easy Hard Hard Easy Easy

L1 ∪ L2 = Union of L1 and L2
L1 ∩ L2 = Intersection of L1 and L2
L̄ = Complement of L
L1 ◦ L2 = Concatenation of L1 and L2
L∗ = Powers of L
Regular Expressions
Example

Example
Arithmetic expression.
(5 + 3) × 4 = 32 = Number
Regular expression.
(a ∪ b)a∗ = {a, b, aa, ba, aaa, baa, . . .} = Regular language
Application
Regular expressions in Linux.
Used to find patterns in filenames, file content etc.
Used in Linux tools such as awk, grep, and Perl.
Google search: http://www.googleguide.com/advanced_
operators_reference.html
What is a regular expression?

Definition
The following are regular expressions.
, φ, a ∈ Σ.
If R1 and R2 are regular expressions, then the following are
regular expressions.
(Union.) R1 ∪ R2
(Concatenation.) R1 ◦ R2
(Kleene star.) R1∗
Examples

Regular language Regular expression

{} φ
{} 
{a} a
{a, b} a∪b
{a}{b} ab
{a}∗ = {, a, aa, aaa, . . .} a∗
{aab}∗ {a, ab} (aab)∗ (a ∪ ab)
({aa, bb} ∪ {a, b}{aa}∗ {ab, ba})∗ (aa ∪ bb ∪ (a ∪ b)(aa)∗ (ab ∪ ba))∗

Equality
Two regular expressions are equal if they describe the same
regular language. E.g.:
(a∗ b∗ )∗ = (a ∪ b)∗ ab(a ∪ b)∗ ∪ b∗ a∗ = (a ∪ b)∗ = Σ∗
Examples

Examples
Let Σ = a ∪ b, R+ = RR∗ , and Rk = R
| ·{z
· · R}
k times
L = {w | |w| = 2}
R = ΣΣ
L = {w | |w| ≤ 2}
R =  ∪ Σ ∪ ΣΣ
L = {w | |w| ≥ 2}
R = ΣΣΣ∗
L = {w | na (w) = 2}
R = b∗ ab∗ ab∗
L = {w | na (w) ≤ 2}
R = b∗ ∪ b∗ ab∗ ∪ b∗ ab∗ ab∗
L = {w | na (w) ≥ 2}
R = b∗ ab∗ ab∗ (ab∗ )∗
Rules

Beware of φ and 
Suppose R is a regular expression.
R∪φ=R
R◦=R
R ∪  may not equal R
(e.g.: R = 0, L(R) = {0}, L(R ∪ ) = {0, })
R ◦ φ may not equal R
(e.g.: R = 0, L(R) = {0}, L(R ◦ φ) = φ)
Rules

Rules
Suppose R1 , R2 , R3 are regular expressions. Then
R1 φ = φR1 = φ
R1  = R1 = R1 ∪ φ = φ ∪ R1 = R1
R1 ∪ R1 = R1
R1 ∪ R2 = R2 ∪ R1
R1 (R2 ∪ R3 ) = R1 R2 ∪ R1 R3
(R1 ∪ R2 )R3 = R1 R3 ∪ R2 R3
R1 (R2 R3 ) = (R1 R2 )R3
φ∗ = 
( ∪ R1 )∗ = ( ∪ R1 )+ = R1∗
R1∗ ( ∪ R1 ) = ( ∪ R1 )R1∗ = R1∗
R1∗ R2 ∪ R2 = R1∗ R2
R1 (R2 R1 )∗ = (R1 R2 )∗ R1
(R1 ∪ R2 )∗ = (R1 ∗ R2 )∗ R1∗ = (R2∗ R1 )∗ R2∗
Construct a regex for Σ = {a, b}

Problem
Construct a regular expression to describe the language
L = {w | na (w) is odd}
Construct a regex for Σ = {a, b}

Problem
Construct a regular expression to describe the language
L = {w | na (w) is odd}
Solution
Incorrect expressions.
b∗ ab∗ (ab∗ a)∗ b∗ B Why?
b∗ a(b∗ ab∗ ab∗ )∗ B Why?
Correct expressions.
b∗ ab∗ (b∗ ab∗ ab∗ )∗ B Why?
b∗ ab∗ (ab∗ ab∗ )∗ B Why?
b∗ a(b∗ ab∗ a)∗ b∗ B Why?
b∗ a(b ∪ ab∗ a)∗ B Why?
(b ∪ ab∗ a)∗ ab∗ B Why?
Construct a regex for Σ = {a, b}

Problem
Construct a regular expression to describe the language L =
{w | w ends with b and does not contain aa}
Construct a regex for Σ = {a, b}

Problem
Construct a regular expression to describe the language L =
{w | w ends with b and does not contain aa}
Solution
A string not containing aa means that every a in the string:
- is immediately followed by b, or
- is the last symbol of the string
Each string in the language has to end with b.
Hence, every a in each string of the language is immediately
followed by b
Regular expression is: (b ∪ ab)+
Construct a regex to recognize identifiers in C

Problem
Identifiers are the names you supply for variables, types, func-
tions, and labels.
Construct a regular expression to recognize the identifiers in the
C programming language i.e., L = {identifiers in C}
Construct a regex to recognize identifiers in C

Problem
Identifiers are the names you supply for variables, types, func-
tions, and labels.
Construct a regular expression to recognize the identifiers in the
C programming language i.e., L = {identifiers in C}
Solution
C identifier = FirstLetter OtherLetters
FirstLetter = English letter or underscore
OtherLetters = Alphanumeric letters or underscore
Let L = {a, . . . , z, A, . . . , Z} and D = {0, 1, . . . , 9}
Regular expression is:
R = FirstLetter ◦ OtherLetters
FirstLetter = (L ∪ _)
OtherLetters = (L ∪ D ∪ _)
Construct a regex to recognize decimals in C

Problem
Construct a regular expression to recognize the decimal num-
bers in the C programming language i.e.,
L = {decimal numbers in C}
Examples: 14, +1, −12, 14.3, −.99, 16., 3E14, −1.00E2, 4.1E−
1, and .3E + 2
Construct a regex to recognize decimals in C

Problem
Construct a regular expression to recognize the decimal num-
bers in the C programming language i.e.,
L = {decimal numbers in C}
Examples: 14, +1, −12, 14.3, −.99, 16., 3E14, −1.00E2, 4.1E−
1, and .3E + 2
Solution
C decimal number = Sign Decimals Exponent
Let D = {0, 1, . . . , 9}
Regular expression is:
R = Sign ◦ Decimals ◦ Exponent
Sign = (+ ∪ − ∪ )
Decimals = (D+ ∪ D+ .D∗ ∪ D∗ .D+ )
Exponent = ( ∪ E Sign D+ )
Nondeterministic Finite Automata (NFA)
Example NFA’s
Examples
e
a x
start 

b u
s 
s u n 
start
p y
s 

Difference DFA NFA

Multiple transitions 1 exiting arrow ≥ 0 exiting arrows
Epsilon transitions 7 3
Missing transitions No missing transitions Missing transitions mean
transitions to sink/reject
state
What is the intuition behind nondeterminism?

Intuition
Nondeterministic computation = Parallel computation
(NFA searches all possible paths in a graph to the accept state)
When NFA has multiple choices for the same input symbol,
think of it as a process forking multiple processes for parallel
computation.
A string is accepted if any of the parallel processes accepts the
string.
Nondeterministic computation = Tree of possibilities
(NFA magically guesses a right path to the accept state)
Root of the tree is the start of the computation.
Every branching point is the decision-making point consisting
of multiple choices.
Machine accepts a string if any of the paths from the root of
the tree to a leaf leads to an accept state.
Why care for NFA’s?

Uses of NFA’s
Constructing NFA’s is easier than directly constructing DFA’s
for many problems.
Hence, construct NFA’s and then convert them to DFA’s.
NFA’s are easier to understand than DFA’s.
Construct NFA for Σ = {0, 1}

Problem
Construct a NFA that accepts all strings from the language
L = {strings containing 11 or 101}
Solution

0, 1 0, 1

1 1
start q1 q2 q3 q4

How does the machine work for the input 010110?

What is the equivalent DFA for solving the problem?
Construct NFA for Σ = {0, 1}

Solution (continued)

Source: Anil Maheshwari and Michiel Smid’s Theory of Computation

Construct NFA for Σ = {a}

Problem
Construct a NFA that accepts all strings from the language
L = {strings of size multiples of 2 or 3 or 5}
Solution
a
2

a
 a a
start 3

a a a a a

5

What is the equivalent DFA for solving the problem?

What is a nondeterministic finite automaton (NFA)?

Nondeterministic = Event paths cannot be determined precisely

Finite = Finite and small amount of space used
Automaton = Computing machine

Definition
A nondeterministic finite automaton (NFA) M is a 5-tuple
M = (Q, Σ, δ, q0 , F ), where,
1. Q: A finite set (set of states). B Space (computer memory)
2. Σ: A finite set (alphabet).
3. δ : Q × (Σ ∪ ) → P (Q) is the transition function, where
P (Q) is the power set of Q. B Time (computation)
4. q0 : The start state (belongs to Q).
5. F : The set of accepting/final states, where F ∈ Q.
Convert NFA to DFA

Problem
Convert the NFA to a DFA.

Source: Anil Maheshwari and Michiel Smid’s Theory of Computation

Construct DFA for the given NFA

Solution
NFA M is specified as
Set of states is Q = {1, 2, 3}
Set of symbols is Σ = {a, b}
Start state is 1
Set of accept states is F = {1}
Transition function δ is:
δ a b 
1 {3} φ {2}
2 {1} φ φ
3 {2} {2, 3} φ

How do you convert this NFA to DFA?

Construct DFA for the given NFA

Solution
NFA M is specified as
Set of states is Q = {1, 2, 3}
Set of symbols is Σ = {a, b}
Start state is 1
Set of accept states is F = {1}
Transition function δ is:
δ a b 
1 {3} φ {2}
2 {1} φ φ
3 {2} {2, 3} φ

How do you convert this NFA to DFA?

If NFA has states Q, then construct a DFA with states P (Q).
Construct DFA for the given NFA

Solution (continued)
a a
φ−→φ {1, 2} −→?
b b
φ→− φ {1, 2} →− ?
a a
{1} −
→ {3} {1, 3} −→?
b b
{1} →
− φ {1, 3} →− ?
a a
{2} −
→ {1, 2} {2, 3} −→?
b b
{2} →
− φ {2, 3} →− ?
a a
{3} −
→ {2} {1, 2, 3} −
→?
b b
{3} →
− {2, 3} {1, 2, 3} →
− ?
Construct DFA for the given NFA
Solution (continued)

Source: Anil Maheshwari and Michiel Smid’s Theory of Computation

Construct DFA for the given NFA

Solution (continued)

Source: Anil Maheshwari and Michiel Smid’s Theory of Computation

Construct DFA for the given NFA

Convert NFA to DFA

Let N = (Q, Σ, δ, q, F ) be the NFA.
Let M = (Q0 , Σ, δ 0 , q 0 , F 0 ) be the DFA. Then
Q0 = P (Q) B Power set of Q
q 0 = C ({q}) B -closure of the start state
F 0 = {S ∈ Q0 | S ∩ F 6= φ} B S ∩ F 6= φ means that S
contains at least one accept state of N
δ 0 : Q0 × Σ → Q0 is defined as follows:
For all state S ∈ Q0 and for all letter a ∈ Σ,
δ 0 (S, a) = s∈S C (δ(s, a))
S
Union of NFA

Source: Margaret Fleck and Sariel Har-Peled’s Notes on Theory of Computation

Concatenation of NFA

Source: Margaret Fleck and Sariel Har-Peled’s Notes on Theory of Computation

Star of NFA

Source: Margaret Fleck and Sariel Har-Peled’s Notes on Theory of Computation

Construct a NFA for (aa ∪ aab)∗ b

Problem
Construct a NFA for the regular expression (aa ∪ aab)∗ b.
Construct a NFA for (aa ∪ aab)∗ b

Problem
Construct a NFA for the regular expression (aa ∪ aab)∗ b.
Solution

Source: John Martin’s Introduction to Languages and the Theory of Computation.

Construct a NFA for (aab)∗ (a ∪ aba)∗

Problem
Construct a NFA for the regular expression (aab)∗ (a ∪ aba)∗ .
Construct a NFA for (aab)∗ (a ∪ aba)∗

Problem
Construct a NFA for the regular expression (aab)∗ (a ∪ aba)∗ .
Solution

Source: John Martin’s Introduction to Languages and the Theory of Computation.

Non-Regular Languages
Regular or non-regular languages

Problems
Let Σ = {a, b} unless mentioned otherwise. Check if the lan-
guages are regular or non-regular (7):
L = {w | w = an and n ≤ 10100 }
L = {w | w = an and n ≥ 1}
L = {w | w = am bn and m, n ≥ 1}
L = {w | w = a∗ b∗ }
L = {w | w = an bn and n ≥ 1}
L = {wwR | |w| = 3}
L = {wwR | |w| ≥ 1}
L = {w | w = wR and |w| ≥ 1}
L = {w | w = a2i+1 b3j+2 and i, j ≥ 1}
L = {w | w = an and n is a square}
L = {w | w = an and n is a prime}
2
L = {w | w = ai bj and i, j ≥ 1}
Regular or non-regular languages

Problems
Let Σ = {a, b} unless mentioned otherwise. Check if the lan-
guages are regular or non-regular (7):
L = {w | w = an and n ≤ 10100 }
L = {w | w = an and n ≥ 1}
L = {w | w = am bn and m, n ≥ 1}
L = {w | w = a∗ b∗ }
L = {w | w = an bn and n ≥ 1} . . . . . . . . . . . . . . . . . . . . . . . . . . 7
L = {wwR | |w| = 3}
L = {wwR | |w| ≥ 1} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
L = {w | w = wR and |w| ≥ 1} . . . . . . . . . . . . . . . . . . . . . . . . . . 7
L = {w | w = a2i+1 b3j+2 and i, j ≥ 1}
L = {w | w = an and n is a square} . . . . . . . . . . . . . . . . . . . . . . 7
L = {w | w = an and n is a prime} . . . . . . . . . . . . . . . . . . . . . . 7
2
L = {w | w = ai bj and i, j ≥ 1} . . . . . . . . . . . . . . . . . . . . . . . . .7
Regular or non-regular languages

Problems (continued)
L = {w | na (w) = nb (w)}
L = {w | na (w) mod 3 ≥ nb (w) mod 5}
L = {w | w = ai bj and j > i ≥ 1}
L = {wxwR | x ∈ Σ∗ , |w|, |x| ≥ 1, and |x| ≤ 5}
L = {wxwR | x ∈ Σ∗ and |w|, |x| ≥ 1}
L = {xwwR y | x, y ∈ Σ∗ and |w|, |x|, |y| ≥ 1}
L = {xwwR | x ∈ Σ∗ and |w|, |x| ≥ 1}
L = {wwR y | y ∈ Σ∗ and |w|, |y| ≥ 1}
Regular or non-regular languages

Problems (continued)
L = {w | na (w) = nb (w)} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
L = {w | na (w) mod 3 ≥ nb (w) mod 5}
L = {w | w = ai bj and j > i ≥ 1} . . . . . . . . . . . . . . . . . . . . . . . 7
L = {wxwR | x ∈ Σ∗ , |w|, |x| ≥ 1, and |x| ≤ 5} . . . . . . . . . . 7
L = {wxwR | x ∈ Σ∗ and |w|, |x| ≥ 1}
L = {xwwR y | x, y ∈ Σ∗ and |w|, |x|, |y| ≥ 1}
L = {xwwR | x ∈ Σ∗ and |w|, |x| ≥ 1} . . . . . . . . . . . . . . . . . . . 7
L = {wwR y | y ∈ Σ∗ and |w|, |y| ≥ 1} . . . . . . . . . . . . . . . . . . . 7
How to prove that certain languages are not regular?

Pumping lemma
Many languages are not regular.
Pumping lemma is a method to prove that certain languages
are not regular.
Pumping property
If a language is regular, then it must have the pumping property.
If a language does not have the pumping property, then the
language is not regular. B Proof by contraposition
How to prove languages non-regular using pumping lemma?
Proof by contradiction.
Assume that the language is regular.
Show that the language does not have the pumping property.
Contradiction! Hence, the language has to be non-regular.
Pumping property of regular languages

Suppose a DFA M with s number of states accepts a very long

string w such that |w| ≥ s from a language L.
From pigeonhole principle, at least one state is visited twice.
This implies that the string went through a loop.
Pumping property of regular languages

Observations
Suppose string w has more characters than the number of states
in the DFA, i.e., |w| ≥ s
String w can be split into three parts i.e., w = xyz where
x: string before the first loop
y: string of the first loop
z: string after the first loop (might contain loops)
Loop must appear i.e., |y| ≥ 1
(x and z can be empty)
Loop must appear in the first s characters of w i.e., |xy| ≤ s
Pumping property of regular languages

Idea
An infinite number of strings can be pumped with loop length
and they must also be in the language.
Formally, for all i ≥ 0, xy i z must be in the language.
xz, xyz, xyyz, xyyyz, etc must also belong to the language.
Pumping lemma for regular languages

Theorem
Suppose L is a regular language over alphabet Σ. Suppose L is
accepted by a finite automaton M having s states. Then, every
long string w ∈ L satisfying |w| ≥ s can be split into three strings
w = xyz such that the following three conditions are true.
|xy| ≤ s.
|y| ≥ 1.
For every i ≥ 0, the string xy i z also belongs to L.
L = {an bn | n ≥ 0} is non-regular

Problem
Prove that L = {an bn | n ≥ 0} is not a regular language.
L = {an bn | n ≥ 0} is non-regular

Problem
Prove that L = {an bn | n ≥ 0} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = as bs .
Let w = xyz = ap aq ar bs
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
Reason: xyyz = ap aq aq ar bs = as+q bs 6∈ L.
xyyz has more a’s than b’s.
Contradiction! Hence, L is not regular.
L = {w | na (w) = nb (w)} is non-regular
Problem
Prove that L = {w | na (w) = nb (w)} is not a regular language.
L = {w | na (w) = nb (w)} is non-regular
Problem
Prove that L = {w | na (w) = nb (w)} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = (ab)s .
Let w = xyz =  (ab)1 (ab)s−1
We have |xy| ≤ s and |y| ≥ 1.
Also, xy i z must belong to L for all i ≥ 0.
xy i z belongs to L for all i ≥ 0.
No contradiction! Hence, L is regular.
L = {w | na (w) = nb (w)} is non-regular
Problem
Prove that L = {w | na (w) = nb (w)} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = (ab)s .
Let w = xyz =  (ab)1 (ab)s−1
We have |xy| ≤ s and |y| ≥ 1.
Also, xy i z must belong to L for all i ≥ 0.
xy i z belongs to L for all i ≥ 0.
No contradiction! Hence, L is regular.
Mistakes
Incorrect solution! Why? Multiple reasons:
1. If we cannot find a contradiction, that does not prove anything.
2. We must try for all possible values of x, y such that |xy| ≤ s.
3. The chosen string (ab)s is a bad string to work on.
L = {w | na (w) = nb (w)} is non-regular

Problem
Prove that L = {w | na (w) = nb (w)} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = as bs .
Let w = xyz = ap aq ar bs
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
Reason: xyyz = ap aq aq ar bs = as+q bs 6∈ L.
xyyz has more a’s than b’s.
Contradiction! Hence, L is not regular.
L = {w | na (w) = nb (w)} is non-regular

Problem
Prove that L = {w | na (w) = nb (w)} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = as bs .
Let w = xyz = ap aq ar bs
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
Reason: xyyz = ap aq aq ar bs = as+q bs 6∈ L.
xyyz has more a’s than b’s.
Contradiction! Hence, L is not regular.
Takeaway
1. Reduction! Reduce a problem to another. Reuse its solution.
Superset of a non-regular language

Problem
{an bn } is a subset of {w | na (w) = nb (w)}.
We used the fact that {an bn } is non-regular to prove that
{w | na (w) = nb (w)} is non-regular.
Is a superset of a non-regular language non-regular?
Superset of a non-regular language

Problem
{an bn } is a subset of {w | na (w) = nb (w)}.
We used the fact that {an bn } is non-regular to prove that
{w | na (w) = nb (w)} is non-regular.
Is a superset of a non-regular language non-regular?
Solution
No!
Σ∗ is a superset of every non-regular language.
But, it is regular.
L = {w | na (w) = nb (w)} is non-regular

Problem
Prove that L = {w | na (w) = nb (w)} is not a regular language.
Solution (without using pumping lemma)
Suppose L is regular.
We know that L0 = {w | w = ai bj , i, j ≥ 0} is regular.
As regular languages are closed under intersection,
L ∩ L0 must also be regular.
We see that L ∩ L0 = {w | w = an bn and n ≥ 0}.
But, this language was earlier proved to be non-regular.
Contradiction! Hence, L is not regular.
L = {ww} is non-regular
Problem
Prove that L = {ww} is not a regular language.
L = {ww} is non-regular
Problem
Prove that L = {ww} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose ww = as as .
Let ww = xyz = ap a1 as−p−1 as
We have |xy| ≤ s and |y| ≥ 1.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
Reason: xyyz = ap a1 a1 as−p−1 ap = as+1 as 6∈ L.
xyyz has odd number of a’s.
Contradiction! Hence, L is not regular.
L = {ww} is non-regular
Problem
Prove that L = {ww} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose ww = as as .
Let ww = xyz = ap a1 as−p−1 as
We have |xy| ≤ s and |y| ≥ 1.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
Reason: xyyz = ap a1 a1 as−p−1 ap = as+1 as 6∈ L.
xyyz has odd number of a’s.
Contradiction! Hence, L is not regular.
Mistakes
Incorrect solution! Why?
1. We must try all possible values of x, y such that |xy| ≤ s.
2. Try pumping with y ∈ {a2 , a4 , . . .} such that |y| ≤ s.
L = {ww} is non-regular

Problem
Prove that L = {ww} is not a regular language.
L = {ww} is non-regular

Problem
Prove that L = {ww} is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose ww = as bs as bs .
Let ww = xyz = ap aq ar bs as bs
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
Reason: xyyz = ap aq aq ar bs as bs = as+q bs as bs 6∈ L.
Contradiction! Hence, L is not regular.
L = {w | w = an , n is a square} is non-regular

Problem
2
Prove that L = {w | w = an } is not a regular language.
L = {w | w = an , n is a square} is non-regular

Problem
2
Prove that L = {w | w = an } is not a regular language.
Solution
Suppose L is regular. Then it must satisfy pumping property.
2
Suppose w = as .
2
Let w = xyz = ap aq ar as −s
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s.
Also, xy i z must belong to L for all i ≥ 0.
But, xyyz is not in L.
2 2
Reason: xyyz = ap aq aq ar as −s = as +q 6∈ L.
Because, s2 < s2 + q < (s + 1)2 .
Contradiction! Hence, L is not regular.
L = {w | w = an , n is prime} is non-regular

Problem
Prove that L = {w | w = an , n is prime} is not regular.
L = {w | w = an , n is prime} is non-regular

Problem
Prove that L = {w | w = an , n is prime} is not regular.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = am , where m is prime and m ≥ s.
Let w = xyz = ap aq ar
where |xy| ≤ s, |y| ≥ 1, and p + q + r = m.
Also, xy i z must belong to L for all i ≥ 0.
But, xy m+1 z is not in L.
Reason: xy m+1 z = ap aq(m+1) ar = am(q+1) 6∈ L.
Contradiction! Hence, L is not regular.
L = {w | w = am bn , m > n} is non-regular

Problem
Prove that L = {w | w = am bn , m > n} is not regular.
L = {w | w = am bn , m > n} is non-regular

Problem
Prove that L = {w | w = am bn , m > n} is not regular.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = as+1 bs .
Let w = xyz = ap aq ar bs
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s + 1.
Also, xy i z must belong to L for all i ≥ 0.
But, xz is not in L. B Pumping down
p r s p+r
Reason: xz = a a b = a b 6∈ L. s

Because, p + r ≤ s i.e., #a’s is not greater than #b’s.

Contradiction! Hence, L is not regular.
L = {w | w = am bn , m 6= n} is non-regular

Problem
Prove that L = {w | w = am bn , m 6= n} is not regular.
L = {w | w = am bn , m 6= n} is non-regular

Problem
Prove that L = {w | w = am bn , m 6= n} is not regular.
Solution
Suppose L is regular. Then it must satisfy pumping property.
Suppose w = as bs+s! .
Let w = xyz = ap aq ar bs+s!
where |xy| ≤ s, |y| ≥ 1, and p + q + r = s.
Also, xy i z must belong to L for all i ≥ 0.
But, xy i z is not in L for some i.
We pump aq to get as+s! bs+s! .
Reason: xy i z = ap aqi ar bs+s! = as+(i−1)q bs+s! 6∈ L.
This means (i − 1)q = s! =⇒ i = s!/q + 1.
Contradiction! Hence, L is not regular.
L = {w | w = am bn , m 6= n} is non-regular

Problem
Prove that L = {w | w = am bn , m 6= n} is not regular.
Solution (without using pumping lemma)
Suppose L is regular.
We know that L0 = {w | w = ai bj , i, j ≥ 0} is regular.
Let L00 = {w | w = an bn , n ≥ 0}.
As regular languages are closed under intersection and comple-
mentation, L = L0 − L00 = L0 ∩ L¯00 is regular.
This implies that L00 is regular.
But, the language L00 was earlier proved to be non-regular.
Contradiction! Hence, L is not regular.