AMBO UNIVERSITY
WOLISO CAMPUS
DEPARTMENT OF COMPUTER SCIENCE
Material for Automata and complexity theory
Prepared by Boki C.
August 24/2024
Chapters of the course
Chapter One: Introduction
Chapter Two: Regular Expressions and Languages
Chapter Three: Context Free Languages
Chapter Four: Push Down Automata
Chapter Five: Context Sensitive grammar
Chapter Six: Turing Machine
Objectives
Introduce concepts in automata theory and theory of
computation
Identify different formal language classes and their relationships
Design grammars and recognizers for different formal
languages
Prove or disprove theorems in automata theory using its
properties
Determine the decidability and intractability of computational
problems
Course Organization
We Categorize the course in to three parts:
Part I) Chapter One and Two
At the end of these chapters you will have mid exam
Part II) Chapter Three and Four
At the end of these chapters you will have assignment
Part III) Chapter Five and Six
At the end of this chapter you will have final exam from part II&III
Part I: Chapter one
Introduction
Definition of Automata and complexity theory
Components of Auomaton
Central Concepts in Automata
Finite Automata and Its Types
Definition(Automata)
Automata is the branch of computer science which used to study
the abstract computing devices.
The word automata comes from the Greek word automaton,
which means "self-acting, self-willed, self-moving".
An automaton (automata in plural) is an abstract self-propelled
computing device which follows a predetermined sequence of
operations automatically.
Examples: Coke dispensers, ATM machines, automatic car
washes, thermostats, automatic pilots of aircraft, missile
guidance systems, telephone networks, and controls of certain
kinds of automatic elevators are all automata.
Automata...
There are four major families of automaton :
Finite-state machine
Pushdown automata
Linear-bounded automata
Turing machine
Complexity Theory
Complexity Theory aims to make general conclusions of the
resource requirements of decidable problems (languages).
The computational model is a Turing Machine.
Time: the number of computation steps a TM machine makes to
decide on an input of size n.
Examples:
the human brain, communication and transport systems, software
and electronic systems, a living cell, and social and economic
organizations are all examples of complexity theory.
Measuring complexity involves an algorithm analysis to determine
how much time it takes while solving a problem (time complexity). To
evaluate an algorithm, a focus is made on relative rates of growth as
the size of the input grows
Components
The components of automata consist of specific materials and
devices, such as wires, transistors, levers, relays, gears, and so
forth, and their operation is based on the mechanics and
electronics of these parts.
Central Concepts of Automata
Symbols
Alphabets
Strings/words
Languages
Grammar
Symbols
Symbols are the representation of any object used for the
specific purposes.
For Example:
The letter ‘A’ is a character representing/symbolizing the first
letter of the english alphabets.
Alphabets
An alphabet is a finite, non-empty set of symbols
We use the symbol ∑ (sigma) to denote an alphabet
Examples:
Binary: ∑ = {0,1}
All lower case letters: ∑ = {a,b,c,..z}
Alphanumeric: ∑ = {a-z, A-Z, 0-9}
Alphabets...
Powers of Alphabets: Let ∑ be an alphabet.
∑k = the set of all strings of length k
∑* = ∑0 U ∑1 U ∑2 U …
∑ + = ∑ 1 U ∑2 U ∑3 U …
Strings/Words
A string or word is a finite sequence of symbols chosen from ∑
Empty string is (or “epsilon”)
Length of a string w, denoted by “|w|”, is equal to the number of
(non- ) characters in the string
E.g., x = 010100 |x| = 6
xy = concatentation of two strings x and y
Languages
L is a said to be a language over alphabet ∑, only if L ∑* this
is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
Let L be the language of all strings consisting of n 0’s
followed by n 1’s: L = {, 01, 0011, 000111,…}
Let L be the language of all strings of with equal number of
0’s and 1’s: L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}
Ø denotes the Empty language
Grammars
Grammar is a standard way of representing the language.
The grammar contains a set of production rules that make up
the language strings.
The set of all possible strings that can be derived from grammar
is known as the language of that grammar.
Grammars...(Type)
Type 0 is known as unrestricted grammar which are accepted
by Turing machines.
Rule 0: α!=Null
Type 1 is known as context-sensitive grammar which are
accepted by linear bound automata.
Rul1: |α| <= |β|
Type 2 is known as a context-free grammar which are accepted
by Push Down Automata.
Rule 2: A ε V
Type 3 Regular Grammar which are accepted by Finite
Automata & RE and recognized by Turing machines. and it is
the most restricted
Rule 3: A -->aB or A-->a
Grammar...(Elements)
Elements of Grammar in Automata
To get a better understanding of grammar, we first understand
its elements. Elements of Grammar depend on the types of
grammar. Represented by G = (V, T, P, S)
The essential elements of grammar are given below.
V (Variables also called Non-Terminal): Capital symbols
represent a finite number of non-empty sets, i.e., A, B, C. It’s
not a part of the string, which makes after-production rules.
T (Terminal): A finite set of Alphabets (Σ), Represented by
small letters, i.e., a,b,c.All variables are replaced with non-
terminals through production rules.Terminals are part of the
string, which makes a production rule.
Grammar...(Elements...)
P (Production Rules): The finite set of non-empty rules to make a
string of language i.e. P = { S → aSb , S → bSa , S → ∈ }
S (Start symbol): The start symbol is used to start the production rule
represented by S. Note: Above grammar generates the strings
through production rules having equal number of a’s and b’s. As
given Below
L = {ab, aabb, aaabbb, …….}
Grammar...(Examples)
Example of Grammar in Automata:
“I am going to school”. It is a valid example of grammar.
I going am to school. It is not a valid example of grammar.
Consider a grammar G = (V, T, P, S): where
V = { S } // Set of Non-Terminal symbols,
T = { 0 , 1 } // Set of Terminal symbols,
P = { S → 0S1S , S → 1S0S , S → ∈ } // Set of production rules,
S = { S } // Start symbol
This grammar generates the strings having an equal number of
0’s and 1’s.
Finite Automata
The finite automata or finite state machine is an abstract
machine that has five elements or tuples.
Input
Output
States of automata
State relation
Output relation
It has a set of states and rules for moving from one state to
another but it depends upon the applied input symbol.
Basically, it is an abstract model of a digital computer.
Finite Automata...(Type)
Deterministic Finite Automata (DFA)
The machine can exist in only one state at any given
time
Non-deterministic Finite Automata (NFA)
The machine can exist in multiple states at the same
time
Deterministic Finite Automaton (DFA)
A Deterministic Finite Automaton (DFA) consists of:
Q ==> a finite set of states
∑ ==> a finite set of input symbols (alphabet)
q0 ==> a start state
F ==> set of accepting states
δ ==> a transition function, which is a mapping between Q x
∑ ==> Q
So A DFA is defined by the 5-tuple:
{Q, ∑ , q0,F, δ }
Deterministic Finite Automaton (DFA)...
Input: a word w in ∑*
Question: Is w acceptable by the DFA?
Steps:
Start at the “start state” q0
For every input symbol in the sequence w do
Compute the next state from the current state, given the
current input symbol in w and the transition function
If after all symbols in w are consumed, the current state is
one of the accepting states (F) then accept w;
Otherwise, reject w.
Q1. construct DFA accepting all strings over (a,b) ending
with ab?
Non-deterministic Finite Automaton (NFA)
A Non-deterministic Finite Automaton (NFA) consists of:
Q ==> a finite set of states
∑ ==> a finite set of input symbols (alphabet)
q0 ==> a start state
F ==> set of accepting states
δ ==> a transition function, which is a mapping between Q x
∑ ==> subset of Q
An NFA is also defined by the 5-tuple:
{Q, ∑ , q0,F, δ }
Q1 set of all strings over (0,1) that start with 0.
Q2 set of all strings over(a,b) ends with bb.
Non-deterministic Finite Automaton
(NFA)..
Input: a word w in ∑*
Question: Is w acceptable by the NFA?
Steps:
Start at the “start state” q0
For every input symbol in the sequence w do
Determine all possible next states from all current states,
given the current input symbol in w and the transition
function
If after all symbols in w are consumed and if at least one of
the current states is a final state then accept w;
Otherwise, reject w.
Chapter Two
Regular Grammar, Expression and
Language
Definition(RG,RE,&RL)
Connection of RG, RE and RL
Pumping lemma
Regular Grammar
Regular grammar generates regular language. They have a
single non-terminal on the left-hand side and a right-hand side
consisting of a single terminal or single terminal followed by a
non-terminal.
The productions must be in the form:
A ⇢ xB
A ⇢ x
A ⇢ Bx where A, B ∈ Variable(V) and x ∈ T* i.e. string of
terminals
Types of Regular Grammar
Left linear grammar(LLG):
In LLG, the productions are in the form if all the productions
are of the form
A ⇢ Bx
A ⇢ x where A,B ∈ V and x ∈ T*
Right linear grammar(RLG):
In RLG, the productions are in the form if all the productions
are of the form
A ⇢ xB
A ⇢ x
where A,B ∈ V and x ∈ T*
Regular Grammar
FA for accepting strings that start with b
∑ = {a,b}
Initial state(q0) = A
Final state(F) = B
The RLG corresponding to FA is
A ⇢ bB
B ⇢ ∈/aB/bB
Regular Expressions
Regular expressions are a compact and convenient way to
define regular languages.
They use a set of special characters and operators to represent
different types of strings and sets of strings.
Example 1: RE = 1*(01*01*)* from this regular expression we
can get L = {ε,0101,011011,10101,11011011...}
Example 2: Write RE for the language accepting all the string
which are starting with 1 and ending with 0 over alphabet (0,1).
RE = 1(1+0)*0
L = {10,110,100,1110,1000...}
Example 3: Write RE for the language accepting all strings
containing any numbers of as and bs.
RE = (a+b)*
L = {ε, a, b, aa, bb, aaa, bbb...}
Convert the above RE to FA
Regular expressions...
Alphabet = {0,1}
L(01) = {01} => L(01) = L(0)L(1)={0}{1}={01}
L(01+0) = {01, 0} => L(01+0) = L(01) U L(0) = (L(0) L(1)) U L(0) = ({0}
{1}) U {0} = {01} U {0}={01,0}
L(0(1+0)) = {01, 00}.
Note order of precedence of operators.
L(0*) = {ε, 0, 00, 000,… }.
L((0+10)*(ε+1)) = all strings of 0’s and 1’s without two consecutive
1’s.
L((0+1)(0+1) ) = {00,01,10,11}
L((0+1)*) = all strings with 0 and 1, including the empty string
Regular Expressions...
All strings of 0’s and 1’s starting with 0 and ending with 1
0(0+1)*1
All strings of 0’s and 1’s with even number of 0’s
1*(01*01*)*
All strings of 0’s and 1’s with at least two consecutive 0’s
(0+1)*00 (0+1)*
All strings of 0’s and 1’s without two consecutive 0’s
((1+01)*(ε+0))
Regular Languages
Regular languages are formal languages that regular
expressions can describe and can also be recognized by finite
automata.
They are used to define sets of strings, such as sequences of
characters or words, that follow specific patterns.
Formally, a regular language can be defined as the collection of
all strings that are recognized by a finite automata (FA).
Some common examples of regular languages include:
Binary strings that represent even numbers
Set of strings that contain exactly two a‘s
The set of all binary numbers that are divisible by 3
The set of all strings that contain the substring “01”
Operations on Languages
Remember: A language is a set of strings and the following are
operations on the languages
Union ==> LUM = {w:w∈L or w∈M}
Concatenation==> L.M = {w:w=xy,x ∈L,y ∈M}
Power ==> L0 ={∈}, L1 = L, Lk+1 = L.Lk
Kleene Closure ==> L* = {∈,L1, L2, L3, and ...}
Regular Languages
Regular = Finite Automata
expressions (DFA, NFA, -NFA)
Syntactical
expressions Automata/machines
Regular
Languages
Formal language
classes
Connection of RG, RE and RL
Regular Expressions are an algebraic way to describe regular
languages.
Regular Expressions describe exactly the regular languages.
If E is a regular expression, then L(E) is the regular language it
defines.
A regular expression is built up of simpler regular expressions
using defining rules which are regular grammar.
Examples:
∑ = {a,b}
Initial state(q0) = A
Final state(F) = B
The RLG corresponding to FA is
A ⇢ bB
B ⇢ ∈/aB/bB = b(a*b*)*
Pumping lemma
Pumping Lemma is to be applied to show that certain languages
are not regular.
It should never be used to show a language is regular. If L is
regular, it satisfies Pumping Lemma.
If L does not satisfy Pumping Lemma, it is non-regular.
Two Pumping Lemmas are defined for
Regular Languages, and
Context – Free Languages
Pumping lemma
Pumping Lemma for Regular Languages
Let L be a regular language. Then there exists a constant ‘c’
such that for every string w in L − |w| ≥ c
We can break w into three strings, w = xyz, such that − |y| >
0 |xy| ≤ c For all k ≥ 0, the string xykz is also in L.
In simple terms, if a string y is ‘pumped’, or if y is inserted any
number of times, the resultant string still remains in L.
Pumping Lemma is used as a proof for irregularity of a
language.
Thus, if a language is regular, it always satisfies pumping
lemma.
If there exists at least one string made from pumping which is
not in L, then L is surely not regular.
Pumping lemma
Method to prove that a language L is not regular
At first, we have to assume that L is regular.
So, the pumping lemma should hold for L.
Use the pumping lemma to obtain a contradiction −
Select w such that |w| ≥ c
Select y such that |y| ≥ 1
Select x such that |xy| ≤ c
Assign the remaining string to z.
Select k such that the resulting string is not in L.
Pumping lemma
Example: L = {0n1n: n>=0} is not regular language and to proof that
it is not RL
1) Assume that L is regular
2) if it is regular take pumping/generating number p
3) Find Regular language S: S = 0p1p
4) Decompose S into xyz or into three.
for the language is regular it must fulfill the following cases
case 1: xyiz∈L for every i>=0.
case 2: |xy|<=p
case 3: |y|>=1
so pumping is based on the y and y should be either in
0, 1 or both 0 and 1 parts.
Pumping lemma
Example Example...
Now let us take 1) xy2z = 0000000011111
p:p=5 here 8 0s!=5 1s
S: S=0000011111
2) xy2z = 0000011111111
Divide S into xyz here 5b0s != 8 1s
1. for y in 0 part
S=0000011111
3) xy2z = 0000000111111
2. for y in 1 part here 7 0s != 6 1s
S = 0000011111
3. for y in both 0 and 1
Hence the language S do
S = 0000011111 not satisfy three rules.
Let us check xyiz not in L by Therefore the language
taking i = 2. A: A=0n1n for n>=0 is not
xyiz = xy2z regular language
Part II
Chapter Three: CFL
Definition(G, CFG, CFL)
Parsing and ambiguity
Sentential forms
Simplification of context free grammar
Chomsky’s hierarchy of grammars
Definition(G, CFG, CFL)
Grammar:
A finite set of formal rules that are generating syntactically correct
sentences.
The formal definition of grammar is that it is defined as four tuples
G=(N, T, P, S), where G is a grammar, which consists of a set of
production rules.
Context Free Grammar:
Context-free grammars (CFGs) are used to describe context-free
languages.
A context-free grammar is a set of recursive rules used to generate
patterns of strings.
A context-free grammar can describe all regular languages and more,
but they cannot describe all possible languages.
Since they are types of Grammar, context-free grammars are also
represented by four tuples G=(V,T,P,S).
Definition...
Context Free Language:
A context-free language is a language generated by a
context-free grammar.
They are more general (and include) regular languages.
The same context-free language might be generated by
multiple context-free grammars.
Parsing and ambiguity
Parser:
Parser is a program that generates a parse tree for the
given string. if the string is generated from the uderlying
grammar.
Example: id = id+id*id;=>parser.
And parser generate parse tree for this string based on
S->i = E;
E->E+T|T
T->T*F|F
F->id
Parsing and ambiguity
A parse tree, also known as a syntax tree, is a tree-like
representation of a string’s semantics according to a grammar.
The root is the start symbol and the leaves are terminals.
The process of generating parse tree is called parsing and are:
Top down(LMD) & Bottom up(RMD)
Parsing and ambiguity
Top down approach Bottom up approach
S->aABe,A->Abc | a, B->d
S-->aABe,A-->Abc | a, B-->d
let us take aabcde
let us take aabcde
S S=>aABe
S S=>aABe
=>aAbcBe
=>aAde
a A B e
=>aabcBe
A B =>aAbcde
=>aabcde
A =>aabcde
A b c d
left most derivation
a a b c d e
a right most derivation
Parsing and ambiguity
Ambiguity
a situation in which something has more than one possible
meaning and may therefore cause confusion.
A CFG is called ambiguous if there exists more than one
LMD or more than one RMD for a string which is generated
by grammar. There will also be more than one derivation
tree for a string in ambiguous grammar.
A CFL is called ambiguous if there is no unambiguous
grammar to define that language and it is also called
inherently ambiguous Context Free Languages.
Sentential forms
A Sentential Form in computer science refers to the
intermediate representation of a string during the parsing
process.
It consists of a sequence of symbols that can be derived from
the start symbol following the production rules of a grammar.
Let G = (V, T, P, S) be a CFG, and α ∈ (V ∪T)∗
* α we say that α is a sentential form.
If S ⇒
If S LM
⇒α we say that α is a left-sentential form,and
if S RM
⇒α we say that α is a right-sentential form
Note: L(G) is those sentential forms that are in T∗
.
Sentential forms
Example:consider Grammar 2) right sentential form(RMD)
E->id | E+E | E*E | (E) ->E*E
to get derivation of id*id+id ->E*E+E
->E*E+id
1) left sentential form(LMD)
->E*E
->E*id+id
->id*id+id
->id*E
->id*E+E
->id*id+E
->id*id+id
