Screenshot 2025-02-01 at 5.32.27 PM
Screenshot 2025-02-01 at 5.32.27 PM
www.keralanotes.com
KTU STUDY MATERIALS
COMPILER DESIGN
Module 1
Related Link :
www.keralanotes.com
https://www.keralanotes.com/
CS304 Compiler Design /B.Tech/S6
MODULE 1
om
.c
Introduction to compilers – Analysis of the source
es
An important role of the compiler is to report any errors in the source program that it
detects during the translation process.
Error Meeages
om
.c
Fig: Compiler
es
Lexical Analysis
Syntax Analysis
Semantic Analysis
Lexical Analysis
In a compiler linear analysis is called lexical analysis or scanning. The lexical analysis
phase reads the characters in the source program and grouped into tokens that are
sequence of characters having a collective meaning.
EXAMPLE
7. The number 60
Blanks separating characters of these tokens are normally eliminated during lexical
analysis.
om
Syntax Analysis
Hierarchical Analysis is called parsing or syntax analysis.
.c
It involves grouping the tokens of the source program into grammatical phrases that
es
are used by the complier to synthesize output. They are represented using a syntax
tree.
ot
A syntax tree is the tree generated as a result of syntax analysis in which the interior
nodes are the operators and the exterior nodes are the operands. This analysis shows
n
Semantic Analysis
This phase checks the source program for semantic errors and gathers type
information for subsequent code generation phase.
Here the compiler checks that each operator has operands that are permitted by the
source language specification.
Lexical Analysis
om
Syntax Analysis
Semantic Analysis
.c
Intermediate Code Generation
es
Code Optimization
Target Code Generation
n ot
la
ra
ke
Lexical Analysis
The first phase of a compiler is called lexical analysis or scanning.
The lexical analyzer reads the stream of characters making up the source program and
groups the characters into meaningful sequences called lexemes.
For each lexeme, the lexical analyzer produces as output a token of the form
In the token, the first component token- name is an abstract symbol that is used during
syntax analysis, and the second component attribute-value points to an entry in the
symbol table for this token.
Information from the symbol-table entry 'is needed for semantic analysis and code
om
generation.
1. position is a lexeme that would be mapped into a token <id, 1>, where id is an
abstract symbol standing for identifier and 1 points to the symbol table entry for
n
position. The symbol- table entry for an identifier holds information about the
identifier, such as its name and type.
la
2. The assignment symbol = is a lexeme that is mapped into the token < = >. Since
ra
3. initial is a lexeme that is mapped into the token < id, 2> , where 2 points to the
ke
< id, l > < = > <id, 2> <+> <id, 3> < * > <60>
Token : Token is a sequence of characters that can be treated as a single logical entity. Typical
tokens are,
Identifiers
keywords
operators
special symbols
constants
Pattern : A set of strings in the input for which the same token is produced as output. This
set of strings is described by a rule called a pattern associated with the token.
Lexeme : A lexeme is a sequence of characters in the source program that is matched by the
pattern for a token.
om
.c
es
n ot
la
Syntax Analysis
ra
The parser uses the first components of the tokens produced by the lexical analyzer to
ke
The tree has an interior node labeled with ( id, 3 ) as its left child and the integer 60 as
its right child.
The node labeled * makes it explicit that we must first multiply the value of rate by 60.
The node labeled + indicates that we must add the result of this multiplication to the
value of initial.
The root of the tree, labeled =, indicates that we must store the result of this addition
into the location for the identifier position.
Semantic Analysis
The semantic analyzer uses the syntax tree and the information in the symbol table to
check the source program for semantic consistency with the language definition.
om
It also gathers type information and saves it in either the syntax tree or the symbol
table, for subsequent use during intermediate-code generation.
An important part of semantic analysis is type checking, where the compiler checks
.c
that each operator has matching operands.
es
For example, many programming language definitions require an array index to be an
integer; the compiler must report an error if a floating-point number is used to index
ot
an array.
For example, if the operator is applied to a floating point number and an integer, the
la
In our example, suppose that position, initial, and rate have been declared to be
floating- point numbers, and that the lexeme 60 by itself forms an integer.
ke
In the following figure, notice that the output of the semantic analyzer has an extra
node for the operator inttofloat , which explicitly converts its integer argument into a
floating-point number.
Syntax trees are a form of intermediate representation; they are commonly used
during syntax and semantic analysis.
After syntax and semantic analysis of the source program, many compilers generate
an explicit low-level or machine-like intermediate representation, which we can think
of as a program for an abstract machine.
om
In our example, the intermediate representation used is three-address code, which
consists of a sequence of assembly-like instructions with three operands per
instruction.
.c
t1 = inttofloat(60)
es
t2 = id3 * t1
t3 = id2 + t2
ot
id1 = t3
n
Code Optimization
la
The objectives for performing optimization are: faster execution, shorter code, or target
ke
t1 = id3 * 60.0
id1 = id2 + t1
Code Generator
The code generator takes as input an intermediate representation of the source
program and maps it into the target language.
If the target language is machine code, registers or memory locations are selected for
each of the variables used by the program.
If the target language is assembly language, this phase generates the assembly code as
its output.
The above code loads the contents of address id3 into register R2, then multiplies it
om
with floating-point constant 60.0.
Finally, the value in register Rl is stored into the address of idl , so the code correctly
implements the assignment statement position = initial + rate * 60.
ot
Symbol Table
n
An essential function of a compiler is to record the variable names used in the source
la
These attributes may provide information about the storage allocated for a name, its
type, its scope (where in the program its value may be used), and in the case of
ke
procedure names, such things as the number and types of its arguments, the method
of passing each argument (for example, by value or by reference), and the type
returned.
The symbol table is a data structure containing a record for each variable name, with
fields for the attributes of the name.
The data structure should be designed to allow the compiler to find the record for each
name quickly and to store or retrieve data from that record quickly.
However, after detecting an error, a phase must somehow deal with that error, so that
compilation can proceed, allowing further errors in the source program to be detected.
A compiler that stops when it finds the first error is not a helpful one.
LEXICAL ANALYZER
< id, l > < = > <id, 2> <+> <id, 3> < * > <60>
SYNTAX ANALYZER
SEMANTIC ANALYZER
om
.c
es
ot
t1 = inttofloat(60)
la
t2 = id3 * t1
t3 = id2 + t2
ra
id1 = t3
ke
CODE OPTIMIZER
t1 = id3 * 60.0
id1 = id2 + t1
CODE GENERATOR
ADDF R1, R2
STF id1, R1
Analysis Phase
Synthesis phase
Analysis Phase
Analysis Phase performs 4 actions namely:
a. Lexical analysis
b. Syntax Analysis
c. Semantic analysis
d. Intermediate Code Generation
om
The analysis part breaks up the source program into constituent pieces and imposes a
grammatical structure on them.
The analysis part also collects information about the source program and stores it in a
n
data structure called a symbol table, which is passed along with the intermediate
la
The analysis part breaks up the source program into constituent pieces and imposes a
ra
If the analysis part detects that the source program is either syntactically ill formed or
semantically unsound, then it must provide informative messages, so the user can take
corrective action.
The analysis part also collects information about the source program and stores it in a
data structure called a symbol table, which is passed along with the intermediate
representation to the synthesis part.
Synthesis Phase
Synthesis Phase performs 2 actions namely:
a. Code Optimization
b. Code Generation
The synthesis part constructs the desired target program from the intermediate
representation and the information in the symbol table.
The analysis part is often called the front end of the compiler; the synthesis part is the
back end.
Parser Generators
Scanner Generators
Syntax-directed translation engine
Automatic code generators
om
Data-flow analysis Engines
Compiler Construction toolkits
Parser Generators.
.c
Input : Grammatical description of a programming language
es
Output : Syntax analyzers.
ot
These produce syntax analyzers, normally from input that is based on a context-free
grammar.
n
In early compilers, syntax analysis consumed not only a large fraction of the running
la
time of a compiler, but a large fraction of the intellectual effort of writing a compiler.
ra
Scanner Generators
ke
The basic organization of the resulting lexical analyzer is in effect a finite automaton.
These produce collections of routines that walk the parse tree, generating intermediate
code.
The basic idea is that one or more "translations" are associated with each node of the
parse tree, and each translation is defined in terms of translations at its neighbour
nodes in the tree.
Such a tool takes a collection of rules that define the translation of each operation of
the intermediate language into the machine language for the target machine.
The rules must include sufficient detail that we can handle the different possible access
methods for data.
om
Data-flow analysis engine gathers the Information that is, the values transmitted from
one part of a program to each of the other parts.
Bootstrap compiler is used to compile the compiler and then you can use this compiled
compiler to compile everything else as well as future versions of itself.
ra
Source Language
Target Language
Implementation Language
Notation : represents a compiler for Source S , Target T , implemented in I . The
T-diagram shown above is also used to depict the same compiler.
om
.c
es
The process illustrated by the T-diagrams is called bootstrapping and can be
summarized by the equation:
n ot
la
ra
ke
LEXICAL
Source Program ANALYSER Sequence of Tokens
om
When the lexical analyzer discovers a lexeme constituting an identifier, it needs to
enter that lexeme into the symbol table. .c
In some cases, information regarding the kind of identifier may be read from the
symbol table by the lexical analyzer to assist it in determining the proper token it must
es
pass to the parser.
Commonly, the interaction is implemented by having the parser call the lexical
n
analyzer.
la
The call, suggested by the getNextToken command, causes the lexical analyzer to read
ra
characters from its input until it can identify the next lexeme and produce for it the
next token, which it returns to the parser.
ke
token
Symbol Table
2. Correlatingerrormessagesgeneratedbythecompilerwiththesourceprogram.Forinstanc
e, the lexical analyzer may keep track of the number of newline characters seen, so it
can associate a line number with each error message.
3. If the source program uses a macro-pre-processor, the expansion of macros may also
be performed by the lexical analyzer.
Simplicity Of Design
om
The separation of lexical analysis and syntactic analysis often allows us to simplify at least
one of these tasks. The syntax analyzer can be smaller and cleaner by removing the
lowlevel details of lexical analysis
.c
Efficiency
es
buffering techniques for reading input characters can speed up the compiler significantly.
n
Portability
la
The most important example is the token id, where we need to associate with the token
a great deal of information.
Normally, information about an identifier - e.g., its lexeme, its type, and the location
at which it is first found (in case an error message about that identifier must be issued)
- is kept in the symbol table.
Thus, the appropriate attribute value for an identifier is a pointer to the symbol-table
entry for that identifier.
Lexical Errors
A character sequence that can’t be scanned into any valid token is a lexical error.
Suppose a situation arises in which the lexical analyzer is unable to proceed because
none of the patterns for tokens matches any prefix of the remaining input.
We delete successive characters from the remaining input, until the lexical analyzer
can find a well-formed token at the beginning of what input is left.
This recovery technique may confuse the parser, but in an interactive computing
environment it may be quite adequate.
om
4. Transpose two adjacent characters.
Transformations like these may be tried in an attempt to repair the input.
.c
The simplest such strategy is to see whether a prefix of the remaining input can be
transformed into a valid lexeme by a single transformation.
es
To ensure that a right lexeme is found, one or more characters have to be looked up
ra
Techniques for speeding up the process of lexical analyzer such as the use of sentinels
to mark the buffer end have been adopted.
There are three general approaches for the implementation of a lexical analyzer:
The three choices are listed in order of increasing difficulty for the implementer
Buffer Pairs
Because of large amount of time consumption in moving characters, specialized
buffering techniques have been developed to reduce the amount of overhead required
to process an input character.
Fig shows the buffer pairs which are used to hold the input data.
om
Scheme
Consists of two buffers, each consists of N-character size which are reloaded
alternatively.
.c
N-Number of characters on one disk block.
es
N characters are read from the input file to the buffer using one system read command.
Pointers
n
lexeme Begin points to the beginning of the current lexeme which is yet to be found.
ra
Once a lexeme is found, lexemebegin is set to the character immediately after the
lexeme which is just found and forward is set to the character at its right end.
This limited lookahead may make it impossible to recognize tokens in situations where
the distance that the forward pointer must travel is more than the length of the buffer.
It cannot determine whether the DECLARE is a keyword or an array name until the
character that follows the right parenthesis.
Sentinels
In the previous scheme, each time when the forward pointer is moved, a check is done
to ensure that one half of the buffer has not moved off. If it is done, then the other half
must be reloaded.
om
Therefore the ends of the buffer halves require two tests for each advance of the
forward pointer
The sentinel is a special character that cannot be part of the source program. (eof
character is used as sent
n
la
ra
ke
Advantages
Most of the time, It performs only one test to see whether forward pointer points to an
eof.
Only when it reaches the end of the buffer half or eof, it performs more tests.
Since N input characters are encountered between eofs, the average number of tests
per input character is very close to 1.
forward : = forward + 1;
if forward ↑ = eof then begin
if forward at end of first half then begin
reload second half;
forward := forward + 1
end
else if forward at end of second half then begin
reload first half;
move forward to beginning of first half
end
else /* eof within a buffer signifying end of input */
terminate lexical analysis
end
om
There are 3 specifications of tokens:
Strings
.c
Language
es
Regular expression
Strings and Languages
ot
A string over an alphabet is a finite sequence of symbols drawn from that alphabet.
la
In language theory, the terms "sentence" and "word" are often used as synonyms for
"string." The length of a string s, usually written |s|, is the number of occurrences of
ke
symbols in s. For example, banana is a string of length six. The empty string, denoted
ε, is the string of length zero.
Operations On Strings
The following string-related terms are commonly used:
Operations On Languages:
The following are the operations that can be applied to languages:
Union
Concatenation
Kleene closure
Positive closure
om
.c
Regular Expressions
es
Define a Pascal identifier- which says the identifier is formed by a letter followed by
n
defining rules.
(Associated with each rule is a specification of the language denoted by the regular
expression being defined)
1. ε is a regular expression that denotes {ε}, i.e. the set containing the empty string.
2. If a is a symbol in Σ, then a is a regular expression that denotes {a}, i.e. the set containing
the stringa.
3. Suppose r and s are regular expressions denoting the languages L(r) and L(s). Then
om
.c
es
Regular Definition
la
d1 → r1
ke
d2 → r2
dn → rn
– Where each di is a distinct name, and each ri is a regular expression over the symbols
in Σ U {d1, d2, … , di-1}, i.e., the basic symbols and the previously defined names.
Example: Identifiers is the set of strings of letters and digits beginning with a letter.
Regular definition for this set:
letter → A|B|…|Z|a|b|…|z
digit → 0|1|…|9
Notational Shorthand
Certain constructs occur so frequently in regular expressions that it is convenient to
introduce notational short hands for them.
Thus the regular expression a+ denotes the set of all strings of one or more a’s.
The operator + has the same precedence and associativity as the operator *.
om
The notation r? is a shorthand for r | ε.
If ‘r’ is a regular expression, then ( r )? is a regular expression that denotes the language
.c
3. Character Classes
es
The notation [abc] where a, b and c are alphabet symbols denotes the regular
expression a | b | c.
ot
We can describe identifiers as being strings generated by the regular expression, [A–
la
Za–z][A– Za–z0–9]*
ra
Non-regular Set
ke
Finite Automata
EXAMPLE
if if
then then
else else
om
rebop <|<=|< >|> |> =
id letter ( letter|digit )*
num digits optional-fraction optional-exponent
.c
where letter and digits are defined previously
es
For this language, the lexical analyzer will recognize the keywords i f , then,and else,
as well as lexemes that match the patterns for relop, id, and number.
ot
To simplify matters, we make the common assumption that keywords are also reserved
n
The num represents the unsigned integer and real numbers of Pascal.
ra
Our lexical analyzer will strip out white space. It will do so by comparing a string
against the regular definition ws, below.
Delim blank|tab|newline
ws delim
If a match for ws is found, the lexical analyzer does not return a token to the parser.
Transition Diagram
As an intermediate step in the construction of a lexical analyzer, we first produce a
flowchart, called a r diagram. Transition diagrams.
Transition diagram depict the actions that take place when a lexical analyzer is called
by the parser to get the next token.
The TD uses to keep track of information about characters that are seen as the forward
pointer scans the input.
It does that by moving from position in the diagram as characters are read.
1. One state is labelled the Start State start It is the initial state of transition
diagram where control resides when we begin to recognize a token.
2. Position is a transition diagram are drawn as circles and are called states.
3. The states are connected by Arrows called edges. Labels on edges are indicating
the input characters.
5. Retract one character use * to indicate states on which this input retraction.
om
.c
es
n ot
la
ra
ke
om
.c
es
n ot
la
ra
ke
**********