NLP: Syntactic and Semantic Analysis
NLP: Syntactic and Semantic Analysis
• Introduction
• Syntactic processing
• Semantic processing
• Discourse processing
• Pragmatic processing
• Statistical NLP
• Spell checking
2 3
Introduction Introduction
4 5
Introduction Property of Language
Problem: Sentences are incomplete descriptions of ideas they want to
convey.
Advantage: When we know a lot, the facts imply each other. Language in AI • What is language understanding?
is intended to be used by agents who know a lot.
• What does a sentence mean?
10 11
12 13
Phases of NLP Phases of NLP
14 15
16 17
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
18 19
• Grammar:
Context Free Grammar (CFG)
Dependency Grammar
22 23
• In DG, the linguistic units, i.e., words are connected to each other by directed links.
• The verb becomes the center of the clause structure. Every other syntactic units are
connected to the verb in terms of directed link. These syntactic units are called
dependencies.
24 25
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
Phrase Structure or Constituency Grammar (CG)
• All the related frameworks view the sentence structure in terms of constituency
relation.
• The basic clause structure is understood in terms of noun phrase NP and verb
phrase VP.
26 27
30 31
32 33
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
The main roles of the parse include −
34 35
• Top-down Parsing
• Bottom-up Parsing
36 37
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
38 39
40 41
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
• Thus parsing converts flat list of words that form sentences into a • Example of syntactically correct but semantically wrong sentences.
hierarchical structure that defines the units that are represented by that flat
list. The lion eats the rabbit.
• These hierarchical structural units will correspond to meaning units when The rabbit eats the lion.
semantic analysis is performed.
44 45
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
Concept of Derivation Sentence: a and b or c
In order to get the input string, we need a sequence of production rules. Derivation
Derivation is a set of production rules. During parsing, we need to decide the
non-terminal, which is to be replaced along with deciding the production rule with N
the help of which the non-terminal will be replaced. ⇒ N ‘or’ N
⇒ N ‘or c’
Types of Derivation ⇒ N ‘and’ N ‘or c’
Left-most Derivation ⇒ N ‘and b or c’
In the left-most derivation, the sentential form of an input is scanned and replaced ⇒ ‘a and b or c’
from the left to the right. The sentential form in this case is called the left-sentential
form. Sentential Form: Rightmost derivation
Right-most Derivation
In the left-most derivation, the sentential form of an input is scanned and replaced • In this derivation each line is called a sentential form.
from right to left. The sentential form in this case is called the right-sentential form. • Furthermore, each line of the derivation applies a rule from the CFG in order to
show that the input can, in fact, be derived from the start symbol N.
• In the above derivation, we restricted ourselves to only expand on the
rightmost non-terminal in each sentential form. This method is called the
46 derivation of the input using a CFG.
rightmost 47
‘a and b or c’
⇒ N ‘and b or c’ # use rule N -> a
⇒ N ‘and’ N ’or c’ # use rule N -> b
⇒ N ‘or c’ # use rule N -> N and N
⇒ N ‘or’ N # use rule N -> c
⇒N # use rule N -> N or N
48 49
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
50 51
Definition A parsing strategy that first A parsing strategy that first looks at My dog ate a frog. Parse Tree Derivation
looks at the highest level of the lowest level of the parse tree and
the parse tree and works works up the parse tree by using the
down the parse tree by using rules of a formal grammar.
the rules of a formal
grammar.
Functionality The parsing occurs from the The parsing occurs from the starting
starting symbol to the input symbol to the input string
string.
Main decision To select what production To select when to use a
rule to use in order to production rule to reduce the
construct the string string to get the starting symbol
Method of Uses leftmost derivation Uses rightmost derivation
construction
52 53
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
Transition Networks Transition Networks
54 55
The boy with the raft near the island in the ocean drowned.
56 57
The boy ....
Phase 2: Syntactic Phase 2: Syntactic
Analysis Analysis
Backtracking Backtracking
• RTNs describe a process • You need to backtrack to the last decision point. In order to backtrack, your
parser must record at every decision point:
• The RTN parser "knows" at each step of the parsing process exactly which choices
are available, or what knowledge will be relevant. On the other hand, a CFG-based • What word in the sentence it was looking at when it made the decision
parser will have to search through the list of rules each step of the way to find the
applicable ones. • What node (or state) in the network it was at
• If there's more than one choice, how does the parser know which one is the • What other arcs (or paths) could have been chosen but weren't
right one? It doesn't, so we have to introduce the notion of backtracking.
• When the parser bumps into a dead end like that described above, it must reset the
The little orange ducks swallow flies. parsing process so that it's looking at the previously stored word, it's restarting from
the previously stored node, and it's traversing the first of the previously stored arcs.
• (If there are more than one arc remaining, just push the information again, without the
first arc, and proceed as usual.)
58 59
• Recursive transition networks are interesting from a theoretical standpoint but are not • To record the structure of the input, we add an action to each arc which stores
of much use by themselves. the word that was processed while traversing that arc in an appropriate register.
• From a computational perspective, a black box that accepts English input and just
says "yes“ or "no" doesn't buy us much.
• What we need is a black box that records the structure of the input as well as
providing an evaluation of syntactic correctness. We do this by adding procedures
to the arcs of the RTN.
• These procedures are then performed when the corresponding arcs are traversed.
The resulting network is called an "augmented transition network" (ATN). (And
sometimes, adding procedures to the arcs of a transition network is called "procedural
attachment.")
62 63
Subject verb agreement: To perform agreement checks in the ATN framework, Subject verb agreement:
we attach "feature tests" to the arcs.
• Now, if we try to parse a sentence like "the dog eat the frog" our ATN parser will
catch the agreement error when it performs the intersection on the number of "dog"
and the number of "eat“.
• If the intersection between the set of all possible values for the number of the subject
and the set of all possible values for the number of the verb is the empty set, then
there is no agreement between the subject and the verb. The sentence must be
syntactically incorrect.
64 65
Phase 2: Syntactic Phase 3: Semantic
Analysis Analysis
Augmented Transition Network (ATN) for feature tests:
• Feature tests can also be used to check for agreement between auxiliary verbs and
the main verb in a verb group.
• They can also be used to check for the correct verb complement structure.
• The feature tests can be used to see if the right number of noun phrases follow the
verb.
• Remember that intransitive verbs expect no noun phrases, transitive verbs expect
one noun phrase, and bitransitive verbs expect two noun phrases.
• It must map individual words into appropriate objects in the knowledge base
• Also note that any given verb might fall into more than one of these categories. or database
• Information about the complement structure is called the "subcategorization" of the • It must create the correct structures to respond to the way the meanings or the
verb. individual words combine with each other.
66 67
• Many words have multiple meanings, may not be possible to choose the correct
one just by looking at the word itself.
• Lexical ambiguity:
• Mean:
❖ To signify
❖ Unpleasant or cheap
❖ Statistical average
• Conceptual parsing: syntactic + semantic knowledge combined into single • Semantic grammars are suitable for use in systems with restricted grammars since
information system, guided by semantic knowledge. its power of computation is limited.
• Approximately compositional semantic interpretation: semantic processing applied • Semantic grammars are grammars whose non-terminals correspond to semantic
to results of syntactic parsing. 70 concepts (e.g., [greeting] or [suggest.time] ) rather than to syntactic
constituen7ts1 (such as Verb or WounPhrase).
• When the parse is complete, the result can be used immediately without the
additional stage of processing that would be required if a semantic interpretation
had not already been performed during the parse.
• Many ambiguities that would arise during a strictly syntactic parse can be avoided
since some of the interpretations do not make sense semantically and thus cannot
be generated by a semantic grammar.
74 75
• To implement a Case Grammar the notation used to express rules must allow • ATNs parsers can be used to translate English sentences into a semantic net
them to produce case frames (ie: structures without a pre-determined form) representing the case structures of the sentences. These semantic nets can
then be used to answer questions about the sentences.
• Grammar notations whose only output reflects the syntactic structure of their input
are not suitable since they cannot be used to construct case frames. • The result of parsing in a case representation I usually not a complete semantic
description of a sentence.
• These rules can be applied in reverse by a parser to determine the
underlying case structure from the superficial syntax. • The constituents that fill the slots may still be English words rather than true
semantic descriptions stated in the target representation.
• Parsing using a case grammar is usually expectation-driven. Once the verb of the
sentence has been located, it can be sed to predict the noun phrases that will
occur and to determine the relationship of those phrases to the rest of the 76 77
sentence.
Phase 3: Semantic Phase 3: Semantic
Analysis Analysis
Conceptual Parsing Conceptual Parsing
• The grammar of the system is bipartite.
• Finds both structure and meaning of a sentence in one step.
• The first part is a universal grammar exemplified by the conceptual rules employed
• Driven by a dictionary hat describes the meanings of words as Conceptual by the system.
Dependency (CD) structures.
• The second part is language-specific and is made up of r e realization rules
• Similar to case grammar. intended to map pieces of the conceptual network into linguistic items.
• Parsing process is heavily driven by a set of expectations that are set up on the • The realization rules may be used for both parsing and generating. However, it is
basis of the sentence’s main verb. not necessary to use all the realization rules in order to parse.
• However, here, the representation of a verb in CD is at a lower level than that of • The system is capable of making sense of a piece of language containing only a
a verb in a case grammar where the representation is often identical to the few words that it knows since its conceptual framework is capable of making
English word that is used as verb. predictions.
• CD usually provides a greater degree of predictive power. • Thus, it can understand while using only a few realization rules, whereas it would
need a great many more to map the same structure back into language. This
• The first step in mapping a sentence into its CD representation involves a phenomenon is similar to that observed in a man attempting to learn a foreign79
syn78tactic processor that extracts the main noun and verb. language.
80 81
Phase 3: Semantic Phase 3: Semantic
Analysis Analysis
Conceptual Parsing Compositional Semantic Interpretation
The tall boy went to the park with a girl • Here, syntactic parsing and semantic interpretation are treated as two separate
steps.
82 83
• The most important features of the theory are its use of model theoretic semantics (1) Mary is singing and dancing.
which is nowadays commonly used for the semantics of logical languages and its
adherence to the principle of compositionality—that is, the meaning of the whole is (2) Mary is singing.
a function of the meanings of its parts and their mode of syntactic combination.
Entailment with inverse relationship, 4 entails 3
• Important relationships are:
(3) No-one is singing and dancing.
• Entailment
• Valid reasoning (4) No-one is singing.
• Synonymy
• Ambiguity Entailment with transitive and intransitive verbs: 5 entails 6
John made a huge wedding cake with chocolate icing • Parts of entities
John made a huge wedding cake with Bill’s mixer.
John made a huge wedding cake with a giant tower covered with roses. Ram opened the dress that he just bought.
John made a cherry pie with a giant tower covered with roses. The collar had a damage.
• Knowledge enables the system to accept meanings that it has not been explicitly • Parts of actions
told about.
Jim went on a business trip to Delhi
John made a huge wedding cake with chocolate icing He left by the early morning flight.
John made a huge wedding cake with Bill’s mixer.
John made a huge wedding cake with a giant tower covered with roses. • Entities involved in actions
John made a cherry pie with a giant tower covered with roses.
The bank was broken into last week. 89
They took the cash and jewels.
88
Phase 4: Discourse and pragmatic processing Phase 4: Discourse and pragmatic processing
The stickers we have in store are the lion, tiger, giraffe and It sure is going to rain today/
octopus.
I will take the octopus. • Implicit presuppositions
92