The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes.

Distribution and modifications of the content is prohibited.

Natural Language Processing

CSDC 7013

Subject In-charge
Ms. Pradnya Sawant
Assistant Professor
Room No. 405
email: [email protected]

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 1
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Module 5
Pragmatic and Discourse Processing

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 2
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Contents
• Discourse: Reference Resolution, Reference
Phenomena, Syntactic & Semantic constraint on
coherence
• Anaphora Resolution using Hobbs and Centering
Algorithm

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 3
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Module 5
Lecture 1
▪ Discourse Integration
▪ Coreference Resolution

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 4
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Example

● Gracie: Oh yeah. . . and then Mr. and Mrs. Jones were having
matrimonial trouble, and my brother was hired to watch Mrs.
Jones.
● George: Well, I imagine she was a very attractive woman.
● Gracie: She was, and my brother watched her day and night
for six months.
● George: Well, what happened?
● Gracie: She finally got a divorce.
● George: Mrs. Jones?
● Gracie: No, my brother’s wife.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 5
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Example
● This is a dialogue part of a movie.
● Language consist of collocated, structured,
coherent groups of sentences.
● Such a coherent structured group of sentences is
called a discourse.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 6
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Dialogue
● The dialogue discourse consist of different types of
communicative acts as:
○ asking questions
○ giving answers
○ making corrections, and so forth.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 7
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Example
Consider the discourse below:
The Tin Woodman went to the Emerald City to see the
Wizard of Oz and ask for a heart. After he asked for it,
the Woodman waited for the Wizard’s response.
● What do pronouns such as he and it denote?
● But doing this disambiguation automatically is a
difficult task.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 8
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Coreference Resolution
● The goal of deciding what pronouns and other
noun phrases refer to is called coreference
resolution.
● Coreference resolution is important for
○ Information Extraction
○ Summarization
○ Conversational Agents
● There are other important discourse structures
beside coreference resolution.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 9
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Coherence Relations

● Consider the task of summarization below:

● Text:
First Union Corp is continuing to wrestle with
○
severe problems. According to industry insiders
at Paine Webber, their president, John R.
George, is planning to announce his retirement
tomorrow.
● Summary:
○ First Union President John R. George is
planning to announce his retirement tomorrow.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 10
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Coherence Relations

● To build such a summary, we need to know that

the second sentence is the more important of the
two, and that the first sentence is just giving
background information.
● Relationships of this sort between sentences in
a discourse are called coherence relations.
● Determining the coherence structures between
discourse sentences is an important discourse
task.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 11
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Coherence Relations

● In automatic essay grading, short student essays

are assigned a grade by measuring the internal
coherence of the essay as well as comparing its
content to source material which is a high-quality
essay.
● It is also used to evaluate the output quality of
natural language generation systems.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 12
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Reference Resolution

Consider the following example:

John went to Bill’s car dealership to check out an
Acura Integra. He looked at it for about an hour.
● A natural language expression used to perform
reference is called a referring expression (He,
it) and the entity that is referred to is called the
referent.(John, Car)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 13
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Reference Resolution

● In the example,
Referring expression: He , it
○
○ Referent : John, Acura Integra
● Co-referring Expression : Two referring
expressions that are used to refer to the same entity.
(Not in this example)
● Antecedent : John is the antecedent of he.
● Anaphora : Reference of an entity that has been
previously introduced into the discourse.(here: he)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 14
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Exercise

Sam left Peter to be mayor of a city with a big

budget problem. It is unclear how he will be
able to handle it during his term.
● Referring expressions: ?
● Referents: ?
● Co-referring expressions: ?
● Antecedent: ?
● Anaphora: ?

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 15
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Exercise

● Referring expressions: he, it, his.

● Referents: Peter, big budget problem
● Co-referring expressions: he, his (Both
referring to Peter)
● Antecedent: Peter, Big budget problem
● Anaphora: he, his , it

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 16
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Module 5
Lecture 2
▪ Reference phenomenon

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 17
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Discourse Model
● The discourse model contains representations of the
entities that have been referred to in the discourse and
the relationships in which they participate.
● There are 2 fundamental operations to discourse
model.
● When a referent is first mentioned in a discourse, we
say that a representation for it is evoked into the
model.
● Upon subsequent mention, this representation is
accessed from the model.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 18
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Discourse Model

Example:
John went to Bill’s car
dealership to check out
an Acura Integra. He
looked at it for about an
hour.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 19
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Consider the example given below:

● But that turned out to be a lie.
● But that was false.
● That struck me as a funny way to describe the
situation.
● That caused Sam to become rather poor.
● That caused them both to become rather poor.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 20
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Example

● But that turned out to be a lie: a speech act

● But that was false: a proposition
● That struck me as a funny way to describe the
situation: a manner of description
● That caused Sam to become rather poor : an event
● That caused them both to become rather poor : a
combination of several events.

Thus the field awaits the development of robust

methods for interpreting these types of reference.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 21
Chatbot with Discourse
Integration
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 23
Chatbot without Discourse
Integration
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 25
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Reference Phenomena

● There are several referential phenomena in natural

languages.
● Five types of referring expressions are:
○ Indefinite noun phrases, definite noun phrases,
pronouns, demonstratives and one-anaphora.
● Three types of referring expressions that
complicate the reference resolution problem:
Inferrables, discontinuous sets, and generics.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 26
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Indefinite Noun Phrases

● Indefinite reference introduces entities that are new
to the hearer into the discourse context.
● Common form of indefinite reference is marked with
the determiner a (or an),by a quantifier (some) or
even determiner this.
○ I saw a Mercedes Benz today.
○ Some Mercedes Benz were being unloaded at the
local dealership today.
○ I saw this awesome Mercedes Benz today.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 27
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Indefinite Noun Phrases

Such noun phrases evoke a representation for a
new entity.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 28
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Definite Noun Phrases

● Definite reference is used to refer to an entity that is

○ identifiable to the hearer
○ it is contained in the hearer’s set of beliefs about the world
○ the uniqueness of the object is implied by the description
itself.
● E.g.
○ I saw an Acura Integra today. The Integra was white and
needed to be washed.
○ The Indianapolis 500 is the most popular one in US.
○ The fastest car in the Indianapolis 500 was an Integra.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 29
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Definite Noun Phrases

A Definite Noun Phrase also evokes a representation of the

referent into the discourse model.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 30
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Pronouns
● Pronominalization is another form of definite
reference
● E.g. I saw an Acura Integra today. It was white and
needed to be washed.
● The constraints on using pronouns is that it should
refer to entities that were introduced no further than
one or two sentences back in the ongoing discourse,
whereas definite noun phrases can often refer further
back.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 31
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Example
a. John went to Bob’s party, and parked next to a beautiful Acura
Integra.
b. He went inside and talked to Bob for more than an hour.
c. Bob told him that he recently got engaged.
d. He also said that he bought it yesterday.
d’. He also said that he bought the Acura yesterday.

● By the time the last sentence is reached, the Integra no longer

has the degree of salience required to allow for pronominal
reference to it.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 32
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Pronouns as Cataphora
● Pronouns are mentioned before their referents are
called Cataphoras
● E.g. Before he bought it, John checked over the
Integra very carefully.
● Here, the pronouns he and it both occur before their
referents are introduced.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 33
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Demonstratives
● Demonstrative pronouns, like this and that, can appear
either alone or as determiners (this Acura, that Acura).
● The choice between two demonstratives is based on spatial
proximity:
● this indicating closeness and
● that signaling distance.
● Spatial distance might be measured with respect to the
discourse participants situational context.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 34
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Demonstratives

E.g. I bought an Integra yesterday. It’s similar to

the one I bought five years ago. That one was really
nice, but I like this one even better.
Here, that one refers to the Acura bought five years
ago (greater temporal distance), whereas this one
refers to the one bought yesterday (closer temporal
distance).

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 35
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

One Anaphora
● One-anaphora, blends properties of definite and
indefinite reference.
● E.g.
○ She has a white car. I also want one
○ She has the best BMW. I also want one like that.
● This usage is paraphrased by one of them, in which
‘them’ refers to a plural referent & one is a member
from this set.
● Thus, one may evoke a new entity which is
dependent on an existing referent.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 36
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

One Anaphora

● This use of one should be distinguished from the

formal, non-specific pronoun usage and its meaning
as the number one as shown below.
● E.g.
○ One shouldn’t pay more than twenty thousand
dollars for an Acura.
○ John has two Acuras, but I only have one.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 37
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Module 5
Lecture 3
▪ Reference phenomenon
▪ Syntactic & semantic constraints on co reference

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 38
 Referents complicate the
reference resolution problem
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Inferrables
Consider cases in which a referring expression does not refer to
an entity that has been explicitly evoked in the text, but instead
one that is inferentially related to an evoked entity. Such referents
are called inferrables.
● E.g. I almost bought an Acura Integra today, but a door had a
dent and the engine seems noisy.
● The indefinite noun phrase a door would evoke a new door
into the discourse context, but in this case infer : that it is not
just any door, but one of the doors of the Integra.
● Similarly, no engine has been explicitly mentioned, but the
hearer infers that the referent is the engine of Integra.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 40
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Discontinuous Sets

● In some cases, references using plural referring expressions

like they and them refer to sets of entities that are evoked
together.
● E.g. John and Mary love their Acuras. They drive them all the
time.
● However, plural references may also refer to sets of entities
that have been evoked by discontinuous phrases in the text:
● E.g. John has an Acura, and Mary has a Mazda. They drive
them all the time.
● Here, they refers to John and Mary, and likewise them refers
to the Acura and the Mazda.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 42
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Generics
• Another kind of expression that does not refer
back to an entity explicitly evoked in the text is
generic reference.
Consider the example given below:
• E.g. I’m interested in buying a Mac laptop. They
are very stylish.
• Here, they refers, not to a particular laptop (or
even a particular set of laptops), but instead to the
class of Mac laptops.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 43
 Syntactic and Semantic
Constraints on Coreference
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Syntactic and Semantic Constraints

● How one develop algorithms for

identifying the referents of referential
expressions.
● One step is to filter the set of possible
referents on the basis of certain relatively
hard-and-fast constraints.
● Let us see some of these constraints

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 45
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Number Agreement
● Referring expressions and their referents
must agree in number
● This means distinguishing between
singular and plural references.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 46
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Number Agreement

● The following examples illustrate constraints on

number agreement.
● E.g.

John has a new Acura. It is red.

John has three new Acuras. They are red.
* John has a new Acura. They are red.
* John has three new Acuras. It is red.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 47
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Person and Case Agreement

English distinguishes between three forms

of person: first, second, and third.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 48
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Person and Case Agreement

E.g.
You and I have Acuras. We love them.
John and Mary have Acuras. They love them.
* John and Mary have Acuras. We love them. (where
We=John and Mary)
* You and I have Acuras. They love them. (where
They=You and I)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 49
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Person and Case Agreement

● Different forms of the pronoun may be required when
placed in
○ subject position ( e.g., he, she, they)
○ object position ( e.g., him, her, them)
○ genitive position ( e.g., his Acura, her Acura, their
Acura).

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 50
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Gender Agreement

● Referents also must agree with the gender

specified by the referring expression.
● English third person pronouns distinguish
between male, female, and nonpersonal genders

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 51
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Gender Agreement

● E.g.
John has an Acura. He is attractive.
(he=John, not the Acura)
John has an Acura. It is attractive. (it=the
Acura, not John)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 52
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Syntactic Constraints

References are constrained by the syntactic

relationships between a referential
expression and a possible antecedent noun
phrase when both occur in the same
sentence.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 53
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Syntactic Constraints

● E.g

John bought himself a new Acura. [himself = John]

John bought him a new Acura. [him ≠John]
John said that Bill bought him a new Acura. [him ≠ Bill]
John said that Bill bought himself a new Acura. [himself
=Bill]
He said that he bought John a new Acura. [He ≠ John] [he
≠ John]

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 54
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Reflexive Constraints

● English pronouns such as himself, herself, and

themselves are called reflexives.
● A reflexive corefers with the subject of the
most immediate clause that contains it
● E.g. John bought himself a new Acura.
[himself = John]
● A nonreflexive cannot corefer with this subject
● E.g. John bought him a new Acura. [him
≠John].

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 55
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Reflexive Constraints
● A nonreflexive pronoun like him can corefer with the
subject of the previous sentence
● E.g. John wanted a new car. Bill bought him a new
Acura. [him=John]
● but it cannot in
● E.g. John wanted a new car. He bought him a new
Acura.[He=John, him ≠John]
● because of its syntactic relationship

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 56
Reflexive Constraints
● There are many cases that they do not cover.
● E.g. John set the pamphlets about Acuras next to
himself. [himself=John]) and
● E.g. John set the pamphlets about Acuras next to
him. [him=John])
● Both can refer to the subject John, even though they
occur in identical syntactic configurations.
Selectional Restrictions
● The selectional restrictions that a verb places on
its arguments may be responsible for eliminating
referents.
● E.g. John parked his Acura in the garage. He had
driven it around for hours.
● A practical NLP system would include a
reasonably comprehensive set of selectional
constraints for the verbs in its lexicon.
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Selectional Restrictions

● Selectional restrictions can be violated in the

case of metaphor.
● E.g. John bought a new Acura. It drinks
gasoline like you would not believe
● E.g. John parked his Acura in the garage. It is
incredibly messy, with old bike and car parts
lying around everywhere.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 59
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Selectional Restrictions

● Thus, any knowledge shared by the discourse

participants might be necessary to resolve a
pronoun reference.
● However, due in part to the vastness of such
knowledge, practical algorithms typically do
not rely on it heavily.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 60
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Preferences in Pronoun Interpretation

Here we look into different

methodologies used for pronoun
resolution

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 61
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

1. Recency

● Most theories assumes that entities

introduced in recent utterances are more
salient than those utterances further back.
● E.g.
● John has an Integra. Bill has a Legend. Mary
likes to drive it.
(Here it is presumed that Mary likes to drive
Legend)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 62
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

2. Grammatical Role

● Some theories specify a salience hierarchy

of entities ordered by the grammatical
position of the expressions.
● It treat entities mentioned in subject
position as more salient than those in
object position, which are in turn more
salient than subsequent positions.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 63
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

2. Grammatical Role

● E.g.
● John went to the Acura dealership with Bill. He
bought an Integra. [ he = John ]
● Bill went to the Acura dealership with John. He
bought an Integra. [ he = Bill ]
● John and Bill went to the Acura dealership. He
bought an Integra. [ he = ?? ].

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 64
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Repeated Mention

● Some theories incorporate the idea that entities that

have been focused on in the prior discourse are more
likely to continue to be focused on in subsequent
discourse
● E.g.
● John needed a car to get to his new job. He decided
that he wanted something sporty. Bill went to the
Acura dealership with him. He bought an Integra. [ he
= John]

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 65
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Parallelism

● There are also strong preferences that appear

to be induced by parallelism effects.
● E.g.
● Mary went with Sita to the Acura dealership.
Sally went with her to the Mercedes
dealership. [ her = Sita ]

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 66
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Heuristics

● This suggests that we might want a heuristic

which says that non-subject pronouns prefer non-
subject referents.
● Consider the following example in which Mary
is the preferred referent of the pronoun instead of
Sita.
● E.g.
● Mary went with Sita to the Acura dealership. Sally
told her not to buy anything. [ her = Mary ]

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 67
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Verb Semantics

● Certain verbs appear to place a semantically-oriented

emphasis on one of their argument positions based on
the manner in which subsequent pronouns are
interpreted.
● Based on Subject:
● E.g. John telephoned Bill. He lost the pamphlet on
Acura.[He = John]
John criticized Bill. He lost the pamphlet on Acura.
[He = Bill]

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 68
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Verb Semantics

● Based on Goal Thematic Role:

● E.g.
● John seized the Acura pamphlet from Bill. He
loves reading about cars. (Goal=John,
Source=Bill)
● John passed the Acura pamphlet to Bill. He loves
reading about cars. (Goal=Bill, Source=John)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 69
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Verb Semantics

● Based on Stimulus Experiencer Role:

● E.g.
● The car dealer admired John. He knows
Acuras inside and out. (Stimulus=John,
Experiencer=the car dealer)
● The car dealer impressed John. He knows
Acuras inside and out. (Stimulus=the car
dealer, Experiencer=John)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 70
Anaphora resolution:
There are following algorithms for anaphora
resolution:

• Hobbs 1978

• Lappin and Leass 1994

• Centering Theory
Lappin and Leass Algorithm
for Pronoun Resolution
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Subject
● The pronoun, noun or noun phrase that does the
action of the verb
● Subject is usually before the verb
● E.g.
○ John Arrived.
○ I love chocolate
● It can also be a group of words
● E.g.
○ An orange cat was sitting beside me

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 73
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Subject
● Some verbs are not actions but linking verbs that
links to more information about the subject
● Linking Verbs : be, seem, become
● E.g.
○ I am hungry
○ John seemed tired
● Subjects can be long
● E.g.
○ The man who lives upstairs lied

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 74
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Subject

● Subjects can include 2 or more nouns

● E.g.
○ Lucy’s grey cat and John’s white cat are
friends

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 75
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Object
● It is a thing or person that the verb is done to or who
receives verb
● It can be a noun, noun phrase, pronoun or a longer
complex object
● Only transitive verb can have object
● E.g.
○ I love chocolate
○ David met Lucy yesterday
○ I put the orange cat into the garden

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 76
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Direct Object
● It is the thing or person to whom or to which we do
the action verb
● E.g.
○ I give the chocolate
○ David met Mary yesterday

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 77
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Indirect Object
● It is the person who receives the direct object
● E.g.
○ I made my friend (IO) some coffee (DO)
○ I threw John (IO) the ball (DO)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 78
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Lappin and Leass Algorithm

● It employs a simple weighting scheme that
integrates the effects of the recency and
syntactically-based preferences
● It employs no semantic preferences
● There are two types of operations performed by the
algorithm:
○ discourse model update
○ pronoun resolution.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 79
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Lappin and Leass Algorithm

● When a noun phrase that evokes a new entity is
encountered, a representation for it must be added to
the discourse model and a degree of salience
computed for it.
● The salience value is calculated as the sum of the
weights assigned by a set of salience factors.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 80
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Lappin and Leass Algorithm

The salience factors used and their corresponding
weights are

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 81
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Subject Emphasis
That means the focus is usually on the subject and what
the subject does.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 82
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Existential Emphasis
● It asserts the existence or non-existence of
something
● English relies on constructions introduced by There
● The verb most often used in existential sentences is
a form of be. Other verbs (e.g., exist, occur) may
follow the existential there.
● E.g.
○ There is always some madness in love
○ There are many exit visas sold in this café

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 83
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Accusative (Direct Object) Emphasis

● It shows the relationship of a direct object to a verb.
● E.g.
○ We cooked our dinner.
○ Verb: cooked
○ Direct object: dinner

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 84
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Indirect Object Emphasis

● The emphasis is on the Indirect Object
● E.g.
○ I made my friend (IO) who came from US some
coffee (DO)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 85
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Oblique Complement Emphasis

● It is prepositional phrase which behaves like a
complement
● E.g.
○ Pat will wait for Kim

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 86
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Adverbial Emphasis
● It is a term used for an intensifier used to give added
force or a greater degree of certainty to another
word in a sentence or a sentence as whole
● Adverbs of emphasis are called emphasizers
● E.g. absolutely, certainly, clearly, definitely
● E.g.
○ He is tapping my phone and I definitely heard it

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 87
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Head Noun Emphasis

● It is the head of the noun that determines what sort
of thing or person the whole noun phrase refers to
● E.g.
○ All the cakes on the table tastes nice
○ Cakes taste nice

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 88
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Lappin and Leass Algorithm

The weights that each factor assigns to an entity in the discourse
model are cut in half each time a new sentence is processed.
● E.g.
○ An Acura Integra is parked in the lot. (subject)
○ There is an Acura Integra parked in the lot. (existential
predicate nominal)
○ John parked an Acura Integra in the lot. (object)
○ John gave his Acura Integra a bath. (indirect object)

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 89
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Lappin and Leass Algorithm

● The weight for any referent is always positive.
● The head noun emphasis factor penalizes referents
which are embedded in larger noun phrases.
● Once we have updated the discourse model with
new potential referents and recalculated the salience
values associated with them, we are ready to
consider the process of resolving any pronouns that
exist within a new sentence.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 90
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Lappin and Leass Algorithm

Two more salience weights, one for grammatical role parallelism
between the pronoun and the potential referent, and one to
disprefer cataphoric reference are considered here

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 91
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Pronoun Resolution Algorithm

1. Collect the potential referents (up to four sentences
back).
2. Remove potential referents that do not agree in
number or gender with the pronoun.
3. Remove potential referents that do not pass
intrasentential syntactic coreference constraints.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 92
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Pronoun Resolution Algorithm

4. Compute the total salience value of the referent by
adding any applicable values to the existing salience
value previously computed during the discourse model
update step.
5. Select the referent with the highest salience value. In
the case of ties, select the closest referent in terms of
string position (computed without bias to direction).

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 93
 The material in this presentation belongs to St. Francis Institute of Technology and is solely for educational purposes. Distribution and modifications of the content is prohibited.

Example
E.g. John saw a beautiful Acura Integra at the
dealership. He showed it to Bob. He bought it.
First Sentence : John saw a beautiful Acura Integra at
the dealership.
Second Sentence : He showed it to Bob.
Third Sentence : He bought it.

St. Francis Institute of Technology NLP

Department of Computer Engineering Ms. Pradnya Sawant 94
Example

• John saw a beautiful Acura Integra at the dealership.

• He showed it to Bob.
• He bought it.
John
Salience Factor Salience Value
Sentence recency 100
Subject emphasis 80
Existential emphasis
Accusative emphasis
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
Integra
Salience Factor Salience Value
Sentence recency 100
Subject emphasis
Existential emphasis
Accusative emphasis 50
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
dealership
Salience Factor Salience Value
Sentence recency 100
Subject emphasis
Existential emphasis
Accusative emphasis
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
 He
Salience Factor Salience Value
Sentence recency 100
Subject emphasis 80
Existential emphasis
Accusative emphasis
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
 It
Salience Factor Salience Value
Sentence recency 100
Subject emphasis
Existential emphasis
Accusative emphasis 50
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
 Bob
Salience Factor Salience Value
Sentence recency 100
Subject emphasis
Existential emphasis
Accusative emphasis
Indirect object emphasis 40
Non-adverbial emphasis 50
Head noun emphasis 80
 He
Salience Factor Salience Value
Sentence recency 100
Subject emphasis 80
Existential emphasis
Accusative emphasis
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
 It
Salience Factor Salience Value
Sentence recency 100
Subject emphasis
Existential emphasis
Accusative emphasis 50
Indirect object emphasis
Non-adverbial emphasis 50
Head noun emphasis 80
Hobbs Algorithm

• Hobbs (1978) proposes an algorithm that searches parse

trees (i.e., basic syntactic trees) for antecedents of a pronoun.

• starting at the NP node immediately dominating the

pronoun
• in a specified search order
• looking for the first match of the correct gender and
number

• Idea: discourse and other preferences will be approximated

by search order.
Hobbs

• This simple algorithm has become a baseline: more complex

algorithms should do better than this.

• Hobbs distance: ith candidate NP considered by the algorithm

is at a Hobbs distance of i.
Multiple parse trees
Because it assumes parse trees, such an algorithm is inevitably
dependent on one’s theory of grammar.

1. Mr. Smith saw a driver in his truck.

2. Mr. Smith saw a driver of his truck.

“his” may refer to the driver in 1, but not 2.

• different parse trees explain the difference:

• in 1, if the PP is attached to the VP, “his” can refer back to the
driver;
• in 2, the PP is obligatorily attached inside the NP, so “his” cannot
refer back to the driver.
Hobbs’s Algorithm
1. Begin at the NP immediately dominating the pronoun.
2. Go up tree to first NP or S encountered.
• Call node X, and path to it, p.
• Search left-to-right below X and to left of p, proposing any NP node which
has an NP or S between it and X.
3. If X is highest S node in sentence,
• Search previous trees, in order of recency, left-to-right, breadth-first,
proposing NPs encountered.
4. Otherwise, from X, go up to first NP or S node encountered,
• Call this X, and path to it p.
5. If X is an NP, and p does not pass through an N-bar that X immediately
dominates, propose X.
6. Search below X, to left of p, left-to-right, breadth-first, proposing NP
encountered.
7. If X is an S, search below X to right of p, left-to-right, breadth-first, but not going
through any NP or S, proposing NP encountered.
8. Go to 2.
Centering Theory
Basic ideas:

• A discourse has a focus, or center.

• The center typically remains the same for a few sentences, then
shifts to a new object.
• The center of a sentence is typically pronominalized.
• Once a center is established, there is a strong tendency for
subsequent pronouns to continue to refer to it.
Some examples
Compare the two discourses:

• a. John went to his favorite music store to buy a piano.

• b. He had frequented the store for many years.
• c. He was excited that he could finally buy a piano.
• d. He arrived just as the store was closing for the day.

• a. John went to his favorite music store to buy a piano.

• b. It was a store John had frequented for many years.
• c. He was excited that he could finally buy a piano.
• d. It was closing just as John arrived.
Another example
• a. Terry really goofs sometimes.
• b. Yesterday was a beautiful day and he was excited
about trying out his new sailboat.
• c. He wanted Tony to join him on a sailing
expedition.
• d. He called him at 6 AM.
• e. He was sick and furious at being woken up so
early.
Replace pronoun with proper name

• a. Terry really goofs sometimes.

• b. Yesterday was a beautiful day and he was excited
about trying out his new sailboat.
• c. He wanted Tony to join him on a sailing expedition.
• d. He called him at 6 AM.
• e. Tony was sick and furious at being woken up so early.
Replace pronoun with proper
name
• a. Terry really goofs sometimes.
• b. Yesterday was a beautiful day and he was excited
about trying out his new sailboat.
• c. He wanted Tony to join him on a sailing expedition.
• d. He called him at 6 AM.
• e. Tony was sick and furious at being woken up so early.
• f. He told Terry to get lost and hung up.
• g. Of course, he hadn’t intended to upset Tony.
Once again, replace pronoun with proper
name
• a. Terry really goofs sometimes.
• b. Yesterday was a beautiful day and he was excited
about trying out his new sailboat.
• c. He wanted Tony to join him on a sailing expedition.
• d. He called him at 6 AM.
• e. Tony was sick and furious at being woken up so early.
• f. He told Terry to get lost and hung up.
• g. Of course, Terry hadn’t intended to upset Tony.
Another example
• Compare the two discourses

• 1 a. John was very worried last night.

• b. He called Bob.
• c. He told him that there was a big problem.

• 2 a. John was very worried last night.

• b. He called Bob.
• c. He told him never to call again at such a late hour.
Again, replace pronoun with
proper name
• Compare the two discourses

• 1 a. John was very worried last night.

• b. He called Bob.
• c. He told him that there was a big problem.

• 2 a. John was very worried last night.

• b. He called Bob.
• c. Bob told him never to call again at such a late hour.
When are pronouns better than proper
names?
• a. Susan gave Betsy a pet hamster.
• b. She reminded her that such hamsters are quite shy.

Compare the following alternative utterances.

• c1. She asked Betsy whether she liked the gift.

• c2. Susan asked her whether she liked the gift.

• c3. Betsy told her that she really liked the gift.
• c4. She told Susan that she really liked the gift.
Centering
• Centering theory was developed by Barbara J.
Grosz, Aravind K. Joshi and Scott Weinstein in the
1980s to explain this kind of phenomena.
Definitions
Utterance – A sentence in the context of a discourse.

Center – An entity referred to in the discourse (our discourse

referents).
Forward looking centers – An utterance Un is assigned a set of
centers Cf(Un) that are referred to in Un (basically, the drefs
introduced / acccessed in a sentence).

Backward looking center – An utterance Un is assigned a single

center Cb(Un), which is equal to one of the centers in Cf(Un-
1)Cf(Un).
If there is no such center, Cb(Un) is NIL.
Ranking of forward looking
centers
• Cf(Un) is an ordered set.

• Its order reflects the prominence of the centers in the utterance.

• The ordering (ranking) is done primarily according to the

syntactic position of the word in the utterance (subject > object(s)
> other).

• The prominent center of an utterance, Cp(Un), is the highest

ranking center in Cf(Un).
Ranking of forward looking
centers
• Think of the backward looking center Cb(Un)
as the current topic.

• Think of the preferred center Cp(Un) as the

potential new topic.
 Constraints on centering
1. There is precisely one Cb.

2. Every element of Cf(Un) must be realized in Un.

3. Cb(Un) is the highest-ranked element of Cf(Un-1)

that is realized in Un.
Another example
• U1. John drives a Ferrari.

• U2. He drives too fast.

• U3. Mike races him often.

• U4. He sometimes beats him.

Let’s see what the centers are…
• U1. John drives a Ferrari.
Cb(U1) = NIL (or: John). Cf(U1) = (John, Ferrari)

• U2. He drives too fast.

Cb(U2) = John. Cf(U2) = (John)

• U3. Mike races him often.

Cb(U3) = John. Cf(U3) = (Mike, John)

• U4. He sometimes beats him.

Cb(U4) = Mike. Cf(U4) = (Mike, John)
Types of transitions
Transition Type Cb(Un) = Cb(Un-1) Cb(Un) = Cp(Un)
from Un-1 to Un
Center + +
Continuation
Center Retaining + -

Center Shifting-1 - +

Center Shifting - -
Let’s see what the transitions
are…
• U1. John drives a Ferrari.
Cb(U1) = John. Cf(U1) = (John, Ferrari)

• U2. He drives too fast.

Cb(U2) = John. Cf(U2) = (John)
 Types of transitions
Transition Type Cb(Un) = Cb(Un-1) Cb(Un) = Cp(Un)
from Un-1 to Un
Center + +
Continuation
Center Retaining + -

Center Shifting-1 - +

Center Shifting - -
Let’s see what the transitions
are…
• U1. John drives a Ferrari.
Cb(U1) = John. Cf(U1) = (John, Ferrari)

• U2. He drives too fast. (continuation)

Cb(U2) = John. Cf(U2) = (John)
Let’s see what the transitions
are…
• U1. John drives a Ferrari.
Cb(U1) = John. Cf(U1) = (John, Ferrari)

• U2. He drives too fast. (continuation)

Cb(U2) = John. Cf(U2) = (John)

• U3. Mike races him often.

Cb(U3) = John. Cf(U3) = (Mike, John)
 Types of transitions
Transition Type Cb(Un) = Cb(Un-1) Cb(Un) = Cp(Un)
from Un-1 to Un
Center + +
Continuation
Center Retaining + -

Center Shifting-1 - +

Center Shifting - -
Let’s see what the transitions
are…
• U1. John drives a Ferrari.
Cb(U1) = John. Cf(U1) = (John, Ferrari)

• U2. He drives too fast. (continuation)

Cb(U2) = John. Cf(U2) = (John)

• U3. Mike races him often. (retaining)

Cb(U3) = John. Cf(U3) = (Mike, John)
Let’s see what the transitions
are…
• U1. John drives a Ferrari.
Cb(U1) = John. Cf(U1) = (John, Ferrari)

• U2. He drives too fast. (continuation)

Cb(U2) = John. Cf(U2) = (John)

• U3. Mike races him often. (retaining)

Cb(U3) = John. Cf(U3) = (Mike, John)

• U4. He sometimes beats him.

Cb(U4) = Mike. Cf(U4) = (Mike, John)
 Types of transitions
Transition Type Cb(Un) = Cb(Un-1) Cb(Un) = Cp(Un)
from Un-1 to Un
Center + +
Continuation
Center Retaining + -

Center Shifting-1 - +

Center Shifting - -
Let’s see what the transitions
are…
• U1. John drives a Ferrari.
Cb(U1) = John. Cf(U1) = (John, Ferrari)

• U2. He drives too fast. (continuation)

Cb(U2) = John. Cf(U2) = (John)

• U3. Mike races him often. (retaining)

Cb(U3) = John. Cf(U3) = (Mike, John)

• U4. He sometimes beats him. (shifting-1)

Cb(U4) = Mike. Cf(U4) = (Mike, John)
 Centering rules in discourse
1. If some element of Cf(Un-1) is realized as a pronoun
in Un, then so is Cb(Un).

2. Continuation is preferred over retaining, which is

preferred over shifting-1, which is preferred over
shifting:
Cont >> Retain >> Shift-1 >> Shift
Violation of rule 1
• Assuming He in utterance U1 refers to John…

• U1. He has been acting quite odd.

• U2. He called up Mike Yesterday.
• U3. John wanted to meet him urgently.
In more detail …
• U1. He has been acting quite odd.
Cb(U1) = John. Cf(U1) = (John)

• U2. He called up Mike Yesterday.

Cb(U2) = John. Cf(U2) = (John, Mike)

• U3. John wanted to meet him urgently.

Cb(U3) = John. Cf(U3) = (John, Mike)
Violation of rule 2
Compare the two discourses we started with:

• U1. John went to his favorite music store to buy a piano.

• U2. He had frequented the store for many years.
• U3. He was excited that he could finally buy a piano.
• U4. He arrived just as the store was closing for the day.

• U1. John went to his favorite music store to buy a piano.

• U2. It was a store John had frequented for many years.
• U3. He was excited that he could finally buy a piano.
• U4. It was closing just as John arrived.
Transitions for the 1st discourse

• U1. John went to his favorite music store to buy a piano.

Cb(U1) = John. Cf(U1) = (John, store, piano).

• U2. He had frequented the store for many years.

Cb(U2) = John. Cf(U2) = (John, store). CONT

• U3. He was excited that he could finally buy a piano.

Cb(U3) = John. Cf(U3) = (John, piano). CONT

• U4. He arrived just as the store was closing for the day.
Cb(U4) = John. Cf(U4) = (John, store). CONT
Transitions for the 2nd discourse

• U1. John went to his favorite music store to buy a piano. Cb(U1) =
John. Cf(U1) = (John, store, piano).

• U2. It was a store John had frequented for many years.

Cb(U2) = John. Cf(U2) = (store, John). RETAIN

• U3. He was excited that he could finally buy a piano.

Cb(U3) = John. Cf(U3) = (John, piano). CONT

• U4. It was closing just as John arrived.

Cb(U4) = John. Cf(U4) = (store, John). RETAIN
Centering algorithm
• An algorithm for centering and pronoun binding
has been presented by Susan E. Brennan, Marilyn
W. Friedman and Carl J. Pollard, based on the
centering theory we have just discussed.
General structure of algorithm
For each utterance perform the following steps
Anchor construction: Create all possible anchors (pairs of forward
centers and a backward center).

Anchor filtering: Filter out the bad anchors according to

various filters.

Anchor ranking: Rank the remaining anchors according to

their transition type.
 General structure of algorithm

This is very similar to the general architecture of the algorithm in

Lappin & Leass 1994:

❑ First: filtering based on hard constraints

❑ Then: ranking based on some soft constraints

Construction of the anchors
1. Create a list of referring expressions (REs) in the utterance,
ordered by grammatical relation.

2. Expand each RE into a center according to whether it is a

pronoun or a proper name. In case of pronouns, the
agreement features must match.

3. Create a set of backward centers according to the forward

centers of the previous utterance, plus NIL.

4. Create a set of anchors, which is the Cartesian product of the

possible backward and forward centers.
Filtering the proposed anchors
• The constructed anchors undergo the following filters.

1. Remove all anchors that assign the same center to two

syntactic positions that cannot co-index (binding theory).

2. Remove all anchors which violate constraint 3, i.e. whose Cb is

not the highest ranking center of the previous Cf which appears
in the anchor’s Cf list.

3. Remove all anchors which violate rule 1. If the utterance has

pronouns then remove all anchors where the Cb is not realized
by a pronoun.
Ranking the anchors
• Classify, every anchor that passed the filters, into its
transition type (cont, retain, shift-1, shift).

• Choose the anchor with the most preferable transition

type according to rule 2.
Let’s look at an example
• U1. John likes to drive fast.
Cb(U1) = John. Cf(U1) = (John)

• U2. He races Mike.

Cb(U2) = John. Cf(U2) = (John, Mike)

• U3. Mike beats him sometimes.

Let’s generate the anchors for U3.

Anchor construction for U3
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Create a list of REs.

REs in U3

Mike him
Anchor construction for U3
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Create a list of REs.
2. Expand into possible forward center lists.

Potential Cfs REs in U3

Mike John
Mike him
Mike Mike
Anchor construction for U3
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Create a list of REs.
2. Expand into possible forward center lists.
3. Create possible backward centers according to Cf(U2).

Potential Cbs Potential Cfs REs in U3

John
Mike John
Mike Mike him
Mike Mike
NIL
Anchor construction for U3
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Create a list of REs.
2. Expand into possible forward center lists.
3. Create possible backward centers according to Cf(U2).
4. Create a list of all anchors (cartesian product).

Potential Cbs Potential Cfs REs in U3

John
Mike John
Mike Mike him
Mike Mike
NIL
Anchor construction for U3
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Create a list of REs.
2. Expand into possible forward center lists.
3. Create possible backward centers according to Cf(U2).
4. Create a list of all anchors (cartesian product).

Cb John John Mike Mike NIL NIL

Mike Mike Mike Mike Mike Mike Mike

him John Mike John Mike John Mike

 Filtering the anchors
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Remove all anchors that assign the same center to two syntactic
positions that cannot co-index.

Cb John John Mike Mike NIL NIL

Mike Mike Mike Mike Mike Mike Mike

him John Mike John Mike John Mike

 Filtering the anchors
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Remove all anchors that assign the same center to two syntactic
positions that cannot co-index.
2. Remove all anchors which violate constraint 3, i.e. whose Cb is not the
highest ranking center in Cf(U2) which appears in the anchor’s Cf.

Cb John John Mike Mike NIL NIL

Mike Mike Mike Mike Mike Mike Mike

him John Mike John Mike John Mike

 Filtering the anchors
• Cb(U2) = John. Cf(U2) = (John, Mike)
• U3. Mike beats him sometimes.
1. Remove all anchors that assign the same center to two syntactic
positions that cannot co-index.
2. Remove all anchors which violate constraint 3, i.e. whose Cb is not the
highest ranking center in Cf(U2) which appears in the anchor’s Cf.
3. Remove all anchors which violate rule 1, i.e. the Cb must be realized by
a pronoun.

Cb John John Mike Mike NIL NIL

Mike Mike Mike Mike Mike Mike Mike

him John Mike John Mike John Mike

Ranking the anchors
• Cb(U2) = John. Cf(U2) = (John, Mike)

• The only remaining anchor –

Cb(U3) = John. Cf(U3) = (Mike, John)

• RETAIN
Sample Questions
1. What is Coreference Resolution?
2. Explain anaphora and cataphora with example.
3. What are the five types of referring expressions
used in reference phenomenon?
4. Explain Hobbs algorithm for pronoun resolution.
5. Which referring expressions complicate the
reference resolution problem?
6. What is coherence relations?
7. What is reference resolution?
8. Explain pronoun resolution with Lapin and Leass
algorithm.