CS498JH: Introduction to NLP (Fall 2012)

http://cs.illinois.edu/class/cs498jh

Lecture 15:
Dependency
Grammars
Julia Hockenmaier
[email protected]
3324 Siebel Center
Office Hours: Wednesday, 12:15-1:15pm

Dependency grammar
Word-word dependencies are a component of many
(most/all?) grammar formalims.
Dependency grammar assumes that syntactic
structure consists only of dependencies.
Many variants. Modern DG began with Tesniere (1959).

DG is often used for free word order languages.

DG is purely descriptive (not a generative system
like CFGs etc.), but certain formal equivalences are
known.
CS498JH: Introduction to NLP

What is a dependency?
Dependencies are (labeled) asymetrical binary
relations between two lexical items (words).
There is a syntactic relation between a head H and a
dependent D in a construction C if:
- the head H determines the syntactic category of the
construction C.
- the head H determines the semantic category of the
construction C; D gives semantic specification.
- the head H is obligatory. D may be optional.
- the head selects D and determines whether D is obligatory or
not.
- The form of D depends on the head H (agreement)
- The linear position of D depends on the head H.
CS498JH: Introduction to NLP

Different kinds of dependencies

Head-argument (exocentric): eat sushi
Arguments may be obligatory, but can only occur once.
The head alone cannot necessarily replace the construction.

Head-modifier (endocentric): fresh sushi

Modifiers are optional, and can occur more than once.
The head alone can replace the entire construction.

Head-specifier (exocentric; Tesnieres transfer): the sushi

Between function words (e.g. prepositions, determiners)
and their arguments. Syntactic head semantic head

Coordination: (Tesnieres junction): sushi and sashimi

Unclear where the head is.
CS498JH: Introduction to NLP

Context-free grammars
CFGs capture only nested dependencies
The dependency graph is a tree
The dependencies do not cross

CS498JH: Introduction to NLP

Beyond CFGs:
Nonprojective dependencies
Dependencies: tree with crossing branches
Arise in the following constructions
- (Non-local) scrambling (free word order languages)
Die Pizza hat Klaus versprochen zu bringen

- Extraposition (The guy is coming who is wearing a hat)

- Topicalization (Cheeseburgers, I thought he likes)

CS498JH: Introduction to NLP

Dependency structures
Dependencies form a graph over the words in a
sentence.
This graph is connected (every word is a node)
and (typically) acyclic (no loops).
Single-head constraint:
Every node has at most one incoming edge.
This implies that the graph is a rooted tree.

CS498JH: Introduction to NLP

Dependency trees and the linear

order of words
Dependency trees do not specify the order of words in
a sentence.
(Sometimes additional linear precedence constraints are introduced).

A dependency tree is projective if there are no

crossing links.
Projective DG is weakly equivalent to CFG.
Parsing is more difficult for non-projective DGs
CS498JH: Introduction to NLP

Dependency Treebanks
Dependency treebanks exist for many languages:
Czech
Arabic
Turkish
Danish,
Portuguese
Estonian,
....

Phrase-structure treebanks (e.g. the Penn Treebank) can

also be translated into dependency trees
(although there might be noise in the translation)
CS498JH: Introduction to NLP

The Prague Dependency Treebank

Three levels of annotation:
morphological: [<2M tokens]
Lemma (dictionary form) + detailed analysis
(15 categories with many possible values = 4,257 tags)
surface-syntactic (analytical): [1.5M tokens]
Labeled dependency tree encoding grammatical functions
(subject, object, conjunct, etc.)
semantic (tectogrammatical): [0.8M tokens]
Labeled dependency tree for predicate-argument structure,
information structure, coreference (not all words included)
(39 labels: agent, patient, origin, effect, manner, etc....)

CS498JH: Introduction to NLP

10

Examples: analytical level

CS498JH: Introduction to NLP

11

METU-Sabanci Turkish
Treebank
Very small -- about 5000 sentences
Turkish is an agglutinative language with free word
order:

Dependencies are at the morpheme level

CS498JH: Introduction to NLP

12

[this and prev. example from Kemal Oflazers talk at Rochester, April 2007]
CS498JH: Introduction to NLP

13

Dependency or phrase structure

annotation?
No clear consensus which is better.
May depend on the language.
It may also depend on the annotation guidelines:
Early phrase-structure treebanks (Penn Treebank)
are not explicit enough (not all nodes have function tags).
Dependency treebanks (e.g. Sabanci) often omit
long-range dependencies.
They also cant express scope relations.

CS498JH: Introduction to NLP

14

Parsing algorithms for DG

Transition-based parsers:
learn a sequence of actions to parse sentences
Models:
State = stack of partially processed items + queue of remaining tokens;
Transitions (actions) = add dependency arcs; stack/queue operations

Graph-based parsers:
learn a model over dependency graphs
Models:
a function (typically sum) of local attachment scores

CS498JH: Introduction to NLP

15

Transition-based parsing
(Nivre et al.)

CS498JH: Introduction to NLP

16

Transition-based parsing
Transition-based shift-reduce parsing processes
the sentence S= w0w1...wn from left to right.
Unlike CKY, it constructs a single tree.
N.B: this only works for projective dependency trees

Notation:
w0 is a special ROOT token.
VS = {w0, w1, ..., wn} is the vocabulary of the sentence
R is a set of dependency relations

The parser uses three data structures:

: a stack of words wi VS
: a buffer of words wi VS
A: a set of dependency arcs (wi, r, wj) VS R VS
CS498JH: Introduction to NLP

17

Parser configurations (, , A)
Stack : a list of words that are partially processed
We push and pop words onto
|w : w is on top of the stack
The buffer is the remaining input words
We read words from and push them onto
w| : w is on top of the buffer
The set of arcs A defines the current tree
Initial configuration: ([w0], [w1, ...., wn], {})
Terminal configuration: (, [], A)
CS498JH: Introduction to NLP

18

Parser actions

words that are attached to other

words are fully processed

SHIFT: push the next input word onto the stack

(, wi|, A) (|wi, , A)
LEFT-ARCr: attach wi (top of stack) to wj (top of buffer)
If stack and buffer not empty, wi not Root:
(|wi, wj|, A) (, wj|, A {(wj, r, wi)})
RIGHT-ARCr: attach wj (top of buffer) to wj (top of stack)
Move wj back to the buffer.
If stack and buffer not empty:
(|wi, wj|, A) (, wi|, A {(wi, r, wj)})
CS498JH: Introduction to NLP

19

An example sentence & parse

2

CHAPTER 1. INTRODUCTION

Figure 1.1: Dependency structure for an English sentence.

The basic assumption underlying all varieties of dependency grammar is the idea that syntactic
structure essentially consists of words linked by binary, asymmetrical relations called dependency
relations (or dependencies for short). A dependency relation holds between a syntactically subordinate
word, called the dependent, and another word on which it depends, called the head.1 This is illustrated
in figure 1.1, which shows a dependency structure for a simple English sentence, where dependency
relations are represented by arrows pointing from the head to the dependent.2 Moreover, each arrow
20
CS498JH: Introduction to NLP
has a label, indicating the dependency type. For example, the noun news is a dependent of the verb
had with the dependency type subject (SBJ). By contrast, the noun effect is a dependent of type object

Economic news had

little effect on financial markets .
HAPTER 3. TRANSITION-BASED
PARSING
Transition Configuration
SH
LAatt
SH
LAsbj
SH
SH
LAatt
SH
SH
SH
LAatt
RApc
RAatt
RAobj
SH
RApu
RApred
SH

([root],
([root, Economic],
([root],
([root, news],
([root],
([root, had],
([root, had, little],
([root, had],
([root, had, effect],
([root, . . . on],
([root, . . . , financial],
([root, . . . on],
([root, had, effect],
([root, had],
([root],
([root, had],
([root],
([ ],
([root],

[Economic, . . . , .],
[news, . . . , .],
[news, . . . , .],
[had, . . . , .],
[had, . . . , .],
[little, . . . , .],
[effect, . . . , .],
[effect, . . . , .],
[on, . . . , .],
[financial, markets, .],
[markets, .],
[markets, .],
[on, .],
[effect, .],
[had, .],
[.],
[had],
[root],
[ ],

)
)
A1 = {(news, ATT, Economic)})
A1 )
A2 = A1 {(had, SBJ, news)})
A2 )
A2 )
A3 = A2 {(effect, ATT, little)})
A3 )
A3 )
A3 )
A4 = A3 {(markets, ATT, financial)})
A5 = A4 {(on, PC, markets)})
A6 = A5 {(effect, ATT, on)})
A7 = A6 {(had, OBJ, effect)})
A7 )
A8 = A7 {(had, PU, .)})
A9 = A8 {(root, PRED, had)})
A9 )

21
CS498JH: Introduction
to NLP
ure 3.2: Transition
sequence
for the English sentence in figure 1.1 (LAr = Left-Arc
r , RA
ht-Arcr , SH = Shift).