Structural and Syntactic

Pattern Recognition

Selim Aksoy

Department of Computer Engineering

Bilkent University
[email protected]

CS 551, Fall 2018

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 1 / 60

 Introduction

I Statistical pattern recognition attempts to classify patterns

based on a set of extracted features and an underlying
statistical model for the generation of these patterns.
I Ideally, this is achieved with a rather straightforward
procedure:
I determine the feature vector,
I train the system,
I classify the patterns.
I Unfortunately, there are also many problems where
patterns contain structural and relational information that
are difficult or impossible to quantify in feature vector form.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 2 / 60

 Introduction

I Structural pattern recognition assumes that pattern

structure is quantifiable and extractable so that structural
similarity of patterns can be assessed.
I Typically, these approaches formulate hierarchical
descriptions of complex patterns built up from simpler
primitive elements.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 3 / 60

 Introduction

I This structure quantification and description are mainly

done using:
I Formal grammars,
I Relational descriptions (principally graphs).
I Then, recognition and classification are done using:
I Parsing (for formal grammars),
I Relational graph matching (for relational descriptions).
I We will study strings, grammatical methods, and
graph-theoretic approaches.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 4 / 60

 Recognition with Strings
I Suppose the patterns are represented as ordered
sequences or strings of discrete items, as in a sequence of
letters in a word or in DNA bases in a gene sequence.
I Pattern classification methods based on such strings of
discrete symbols differ in a number of ways from the more
commonly used techniques we have discussed earlier.
I Definitions:
I String elements are called characters (or letters, symbols).
I String representation of a pattern is also called a word.
I A particularly long string is denoted text.
I Any contiguous string that is part of another string is called a
factor (or substring, segment) of that string.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 5 / 60
 Recognition with Strings
I Important pattern recognition problems that involve
computations on strings include:
I String matching: Given string x and text, determine whether
x is a factor of text, and if so, where it appears.
I String edit distance: Given two strings x and y, compute the
minimum number of basic operations — character insertions,
deletions and exchanges — needed to transform x into y.
I String matching with errors: Given string x and text, find the
locations in text where the “distance” of x to any factor of text
is minimal.
I String matching with the “don’t care” symbol: This is the
same as basic string matching but the special “don’t care”
symbol can match any other symbol.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 6 / 60
 String Matching

I The most fundamental and useful operation in string

matching is testing whether a candidate string x is a factor
of text.
I The number of characters in text is usually much larger than
that in x, i.e., |text|  |x|, where each discrete character is
taken from an alphabet.
I A shift, s, is an offset needed to align the first character of x
with character number s + 1 in text.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 7 / 60

 String Matching

I The basic string matching problem is to find whether there

exists a valid shift, i.e., one where there is a perfect match
between each character in x and the corresponding one in
text.
I The general string matching problem is to list all valid shifts.
I The most straightforward approach is to test each possible
shift in turn.
I More sophisticated methods use heuristics to reduce the
number of comparisons.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 8 / 60

 String Matching

Figure 1: The general string matching problem is to find all shifts s for which
the pattern x appears in text. Any such shift is called valid. In this example,
x = “bdac00 is indeed a factor of text, and s = 5 is the only valid shift.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 9 / 60

 String Edit Distance

I The fundamental idea underlying pattern recognition using

edit distance is based on the nearest neighbor algorithm.
I We store a full training set of strings and their associated
category labels.
I During classification, a test string is compared to each
stored string, a “distance” is computed, and the string is
assigned the category label of the nearest string in the
training set.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 10 / 60

 String Edit Distance

I Edit distance between x and y describes how many of the

following fundamental operations are required to transform
x and y.
I Substitutions: A character in x is replaced by the
corresponding character in y.
I Insertions: A character in y is inserted into x, thereby
increasing the length of x by one character.
I Deletions: A character in x is deleted, thereby decreasing
the length of x by one character.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 11 / 60

 String Edit Distance

x excused source string

exhused substitute h for c
exhaused insert a
exhausted insert t
y exhausted target string
Figure 2: Transformation of x = “excused” to y = “exhausted” through one
substitution and two insertions.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 12 / 60

 String Matching with Errors

I Given a pattern x and text, string matching with errors

algorithm finds the shift for which the edit distance between
x and a factor of text is minimum.
I The algorithm for the string matching with errors problem is
very similar to that for edit distance but some additional
heuristics can reduce the computational burden.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 13 / 60

 String Matching with Errors

Figure 3: Finding the shift s for which the edit distance between x and an
aligned factor of text is minimum. In this figure, the minimum edit distance is
1, corresponding to the character exchange u → i, and the shift s = 11 is the
location.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 14 / 60

 String Matching with “Don’t Care”
I String matching with the “don’t care” symbol, ∅, is formally
the same as basic string matching, but the ∅ in either x or
text is said to match any character.
I The straightforward approach is to modify the string
matching algorithm to include a condition for matching the
“don’t care” symbol.

Figure 4: The problem of string matching with the “don’t care” symbol is the
same as the basic string matching except that the ∅ symbol can match any
character. The figure shows the only valid shift.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 15 / 60
 Grammatical Methods

I Grammars provide detailed models that underlie the

generation of the sequence of characters in strings.
I For example, strings representing telephone numbers
conform to a strict structure.
I Similarly, optical character recognition systems that
recognize and interpret mathematical equations can use
rules that constrain the arrangement of the symbols.
I In pattern recognition, we are given a sentence (a string
generated by a set of rules) and a grammar (the set of
rules), and seek to determine whether the sentence was
generated by this grammar.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 16 / 60

 Grammatical Methods
I Formally, a grammar consists of four components:
I Symbols: Every sentence consists of a string of characters
(or primitive symbols, terminal symbols) from an alphabet.
I Variables: These are called the nonterminal symbols (or
intermediate symbols, internal symbols).
I Root symbol: It is a special variable, the source from which
all sequences are derived.
I Productions: The set of production rules (or rewrite rules)
specify how to transform a set of variables and symbols into
other variables and symbols.
I For example, if A is a variable and c a terminal symbol, the
rewrite rule cA → cc means that any time the segment cA
appears in a string, it can be replaced by cc.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 17 / 60
 Grammatical Methods
I The language L(G) generated by a grammar G is the set of
all strings (possibly infinite in number) that can be
generated by G.

Figure 5: The derivation tree illustrates how a portion of English grammar

can transform the root symbol into a particular sentence.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 18 / 60
 Types of String Grammars

I Type 0: Free or Unrestricted: Free grammars have no

restrictions on the rewrite rules, and thus they provide no
constraints or structure on the strings they can produce.
I Type 1: Context-Sensitive: A grammar is called
context-sensitive if every rewrite rule is of the form

αIβ → αxβ

where α and β are any strings made up of intermediate and

terminal symbols, I is an intermediate symbol, and x is an
intermediate or terminal symbol, i.e., I can be rewritten as x
in the context of α on the left and β on the right.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 19 / 60
 Types of String Grammars

I Type 2: Context-Free: A grammar is called context-free if

every production rule is of the form

I→x

where I is an intermediate symbol, and x is an intermediate

or terminal symbol, i.e., there is no need for a context for
the rewriting of I by x.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 20 / 60

 Types of String Grammars

I Type 3: Finite State or Regular: A grammar is called regular

if every rewrite rule is of the form

α → zβ or α→z

where α and β are made up of intermediate symbols and z

is a terminal symbol.
I The class of grammars of type i includes all grammars of
type i + 1.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 21 / 60

 Recognition Using Grammars

I Suppose we are given a test sentence x that was

generated by one of c different grammars G1 , G2 , . . . , Gc
which can be considered as different models or classes.
I The test sentence x is classified according to which
grammar could have produced it, or equivalently, the
language L(Gi ) of which x is a member.
I Parsing is the inverse process that, given a particular x,
finds a derivation in G that leads to x.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 22 / 60

 Recognition Using Grammars

I Bottom-up parsing starts with the test sentence x, and

seeks to simplify it, so as to represent it as the root symbol.
I The basic approach is to use candidate productions
backwards, i.e., find rewrite rules whose right hand side
matches part of the current string, and replace that part
with a segment that could have produced it.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 23 / 60

 Recognition Using Grammars

I Top-down parsing starts with the root node and

successively applies productions with the goal of finding a
derivation of the test sentence x.
I Since it is rare that the sentence is derived in the first
production attempted, it is necessary to specify some
criteria to guide the choice of which rewrite rule to apply.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 24 / 60

 Pattern Description Using Grammars

Figure 6: A 2-D line drawing picture description grammar with the set of
terminal symbols {t, b, u, o, s, ∗, ¬, +} where + represents head to tail
concatenation, ∗ represents head-head and tail-tail attachment, and ¬
represents head and tail reversal. H represents heads of lines and T
represents the tails. (Schalkoff, Pattern Recognition: Statistical, Structural
and Neural Approaches, 1992)

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 25 / 60

 Pattern Description Using Grammars

Figure 7: Representation of a cylinder using the line drawing picture

description grammar.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 26 / 60

 Pattern Description Using Grammars

Figure 8: Representation of four characters using the line drawing picture

description grammar. (a) Pattern data. (b) Primitive representation and
interconnection. (c) Corresponding descriptions.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 27 / 60
 Pattern Description Using Grammars
I A grammar describing four blocks arranged in 2-block
stacks:
VT = {table, block , +, ↑} (terminal symbols)
VN = {DESC , LEFT STACK , RIGHT STACK }
(non-terminal symbols)
S = DESC ∈ VN (root symbol)
P = {DESC → LEFT STACK + RIGHT STACK
DESC → RIGHT STACK + LEFT STACK
LEFT STACK → block ↑ block ↑ table
RIGHT STACK → block ↑ block ↑ table}
(production rules)
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 28 / 60
 Pattern Description Using Grammars

Figure 9: A grammar describing four blocks arranged in 2-block stacks. (i)

An example. (ii) Graphical description corresponding to (i). (iii) Another
example. (iv) Graphical description corresponding to (iii).

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 29 / 60

 Pattern Description Using Grammars
I A grammar describing four blocks in 3-block and 1-block
stacks:
VT = {table, block , +, ↑} (terminal symbols)
VN = {DESC , LEFT STACK , RIGHT STACK }
(non-terminal symbols)
S = DESC ∈ VN (root symbol)
P = {DESC → LEFT STACK + RIGHT STACK
LEFT STACK + RIGHT STACK →
block ↑ table + block ↑ block ↑ block ↑ table
LEFT STACK + RIGHT STACK →
block ↑ block ↑ block ↑ table + block ↑ table}
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 30 / 60
 Pattern Description Using Grammars

Figure 10: A grammar describing four blocks arranged in 3-block and

1-block stacks. (i) An example. (ii) Graphical description corresponding to (i).
(iii) Another example. (iv) Graphical description corresponding to (iii).

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 31 / 60

 Pattern Description Using Grammars

Figure 11: Tree grammar-based representation of a cube.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 32 / 60

 Pattern Description Using Grammars

Figure 12: Quadtree representation of an 8 × 8 binary image.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 33 / 60

 Graph-Theoretic Methods — Definitions

I Graphical alternatives for structural representations are

natural extensions of higher dimensional grammars
because graphs are valuable tools for representing
relational information.
I A graph G = {N, R} is an ordered pair represented using:
I a set of nodes (vertices), N ,
I a set of edges (arcs), R ⊆ N × N .
I A subgraph of G is itself a graph Gs = {Ns , Rs } where
Ns ⊆ N and Rs consists of edges in R that connect only the
nodes in Ns .

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 34 / 60

 Graph-Theoretic Methods — Definitions

I A graph is connected if there is a path between all pairs of

its nodes.
I A graph is complete if there is an edge between all pairs of
its nodes.
I A relation from set A to set B is a subset of A × B.
I It is usually shown using a function f : A → B or b = f (a).

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 35 / 60

 Graph-Theoretic Methods — Definitions

I For example, the relation “lies on” can contain:

R = {(floor, foundation), (rug, floor),

(chair, rug), (person, chair)}.

I Note that relations have directions, i.e., the order in which

an entity appears in the pair is significant.
I Higher-order relations can be shown as ordered n-tuples
that can also be viewed as ordered pairs of an (n − 1)-tuple
and a single element, e.g., (((A × B) × C) × D).

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 36 / 60

 Graph-Theoretic Methods — Definitions

I In directional graphs (digraphs), edges have directional

significance, i.e., (a, b) ∈ R means there is an edge from
node a to node b.
I When the direction of edges in a graph is not important, i.e.,
specification of either (a, b) or (b, a) ∈ R is acceptable, the
graph is an undirected graph.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 37 / 60

 Graph-Theoretic Methods — Definitions

I A relational graph represents a particular relation

graphically using arrows to show this relation between the
elements as a directed graph.
I A semantic net is a relational graph showing all the
relations between its nodes using labeled edges.
I A tree is a finite acyclic (containing no closed loops or paths
or cycles) digraph.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 38 / 60

 Comparing Relational Graph Descriptions

I One way to recognize structure using graphs is to let each

pattern structural class be represented by a prototypical
relational graph.
I An unknown input pattern is then converted into a structural
representation in the form of a graph, and this graph is then
compared with the relational graphs for each class.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 39 / 60

 Comparing Relational Graph Descriptions

I The observed data rarely matches a stored relational

representation “exactly”, hence, graph similarity should be
measured.
I One approach is to check whether the observed data match
a “portion” of a relational model.
I Case 1: Any relation not present in both graphs is a failure.
I Case 2: Any single match of a relation is a success.
I A realistic strategy is somewhere in between these
extremes.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 40 / 60

 Graph Isomorphism
I A digraph G with p nodes can be converted to an adjacency
matrix:
I Number each node by an index {1, . . . , p}.
I Represent the existence or absence of an edge as

1 if G contains an edge from node i to node j,
Adj(i, j) =
0 otherwise.

I Consider two graphs G1 = {N1 , R1 } and G2 = {N2 , R2 }.

I A homomorphism from G1 to G2 is a function f from N1 to
N2 :
(v1 , w1 ) ∈ R1 ⇒ (f (v1 ), f (w1 )) ∈ R2 .
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 41 / 60
 Graph Isomorphism
I A stricter test is that of isomorphism, where f is required to
be 1:1 and onto:
(v1 , w1 ) ∈ R1 ⇔ (f (v1 ), f (w1 )) ∈ R2 .

I Isomorphism simply states that relabeling of nodes yields

the same graph structure.
I Given two graphs G1 and G2 each with p nodes, to
determine isomorphism:
I Label the nodes of each graph with labels 1, . . . , p.
I Form the adjacency matrices M1 and M2 for both graphs.
I If M1 = M2 , G1 and G2 are isomorphic.
I Otherwise, consider all the p! possible labelings on G2 .
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 42 / 60
 Graph Isomorphism

a b c d a’ b’ c’ d’ a” b” c” d”
f (a) = a00
a 0 1 0 1 a’ 0 1 1 0 a” 0 1 0 1
f (b) = b00
b 1 0 1 0 b’ 1 0 0 1 b” 1 0 1 0
f (c) = c00
c 0 1 0 1 c’ 1 0 0 1 c” 0 1 0 1
f (d) = d00
d 1 0 1 0 d’ 0 1 1 0 d” 1 0 1 0

Figure 13: An example of isomorphism of two undirected graphs with p = 4.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 43 / 60

 Graph Isomorphism
I Unfortunately, determining graph isomorphism is
computationally expensive.
I Furthermore, it is also not a practical similarity measure
because it allows only exact matches but not all existing
relations for a given class are observed in practical
problems.
I G1 and G2 are called subisomorphic if a subgraph of G1 is
isomorphic to a subgraph of G2 .
I Clearly, this is a less restrictive structural match than that of
isomorphism.
I However, determining subisomorphism is also
computationally expensive.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 44 / 60
 Extensions to Graph Matching

I To allow structural deformations, numerous extensions to

graph matching have been proposed.
I Extract features from graphs G1 and G2 to form feature
vectors x1 and x2 , respectively, and use statistical pattern
recognition techniques to compare x1 and x2 .
I Use a matching metric as the minimum number of
transformations necessary to transform G1 into G2 .

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 45 / 60

 Extensions to Graph Matching

I Common transformations include:

I Node insertion,
I Node deletion,
I Node splitting,
I Node merging,
I Edge insertion,
I Edge deletion.
I Note that computational complexity can still be high and it
may be difficult to design a distance measure that can
distinguish structural deformations between different
classes.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 46 / 60

 Relational Graph Similarity
I Given a set of nodes N , a corresponding relational
description is defined as a set of relations
DN = {R1 , R2 , . . . , Rn } where each Ri ⊆ N × N (in general,
Ri ⊆ N × N × · · · × N ).
I Given two node sets A and B, with |A| = |B|, and
corresponding relational descriptions DA = {R1 , R2 , . . . , Rn }
and DB = {S1 , S2 , . . . , Sn }, the composition Ri ◦ f maps
n-tuples of A into n-tuples of B as

Ri ◦ f = {(b1 , b2 , . . . , bn ) ∈ B n | ∃(a1 , a2 , . . . , an ) ∈ An }

with f (ai ) = bi , i = 1, 2, . . . , n where An = A × A × · · · × A (n

times) and B n = B × B × · · · × B.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 47 / 60
 Relational Graph Similarity

I Based upon the i’th relation Ri in DA and the i’th relation Si

in DB , the structural error is computed as

E i (f ) = |Ri ◦ f − Si | + |Si ◦ f −1 − Ri |.

I E i (f ) simply measures the number of elements in Ri that

are not in Si and the number of elements of Si that are not
in Ri .

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 48 / 60

 Relational Graph Similarity

I The total structural error for this mapping is the sum over all
relations n
X
E(f ) = E i (f )
i=1

that defines the relational distance between DA and DB as

RD(DA , DB ) = min E(f )

f

which becomes 0 when DA and DB are isomorphic.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 49 / 60

 Attributed Relational Graphs

I In addition to representing pattern structure, the

representation may be extended to include numerical and
symbolic attributes of pattern primitives.
I An attributed graph G = {N, P, R} is a 3-tuple where
I N is a set of nodes,
I P is a set of properties of these nodes,
I R is a set of relations between nodes.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 50 / 60

 Attributed Relational Graphs

I Let piq (n) denote the value of the q’th property of node n of
graph Gi .
I Nodes ni ∈ Ni and n2 ∈ N2 are said to form an agreement
(n1 , n2 ) if
p1q (n1 ) ∼ p2q (n2 )
where “∼” denotes similarity.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 51 / 60

 Attributed Relational Graphs

I Let rji (nx , ny ) denote the j’th relation involving nodes

nx , ny ∈ Ni .
I Two assignments (n1 , n2 ) and (n01 , n02 ) are considered
compatible if

rj1 (n1 , n01 ) ∼ rj2 (n2 , n02 ) ∀j.

I Two attributed graphs G1 and G2 are isomorphic if there

exists a set of 1:1 assignments of nodes in G1 to nodes in
G2 such that all assignments are compatible.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 52 / 60

 Comparing Attributed Graph Descriptions
I A strategy for measuring the similarity between two
attributed graphs is to find node pairings using the cliques
of a match graph.
I A clique of a graph is a totally connected subgraph.
I A maximal clique is not included in any other clique.
I A match graph is formed from two graphs G1 and G2 as
follows:
I Nodes of the match graph are assignments from G1 to G2 .
I An edge in the match graph exists between two nodes if the
corresponding assignments are compatible.
I The maximal cliques of the match graph provide starting
points for good candidate node pairings between two
graphs.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 53 / 60
 Comparing Attributed Graph Descriptions

I Another similarity measure between two attributed graphs is

the editing distance which is defined as the minimum cost
taken over all sequences of operations (error corrections)
that transform one graph to the other.
I These operations are defined as substitution, insertion and
deletion.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 54 / 60

 Comparing Attributed Graph Descriptions

I Let G1 and G2 be two graphs where each node and edge

are assigned labels with an additional confidence value for
each label.
I Let f1 be the confidence value of the label of a node in the
first graph and f2 be the confidence value of a node with the
same label in the second graph.
I The cost of node substitution is |f1 − f2 | and the cost of
node insertion or deletion is f1 .

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 55 / 60

 Comparing Attributed Graph Descriptions

I Let g1 be the confidence value of the label of an edge in the

first graph and g2 be the corresponding value for an edge in
the second graph.
I The cost of edge substitution is |g1 − g2 | and the cost of
edge insertion or deletion is g1 .
I The computation of the distance between two attributed
graphs involves not only finding a sequence of error
corrections that transforms one graph to the other, but also
finding the one that yields the minimum total cost.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 56 / 60

 Comparing Attributed Graph Descriptions

The editing distance between these two graphs is computed as:

I node substitution:
I object 0 in (b) with object 0 in (a) with cost |1.0 − 1.0| = 0
I object 1 in (b) with object 1 in (a) with cost |0.4 − 0.3| = 0.1
I object 2 in (b) with object 3 in (a) with cost |0.1 − 0.2| = 0.1
I edge substitution for these nodes:
I |0.7 − 0.7| = 0, |0.3 − 0.4| = 0.1, |0.3 − 0.3| = 0
I node deletion: object 2 in (a) with cost 0.2
I edge deletion for this node: 0.5, 0.2, 0.1
Total cost of matching is 0.1 + 0.1 + 0.1 + 0.2 + 0.5 + 0.2 + 0.1 = 1.3.
(Taken from Petrakis et al. “ImageMap: An Image Indexing Method Based on Spatial Similarity,” IEEE
Trans. on Knowledge and Data Engineering, 14(5):979–987, 2002.)

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 57 / 60

 Comparing Attributed Graph Descriptions

Figure 14: An example image scene and its attributed graph. Nodes
correspond to image regions marked with white boundaries and edges
correspond to the spatial relationships between these regions.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 58 / 60

 Comparing Attributed Graph Descriptions

The editing distance between these two graphs is computed as:

I node substitution:
I park in (a) with park in (b) with cost |0.8 − 0.9| = 0.1
I city in (a) with city in (b) with cost |0.9 − 0.9| = 0
I water in (a) with water in (b) with cost |1.0 − 1.0| = 0
I edge substitution for these nodes:
I |0.9 − 0.8| + 0.8 = 0.9, |1.0 − 1.0| + |0.8 − 0.9| = 0.1, |0.6 − 0.7| = 0.1,
I node insertion: field in (a) with cost 0.7
I edge insertion for this node: 0.4, 0.7, 0.6
Total cost of matching is 3.6.
CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 59 / 60
 Comparing Attributed Graph Descriptions

(a) (b) (c) (d)

Figure 15: Scene matching using attributed graphs. (a) Query scene
marked using the red rectangle. (b) Query graph: red is city, green is park,
blue is water. (c) Nodes with labels similar to those in the query. (d)
Subgraphs matching to the query.

CS 551, Fall 2018 c 2018, Selim Aksoy (Bilkent University) 60 / 60