SlideShare a Scribd company logo
Entropy based measures for graphs
Master in Web Science, Department of Mathematics,
Aristotle University of Thessaloniki
Giorgos Bamparopoulos, Nikos Papathanasiou
Introduction
In many scientific areas, e.g. sociology, psychology, physics, chemistry,
biology etc., systems can be described as interaction networks where the
interactions correspond to edges and the elements to vertices. A variety of
problems in those fields can deal with network comparison and
characterization.
The problem of comparing networks is the task of measuring their
structural similarity. Moreover, “characterize networks” means that we are
searching for network characteristics which capture structural
information of networks. To analyze complex networks, several methods
can be combined, such as graph theory, information theory, and statistics.
In this project, in order to characterize and compare network structures,
we describe methods for measuring Shannon’s entropy of graphs.
To introduce entropy based measures for graphs, we first mention some
basic graph-theoretical preliminaries:
Preliminaries
A graph is a non-empty finite set V of elements called vertices
together with a possibly empty set E of pairs of vertices called
edges. It is denoted G=(V,E)
The degree of a vertex u is the number of links plus twice the
number of loops incident with u. It is denoted d(u). The degree
distribution is the relative frequency of vertices with different
degrees. (P(k) = fraction of vertices with degree k)
The j-sphere of a vertex vi in G is defined by the set Sj(vi, G): =
Preliminaries
A path of length n , n ≥1 , from vertex a to vertex b is a sequence
of directed edges e1, e2, ...,en such that the initial vertex of e1 is a ,
and the terminal vertex of en is b, and for i=2, ..., n , the initial
vertex of ei is the terminal vertex of ei-1 . When it is more
convenient, vertices can be listed in order (rather than the
edges) to represent the path.
If the first and last nodes of a path coincide it is called cycle.
d(u,v) denotes the distance between u∈V and v ∈ V expressed
as the minimum length of a path between u,v.
Preliminaries
A coloring of a graph G(V,E) is an assignment of colors to the
vertices such that no two adjacent vertices have the same color
 An n-coloring of a graph G = (V, E) is a coloring with n colors.
More precisely, a mapping f of V onto the set {1, 2, . . . , n} such that
whenever [u, v]∈E, f(u)≠ f(v).
The chromatic number X(G) of a graph G is the minimum
number of colors needed to color the vertices of G so that no two
adjacent vertices have the same color.
An n-coloring is complete if, ∀ i, j with i≠j, there exist adjacent
vertices u and v such that f(u) = i and f(v) = j. The sets of vertices
𝑉𝑖 𝑖=1
𝑛
with the same color is called color classes.
Preliminaries
We call the quantity σ(v)= )max 𝑢𝑒𝑣 𝑑(𝑢, 𝑣 the eccentricity of
v∈V. The diameter ρ of a connected graph G is the maximum
eccentricity among all nodes of G.
 For a given path P = {u0, u1, . . . , un}, a downstream edge of a
vertex ut ∈ P is any edge (ut,w) ∈E with w≠us for s < t.
A downstream vertex of ut is any vertex w such that (ut ,w) is a
downstream edge.
The downstream degree, D(ut), of a vertex is the number of
downstream edges it has.
Basic concepts
Information content of graphs
Entropy of degree distribution
Entropy as a measure of centrality
Entropy as a measure of connectivity
Topology entropy of a network
Information content of graphs
Rashevsky, Trucco and Mowshowitz were the first researchers to
define and investigate the Shannon-entropy of graphs.
Rashevsky in 1955, defined the topological information content of
the graph, dealing with the complexity of organic molecules, where
vertices represent physically indistinguishable atoms and edges
represent chemical bonds.
His definition is based on the partitioning of the n vertices of a
given graph into k classes of equivalent vertices, according to their
degree dependence. Then, assigned a probability to each partition
obtained as the fraction of vertices in this partition divided by the
total number of vertices.
Examples:
Graph has 3 vertices of degree two (1, 3, and 4), and two vertices (2 and
5) of degree three. However, vertex 1 is topologically different from
vertices 3 and 4. Vertex 1 is adjacent to two vertices of degree three,
while each of the vertices 3 and 4 is adjacent to one vertex of degree
three and one of degree two. Hence we have classes of vertices, 1 with
probability
1
5
, 2 and 5 with probability
2
5
, 3 and 4 with probability
2
5
.
The information content of is:
By adding the vertex 6, vertices 2 and 5 have different degrees.
Moreover vertex 3 is adjacent to one vertex of degree four and to
one of degree two, while the vertex 4 is adjacent to one vertex of
degree three and one of degree two. So, we have six distinct classes
each of them have probability
1
6
. Hence, the information content of
is:
𝑰 𝒄 = −(
𝟏
𝟓
𝒍𝒅
𝟏
𝟓
+
𝟐
𝟓
𝒍𝒅
𝟐
𝟓
+
𝟐
𝟓
𝒍𝒅
𝟐
𝟓
) =
𝟏
𝟓
𝒍𝒅𝟓 +
𝟐
𝟓
𝒍𝒅
𝟓
𝟐
+
𝟐
𝟓
𝒍𝒅
𝟓
𝟐
= 𝟏. 𝟓𝟑
𝑰 𝒅 = 𝒍𝒅𝟔 = 𝟐. 𝟔
Trucco in 1956 made this definition more precision. He considered
that two vertices are equivalent if they belong to the same orbit of this
group, i. e., if they can interchange preserving the adjacency of the
graph.
We denote the group of all automorphisms of a graph X by G(X).
X is a graph with 𝑉(𝑋) = {𝑥1, . . . 𝑥 𝑛 . If f∈G(X), 𝑓(𝑥𝑖) = 𝑥𝑗 for 1 < i < n.
Hence, to each f∈G(X), there corresponds a unique permutation of the
elements 1,2,...,n. We can regard G(X) as a subgroup of 𝑆𝑛, the
symmetric group of degree n.
Suppose K is a subgroup of 𝑆𝑛 and 1 ≤ 𝑖 ≤ 𝑛. Then 𝑓(𝑖 ) 𝑓 ∈ 𝐾 is
called an orbit of K. Let 𝐴𝑖, 1 ≤ 𝑖 ≤ 𝑛 be the distinct orbits of K. Then
𝐴𝑖 ∩ 𝐴𝑗 = ∅ , if i≠j and ∪
𝑖=1
𝑛
𝐴𝑖 = 1, . . . , 𝑛 , hence the orbits form a
partition of the set {1, 2 , . . . , n}. We assigned probability 𝑝𝑖 =
|𝐴 𝑖|
𝑛
to
each orbit, where |𝐴𝑖| is the cardinality of 𝐴𝑖.
Examples:
)𝐺(𝑋1 ={e, (12)}, Orbits of )𝐺(𝑋1 : {1, 2}, {3},{4}. Hence,
)𝐺(𝑋2 ={e, (23), (24), (34), (234), (243)}, Orbits of )𝐺(𝑋2 :
{2,3,4},{1} Hence,
𝑰 𝒈(𝑿 𝟏) = −(
𝟐
𝟒
𝒍𝒅
𝟑
𝟒
+
𝟏
𝟒
𝒍𝒅
𝟏
𝟒
+
𝟏
𝟒
𝒍𝒅
𝟏
𝟒
) =
𝟑
𝟐
𝑰 𝒈(𝑿 𝟐) = −(
𝟑
𝟒
𝒍𝒅
𝟑
𝟒
+
𝟏
𝟒
𝒍𝒅
𝟏
𝟒
) = 𝟐 −
𝟑
𝟒
𝒍𝒅𝟑
)𝐺(𝑋3 ={e, (13), (24), (13) (24)}. Orbits of )𝐺(𝑋3 :
{1, 3},{2, 4}. Hence,
G(𝑋4)= {(1234), (13), (24), (13) (24), (12) (34),
(14) (23) (1432)}. Orbits of G(𝑋4): {1,2,3,4}
•
𝑰 𝒈(𝑿 𝟑) = −(
𝟐
𝟒
𝒍𝒅
𝟐
𝟒
+
𝟐
𝟒
𝒍𝒅
𝟐
𝟒
) = −𝒍𝒅
𝟐
𝟒
= 𝒍𝒅𝟒 − 𝒍𝒅𝟐 = 𝟏
𝑰 𝒈(𝑿 𝟒) = −(
𝟒
𝟒
𝒍𝒅
𝟒
𝟒
) = −𝒍𝒅𝟏 = 𝟎
In 1968 Mowshovitz has studied in detail the information
content of the graphs and has formulated, on the basis of the
chromatic properties of the graphs.
 A decomposition 𝑉𝑖 𝑖=1
𝑛
of the set of vertices V is called a
chromatic decomposition of G if u, v ∈ Vi imply that [u, v]
∉ E. If f is an n-coloring, the collection of sets { 𝑣 ∈
For the graph G(V,E), |𝑉| = 6, X(G)=3
Chromatic decomposition is :
A:{1,3}, {2,5}, {4,6}
B: {2}, {1,3,} {4,5,6}
C:{2}, {3}, {1,4,5,6}
Information content base on functionals
M. Dehmer and F. Emmert-Streib (2007) presented a method to
determine the structural information content which is not based on the
problem of finding partitions of the vertices, and the overall time
complexity is polynomial.
They assigned a probability to each vertex 𝒗𝒊 as,
where f represents an arbitrary information functional.
In this case 𝒇 𝒗𝒊 = 𝒂 𝒄 𝟏|𝒔 𝟏 𝒗 𝒊,𝑮 |+...+𝒄 𝝆|𝒔 𝝆 𝒗 𝒊,𝑮 |
, 1 ≤ k ≤ ρ, α >0, 𝑐 𝑘 are real
positive coefficients and they are chosen in such a way to emphasize
certain structural characteristics e.g. high vertex degrees.
𝒑(𝒗𝒊) =
)𝒇(𝒗 𝒊
𝒋=𝟏
|𝑽|
𝒇(𝒗 𝒋
The process of defining vertex probabilities using graph-theoretical
quantities is not unique. Each such quantity captures different
structural information of a graph. In that way, M. Dehmer induced
another functional.
The associated paths of the j-sphere 𝑺𝒋(𝒗𝒊, 𝑮) = {𝒗 𝒖 𝒋
, 𝒗 𝒘 𝒋
, . . . , 𝒗 𝒙 𝒋
are
and their edge sets :
𝐸1 = {{𝑣𝑖, 𝑣 𝑢1
, . . . , {𝑣 𝑢 𝑗−1
, 𝑣 𝑢 𝑗
𝐸2 = {{𝑣𝑖, 𝑣 𝑤1
, . . . , {𝑣 𝑤 𝑗−1
, 𝑣 𝑤 𝑗
⋮
𝐸 𝑘 𝑗
= {{𝑣𝑖, 𝑣 𝑥1
, . . . , {𝑣 𝑥 𝑗−1
, 𝑣 𝑥 𝑗
For each vertex 𝑣𝑖∈V he determined the local information graph
𝐿 𝐺(𝑣𝑖, 𝑗) = (𝑉𝜑, 𝐸 𝜑 which is induced by the paths 𝑃1
𝑗
(𝑣𝑖), . . , 𝑃𝑘 𝑗
𝑗
(𝑣𝑖
where 𝛦 𝜑 = 𝛦1 ∪. . .∪ 𝛦 𝑘 𝑗
and 𝑉𝜑 = {𝑣𝑖, 𝑣 𝑢1
, . . . , 𝑣 𝑢 𝑗
∪. . .∪
Example:
)𝜌(𝐺1 =4 and it is considered that: 𝑐1 = 𝑏1 = 4, 𝑐2 = 𝑏2 = 3,
𝑐3 = 𝑏3 = 2, 𝑐4 = 𝑏4 = 1
𝑓 𝑉(𝑣1) = 𝑓 𝑉(𝑣4) = 𝑓 𝑉(𝑣5) = 𝑓 𝑉(𝑣8) = 𝑎2𝑐1+2𝑐2+2𝑐3+𝑐4 = 𝑎19
𝑓 𝑉
(𝑣2) = 𝑓 𝑉
(𝑣7) = 𝑎2𝑐1+3𝑐2+2𝑐3 = 𝑎21
𝑓 𝑉
(𝑣3) = 𝑓 𝑉
(𝑣6) = 𝑎3𝑐1+3𝑐2+𝑐3 = 𝑎23
𝐼 𝑓 𝑉(𝐺1) = −
𝑖=1
8
)𝑝 𝑣
(𝑣𝑖)𝑙𝑑(𝑝 𝑣
(𝑣𝑖) =
= − 4
𝑎19
4𝑎19 + 2𝑎21 + 2𝑎23
𝑙𝑑 4
𝑎19
4𝑎19 + 2𝑎21 + 2𝑎23
+ 2
𝑎21
4𝑎19 + 2𝑎21 + 2𝑎23
𝑙𝑑
𝑎21
4𝑎19 + 2𝑎21 + 2𝑎23
𝒇 𝑷
(𝒗 𝟏) = 𝒇 𝑷
(𝒗 𝟒) = 𝒇 𝑷
(𝒗 𝟓) = 𝒇 𝑷
(𝒗 𝟖) = 𝒇 𝑷
(𝒗 𝟑) = 𝒇 𝑷
(𝒗 𝟔) = 𝒂 𝟐𝒄 𝟏+𝟒𝒄 𝟐+𝟔𝒄 𝟑+𝟒𝒄 𝟒 = 𝒂 𝟑𝟔
𝒇 𝑷
(𝒗 𝟐) = 𝒇 𝑷
(𝒗 𝟕) = 𝒂 𝟐𝒄 𝟏+𝟔𝒄 𝟐+𝟔𝒄 𝟑 = 𝒂 𝟑𝟖
𝐼 𝑓 𝑃(𝐺1) = −
𝑖=1
8
)𝑝 𝑃
(𝑣𝑖)𝑙𝑑(𝑝 𝑃
(𝑣𝑖) =
= 4
𝑎36
6𝑎36 + 2𝑎38
𝑙𝑑
𝑎36
6𝑎36 + 2𝑎38
The degree distribution network entropy
The entropy of the degree distribution had be defined as 𝑯 =
− 𝒌=𝟏
𝜨−𝟏
)𝒑(𝒌 𝒍𝒅𝒑 𝒌 , where p(k) is the probability that a node has
degree k an N is the number of nodes. The maximum value of
entropy is obtained for a uniform degree distribution and the
minimum value is achieved when all vertices have the same
degree.
Entropy provides an average measure of heterogeneity, since it
measures the diversity of the link distribution. Heterogeneity is in
relationship with the network’s resilience to attacks.
B.Wang, H.Tang, C. Guo and Z. Xiu studied the robustness of scale-
free networks to random failures ,with entropy of the degree
distribution. By maximizing the entropy of the degree distribution,
we get an optimal design of scale-free network’s robustness to
random failures.
Solé and Valverde used entropy of the remaining degree .
The remaining degree of a vertex at one end of an edge is
the number of edges connected to that vertex not counting
the original edge. The remaining degree distribution q(k)
is obtained from:
𝑞 𝑘 =
𝑘+1)𝑃(𝑘+1
<𝑘>
, 𝑤ℎ𝑒𝑟𝑒 < 𝑘 >= 𝑘 𝑘𝑃𝑘 .
Fig. Here two given, connected nodes 𝑠𝑖 , 𝑠𝑗are shown, displaying different degrees 𝑘𝑖 ,
𝑘𝑗. Since we are interested in the remaining degrees, a different value needs to be
considered (here indicated as 𝑞𝑖 , 𝑞𝑗)
Entropy as a measure of centrality in
networks
Frank Tutzauer (2007) has proposed a measure of centrality for
networks characterized by path-based transfer flows.
To model a path-transfer process, it is helpful to think of a specific
object being passed from one node to another. If the starting node
is chosen, the flow either stops, otherwise, the object passes to the
chosen node.
The next node then randomly chooses from among its neighbors
and again the flow either stops or continues. The object thus
traverses a path in the network, traveling along links, stopping
when a loop is chosen. Each neighbor is assumed to be chosen with
equal likelihood.
What is the probability that a flow beginning at
vertex 5 ends at vertex 2, in the network below?
Consequently there is a 1/4 probability that
vertex 5 passes control to vertex 3 (because 5
chooses with equal likelihood from among nodes
3, 4, 5, and 6). Once node 3 receives control, the
flow will not pass back to node 5, so there is a 1/2
probability that node 3 stops the flow, and a 1/2
probability that control passes to node 2.
Likewise, vertex 2 chooses between stopping the
flow or continuing it to vertex 1. As a result, the
probability that a flow beginning at vertex 5 ends
at vertex 2 is (1/4)(1/2)(1/2) = 1/16
The transfer and the stopping probability of a vertex 𝑣 𝑡 ∈ 𝑃𝑘 is
given by:
To obtain the single path probability – i.e., the likelihood that a flow
beginning at 𝑖 = 𝑢0 ends at 𝑗 = 𝑢 𝑛 𝑘 by traveling along the path
𝑃𝑘 = { 𝑢0, . . . , 𝑢 𝑛 𝑘 } – simply multiply by the transfer probability of
each of the first 0, . . ., 𝑛 𝑘–1 vertices and the stopping probability of the
last vertex in the path.
Then the overall probability that a flow starting at i ends at j is given by
the combined path probability, which is simply the single path
probabilities summed across the K(i, j) paths from i to j:
𝑝𝑖𝑗 =
𝑘=1
)𝐾(𝑖,𝑗
𝜎 𝑘(𝑗) 𝑡=0
𝑛(𝑘)−1
𝜏 𝜅 𝑢 𝑡 =
𝑘=1
)𝐾(𝑖,𝑗
𝑡=0
𝑛(𝑘)−1
𝜏 𝜅 𝑢 𝑡
The path-transfer centrality of vertex i is then given by the entropy
𝐶 𝐻(𝑖) = −
𝑗=1
𝑁
𝑝𝑖𝑗 𝑙𝑑𝑝𝑖𝑗
Entropy as a connectivity measure
Entropy as a connectivity measure of a graph is defined as
𝐻 𝐺 = − 𝑣∈𝑉
)𝑝(𝑣)𝑙𝑑𝑝(𝑣 , where 𝑝(𝑣) =
)indeg(𝑣
2|𝐸| 𝑣∈𝑉
To obtain a measure with a [0, 1] range, we divide H(G) by
the maximum entropy given by ld|V|
High entropy indicates that many vertices are equally
important, whereas low entropy indicates that only a few
vertices are relevant.
Topology entropy of network
In order to describe the uncertainty of complex networks, we
proposed another network entropy concept based on the topology
configuration of network. For a complex networks generated
according to certain rules, in the given parameters, each test can
generate a specific network configuration. Repeated tests, a wide
range of configurations will be produced.
The topology entropy of network is defined as 𝑺(𝜴, 𝑷) =
The number of all possible edges is equal to M=3*2/2=3. Here 3
possible connected processes can be treated as 3 independent
random events.
The actual link edge number 𝑚 is a random variable, it will comply
the binomial distribution, 𝑃 𝑚 = 𝐶3
𝑚
0.2 𝑚
1 − 0.2 3−𝑚
, where
𝐶3
𝑚
=
3
𝑚
. The possible configurations are 𝛺(3,0.2) =
𝑚=0
3
𝐶3
𝑚
=8.
All the configurations in these 𝐶3
𝑚
different configurations appear
with probability 1 𝐶3
𝑚
. This is a conditional probability which
links numbers are 𝑚, so the realization of a G(3,0.02) random
network, here 𝑚 edges can be regard as a random incident A𝑖
which has occurrence probability: 𝑃𝑖 = 𝑃(𝑚 ) 𝐶3
𝑚
= 0.2 𝑚(1 −
As the probabilities of each 𝐶3
𝑚
configuration for certain 𝑚 are
the same, the entropy can be calculated for different 𝑚
respectively, so 𝑆 can be written as:
Example
For this network, we compute entropy as a connectivity and centrality measure
and entropy of degree and remaining degree distribution. Then we compare
them with some existing measures.
Centrality
The table above depicts entropy-based centrality, degree centrality,
betweenness centrality and closeness centrality.
Node Entropy Degree Closeness Betweenness
11 3.407 2 (9-13) 0.319 (3) 50 (3-4)
7 3.239 3 (4-8) 0.341 (2) 54 (2)
10 3.211 4 (1-3) 0.349 (1) 57.5 (1)
6 3.131 2 (9-13) 0.313 (4) 50 (3-4)
8 3.061 3 (4-8) 0.306 (5) 3.5 (10)
12 3.003 4(1-3) 0.283 (6) 44 (6)
5 2.944 3 (4-8) 0.278 (7) 47 (5)
14 2.87 4 (1-3) 0.238 (9) 14 (8)
9 2.868 2 (9-13) 0.268 (8) 0 (11-16)
13 2.861 3 (4-8) 0.234 (10-12) 0 (11-16)
15 2.787 3 (4-8) 0.234 (10-12) 0 (11-16)
3 2.7 2 (9-13) 0.234 (10-12) 26 (7)
4 2.421 1 (14-16) 0.221 (13) 0 (11-16)
16 2.343 1 (14-16) 0.195 (15) 0 (11-16)
2 2.308 2 (9-13) 0.197 (14) 14 (8)
1 2.043 1 (14-16) 0.167 (16) 0 (11-16)
The table below portrays Spearman’s rank-order correlation:
If we are interesting about centrality as a score, rather than
simply the ranking, Pearson’s correlation coefficient is more
appropriate. So the table below illustrate the Pearson’s r
correlation:
Entropy, betweenness, and closeness all agree on the top four
nodes, though they rank them differently.
Measures are highly, but not perfectly correlated. Degree
centrality provides the minimum correlation with entropy
based centrality, in comparison with betweenness and
closeness centrality.
In contrast, closeness produces the rankings most similar to
the rankings of the entropy centrality.
Scatter plots and linear regression
Entropy-Closeness
Scatter plots and linear regression
Entropy-Betweenness
Scatter plots and linear regression
Entropy-Degree
Another visualization of the network:
The size of the nodes depends on the value of the entropy.
Connectivity
For the graph with loops the number of edges is: |E|=36. We
obtain H(G)=3,967. To normalize H(G), we divide it by the
maximum entropy given by ld(|V|)=ld(16)=4. So, we obtain
0,991.
If we simplify the loops, the number of edges is: |E|=20. Thus,
H(G)=3,877
Normalize: H(G)/ld(|V|)=H(G)/ld(16)=3,877/4=0,969.
Degree distribution
 Entropy of degree
distribution: H(p)=1,9544
 Entropy of the remaining
degree distribution:
H(q)=1,832
Remainin
g
degree
q(k)
0 0,075
1 0,25
2 0,375
3 0,3
Summary and conclusion
This project has attempted to demonstrate a variety of methods for
measuring the entropy of graphs. We started with a review of classical
measures for determining the structural information content of graphs.
Furthermore, we represented some other approaches characterized by
information functionals. Then we represented topological network
entropy. Moreover, we represented the entropy as centrality and
connectivity measure, the entropy of degree and remaining degree
distribution. Finally, we gave an example of a graph and computed the
relative measures.
Nowadays, graph-based models are applied in a wide range of disciplines.
Shannon’s entropy, as a measure of structural characteristics has been
proven very useful. Further development of that theory and more efficient
algorithms for computing entropy is needed.
References
• N. Rashevsky, Life information theory and topology, Bulletin of Mathematical Biophysics 17 (1955) 229–235.
• E. Trucco, A note on the information content of graphs, Bulletin of Mathematical Biology 18 (2) (1956) 129–135.
• A. Mowshowitz, Entropy and the complexity of the graphs I: an index of the relative complexity of a graph, Bulletin of
Mathematical Biophysics 30 (1968) 175–204.
• A. Mowshowitz, Entropy and the complexity of graphs IV: entropy measures and graphical structure, Bulletin of Mathematical
Biophysics 30 (1968) 533–546.
• Mowshowitz , V. Mitsou, Entropy,Orbits and Spectra of Graphs Analysis of Complex Networks: From Biology to Linguistics,
(2009) Ch. 1.
• M. Dehmer, Information-theoretic concepts for the analysis of complex networks, Applied Artificial Intelligence 22 (7 & 8) (2008)
684–706.
• M. Dehmer, A novel method for measuring the structural information content of networks, Cybernetics and Systems 39 (2008)
825–843.
• M. Dehmer, F. Emmert-Streib, Structural information content of networks: graph entropy based on local vertex functionals,
Computational Biology and Chemistry 32 (2008) 131–138.
• M. Dehmer, Information processing in complex networks:Graph entropy and information functionals, Applied Mathematics and
Computation 201 (2008) 82–94.
• Dehmer Μ., Mowshowitz A., A history of Graph Εntropy measures,Information Sciences 181, 57‐78 (2011)
• F. Tutzauer, Entropy as a measure of centrality in networks characterized by path-transfer flow, Social Networks 29 (2007) 249–
265.
• R.V. Sole, S. Valverde, Information theory of complex networks: on evolution and architectural constraints, in: Lecture Notes in
Physics, (2004), vol. 650, pp. 189–207.
• B. Wang, H. Tang ,_ C. Guo, Z. Xiu, Entropy Optimization of Scale-Free Networks Robustness to Random Failures (2005).
• Li J. ,ea 2008, Network Entropy Based on Topology Configuration and Its Computation to Random Networks, Chin. Phys. Letters
25, 4177-4180
• R. Navigli, M. Lapata, Graph Connectivity Measures for Unsupervised Word Sense Disambiguation
• L. da F. Costa, F. A. Rodrigues, G. Travieso, P. R. Villas Boas, Complex Nets Characterization Measurements Entropy (2008)
Questions?
Thank you for your attention!!!

More Related Content

PDF
Jones matrix formulation by the unitary matrix
PDF
COMPUTING WIENER INDEX OF FIBONACCI WEIGHTED TREES
PDF
Introduction to Graph Theory
PDF
Fixed Point Results for Weakly Compatible Mappings in Convex G-Metric Space
PDF
Generalized fixed point theorems for compatible mapping in fuzzy 2 metric spa...
PPT
Applications of graphs
PDF
Pixel Relationships Examples
PDF
A high accuracy approximation for half - space problems with anisotropic scat...
Jones matrix formulation by the unitary matrix
COMPUTING WIENER INDEX OF FIBONACCI WEIGHTED TREES
Introduction to Graph Theory
Fixed Point Results for Weakly Compatible Mappings in Convex G-Metric Space
Generalized fixed point theorems for compatible mapping in fuzzy 2 metric spa...
Applications of graphs
Pixel Relationships Examples
A high accuracy approximation for half - space problems with anisotropic scat...

What's hot (19)

PDF
Graph theory
PPTX
Graph theory and life
PPTX
Beginning direct3d gameprogrammingmath04_calculus_20160324_jintaeks
PDF
INDEPENDENT AND CONNECTED DOMINATION NUMBER IN MIDDLE AND LINE GRAPHS
PDF
A study on connectivity in graph theory june 18 pdf
PDF
Generalized fixed point theorems for compatible mapping in fuzzy 3 metric spa...
PDF
project
PDF
PDF
Lines and planes in space
PPT
1525 equations of lines in space
PDF
Fixed points theorem on a pair of random generalized non linear contractions
PDF
Clustering in Hilbert geometry for machine learning
PDF
RETOOLING OF COLOR IMAGING IN THE QUATERNION ALGEBRA
PDF
hone_durham
PDF
RachelKnakResearchPoster
PDF
Clustering in Hilbert simplex geometry
PPTX
Introduction to graph theory (All chapter)
PDF
The Generalized Difference Operator of the 퐧 퐭퐡 Kind
Graph theory
Graph theory and life
Beginning direct3d gameprogrammingmath04_calculus_20160324_jintaeks
INDEPENDENT AND CONNECTED DOMINATION NUMBER IN MIDDLE AND LINE GRAPHS
A study on connectivity in graph theory june 18 pdf
Generalized fixed point theorems for compatible mapping in fuzzy 3 metric spa...
project
Lines and planes in space
1525 equations of lines in space
Fixed points theorem on a pair of random generalized non linear contractions
Clustering in Hilbert geometry for machine learning
RETOOLING OF COLOR IMAGING IN THE QUATERNION ALGEBRA
hone_durham
RachelKnakResearchPoster
Clustering in Hilbert simplex geometry
Introduction to graph theory (All chapter)
The Generalized Difference Operator of the 퐧 퐭퐡 Kind
Ad

Viewers also liked (7)

PDF
Topological algebra for wireless networks
PPTX
Statistical classification: A review on some techniques
PPTX
Classification of data
PPT
M H C
PDF
Role of computers in research
PPSX
Research Methodology Lecture for Master & Phd Students
PPT
Major Histocompatibility Complex
Topological algebra for wireless networks
Statistical classification: A review on some techniques
Classification of data
M H C
Role of computers in research
Research Methodology Lecture for Master & Phd Students
Major Histocompatibility Complex
Ad

Similar to Entropy based measures for graphs (20)

PDF
On algorithmic problems concerning graphs of higher degree of symmetry
PDF
ON ALGORITHMIC PROBLEMS CONCERNING GRAPHS OF HIGHER DEGREE OF SYMMETRY
PDF
A Quest for Subexponential Time Parameterized Algorithms for Planar-k-Path: F...
PDF
A PROBABILISTIC ALGORITHM FOR COMPUTATION OF POLYNOMIAL GREATEST COMMON WITH ...
PDF
A Probabilistic Algorithm for Computation of Polynomial Greatest Common with ...
PDF
A PROBABILISTIC ALGORITHM FOR COMPUTATION OF POLYNOMIAL GREATEST COMMON WITH ...
PDF
ENUMERATION OF 1-REGULAR SEMIGRAPHS OF ORDER p WITH THREE MUTUALLY ADJACENT m...
PDF
EVEN GRACEFUL LABELLING OF A CLASS OF TREES
PDF
The Optimization of Weight Functions of R3,1 Space Time by the Echo State Net...
PDF
4. [22 25]characterization of connected vertex magic total labeling graphs in...
PDF
4. [22 25]characterization of connected vertex magic total labeling graphs in...
PDF
4. [22 25]characterization of connected vertex magic total labeling graphs in...
PDF
11.characterization of connected vertex magic total labeling graphs in topolo...
PDF
Dual Spaces of Generalized Cesaro Sequence Space and Related Matrix Mapping
PDF
Steven Duplij, "Polyadic Hopf algebras and quantum groups"
PDF
Homometric Number of Graphs
PDF
Fuzzy algebra
On algorithmic problems concerning graphs of higher degree of symmetry
ON ALGORITHMIC PROBLEMS CONCERNING GRAPHS OF HIGHER DEGREE OF SYMMETRY
A Quest for Subexponential Time Parameterized Algorithms for Planar-k-Path: F...
A PROBABILISTIC ALGORITHM FOR COMPUTATION OF POLYNOMIAL GREATEST COMMON WITH ...
A Probabilistic Algorithm for Computation of Polynomial Greatest Common with ...
A PROBABILISTIC ALGORITHM FOR COMPUTATION OF POLYNOMIAL GREATEST COMMON WITH ...
ENUMERATION OF 1-REGULAR SEMIGRAPHS OF ORDER p WITH THREE MUTUALLY ADJACENT m...
EVEN GRACEFUL LABELLING OF A CLASS OF TREES
The Optimization of Weight Functions of R3,1 Space Time by the Echo State Net...
4. [22 25]characterization of connected vertex magic total labeling graphs in...
4. [22 25]characterization of connected vertex magic total labeling graphs in...
4. [22 25]characterization of connected vertex magic total labeling graphs in...
11.characterization of connected vertex magic total labeling graphs in topolo...
Dual Spaces of Generalized Cesaro Sequence Space and Related Matrix Mapping
Steven Duplij, "Polyadic Hopf algebras and quantum groups"
Homometric Number of Graphs
Fuzzy algebra

Recently uploaded (20)

PDF
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
PDF
Unveiling a 36 billion solar mass black hole at the centre of the Cosmic Hors...
PDF
AlphaEarth Foundations and the Satellite Embedding dataset
PPTX
G5Q1W8 PPT SCIENCE.pptx 2025-2026 GRADE 5
PDF
. Radiology Case Scenariosssssssssssssss
PPTX
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
PPTX
Introduction to Fisheries Biotechnology_Lesson 1.pptx
PPTX
INTRODUCTION TO EVS | Concept of sustainability
PPTX
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
PPTX
famous lake in india and its disturibution and importance
PDF
CAPERS-LRD-z9:AGas-enshroudedLittleRedDotHostingaBroad-lineActive GalacticNuc...
PPT
The World of Physical Science, • Labs: Safety Simulation, Measurement Practice
PDF
Biophysics 2.pdffffffffffffffffffffffffff
PPTX
The KM-GBF monitoring framework – status & key messages.pptx
PPTX
Classification Systems_TAXONOMY_SCIENCE8.pptx
PPTX
ANEMIA WITH LEUKOPENIA MDS 07_25.pptx htggtftgt fredrctvg
PDF
Phytochemical Investigation of Miliusa longipes.pdf
PPTX
Microbiology with diagram medical studies .pptx
PPTX
2. Earth - The Living Planet Module 2ELS
PPTX
Cell Membrane: Structure, Composition & Functions
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
Unveiling a 36 billion solar mass black hole at the centre of the Cosmic Hors...
AlphaEarth Foundations and the Satellite Embedding dataset
G5Q1W8 PPT SCIENCE.pptx 2025-2026 GRADE 5
. Radiology Case Scenariosssssssssssssss
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
Introduction to Fisheries Biotechnology_Lesson 1.pptx
INTRODUCTION TO EVS | Concept of sustainability
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
famous lake in india and its disturibution and importance
CAPERS-LRD-z9:AGas-enshroudedLittleRedDotHostingaBroad-lineActive GalacticNuc...
The World of Physical Science, • Labs: Safety Simulation, Measurement Practice
Biophysics 2.pdffffffffffffffffffffffffff
The KM-GBF monitoring framework – status & key messages.pptx
Classification Systems_TAXONOMY_SCIENCE8.pptx
ANEMIA WITH LEUKOPENIA MDS 07_25.pptx htggtftgt fredrctvg
Phytochemical Investigation of Miliusa longipes.pdf
Microbiology with diagram medical studies .pptx
2. Earth - The Living Planet Module 2ELS
Cell Membrane: Structure, Composition & Functions

Entropy based measures for graphs

  • 1. Entropy based measures for graphs Master in Web Science, Department of Mathematics, Aristotle University of Thessaloniki Giorgos Bamparopoulos, Nikos Papathanasiou
  • 2. Introduction In many scientific areas, e.g. sociology, psychology, physics, chemistry, biology etc., systems can be described as interaction networks where the interactions correspond to edges and the elements to vertices. A variety of problems in those fields can deal with network comparison and characterization. The problem of comparing networks is the task of measuring their structural similarity. Moreover, “characterize networks” means that we are searching for network characteristics which capture structural information of networks. To analyze complex networks, several methods can be combined, such as graph theory, information theory, and statistics. In this project, in order to characterize and compare network structures, we describe methods for measuring Shannon’s entropy of graphs. To introduce entropy based measures for graphs, we first mention some basic graph-theoretical preliminaries:
  • 3. Preliminaries A graph is a non-empty finite set V of elements called vertices together with a possibly empty set E of pairs of vertices called edges. It is denoted G=(V,E) The degree of a vertex u is the number of links plus twice the number of loops incident with u. It is denoted d(u). The degree distribution is the relative frequency of vertices with different degrees. (P(k) = fraction of vertices with degree k) The j-sphere of a vertex vi in G is defined by the set Sj(vi, G): =
  • 4. Preliminaries A path of length n , n ≥1 , from vertex a to vertex b is a sequence of directed edges e1, e2, ...,en such that the initial vertex of e1 is a , and the terminal vertex of en is b, and for i=2, ..., n , the initial vertex of ei is the terminal vertex of ei-1 . When it is more convenient, vertices can be listed in order (rather than the edges) to represent the path. If the first and last nodes of a path coincide it is called cycle. d(u,v) denotes the distance between u∈V and v ∈ V expressed as the minimum length of a path between u,v.
  • 5. Preliminaries A coloring of a graph G(V,E) is an assignment of colors to the vertices such that no two adjacent vertices have the same color  An n-coloring of a graph G = (V, E) is a coloring with n colors. More precisely, a mapping f of V onto the set {1, 2, . . . , n} such that whenever [u, v]∈E, f(u)≠ f(v). The chromatic number X(G) of a graph G is the minimum number of colors needed to color the vertices of G so that no two adjacent vertices have the same color. An n-coloring is complete if, ∀ i, j with i≠j, there exist adjacent vertices u and v such that f(u) = i and f(v) = j. The sets of vertices 𝑉𝑖 𝑖=1 𝑛 with the same color is called color classes.
  • 6. Preliminaries We call the quantity σ(v)= )max 𝑢𝑒𝑣 𝑑(𝑢, 𝑣 the eccentricity of v∈V. The diameter ρ of a connected graph G is the maximum eccentricity among all nodes of G.  For a given path P = {u0, u1, . . . , un}, a downstream edge of a vertex ut ∈ P is any edge (ut,w) ∈E with w≠us for s < t. A downstream vertex of ut is any vertex w such that (ut ,w) is a downstream edge. The downstream degree, D(ut), of a vertex is the number of downstream edges it has.
  • 7. Basic concepts Information content of graphs Entropy of degree distribution Entropy as a measure of centrality Entropy as a measure of connectivity Topology entropy of a network
  • 8. Information content of graphs Rashevsky, Trucco and Mowshowitz were the first researchers to define and investigate the Shannon-entropy of graphs. Rashevsky in 1955, defined the topological information content of the graph, dealing with the complexity of organic molecules, where vertices represent physically indistinguishable atoms and edges represent chemical bonds. His definition is based on the partitioning of the n vertices of a given graph into k classes of equivalent vertices, according to their degree dependence. Then, assigned a probability to each partition obtained as the fraction of vertices in this partition divided by the total number of vertices.
  • 9. Examples: Graph has 3 vertices of degree two (1, 3, and 4), and two vertices (2 and 5) of degree three. However, vertex 1 is topologically different from vertices 3 and 4. Vertex 1 is adjacent to two vertices of degree three, while each of the vertices 3 and 4 is adjacent to one vertex of degree three and one of degree two. Hence we have classes of vertices, 1 with probability 1 5 , 2 and 5 with probability 2 5 , 3 and 4 with probability 2 5 . The information content of is: By adding the vertex 6, vertices 2 and 5 have different degrees. Moreover vertex 3 is adjacent to one vertex of degree four and to one of degree two, while the vertex 4 is adjacent to one vertex of degree three and one of degree two. So, we have six distinct classes each of them have probability 1 6 . Hence, the information content of is: 𝑰 𝒄 = −( 𝟏 𝟓 𝒍𝒅 𝟏 𝟓 + 𝟐 𝟓 𝒍𝒅 𝟐 𝟓 + 𝟐 𝟓 𝒍𝒅 𝟐 𝟓 ) = 𝟏 𝟓 𝒍𝒅𝟓 + 𝟐 𝟓 𝒍𝒅 𝟓 𝟐 + 𝟐 𝟓 𝒍𝒅 𝟓 𝟐 = 𝟏. 𝟓𝟑 𝑰 𝒅 = 𝒍𝒅𝟔 = 𝟐. 𝟔
  • 10. Trucco in 1956 made this definition more precision. He considered that two vertices are equivalent if they belong to the same orbit of this group, i. e., if they can interchange preserving the adjacency of the graph. We denote the group of all automorphisms of a graph X by G(X). X is a graph with 𝑉(𝑋) = {𝑥1, . . . 𝑥 𝑛 . If f∈G(X), 𝑓(𝑥𝑖) = 𝑥𝑗 for 1 < i < n. Hence, to each f∈G(X), there corresponds a unique permutation of the elements 1,2,...,n. We can regard G(X) as a subgroup of 𝑆𝑛, the symmetric group of degree n. Suppose K is a subgroup of 𝑆𝑛 and 1 ≤ 𝑖 ≤ 𝑛. Then 𝑓(𝑖 ) 𝑓 ∈ 𝐾 is called an orbit of K. Let 𝐴𝑖, 1 ≤ 𝑖 ≤ 𝑛 be the distinct orbits of K. Then 𝐴𝑖 ∩ 𝐴𝑗 = ∅ , if i≠j and ∪ 𝑖=1 𝑛 𝐴𝑖 = 1, . . . , 𝑛 , hence the orbits form a partition of the set {1, 2 , . . . , n}. We assigned probability 𝑝𝑖 = |𝐴 𝑖| 𝑛 to each orbit, where |𝐴𝑖| is the cardinality of 𝐴𝑖.
  • 11. Examples: )𝐺(𝑋1 ={e, (12)}, Orbits of )𝐺(𝑋1 : {1, 2}, {3},{4}. Hence, )𝐺(𝑋2 ={e, (23), (24), (34), (234), (243)}, Orbits of )𝐺(𝑋2 : {2,3,4},{1} Hence, 𝑰 𝒈(𝑿 𝟏) = −( 𝟐 𝟒 𝒍𝒅 𝟑 𝟒 + 𝟏 𝟒 𝒍𝒅 𝟏 𝟒 + 𝟏 𝟒 𝒍𝒅 𝟏 𝟒 ) = 𝟑 𝟐 𝑰 𝒈(𝑿 𝟐) = −( 𝟑 𝟒 𝒍𝒅 𝟑 𝟒 + 𝟏 𝟒 𝒍𝒅 𝟏 𝟒 ) = 𝟐 − 𝟑 𝟒 𝒍𝒅𝟑
  • 12. )𝐺(𝑋3 ={e, (13), (24), (13) (24)}. Orbits of )𝐺(𝑋3 : {1, 3},{2, 4}. Hence, G(𝑋4)= {(1234), (13), (24), (13) (24), (12) (34), (14) (23) (1432)}. Orbits of G(𝑋4): {1,2,3,4} • 𝑰 𝒈(𝑿 𝟑) = −( 𝟐 𝟒 𝒍𝒅 𝟐 𝟒 + 𝟐 𝟒 𝒍𝒅 𝟐 𝟒 ) = −𝒍𝒅 𝟐 𝟒 = 𝒍𝒅𝟒 − 𝒍𝒅𝟐 = 𝟏 𝑰 𝒈(𝑿 𝟒) = −( 𝟒 𝟒 𝒍𝒅 𝟒 𝟒 ) = −𝒍𝒅𝟏 = 𝟎
  • 13. In 1968 Mowshovitz has studied in detail the information content of the graphs and has formulated, on the basis of the chromatic properties of the graphs.  A decomposition 𝑉𝑖 𝑖=1 𝑛 of the set of vertices V is called a chromatic decomposition of G if u, v ∈ Vi imply that [u, v] ∉ E. If f is an n-coloring, the collection of sets { 𝑣 ∈
  • 14. For the graph G(V,E), |𝑉| = 6, X(G)=3 Chromatic decomposition is : A:{1,3}, {2,5}, {4,6} B: {2}, {1,3,} {4,5,6} C:{2}, {3}, {1,4,5,6}
  • 15. Information content base on functionals M. Dehmer and F. Emmert-Streib (2007) presented a method to determine the structural information content which is not based on the problem of finding partitions of the vertices, and the overall time complexity is polynomial. They assigned a probability to each vertex 𝒗𝒊 as, where f represents an arbitrary information functional. In this case 𝒇 𝒗𝒊 = 𝒂 𝒄 𝟏|𝒔 𝟏 𝒗 𝒊,𝑮 |+...+𝒄 𝝆|𝒔 𝝆 𝒗 𝒊,𝑮 | , 1 ≤ k ≤ ρ, α >0, 𝑐 𝑘 are real positive coefficients and they are chosen in such a way to emphasize certain structural characteristics e.g. high vertex degrees. 𝒑(𝒗𝒊) = )𝒇(𝒗 𝒊 𝒋=𝟏 |𝑽| 𝒇(𝒗 𝒋
  • 16. The process of defining vertex probabilities using graph-theoretical quantities is not unique. Each such quantity captures different structural information of a graph. In that way, M. Dehmer induced another functional. The associated paths of the j-sphere 𝑺𝒋(𝒗𝒊, 𝑮) = {𝒗 𝒖 𝒋 , 𝒗 𝒘 𝒋 , . . . , 𝒗 𝒙 𝒋 are and their edge sets : 𝐸1 = {{𝑣𝑖, 𝑣 𝑢1 , . . . , {𝑣 𝑢 𝑗−1 , 𝑣 𝑢 𝑗 𝐸2 = {{𝑣𝑖, 𝑣 𝑤1 , . . . , {𝑣 𝑤 𝑗−1 , 𝑣 𝑤 𝑗 ⋮ 𝐸 𝑘 𝑗 = {{𝑣𝑖, 𝑣 𝑥1 , . . . , {𝑣 𝑥 𝑗−1 , 𝑣 𝑥 𝑗
  • 17. For each vertex 𝑣𝑖∈V he determined the local information graph 𝐿 𝐺(𝑣𝑖, 𝑗) = (𝑉𝜑, 𝐸 𝜑 which is induced by the paths 𝑃1 𝑗 (𝑣𝑖), . . , 𝑃𝑘 𝑗 𝑗 (𝑣𝑖 where 𝛦 𝜑 = 𝛦1 ∪. . .∪ 𝛦 𝑘 𝑗 and 𝑉𝜑 = {𝑣𝑖, 𝑣 𝑢1 , . . . , 𝑣 𝑢 𝑗 ∪. . .∪
  • 18. Example: )𝜌(𝐺1 =4 and it is considered that: 𝑐1 = 𝑏1 = 4, 𝑐2 = 𝑏2 = 3, 𝑐3 = 𝑏3 = 2, 𝑐4 = 𝑏4 = 1 𝑓 𝑉(𝑣1) = 𝑓 𝑉(𝑣4) = 𝑓 𝑉(𝑣5) = 𝑓 𝑉(𝑣8) = 𝑎2𝑐1+2𝑐2+2𝑐3+𝑐4 = 𝑎19 𝑓 𝑉 (𝑣2) = 𝑓 𝑉 (𝑣7) = 𝑎2𝑐1+3𝑐2+2𝑐3 = 𝑎21 𝑓 𝑉 (𝑣3) = 𝑓 𝑉 (𝑣6) = 𝑎3𝑐1+3𝑐2+𝑐3 = 𝑎23 𝐼 𝑓 𝑉(𝐺1) = − 𝑖=1 8 )𝑝 𝑣 (𝑣𝑖)𝑙𝑑(𝑝 𝑣 (𝑣𝑖) = = − 4 𝑎19 4𝑎19 + 2𝑎21 + 2𝑎23 𝑙𝑑 4 𝑎19 4𝑎19 + 2𝑎21 + 2𝑎23 + 2 𝑎21 4𝑎19 + 2𝑎21 + 2𝑎23 𝑙𝑑 𝑎21 4𝑎19 + 2𝑎21 + 2𝑎23
  • 19. 𝒇 𝑷 (𝒗 𝟏) = 𝒇 𝑷 (𝒗 𝟒) = 𝒇 𝑷 (𝒗 𝟓) = 𝒇 𝑷 (𝒗 𝟖) = 𝒇 𝑷 (𝒗 𝟑) = 𝒇 𝑷 (𝒗 𝟔) = 𝒂 𝟐𝒄 𝟏+𝟒𝒄 𝟐+𝟔𝒄 𝟑+𝟒𝒄 𝟒 = 𝒂 𝟑𝟔 𝒇 𝑷 (𝒗 𝟐) = 𝒇 𝑷 (𝒗 𝟕) = 𝒂 𝟐𝒄 𝟏+𝟔𝒄 𝟐+𝟔𝒄 𝟑 = 𝒂 𝟑𝟖 𝐼 𝑓 𝑃(𝐺1) = − 𝑖=1 8 )𝑝 𝑃 (𝑣𝑖)𝑙𝑑(𝑝 𝑃 (𝑣𝑖) = = 4 𝑎36 6𝑎36 + 2𝑎38 𝑙𝑑 𝑎36 6𝑎36 + 2𝑎38
  • 20. The degree distribution network entropy The entropy of the degree distribution had be defined as 𝑯 = − 𝒌=𝟏 𝜨−𝟏 )𝒑(𝒌 𝒍𝒅𝒑 𝒌 , where p(k) is the probability that a node has degree k an N is the number of nodes. The maximum value of entropy is obtained for a uniform degree distribution and the minimum value is achieved when all vertices have the same degree. Entropy provides an average measure of heterogeneity, since it measures the diversity of the link distribution. Heterogeneity is in relationship with the network’s resilience to attacks. B.Wang, H.Tang, C. Guo and Z. Xiu studied the robustness of scale- free networks to random failures ,with entropy of the degree distribution. By maximizing the entropy of the degree distribution, we get an optimal design of scale-free network’s robustness to random failures.
  • 21. Solé and Valverde used entropy of the remaining degree . The remaining degree of a vertex at one end of an edge is the number of edges connected to that vertex not counting the original edge. The remaining degree distribution q(k) is obtained from: 𝑞 𝑘 = 𝑘+1)𝑃(𝑘+1 <𝑘> , 𝑤ℎ𝑒𝑟𝑒 < 𝑘 >= 𝑘 𝑘𝑃𝑘 . Fig. Here two given, connected nodes 𝑠𝑖 , 𝑠𝑗are shown, displaying different degrees 𝑘𝑖 , 𝑘𝑗. Since we are interested in the remaining degrees, a different value needs to be considered (here indicated as 𝑞𝑖 , 𝑞𝑗)
  • 22. Entropy as a measure of centrality in networks Frank Tutzauer (2007) has proposed a measure of centrality for networks characterized by path-based transfer flows. To model a path-transfer process, it is helpful to think of a specific object being passed from one node to another. If the starting node is chosen, the flow either stops, otherwise, the object passes to the chosen node. The next node then randomly chooses from among its neighbors and again the flow either stops or continues. The object thus traverses a path in the network, traveling along links, stopping when a loop is chosen. Each neighbor is assumed to be chosen with equal likelihood.
  • 23. What is the probability that a flow beginning at vertex 5 ends at vertex 2, in the network below? Consequently there is a 1/4 probability that vertex 5 passes control to vertex 3 (because 5 chooses with equal likelihood from among nodes 3, 4, 5, and 6). Once node 3 receives control, the flow will not pass back to node 5, so there is a 1/2 probability that node 3 stops the flow, and a 1/2 probability that control passes to node 2. Likewise, vertex 2 chooses between stopping the flow or continuing it to vertex 1. As a result, the probability that a flow beginning at vertex 5 ends at vertex 2 is (1/4)(1/2)(1/2) = 1/16 The transfer and the stopping probability of a vertex 𝑣 𝑡 ∈ 𝑃𝑘 is given by:
  • 24. To obtain the single path probability – i.e., the likelihood that a flow beginning at 𝑖 = 𝑢0 ends at 𝑗 = 𝑢 𝑛 𝑘 by traveling along the path 𝑃𝑘 = { 𝑢0, . . . , 𝑢 𝑛 𝑘 } – simply multiply by the transfer probability of each of the first 0, . . ., 𝑛 𝑘–1 vertices and the stopping probability of the last vertex in the path. Then the overall probability that a flow starting at i ends at j is given by the combined path probability, which is simply the single path probabilities summed across the K(i, j) paths from i to j: 𝑝𝑖𝑗 = 𝑘=1 )𝐾(𝑖,𝑗 𝜎 𝑘(𝑗) 𝑡=0 𝑛(𝑘)−1 𝜏 𝜅 𝑢 𝑡 = 𝑘=1 )𝐾(𝑖,𝑗 𝑡=0 𝑛(𝑘)−1 𝜏 𝜅 𝑢 𝑡 The path-transfer centrality of vertex i is then given by the entropy 𝐶 𝐻(𝑖) = − 𝑗=1 𝑁 𝑝𝑖𝑗 𝑙𝑑𝑝𝑖𝑗
  • 25. Entropy as a connectivity measure Entropy as a connectivity measure of a graph is defined as 𝐻 𝐺 = − 𝑣∈𝑉 )𝑝(𝑣)𝑙𝑑𝑝(𝑣 , where 𝑝(𝑣) = )indeg(𝑣 2|𝐸| 𝑣∈𝑉 To obtain a measure with a [0, 1] range, we divide H(G) by the maximum entropy given by ld|V| High entropy indicates that many vertices are equally important, whereas low entropy indicates that only a few vertices are relevant.
  • 26. Topology entropy of network In order to describe the uncertainty of complex networks, we proposed another network entropy concept based on the topology configuration of network. For a complex networks generated according to certain rules, in the given parameters, each test can generate a specific network configuration. Repeated tests, a wide range of configurations will be produced. The topology entropy of network is defined as 𝑺(𝜴, 𝑷) =
  • 27. The number of all possible edges is equal to M=3*2/2=3. Here 3 possible connected processes can be treated as 3 independent random events. The actual link edge number 𝑚 is a random variable, it will comply the binomial distribution, 𝑃 𝑚 = 𝐶3 𝑚 0.2 𝑚 1 − 0.2 3−𝑚 , where 𝐶3 𝑚 = 3 𝑚 . The possible configurations are 𝛺(3,0.2) = 𝑚=0 3 𝐶3 𝑚 =8. All the configurations in these 𝐶3 𝑚 different configurations appear with probability 1 𝐶3 𝑚 . This is a conditional probability which links numbers are 𝑚, so the realization of a G(3,0.02) random network, here 𝑚 edges can be regard as a random incident A𝑖 which has occurrence probability: 𝑃𝑖 = 𝑃(𝑚 ) 𝐶3 𝑚 = 0.2 𝑚(1 −
  • 28. As the probabilities of each 𝐶3 𝑚 configuration for certain 𝑚 are the same, the entropy can be calculated for different 𝑚 respectively, so 𝑆 can be written as:
  • 29. Example For this network, we compute entropy as a connectivity and centrality measure and entropy of degree and remaining degree distribution. Then we compare them with some existing measures.
  • 30. Centrality The table above depicts entropy-based centrality, degree centrality, betweenness centrality and closeness centrality. Node Entropy Degree Closeness Betweenness 11 3.407 2 (9-13) 0.319 (3) 50 (3-4) 7 3.239 3 (4-8) 0.341 (2) 54 (2) 10 3.211 4 (1-3) 0.349 (1) 57.5 (1) 6 3.131 2 (9-13) 0.313 (4) 50 (3-4) 8 3.061 3 (4-8) 0.306 (5) 3.5 (10) 12 3.003 4(1-3) 0.283 (6) 44 (6) 5 2.944 3 (4-8) 0.278 (7) 47 (5) 14 2.87 4 (1-3) 0.238 (9) 14 (8) 9 2.868 2 (9-13) 0.268 (8) 0 (11-16) 13 2.861 3 (4-8) 0.234 (10-12) 0 (11-16) 15 2.787 3 (4-8) 0.234 (10-12) 0 (11-16) 3 2.7 2 (9-13) 0.234 (10-12) 26 (7) 4 2.421 1 (14-16) 0.221 (13) 0 (11-16) 16 2.343 1 (14-16) 0.195 (15) 0 (11-16) 2 2.308 2 (9-13) 0.197 (14) 14 (8) 1 2.043 1 (14-16) 0.167 (16) 0 (11-16)
  • 31. The table below portrays Spearman’s rank-order correlation: If we are interesting about centrality as a score, rather than simply the ranking, Pearson’s correlation coefficient is more appropriate. So the table below illustrate the Pearson’s r correlation:
  • 32. Entropy, betweenness, and closeness all agree on the top four nodes, though they rank them differently. Measures are highly, but not perfectly correlated. Degree centrality provides the minimum correlation with entropy based centrality, in comparison with betweenness and closeness centrality. In contrast, closeness produces the rankings most similar to the rankings of the entropy centrality.
  • 33. Scatter plots and linear regression Entropy-Closeness
  • 34. Scatter plots and linear regression Entropy-Betweenness
  • 35. Scatter plots and linear regression Entropy-Degree
  • 36. Another visualization of the network: The size of the nodes depends on the value of the entropy.
  • 37. Connectivity For the graph with loops the number of edges is: |E|=36. We obtain H(G)=3,967. To normalize H(G), we divide it by the maximum entropy given by ld(|V|)=ld(16)=4. So, we obtain 0,991. If we simplify the loops, the number of edges is: |E|=20. Thus, H(G)=3,877 Normalize: H(G)/ld(|V|)=H(G)/ld(16)=3,877/4=0,969.
  • 38. Degree distribution  Entropy of degree distribution: H(p)=1,9544  Entropy of the remaining degree distribution: H(q)=1,832 Remainin g degree q(k) 0 0,075 1 0,25 2 0,375 3 0,3
  • 39. Summary and conclusion This project has attempted to demonstrate a variety of methods for measuring the entropy of graphs. We started with a review of classical measures for determining the structural information content of graphs. Furthermore, we represented some other approaches characterized by information functionals. Then we represented topological network entropy. Moreover, we represented the entropy as centrality and connectivity measure, the entropy of degree and remaining degree distribution. Finally, we gave an example of a graph and computed the relative measures. Nowadays, graph-based models are applied in a wide range of disciplines. Shannon’s entropy, as a measure of structural characteristics has been proven very useful. Further development of that theory and more efficient algorithms for computing entropy is needed.
  • 40. References • N. Rashevsky, Life information theory and topology, Bulletin of Mathematical Biophysics 17 (1955) 229–235. • E. Trucco, A note on the information content of graphs, Bulletin of Mathematical Biology 18 (2) (1956) 129–135. • A. Mowshowitz, Entropy and the complexity of the graphs I: an index of the relative complexity of a graph, Bulletin of Mathematical Biophysics 30 (1968) 175–204. • A. Mowshowitz, Entropy and the complexity of graphs IV: entropy measures and graphical structure, Bulletin of Mathematical Biophysics 30 (1968) 533–546. • Mowshowitz , V. Mitsou, Entropy,Orbits and Spectra of Graphs Analysis of Complex Networks: From Biology to Linguistics, (2009) Ch. 1. • M. Dehmer, Information-theoretic concepts for the analysis of complex networks, Applied Artificial Intelligence 22 (7 & 8) (2008) 684–706. • M. Dehmer, A novel method for measuring the structural information content of networks, Cybernetics and Systems 39 (2008) 825–843. • M. Dehmer, F. Emmert-Streib, Structural information content of networks: graph entropy based on local vertex functionals, Computational Biology and Chemistry 32 (2008) 131–138. • M. Dehmer, Information processing in complex networks:Graph entropy and information functionals, Applied Mathematics and Computation 201 (2008) 82–94. • Dehmer Μ., Mowshowitz A., A history of Graph Εntropy measures,Information Sciences 181, 57‐78 (2011) • F. Tutzauer, Entropy as a measure of centrality in networks characterized by path-transfer flow, Social Networks 29 (2007) 249– 265. • R.V. Sole, S. Valverde, Information theory of complex networks: on evolution and architectural constraints, in: Lecture Notes in Physics, (2004), vol. 650, pp. 189–207. • B. Wang, H. Tang ,_ C. Guo, Z. Xiu, Entropy Optimization of Scale-Free Networks Robustness to Random Failures (2005). • Li J. ,ea 2008, Network Entropy Based on Topology Configuration and Its Computation to Random Networks, Chin. Phys. Letters 25, 4177-4180 • R. Navigli, M. Lapata, Graph Connectivity Measures for Unsupervised Word Sense Disambiguation • L. da F. Costa, F. A. Rodrigues, G. Travieso, P. R. Villas Boas, Complex Nets Characterization Measurements Entropy (2008)
  • 41. Questions? Thank you for your attention!!!