Noisy information transmission through molecular interaction networks

Noisy information transmission through molecular
interaction networks

Michael P.H. Stumpf

Theoretical Systems Biology Group

09/05/2012

Noise in Biological Networks Michael P.H. Stumpf 1 of 24

Cellular Decision Making Processes

Noise in Biological Networks Michael P.H. Stumpf Cellular Decision Making Processes 2 of 24

What Do We Mean by a Cellular Decision?
Cell Fates and Phenotypes
• How are decisions made?
• How non-random are these
decisions?
• Why and how do cellular decision
Differentiation Proliferation Apoptosis making processes diverge?


What Do We Mean by a Cellular Decision?
Cell Fates and Phenotypes
• How are decisions made?
• How non-random are these
decisions?
• Why and how do cellular decision
Differentiation Proliferation Apoptosis making processes diverge?

Information Processing
EGF

EGFR Akt S6

How reliable is the processing and transmission of information from
the cell’s environment into the nucleus?


Signal Transduction vs. Signal Processing

X X X

Y Y

Z Z Z

Noise in Biological Networks Michael P.H. Stumpf Information Transmission 4 of 24

Signal: S = {S1 , S2 , . . .}

X X X

Y Y

Z Z Z


Cellular Information
Signal: S = {S1 , S2 , . . .}
Processing
X X X We assume that X is a
receptor and Z an effector (e.g.
a transcription factor).
We are interested in how
Y Y
• Z (t ) reﬂects the stimulus
pattern Si (t ).
Z Z Z • the molecular architecture
Mj modulates the signal.


Cellular Information
Signal: S = {S1 , S2 , . . .}
Processing
X X X We assume that X is a
receptor and Z an effector (e.g.
a transcription factor).
We are interested in how
Y Y
• Z (t ) reﬂects the stimulus
pattern Si (t ).
Z Z Z • the molecular architecture
Mj modulates the signal.

S (t )

Z (t ) Z (t )

t

t t


Information Theory
Notions of Information (according to C. Shannon)
• Information is closely associated with uncertainty.
• What is signiﬁcant is the difﬁculty in transmitting the message from
one point to another.
• Information in entropy: H (X ) = − p(x ) log (p(x ))


Information Theory
Notions of Information (according to C. Shannon)
• Information is closely associated with uncertainty.
• What is signiﬁcant is the difﬁculty in transmitting the message from
one point to another.
• Information in entropy: H (X ) = − p(x ) log (p(x ))

H (X , Y )

H (X |Y ) I (X , Y ) H (Y |X )

H (X ) H (Y )

Facets of Information
Gaussian Channels

Transmitter Noisy Channel Receiver

If X is the message emanating from the transmitter and Y is the
message arriving at the receiver we model the channel as

Pr(Y = y |X = x ) ∼ N(x , σ2 ).
C


Facets of Information
Gaussian Channels

Transmitter Noisy Channel Receiver

If X is the message emanating from the transmitter and Y is the
message arriving at the receiver we model the channel as

Pr(Y = y |X = x ) ∼ N(x , σ2 ).
C

The mutual information I (Y , X ) captures the level of reduction in
uncertainty about the state of Y once X is known.
The channel capacity is the maximum information that can ﬂow
through the channel,
C = max I (Y , X ).
p (x )
This is a variational problem but straightforwardly solvable for
Gaussian and some other simple information channels.

Information Transmission
The Mutual Information between the signal, S, and
the effector, Z , is given by

Pr(z , s)
I (Z , S ) = Pr(z , s)
S Pr(z )Pr(s)
s∈S,z ∈Z

= H (Z ) − H (Z |S ).
X
High mutual information means that Z faithfully
reproduces the signal S (as taken up by X ).

Z


Information Transmission
The Mutual Information between the signal, S, and
the effector, Z , is given by

Pr(z , s)
I (Z , S ) = Pr(z , s)
S Pr(z )Pr(s) S
s∈S,z ∈Z

= H (Z ) − H (Z |S ).
X X
High mutual information means that Z faithfully
reproduces the signal S (as taken up by X ).
Y
Signal Processing
Ultimately, however, the signal is processed and
altered in a way not normally considered in e.g. the
Z Z
analysis of “Gaussian Channels”.
If we transduce a signal through a signalling
network/pathway it will get processed and distorted
by intrinsic and extrinsic noise.

Extrinsic vs. Intrinsic Noise

S

X
Extrinsic
Intrinsic
Extrinsic
+ Intrinsic

Z


Extrinsic vs. Intrinsic Noise

S

X
Extrinsic
Intrinsic
Extrinsic
Y
+ Intrinsic

Z


Some Strange Results

Signal: S = {0, S }

X X X

Y Y

Z Z Z
I (Z,X)

0 .2

0 .1



Signal: S = {0, S }

X X X

Y Y

Z Z Z
I (Z,X) Stimulus “On”
0 .2

0 .1



∅→X
Signal: S = {0, S }
X →∅

X X X X → Y /Z
Y /Z → ∅

Y Y
X +Y →Z
Z →∅
Z Z Z
I (Z,X) Stimulus “On”
0 .2

0 .1


Information Propagation: Conclusions

• The dynamics of signal transduction cascades lead to dissipation
of the signal, that is probably poorly described in terms of
Gaussian channels.
• Under certain circumstances intrinsic and extrinsic noise can
“swamp” or “overwhelm” the signal. Systematic differences
between parameters can even induce apparent correlations
between initial and terminal nodes in networks.
• Some of the results observable in stochastic treatments of
information propagation along signal-transduction networks appear
counter-intuitive.



Gaussian channels.
counter-intuitive.
• Such results appear reﬂect the interplay between the
(deterministic) dynamics and the extrinsic and intrinsic sources of
noise.



Gaussian channels.
counter-intuitive.
• Such results appear reﬂect the interplay between the
(deterministic) dynamics and the extrinsic and intrinsic sources of
noise.
• Rather than studying the propagation of information, we next
consider the propagation of noise along a network.

Molecular Noise

Noise in Biological Networks Michael P.H. Stumpf Modelling Molecular Noise 11 of 24

Molecular Noise

∅ ∅


Molecular Noise

∅ ∅ ∅


Modelling Intrinsic Noise
The stoichiometric matrix
 
s11 212 ... s1r
 .. .. . 
. 
 s21 . . . 
S= . . 
 . .. .. . 
 . . . .
sm1 ... . . . smr

describes the changes that can occur
among the r reactions for m molecular
species. I.e. when reaction j occurs then
xi −→ xi + sij .

Komorowski et al., PNAS (2011); Komorowski et al., Bioinformatics (2012).


s11 212 ... s1r
 R1: m −→ m + 1
 .. .. . 
.  R2: m −→ m − 1
 s21 . . . 
S= . .  R3: (m, p) −→ (m, p + 1)
 . .. .. . 
 . . . .
R4: p −→ p − 1
sm1 ... . . . smr
R5: (p, p ) −→ (p − 1, p∗ + 1)
∗

describes the changes that can occur R6: (p, p∗ ) −→ (p + 1, p∗ − 1)
R7: p∗ −→ p∗ − 1



s11 212 ... s1r
 R1: m −→ m + 1
 .. .. . 
.  R2: m −→ m − 1
 s21 . . . 
S= . .  R3: (m, p) −→ (m, p + 1)
 . .. .. . 
 . . . .
R4: p −→ p − 1
sm1 ... . . . smr
R5: (p, p ) −→ (p − 1, p∗ + 1)
∗

describes the changes that can occur R6: (p, p∗ ) −→ (p + 1, p∗ − 1)
R7: p∗ −→ p∗ − 1
Intrinsic vs. Extrinsic Noise
are due to stochastic effects because of small numbers of molecules
and factors outside of the model, respectively. Extrinsic noise can, for
example, affect the rates at which reactions occur.


Stochastic Dynamics

x (t ) =


Stochastic Dynamics

x (t ) = x (0)


Stochastic Dynamics

x (t ) = x (0) + S·j

Ball et al., Ann. Appl. Probab. (2006).

S·j Change in the state due to occurrence of jth reaction


Stochastic Dynamics
t
x (t ) = x (0) + S·j Yj fj (x , s)ds
0



Yj (u ) is a Poisson point process (How many times did reaction j ﬁre?)


Stochastic Dynamics
r t
x (t ) = x (0) + S·j Yj fj (x , s)ds
j =1 0





Stochastic Dynamics
r t
x (t ) = x (0) + S·j Yj fj (x , s)ds
j =1 0




Yj (u ) can be approximated as
Y (u ) ≈ u + N(0, u )


Stochastic Dynamics
r t
x (t ) = x (0) + S·j Yj fj (x , s)ds
j =1 0




Yj (u ) can be approximated as
Y (u ) ≈ u + N(0, u )

Deterministic approximation
r t
φ(t ) = φ(0) + S·j fj (φ, s)ds
j =1 0


Decomposing Noise — 1
We want to know how much each reaction contributes to the overall
noise in the system and therefore deﬁne

x (t ) |−j

as the expectation of x (t ) conditioned on the processes
Y1 (t ), ..., Yj −1 (t ), Yj +1 (t ), ..., Yr (t ). It is a random variable where
timings of reaction j have been averaged over all possible times,
keeping all other reactions ﬁxed.

Noise in Biological Networks Michael P.H. Stumpf Noise Decomposition 14 of 24


x (t ) |−j

Thus
x (t ) − x (t ) |−j
is a random variable representing the difference between the native
process x (t ) and a process with time-averaged jth reaction.



x (t ) |−j

Thus
x (t ) − x (t ) |−j
is a random variable representing the difference between the native
process x (t ) and a process with time-averaged jth reaction.
Now the contribution of the jth reaction to the total variability of x (t ) is

Σ (j ) ( t ) ≡ ( x ( t ) − x ( t ) |−j )(x (t ) − x (t ) |−j )
T


The Linear Noise Approximation (LNA)

ξ(t ) ≡ x (t ) − φ(t ) =



ξ(t ) ≡ x (t ) − φ(t ) =
r t t
= x (0) − φ(0) + S·j fj (x (s), s)ds − fj (φ(s), s)ds
j =1 0 0

r t t
+ S·j Yj fj (x , s)ds − fj (x (s), s)ds
j =1 0 0



ξ(t ) ≡ x (t ) − φ(t ) =
r t t
j =1 0 0

t
φ fj (φ(s), s)ξ(s)ds
0
r t t
j =1 0 0



ξ(t ) ≡ x (t ) − φ(t ) =
r t t
j =1 0 0

t
0
r t t
j =1 0 0

t
Wj fj (φ(s), s)ds
0



ξ(t ) ≡ x (t ) − φ(t ) =
r t t
j =1 0 0

t
0
r t t
j =1 0 0

t
Wj fj (φ(s), s)ds
0

dξ = S φ F (φ)ξdt + S diag F (φ) dW


In summary we have in the LNA:

x (t ) = φ(t ) + ξ(t )



x (t ) = φ(t ) + ξ(t )
• The LNA implies a Gaussian distribution,
x (t ) ∼ MVN (φ(t ), Σ(t ))



x (t ) = φ(t ) + ξ(t )
x (t ) ∼ MVN (φ(t ), Σ(t ))
• with mean φ(t ) given as s solution of the rate equation



x (t ) = φ(t ) + ξ(t )
x (t ) ∼ MVN (φ(t ), Σ(t ))
• variances given as the solution of
d Σ(t )
= A(φ, t )Σ + ΣA(φ, t )T + E (φ, t )E (φ, t )T
dt



x (t ) = φ(t ) + ξ(t )
x (t ) ∼ MVN (φ(t ), Σ(t ))
• variances given as the solution of
d Σ(t )
= A(φ, t )Σ + ΣA(φ, t )T + E (φ, t )E (φ, t )T
dt
• and covariances by
cov(x (s), x (t )) = Σ(s)Φ(s, t )T for s t
d Φ(t , s)
= A(φ, s)Φ(t , s), Φ(t , t ) = I .
ds


The LNA gives us much more ﬂexibility and allows us to write the
overall variance in x (t ), Σ(t ), as the sum of the variances of individual
reactions,
Σ(t ) = Σ(1) (t ) + . . . + Σ(r ) (t ).
where we can obtain Σ(t ) from

dΣ
= A(t )Σ + Σ A(t )T + D (t )
dt
and the contribution to the overall variance resulting from reaction j as

d Σ (j )
= A(t )Σ(j ) + Σ(j ) A(t )T + D (j ) (t ).
dt


Birth and death process
We consider the simple birth and death process,

k+
∅− x
→
k−
x− ∅
→



k+
∅− x
→
k−
x− ∅
→

How much of the noise comes from birth and how much comes from
death?



k+
∅− x
→
k−
x− ∅
→

death?
√
dx = (k + − k − x (t ))dt + k + dW1 + k − x (t ) dW2
birth noise death noise



k+
∅− x
→
k−
x− ∅
→

death?
√
dx = (k + − k − x (t ))dt + k + dW1 + k − x (t ) dW2

We therefore have
1 1
Σ= x + x
2 2


Noise contributions - general results
Proposition 1
In any open conversion system with only ﬁrst order reactions the
contribution of the product degradation to the variability in the product
abundance is precisely one half of the total variability.
1
Σ (r ) = [Σ]nn
nn 2

Komorowski M., Miekisz J., Stumpf M.P.H. Decomposing Noise in Biochemical Signalling Systems Highlights the Role of Protein
Degradation, Submitted, available on arXiv.


Noise contributions - general results
Proposition 1
In any open conversion system with only ﬁrst order reactions the
contribution of the product degradation to the variability in the product
abundance is precisely one half of the total variability.
1
Σ (r ) = [Σ]nn
nn 2

Proposition 2
In a general system the contribution of the product degradation to the
variability in the product abundance is one half its mean.
1
Σ (r ) = xn
nn 2

Komorowski M., Miekisz J., Stumpf M.P.H. Decomposing Noise in Biochemical Signalling Systems Highlights the Role of Protein
Degradation, Submitted, available on arXiv.


Linear cascades

(A)
X1 X2 X3
1 2 3
4 5 6

(B)
X1 X2 X3
1 2 3
4 5 6


Linear cascades
conversion slow conversion fast
7%

6%

36%

(A) 25%
50% 50%

X1 X2 X3
1 2 3
4 5 6 2%
1
3% 5%
2
8% 3%
3 4%
catalytic slow 4 catalytic fast
(B) 17%
2%
5
20% 20%
6
X1 X2 X3 31%
1 2 3
4 5 6
10% 10%

33%

16% 20% 20%

2%


Linear cascades
conversion slow conversion fast
7%

6%

36%

(A) 25%
50% 50%

X1 X2 X3
1 2 3
4 5 6 2%
1
3% 5%
2
8% 3%
3 4%
catalytic slow 4 catalytic fast
(B) 17%
2%
5
20% 20%
6
X1 X2 X3 31%
1 2 3
4 5 6
10% 10%

33%

16% 20% 20%

2%

Conversion reactions: Increasing the rates increases the contribution
of earlier reactions to the overall noise.
Catalytic reactions: Now also early degradation reactions start to
contribute as the rates increase.

Controlling Cell-to-Cell Variability

∅ ∅ ∅



R1 R3 R5
R6

R2 R4 R7

∅ ∅ ∅



R1 R3 R5
R6

R2 R4 R7

∅ ∅ ∅
1
2 7%
3
4 7%
5
6
7
9%

43%

6%

28% < 1%

No dephosphorylation.



R1 R3 R5
R6

R2 R4 R7

∅ ∅ ∅
1
1
2 7% 7%
2
3
3
4 7% 7%
4
5
5
6
6
7
7
8%
9%
40%
43%

6%
6%

3%
28% < 1% 30%

No dephosphorylation. Slow dephosphorylation.



R1 R3 R5
R6

R2 R4 R7

∅ ∅ ∅
1 1
1
2 7% 7% 2 5%
2
3 3 5%
3
4 7% 7% 4
4 24%
5 5
5 6%
6 6
6
7 7
7
8%
9% 5%
40%
43%

6%
6%

18%

36%
3%
28% < 1% 30%

No dephosphorylation. Slow dephosphorylation. Fast dephosphorylation.


Control of Noise
Let us consider the simple birth-death process with

dx = (f (x ) − g (x ))dt + f ( x )dW1 + g ( x )dW2 .


Control of Noise


At steady state we have
1
2 g( x )
Σ(death) = .
− ∂f ( x ) + ∂g∂xx )
∂
x (

For f (x ) = k , g (x ) = γx this yields
1
Σ(death) = 2 x .


Control of Noise


At steady state we have
1
2 g( x )
Σ(death) = .
− ∂f ( x ) + ∂g∂xx )
∂
x (

1
Σ(death) = 2 x .
But if f (x ), g (x ) are Hill or
Michaelis-Menten functions, then
noise can be reduced arbitrarily.


Control of Noise


At steady state we have Control of Degradation

1.0

1.0
1
2 g( x )
Σ(death) = .

0.8

0.8
− ∂f ( x ) + ∂g∂xx )
x (

production and degradation rates
∂

normalised density
0.6

0.6
1
Σ(death) = 2 x .
0.4

0.4
But if f (x ), g (x ) are Hill or
0.2

0.2
Michaelis-Menten functions, then
0.0

0.0
noise can be reduced arbitrarily. 0 10 20 30 40

x


Control of Noise


Noise Propagation: Conclusions
• We can numerically decompose the variance of stochastic
chemical reaction networks and obtain easily interpretable and
useful insights.
• For a large class of cases we obtain exact analytical insights:
◦ For linear open conversion reactions we ﬁnd that output degradation
contributes half the variance.
◦ For general systems the contribution of the product degradation
reaction is half of the expected product abundance.
• Other measures of variability/dispersion can also be expressed in
this decomposition.
Coefﬁcient of Variation:
σ 1
µ 2 x
Fano Factor:
1 σ2
2 µ
• Noise can be controlled by regulating production and degradation
reactions.

Open Questions (A Small Selection)
• How do non-linear dynamics and stochastic effects jointly affect
the dynamics and behaviour of biological systems?
• Is mutual information a good measure for
◦ the ﬁdelity of information transmission?
◦ the efﬁciency of biological information processing?
Similar questions arise for other correlative measures (MIC etc ).
• Does the noise decomposition break down for
◦ small systems?
◦ extrinsic noise?
◦ non-linear dynamics?
• Can we manipulate noise through regulating the protein
degradation reaction?
• Do cells with controlled degradation of key-molecules behave in a
more predictable manner?
• Can we develop stimulus patterns that increase or decrease
cell-to-cell variability?
Noise in Biological Networks Michael P.H. Stumpf Conclusions 23 of 24

Acknowledgements

Noise in Biological Networks Michael P.H. Stumpf Conclusions 24 of 24

Noisy information transmission through molecular interaction networks

More Related Content

Viewers also liked

Similar to Noisy information transmission through molecular interaction networks

More from Michael Stumpf

Recently uploaded

Noisy information transmission through molecular interaction networks