Modelos Estocasticos 2

Christian Ojeda
Tarea 2
Exercise 1. The rooted binary tree is an infinite graph T with one distinguished vertex R from
which comes a single edge; at ever other vertex there are three edges and there are no closed loops.
The random walk on T jumps from a vertex along each available edge with equal probability. Show
that the random walk is transient.
Proof. We will index the vertices of T with binary numbers as follows. Denote R with 0 and the
vertex connected below R with 1. Next, denote the vertices connected with 1 with the numbers
10 and 11. In the same way, we will index the two vertices connected below the vertex indexed
by i1 i2 . . . in , i1 , . . . , in {0, 1}, with the numbers i1 . . . in 0 and i1 . . . in 1. It is easy to see that the
transition matrix of the random walk on T is irreducible, because
i1 . . . in i1 . . . in1 ,

i1 . . . in i1 i2 . . . in 0 and i1 . . . in i1 . . . in 1,

i1 , . . . , in {0, 1}.

and so, 0 i1 . . . in for all i1 , . . . , in {0, 1}. Then, we only have to proof that the estate 0 (or
...in
denote the rooted binary tree that
R) is transient. Let X be the random walk on T , and Tii11...i
n in+1
is under the vertex i1 . . . in+1 and with root i1 . . . in . It is evident that X has the same behavior that
...in
the random walk on Tii11...i
. This implies that
n in+1
P(H0n < |Xn = 1) = P(H1n < |X1 = 10) = P(H1n < |Xn = 11),
where Hin1 ...im := inf{k n : Xk = i1 . . . im } and Hi1 ...in = Hi11 ...in . Since P0 (X1 = 1) = 1 we have
P0 (H0 < ) = P(H0 < |X1 = 1)
= P(X2 = 0|X1 = 1) + P(H0 < , X2 = 10|X1 = 1) + P(H0 < , X2 = 11|X1 = 1)
P(X2 = 0|X1 = 1) + P(H12 < , X2 = 10|X1 = 1) + P(H12 < , X2 = 11|X1 = 1)
1
= + P(H12 < |X2 = 10, X1 = 1)P(X2 = 10|X1 = 1)
3
+ P(H12 < |X2 = 11, X1 = 1)P(X2 = 11|X1 = 1)
1 P(H12 < |X2 = 10) P(H12 < |X2 = 11)
+
(M. p.) = +
3
3
3
1 2P(H02 < |X2 = 1)
= +
3
3
1 + 2P0 (H0 < )
=
.
3
Then,
P0 (H0 < ) < 1,
and therefore R is transient, as required.
Exercise 2. Find all invariant distributions of the transition matrix

1
0 0 0 21
2
0 1 0 1 0
2
2

.
0
0
1
0
0
P =

0 1 1 1 1
4
4
4
4
1
0 0 0 21
2

Tarea 2

Solution:. Let = (1 , 2 , 3 , 4 , 5 ) be an invariant distribution of M , that is P = . Then we

have the relations
1 + 5
= 1
(1)
2
22 + 4
= 2
(2)
4
43 + 4
= 3
(3)
4
22 + 4
= 4
(4)
4
21 + 25 + 4
= 5
(5)
4
From (2) and (4) we obtain that
4
= 2
2

and

34
= 2 ,
2

and so, 2 = 4 = 0. From (1) and (5) we have that 1 = 5 . Therefore has the form =
(1 , 0, 3 , 0, 1 ), with the condition 21 + 3 = 1. So, the set of all invariant distributions of P is



1
.
= (x, 0, 1 2x, 0, x) : x 0,
2
Exercise 3. Each bacterium in a colony splits into two identical bacteria after an exponential time
of parameter , which then split in the same way but independently. Let Xt denote the size of
 the

colony at time t, and suppose X0 = 1. Show that the probability generating function (t) = E1 z Xt
satisfies
Z
t

es (t s)2 ds.

(t) = zet +

Make a change of variables u = t s in the integral and deduce that d/dt = ( 1). Hence
deduce that, for q = 1 et and n = 1, 2, . . .
P(Xt = n) = q n1 (1 q).
Proof. Note that the population bacteria never goes down. Note also that in a short time, only one
bacterium can split into two bacteria. This implies that when h 0, uniformly in t we have

 
n

(h + o(h))(1 h + o(h))n1 + o(h), j = n + 1,

 
n
P(Xt+h = j|Xt = n) =
1
(h + o(h))(1 h + o(h))n1 + o(h), j = n,

o(h), j 6= n, n + 1.
The above relation holds because we choose the bacterium that splits with
of that bacterium splits is (h + o(h)). We can rewrite this as
P(Xt+h = j|Xt = n) = nj + qnj + o(h),

n
1

and the probability

Calculo Estocastico

where qn,n+1 = n, qn n = n and qn , j = 0 if j 6= n, n + 1. This process is called Yule process of

intensity , and it satisfies that
Xt |X0 = 2 Xt1 + Xt2 |X01 = 1, X02 = 1,

(6)

where X 1 and X 2 are two independent Yule process of intensity . This relation holds because each
individual splits independently of the others whit intensity and only matters the initial population.
Let J1 be the first jump time of X. Conditional to X0 = 1 we have that J1 exp() and then




 
(t) = E1 z Xt = E1 z Xt 1{J1 >t} + E1 z Xt 1{J1 t}




= E1 z X0 1{J1 >t} + E1 z Xt 1{J1 t}
Z t


e E1 z Xt |J1 = s ds
= zP(J1 > t) +
0
Z t


e E1 z XJ1 +ts |J1 = s ds.
= zet +
0

As XJ1 = 2, the strong Markov property impies that

Z t
h
i
e
t
e0 = 2 ds
(t) = ze +
e E z Xts |X
0

e is a Yule process of intensity . By (6) we obtain

where X
Z t
h 1
i
2
t
(t) = ze +
e E z Xts +Xts |X01 = 1, X02 = 1 ds
Z0 t
h 1 i h 2 i
t

(independence) = ze +
e E1 z Xts E1 z Xts ds
Z0 t
= zet +
e (t s)2 ds,
0

as required. With the change of variable u = t s in the above integral it results that
Z t
t
t
(t) = ze + e
eu (u)2 ds,
0

which proves that is differentiable and



Z t
0
t
t
u
2
(t) = ze + e
e (u) ds + et et (t)2
0

= (t)((t) 1).
This expression can be rewritten as
et 0 (t) et
d
+
=
2
(t)
(t)
dt
It follows that

et
1

+
=
(t) (0)

Z
0

 t 
e

= et .
(t)

es ds = 1 et .

Tarea 2

Since (0) = z, we obtain that

et
zet
=
.
1
1 z(1 et )
1 + et
z


1
Finally, we note that for all z 0,
and t > 0
1 et
(t) =

z n P(Xt = n) = (t)

n=1

zet
1 z(1 et )

X
t
(z(1 et ))n
= ze
=

n=0

z n (1 et )n1 et ,

n=1

and therefore
P(Xt = n) = (1 et )n1 et ,

n N,

as required (the case t = 0 is obvious).

Exercise 4. Let X = (Xt )t0 be a Markov chain on the integers with transition rates
qi,i+1 = qi ,

qi,i1 = qi

and qij = 0 if |j i| 2, where + = 1 and qi > 0 for all i. Find for all integers i:
(a) the probability, starting from 0, that X hits i;
(b) the expected total time spent in state i, starting from 0.
In the case where = 0, write down a necessary and sufficient condition for X to be explosive. Why
is this condition necessary for X to be explosive for all [0, 1/2)?
Show that, in general, X is non-explosive if and only if one of the following conditions holds:
(i) = ;
(ii) > and

1/qi = ;

(iii) < and

1/qi1 = .

i=1
i=1

Proof. ...
Exercise 5. Two fleas are bound together to take part in a nine-legged race on vertices A, B, C of a
triangle. Flea 1 hops at random times in the clockwise direction; each hop takes pair from one vertex
to the next and the times between successive fops of Flea 1 are independent random variables, each
with exponential distribution, mean 1/. Flea 2 behaves similarly, but hops in the anticlockwise
direction, the times between his hops having mean 1/. Show that the probability that they are at
A at a given time t > 0 (starting form A at time t = 0) is
(
)


1 2
3( + )t
3( )t
+ exp
cos
.
3 3
2
2

Calculo Estocastico

Proof. ...
Exercise 6. Let X = (Xt )tt be a birth-and-death process with rates n = n and n = n, and
assume that X0 = 1. Show that h(t) = P1 (Xt = 0) satisfies
Z t
h(t) =
e(+)s { + h(t s)2 } ds
0

and deduce that if 6= then

h(t) =

et et
.
et et

Proof. The Q-matrix of X has the form

qii = ( + )i,

qi,i+1 = i and qi,i1 = i,

and so, the transition matrix of the jump chain has the form
i,i1 =

and i,i+1 =

.
+

This process models the behavior of a population where each individual splits whit intensity and
dies with intensity , independently from the others. As in exercise 3, we have that
Xt |X0 = 2 Xt1 + Xt2 |X01 = 1, X02 = 1,

(7)

where X 1 and X 2 are birth-and-death processes with the same rates than X, because it only matters
the initial population. Then,
h(t) = P1 (Xt = 0)
= P1 (Xt = 0, J1 t) + P1 (Xt = 0, J1 > t)
= P1 (Xt = 0, XJ1 , J1 t) + P1 (Xt = 0, XJ1 = 2, J1 t)
Z t
( + )e(+)s P1 (Xt = 0, Y1 = 2|J1 = s) ds
= P1 (Y1 = 0, J1 t) +
0
Z t

(+)t
(1 e
)+
( + )e(+)s P1 (XJ1 +ts = 0, Y1 = 2|J1 = s) ds
=
+
Z0 t

ets = 0) ds
(by s.M.p.) =
(1 e(+)t ) +
( + )e(+)s P2 (X
+
Z0 t

1
2
(by (7)) =
(1 e(+)t ) +
( + )e(+)s P(Xts
+ Xs+t
= 0|X01 = 1, X02 = 1) ds
+
Z0 t

1
2
(X 1 , X 2 0) =
(1 e(+)t ) +
( + )e(+)s P(Xts
= 0, Xs+t
= 0|X01 = 1, X02 = 1) ds
+
Z0 t

1
2
(1 e(+)t ) +
( + )e(+)s P1 (Xts
= 0)P1 (Xts
= 0) ds
(ind.) =
+
0
Z t
=
e(+)s { + h(t s)2 } ds.
0

With the change of variable u = t s we have

Z t
(+)t
h(t) = e
e(+)u ( + h(u)2 ) du.
0

Tarea 2

The above expression shows that h is differentiable and

h0 (t) = ( + )h(t) + + h(t)2 ,
and it is to check that the solution to this differential equation with initial condition h(0) = 0, when
6= is
et et
h(t) = t
.
e et