Module iii sp

Module III
Stochastic Processes
1

Vector Random Variable
• Assigning a vector of real numbers to each outcome of
Sample Space of a random experiment.
• Example: Random Experiment consists of selecting student’s
name from an urn based on the Height, Weight and Age of
the student.
• - Height of student in inches• - Height of student in inches
• - Age of student in years
• - Weight of student in Kg
• Then the vector is the vector random
variable.
)(H
)(A
)(W
       WAH ,,
2Vijaya Laxmi, Dept. of EEE

• We have
y
y2
   221 yYxXx 
x1 x2 x
x
y1
y2
y
x2x1
   2121 yYyxXx 

y
y2
y1
   211 yYyxX 
x1 y2
y
y2
y1
y2x1-x1
   211 yYyxX 

MULTIPLE RANDOM VARIABLESMULTIPLE RANDOM VARIABLES

Joint Distribution
• If there are two RVs X and Y and the sets and are
events with probabilities
consisting of all outcomes ξ such that
is also an event, then
 xX   yY 
   
    yYxXyYxX
producttheandyFyYPandxFxXP YX


,
)()(
yYandxX  )()( 
is also an event, then
is called the Joint Distribution of the RVs X and Y.
• It is given by
•
 yYxXP  ,
 yYxXPyxFXY  ,),(

Discrete Random Variable
• For discrete RV,
 


iyxfWhere
yxfyYxXP
).....(..........0),(,
),(,
 

x y
iiyxf
iyxfWhere
).....(..........1),(
).....(..........0),(,

• Let X be a RV which assumes any one value of
• And, Y be a RV which assumes any one of
mxxx .,..........,, 21
nyyy .,..........,, 21
• Then, the probability of an event that X=xj and Y=yk
is given by
n21
  ),(, kjkj yxfyYxXP 

X Y y1 y2 . . yn Total
f(x1,y1) f(x1,y2) . . f(x1,yn) f1(x1)
x2 f(x2,y1) f(x2,y2) . . f(x2,yn) f1(x2)
x3 f(x3,y1) f(x3,y2) . . f(x3,yn) f1(x3)
x1
. . . . . . .
. . . . . . .
xm f(xm,y1) f(xm,y2) . . f(xm,yn) f1(xm)
Total f2(y1) f2(y2) . . f2(yn) 1
Grand Total

• The probability that X=xj is obtained by adding all
entries in the row corresponding to xj is given by
• The probability that Y=yk is obtained by adding all
   

n
k
kjjj yxfxfxXP
1
1 ,)(
• The probability that Y=yk is obtained by adding all
entries in the row corresponding to yk is given by
   

m
j
kjkk yxfyfyYP
1
2 ,)(

• f1(xj) and f2(yk) or simply f1(x) and f2(y) are obtained from the
margins of table, hence called Marginal Probability function of
X and Y
• Or,
which, can be written as
    11
1 1
21   
m
j
n
k
kj yfandxf
  1,  
m n
yxf
which, can be written as
• i.e., Total probability of all entries is 1.
• Joint distribution function of X and Y is given by
• This is the sum of all entries for which
  1,
1 1
  j k
kj yxf
      

xu yv
vufyYxXPyxF ),(,,
yyandxx kj 

Continuous Random Variables
• The properties of joint PDF for the continuous RVs
are
  )..(....................0, iyxf   )..(....................0, iyxf 
12
)........(1),( iidxdyyxf 





Vijaya Laxmi, Dept. of EEE

Joint Distribution function for
Continuous RV
• The joint distribution function of two RVs X
and Y is given by
 
 
dudvvufyYxXPyxF
x
u
y
v
 

),(,),(
  FunctionDensityyxf
yx
yxF



),(
,2

Marginal Distribution Function
• The marginal distribution function of X is given by
• The marginal distribution function of Y is given by
   



x
u v
dudvvufxFxXP ),()(1
• The marginal distribution function of Y is given by
   

 

u
y
v
dudvvufyFyYP ),()(2

Marginal Density Function
• The marginal density function of RV X is given by
• The marginal density function of RV Y is given by
  



v
dvvxfxF
dx
d
xf ),()(11
• The marginal density function of RV Y is given by
  



u
duyufxF
dy
d
yf ),()(22

• Or, If for all x and y, f(x,y) is the product of a function
of x alone and a function of y alone (which are the
marginal probability of X and Y), then, X and Y are
independent.
• If, f(x,y) cannot be expressed as function of x and y,
then, X and Y are dependent.

Independence
• If X and Y are two independent random variables, events that
involve only X should be independent of the events that
involve only Y.
• If A1 is any event that involve X only and A2 is any event that
involve only Y, then for discrete RVs,
     , yYPxXPyYxXP 
• In general, n random variables are independent,
when
• Knowledge about probabilities of RVs in isolation is sufficient to specify
the probabilities of joint events.
     
)()(),(,
,
21 yfxfyxfor
yYPxXPyYxXP


nXXX ,......, 21
       nnnn xXPxXPxXPxXxXxXP  ............,,, 22112211

• X and Y (continuous RVs)
• If, the events are independent events for
all x and y,
   yYandxX 
     , yYPxXPyYxXP 
)()(),(
),()(),(
21
21
yfxfyxf
andyFxFyxF



Properties of Joint CDF
• The joint CDF is non-decreasing in the
‘northeast’ direction, i.e.,
),(),( yyandxxifyxFyxF  21212211 ),(),( yyandxxifyxFyxF XYXY 
(x1,y1)
(x2,y2)
x
y

x
y
x1
  YxXPxFX ,)( 11

x
y
y1
 11 ,)( yYXPyFY 

• It is impossible for either X or Y to assume a value less than -
∞, therefore
• It is certain that X and Y will assume values less than infinity,
therefore
    0,, ,,  xFyF YXYX
therefore
• If, one of the variables approach infinity while keeping the
other fixed, marginal cumulative distribution functions are
obtained as
1),(, YXF
   
   yYPyYXPyxFyF
and
xXPYxXPyxFxF
YXY
YXX


,),()(
,),()(
,
,

• The joint CDF is continuous from the north and from
the east, i.e.,


ax
YXYX
and
yaFyxF ),(),(lim ,,


by
YXYX bxFyxF
and
),(),(lim ,,

Problem
• If X and Y are two discrete RVs, whose joint PDF
is given by
 
0
3020,2
),(


 

otherwise
yandxwhereyxc
yxf
 
 2,1)(
1,2)(
)(
0



YXPc
YXPb
caFind
otherwise

Solution: (a)
• We haveX Y 0 1 2 3 Total
0 0 C 2c 3c 6c
1 2c 3c 4c 5c 14c
2 4c 5c 6c 7c 22c2 4c 5c 6c 7c 22c
Total 6c 9c 12c 15c 42c
C=1/42

• (b)
• (c)
 
42
5
51,2  cYXP
  ),(2,1
1 2

  
yxfYXP
X Y
7
4
42
24
24
)654()432(


c
cccccc

Problem
• Find the marginal probability of (a) X and (b) Y

Solution
• Marginal Prob. function of X,
 













2
21
11
22
1
3
1
14
0
7
1
6
)()(1
xforc
xforc
xforc
xfxfxXP X
• Marginal Prob. function of Y,
 
















3
14
5
15
2
7
2
12
1
14
3
9
0
7
1
6
)()(2
yforc
yforc
yforc
yforc
yfyfyYP Y

Problem
• Show that the RVs X and Y are dependent

Solution
• If x and y are independent, then for all x and y
     yYPxXPyYxXP  ,
 
42
5
1,2  YXP
30
   
14
3
1,
21
11
2,  YPandXPBut
dependentareYandX

14
3
.
21
11
42
5

Problem
• If X and Y are two cont. RV, whose joint density
function is given by
0
51,40
),(


 

otherwise
yxforcxy
yxf
 
 2,3)(
32,21)(
)(


YXPc
YXPb
caFind

Solution
• (a)
• (b) 96
1
96
1),(
4
0
5
1
4
0
5
1











  
 
  




cc
dxxydyccxydxdy
dxdyyxf
x yx y
• (c)
 
128
5
96
32,21
2
1
3
2

   x y
dxdy
xy
YXP
 
128
7
96
2,3
4
3
2
1

   x y
dxdy
xy
YXP

Problem
• Find the Marginal distribution function of X
and Y

Solution
• Marginal Distribution function of X
 
1696
1
96
),()(
2
0
5
10
5
1
x
duuvdvdudv
uv
dudvvufxXPxF
x
u v
x
u v
x
u v
X

















  
 
  



34
.0)(,0
1)(,4
40,



xFx
xFxFor
xBecause
X
X











41
40
16
00
)(
2
xfor
xfor
x
xfor
xFX

• Marginal Prob. function of Y,
   

 


u
y
v
Y
y
dudvvufyYPyF
24
1
),()(
2
51,  yBecause
35
.0)(,1
1)(,5


yFy
yFyFor
Y
Y












51
51
24
1
10
)(
2
yfor
yfor
y
yfor
yFY

Change of Variables (Discrete RVs)
• Theorem 1: If X is a discrete RV having Prob. Function
f(x) and another RV U is defined by U=ø(X)
• For each value of X there corresponds one and only
one value of U,one value of U,
• Proof:
 )()(,..,
)(
ufugUoffnprobThen
UX




   
    UfuXP
uXPuUPug




)(
)()(

• Theorem 2: If X and Y is discrete RVs having joint
prob. Function f(x,y)
• Another RV U and V defined by
   YXVandYXU ,, 21  
• For each pair of values of X and Y there corresponds
one and only one pair of values of U & V
• Then, joint Prob. Function of U & V is given by
   VUYandVUX ,, 21  
    vuvufvug ,,,),( 21 

• Proof:
      
    
    vuvuf
vuYvuXP
vYXuYXPvVuUPvug
,,,
,,,
,,,,),(
21
21
21







Change of Variables (Continuous RVs)
• Theorem 1: If X is a continuous RV having Prob.
Function f(x) and another RV U is defined by U=ø(X),
where
• Then, prob. Density function of U is given by g(u),
)(UX 
Then, prob. Density function of U is given by g(u),
where
   uuf
du
dx
xfug
dxxfduug
'
)()()(
)()(



• Proof: If u is an increasing function i.e., if x increases
then u increases.
u
u2
u1
   
)()(
2 2
2121


  dxxfduug
xXxPuUuP
u
u
v
v
x1 x2
u1
x
   
    0)('')()(
)()(
2
1
2
1
1 1
'

 
 
uhereuufug
duuufduug
u
u
u
u
u v


This can be proved for 0)('0)('  uoru 

• Theorem 2:
• The joint density function g(u,v) of U and V is given
by
   
   VUYVUXwhere
YXVYXUDefine
,,,,
,,,
21
21




by
• Where,
    Jvuvufvug
dxdyyxfdudvvug
,,,),(
),(),(
21 

 
 
v
y
u
y
v
x
u
x
vu
yx
J












,
,

• Proof:
• If x and y increases then, u and v also increases
   21212121 ,, yYyxXxPvVvuUuP 
2 22 2 x yu v
This can be proved also for 0J
42
    
 


2
1
2
1
2
1
2
1
2
1
2
1
,,,
),(),(
21
u
u
v
v
x
x
y
y
u
u
v
v
Jdudvvuvuf
dxdyyxfdudvvug

     0,,,),( 21  JhereJvuvufvug 

Problem
• If X is a discrete RV with prob. Function
• Find the prob. Function of RV


 


otherwise
xfor
xf
x
0
....3,2,12
)(
14
XU
• Find the prob. Function of RV

Solution


uwhereuxgives
xu
.......,82,17,2,1
,1
4
4



 



otherwise
u
ug
uwhereuxgives
u
0
.....,82,17,2,2
)(
.......,82,17,2,1
4
1
4

Problem
• If X is a Cont. RV with density function
• Find the prob. Density Function for


 

otherwise
xforx
xf
0
6381/
)(
2
 XU  12
3
1
• Find the prob. Density Function for 3

Solution
 
2,6
,5,3
3'



ux
anduxFor
dx
du
u 
uxux
xu
312312
12
3
1


Check:
 
 

5
2
2
.1
27
312
du
u
46
 








otherwise
u
u
ug
0
52,
27
312
)(
2

Problem
• If X and Y are two cont. RVs whose joint density
function is given by






otherwise
yandx
xy
yxf
0
5140
96),(
• Find the density function of U=X+2Y
 otherwise0

Solution
 
102,4051,40
2
1
,
)(2



vuvyxrangethe
vuyvx
chosenyarbitrarilxvandyxu
v
u-v=2
u
u-v=2
u-v=10
v=4
v=0 102
I
II
III
2
1
2
1
2
1
10












v
y
u
y
v
x
u
x
J
 
otherwise
vvuvuvvug
0
40,102384/),( 

• Marginal density function of U is given by
ufordv
vuv
ufordv
vuv
ug
v
u
v
106
384
)(
62
384
)(
)(
4
0
2
0
1










 




   
 
 
otherwise
u
uu
u
u
u
uu
ufordv
vuv
uv
v
,0
1410,
2304
2128348
106,
144
83
62,
2304
42
1410
384
)(
384
3
2
4
10
0







 











Expected Value of function of random variables
• The expected value of Z=g(X,Y) can be obtained by
 



 
 
continuouslyjoYXdxdyyxfyxg
ZE
YX int,,),(
)(
,
   






  
RVsdiscreteYXyxpyxg
ZE
i
niYX
n
ni ,,,
)(
,

Sum of Random Variables
• Let Z=X+Y, Find E(Z)
    






''','''
)()(
, dydxyxfyx
YXEZE
YX
     








 ''',''''','' ,, dydxyxfydydxyxfx YXYX
The expected value of the sum of two random variables is equal to
the sum of individual expected values.
X and Y need not be independent.
51
     
  








  













 ''',''''','' ,, dydxyxfydxdyyxfx YXYX
   
)()(
''''''
YEXE
dyyfydxxfx YX

 





• The expected values of sum of n random variables is
equal to the sum of the expected values.
       nn XEXEXEXXXE  ........ 2121       nn XEXEXEXXXE  ........ 2121

Sum of Discrete RVs
• If X and Y are two discrete RVs
   
yxyfyxxf
yxfyxYXE
x y




),(),(
),(
   YEXE
yxyfyxxf
x yx y

  ),(),(

Expected value of Product of Two RVs
• If X and Y are two independent RVs
• Proof: (continuous RV)
     YEXEXYE 
   
 
   
 
   
   YEXE
dyyfydxxfx
dydxyfxfyx
dydxyxfyxXYE
YX
YX
YX

















 
 












''''''
'''()'''
''',''' ,

Product of two discrete RVs
• If X and Y are two independent RVs
)()(),( 21 yfxfyxf 
 
 
 


x y
yfxxyf
yxxyfXYE
)()(
),(
 
x y
yfxxyf )()( 21
55
  
   YEXE
YExxf
yyfxxf
x
x y










 
)(
)()(
1
21

Product of functions of Random variables
• If X and Y are two independent RVs and g(X,Y)=g1(X)g2(Y)
• Find
• Solution:
             dydxyfxfygxgYgXgE YX2121 ''''''  
 
       YgEXgEYXgE 21),( 
            
       
     YgEXgE
dyyfygdxxfxg
dydxyfxfygxgYgXgE
YX
YX
21
21
2121
''''''
''''''
















 




 

• In general, for n independent RVs,
nXXX ,........, 21
               nnnn XgEXgEXgEXgXgXgE ................. 22112211                nnnn 22112211

Conditional Probability Distribution
• For P(A)>0, Probability of event B given that A has
occurred
        )(|
)(
| APABPBAPor
AP
BAP
ABP 


• If X and Y are discrete RVs with events (A:X=x) and
(B:Y=y)
  Xofprobinalmtheisxfwhere
xf
yxf
xyf .arg)(,
)(
),(
| 1
1


• Conditional prob. function of Y given X is given by
• Conditional prob. function of X given Y is given by
 
)(
),(
|(
xf
yxf
xyf
X

• Conditional prob. function of X given Y is given by
 
)(
),(
|
yf
yxf
yxf
Y


Conditional Probability Distribution for
Cont. RV
• If X and Y are two cont. RVs, the conditional prob.
Density function of Y given X is given by
 
)(
),(
|
xf
yxf
xyf 
)( xf X

Problem
• The joint density function of two cont. RVs X and Y
are given by






otherwise
yxxy
yxf
0
10,10,
4
3
),3(
• Find (a)
• (b)
 otherwise0
 xyf






 dxXYP
2
1
2
1
2
1

Solution
• (a) For 0<x<1
  











 
10,
43
),(
4
23
24
3
4
3
)(
1
0
1
y
xy
yxf
xx
dyxyxf
• (b)
 





)(0
10,
23
)(
),(
1
definednototherwise
y
x
xf
yxf
xyf
16
9
4
23
2
1
2
1
2
1
2
1
2/1
2/1




















dy
y
dyyfdxXYP

Variance of Sum of Two RVs
• For two independent RVs X and Y,
Var(X+Y)=Var(X)+Var(Y)
• Proof:
     
     
)(
2
YXEYXVar YX  
     
       
         
           
     
)()(
2
2
2
22
22
22
22
2
YVarXVar
YEXE
YEYEXEXE
YEYXEXE
YYXXE
YXE
YX
YYXX
YYXX
YYXX
YX












• Var(X-Y)=Var(X)+Var(Y)
• Proof:
     
     
)(
2
2
YXE
YXEYXVar
YX
YX




     
       
         
     
)()(
2
2
22
22
22
YVarXVar
YEXE
YEYEXEXE
YYXXE
YXE
YX
YYXX
YYXX
YX










Variance of Sum of Two RVs
• In general,
XYYXYXor
YXCovYVarXVarYXVar
 2,
),(2)()()(
222




Correlation and Covariance of Two RVs
• The jkth joint moment of two RVs X and Y is defined
as
   
 
 









kj
YX
kjKj
ContinuouslyjoYandXFordxdyyxfyxYXE int,,
 
i n
niYX
k
n
j
i RVsdiscreteYandXyxpyx ,,
If j=0, moments of Y are obtained
If k=0, moments of X are obtained
If j=k=1, E[XY] gives the correlation of X and Y
IF E[XY]=0, X and Y are orthogonal

Jkth central moment about the mean
• The jkth central moment of X and Y about the mean is
given by
    k
Y
j
X YXE  
If j=2, k=0, variance of X is obtained
If j=0, k=2, variance of Y is obtained
If j=k=1, Covariance of X and Y is obtained
    YandXofianceCoYXCovYXE YX var),(  

Covariance of Two RVs
   
 
     
         YEXEYEXEXYE
YEXEXYE
YXXYE
YXEYXCov
YXXY
YXXY
YX




2
),(



         
     YEXEXYE
YEXEYEXEXYE

 2
Hence,
Cov(X,Y)=E[XY], if either of the RVs has mean value equal to zero

Problem
• If X and Y are two independent RVs each having
density function


 


otherwise
ufore
uf
u
0
02
)(
2
• Find
     XYEYXEYXE ,, 22


Solution
     
1
2.2.
0
2
0
2








dyeydxex
YEXEYXE
yx
  1
4 22
  
 

dxdyexyeXYE yx
Check E[X+Y]=E[X]+E[Y]
E[XY]=E[X]E[Y]
70
 
4
4
0 0
22
  

dxdyexyeXYE yx
  122
0
22
0
2222
 




dyeydxexYXE yx

Problem
• If X and Y are two discrete independent RVs







4/3.2
3/2.0
3/1.1
probwith
Y
probwith
probwith
X
• Find





4/1.3 probwith
Y
       YXEXYEYXEYXE 222
,,2,23 

Solution
     
3
1
0
3
2
1
3
1
XE
     
4
3
3
4
1
2
4
3
YE
72
     
2
5
4
3
2
3
1
3
2323













 YEXEYXE

     22
 YEXEYXE
     
4
1
4
3
.
3
1

 YEXEXYE
     
4
1
4
3
.
3
1

 YEXEYXE

     
3
1
0
3
2
1
3
1 22
XE
     
4
21
4
9
33
4
1
2
4
3 222
YE
74
     
12
55
4
21
3
2
4
21
3
1
2
22 2222













 YEXEYXE

Problem
• If X and Y are two independent RVs, find the Covariance
• Solution:
   
     
0
),(



YX
YX
YEXE
YXEYXCov


0
For the pairs of independent RVs, Covariance is zero.

Correlation Coefficient of X and Y
• The correlation coefficient between the two RVs, X and Y is
given by
     YEXEXYEYXCov
YXYX
YX
),(
,






deviationdardSYVarandXVarwhere YX tan)()(,  
11 ,  YX
X and Y are uncorrelated, if 0, YX
If X and Y are independent, Cov(X,Y)=0, 0,  YX
If X and Y are independent, they are uncorrelated.
-------From Theorem 4

Theorems on Covariance
• Theorem 1:
• Theorem 2: If X and Y are independent RVs
• Theorem 3:
     YEXEXYEXY 
0),(  YXCovXY
• Theorem 3:
• Theorem 4:
 
XYYXYXor
YXCovYVarXVarYXVar
 2,
),(2)()(
222



YXXY  
If X and Y are independent, Theorem 3 reduces to Theorem 4.
The converse of Theorem 3 is not necessarily true.

Problem
• If X and Y are two discrete RVs, whose joint density function is
given by
• Find E[X], E[Y], E[XY], E[X2], E[Y2], Var(X), Var(Y), Cov(X,Y) and
 
otherwise
yxforyxcyxf
0
30,202),( 
• Find E[X], E[Y], E[XY], E[X2], E[Y2], Var(X), Var(Y), Cov(X,Y) and
correlation coeff.

Solution
• We have
X Y 0 1 2 3 Total
0 0 C 2c 3c 6c
1 2c 3c 4c 5c 14c1 2c 3c 4c 5c 14c
2 4c 5c 6c 7c 22c
Total 6c 9c 12c 15c 42c

   
cccc
xfxyxfxyxxfXE
x y xx y
5822.214.16.0
)(,),(







   
      yyfyxfyyxyfYE
x y yy x
)(,, 





   
80
ccccc
x y yy x
7815.312.29.16.0 

   
c
ccc
yxxyfXYE
x y
102
........3.3.02.2.0.1.00.0.0
,


 

     
                ccccc
yxfyyxfyYE
x y y x
1921531229160
,,
2222
222







   
     
            cccc
yxfxyxfxXE
x y x y
10222214160
,,
222
222







   
81
      
      2222
2222
78192)(
58102)(
ccYEYEYVar
ccXEXEXVar
Y
X




     
YX
YX
YXCov
ccc
YEXEXYEYXCov



),(
78.58102
),(,




Problem
• If X and Y are two Cont. RVs
• Find (a) c
• (b) E[X]
 
 


 

otherwise
yxforyxc
yxf
0
50,622
,
• (c) E[Y]
• (d) E[XY]
• (e) E[Y2]
• (f) Var(X)
• (g) Var(Y)
• (h) Cov(X,Y)
• (i) Correlation Coeff.

Solution
 
 
210
1
12
1,sin
6
2
5
0



 
 
 




c
dxdyyxc
dxdyyxfgU
x y
    2681
6 5
 
83
   
63
268
2
210
1
6
2
5
0
   x y
dxdyyxxXE
   
7
80
2
210
1
  




dxdyyxxyXYE
   
63
170
2
210
1
  




dxdyyxyYE

  63
12202
XE
    
3969
5036
)(
222
 XEXEXVarX
7938
16225
)(2
 YVarY
  126
11752
YE
84
     
3969
200
),(  YEXEXYEYXCov
03129.0
YX
XY



7938
Y

Problem
• Let Ɵ be uniformly distributed RV in the interval
(0,2π) and X and Y are two RVs defined as X=Cos Ɵ,
Y=SinƟ.
• Show that X and Y are uncorrelated.Show that X and Y are uncorrelated.

Solution
• The marginal PDF of X and Y are arcsine functions which are
nonzero in the interval
• So, if X and Y were independent the point (X,Y) would assume
all values in the square.
• But, this is not the case, hence X and Y are dependent.
1111  yandx
• But, this is not the case, hence X and Y are dependent.
(CosƟ, SinƟ)
Ɵ
y
x
1
-1
-1
1

   
02
4
1
2
1
2
0
2
0











dSin
dCosSinCosSinEXYE
Since E[X]=E[Y]=0, X and Y are uncorrelated
Uncorrelated but dependent Random Variables

Conditional Expectation, Variance and
Moments
• The conditional expectation of a RV Y given X is expressed as
• Properties:
 If X and Y are independent,
    dxxXxmeansxXwheredyxyyfxXYE  


,
 If X and Y are independent,

   YExXYE 
        XYEEdxxfxXYEYE  


1

• Hence,
         YoffunctionaisYhwhereXYhEEYhE ,
    
and
thKKK
,

89
     origintheaboutmomentktheisYwhereXYEEYE thKKK
,

Theorem
    XYEEYE 
Proof:
      
   






dxxfXYE
dxxfXYEXYEERHS
X
:
   
 dxxfXYE X
90
   
 
 
  
 










dxdyxf
xf
yxf
y
dxxfdyxyyf
XY
XY
,
 
   YEdyyyf
dxdyyxfy



 






,

Problem
• If X and Y are two RVs whose density function is
given by
   
otherwise
yxforyxcyxf
0
30,202, 
• Find E[Y|X=2]

Solution
• The marginal density function of RV X is given by
• And, marginal density function of Y is given by
 









222
114
06
1
xforc
xforc
xforc
xf
• And, marginal density function of Y is given by
 












315
212
19
06
2
yforc
yforc
yforc
yforc
yf

• The conditional density function of Y given X=2 is
obtained as
   
 
 
22
4
42/22
42/2,
2
1
yyx
xf
yxf
yf




93
     




 

yy
y
yyyfXYE
22
4
22
       
11
19
22
7
3
22
6
2
22
5
1
22
4
0 

























Problem
• The average time of travel from city ‘A’ to city ‘B’ is ‘c’
hours by car and ‘b’ hours by bus. A man cannot
decide whether to drive the car or to take bus, so he
tosses a coin. What is the expected time of travel?

Solution
• X is RV, which is the outcome of toss
• Y is travel time 
1
0


XifY
XifYY car
The RVs X and Y are independent, because Ycar and Ybus
are independent of X
• Using Property 1,
• Using Property 2, (for discrete RVs)
•
1XifYbus
     
      bYEXYEXYE
cYEXYEXYE
busbus
carcar


11
00
         
2
1100
bc
XPXYEXPXYEYE




Conditional Variance
• The conditional variance of Y given X is defined
as
• The rth conditional moment of Y about any value
a is given as
      
 xXYEwhere
ydxyfyxXYE

 


2
2
2
2
2


a is given as
      


 dyxyfayxXaYE
rr
The usual theorems for variance and moments extend to conditional
variance and moments.

Gaussian PDF and CDF
 
 
2
2
2
2
1
: 

mx
X exfXofPDF


 
 



x mx
dxexXPCDF '
2
1
:
2
2
2
'


FX(x) is the CDF of Gaussian RV with zero mean and unit variance.
97
 

mxtLet  '
 
 





 

 







mx
dtexFThen
mx
t
X
2
2
2
1
,
  


x
t
dtexwhere 2
2
2
1



Gaussian PDF
• Show that Gaussian PDF integrates to one.
• Solution:
• Take Square of PDF of Gaussian RV











dyedxedxe
yxx
22
2
2
222
11

 
 












dxdye
dyedxedxe
yx
2
22
2
1
22


98
1
2
1
,
0
2
0
2
0
2
22



 




drrerdrde
rSinyrCosxLet
rr





Joint Characteristic Function
• The Joint Ch. Fn of n RVs, is given by
• And, the joint ch. fn. of two RVs X and Y is given by
   
 nn
n
XXXj
nXXX eE 
 
 ...
21.....
2211
21
,...,,
nXXX ,...,, 21
   
 
 eE YXj
YX
21
, 21,


   
 





 dxdyeyxf yxj
YX
21
,,

Joint Characteristic function is the 2-dimensional Fourier
transform of joint PDF of X and Y
     
 





 2121,2,
21
,
4
1
, 


ddeyxf yxj
YXYX

Marginal Characteristic Function
   
   

,0
0,
XYY
XYX


If X and Y are independent
   
   
       21
21
21
2121
,




YX
YjXj
YjXjYXj
XY
eEeE
eeEeE

 

   
   
    


 













21
0
2
0
1
21
!!
, 21
ki
ki
k
k
i
i
YjXj
XY
jj
YXE
k
Yj
i
Xj
E
eeE


 
    
 

0 0
21
!!i k
ki
k
j
i
j
YXE

101
    0,021
21
21
|,
1
,



 

XYki
ki
ki
ki
j
YXE
And

Problem
• If Z=aX+bY, find the ch. Fn. Of Z
• Solution:
   
   
 
 
 
ba
eEeE
XY
bYaXjbYaXj
Z
,
 
• If X and Y are independent
        baba YXXYZ  ,

Problem
• If U and V are independent zero mean, unit
variance Gaussian RV, and
• X=U+V and Y=2U+V
• Find the joint ch. Fn of X and Y and find E[XY]• Find the joint ch. Fn of X and Y and find E[XY]

Solution
• The joint Ch. Fn. Of X and Y is given by
• The joint Ch. Fn. Of U and V is given by
   
      
 
   
 )2(
2
21
2121
2121
,
VUj
VUVUjYXj
XY
eE
eEeE







   
   
 2
, 
 
 eEeE VjUj
   
   
 
   
   2
21
2
21
2121
2
1
2
2
1
2121
2
21
2
,









ee
eEeE
VU
VjUj
XY
  2/22

 
 jm
X e
104
   
3
|,
1
0,021
21
2
2 21



  
 XY
j
XYE
We have

Jointly Gaussian RV
• The RVs X and Y are said to be jointly Gaussian, if
their joint PDF has the form
 
 
2
2
2
2
2
2
1
1
,
2
1
1
2
,
12
2
12
1
exp
,
YX
YX
XY
mymymxmx
yxf


























 





 





 





 



• for
•
2
,21 12 YX 
 yandx

Problem
• The PDF for jointly Gaussian RVs is given by
 








16168
3
8
3
16
3
4
2
1
,
22
2
1
,
yx
xy
yx
YX eyxf

• Find E[X], E[Y], Var(X), Var(Y) and Cov(X,Y)

Problem
• The joint PDF of X and Y is given by
• Find the marginal PDF.
      

 
yxeyxf yxyx
YX ,,
12
1
,
222
12/2
2,


• Find the marginal PDF.

Solution
• The marginal PDF of X is obtained by integrating f(x,y) over y as
 
 
   
 
     






2,
12
12
2222212/
2
12/
12/2
2
12/
2222
22
22
22
xxy
x
xyy
x
X
xxxyyHeredye
e
dye
e
xf














   
 





2
122
2,
12
2
2
12/2
2
2
22
2
x
xy
x
e
dy
ee
xxxyyHeredye














• Hence
 The last integral equals one because its integrand is a Gaussian
PDF with mean and variance
 The marginal PDF of X is a one-dimensional Gaussian PDF with mean
x 2
1 
 The marginal PDF of X is a one-dimensional Gaussian PDF with mean
0 and variance 1. Form symmetry, the marginal PDF of Y will also be a
Gaussian PDF with mean 0 and variance 1.

N jointly Gaussian RV
• The RVs are said to be jointly Gaussian RVs
if their joint PDF is given by
•
nXXX ,...,, 21
 
   
 
 









n
T
nXXXX
bydefinedvectorscolumnaremandXwhere
K
mXKmX
xxxfXf n
,
2
2
1
exp
,...,,)(
2
1
2
1
21,...,, 21

•
 
 
 
     
     
      



































































nnn
n
n
nnn
XVarXXCovXXCov
XXCovXVarXXCov
XXCovXXCovXVar
Kand
XE
XE
XE
m
m
m
m
x
x
x
X
..,,
.....
.....
,..,
,..,
,
.
.
.
.,
.
.
21
2212
1211
2
1
2
1
2
1
Where, K is called the Covariance Matrix 110Vijaya Laxmi, Dept. of EEE

Module iii sp

More Related Content

What's hot (19)

Similar to Module iii sp (20)

More from Vijaya79 (7)

Recently uploaded (20)

Module iii sp