Scribd
Upload
296 views9 pages

Jensen Inequality and Related Proofs

The Jensen inequality states that if p(x) and q(x) are two discrete probability distributions, the entropy of p(x) is greater than or equal to the entropy of p(x) weighted by the log of q(x). The proof of the Jensen inequality uses properties of logarithms and convexity to show the inequality holds for any probability distributions p(x) and q(x).

Uploaded by

Vikas Kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
296 views9 pages

Jensen Inequality and Related Proofs

The Jensen inequality states that if p(x) and q(x) are two discrete probability distributions, the entropy of p(x) is greater than or equal to the entropy of p(x) weighted by the log of q(x). The proof of the Jensen inequality uses properties of logarithms and convexity to show the inequality holds for any probability distributions p(x) and q(x).

Uploaded by

Vikas Kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 9

JENSEN INEQUALITY

If p(x) and q(x) are two discrete probability


distributions, if and only if p(x) = q(x) for all x.

 p( x) log p( x)   p( x) log q( x)
x x
PROOF OF JENSEN INEQUALITY
 p( x) log p( x)   p( x) log q( x)  0
x x

 p( x) log  p( x)  q( x)  0
x

 p( x) 
x p( x) log  q( x)   0
 
q( x)
x p( x) log p( x)  0
q( x) q ( x)
x p( x) log p( x)  x p( x)( p( x)  1)
LOG SUM INEQUALITY
PROOF OF LOG SUM INEQUALITY
CONSEQUENCE OF LOG SUM
EQUALITY
DATA PROCESSING INEQUALITY
PROOF OF DATA PROCESSING
INEQUALITY
FANO’S INEQUALITY
PROOF OF FANO’S INEQUALITY

You might also like