Page 101 - Fundamentals of Communications Systems

Review of Probability and Random Variables 3.15

Definition 3.12 Let X and Y be random variables defined on the probability space
( , F, P ). The conditional or a posteriori PDF of Y given X = x,is
p XY (x, y)
p Y |X (y|X = x) = (3.22)
p X (x)
The conditional PDF, p Y |X (y|X = x), is the PDF of the random variable Y after
the random variable X is observed to take the value x.
Definition 3.13 Two random variables x and y are independent if and only if

p XY (x, y) = p X (x) p Y (y)
Independence is equivalent to

p Y |X (y|X = x) = p Y (y)

i.e., Y is independent of X if no information is in the RV X about Y in the sense
that the conditional PDF is not different than the unconditional PDF.

EXAMPLE 3.15
In Matlab each time rand(•) or randn(•) is executed it returns an ostensibly inde-
pendent sample from the corresponding uniform or Gaussian distribution.

Theorem 3.4 Total Probability For two random variables X and Y the marginal density
of Y is given by

∞
p Y (y) = p Y |X (y|X = x) p X (x)dx
−∞
Proof: The marginal density is given as (Property 3 of PDFs)
∞

p Y (y) = p XY (x, y)dx
−∞
Rearranging Eq. (3.22) and substituting completes the proof.

Theorem 3.5 Bayes For two random variables X and Y the conditional density is
given by
p X|Y (x|Y = y) p Y (y) p X|Y (x|Y = y) p Y (y)
p Y |X (y|X = x) = =
p X (x) ∞ p XY (x, y)dy
−∞
Proof: The definition of conditional probability gives
p XY (x, y) = p Y |X (y|X = x) p X (x) = p Y |X (x|Y = y) p Y (y)

Rearrangement and total probability completes the proof.