Page 26 - Probability and Statistical Inference

1. Nations of Probability 3

Since the coin is assumed to be fair, this particular random experiment gener-
ates the following probability scheme: P(HHH) = P(HHT) = P(HTH) = P(HTT)
= P(THH) = P(THT) = P(TTH) = P(TTT) = 1/8. !
Example 1.1.2 Suppose that we toss two fair dice, one red and the other
yellow, at the same time and record the scores on their faces landing upward.
Then, each simple event would constitute, for example, a pair ij where i is the
number of dots on the face of the red die that lands up and j is the number of
dots on the face of the yellow die that lands up. The sample space is then given
by S = {11, 12, ..., 16, 21, ..., 26, ..., 61, ..., 66} consisting of exactly 36 pos-
sible simple events. Since both dice are assumed to be fair, this particular ran-
dom experiment generates the following probability scheme: P(ij) = 1/36 for
all i, j = 1, ..., 6. !
Some elementary notions of set operations are reviewed in the Section 1.2.
The Section 1.3 describes the setup for developing the formal theory of prob-
ability. The Section 1.4 introduces the concept of conditional probability fol-
lowed by the notions such as the additive rules, multiplicative rules, and Bayes’s
Theorem. The Sections 1.5-1.6 respectively introduces the discrete and con-
tinuous random variables, and the associated notions of a probability mass
function (pmf), probability density function (pdf) and the distribution function
(df). The Section 1.7 summarizes some of the standard probability distributions
which are frequently used in statistics.

1.2 About Sets

A set S is a collection of objects which are tied together with one or more
common defining properties. For example, we may consider a set S = {x : x is
an integer} in which case S can be alternately written as {..., –2, –1, 0, 1, 2,
...}. Here the common defining property which ties in all the members of the set
S is that they are integers.
Let us start with a set S. We say that A is a subset of S, denoted by A ⊆ S,
provided that each member of A is also a member of S. Consider A and B which
are both subsets of S. Then, A is called a subset of B, denoted by A ⊆ B, pro-
vided that each member of A is also a member of B. We say that A is a proper
subset of B, denoted by A ⊂ B, provided that A is a subset of B but there is at
least one member of B which does not belong to A. Two sets A and B are said to
be equal if and only if A ⊆ B as well as B ⊆ A.
Example 1.2.1 Let us define S = {1, 3, 5, 7, 9, 11, 13},A = {1, 5, 7}, B =
{1, 5, 7, 11, 13}, C = {3, 5, 7, 9}, and D = {11, 13}. Here, A, B, C and D are