CSE 736: Markov Chains Lecturer: Hung Q. Ngo SUNY Buffalo, Spring 2003 Scribe: Hung Q. Ngo

Lecture 2: Review of Probability Theory

Some texts on Probability Theory are [1, 4–7]. The absolute classic is William Feller’s [2, 3]. There are also two excellent free books on the Internet at

   • http://www.dartmouth.edu/~chance/teaching aids/
     books articles/probability book/book.html,
   • http://www.mit.edu/people/dimitrib/Probability.html

1 Preliminaries

Through out this note, we often use [n] to denote \{1, 2, …, n\}, \binom{Ω}{k} to denote the set of all k-subsets of Ω, and 2^{Ω} to denote the power set of Ω.

An event E is a subset of the sample space (or probability space) Ω that we are considering. For any two events E and F, we use EF to denote their intersection and E ∪ F their union. Assume there is a function P : 2^{Ω} → [0, 1], defined on each event E, such that

(i) 0 ≤ P [E] ≤ 1.

(ii) P [Ω] = 1.

(iii) For any (finite or infinite) sequence E_{1}, E_{2}, … of events such that E_{i}E_{j} = ∅ whenever i ≠ j (mutually exclusive),

P (⋃_{i} E_{i}) = ∑_{i} P [E_{i}].

P [E] is called the probability of event E. By the inclusion-exclusion principle, the following identity holds:

P [E_{1} ∪ ⋯ ∪ E_{n}] = ∑_{k=1}^{n}(−1)^{k+1} ∑_{\{i_{1},...,i_{k}\}∈\binom{[n]}{k}} P [E_{i_{1}} … E_{i_{k}}].

Let E and F be any two events. The conditional probability that E occurs given that F has occurred, denoted by P [E|F] is defined to be

P [E|F] := \frac{P [EF]}{P [F]}

Note that this definition is only valid when P [F] > 0. We can think of F as being the new sample space. The phrase “given F ” implies that the outcome belongs to F, so the probability that the outcome also belongs to E (which must thus be in EF) is the fraction given.

Two events E and F are said to be independent events if P [EF] = P [E]P [F], or equivalently, P [E|F] = P [E]. Intuitively, E and F are independent if the density of E over the whole space is equal to the density of EF over F and vice versa. Put it another way, the probability that E occurs does not change even when we know that F has occured. If E and F are not independent, they are said to be dependent.

Example 1.1. Suppose we are tossing 2 dice fairly. Let F be the event that the first die is 4, E_{1} be the event that the sum is 6 and E_{2} be the event that the sum is 7. Then, E_{1} and F are dependent and E_{2} and F are independent. For, P [F] = 1/6, P [E_{1}] = 5/36, P [E_{2}] = 1/6, P [E_{1}|F] = 1/6, and P [E_{2}|F] = 1/6.

A set E_{1}, …, E_{n} of events are said to be independent iff for any {i_{1}, …, i_{k}} ⊆ {E_{1}, …, E_{n}} we have

P [E_{i_{1}} … E_{i_{k}}] = P [E_{i_{1}}] … P [E_{i_{k}}].

Intuitively, this means that the knowledge on the occurence of any subset of these events does not affect the probability of any other event.

Example 1.2. Rolling 3 dice, the events that each die takes a particular value are independent.

Example 1.3 (Pair-wise independence does not imply independence.). Flip 2 coins in a row. Let E_{1} be the event that the first coin turns up head, E_{2} be the event the the second coin turns up tail, E_{3} be the event that the two coins are both heads or both tails. Then, P [E_{1}] = P [E_{2}] = P [E_{3}] = 1/2. Moreover,

P [E_{1}E_{2}] = 1/4 = P [E_{1}]P [E_{2}]
P [E_{1}E_{3}] = 1/4 = P [E_{1}]P [E_{3}]
P [E_{2}E_{3}] = 1/4 = P [E_{2}]P [E_{3}],

hence the three events are pair-wise independent. However,

P [E_{1}E_{2}E_{3}] = 0 ≠ P [E_{1}]P [E_{2}]P [E_{3}] = 1/8,

namely they are not independent.

Suppose F_{1}, …, F_{n} are mutually exclusive events (i.e. F_{i}F_{j} = ∅ whenever i ≠ j), and that ⋃_{i} F_{i} = Ω. (We also say the events F_{1}, …, F_{n} partition the sample space.) Intuitively, exactly one and only one of the events F_{i} will occur. Firstly, notice that for any event E, E = ∪_{i}EF_{i}, and that all events EF_{i} are also mutually exclusive. We have

P [E] = ∑_{i} P [EF_{i}] = ∑_{i} P [E|F_{i}]P [F_{i}]  (1)

This states that P [E] is the weighted average of the P [E|F_{i}]. The equation also implies that

P [F_{j}|E] = \frac{P [F_{j}E]}{P [E]} = \frac{P [E|F_{j}]P [F_{j}]}{∑_{i} P [E|F_{i}]P [F_{i}]}  (2)

Equation (2) is known as Bayes’ formula.

Example 1.4. In multiple choice tests, assume a student knows the answer to any question with probability p (i.e. she guesses with probability 1 − p). Let m be the number of alternatives for each question. What is the probability that she knew the answer to a question given that she answered it correctly? What is the sample space in this example?

Answer. Let A be the event that she answered it correctly, B be the event that she guessed and C she knew the answer. We know B ∪ C is the sample space. We want P [C|A]. Using Bayes’ formula we get

P [C|A] = \frac{P [A|C]P [C]}{P [A|B]P [B] + P [A|C]P [C]} = \frac{p}{\frac{1}{m}(1 − p) + p}

The sample space consists of pairs (x, y) where x is the outcome of her answer (true, false) and y is the fact that she guessed or not.

Exercise 1.5 (Monty Hall Problem). There are three closed doors at the conclusion of a game show. A contestant chooses one of the doors, behind which he or she hopes lies the GRAND PRIZE. The host of the show opens one of the remaining two doors to reveal a wimpy prize. The contestant is then given the choice to stay with the original choice of a door or to switch and choose the remaining door that the host did not open. The problem is: Should the contestant stick with the original door choice or switch and choose the other door? Argue intuitively first and rigorously later. There is an intriguing story behind this also called “let’s make a deal” problem. For more information, see, e.g.,

http://www.nadn.navy.mil/MathDept/courses/pre97/sm230/MONTYHAL.HTM

for a sample discussion.

Exercise 1.6 (Red or Black problem). An acquaintance of mine Harry enjoys a good bet. He always keeps the bet small enough to be affordable, but big enough to keep him in pocket money. One of his favorites uses three cards and a brown bag:

Harry: I have an empty brown bag and three cards. One card is black on both sides, one card is red on both sides and the other is black on one side and red on the other. I’ll put all three cards in the bag and mix them well. Now you carefully pull one card out so that we see only one side. Look, its red. That means it can’t be the card that is black on both sides. So its one of the other two cards and an even bet that its black on the other side. What do you say we each bet $1. You get black and I get red.

Harry likes this game so much he wants to continue playing, always giving you the color not showing. Since the color showing has already been used once making it less likely and he just plays for the entertainment.

One of the following statements about Harry is true. Select the true statement and show that it is true.

1. In the long run Harry will win about as much as he loses.

2. In the long run Harry will win a lot more than he loses.

3. In the long run Harry will lose a lot more than he wins.

2 Random variables

In many cases, we are more interested in some function on the events rather than the event itself. For example, we might want to know the probability of tossing two dice whose sum is 7. These quantities of interest, or more formally these real-valued functions defined on the events, are called random variables.

For example, suppose a coin has probability p of coming up head. Let X be the random variable that is defined as the number of fair tosses of a coin until it comes up head, then clearly

P [X = n] = (1 − p)^{n−1}p.

We usually want to know if an event E happens or not, and define a random variable I to be 1 if E happens and 0 otherwise. The variable I is said to be the indicator random variable for event E.

The cumulative distribution function F (·) (cdf), often referred to as the distribution function, of a random variable X is defined for every x ∈ ℝ by

F (x) = P [X ≤ x]

2.1 Discrete random variables

Random variables that can take on only a countable number of values are called discrete random variables. To each discrete random variable X, we associate a probability mass function p(a), or simply probability function, defined by

p(a) = P [X = a].

If X can take on only values x_{1}, x_{2}, …, then we must have

∑_{i=1}^{∞} p(x_{i}) = 1
F (a) = ∑_{x_{i}≤a} p(x_{i}),

where F is the distribution function of X.

Definition 2.1 (Bernulli random variable). Suppose an experiment is performed with two outcomes “success” and “failure”, where the probability of a success is p (and a failure is 1 − p). Let X be 1 if the outcome is a success and 0 if the outcome is a failure, then

p(0) = 1 − p
p(1) = p  (3)

X is said to be a Bernulli random variable if its probability function is defined by (3).

Definition 2.2 (Binomial distribution). Suppose n independent trials are performed, each with success probability p. Let X be the number of successes in the n trials, then clearly

p(i) = \binom{n}{i}p^{i}(1 − p)^{n−i}, ∀i = 0, …, n

Such an X is called a binomial random variable, or is said to have a binomial distribution with parameters (n, p). We also write X ∈ binomial(n, p).

Definition 2.3 (Geometric distribution). Suppose n independent trials, each with success probability p, are performed until a success occurs. Let X be the number of trials, then Let X be the number of independent trials of coin flipping until a head shows up, then

p(i) = (1 − p)^{i−1}p, ∀i = 1, 2, ….

Such an X is called a geometric random variable, and said to have a geometric distribution with parameter p. We also write X ∈ geometric(p).

Definition 2.4 (Poisson distribution). X is called a Poisson random variable with parameter λ > 0 if it takes on values in ℕ, and its probability function is defined by

p(i) = P [X = i] = e^{−λ} \frac{λ^{i}}{i!} , ∀i ∈ ℕ.

Such an X is said to have a Poisson distribution with parameter λ, and we write X ∈ Poisson(λ).

It is easy to verify that p(i) above actually defines a proper probability function since ∑_{i=0}^{∞} p(i) = 1. Given that n is large and p is small, a Poisson random variable with λ = np can be used to approximate a binomial random variable with parameters (n, p). Basically, if p → 0, np → λ as n → ∞, then \binom{n}{i}p^{i}(1 − p)^{n−i} → e^{−λ} \frac{λ^{i}}{i!} .

2.2 Continuous random variables

A random variable X taking on uncountably many possible values is said to be a continuous random variable if there exists a function f : ℝ → ℝ, having the property that for every B ⊆ ℝ:

P [X ∈ B] = ∫_{B} f(x)dx

The function f(x) is called the probability density function of X. Obviously, we must have

1 = P [X ∈ (−∞, ∞)] = ∫_{−∞}^{∞} f(x)dx.

Notice that P [X = a] = 0, and that the distribution function F (·) of X could be calculated as

F (a) = P [X ∈ (−∞, a]] = ∫_{−∞}^{a} f(x)dx.

Differentiating both sides of the preceding equation yields \frac{d}{da}F (a) = f(a), that is the density is the derivative of the distribution.

Definition 2.5 (Uniform distribution). X is said to be uniformly distributed on the interval (α, β) if its density is

f(x) = \begin{cases}\frac{1}{β−α} & if x ∈ (α, β) \\ 0 & otherwise \end{cases}

As F (a) = ∫_{−∞}^{a} f(x)dx, we get

F (a) = \begin{cases}0 & a ≤ α \\ \frac{a−α}{β−α} & a ∈ (α, β) \\ 1 & a ≥ β \end{cases}

Definition 2.6 (Exponential distribution). X is said to be exponentially distributed with parameter λ if its density is

f(x) = \begin{cases}λe^{−λx} & if x ≥ 0 \\ 0 & if x < 0 \end{cases}

Clearly, its cdf F is

F (a) = ∫_{−∞}^{a} f(x)dx = 1 − e^{−λa}, a ≥ 0.

This kind of random variables occur a lot in the studies of Poisson processes. We shall have much more to say about the exponential distribution later on.

Definition 2.7 (Gamma distribution). X is called a gamma random variable with parameters α, λ if its density is

f(x) = \begin{cases}\frac{λe^{−λx}(λx)^{α−1}}{Γ(α)} & if x ≥ 0 \\ 0 & if x < 0 \end{cases}

where the gamma function is defined by

Γ(α) = ∫_{0}^{∞} e^{−x}x^{α−1}dx.

This is a generalization of the factorial function, as it is easy to show by induction on n that Γ(n) = (n − 1)!.

Definition 2.8 (Normal distribution). Lastly, a continuous random variable X is normally distributed with parameters μ and σ^{2} if the density of X is given by

f(x) = \frac{1}{\sqrt{2π}σ} e^{−(x−μ)^{2}/2σ^{2}}, x ∈ ℝ.

This function is a Bell-shaped curve peaking at μ and symmetric around μ. More importantly, if X is normally distributed with parameters μ and σ^{2}, then Y = αX + β is normally distributed with parameters αμ + β and (ασ)^{2} (why?). When μ = 0 and σ^{2} = 1, X is said to have unit or standard normal distribution.

3 Expectations, moments and variances

Definition 3.1 (Expectation of a discrete variable). Given a discrete random variable X with probability mass function p(x), then the expected value of X is defined as

E[X] := ∑_{x} xp(x).

In other words, the expected value of X is the weighted average of all possible values of X, each weighted by the corresponding probability.

It is not difficult to see that

• If X is a Bernulli random variable with parameter p, then E[X] = p.

• If X is a binomial random variable with parameters (n, p), then E[X] = np.

• If X is a geometric random variable with parameter p, then E[X] = 1/p.

• If X is a Poisson random variable with parameter λ, then E[X] = λ.

Definition 3.2 (Expectation of a continuous variable). Given a continuous random variable X with density f(x), we define the expected value of X as

E[X] = ∫_{−∞}^{∞} xf(x)dx.

The above definition is “expected” since the integral is the continuous analog of the sum. Simple calculations show that

• When X is uniformly distributed over (α, β), E[X] = (α + β)/2.

• When X is exponentially distributed with parameter λ, E[X] = 1/λ.

• When X is normally distributed with parameters (μ, σ^{2}), E[X] = μ. This is why we use μ, which stands for “mean”.

Many times, we are interested in calculating the expectation of a random variable Y whose value is a function of a random variable X, say Y = g(X) for some g : ℝ → ℝ. The following proposition comes almost directly from definitions.

Theorem 3.3. Let g be any real-valued function, then (i) If X is a discrete random variable with probability mass function p(x), then

E[g(X)] = ∑_{x} g(x)p(x)

(ii) If X is a continuous random variable with probability density function f(x), then

E[g(X)] = ∫_{−∞}^{∞} g(x)f(x)dx

Corollary 3.4. If a and b are constants, then E[aX + b] = aE[X] + b. Definition 3.5 (Moments). E[X] is usually called the first moment of X. Similarly, E[X^{2}] is the second moment and E[X^{n}] is the nth moment of X. Note that

E[X^{n}] = \begin{cases}∑_{x} x^{n}p(x) & if X is discrete \\ ∫_{−∞}^{∞} x^{n}f(x)dx & if X is continuous \end{cases}

We shall see the use of these moments later on. Definition 3.6 (Variance). Another quantity of interest is the variance of X denoted by Var [X], which is defined by

Var [X] = E[(X − E[X])^{2}].

The variance of X is often denoted by σ_{X}^{2}, or simply σ^{2} if X is clear from context. The number σ itself is called the standard deviation of X. The following very useful identity can be proven easily by considering separately the continuous and discrete cases. Theorem 3.7. Var [X] = E[X^{2}] − (E[X])^{2}

We list here some simple facts without proof. In the discrete case, we have

• The variance of X ∈ Poisson(λ) is Var [X] = λ.

• The variance of X ∈ geometric(p) is Var [X] = (1 − p)/p^{2}. In the continuous case, we get

• The variance of X exponentially distributed with parameter λ is Var [X] = 1/λ^{2}.

• The variance of X uniformly distributed over (a, b) is Var [X] = (b − a)^{2}/12.

• The variance of a normal random variable X with parameters (μ, σ^{2}) is Var [X] = σ^{2}.

4 Multiple Random Variables

We are often interested in probability statements concerning a set of random variables. Let X_{1}, …, X_{n} be a set of n random variables, the joint cumulative probability distribution function of X_{1}, …, X_{n} is defined by

F (a_{1}, …, a_{n}) = P [X_{1} ≤ a_{1}, …, X_{n} ≤ a_{n}]

The distribution (cdf) F_{X_{i}}(·) of any X_{i} can be obtained from F (·) as follows.

F_{X_{i}}(a) = P [X_{i} ≤ a]
= P [X_{1} ≤ ∞, …, X_{i} ≤ a, …, X_{n} ≤ ∞]
= F (∞, …, a, ∞, …, ∞).

When all X_{i} are discrete, we define the joint probability mass function of the X_{i} by

p(x_{1}, …, x_{n}) = P [X_{1} = x_{1}, …, X_{n} = x_{n}].

Given this, the probability function of each variable X_{i} (denoted by p_{X_{i}}(x)) can be obtained by

p_{X_{i}}(x) = ∑_{x_{1},...,x_{i−1},x_{i+1},...,x_{n}} p(x_{1}, …, x, …, x_{n}).

The functions p_{X_{i}}(x) gives the marginal distributions of the variables X_{i}.

We say that X_{1}, …, X_{n} are jointly continuous if there is a function f(x_{1}, …, x_{n}) on ℝ^{n} having the property that for all sets A_{1}, …, A_{n} of real numbers

P [X_{1} ∈ A_{1}, …, X_{n} ∈ A_{n}] = ∫_{A_{1}} … ∫_{A_{n}} f(x_{1}, …, x_{n})dx_{1} … dx_{n}

This function is naturally called the joint probability density function of the X_{i}. The marginal density f_{X_{i}} of each individual X_{i} can be obtained by observing that

P [X_{i} ∈ A] = P [X_{1} < ∞, …, X_{i} ∈ A, …, X_{n} < ∞]
= ∫_{−∞}^{∞} … ∫_{A} … ∫_{−∞}^{∞} f(x_{1}, …, x_{i}, …, x_{n})dx_{1} … dx_{n}
= ∫_{A} (∫_{−∞}^{∞} … ∫_{−∞}^{∞} f(x_{1}, …, x, …, x_{n})dx_{1} … dx_{n}) dx

Hence,

f_{X_{i}}(x) = ∫_{−∞}^{∞} … ∫_{−∞}^{∞} f(x_{1}, …, x, …, x_{n})dx_{1} … dx_{n}

Similar to Theorem 3.3 and Corollary 3.4 we obtain:

Theorem 4.1. Let g(x⃗) be any real-valued function on x⃗ ∈ ℝ^{n}, then

(i) If X_{i} are discrete random variables with joint probability mass function p(x⃗), then

E[g(X_{1}, …, X_{n})] = ∑_{x⃗:p(x⃗)>0} g(x⃗)p(x⃗)

(ii) If X_{i} are continuous random variables with joint probability density function f(x⃗), then

E[g(X_{1}, …, X_{n})] = ∫_{−∞}^{∞} g(x⃗)f(x⃗)dx_{1} … dx_{n}

Corollary 4.2 (Linearity of Expectation). If X_{1}, …, X_{n} are n random variables, then for any n constants a_{1}, …, a_{n},

E[a_{1}X_{1} + ⋯ + a_{n}X_{n}] = a_{1}E[X_{1}] + ⋯ + a_{n}E[X_{n}]

This corollary holds regardless of the dependence or independence of the X_{i}. This fact is extremely useful in applying the probabilistic methods to solve problems. We will see many applications of this corollary later on. Let’s start with a simple example.

Example 4.3. What is the expected number of people getting their own hat back among n people who throw their hats to a pile and then each gets one back at random?

Answer. Let X be the number of people with their own hat back. Let X_{i} be the indicator variable indicating if person i gets his own hat back, then

X = X_{1} + ⋯ + X_{n}.

Moreover, since E[X_{i}] = P [X_{i} = 1] = 1/n we get

E[X] = ∑_{i} E[X_{i}] = n(1/n) = 1.

This says than only one person gets his own hat back on average, independent of the number of people. Thus, if those people who haven’t got their hats back keep playing the game, intuitively on average it will take n rounds until no-one is left to play. This is indeed the truth as we shall see.

Definition 4.4 (Independent random variables). Two random variables X and Y are independent if for all a,b,

P [X ≤ a, Y ≤ b] = P [X ≤ a]P [Y ≤ b]

In other words, X and Y are independent iff for all a and b, the events E_{a} = {X ≤ a} and F_{b} = {Y ≤ b} are independent. In terms of the joint distribution function F of X and Y. X and Y are independent if F (a, b) = F (a)F (b).

When X and Y are discrete, the independence condition reduces to

p(x, y) = p_{X}(x)p_{Y} (y),

while when they are jointly continuous, we have

f(x, y) = f_{X}(x)f_{Y} (y).

It is not difficult to show the above two facts. Another important fact is the following theorem.

Theorem 4.5. If X and Y are independent then for any function h and g,

E[h(X)g(Y)] = E[h(X)]E[g(Y)].

Exercise 4.6. Suppose X ∈ Poisson(λ_{1}) and Y ∈ Poisson(λ_{2}) are independent. Show that Z = X+Y ∈ Poisson(λ_{1} + λ_{2}).

Exercise 4.7. Suppose X ∈ binomial(n, p) and Y ∈ binomial(m, p) are independent. Show that Z = X + Y ∈ binomial(m + n, p).

Definition 4.8 (Covariance and variance of sums of random variables). The covariance of two random variables X and Y is defined by

Cov [X, Y] := E[(X − E[X])(Y − E[Y])].

Expending out this expression definition out, we get

Cov [X, Y] := E[XY] − E[X]E[Y],

so that Cov [X, Y] = 0 when X and Y are independent. It is also easy to see the following 1. Cov [X, X] = Var [X].

2. Cov [X, Y] = Cov [Y, X].

3. Cov [cX, Y] = cCov [X, Y].

4. Cov [X, Y + Z] = Cov [X, Y] + Cov [X, Z].

Given this observation, we could calculate the variance of a sum as follows.

Var [∑_{i} X_{i}] = Cov [∑_{i} X_{i}, ∑_{i} X_{i}]
= ∑_{i} ∑_{j} Cov [X_{i}, X_{j}]
= ∑_{i} Var [X_{i}] + ∑_{i} ∑_{j≠i} Cov [X_{i}, X_{j}]

In particular, when the X_{i} are pairwise independent, we have

Var [∑_{i} X_{i}] = ∑_{i} Var [X_{i}]

Definition 4.9 (Sample mean and sample variance). If X_{i}, i = 1, …, n are independent and identically distributed random variables, then the random variable

X̄ := (∑_{i=1}^{n} X_{i})/n

is called the sample mean, and the random variable S^{2} defined by

S^{2} = ∑_{i=1}^{n} \frac{(X_{i} − X̄)^{2}}{n − 1}

is called the sample variance of these data. Proposition 4.10. If X_{i}, i = 1, …, n are independent and identically distributed random variables with mean μ and variance σ^{2}, then

(a) E[X̄] = μ.

(b) Var \left[{ \atop X̄}\right] = σ^{2}/n.

(c) Cov [X̄, X_{i} − X̄] = 0, for i = 1, 2, …, n.

(d) E[S^{2}] = σ^{2}. Proof. Except part (d), other parts are easy to prove. To show part (d) we need to use the moment generating function introduced next.

Definition 4.11 (Moment generating function). The moment generating function ϕ(t) of the random variable X is defined for all values t by

ϕ(t) := E[e^{tX}] = \begin{cases}∑_{x} e^{tx}p(x) & if X is discrete \\ ∫_{−∞}^{∞} e^{tx}f(x)dx & if X is continuous \end{cases}

ϕ(t) gets its name because all moments of X can be obtained by taking derivatives of ϕ(t) evaluated at 0. In general, it’s easy to see that

ϕ^{(n)}(0) = E[X^{n}], for all n ≥ 1.

The following theorem is important, where the first part is easy to prove.

Theorem 4.12. We have

(i) the moment generating function of the sum of independent random variables is just the product of the individual moment generating functions.

(ii) the moment generating function of X uniquely determines the distribution of X (i.e. its cdf and mass or density.)

5 Limit Theorems

Proposition 5.1 (Markov’s Inequality). If X is a random variable taking only non-negative values, then for any a > 0

P [X ≥ a] ≤ \frac{E[X]}{a}

Proof. We show this for the discrete case only, the continuous case is similar. By definition, we have

E[X] = ∑_{x} xp(x)
= ∑_{x<a} xp(x) + ∑_{x≥a} xp(x)
≥ ∑_{x≥a} ap(x)
= aP [X ≥ a]

Proposition 5.2 (Chebyshev’s Inequality). If X is a random variable with mean μ and variance σ^{2}, then for any k > 0,

P [|X − μ| ≥ k] ≤ \frac{σ^{2}}{k^{2}}

Proof. This inequality makes a lot of sense. The probability that X is far from its mean gets smaller when X is further, and smaller when its variance is smaller. The proof is almost an immediate corollary of Markov’s. Let Z = (X − μ)^{2}, then E[Z] = σ^{2} by definition of variance. Since |X − μ| ≥ k iff Z ≥ k^{2}, applying Markov’s inequality completes the proof.

The following are the two most well known results in probability theory.

Theorem 5.3 (Strong law of large number). Let X_{1}, X_{2}, … be a sequence of independent random variables having a common distribution, and let E[X_{i}] = μ. Then with probability 1,

\frac{X_{1} + X_{2} + ⋯ + X_{n}}{n} → μ, as n → ∞

Theorem 5.4 (The Central Limit Theorem). Let X_{1}, X_{2}, … be a sequence of independent and identically distributed random variables with mean μ and variance σ^{2}. Then the distribution of

\frac{X_{1} + X_{2} + ⋯ + X_{n} − nμ}{σ\sqrt{n}}

tends to the standard normal distribution as n → ∞. That is

P [\frac{X_{1} + X_{2} + ⋯ + X_{n} − nμ}{σ\sqrt{n}} ≤ a] → \frac{1}{\sqrt{2π}} ∫_{−∞}^{a} e^{−x^{2}/2}dx

as n → ∞.

6 Conditional expectation and probability

Similar to conditional probability of events, we define the followings.

Definition 6.1. If X and Y are discrete random variables, then the conditional probability mass function of X given that Y = y is

p_{X|Y} (x|y) := P [X = x|Y = y]
= \frac{P [X = x, Y = y]}{P [Y = y]}
= \frac{p(x, y)}{p_{Y} (y)}.

Similarly, the conditional probability distribution function of X given Y = y is defined for all Y such that P [Y = y] > 0 by

F_{X|Y} (x|y) := P [X ≤ x|Y = y] = ∑_{a≤x} p_{X|Y} (a|y).

Finally, the conditional expectation E[X|Y = y] of X given Y = y is defined by

E[X|Y = y] := ∑_{x} xP [X = x|Y = y] = ∑_{x} xp_{X|Y} (x|y).

Definition 6.2. We have an identical analog in the continuous case as follows. Suppose X and Y are continuous random variables with joint density function f(x, y), then the conditional probability density function of X given that Y = y, is defined for all values of y such that f_{Y} (y) > 0, by

f_{X|Y} (x|y) := \frac{f(x, y)}{f_{Y} (y)}

The conditional expectation of X given that Y = y, is defined for all values of y such that f_{Y} (y) > 0, by

E[X|Y = y] := ∫_{−∞}^{∞} xf_{X|Y} (x|y)dx.

The following two theorems are easy to show, but are very important tools whose values can not be overestimated.

Theorem 6.3. Let E[X|Y] denote the function with indeterminate Y whose value at Y = y is E[X|Y = y]. Note that E[X|Y] is itself a random variable. We have

E[X] = E[E[X|Y]],  (4)

which in the discrete case means

E[X] = ∑_{y} E[X|Y = y]P [Y = y],  (5)

and in the continuous case means

E[X] = ∫_{−∞}^{∞} E[X|Y = y]f_{Y} (y)dy.  (6)

Example 6.4. A rat is trapped in a cage. There are three doors from the cage. The first door leads to a tunnel going back to the cage which takes the rat 2 minutes. The second door leads to another tunnel which takes 3 minutes. The third door leads to a tunnel out of the cage in 4 minutes. Suppose at any moment the rat is equally likely to take any one of the three doors. What is the expected number of minutes before she goes free?

Answer. Let X be the number of minutes, Y be the door number she takes in the first trial. We have

E[X] = E[X|Y = 1]\frac{1}{3} + E[X|Y = 2]\frac{1}{3} + E[X|Y = 3]\frac{1}{3}
= \frac{1}{3}(2 + E[X] + 3 + E[X] + 4 + 0),

namely E[X] = 9.

Exercise 6.5 (The Matching Round Problem). Consider the hat game again. Suppose those people who got their own hats back leave the game, the rest keep playing. Let n be the initial number of people, R_{n} the number of rounds until noone is left, S_{n} the number of selections made in all rounds, C_{n} the number of selections made by person number 1. Calculate the expected values of R_{n}, S_{n} and C_{n}.

Exercise 6.6. A coin is flipped until there are k consecutive heads. What’s the mean number of flips?

Example 6.7 (Analysis of Quick Sort). We can sort a sequence of n distinct numbers a_{1}, …, a_{n} by selecting a number a = a_{i} at random, partition the rest into two sequences S_{1} and S_{2} consisting of other elements less than and larger than a_{i}, respectively. Sort S_{1}, S_{2} recursively. Then, concatenate them with a_{i} in between. What’s the expected number of comparisons?

Answer. Let M_{n} be the expected number of comparisons we are looking for. Let X be the number of comparisons. Conditioning on how large a is relative to the rest, we get

M_{n} = E[X] = ∑_{j=1}^{n} E[X | a is the jth least number]\frac{1}{n}

= \frac{1}{n} ∑_{j=1}^{n}((n − 1) + M_{j−1} + M_{n−j})

= \frac{n − 1}{n} + \frac{2}{n} ∑_{j=0}^{n−1} M_{j}

Replacing n by n − 1 we have

∑_{j=0}^{n−2} M_{j} = \frac{n − 1}{2}M_{n−1} − \frac{(n − 1)(n − 2)}{2},

hence,

M_{n} = \frac{2(n − 1)}{n} + \frac{(n + 1)}{n}M_{n−1}.

Devide both sides by n + 1, solve the recurence for the sequence M_{n}/(n + 1) we get M_{n} = θ(n log n).

Example 6.8 (Analysis of Slow Sort). We can also sort a sequence of n distinct numbers a_{1}, …, a_{n} by repeating the following two steps:

1. Check if the sequence is sorted. If it is, then stop.

2. Pick two distinct elements at random and swap them. Go back to step 1.

What’s the expected number of comparisons? (Which is n − 1 times the number of rounds.)

Answer. This problem actually relates to random walks on graphs. We shall come back to this problem later.

Theorem 6.9. Let E be a random event, then we have

P (E) = \begin{cases}∑_{y} P [E|Y = y]P [Y = y] & if Y is discrete \\ ∫_{−∞}^{∞} P [E|Y = y]f_{Y} (y)dy & if Y is continuous \end{cases}  (7)

Example 6.10 (The Best Prize Problem). Suppose you are presented with n boxes of money in a sequence. You don’t know how much money is in each box, but you are told when seeing a new box how the amount of money in the box compared to all the ones you have seen. You will either accept the box or go on. Devise a strategy to attempt the get the box with the most amount of money, and calculate the probability of getting that box.

Solution. One strategy is to reject the first k boxes (0 ≤ k < n) and accept the first box whose amount of money is larger than all the first k boxes’ amounts. We will calculate P_{k}, the probability that this strategy gives the best box, and then optimize P_{k} over all values of k. Do you think that P_{k} → ∞ as n → ∞?

Let E be the event that we succeed and X be the position of the best price. We have

P [E] = ∑_{i=1}^{n} P [E | X = i]P [X = i]

= ∑_{i=k+1}^{n} P [E | X = i]P [X = i]

= \frac{1}{n} ∑_{i=k+1}^{n} P [best of first (i − 1) is among first k]

= \frac{1}{n} ∑_{i=k+1}^{n} \frac{k}{i − 1}

≈ \frac{k}{n} ∫_{k}^{n−1} \frac{1}{x}dx
= \frac{k}{n} log (\frac{n − 1}{k})
≈ \frac{k}{n} log (\frac{n}{k})

As a function of k, this function is optimized at k ≈ n/e, giving a probability 1/e ≈ 0.36788 of success. Surprising?

Example 6.11. Given a complete graph network each of whose link fails with probability p. Given two fixed vertices u and v.

(i) Calculate the P_{2}, probability that there is a path of length 2 available from u to v given that all paths of length 1 are not.

(ii) Repeat part (i) with 1, 2 replaced by 2, 3 respectively.

Answer. (i) There are exactly n − 2 paths of length 2 from u to v, which are edge disjoint.

P_{2} = 1 − P [no path of length 2 available]
= 1 − (1 − (1 − p)^{2})^{n−2}

(ii) TBD.

Exercise 6.12. Given the same assumption as in Example 6.11, calculate the probability that a path of length k is available given that all shorter paths are not.

References

[1] R. Durrett, Essentials of stochastic processes, Springer Texts in Statistics, Springer-Verlag, New York, 1999.

[2] W. Feller, An introduction to probability theory and its applications. Vol. I, John Wiley & Sons Inc., New York, 1968.

[3] , An introduction to probability theory and its applications. Vol. II., John Wiley & Sons Inc., New York, 1971.

[4] S. M. Ross, A first course in probability, Macmillan Co., New York, second ed., 1984.

[5] , Applied probability models with optimization applications, Dover Publications Inc., New York, 1992. Reprint of the 1970 original.

[6] , Stochastic processes, Wiley Series in Probability and Statistics: Probability and Statistics, John Wiley & Sons Inc., New York, second ed., 1996.

[7] , Introduction to probability models, Harcourt/Academic Press, San Diego, CA, seventh ed., 2000.