Introduction

So far this semester we have been looking at various techniques to do probabilistic analysis of algorithms.
For example, under the assumption that candidates arrived in a uniformly at random fashion, we used indicator random variables and linearity of expectation to show that the expected number of candidates we would hire and fire in the Hiring problem was `ln n +O(1)`.
To ensure the random ordering of candidates we gave two algorithms for generating random permutations: sort-by-priorities and random-swaps-to-the-right.
To analyze the first, we argued sorting by distinct priorities produces a random permutation and then that the odds of getting distinct priorities if they were chosen in the range `1` to `n^3` was high. Both arguments made use of conditional probabilities. The main trick in the latter was using the inequality `(1 - a)(1 -b) > 1 -a -b` for `a,b` positive to get a lower bound for an expression of the form `(1 - 1/n^k)^(n-1)`.
The analysis of the second random permutation algorithm was a proof by induction that the odds that it produced a particular `(i-1)`-permutation after `i` swaps had a particular form.
On Wednesday, we then considered a common argument for probabilistic analysis: the birthday problem, which often come up in analyzing things such as hashing.
To compute the odds of two people sharing a birthday we actually bounded the probability of the event that no one shares a birthday. We used the inequality `1 +x le e^x` to bound a product that would be hard to evaluate in terms of e to a sum we could.
Today, we begin by looking at another common class of probabilistic analysis arguments known as ball and bin arguments. They too are useful in analyzing hashing.

Balls and Bins

Consider the process of tossing identical balls into `b` bins.
Tosses will be assumed to be independent and any ball is equally likely to end up in any bin.
The odds of ending up in a particular bin are thus `1/b`.
Successfully landing in a given bin can be viewed as a so-called Bernoulli trial. A Bernoulli trial is an experiment with two possible outcomes.
Balls and bins arguments are useful for modeling a variety of processes in computer science such as hashing.

Ball and Bin Questions

We can ask a variety of questions about the ball tossing process. As a first question, we ask:

How many balls fall in a given bin assuming `n` tosses?

Since tossing a ball into a given bin is a Bernoulli trial, the odds of `k` successes given `n` tosses follows the binomial distribution `b(k;n, x_(i\n\-\b\i\n))`, where `x_(i\n\-\b\i\n)= Pr{\text{ball in bin}} = 1/b`; `x_(n\ot\-i\n\-b\i\n) = Pr{\text(ball not in bin)}= 1-1/b`.
The sample space for the binomial distribution is `{1,2,...}` the possible values for `k`.
The probability of a given event can be determined by considering
`(x_(i\n\-\b\i\n) + x_(n\ot\-i\n\-b\i\n))^n=(1/b +(1-1/b))^n = 1`.
Expanding the left hand side and right hand side gives terms
`((n),(k)) cdot x_(i\n\-\b\i\n)^k cdot x_(n\ot\-i\n\-b\i\n)^(n-k) = ((n),(k)) 1/(b^k) cdot (1 - 1/b)^(n-k)`
which represent the probability of getting `k` balls after `n` tosses.
Let `X` be a random variable whose value is the number of tosses to fall in the bin. Then
`E[X]= sum_(k=1)^n k ((n),(k)) 1/(b^k) cdot (1 - 1/b)^(n-k)`
`= n(1/b) sum_(k=1)^n ((n - 1),(k -1)) 1/(b^(k-1)) cdot (1 - 1/b)^(n-k)`
`= (n/b) sum_(k=0)^(n -1) ((n - 1),(k))1/(b^k) cdot (1 - 1/b)^((n - 1) -k)` notice the sum is `(x_(i\n\-\b\i\n) + x_(n\ot\-i\n\-b\i\n))^(n -1) = 1` so
`=n/b`

Quiz

Which of the following statements is true?

Suppose we wanted to modify the definition of random variable from `X: S -> RR` to `X: S -> W` where `W` is some other set rather than the reals. For expectations of `X` to make sense, we would still need some way to multiply elements of the interval `[0,1]` with elements of `W`.
Our analysis of the hiring problem did not involve calculus as argued.
The Knuth shuffle is easily parallelized.

Second Ball and Bin Question

How many balls must one toss on average, until a given bin contains a ball?

Let `b` still be the number of bins.
Our sample space `{1,2, ...}` where an event `k` is supposed to indicate that the first time one got into the bin was the `k`th toss.
If `X` is the number of trials to succeed `Pr{X=k} = (1- 1/b)^(k-1) cdot 1/b`. This is a geometric distribution.
Before we work out `E[X]` let's recall the following fact about series:
`sum_(k=0)^(\infty) q^k = lim_(n->infty) sum_(k=0)^(n-1) q^k`
`= lim_(n->infty) (1 - q^n)/(1-q)` as `(1 - q)sum_(k=0)^(n-1) q^k = 1 - q^n`
`= 1/(1 - q)`
Given this, `E[X] = `
`= sum_(k=1)^(infty) k (1 - 1/b)^(k-1) cdot (1/b)`
`= 1/b sum_(k=1)^(infty) k (1 - 1/b)^(k-1)` let `q = 1- 1/b`.
`= (1/b) d/(dq) (sum_(k=0)^(infty) q^k) = (1/b) d/(dq) (1/(1 - q)) = (1/b) 1/((1 - q)^2)`
`= (1/b) 1/((1 - (1 - 1/b))^2) = b^2/b = b`.

Third Ball and Bin Question

How many balls must one toss on average, until every bin contains at least one ball?

Call a toss into an empty bin a hit.
We want to know the expected number `n` of tosses to get `b` hits.
Can partition the `n` tosses into stages where the `i`th stage is the number of tosses after the `(i-1)`st hit until the `i`th hit. The first stage is thus just the first toss.
The probability of there being a hit for a given toss in stage `i` is `(b - i+1)/b`
Let `n_i` denote the number of tosses in stage `i`. So the number of tosses to get `b` hits is
`n = sum_(i=1)^b n_i`.
Each random variable `n_i` follows a geometric distribution with probability of success `(b-i+1)/b`. Using the same kind of calculation as the last day `E[n_i]=b/(b-i+1)`.
Using linearity of expectation:
`E[n] = E[sum_(i=1)^b n_i] = sum_(i=1)^b E[n_i] = sum_(i=1)^b b/(b-i+1) = b sum_(i=1)^b 1/i` using the integral bound for 1/i
`= b (ln b + O(1))`.
This problem is also called the coupon collector's problem.

Streaks

Suppose you flip a fair coin n times. What is the longest streak of consecutive heads you expect to see?

The answer is `Theta(log n)`. We will show it is `O(log n)` and give some intuitions on the lower bound -- the full argument is in the book.
What is interesting in the upper bound is we break the expected number into two sums and show how to bound each independently.

Upper Bound on Streaks

Since we are assuming the coin is fair, a coin flip is heads with probability `1/2`.
Let `A_(ik)` be the event that a streak of heads of length at least `k` begins with the `i`th coin flip.
Since coin flips are mutually independent, `Pr{A_(ik)} = 1/(2^k)`
For `k= 2 |~ log n ~|`,
`Pr{A_(i,2 |~ log n ~|)} = 1/2^(2 |~ log n ~|) le 1/(2^(2 log n )) = 1/n^2`.
There are at most `n- 2 |~ log n ~| + 1` positions such a streak can begin. So
`Pr{cup_(i=1)^(n - 2 |~ log n ~| +1) A_(i,2 |~ log n ~|)} le sum_(i=1)^(n - 2 |~ log n ~| +1) 1/(n^2) lt sum_(i=1)^n 1/(n^2) = 1/n` (*).
For example, plugging `n=1000` in the above and noting `2 log n le 20`, the odds of getting a streak of length `20` is less than `1/1000`.
Let `L_j` be the event that the longest streak has length exactly `j`. and let `L` be the length of the longest streak. Thus,
`E[L] = sum_(j = 0)^n j Pr{L_j}`.

Ball and Bins Arguments

Outline