Outline

• Coin Flip Streaks
• Quiz
• Online Hiring Problem
• Start Threads

Introduction

• So far this semester we have been looking at various techniques to do probabilistic analysis of algorithms.
• For example, under the assumption that candidates arrived in a uniformly at random fashion, we used indicator random variables and linearity of expectation to show that the expected number of candidates we would hire and fire in the Hiring problem was `ln n +O(1)`.
• To ensure the random ordering of candidates we gave two algorithms for generating random permutations: sort-by-priorities and random-swaps-to-the-right.
• To analyze the first, we argued sorting by distinct priorities produces a random permutation and then that the odds of getting distinct priorities if they were chosen in the range `1` to `n^3` was high. Both arguments made use of conditional probabilities. The main trick in the latter was using the inequality `(1 - a)(1 -b) > 1 -a -b` for `a,b` positive to get a lower bound for an expression of the form `(1 - 1/n^k)^(n-1)`.
• The analysis of the second random permutation algorithm was a proof by induction that the odds that it produced a particular `(i-1)`-permutation after `i` swaps had a particular form.
• Last week, we started considering two common arguments for probabilistic analysis: the birthday problem, and ball and bins arguments. Both of these often come up in analysing things such as hashing.
• To compute the odds of two people sharing a birthday we actually bounded the probability of the event that no one shares a birthday. We used the inequality `1 +x le e^x` to bound a product that would be hard to evaluate in terms of e to a sum we could.
• For ball and bin arguments, we answered three questions: How many balls fall in a given bin? (The expected value of the binomial distribution); How many balls must one toss on average, until a given bin contains a ball? (The expected value of the geometric distribution); How many balls must one toss on average, until all bins contains a ball? (We broke this argument into stages where a new bin gets occupied, this is also called the coupon collector problem).
• Today, we begin by analyzing how likely a streak of a given length occurs when tossing coins. Then, we consider an online version of our hiring problem.

Streaks

Suppose you flip a fair coin n times. What is the longest streak of consecutive heads you expect to see?

• The answer is `Theta(log n)`. We will show it is `O(log n)` and give some intuitions on the lower bound -- the full argument is in the book.
• What is interesting in the upper bound is we break the expected number into two sums and show how to bound each independently.

Upper Bound on Streaks

• Since we are assuming the coin is fair, a coin flip is heads with probability `1/2`.
• Let `A_(ik)` be the event that a streak of heads of length at least `k` begins with the `i`th coin flip.
• Since coin flips are mutually independent, `Pr{A_(ik)} = 1/(2^k)`
• For `k= 2 |~ log n ~|`,
  `Pr{A_(i,2 |~ log n ~|)} = 1/2^(2 |~ log n ~|) le 1/(2^(2 log n )) = 1/n^2`.
• There are at most `n- 2 |~ log n ~| + 1` positions a streak can begin. So
  `Pr{cup_(i=1)^(n - 2 |~ log n ~| +1) A_(i,2 |~ log n ~|)} le sum_(i=1)^(n - 2 |~ log n ~| +1) 1/(n^2) lt sum_(i=1)^n 1/(n^2) = 1/n` (*).
• For example, plugging `n=1000` in the above and noting `2 log n le 20`, the odds of getting a streak of length `20` is less than `1/1000`.
• Let `L_j` be the event that the longest streak has length exactly `j`. and let `L` be the length of the longest streak. Thus,
  `E[L] = sum_(j = 0)^n j Pr{L_j}`.

Bounding `E[L]`

• The probability that a streak of heads of length `2 |~ log n ~|` begins anywhere is ` sum_(j = 2 |~ log n ~|)^n Pr{L_j}`.
• By (*) on the previous slide, `sum_(j = 2 |~ log n ~|)^n Pr{L_j} lt 1/n`.
• Notice also that `sum_(j = 0)^(2 |~ log n ~|) Pr{L_j} le sum_(j = 0)^n Pr{L_j} = 1`.
• Hence, we have
  `E[L] = sum_(j = 0)^n j Pr{L_j}`
  `= sum_(j = 0)^(2 |~ log n ~|) j Pr{L_j} + sum_(j = 2 |~ log n ~|)^n j Pr{L_j}`
  `lt sum_(j = 0)^(2 |~ log n ~|) (2 |~ log n ~|) Pr{L_j} + sum_(j = 2 |~ log n ~|)^n n Pr{L_j}`
  `= (2 |~ log n ~|) sum_(j = 0)^(2 |~ log n ~|)Pr{L_j} + n sum_(j = 2 |~ log n ~|)^n Pr{L_j}`
  `lt (2 |~ log n ~|) cdot 1 + n cdot (1/n)`
  `= O(log n)`. So we have proven the upper bound.

Using Indicator Variables for Lower Bound Intuition

• Let `X_(ik) = I(A_(ik))`. i.e., `X_(ik)` indicates a streak of length at least `k` beginning with the `i`th location.
• We can count the total number of such streaks with the random variable
  `X = sum_(i=1)^(n - k + 1) X_(ik)`.
• The expectation of this is:
  `E[X] = E[sum_(i=1)^(n - k + 1) X_(ik)]`
  `= sum_(i=1)^(n - k + 1) E[X_(ik)]`
  `= sum_(i=1)^(n - k + 1) Pr{A_(ik)}`
  `= sum_(i=1)^(n - k + 1) 1/(2^k)`
  `= (n - k + 1)/(2^k)`.
• If we plug in `1/2 log n` for `k`, we get `E[X] = (n - (1/2 log n) + 1)/(2^(1/2 log n)) = n^(1/2) - O(1)`.
• So the expected number of streaks of length at least `1/2 log n` is an increasing function of `n` greater than 1, thus, giving a lower bound. on `E[L]` as defined on previous slides.

Quiz

Which of the following statements is true?

1. Two events `A, B` are independent if `Pr{A cap B} = 0`.
2. Assuming all birthdays are equally likely, and the number of distinct choices of birthday is 365, the probability that two people share a birthday is 1/(365).
3. The Bernoulli distribution was described by the formula `Pr{X=k} = (1- 1/b)^(k-1) cdot 1/b`.

The On-Line Hiring Problem

• Suppose we don't want to interview everyone in the hiring problem to find the best candidate.
• Suppose further we only want to hire once.
• So we follow the following algorithm:

  On-Line-Maximum(k,n)
  1 bestscore = -infinity 
  2 for i := 1 to k
  3    if score(i) > bestscore
  4        bestscore := score(i)
  5 for i := k + 1 to n
  6   if score(i) > bestscore then return i
  7 return n
  

• I.e., we interview the first `k` applicants and determine the best score, which we will denote bestscore. Then we start interviewing the remaining canididates and hire the first candidate that has a score higher than bestscore.
• We want to determine the odds of getting the best candidate as a function of `k`.

More on the On-line Hiring Problem

• Let `M(j) = max_(1 le i le j){mbox(score)(i)}`
• Let `S` be the event of choosing the best qualified applicant.
• Let `S_i` be the event that we choose the `i`th applicant and the applicant is the best qualified. So the `S_i`'s are disjoint events.
• Note we never succeed in choosing the best candidate if `i=1,...,k`. So `Pr{S_i}=0` for these `i`.
• Thus, `Pr{S}= sum_(i = k+1)^n Pr{S_i}`
• Suppose the best qualified applicant was in location `i`. Call this event `B_i`. For this to be true, the algorithm also cannot select any candidate from among `k+1` through `i-1`, which happens only if, for `j` in this range `mbox(score)(j) lt mbox(bestscore)`. So `mbox(score)(k+1),...,mbox(score)(i-1) lt M(k)`. Let `O_i` denote this second event.

Still More on the On-Line hiring Problem

• `O_i` only depends on the relative order of the values in the positions `1` through `i - 1`.
• `B_i` depends only on whether the value at position `i` is greater than all other positions.
• This turns out to imply `B_i` and `O_i` are independent.
• So `Pr{S_i} = Pr{B_i cap O_i} = Pr{B_i}Pr{O_i} = (1/n)cdot(k/(i-1)) = k/(n(i-1))`.
  • `Pr{B_i} =1/n` since the best value is equally likely to be in any position
  • `Pr{O_i} =k/(i-1)` as the maximum value in position `1.. i-1` is equally likely to be in any position. So the odds its in the first `k` of these `i-1` positions, is `k/(i-1)`.
• Thus,
  `Pr{S} = sum_(i = k+1)^n Pr{S_i} = sum_(i = k+1)^n k/(n(i-1)) = k/n sum_(i = k+1)^n 1/(i-1) = k/n sum_(i = k)^(n-1) 1/i`
• Bounding the sum in terms of the integral for `1/i`. We get
  `k/n(ln n - ln k) le Pr{S} le k/n (ln (n-1) - ln (k-1) )`
• Differentiating with respect to ``k and setting to `0` one can show this is maximized when `k= n/e`.

Multithreaded Algorithms

• In most undergrad algorithms classes all of the algorithms are serial algorithms. That is, they are suitable for running on a uniprocessor computer in which only one instruction executes at a time.
• For our next topic, we are going to extend our algorithmic model in different ways to encompass parallel algorithms. These algorithms will be designed to run on a multiprocessor computer that permits multiple instructions to execute concurrently.
• Most laptops today contain chip multiprocessors. Each such chip contains several cores each of which contains a full-fledged processor that can access a common memory.
• This is a shared memory.
• It is also possible to have models of parallel processing in which each processor has it own memory. This is called a distributed memory set-up
• To program a shared memory multiprocessor, it is common to use a software abstraction, static threading, that provides "virtual processors", or threads, sharing a common memory.
• Since creating and destroying threads is often slow compared to a typical computation, we will assume that our threads "live" for the durations of our computations. Hence, the number of threads is "static".
• As directly programming threads can be a pain to do correctly, various concurrency platforms have been proposed that provide a layer of software that coordinates, schedules, and manages the parallel-computing resources.

Dynamic Multithreaded Programming

• Dynamic multithreading is an important particular type of concurrency platform.
• It contains a scheduler which load balances the computation automatically, simplifying life for the programmer.
• It supports nested parallelism and parallel loops.
• Nested Parallelism means that a subroutine can by "spawned", allowing the caller to continue while the spawned subroutine is computing results.
• Parallel loops are like for loops, except that the iterations of the loop can execute concurrently.