Introduction

On Monday, we were analyzing the expected length of the longest streak of heads in `n` coin tosses of a fair coin.
We let `A_(ik)` be the event that a streak of heads of length at least `k` begins with the `i`th coin flip.
We showed
`Pr{cup_(i=1)^(n - 2 |~ log n ~| +1) A_(i,2 |~ log n ~|)} lt 1/n` (*).
Let `L_j` be the event that the longest streak has length exactly `j`. and let `L` be the length of the longest streak. Thus,
`E[L] = sum_(j = 0)^n j Pr{L_j}`.
We begin today by continuing our analysis of the longest streak.

Bounding `E[L]`

The probability that a streak of heads of length at least `2 |~ log n ~|` is ` sum_(j = 2 |~ log n ~|)^n Pr{L_j}`.
By (*) on the previous slide, `sum_(j = 2 |~ log n ~|)^n Pr{L_j} lt 1/n`.
Notice also that `sum_(j = 0)^(2 |~ log n ~|) Pr{L_j} le sum_(j = 0)^n Pr{L_j} = 1`.
Hence, we have
`E[L] = sum_(j = 0)^n j Pr{L_j}`
`= sum_(j = 0)^(2 |~ log n ~|) j Pr{L_j} + sum_(j = 2 |~ log n ~|)^n j Pr{L_j}`
`lt sum_(j = 0)^(2 |~ log n ~|) (2 |~ log n ~|) Pr{L_j} + sum_(j = 2 |~ log n ~|)^n n Pr{L_j}`
`= (2 |~ log n ~|) sum_(j = 0)^(2 |~ log n ~|)Pr{L_j} + n sum_(j = 2 |~ log n ~|)^n Pr{L_j}`
`lt (2 |~ log n ~|) cdot 1 + n cdot (1/n)`
`= O(log n)`. So we have proven the upper bound.

Using Indicator Variables for Lower Bound Intuition

Let `X_(ik) = I(A_(ik))`. i.e., `X_(ik)` indicates a streak of length at least `k` beginning with the `i`th location.
We can count the total number of such streaks with the random variable
`X = sum_(i=1)^(n - k + 1) X_(ik)`.
The expectation of this is:
`E[X] = E[sum_(i=1)^(n - k + 1) X_(ik)]`
`= sum_(i=1)^(n - k + 1) E[X_(ik)]`
`= sum_(i=1)^(n - k + 1) Pr{A_(ik)}`
`= sum_(i=1)^(n - k + 1) 1/(2^k)`
`= (n - k + 1)/(2^k)`.
If we plug in `1/2 log n` for `k`, we get `E[X] = (n - (1/2 log n) + 1)/(2^(1/2 log n)) = n^(1/2) - O(1)`.
So the expected number of streaks of length at least `1/2 log n` is an increasing function of `n` greater than 1, thus, giving a lower bound on `E[L]` as defined on the earlier slides.

The On-Line Hiring Problem

Suppose we don't want to interview everyone in the hiring problem to find the best candidate.
Suppose further we only want to hire once.

So we follow the following algorithm:

On-Line-Maximum(k,n)
1 bestscore = -infinity 
2 for i := 1 to k
3    if score(i) > bestscore
4        bestscore := score(i)
5 for i := k + 1 to n
6   if score(i) > bestscore then return i
7 return n

I.e., we interview the first `k` applicants and determine the best score, which we will denote bestscore. Then we start interviewing the remaining canididates and hire the first candidate that has a score higher than bestscore.
We want to determine the odds of getting the best candidate as a function of `k`.

Still More on the On-Line hiring Problem

`O_i` only depends on the relative order of the values in the positions `1` through `i - 1`.
`B_i` depends only on whether the value at position `i` is greater than all other positions.
This turns out to imply `B_i` and `O_i` are independent.
So `Pr{S_i} = Pr{B_i cap O_i} = Pr{B_i}Pr{O_i} = (1/n)cdot(k/(i-1)) = k/(n(i-1))`.
- `Pr{B_i} =1/n` since the best value is equally likely to be in any position
- `Pr{O_i} =k/(i-1)` as the maximum value in position `1.. i-1` is equally likely to be in any position. So the odds its in the first `k` of these `i-1` positions, is `k/(i-1)`.
Thus,
`Pr{S} = sum_(i = k+1)^n Pr{S_i} = sum_(i = k+1)^n k/(n(i-1)) = k/n sum_(i = k+1)^n 1/(i-1) = k/n sum_(i = k)^(n-1) 1/i`
Bounding the sum in terms of the integral for `1/i`. We get
`k/n(ln n - ln k) le Pr{S} le k/n (ln (n-1) - ln (k-1) )`
Differentiating with respect to ``k and setting to `0` one can show this is maximized when `k= n/e`.

In-Class Exercise

If we set `k = n/e`, what is the odds we choose the best candidate using the online hiring algorithm?
Give a lower bound on the odds of hiring the worst candidate with this algorithm.
Suppose we just randomly permuted the candidates and hire the first candidate. What would be the odds of hiring the worst candidate doing this?
Please post your solutions to the Feb 9 In-Class Exercise thread.

Multithreaded Algorithms

In most undergrad algorithms classes all of the algorithms are serial algorithms. That is, they are suitable for running on a uniprocessor computer in which only one instruction executes at a time.
For our next topic, we are going to extend our algorithmic model in different ways to encompass parallel algorithms. These algorithms will be designed to run on a multiprocessor computer that permits multiple instructions to execute concurrently.
Most laptops today contain chip multiprocessors. Each such chip contains several cores each of which contains a full-fledged processor that can access a common memory.
This is a shared memory.
It is also possible to have models of parallel processing in which each processor has it own memory. This is called a distributed memory set-up.
To program a shared memory multiprocessor, it is common to use a software abstraction, static threading, that provides "virtual processors", or threads, sharing a common memory.
Since creating and destroying threads is often slow compared to a typical computation, we will assume that our threads "live" for the durations of our computations. Hence, the number of threads is "static".
As directly programming threads can be a pain to do correctly, various concurrency platforms have been proposed that provide a layer of software that coordinates, schedules, and manages the parallel-computing resources.

Dynamic Multithreaded Programming

Dynamic multithreading is an important particular type of concurrency platform.
It contains a scheduler which load balances the computation automatically, simplifying life for the programmer.
It supports nested parallelism and parallel loops.
Nested Parallelism means that a subroutine can by "spawned", allowing the caller to continue while the spawned subroutine is computing results.
Parallel loops are like for loops, except that the iterations of the loop can execute concurrently.

Going from Serial Algorithm to Parallel Algorithm

Recall the Fibonacci numbers (Fibonnaci, Liber Abaci, 1202) are defined by the recurrence:
`F_0 = 0`,
`F_1 = 1`,
`F_i = F_(i-1) + F_(i-2)` for `i ge 2`.
A closed form solution to find the `F_n` can be found using generating functions (DeMoivre, 1730), but let's first consider a simple serial algorithm to compute `F_n`:
```
Fib(n)
1 if n ≤ 1
2     return n
3 else 
    x = Fib(n - 1)
4   y = Fib(n - 2)   
5   return x + y
```
In a moment we will show how to parallelize this, but first let's analyze it.
The runtime of this algorithm, because of the calls to subproblems, satisfies the recurrence: `T(n) = T(n-1) + T(n-2) + Theta(1)`.
We can bound this as `T(n) = Theta(F_n)`. To see the upper bound, let's use the substitution method.
Assume that `T(n') le a F_(n') - b` for all `n' lt n` where `a > 1` and `b >0` are constants. Substituting into our recurrence we obtain:
`T(n) le (a F_(n-1) - b) + (a F_(n-2) - b) + Theta(1)`
`= a(F_(n-1) + F_(n-2)) - b - (b - Theta(1))`
`le a F_n - b`
for suitable large `a` and `b`.

Finish Streaks, Online Hiring Problem, Multithreaded Algorithms

Outline

Introduction

Bounding `E[L]`

Using Indicator Variables for Lower Bound Intuition

The On-Line Hiring Problem

More on the On-line Hiring Problem

Still More on the On-Line hiring Problem

In-Class Exercise

Multithreaded Algorithms

Dynamic Multithreaded Programming

Going from Serial Algorithm to Parallel Algorithm