Randomized Algorithms

We now begin to investigate the power of the Turing Machines which have the ability to flip coins (which we assume are random and fair).
To make this precise, we first define a notion of probabilistic TM and define some probabilistic complexity classes.
We then consider some randomized algorithms for some common computational problems.

Probabilistic Turing Machines

Definition. A probabilistic Turing machine (PTM) is a TM with two transition functions `delta_0`, `delta_1`. To execute a PTM `M` on input `x`, we choose in each step with probability `1/2` to apply `delta_0` or to apply `delta_1`.

The machine only outputs `1` (Accept) or `0` (Reject). We denote by `M(x)` the random variable corresponding to the value `M` writes at the end of the process. For a function `T: NN -> NN`, we say that `M` runs in time `T(n)` if for any input `x`, `M` halts on `x` within `T(|x|)` steps regardless of the choices `M` makes.

BPTIME and BPP

So after `t` steps, a PTM might be on any one `2^t` computation branches.
The probability that it is on a particular branch is `1/2^t`.
To make a reasonable complexity class out of this model we would like that when a string is in some randomized language, then we will accept on "lot" or branches.
Depending on how we make "lot" precise, we will end up with different complexity classes.
For example:
Definition (BPTIME and BPP). For `T:NN -> NN` and `L subseteq {0, 1}^star` we say that a PTM `M` decides `L` in time `T(n)` if for every `x in {0,1}^star`, `M` halts in `T(|x|)` steps regardless of its random choices and `Pr[M(x) = L(x)] ge 2/3`.
We let `BPTIME(T(n))` be the class of languages decided by PTMs in `O(T(n))` time and define `BPP = cup_c BPTIME(n^c)`.

BPP, an alternative definition

Recall we defined `NP` first with a verifier, later using NDTMs.
You might ask if there is a "verifier" way to define `BPP` (or not -- but I'm going to tell you anyway).
In fact, there is:
Definition. A language `L` is in `BPP` if there exists a p-time TM `M` and a polynomial `p:NN->NN` such that for every `x in {0,1}^star`:
`Pr_(r in_R {0,1}^(p(|x|)))[M(x,r) = L(x)] ge 2/3.`
This equivalent formulation makes it clear that `BPP subseteq EXP` as in time `2^(poly(n))` we can cycle and count all the possible random strings and what `M`'s output on them is.
It is still open if `BPP = NEXP`. It is believed because of progress that has been made on derandomization that `BPP =P`, but people really don't know.

Quiz

Which of the following is known to be true?

There does not exist a size hierarchy theorem for circuits.
`P``/poly` contains undecidable languages.
Meyer's Theorem says that: If `NP subseteq P``/poly`, then `PH = Sigma_2^p`.

Some examples of PTMs

A median of a set on numbers `{a_1, ..., a_n}` is any number `x` such that at least `\lfloor n/2 rfloor` of the `a_i`'s are smaller or equal to `x` and at least `\lfloor n/2 rfloor` of them are larger or equal to `x`.
One way to find the median is to sort them and then output the `lfloor n/2 rfloor`th smallest from this list.
This takes `O(n log n)` time.
Here is a randomized algorithm for this problem (actually finding the `k`th smallest element) that run is expected `O(n)`.
Algorithm FindKthSmallest(`k, a_1, ..., a_n`)
1. Pick a random `i in [n]` and let `x=a_i`.
2. Scan the list `{a_1, .., a_n}` and count the number `m` of `a_i`'s such that `a_i le x`.
3. If `m=k` then output `x`.
4. Otherwise, if `m > k`, then copy to a new list `L` all elements such that `a_i le x` and run FindKthElement(k, L)
5. Otherwise, if `m < k` copy to a new list `H` all elements such that `a_i > x` and run FindKthElement(`k-m, H`).

Runtime of FindKthElement

Claim. For every input `k, a_1, ..., a_n` to FindKthElement, let `T(k, a_1, ..., a_n)` be the expected number of steps the algorithm takes on this input. Let `T(n)` be the maximum of `T(k, a_1, ..., a_n)` over all length `n` inputs. Then `T(n) = O(n)`.

Proof. We can prove by induction that `T(n) < 10cn`. Fix some inputs `k, a_1, ..., a_n`. For every `j in [n]` we choose `x` to be the `j`th smallest of `a_1, ..., a_n` with probability `1/n`, and then we perform either at most `T(j)` steps or `T(n-j)` steps. Thus, we have:
`T(k, a_1, ..., a_n) le cn + 1/n(sum_(j>k)T(j) + sum_(j < k)T(n-j))`
Plugging in our induction hypothesis that `T(j) le 10cj` for `j < n`, gives
`T(k, a_1, ..., a_n) le cn + 10 frac(c)(n)(sum_(j>k) j + sum_(j < k)(n-j)) le cn +10 frac(c)(n)(sum_(j > k) j +kn - sum_(j < k) j)`
Next using `sum_(j > k) j le (n(n-k))/2` and `sum_(j < k)j ge frac(k^2)(2)(1 - o(1)) ge k^2/2.5`, we get
`T(k, a_1, ..., a_n) le cn + 10 frac(c)(n)((n(n-k))/2 +kn - k^2/2.5) = cn + 10 frac(c)(n)(n^2/2 + (kn)/2 - k^2/2.5)
Considering separately the cases `k < n/2` and `k> n/2` we can see this last equation is always less than `cn + (10c)/n (9n^2)/10 = 10cn`.

Randomized Complexity Classes