Outline

• Error Reduction
• `RTime(T(n))`
• `BPP subseteq P``/poly`
• `BPP subseteq Sigma_2^p cap Pi_p^2`

RTIME and RP

• In our 2SAT algorithm from Monday, if we output 1, satisfiable, we were always right, but if we output 0, unsatisfiable, we were sometimes wrong.
• We would like a complexity class that captures this notion of randomized algorithm as it doesn't fit exactly in the BPP framework.

Definition. `RTIME(T(n))` contains every language `L` for which there is a probabilistic TM `M` running in `T(n)` time such that:
`x in L => Pr_r[M(x, r) = 1] ge 2/3`
`x !in L => Pr_r[M(x, r) = 1] = 0`
We define `RP = cup_c RTIME(n^c)`.

• In a couple slides we'll give an argument that when `x in L` it suffices that the probability is greater than 0.
• So our 2SAT algorithm was a coRP algorithm. Sometimes BPP-style randomness is called Las Vegas randomness, and RP-style is called Monte-Carlo.

`ZTIME(T(n))` and relationships

RP has "one-sided error"; one might ask for randomized algorithms that have "zero-sided error":

Definition. The class `ZTIME(T(n))` contains all the languages `L` for which there is a machine `M` that run in an expected-time `O(T(n))` such that for every input `x`, whenever `M` halts on `x`, the output `M(x)` it produces is exactly `L(x)`.

We define `ZPP = cup_c ZTIME(n^c)`.

One can show:
`ZPP = RP cap coRP`
`RP subseteq BPP`
`coRP subseteq BPP`

ZPP can also be formulated as those languages that cab be decided without error using randomness in expected polynomial time. The language version of our FindKthSmallest algorithm is a ZPP algorithm.

Error Reduction -- Chernoff Bounds

• When we defined BPP and RP we use `2/3`. Why?
• Would other choices bounded away from 1/2 work? Say 3/4?
• The answer is "Yes". For example if the bound was `3/4` we can run a `2/3` algorithm repeatedly and take the majority output to reduce the error.
• The main tool to show that this works is Chernoff Bounds:
  Lemma (Chernoff). Suppose `X_1,...,X_n` are independent random variables taking the values `1` and `0` with probabilities `p` and `1-p`. Let `X = sum_(i=1)^n X_i`. Then for all `0 le c le 1`, `Pr[X ge (1+c)pn] le e^(-(c^2pn)/2)`.

Proof of Chernoff Bounds

If t is a positive real number, then
`Pr[X ge (1+c)pn]= Pr[e^(tX) ge e^(t(1+c)pn)]` (*)
By Markov's Inequality,
`Pr[e^(tX) > k cdot E(e^(tX))] le 1/k` for any real `k > 0`.
Taking `k=e^(t(1+c)pn)/[E(e^(tX))]` and using (*) gives
`Pr[X ge (1+c)pn] le [E(e^(tX))]cdot e^(-t(1+c)pn)`. (**)
Since `X= sum_(i=1)^n X_i`, we have `E(e^(tX) )=[E(e^(tX_1))]^n` which in turn equals `(1 + p(e^t-1))^n`. Substituting this into (**) gives:
`le e^(-t(1+c)pn) cdot e^(pn (e^t-1))`, since `(1+a)^n le e^(an)`. Take `t=ln(1+c)` to get `Pr[X ge (1+c)pn] le e^(pn(c-(1+c)ln(1+c)))`. Taylor expanding `ln(1+c)` as `c - c^2/2 + ...` and substituting gives the result. i/e.,
`e^(pn(c-(1+c)ln(1+c))) le e^(pn(c-(1+c)(c- c^2/2 +c^3/3 + ...))) le e^(-(c^2 pn)/2)`

Corollary. If `p=1/2 + epsilon` for some `epsilon > 0`, then the probability that `sum_(i=1)^n X_ i le n/2` is at most `e^(-epsilon^2n/4)`.

Proof. Take `c = epsilon/(1/2+ epsilon)`. Q.E.D.

Robustness of BPP

• Notice if we had chosen `1/2+epsilon` in the definition for some `0 < epsilon <1/4`, in our definition, then it would not have made a difference.
• Let `k = [(4ln 2)/(epsilon^2)]`. Run the machine that accepts `L` according to the probabilities `1/2+epsilon` a total of `2k+1` times and accept the majority of the outcomes.
• So by Chernoff bounds, the odds that the majority vote of these runs is wrong is at most `e^(-epsilon^2(2k+1)/4) le e^(-epsilon^2(2k)/4) = e^(-(8ln2)/4)= 2^(-2) = 1/4`.
• Thus, we will accept with the `3/4`'s probability if its in the languages and reject with `3/4` probability if its not.
• By taking `k = [(2(n + 1)ln 2)/(epsilon^2)]`, we can reduce the error rate to `e^(-epsilon^2(2k)/4) = e^(-(4(n+1)ln2)/4)= 2^(-n-1)`.
• We can even allow `\epsilon` to vary with `n`. For example, if `epsilon = 1/n^c`, then the number of repetitions will still be polynomial and we can still achieve error reduction to `2^(-n-1)`.

`BPP subseteq P``/poly`

The next result shows that error reduction can be used to derandomized BPP to a certain degree:

Theorem. `BPP subseteq P``/poly`. (Bennett-Gill, 1981)

Suppose `L in BPP`, then there exists a TM `M` that on inputs of size `n` uses `m` random bits and such that for every `x in {0,1}^n`, `Pr_r[M(x,r) ne L(x)] le 2^(-n-1)`. Say that a string `r in {0,1}^m` is bad for an input `x in{0,1}^n` if `M(x,r) ne L(x)` and otherwise call `r` good for x. For every `x`, at most `2^m/(2^(n+1))` strings are bad for `x`. Adding over all `x in {0,1}^n`, there are at most `2^n(2^m/(2^(n+1))) = 2^m/2` strings that are bad some some `x`. In particular, there exists a string `r_0 in {0,1}^m` that is good for every `x in {0,1}^n`. We can hardwire such a string `r_0` to obtain a circuit `C` (of size at most quadratic in the running time of `M`) that on input `x` outputs `M(x,r_0)`. The circuit `C` will satisfy `C(x) = L(x)` for every `x in {0,1}^n`.

Corollary. If `NP subseteq BPP` then the polynomial hierarchy collapses to `\Sigma_2^p`.

Proof. From (Karp Lipton 1980) we know if `NP subseteq P`/`poly` then `PH=\Sigma_2^p`, so the result follow from `BPP subseteq P``/poly`.

In-Class Exercise

Bulma, at Capsule Corporation, has developed a new BPP compression algorithm. She would like to implement it in hardware for her company's new line of storage capsules. These capsules might feed this algorithm with inputs of size `10^12` bits. In a paragraph describe how you would implement her algorithm in silicon.

Post your solution to the Apr 26 In-Class Exercise Thread.

`BPP subseteq Sigma_2^p cap Pi_p^2`

Theorem. `BPP subseteq Sigma_2^p cap Pi_p^2`. (Sipser-Gacs-Lautemann, 1983).

Proof. It suffices to show `BPP subseteq Sigma_2^p` as `BPP` is closed under complement.

Suppose `L in BPP`. Then there is a TM `M` for `L` that on inputs of length `n` use `m = poly(n)` random bits and satisfies:
`x in L => Pr_r[M(x,r) mbox( accepts )] ge 1 - 2^(-n)`
`x !in L => Pr_r[M(x,r) mbox( accepts )] le 2^(-n)`
For `x in {0, 1}^n`, let `S_x` denote the set of `r`'s for which `M` accepts the input pair `langle x,r rangle`. Then either `|S_x| ge (1-2^(-n))2^m` or `|S_x| le 2^(-n)2^m` depending on whether `x` is in the language or not.

For set `S subseteq {0,1}^m` and a string `u in {0,1}^m` we denote by `S + u` the "shift" of the set `S` by `u`: `S + u = {x+u | x in S}` where `+` denote vector addition mod 2. Let `k = |~ m / n~|+1`

Claim. For every set `S subseteq {0,1}^m` with `|S| le 2^(m-n)` and `k` vectors `u_1, ... u_k`, `cup_(i=1)^k(S+u_i) ne {0,1}^m`.

Proof. Since `|S + u_i| = |S|`, by the union bound we have `|cup_(i=1)^k(S+u_i)| le k |S| < 2^m` asymptotically. The last inequality is because `m = poly(n)` so is bounded by `n^j` for some `j` and so
`k|S| = (|~ m / n~|+1)|S| < (n^(j-1) + 1)2^(n^j - n) = 2^(n^j + log(n^(j-1) + 1) - n) < 2^(n^j)=2^m`
As polylog in `n` is asymptotically small in `n`.

`BPP subseteq Sigma_2^p cap Pi_p^2` proof cont'd

Claim. For every set `S subseteq {0, 1}^m` with `|S| ge (1 - 2^(-n))2^m`, there exist `u_1, ..., u_k` such that `|cup_(i=1)^k(S+u_i)| = {0,1}^m`.

Proof. We claim if `u_1, ... u_k` are chosen uniformly at random `Pr[cup_(i=1)^k(S+u_i) = {0, 1}^m] > 0`. So some choice of `u_i`'s must work. For `r in {0, 1}^m`, let `B_r` denote the "bad event" that `r` is not in `cup_(i=1)^k(S+u_i)`. It suffices to prove that `Pr[exists_(r in {0,1}^m)B_r] < 1`, which will follow from the union bound if we can show for every `r` that `Pr[B_r] < 2^(-m)`. Notice `B_r = cap_(i in [k]) B_r^i` where `B_r^i` is the event that `r !in S +u _i`, or equivalently, that `r + u_i !in S`. Yet, `r + u_i` is a uniform element in `{0,1}^m`, and so it will be in `S` with probability at least `1-2^(-n)`. Furthermore, the events `B_r^i` are independent for different `i`'s implying:
`Pr[B_r] = Pr[B_r^i]^k le 2^(-nk) < 2^(-m)`.

`BPP subseteq Sigma_2^p cap Pi_p^2` proof conclusion

Together the two claims show that `x in L` iff the following statements is true:
`exists u_1, ... u_k in {0,1}^m forall r in {0,1}^m r in cup_(i=1)^k(S_x+u_i)`
or, equivalently,
`exists u_1, ... u_k in {0,1}^m forall r in {0,1}^m vv_(i=1)^k M(x, r oplus u_i) mbox( accepts)`.
Notice this is a `Sigma_2^p` check.