Outline

• Hierarchy Theorems
• In-Class Exercise
• Complete Problems
• More Reductions

Hierarchy Theorems

• So far, we have used diagonalization for three results: We showed `P(NN)` is uncountable, we showed that there is a set that is not recursively enumerable (i.e., not Turing recognizable), and we showed that the set `A_{TM}` is not decidable.
• To see the diagonalization argument one more time, we next turn to the question of when can we show a language is in `\mbox{TIME}(f)` but not in `\mbox{TIME}(g)`? Here `g` is a slower growing proper function than `f`.
• By proper, we mean `f` is nondecreasing and there is a deterministic TM `M_f` that when started with `x` on the input tape, runs for `t=O(|x| + f(|x|))` steps, uses at most `O(f(|x|))` space, and outputs to an output tape the string `I^{f(|x|)}`. We measure space disregarding the input and output tapes: The input tape is assumed to be read-only and we assume we also have a dedicated, write-only output tape.
• For example, a constant `c`, `n`, `|~ log n~|` are all proper complexity functions. One can show that if `f` and `g` are proper so are: `f+g`, `f cdot g`, and `2^g`.
• The result we will give illustrates a common heuristic: often we can take arguments about general Turing machine computations and then "scale them down" to arguments about specific time or space complexity classes.
• To carry out our argument, we are going to look at a clocked-based variant of the `A_{TM}` problem. For a proper function`f(n)`, define: `A_{f} = { langle M, x rangle | \mbox{ M accepts input x after at most f(|x|) steps}}`

When Can We Recognize `A_f`?

Lemma. `A_{f} \in \mbox{TIME}((f(n))^3 )`.

Proof. We will use a variation on our universal Turing machine from Monday to prove this. Our machine, `U_f`, will have `4` tapes. Let `n= | langle M,x rangle|`. First, `U_f` uses `M_f` to write an "alarm clock" amount of time `I^{f(n)}` on the fourth tape. The fourth head is then rewound. Using the input `langle M, x rangle`, the second tape is initialized to encode the start state `s` of `M`, `langle M rangle` is copied to the third tape and converted into a 1-tape machine using our earlier simulation (this machine executes one step of the original machine in at most `O(f(n))` steps and we could linear speed this up to `f(n)` ). The first tape is set up with just `x` on it. Doing all this, takes `O(f(n))` time. The modified `M` is then simulated step by step. This involves:

1. Comparing the current state, and current head position of tape `1` against the transition function of `M` on tape `3` until a match its found,
2. applying the appropriate change to the first tape and updating the current state on the second tape, and
3. advancing the alarm clock tape.

Simulating one step of the modified `M` take `O(f(n))` steps on `U_f`. So simulating one step of the original takes `O(f(n)^2)` steps on `U_f`. We simulate exactly `f(n)` steps of `M` by using our clock and determine if we have accepted by then. So the total time is `O(f(n)^3)`, which gives `A_f` in `\mbox{TIME}((f(n))^3 )` as desired.

When Can't We Recognize `A_f`?

Lemma. `A_{f} !in \mbox{TIME}( f( lfloor n/2 rfloor ))`.

Proof. Suppose `M_{A_f}` decides `A_f` in time `f(lfloor n/2 rfloor)`. Let `D_f` be the diagonalizing machine that computes:
`D_{f}( langle M rangle)`: if `M_{A_f}( langle M, M rangle)` accepts then "reject" else "accept".
This machine on input `langle M rangle` runs in the same time as `M_{A_f}` on input `langle M, M rangle` (which is twice as long `+ 1` as `langle M rangle`). That is, it runs in time `f(lfloor (2n+1)/2 rfloor) = f(n)`. Does `D_f` accept its own description? Suppose `D_{f} ( langle D_f rangle)` accepts. By our definition of `D_f` this is done in at most `f(n)` steps. This means `M_{A_f}( langle D_f, D_f rangle)` rejects, which means `\langle D_f, D_f rangle !in A_f`. This means `D_f` on input `D_f` did not accept in `f(n)` steps, contradicting our starting assumption. Starting with `D_{f}(langle D_f rangle)` rejecting, we can follow a similar chain of reasoning to get a contradiction as well.

Hierarchy Theorems

• Using the last two lemmas together one can show:
  The Deterministic Time Hierarchy Theorem. If `f(n) ge n` is a proper complexity function, then the class `\mbox{TIME}(f(n))` is strictly contained in `\mbox{TIME}(f(2n+1)^3 )`.
• This can be improved to `\mbox{TIME}(f(n) cdot log^2 f(n))`.
• A similar technique as above can be used to show:
  The Space Hierarchy Theorem. If `f(n)` is a proper function, then `\mbox{SPACE}(f(n))` is a proper subset of `SPACE(f(n)log f(n))`.
• These results are due to Hartmanis and Stearns 1965.
• For nondeterministic time, a hierarchy theorem was first proven by Cook 1971 and later improved by Seiferas, Fischer and Meyer 1978. A relatively easy to prove variant due to Fortnow and Santhanam 2010 is:
  The Nondeterministic Time Hierarchy Theorem. If `f` and `g` are proper complexity functions and suppose `f(n+1)=o(g(n))` then `\mbox{NTIME}(f(n))` is strictly contained in `\mbox{NTIME}(g(n))`.

Hierarchy Theorem Implications and Open Problems

• From the hierarchy theorems, it follows that:
  1. `\mbox{P} := \cup _k \mbox{TIME}(n^k) ⊊ \mbox{EXP} := \cup _k \mbox{TIME}(2^{n^k})`.
  2. `\mbox{NP} := \cup _k \mbox{NTIME}(n^k) ⊊ \mbox{NEXP} := \cup _k \mbox{NTIME}(2^{n^k})`.
  3. `\mbox{L} := \mbox{SPACE}(\log n) ⊊ \mbox{PSPACE} := \cup _k \mbox{SPACE}(n^k)`.
• Some open problems are: `\mbox{L} = \mbox{P}`?, `\mbox{P} = \mbox{NP}`?, `\mbox{P} = \mbox{PSPACE}`?, `\mbox{PSPACE} = \mbox{EXP}`?, and `\mbox{EXP} = \mbox{NEXP}`?. They can't all be equal.
• It is known that `\mbox{TIME}(n) ne \mbox{NTIME}(n)` (Paul and Reischuk (1980), Paul, Pippenger, Szemeredi, Trotter (1983)) (A slightly stronger result due to Santhanam 2001 is also known) and that `\cup_k \mbox{TIME}(2_k(n)) = \cup_k mbox{NTIME}(2_k(n))` where `2_1(n) := 2^n, 2_{k+1}(n) := 2^{2_k(n)}`.

In-Class Exercise

• Carefully, show why `\mbox{P} ⊊ \mbox{EXP}` follows from the time hierarchy theorem.
• Post your solution to the Apr 29 In-Class Exercise Thread.

A Specific Nonrecursively Enumerable Language I

• We gave a counting argument to show a non recursively enumerable language must exist - our argument though doesn't give a specific example language.
• We'll use the next theorem to give an example.
• First, call a language co-recursively enumerable if its complement is recursively enumerable.
  Theorem. A language is decidable iff it is recursively enumerable and co-recursively enumerable.
  Proof. Suppose `L` is decidable by `M`. Then it is also r.e. Further, let `bar M` be the machine which rejects when `M` accepts and accepts when `M` rejects. Then `\bar M` recognizes the complement of `L`, so `L` is co-r.e. On the other hand, suppose `L'` is Turing recognized by `M'` and co-Turing recognized by `M''`. Then let `D` be the machine which on input `w` simulates each of `M'` and `M''` first for `1` step, then for `2` steps, etc. If `M'` ever accepts the `D` accepts and if `M''` ever accepts then `D` rejects. Since a string is either in `L'` or not, one of these two machines must accept eventually, and so then `D` will decide that string.

A Specific Non-Recursively Enumerable Language II

Corollary. `\bar{A}_{TM}` is not r.e.

Proof. We proved in an earlier lecture `A_{TM}` is recursively enumerable. So if `\bar A_{TM}` were r.e., then `A_{TM}` would be decidable giving a contradiction with our result that `A_{TM}` is undecidable.

Reducibility

• We next consider what other problems are undecidable.
• Our approach to showing languages are undecidable will be to use a notion called reducibility.
• A reduction `r` is a mapping from possible inputs (instances) `I_A` to a problem `A` to instances of problem `B`, with the property that `I_A in A` if and only if `r(I_A) in B`.
• If the reduction can be computed by a TM, it is called a Turing reduction. Such reductions were first defined by Post in 1944.
• If the reduction can be computed by a TM in polynomial time, it is called a Karp reduction. These reductions were first defined by Karp 1972.
• Turing reductions have the property that if we have a Turing reduction from `A` to `B`, and `B` is decidable, then `A` will be too. Conversely, if `A` is not decidable, then `B` also won't be decidable.
• We write `A le_m B` to say `A` is Turing reducible to `B` and `A le_m^p B` to say `A` is Karp reducible to `B`.
• `m` is for many-one (because two instances of A might be mapped to the same instance of B). Sometimes this term is used for both kinds of reductions. `p` is for polynomial time.

Example

Let `\mbox{HALT}_{TM} = { langle M, w rangle | \mbox{M is a TM and M halts on input w}}`.

Theorem. `\mbox{HALT}_{TM}` is undecidable.

Proof. Suppose `H` decides `\mbox{HALT}_{TM}` . From `H` we can construct a machine `S` which decides `A_{TM}` as follows:

`S =`" On input `langle M, w rangle` an encoding of a TM `M` and a string `w`:

1. We build a new string `langle M',w rangle` where `M'` is a machine which simulates `M` until `M` halts (if it does) and if `M` accepts `M'` accepts. Otherwise, if `M` rejects than `M'` moves right forever, one square at a time. The map `langle M,w rangle -> langle M', w rangle` is well-defined enough that it can be computed by a TM. This is our reduction.
2. We then ask our decider for `H` if `langle M', w rangle` is in `H`? If `H` accepts we accept and if `H` rejects we reject."

Since the only way `M'` on input `w` halts is if `M` accepts `w`, we know `langle M,w rangle` is in `A_{TM}` iff `langle M', w rangle` is in `\mbox{HALT}_{TM}`. So if `H` was a decision procedure for `\mbox{HALT}_{TM}`, then `S` would be a decision procedure for `A_{TM}`. As we know there is no decision procedure for `A_{TM}` we know that the supposed `H` can't exist.

Complete Problems

• A language `L` is said to Turing-hard (resp. Karp-hard) for a class of languages `C` if every `L'` in `C` Turing-reduces (Karp-reduces) to `L`.
• If, in addition, `L` is in `C`, then `L` is called Turing-complete (resp. Karp-complete) for `C`.
• Basically, the notion of completeness makes more precise the notion of being the hardest kind of problem in a class of languages.

Complete r.e. and NP problems

Theorem.

1. `A_{TM}` is Turing-complete for r.e.
2. `A_{\mbox{clocked TM}} := {\langle M, w, 1^t rangle | \mbox{M is a nondeterministic machine that accepts w in at most t steps} }` is Karp-complete for NP.

Proof. For (1), we already have show that `A_{TM}` is in r.e. (the set of recursively enumerable languages). Suppose `L` in r.e. is recognized by a machine `M`. The function `w -> langle M, w rangle` is Turing computable and `w in L` iff `langle M, w rangle in A_{TM}`. Therefore, `A_{TM}` is Turing-complete. For (2), we first show `A_{\mbox{clocked TM}}` is in NP. An nondeterministic algorithm to recognize this language is as follows: On input `\langle M, w, 1^t rangle`, nondeterministically guess a string `v` of length `t^2`, then verify that `v` codes a sequence of configurations of `M` of length `t`, the first being the start configuration of `M` on `w` and such that the `i+1`st configuration follows from the `i`th. If it does, and `M` accepts `w` then accept, else reject. Now suppose `L` is in NP. Then it can be decided by some nondeterministic machine `M` in time `p(n)` for some polynomial `p`. The function `w -> langle M, w, 1^{p(|w|)} rangle` is polynomial time computable and gives a reduction from `L` to `A_{\mbox{clocked TM}}`.