Outline

• Hierarchy Theorems
• HW Problem
• co-r.e. (co-c.e.)
• Reductions

Hierarchy Theorems

• So far, we have used diagonalization for three results: We showed `P(NN)` is uncountable, we showed that there is a set that is not recursively enumerable (i.e., not Turing recognizable), and we showed that the set `A_{TM}` is not decidable.
• To see the diagonalization argument one more time, we next turn to the question of when can we show a language is in `\mbox{TIME}(f)` but not in `\mbox{TIME}(g)`? Here `g` is a slower growing proper function than `f`.
• By proper, we mean `f` is nondecreasing and there is a deterministic TM `M_f` that when started with `x` on the input tape, runs for `t=O(|x| + f(|x|))` steps, uses at most `O(f(|x|))` space, and outputs to an output tape the string `I^{f(|x|)}`. We measure space disregarding the input and output tapes: The input tape is assumed to be read-only and we assume we also have a dedicated, write-only output tape.
• For example, a constant `c`, `n`, `|~ log n~|` are all proper complexity functions. One can show that is `f` and `g` are proper so are: `f+g`, `f cdot g`, and `2^g`.
• The result we will give illustrates a common heuristic: often we can take arguments about general Turing machine computations and then "scale them down" to arguments about specific time or space complexity classes.
• To carry out our argument, we are going to look at clocked-based variant of the `A_{TM}` problem. For a proper function`f(n)`, define: `A_{f} = { langle M, x rangle | \mbox{ M accepts input x after at most f(|x|) steps}}`

When Can We Recognize `A_f`?

Lemma. `A_{f} \in \mbox{TIME}((f(n))^3 )`.

Proof. We will use a variation on our universal Turing machine from last week to prove this. Our machine, `U_f`, will have `4` tapes. Let `n= | langle M,x rangle|`. First, `U_f` uses `M_f` to write an alarm clock amount of time `I^{f(n)}` on the fourth tape. The fourth head is then rewound. Using the input `langle M, x rangle`, the second tape is initialized to encode the start state `s` of `M`, `langle M rangle` is copied to the third tape and converted into a 1-tape machine using our earlier simulation (this machine executes one step of the original machine in at most `O(f(n))` steps and we could linear speed this up to `f(n)` ). The first tape is set up with just `x` on it. Doing all this, takes `O(f(n))` time. The modified `M` is then simulated step by step. This involves:

1. Comparing the current state, and current head position of tape `1` against the transition function of `M` on tape `3` until a match its found,
2. applying the appropriate change to the first tape and updating the current state on the second tape, and
3. advancing the alarm clock tape.

Simulating one step of the modified `M` take `O(f(n))` steps on `U_f`. So simulating one step of the original takes `O(f(n)^2)` steps on `U_f`. We simulate exactly `f(n)` steps of `M` by using our clock and determine if we have accepted by then. So the total time is `O(f(n)^3)`, which gives `A_f` in `\mbox{TIME}((f(n))^3 )` as desired.

When Can't We Recognize `A_f`?

Lemma. `A_{f} !in \mbox{TIME}( f( lfloor n/2 rfloor ))`.

Proof. Suppose `M_{A_f}` decides `A_f` in time `f(lfloor n/2 rfloor)`. Let `D_f` be the diagonalizing machine that computes:
`D_{f}( langle M rangle)`: if `M_{A_f}( langle M, M rangle)` accepts then reject else accept.
This machine on input `langle M rangle` runs in the same time as `M_{A_f}` on input `langle M, M rangle` (which is twice as long `+ 1` as `langle M rangle`). That is, it runs in time `f(lfloor (2n+1)/2 rfloor) = f(n)`. Does `D_f` accept its own description? Suppose `D_{f} ( langle D_f rangle)` accepts. By our definition of `D_f` this is done in at most `f(n)` steps. This means `M_{A_f}( langle D_f, D_f rangle)` rejects, which means `\langle D_f, D_f rangle !in A_f`. This means `D_f` on input `D_f` did not accept in `f(n)` steps, contradicting our starting assumption. Starting with `D_{f}(langle D_f rangle)` rejecting, we can follow a similar chain of reasoning to get a contradiction as well.

Hierarchy Theorems

• Using the last two lemmas together one can show:
  The Deterministic Time Hierarchy Theorem. If `f(n) ge n` is a proper complexity function, then the class `\mbox{TIME}(f(n))` is strictly contained in `\mbox{TIME}(f(2n+1)^3 )`.
• This can be improved to `\mbox{TIME}(f(n) cdot log^2 f(n))`.
• A similar technique as above can be used to show:
  The Space Hierarchy Theorem. If `f(n)` is a proper function, then `\mbox{SPACE}(f(n))` is a proper subset of `SPACE(f(n)log f(n))`.
• These results are due to Hartmanis and Stearns 1965.
• For nondeterministic time, hierarchy theorem was first proven by Cook 1971 and later improved by Seiferas, Fischer and Meyer 1978. A relatively easy to prove variant due to Fortnow and Santhanam 2010 is:
  The Nondeterministic Time Hierarchy Theorem. If `f` and `g` are proper complexity functions and suppose `f(n+1)=o(g(n))` then `\mbox{NTIME}(f(n))` is strictly contained in `\mbox{NTIME}(g(n))`.

Hierarchy Theorem Implications and Open Problems

• From the hierarchy theorems, it follows that:
  1. `\mbox{P} := \cup _k \mbox{TIME}(n^k) ⊊ \mbox{EXP} := \cup _k \mbox{TIME}(2^{n^k})`.
  2. `\mbox{NP} := \cup _k \mbox{NTIME}(n^k) ⊊ \mbox{NEXP} := \cup _k \mbox{NTIME}(2^{n^k})`.
  3. `\mbox{L} := \mbox{SPACE}(\log n) ⊊ \mbox{PSPACE} := \cup _k \mbox{SPACE}(n^k)`.
• Some open problems are: `\mbox{L} = \mbox{P}`?, `\mbox{P} = \mbox{NP}`?, `\mbox{P} = \mbox{PSPACE}`?, `\mbox{PSPACE} = \mbox{EXP}`?, and `\mbox{EXP} = \mbox{NEXP}`?. They can't all be equal.
• It is known that `\mbox{TIME}(n) ne \mbox{NTIME}(n)` (Paul and Reischuk (1980), Paul, Pippenger, Szemeredi, Trotter (1983)) (A slightly stronger result due to Santhanam 2001 is also known) and that `\cup_k \mbox{TIME}(2_k(n)) = \cup_k mbox{NTIME}(2_k(n))` where `2_1(n) := 2^n, 2_{k+1}(n) := 2^{2_k(n)}`.

Homework Problems

Problem 1. Give an offline TM which when started with `x#y` on its read only input tape outputs `lfloor x/y rfloor` on its output tape. Assume `{0,1, #}` is the input alphabet and numbers are in binary (lead zeros not allowed). On bad inputs your TM should halt, with `#` on the tape. If `y` is `0` output `#`. If `x` is `0` and `y` is a nonzero integer the output should be `0`.

Answer. We give a description of our TM at a level such that each of the submachines should be easily spec'd out so that you could do problems (2) and (3). We assume the tape alphabet has symbols `\ul{0}, \ul{1}, \ul{square}`. Let `s` be the start state of our machine, `\h\a\l\t` be a halt state. If the machine starts on a blank space, it writes `#` to the output tape and halts. i.e., `delta(s, square, square) = (\h\a\l\t, square, #, R, R)`. Similarly, `delta(s, #, square) = (\h\a\l\t, #, #, R, R)` The first sub-machine scans the input tape left-to-right until the # symbol is observed and moves 1 square beyond this. (On the work tape it can be moving right as well). It remembers as it did this if it ever saw a `1` or if it saw lead `0`'s on the input tape. If the machine is reading a blank square it halts and writes `#`. If then continues to the right until it sees a `1` or lead `0`'s or a blank square. If it never sees a `1` or ever sees lead 0's it writes `#` to the work tape and halt. Otherwise, if before the `#` on the work tape it never saw a `1`, it writes `0` and halts. If we haven't halted, we have now verified the input is in the correct format and both `x` and `y` are non-zero. We rewind the input tape to its left hand side. We scan left-to-right and copy `y` to the work tape. We then write #0 after `y` on the work tape so that the work tape looks like `y#0`. We next move to the `#` on both tapes. We then scan right-to-left on both tapes, until we get to a blank square on the input tape. If we are not reading a square on the work tape we erase everything before the # on the work tape and halt. If as we are scanning right-to-left on both tapes, we read a blank on the work tape, we write a 0. After this phase, we know `y` is less than `x` and has been padded with lead `0`'s to be of the same length. We next repeatedly, subtract `y` from `x` in passes. After the `m`th pass, the work tapes configuration will look like: `x-y cdot m#y#m `. To do the subtractions we use the underscore versions of symbols to keep track of which pair of squares we are subtracting. If the term before the first # is all 0's then, we erase up to the second sharp on the tape and output `m` and halt.

Problem 4. Give a nondeterministic Turing machine which recognizes the language of binary strings of integers `n` such that `n` is a product of integers `x` and `y` both of which are greater than `1`. You can give a high level description of your NTM.

Answer. We describe a 6-tape NTM to do this. We assume the input alphabet is `{0, 1}`. We assume the input is big-endian. On input `w`, the machine first runs a program which at each step reads an input square, if this square is not a blank space, it nondeterministically chooses to write a 0 or a 1 on the 2nd tape, and it independently, nondeterministically chooses to write a `0` or a `1` on the third tape. For the the 1st, 2nd, and 3rd tapes it moves right, all other tapes are assumed to stay put. Once it sees a blank, it rewinds tapes 1,2,3. After this pass the machine operates deterministically. So far the computation has been in linear time. Let `x` be the contents of the 2nd tape, `y` be the contents of the third tape. These numbers may have lead `0`'s. We now check to make sure both of these numbers are greater than `1`, by doing a single left right scan of the numbers verify that there was a position other than the low order position that was a `1`. If both numbers are not greater than one it halts and rejects. We next describe how to multiply `x` and `y` together to verify whether they equal `w`. We use the 4th tape to hold the total so far, the 5th tape to hold the next row of the long multiplication, and the 6th tape to hold a temporary total. Initialize tape 4 to 0. We then do the following steps until tape 3 is a string of `0`'s.

1. Rewind all tapes to the left hand size of their values.
2. Scan left-to-right on tape 2 and tape 5 until we see a space on both tapes. As we move right copy the value of the square of tape 2 to tape 5. Rewind tape 2 to its left hand side. Put tape 5 on the blank square next to its rightmost non-blank square. This copies the contents of tape 2 to 5.
3. Scan left-to-right on tape 3, to see if there are any 1 squares. If not, break out of this sequence of steps.
4. Now on tape 3, scan right-to-left, if one sees a 0 move left on tape 3, move right on tape 5 writing a 0 on tape 5. When one sees the first 1 on tape 3, stop, write a 0 over the 1 on tape 3, and add the contents of tape 5 to tape 4 with the output going to tape 6. This last thing can be done using a right-to-left scan of tapes 5 and 4 using carry save addition.
5. Copy tape 6 to tape 4 and erase tape 6.
6. Go to step 1.

Finally, after doing all these steps we rewind to the left tapes 1 and tape 4. Then we scan left-to-right and verify each square is the same. If we ever see a difference we reject. If we get to the end of the string without rejecting we accept. This whole algorithm is at most linear space and quadratic time.

A Specific Nonrecursively Enumerable Language I

• Last day we gave a counting argument to show a non recursively enumerable language must exist - our argument though doesn't give a specific example language.
• We'll use the next theorem to give an example.
• First, call a language co-recursively enumerable if its complement is recursively enumerable.
  Theorem. A language is decidable iff it is recursively enumerable and co-recursively enumerable.
  Proof. Suppose `L` is decidable by `M`. Then it is also r.e. Further, let `bar M` be the machine which rejects when `M` accepts and accepts when `M` rejects. Then `\bar M` recognizes the complement of `L`, so `L` is co-r.e. On the other hand, suppose `L'` is Turing recognized by `M'` and co-Turing recognized by `M''`. Then let `D` be the machines which on input `w` simulates each of `M'` and `M''` first for `1` step, then for `2` steps, etc. If `M'` ever accepts the `D` accepts and if `M''` ever accepts then `D` rejects. Since a string is either in `L'` or not, one of these two machines must accept eventually, and so then `D` will decide that string.

A Specific Non-Recursively Enumerable Language II

Corollary. `\bar{A}_{TM}` is not r.e.

Proof. We proved in an earlier lecture `A_{TM}` is recursively enumerable. So if `\bar A_{TM}` were r.e., then `A_{TM}` would be decidable giving a contradiction with our result that `A_{TM}` is undecidable.

Reducibility

• We next consider what other problems are undecidable.
• Our approach to showing languages are undecidable will be to use a notion called reducibility.
• A reduction `r` is a mapping from possible inputs (instances) `I_A` to a problem `A` to instances of problem `B`, with the property that `I_A in A` if and only if `r(I_A) in B`.
• If the reduction can be computed by a TM, it is called a Turing reduction. Such reductions were first defined by Post in 1944.
• If the reduction can be computed by a TM in polynomial time, it is called a Karp reduction. These reductions were first defined by Karp 1972.
• Turing reductions have the property that if we have a Turing reduction from `A` to `B`, and `B` is decidable, then `A` will be too. Conversely, if `A` is not decidable, then `B` also won't be decidable.
• We write `A le_m B` to say `A` is Turing reducible to `B` and `A le_m^p B` to say `A` is Karp reducible to `B`.
• `m` is for many-one (because two instances of A might be mapped to the same instance of B). Sometimes this term is used for both kinds of reductions. `p` is for polynomial time.

Example

Let `\mbox{HALT}_{TM} = { langle M, w rangle | \mbox{M is a TM and M halts on input w}}`.

Theorem. `\mbox{HALT}_{TM}` is undecidable.

Proof. Suppose `H` decides `\mbox{HALT}_{TM}` . From `H` we can construct a machine `S` which decides `A_{TM}` as follows:

`S =` On input `langle M, w rangle` an encoding of a TM `M` and a string `w`:

1. We build a new string `langle M',w rangle` where `M'` is a machine which simulates `M` until `M` halts (if it does) and if `M` accepts `M'` accepts. Otherwise, if `M` rejects than `M'` moves right forever, one square at a time. The map `langle M,w rangle -> langle M', w rangle` is well-defined enough that it can be computed by a TM. This is our reduction.
2. We then ask our decider for `H` if `langle M', w rangle` is in `H`? If `H` accepts we accept and if `H` rejects we reject.

Since the only way `M'` on input `w` halts is if `M` accepts `w`, we know `langle M,w rangle` is in `A_{TM}` iff `langle M', w rangle` is in `\mbox{HALT}_{TM}`. So if `H` was a decision procedure for `\mbox{HALT}_{TM}`, then `S` would be a decision procedure for `A_{TM}`. As we know there is no decision procedure for `A_{TM}` we know that the supposed `H` can't exist.

Complete Problems

• A language `L` is said to Turing-hard (resp. Karp-hard) for a class of languages `C` if every `L'` in `C` Turing-reduces (Karp-reduces) to `L`.
• If, in addition, `L` is in `C`, then `L` is called Turing-complete (resp. Karp-complete) for `C`.
• Basically, the notion of completeness makes more precise the notion of being the hardest kind of problem in a class of languages.

Complete r.e. and NP problems

Theorem.

1. `A_{TM}` is Turing-complete for r.e.
2. `A_{\mbox{clocked TM}} := {\langle M, w, 1^t rangle | \mbox{M is a nondeterministic machine that accepts w in at most t steps} }` is Karp-complete for NP.

Proof. For (1), we already have show that `A_{TM}` is in r.e. (the set of recursively enumerable languages). Suppose `L` in r.e. is recognized by a machine `M`. The function `w -> langle M, w rangle` is Turing computable and `w in L` iff `langle M, w rangle in A_{TM}`. Therefore, `A_{TM}` is Turing-complete. For (2), we first show `A_{\mbox{clocked TM}}` is in NP. An nondeterministic algorithm to recognize this language is as follows: On input `\langle M, w, 1^t rangle`, nondeterministically guess a string `v` of length `t^2`, then verify that `v` codes a sequence of configurations of `M` of length `t`, the first being the start configuration of `M` on `w` and such that the `i+1`st configuration follows from the `i`th. If it does, and `M` accepts `w` then accept, else reject. Now suppose `L` is in NP. Then it can be decided by some nondeterministic machine `M` in time `p(n)` for some polynomial `p`. The function `w -> langle M, w, 1^{p(|w|)} rangle` is polynomial time computable and gives a reduction from `L` to `A_{\mbox{clocked TM}}`.

Emptiness Testing for TMs

• Using reducibility is the most common way to show a language is undecidable or to show it is Karp-hard for NP.
• For this reason we are going to look at some more examples of these kinds of arguments.
• Consider the language:
  `E_{TM}={ langle M rangle | \mbox{M is a TM and }L(M)=emptyset}`.
  Theorem. `E_{TM}` is Turing complete for co-r.e.
  Proof. Notice we can make a Turing machine `M'` that given an encoding of `langle M rangle` cycles over lengths `k= 1, 2, \ldots` and for each `k` checks for each string `w` of length `leq k` whether `M` accepts `w` in `k` step. If it does, it halts accepting; otherwise it continues. This machine recognizes whether `M`'s language is non-empty, thus showing `E_{TM}` is in co-r.e. Now suppose `L` is co-r.e. That means its complement, `bar{L}` is recognized by some TM `M`. So `w` is in `L` iff `M` does not accept `w`. So for any such `M` which is derived from a co-r.e. language and for any input `w`, consider the following machine:
  "`M_1 =` "On input `x`:
  1. If `x ne w`, reject.
  2. If `x = w`, run `M` on input `w` and accept if `M` does."
  This machine is a modification of `M` and it accepts at most one input `w`, and it only accepts this if `M` does. For a fixed machine `M`, the map `w -> \langle M_1 rangle` is computable by a TM. Also, we have `w in L` iff `M` does not accept `w` iff `M_1`'s language is empty iff `langle M_1 rangle` is in `E_{TM}`. So we have shown how to reduce any language in co-r.e. to `E_{TM}`. Therefore, `E_{TM}` is Turing-complete for co-r.e.
  Corollary. `E_{TM}` is undecidable.
  Proof. If `E_{TM}` were decidable. Then every co-r.e. language would be decidable; and hence, also r.e. So the complement of `A_{TM}` would be r.e., which would mean that `A_{TM}` is both r.e. and co-r.e. and hence decidable, giving a contradiction.

A Problem about Regular Languages

Even problems about regular languages can sometimes be hard...

Let `\mbox{REGULAR}_{TM} = { \langle M \rangle | mbox {M is a TM and L(M) is a regular language}}`.

Theorem. `\mbox{REGULAR}_{TM}` is undecidable.

Proof. Suppose `R` decides `\mbox{REGULAR}_{TM}`. Then the following machine decides `A_{TM}`:
"`S =` On input `langle M,w rangle`, where `M` is a TM and `w` is a string:

1. Construct the following machine `M'`:
   `M' = ''` On input `x`:
   If `x` has the form `0^n1^n`, accept.
   If `x` does not have this form, run `M` on input `w` and accept if `M` accepts `w`.
    // So if `M` accepts `w`, then `M'` accepts all strings; otherwise, `M'` only accepts strings of the form `0^n1^n`.' '
2. Run `R` on input `langle M' rangle`.
3. If `R` accepts, accept; otherwise, if `R` rejects, reject."

In the above, the map `langle M,w rangle -> langle M' rangle` can be viewed as a reduction from `A_{TM}` to `\mbox{REGULAR}_{TM}`.

Using reducibility from languages other than `A_{TM}`

• We can use languages other than `A_{TM}` to show a language is undecidable.
• For instance, if `E_{TM}` reduces to some language `A`, then `A` will be undecidable.
• For example, let `EQ_{TM} = { langle M_1, M_2 rangle | M_1\mbox{ and }M_2\mbox{ are TMs and }L(M_1) = L(M_2) }`.
  Theorem. `EQ_{TM}` is undecidable.
  Proof. Suppose `R` decides `EQ_{TM}`, then we can build an `S` solving `E_{TM}` as follows (hence, giving a contradiction):
  "`S = ` On input `langle M rangle`, where `M` is a TM:
  1. Run `R` on `langle M, M_2 rangle`, where `M_2` is the machine that rejects all inputs.
  2. If `R` accepts, accept; otherwise if `R` rejects, reject."
• Since every problem in co-r.e. is Turing reducible to `E_{TM}`, the fact that Turing computable functions are closed under composition and the Turing computable map `langle M rangle -> \langle M, M_2 rangle` shows `EQ_{TM}` is Turing-hard for co-r.e.
• We can show `A_{TM}` is Turing reducible to `EQ_{TM}` using the map `\langle M, w rangle -> langle M_1, M_2 rangle` where `M_1` is the machine from Slide 3. Thus, `EQ_{TM}` is Turing-hard for r.e. Hence, we have:
  Theorem. `EQ_{TM}` is neither r.e. or co-r.e.