Equivalence classes of automata

Before the midterm, we introduced the Church-Turing Thesis which roughly says that if a language is computable by a process that has some hope of being implementable in the real world then it can be simulated by a Turing Machine.
To make this statement meaningful, we need to first say what it means for two computational models to be of equivalent strength.
Independent of the status of the Church-Turing Thesis, being able to compare the strengths and weaknesses of different computational models is helpful in many real-word situations. For instance, in the design and implementation of new programming languages.
Given two machine models `F` and `F'` we say they are equivalent if for each `M \in F` there is an `M' in F'` such that `L(M) = L(M')` and vice-versa.
We are interested in what models are equivalent to TMs, especially if the model appears stronger at first.

Stay put machines

Suppose rather than having a transition function `delta : Q times Gamma -> Q times Gamma times {L,R}` we instead have: `delta : Q times Gamma -> Q times Gamma times {L,R,S}` where `S` denotes stay put.
It is obvious that machines with this extra stay put ability can simulate machines that don't have it.
On the other hand, given a transition: `delta(q,a) ->(r, b, S)`, we could simulate it by having transitions:
`delta (q, a) ->(r', b, R)`
`delta ( r', c) ->(r, c, L)` where `r'` is a new state and where we have the second kind of transition for each symbol `c`.
Hence, stay-put machines are equivalent to usual TMs.

Quiz (Sec 1)

Which of the following is true?

A Turing machine which computes a function on a domain D, must halt on every input from domain D.
By definition every Turing recognizable language is decidable.
The Church Turing Thesis says that every language that can in principle be recognized by a physical process can be given a CFG.

Quiz (Sec 3)

Which of the following is true?

It is impossible to use smaller Turing Machines to build bigger ones.
`L={w#w | w in {0,1}^{\star}}` is Turing Recognizable but not decidable.
The Church-Turing Thesis is that any computational process that can be effectively carried out on a real-world computational device can be simulated by a Turing Machine.

Semi-infinite tape

Suppose rather than having a two way infinite tape we instead have a tape which is infinite only to the right and where the machine starts off on the left hand square of the input.
If the machine ever tries to move off to the left hand side of the tape it just stays on that left hand square.
This can be simulated by a stay-put TM (and hence, by the last slide, a usual TM) by having our machine as follows:
1. For each tape symbol `a` we have a new symbol `\ul{a}`. Here `\ul{a}` will be used to mean we are on the left hand side of the tape reading an `a`.
2. The first move of our machine replaces whatever symbol `a` it was reading with `\ul{a}`.
3. Thereafter, our machine acts like a usual TM. For each transition which moves right `delta(q,a)->(r,b,R)` we also add a transition `delta(q, \ul{a}) -> (r, \ul{b}, R)`.
4. For each transition that moves left `delta (q,a) ->(r, b, L)`, we add `delta( q, \ul{a})->(r, \ul{b}, S)`.

Semi-infinite tape machines can simulate usual TMs.

The idea is to again increase the tape alphabet `Gamma`. Let `Gamma'` be `Gamma cup {\ul{a} | a in Gamma}`, and let the new alphabet be `Gamma cup ( Gamma' times Gamma')`. This is still finite.
The simulator acts by first replacing the input `w_1, ldots, w_n` with `(\square, \ul{w_1}), (\square, w_2) ldots (\square, w_n)`.
Now we have for each original transition a transition which acts on the left hand side of a symbol.
Similarly, we have for each original transition a transition which acts on the right hand side of a symbol.
For each state in the original machine, we have two states corresponding to whether we are acting on the left or right hand coordinate.
We also have transitions which when we move left in a position with an underscore in the right coordinate we move onto the left hand coordinate. I.e., we can start affecting it. Then for transitions in the original machine to the left again we would have transitions which move left on the left hand coordinate and vice versa. The right hand cordinate thus, behaves as the squares in the original machine to the right of the first square of the input.
To make things complete, we need also some transitions to handle if we move off the right hand side of the squares seen so far onto a square labelled with just `\square` to convert this to `(\square, \square)`.

Off-line Turing Machines

Offline Turing Machines are a TM variant defined in Stearns, Hartmanis, Lewis (1965). An offline TMs have two tapes:
- They have an input tape which is read only on which the input is originally written.
- They have a work tape which is initially blank but which is read write.
Offline machines can simulate usual machines by first copying the input tape to the work tape and then doing all further work on the work tape like a usual TM.
Now let's show an offline TM can be simulated by a usual TM.
The basic idea is to introduce a new symbol `#` to denote the stop and end of tape configuration. We also introduce underscore versions of each tape symbol.
If the input to the offline machine is `w_1 ldots w_n`, our simulator first converts it to the single string `# \ul{w_1}w_2 ldots w_n#\ul{\square}#`. Between the first two `#`'s represents the contents of the input tape, between the second two represents the contents of the work tape that we might have affected (initially, only the square we start at). The underscores indicate which square is being read.
To simulate one step of the offline TM we scan our tape left to right to determine which symbols are under the tape head. Then we scan right to left to update the tapes according to the offline tape's transition.

`k`-tape Turing Machines

One way you might try to improve the power of a TM is to allow multiple tapes.

Definition. A `k`-tape TM, where `k ge 1` is an integer, is a six-tuple `M=(Q, Sigma, Gamma, delta, q_0, F)` where `Q, Sigma, Gamma, q_0` are as in the 1-tape case. Now, however, the transition functions is a map `delta:Q times Gamma^k -> Q times Gamma^k times {L,R}^k`

The transition allows the heads on each tape to move independently of each other.
At the start of the computation the first tape has the input written on it and all the other tapes are blank.
With a two tape machine, an algorithm for palindrome testing is easy:
- We set up the transition function so it first copies the first tape input to the second tape.
- Then it rewinds the first tape and leaves the second tape at the end of the input.
- Then the first tape moves right while the second tape moves left and we compare the two tape symbol by symbol. If they don't match we halt and reject. If the second tape gets back to the `\square` then we accept.

TIME and SPACE classes.

We shall use the k-tape model of TM as our basic model to study time and space complexity.
Let `f:N -> N`. We say that machine `M` operates within time `f(n)` if for any input string `x`, the time require by `M` on input `x` is at most `f(|x|)`. Here `|x|` is the length of `x` as a string. We can make a similar definition for space.

Definition. We say that a language `L` is in `\mbox{TIME}(f(n))` (resp. `\mbox{SPACE}(f(n))`) if it is decided by some `k`-tape TM in time `f(n)` (resp. space `f(n)`).

For example, the algorithm for palindrome is in time `\mbox{TIME}(3(n+2))`.
You can show using an information/Komolgorov-theoretic style argument that for a single tape machine you need at least time `Omega(n^2)` to test for palindromes.

Linear Speed-Up

You might wonder why we didn't say a language is in time `\mbox{TIME}(f(n))` if it can be computed in `O(f(n))` steps rather than the more fragile seeming definition of `f(n)` steps.
The reason is because of the following linear speed-up theorem from Hartmanis and Stearns. Trans AMS #117. May, 1965:
Theorem. Let `L` be in `\mbox{TIME}(f(n))`. Then for any `epsilon > 0`, `L` is also in `\mbox(TIME(f '(n)))` where `f'(n) = epsilon f(n) + n + 2`.
Proof. Let `M=(K, Sigma, Gamma, delta, s)` be a `k`-tape machine that decides `L` in `\mbox{TIME}(f(n))`. Let `k'` be `2` if `k=1` and be `k` otherwise. We will get the speedup by making a simulating `k'` tape machine `M'=(K', Sigma', Gamma', delta', s')` that encodes several of moves of `M` by one move of `M'`. To be specific let `Gamma = Gamma cup Gamma^m` -- we'll fix `m` in a moment. To begin `M'` scans the input `x = x_1 ldots x_n` writing for each `m` adjacent symbols a single symbol in `Gamma'` . At the end of this `m cdot |~ |x|/m ~| + 2` step process the second tape has `n/m` squares with symbols. We then delete the first tape and use the second tape as the input from now on. The machine `M'` then simulates `m` steps of `M` using `6` or fewer steps. At the beginning of a simulating stage the state of `M'` consists of a `k+1` tuple `(q, j_1, ldots, j_k)` where `q` is `M's` state at the start of the stage, and the `j_i`'s represent where within the current `m`-block on each tape `M`'s head would be. For each tape `M'` then scans one square left, two right, one left. It remembers in its controls the symbols it saw to the left and right. Using this information `M'` can now completely predict where `M` would go in its next `M` moves. It then updates the at most two tape square it needs to update and starts simulating the next. If `M` accepts or reject, so does `M'`. Simulating `f(n)` steps takes time `|x|+2 +6 \cdot |~ f(|x|)/m ~|` steps. So for `m = |~ 6/epsilon ~|` the theorem holds.
We now turn to the question of how well a `1`-tape machine simulate a `k`-tape machine?

Equivalence of Turing Machine Variants

Outline