Outline

• Turing Machines
• Quiz
• Expressive power of TMs

Modeling Computation

• Before we give an exact model of computation, the Turing Machine, we will first give an intuitive overview of what we want from such a model.
• As we said last Wednesday, for thousands of years the term "computation" was understood to mean the application of mechanical rules to manipulate numbers.
• The machine or person doing the manipulation was allowed a scratch pad on which to write intermediate results.
• A Turing Machine is just a concrete embodiment of this notion. It can be viewed as equivalent to any modern programming language -- albeit one with no built-in prohibition on memory size.

An Algorithm

• Let's try to add a little detail to the above description. Suppose `f` is a function from strings of bits, `\{0, 1\}^star`, to `\{0, 1\}`.
• An algorithm for computing `f` is a set of mechanical rules, such that by following them we can compute `f(x)` on inputs `x in \{0, 1\}^star`.
• The "set of rules is fixed" means that we must use the same rules for all inputs
• We allow each rule to be applied arbitrarily many times (0 or more times) while carrying out the algorithm.
• We also require that each rule be made up of one or more of the following "elementary" operations:
  1. Read a bit of the input.
  2. Read a symbol (might be a larger alphabet than `\{0,1\}`) from the scratch pad.
  We assume in the above we have some means of keeping track of where we are in the input and in the scratch pad. Based on the information we got from the above we can:
  1. Write a bit/symbol to the scratch pad.
  2. Adjust where we are in the input or scratch pad.
  3. Either stop and output `0` or `1` or choose a new rule to execute.

Runtime and Robustness

• The running time is the number of these basic operations that we performed while computing the output.
• We say a machine runs in time `T(n)` if it performs at most (could be less) `T(n)` basic operations before halting on inputs of length `n`.
• Here are some facts about our model:
  1. The model is robust to almost any tweak in the definition such as changing the scratch alphabet say from `\{0, .., 9\}` to `\{0, 1\}` or allowing multiple scratch pads, and so on. The most basic version can simulate the more complicated versions with at most a polynomial slowdown. That is, `t` steps on the more complicated model can be simulated in `O(t^c)` steps for some fixed `c` on the simpler model.
  2. An algorithm can be represented as a bit string once we decide on some canonical encoding. So the string for an algorithm can be used as an input to another algorithm. For example, you could imagine that we have a program that takes some input program compiles it, optimizes the results, etc, then runs the result on some input. We use `M_\alpha` to denote the machine encoded by the string `alpha`.
  3. There is a universal Turing Machine, `U`, that can simulate any other Turing machine given its bit representation. So given a pair `langle alpha, x rangle`, `U` can simulate `M_alpha` on input `x`. Furthermore, if the running time of `M_alpha` was `T(|x|)`, the running time of `U` is `O(T(|x|)log T(|x|))`
• The previous two facts can be used to prove the existence of function that are not computable by any Turing Machine.

Turing Machines - making scratch pads precise

• Now that we have given a high level overview of models of computation let's try to make the notion of Turing Machine precise and prove some of those "facts" we just had.
• First, to make the notion of scratch pad precise we assume a scratch pad is a tape, a one-dimensional line of cells where each cell can hold a symbol from a finite set `Gamma` called the alphabet of the machine.
• Each tape is equipped with a tape head that can read or write one tape cell at a time.
• The machine's computation is divided into discrete time steps and the head can move left or right one cell in each step.
• We assume our machine has `k` tapes, the first of which is called the input tape. This is a read-only tape.
• The remaining tapes are called work tapes, and the last of these is designated as an output tape. The output is used to write the final answer of the computation.

Turing Machines -- Finite set of operations/rules

• A Turing Machine has a finite set of states, denoted by `Q`.
• The machine has a register that holds a single symbol from `Q`. This can be viewed as the current state.
• This state determines the machine's next computational step. A computation step involves: (1) read the symbol on each of the `k` tapes; (2) for the `k-1` read/write tapes replace the current symbol with a potentially different one; (3) change the value of the current state register to some other state; (4) move each head one cell to the left or right or stay put.

Formal definition

A TM `M` is described by a tuple `(Gamma, Q,delta)` containing:

• A finite set `Gamma` of the symbols that `M`'s tapes can contain. We assume that `Gamma` contains a designated "blank" symbol, denote `square`; a designated "start-of-input symbol" denoted `Delta`; and the numbers `0` and `1`. We call `Gamma` the alphabet of `M`.
• A finite set `Q` of possible states `M`'s register can be in. We assume that `Q` contains a designated start state called `q_(mbox(start))`, and a designated stop state called `q_(mbox(halt))`.
• A function called `delta: Q times Gamma^k -> Q times Gamma^(k-1) times \{L,S,R\}^k`, where `k ge 2`, describing the rules `M` uses in performing each step. This is called the transition function of `M`.

If the machine in in state `q in Q` and `\(sigma_1, ... sigma_k\)` are the symbols currently being read on the `k` tapes, and
`delta(q, \(sigma_1, ... sigma_k\)) = (q', \(sigma_2', ... sigma_k'\), z)`
where `z in \{L,S, R\}^k` then at the next step the `sigma` symbols in the last `k-1` tapes will be replaced by the `sigma'` symbols, the machine will be in state `q'` and the `k` heads will move left, right or stay put according to the corresponding coordinate of `z`. At the start of a computation the start configuration of a machine has the input tape with `Delta x_1 ... x_n` on it where `x_1, ... x_n` is the input string. All other squares are blank. Each other tape in the start configuration only has `Delta` on it and all other squares blank. In the start configuration, the tape heads are all on their `Delta` symbol. The computation proceeds according to applications of the transition function until the halt state is reached.

Quiz

Which of the following is true?

1. `nlog n = Theta(2^{log n} log n)`
2. CLIQUE is not a decision problem since it is formulated over graphs rather than strings.
3. The length of a number `n` in binary is `o(n)`.

Palindromes

• Let `mbox(PAL)` be the Boolean function defined as follows: for every `x in \{0, 1\}^star`, `mbox(PAL)(x)` is equal to `1` if `x` is a palindrome and `0` otherwise. That is, `mbox(PAL)(x) = 1` iff `x_1x_2 ... x_n = x_n x_(n-1) ... x_1`.
• We can show there is a TM `M` that computes `mbox(PAL)(x)` in `3n` steps.
• To do this we can use a three tape machine which operates as follows:
  1. Copy the input to the read-write work tape.
  2. Move the input-tape to the beginning of the input.
  3. Move the input-tape head to the right while moving the work-tape head to the left. If at any step we observe different symbols, we halt and write 0 to the output tape.
  4. If we see the start symbol of the work tape, we halt and output 1 to the output tape.

The book shows how to implement the above algorithm using the formal definition.

Expressive Power of TMs

The Church-Turing thesis states that any implementable computational model can be simulated by a Turing Machine.

Here is a hand-wavy proof that any program written in a language like C or Java could be simulated with a TM.

• First, recall that to run a C or Java program we first compile it to machine code, and then the computer executes that machine code.
• So it suffices to show machine code can be simulated by a TM.
• A machine language program consists of a sequence of instructions of a few simple types.
• For example, (a) read from memory into one of a finite set of registers, (b) write a register's contents to memory, (c) add the contents of two registers and store the results in a third, (d) perform (c) but do multiplication instead of addition.
• All of these operations can be implemented using a TM's tapes, while the instructions can be encoded by the machines trasition function.
• The transition function can also be used to simulate branch instructions.