Outline

• Turing Machines
• In-class Exercise
• Expressive power of TMs

Modeling Computation

• Before we give an exact model of computation, the Turing Machine, we will first give an intuitive overview of what we want from such a model.
• As we said last Monday, for thousands of years the term "computation" was understood to mean the application of mechanical rules to manipulate numbers.
• The machine or person doing the manipulation was allowed a scratch pad on which to write intermediate results.
• A Turing Machine is just a concrete embodiment of this notion. It can be viewed as equivalent to any modern programming language -- albeit one with no built-in prohibition on memory size.

An Algorithm

• Let's try to add a little detail to the above description. Suppose `f` is a function from strings of bits, `\{0, 1\}^star`, to `\{0, 1\}`.
• An algorithm for computing `f` is a set of mechanical rules, such that by following them we can compute `f(x)` on inputs `x in \{0, 1\}^star`.
• The "set of rules is fixed" means that we must use the same rules for all inputs.
• We allow each rule to be applied arbitrarily many times (0 or more times) while carrying out the algorithm.
• We also require that each rule be made up of one or more of the following "elementary" operations:
  1. Read a bit of the input.
  2. Read a symbol (might be a larger alphabet than `\{0,1\}`) from the scratch pad.
  We assume in the above we have some means of keeping track of where we are in the input and in the scratch pad. Based on the information we got from the above we can:
  1. Write a bit/symbol to the scratch pad.
  2. Adjust where we are in the input or scratch pad.
  3. Either stop and output `0` or `1` or choose a new rule to execute.

Runtime and Robustness

• The running time is the number of these basic operations that we performed while computing the output.
• We say a machine runs in time `T(n)` if it performs at most (could be less) `T(n)` basic operations before halting on inputs of length `n`.
• Here are some facts about our model:
  1. The model is robust to almost any tweak in the definition such as changing the scratch alphabet say from `\{0, .., 9\}` to `\{0, 1\}` or allowing multiple scratch pads, and so on. The most basic version can simulate the more complicated versions with at most a polynomial slowdown. That is, `t` steps on the more complicated model can be simulated in `O(t^c)` steps for some fixed `c` on the simpler model.
  2. An algorithm can be represented as a bit string once we decide on some canonical encoding. So the string for an algorithm can be used as an input to another algorithm. For example, you could imagine that we have a program that takes some input program compiles it, optimizes the results, etc, then runs the result on some input. We use `M_\alpha` to denote the machine encoded by the string `alpha`.
  3. There is a universal Turing Machine, `U`, that can simulate any other Turing machine given its bit representation. So given a pair `langle alpha, x rangle`, `U` can simulate `M_alpha` on input `x`. Furthermore, if the running time of `M_alpha` was `T(|x|)`, the running time of `U` is `O(T(|x|)log T(|x|))`
• The previous two facts can be used to prove the existence of functions that are not computable by any Turing Machine.

Turing Machines - making scratch pads precise

• Now that we have given a high level overview of models of computation let's try to make the notion of Turing Machine precise and prove some of those "facts" we just had.
• First, to make the notion of scratch pad precise we assume a scratch pad is a tape, a one-dimensional line of cells where each cell can hold a symbol from a finite set `Gamma` called the alphabet of the machine.
• Each tape is equipped with a tape head that can read or write one tape cell at a time.
• The machine's computation is divided into discrete time steps and the head can move left or right one cell in each step.
• We assume our machine has `k` tapes, the first of which is called the input tape. This is a read-only tape.
• The remaining tapes are called work tapes, and the last of these is designated as an output tape. The output is used to write the final answer of the computation.
• Later, when we measure the space used by a computation, we will only count how many of the work tape squares have been used.

A Real-Life, One-Tape Turing Machine

• For fun, I thought I'd include a video of a real-life implementation of a TM.
• This machine is a one tape machine that allows the input tape to be written to, but is otherwise close to the model just presented.

Turing Machines -- Finite set of operations/rules

• A Turing Machine has a finite set of states, denoted by `Q`.
• The machine has a register that holds a single symbol from `Q`. This can be viewed as the current state.
• This state determines the machine's next computational step. A computation step involves:
  1. read the symbol on each of the `k` tapes;
  2. for the `k-1` read/write tapes replace the current symbol with a potentially different one;
  3. change the value of the current state register to some other state;
  4. move each head one cell to the left or right or stay put.

Formal definition

A TM `M` is described by a tuple `(Gamma, Q,delta)` containing:

• A finite set `Gamma` of the symbols that `M`'s tapes can contain. We assume that `Gamma` contains a designated "blank" symbol, denote `square`; a designated "start-of-input symbol" denoted `Delta`; and the numbers `0` and `1`. We call `Gamma` the alphabet of `M`. (I am using `Delta` for what the book draws as a right triangle, because of my inability to get MathJax to draw a right triangle.)
• A finite set `Q` of possible states `M`'s register can be in. We assume that `Q` contains a designated start state called `q_(mbox(start))`, and a designated stop state called `q_(mbox(halt))`.
• A function called `delta: Q times Gamma^k -> Q times Gamma^(k-1) times \{L,S,R\}^k`, where `k ge 2`, describing the rules `M` uses in performing each step. This is called the transition function of `M`.

Machine Operation

• At the start of a computation the start configuration of a machine has the input tape with ` Delta x_1 ... x_n` on it where `x_1, ... x_n` is the input string. All other squares are blank. Each other tape in the start configuration only has `Delta` on it and all other squares blank. In the start configuration, the tape heads are all on their `Delta` symbol. The computation proceeds according to applications of the transition function until the halt state is reached.
• Suppose the machine in in state `q in Q` and `(\sigma_1, \ldots \sigma_k)` are the symbols currently being read on the `k` tapes, and there is a transition rule:
  `delta(q, (sigma_1, ... sigma_k)) = (q', (sigma_2', ... sigma_k'), z)`
  where `z in \{L,S, R\}^k`.
• Then at the next step:
  1. The `(sigma_2, ... sigma_k)` symbols in the last `k-1` tapes will be replaced by the symbols `(sigma_2', ... sigma_k')`. The first tape is unchanged because, as we said earlier, it is read only.
  2. The machine will be in state `q'` and the `k` heads will move left, right or stay put according to the corresponding coordinate of `z`.

In-class Exercise

• If you haven't already done so, sign up for an account on https://yioop.com/.
• Then join the class CS 254 Spring 2017 discussion group.
• For this exercise, I would like you to carefully show that `n! = O((\frac{n}{2})^n)`.
• Once you are done writing your solution, post it to the Feb 1 Class Thread.
• To get the pre-point, you need to post your solution before Feb 1, midnight.

Palindromes

• Let `mbox(PAL)` be the Boolean function defined as follows: for every `x in \{0, 1\}^star`, `mbox(PAL)(x)` is equal to `1` if `x` is a palindrome and `0` otherwise. That is, `mbox(PAL)(x) = 1` iff `x_1x_2 ... x_n = x_n x_(n-1) ... x_1`.
• We can show there is a TM `M` that computes `mbox(PAL)(x)` in `3n` steps.
• To do this we can use a three tape machine which operates as follows:
  1. Copy the input to the read-write work tape.
  2. Move the input-tape to the beginning of the input.
  3. Move the input-tape head to the right while moving the work-tape head to the left. If at any step we observe different symbols, we halt and write 0 to the output tape.
  4. If we see the start symbol of the work tape, we halt and output 1 to the output tape.

The book shows how to implement the above algorithm using the formal definition.

Expressive Power of TMs

The Church-Turing thesis states that any implementable computational model can be simulated by a Turing Machine.

Here is a hand-wavy proof that any program written in a language like C or Java could be simulated with a TM.

• First, recall that to run a C or Java program we compile it to machine code, and then the computer executes that machine code.
• So it suffices to show machine code can be simulated by a TM.
• A machine language program consists of a sequence of instructions of a finite number of simple types.
• For example, (a) read from memory into one of a finite set of registers, (b) write a register's contents to memory, (c) add the contents of two registers and store the results in a third, (d), (e) perform (c) but do multiplication or limited subtraction instead of addition, (f) jump (also called branch) to a location based on whether a register was 0.
• A machine code program operates starting from an initial state of a computer's memory together with the state of its register, and the state of its program counter. (The program counter says which word in memory to look at for the next instruction.)
• We can imagine this starting configuration being suitably encoded as a tuple on the read only input tape of a simulating TM.
• The simulating TM could have a finite set of read-write tapes for the computer's memory, each of its registers, and the program counter. We could maybe have a finite number additional tapes for scratch work.
• At the start of the simulation the TM copies the read only input tape to initialize the computer memory tape, register tape, and program counter tape.
• The TM would then have a subroutine to simulate executing one instruction from the machine code program.
• This might involve (a) rewinding each of the read-write tapes to their start of tape symbols, (b) copying the program counter tape to an auxiliary tape, (c) using the auxiliary tape to count down while advancing memory locations through the memory tape to get to the location in memory of the current instruction, (d) from this location, since there are only finitely many machine code instruction types, we can determine the instruction type using a finite amount of transition function control logic, and then switch to one of finitely many subroutine to simulate the particular instruction, (f) once done simulating the instruction we adjust the program counter to the appropriate next instruction.