Computation and complexity

Historically, when people thought of the term computation they might have thought of something like doing a calculation on a piece of scratch paper according to some rules. For example, multiply two 3 digit numbers.
In the first half of the twentieth century, a more precise definition of computation was developed.
From this definition, it became apparent that computation can happen in diverse physical and mathematical systems - Turing Machines, lambda calculus, cellular automata, bouncing billiard balls, etc.
It also turned out that the computational power of all these forms of computation was equivalent, so people started focusing on figuring out which problems which were computable by any of these systems and which problems none of these computation systems would be able to solve.
As electronic computers became more common, it became clear that although a problem may be in principle doable on a computer, that if solving it would take longer than the lifetime of the universe, then the problem still wasn't doable. So people started looking at the computational complexity of problem: How efficiently they can be solved.
These are the two main kinds of problems we are going to look at: The first kind mentioned above, which is the question of computability versus non-computability and the second which are question of complexity theory, what problems can be solved within a given resource bound.
In order to begin, we need to fix our notations for resource bounds, problems, etc...

Notations

We write `ZZ` for the integers `\{0, \pm 1, \pm 2 , ldots\}`. We write `NN` for `\{0, 1, 2, ldots\}`.
`i, j, k, l, m, n` will all be used as integer variables.
We'll write `[n]` for the set `\{0, ..., n \}`.
For a real number `x`, we write `|~ x ~|` for the nearest integer to x rounded up and `\lfloor x \rfloor` for the nearest integer rounded down.
`log x` denotes the logarithm of `x` base `2`.
We say the condition `P(n)` holds for sufficiently large `n` if there exists some `N` such that `P(n)` holds for all `n > N`.
We write `sum_i f(i)` rather than `sum_(i=1)^n f(i)` when the range of `i` is obvious.

Strings

If `S` is a finite set, then a string over the alphabet `S` is a finite ordered tuple of elements from `S.` For example, `(a, b, b, a)`, usually written as `\a\b\b\a`, is a string over `\{a,b\}`.
We write `S^n` to denote the strings over `S` of length `n`. So `\{0, 1\}^6` would be the strings of length `6` over `0` and `1`.
We write `S^\star` for `cup_(n ge 0) S^n`.
We write `x circ y` for the concatenation of the strings `x` and `y`.
We write `|x|` to mean the length of the string `x`. So `|aaaaa| = |a^5| = 5`.

Representing Objects as Strings

This semester we will often be interested in computing functions from `\{0,1\}^m` to `\{0,1\}^n`. However, these inputs and outputs might be used to represent something more complicated than just strings of bits.
For example, we might want to represent an integer, ordered pairs of integers, graphs, vectors, etc -- just using strings over `0` and `1`.
We often won't be too specific on the exact representation, but you should always convince yourself that such a representation is possible. For example, `\langle x, y rangle` might be represented as `x # y` where the strings `x`, `y` and the separator `#` are encoded by using the map `0 -> 00,` `1 -> 11,` and `# -> 01`.
Using representations we can talk about functions whose inputs and outputs are not strings. We can also talk about computing functions which take a vector of inputs or produce a vector of outputs.

Decision Problems/Languages

Functions from strings to strings which produce a single bit as output are called Boolean Functions.
We identify such a function `f` with the subset `L_{f} \= \{ x | f(x) = 1\}` of `{0, 1}^star`.
This set is called a language or a decision problem for `f`.
For example, we could have a representation for graphs over `0`, `1`. We could then consider those graphs which have subgraphs of size `k` in which every vertex in the subgraph was connected.
We could write this as the decision problem:
`CLIQUE := \{ langle G,k rangle | exists S subseteq V(G) \mbox{ s.t. } |S| ge k mbox{ and } forall u,v in S, (u,v) \in E(G) \}`

Big-Oh Notation

We will typically measure the computational efficiency of an algorithm as the number of basic operations it performs as a function of its input length.
That is, a function `T(n)` where `n` is the input length and `T(n)` is the number of steps to do the computation.
We don't want our discussion to be overly dependent on the low-level implementation and representation of our algorithm, so we introduce the concept of Big-Oh notation to avoid this.

Definition. If `f,g` are two functions from `NN` to `NN`, then we say that (1) `f=O(g)` if there exists a constant `c` such that `f(n) le c cdot g(n)` for every sufficiently large `n`, (2) we say `f= Omega(g)` if `g= O(f)`, and (3) say that `f=Theta(g)` if `f=O(g)` and `f=Omega(g)`. (4) We say that `f = o(g)` if for every `epsilon > 0`, `f(n) le epsilon g(n)` for sufficiently large `n` and say that `f = omega(g)` if `g = o(f)`.

As an example, you should work out formally that if `f(n) = 100n log n` and `g(n) = n^2` that `f= O(g)`, `g=Omega(f)`, `f=o(g)`, and `g=omega(f)` .