Outline

• O-Notation
• Equivalence Relations
• Homework Problems
• Graphs and Trees
• Strings
• Proofs and Proof Strategies

Growth Rates of Functions

In this class we will frequently examine the efficiency of algorithms. We will use the notions on this slide to do this.

Definition: Let `NN` be the nonnegative integers. Let `f` and `g` be functions from `NN` to `NN`.

• We write `f(n) = O(g(n))` if there are positive integers `c`, `m` such that `f(n) leq c cdot g(n)` for all `n geq m`. "f grows as g or slower"
• We write `f(n) = Omega (g(n))` if `g(n) = O(f(n))`.
• We write `f(n) = Theta (g(n))` if `f(n) = Omega(g(n))` and `f(n) = O(g(n))`.

For example, `n^2+1 = O(n^2)`.

To see this notice, for all `n geq 1`, `n^2+1 leq n^2+ n^2 leq 2cdot n^2`. So `m=1`, `c=2` work for the above definition.

You might want to convince yourself that: `n^3 = Omega(n^2+n+1)` and `n^3 + n^2= Theta(n^3)`.

Equivalence Relations

• One particularly useful kind of relation is an equivalence relation. Such a relation acts like `=` in situations where we might not usually have an equals around.
• Like the binary relation equals we will write equivalence relations in infix notation. i.e., we'll write `xRy` rather than `(x,y) in R` or `R(x,y)`.
• A binary relation `R` is an equivalence relation if for each `x`, `y`, `z`:
  • `R` is reflexive, that is, `xRx`.
  • `R` is symmetric, that is, `xRy` implies `yRx`.
  • `R` is transitive, that is, `xRy` and `yRz` implies `xRz`.
• The equivalence class of `x`, denoted `[x]`, is the set: `{y | xRy }`
• We often write `equiv` or `~` rather than `R` for equivalence relations.

Example Equivalence Relations

• Last day, we defined the natural numbers in terms of sets.
• Let `-` be coded as `0`, and `+` be coded as `1`.
• The integers `ZZ` are defined as `{[(sgn,n)]| sgn in {-,+} ^^ n in NN}` under the equivalence relation: `(sgn, n)~(sgn', n')` if `n=n'` and `sgn = sgn'` or if `n=n'=0`.
• To keep things simple we abbreviate `(+, n)` as `n` and `(-, n)` as `-n`. The `n=n'=0` case is so that `-0~0`.
• You might want to think how addition, subtraction, and less than can be defined within this definition of the integers.
• Once we do this, we get the usual view of the integers as `ldots -2, -1, 0, 1, 2 ldots`
• The rationals, `QQ`, are defined as the set of equivalence classes of pairs of integers `(p,q)` (which we write as `frac(p)(q)`) such that `q geq 1` and where `frac(p)(q) ~ frac(p')(q')` if and only if `p cdot q'= p' cdot q`.
• For example, `frac(1)(2) ~ frac(2)(4)` as `1cdot 4 = 2cdot 2`.
• The real numbers `RR` can be defined in terms equivalence classes of Cauchy sequences `{a_n}` of rationals. (A Cauchy sequence is a function `a:NN rightarrow QQ`, where `a(n)` is usually written `a_n`, such that for any positive rational `epsilon` we can find an `N` such that for all `n`, `m` bigger than `N`, `|a_n -a_m| lt epsilon` ). Two reals are said to be equivalent if their defining Cauchy sequence converges to the same value. So the equivalence class of the sequence `9/10, 99/100, 999/1000, ldots` and that of `1, 1, 1, 1, ldots` would be the same.
• One can show that both the integers and the rationals are countable; however, the reals are not. (We'll do this later in the semester).

Homework Problems (Sec 1)

Problem 1. Let `A = {1, 4, 5, 9}` and `B = {3, 4, 9}`. Write out fully the elements in each of the following sets (a) `A \cap B` (b) `B - A` (c) `S(A)` (d) `2^B` (e) `A times B`. Give the cardinality of each set.

Answer. (a) `A \cap B = {4, 9}` and `|A \cap B| = 2`, (b) `B - A = {3}` and `|B - A| = 1`, (c) `S(A) = {1, 4, 5, 9, {1, 4, 5, 9}}` and `|S(A)| = 5`, (d) `2^B = {emptyset, {3}, {4}, {9}, {3, 4}, {4,9}, {3,9}, {3,4,8}}` and `|2^B| = 8` (e) `A times B = {(1,3)`,`(1,4)`, `(1,9)`, `(4,3),(4,4), (4,9), (5,3),(5,4), (5,9), (9,3),(9,4), (9,9) }` and `|A times B| = 12`.

Problem 3. Show using only the properties (a)-(d) of this slide that `S^3(0) cdot S^6(0) = S^{18}(0)`.

Answer. The point of this exercise is to realize that axioms for an operation like plus or times should give us an algorithm for computing these operations based on the simpler operations we already have defined. So the two axioms for times say how it is computed in terms of plus and successor, the axioms for plus tell us how plus is defined in terms of successor. To see that `S^3(0) cdot S^6(0) = S^{18}(0)` we will start with the left-hand-side and repeatedly use our rules to expand it until we get the right-hand-side:

`S^3(0) cdot S^6(0) =` (by d)
`S^3(0) cdot S^5(0) + S^3(0) =` (by d -- the first sequence of steps can be viewed as saying multiply 3 times 6 by adding three, six times)
`S^3(0) cdot S^4(0) + S^3(0) + S^3(0) =` (by d)
`S^3(0) cdot S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by d)
`S^3(0) cdot S^2(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by d)
`S^3(0) cdot S(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by d)
`S^3(0) cdot 0 + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by d)
`((((((0 + S^3(0)) + S^3(0)) + S^3(0)) + S^3(0)) + S^3(0)) + S^3(0)) =` (by c -- and by being less sloppy on parentheses)
`S(0 + S^2(0)) + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by b -- as there are no more multiplies left we start using properties of addition)
`S(S(0 + S(0))) + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S(S(S(0 + 0))) + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S(S(S(0))) + S^3(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by a -- now note left hand term is `S^3(0)`)
`S(S^3(0) + S^2(0)) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S(S(S^3(0) + S(0))) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S(S(S(S^3(0) + 0))) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S^6(0) + S^3(0) + S^3(0) + S^3(0) + S^3(0) =` (by a)
`S(S^6(0) + S^2(0)) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S(S(S^6(0) + S(0))) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S(S(S(S^6(0) + 0))) + S^3(0) + S^3(0) + S^3(0) =` (by b)
`S^9(0) + S^3(0) + S^3(0) + S^3(0) =` (by a)
`S(S^9(0) + S^2(0)) + S^3(0) + S^3(0) =` (by b)
`S(S(S^9(0) + S(0))) + S^3(0) + S^3(0) =` (by b)
`S(S(S(S^9(0) + 0))) + S^3(0) + S^3(0) =` (by b)
`S^(12)(0) + S^3(0) + S^3(0) =` (by a)
`S(S^(12)(0) + S^2(0)) + S^3(0) =` (by b)
`S(S(S^(12)(0) + S(0))) + S^3(0) =` (by b)
`S(S(S(S^(12)(0) + 0))) + S^3(0) =` (by b)
`S^(15)(0) + S^3(0) =` (by a)
`S(S^(15)(0) + S^2(0)) =` (by b)
`S(S(S^(15)(0) + S(0))) =` (by b)
`S(S(S(S^(15)(0) + 0))) =` (by b)
`S^(18)(0)` (by a, we're done!!)

Homework Problems (Sec 3)

Problem 2. Write down all possible partitions of the set `{1,2,3,4}`

Answer.`{{1,2,3,4}}`, `{{1},{2,3,4}}`, `{{2},{1,3,4}}`, `{{3},{1,2,4}}`, `{{4},{1,2,3}}`, `{{1, 2},{3, 4}}`, `{{1, 3}, {2, 4}}`, `{{1, 4}, {2, 3}}`, `{{1},{2}, {3,4}}`, `{{1},{3}, {2,4}}`, `{{1},{4}, {2,3}}`, `{{2},{3}, {1,4}}`, `{{2},{4}, {1,3}}`, `{{3}, {4}, {1,2}}`, `{{1}, {2}, {3}, {4}}`.

Problem 4. Construct using `^^`, `vv`, `not` gates a boolean function `{0, 1}^4 rightarrow {0, 1}` which returns `1` if and only if at least half of the boolean inputs are `1`.

Answer. Here we are view 1 as true and 0 as false, and otherwise are using our definitions of `^^`, `vv`, `not` from last day. First, let's call this function `f` and make a truth table of this function:

Two ways we could express this as a boolean formula are: (1) Write a formula that expresses that at least one of the rows where `f` is `1` is true (disjunctive normal form), or (2) Write a formula that expresses that none of the rows where `f` is `0` is true (conjunctive normal form). We'll use approach (1), although it gives a longer formula. This give the formula:
`(not x_1 ^^ not x_2 ^^ x_3 ^^ x_4) vv`
`(not x_1 ^^ x_2 ^^ not x_3 ^^ x_4) vv`
`(not x_1 ^^ x_2 ^^ x_3 ^^ not x_4) vv`
`(not x_1 ^^ x_2 ^^ x_3 ^^ x_4) vv`
`(x_1 ^^ not x_2 ^^ not x_3 ^^ x_4) vv`
`(x_1 ^^ not x_2 ^^ x_3 ^^ not x_4) vv`
`(x_1 ^^ not x_2 ^^ x_3 ^^ x_4) vv`
`(x_1 ^^ x_2 ^^ not x_3 ^^ not x_4) vv`
`(x_1 ^^ x_2 ^^ not x_3 ^^ x_4) vv`
`(x_1 ^^ x_2 ^^ x_3 ^^ not x_4) vv`
`(x_1 ^^ x_2 ^^ x_3 ^^ x_4)`

Graphs

• A graph (sometimes called a directed graph) is a pair `G=(V,E)` where `V` is a set of vertices (aka points or nodes) and `E subseteq V times V` is a set of edges between points. For example, `({1,2,3,4}, {(1,2),(2,4), (1,1),(3,4)})`.
• We can draw a graph like this pictorially:
  
• An edge of the form `(v,v)` is called a loop. For example, `(1,1)` above.
• An undirected graph (or just a graph) is graph in which we can ignore the direction on the edges. One way to do this is to require that if `(v,w)` is in `E` then `(w,v)` is also in `E`.
• For example, the undirected version of the above graph would be : `({1,2,3,4}, {(1,2),(2,1), (2,4),(4,2), (1,1),(3,4), (4,3)})`
  

More on Graphs

• Last day, we defined the cartesian power of a set `A^n=A stackrel(n)(\hat(times cdots times)) A`.
• A sequence of elements from a set `A` is a tuple in `A^n` for some `n`.
• A sequence of edges of the form `( (v_1, v_2), (v_2, v_3), ...,(v_(n-1),v_n))` in a graph is called a walk. For example, `w=(` `(1,2), (2,4), (4,1), (1,2)` `)` is a walk.
• The length of a walk is the number of edges in it. `mbox(length(w)) = 4`.
• A path is a walk in which no edge is repeated. For example, `p=(` `(1,2), (2,4), (4,1), (1,3)` `)` is a path but `w` is not.
• A simple path is a path that does not go out of any vertex more than once. For example, `p'=(` `(1,2), (2,4), (4,3)` `)` is a simple path, but `p` is not a simple path.
• A cycle is a path which begins and ends at the same node. A cycle is simple if it does not repeat nodes except the end point twice. For example,
  `(` `(1,2), (2,5), (5,1), (1,3), (3,4), (4,1)` `)`
  is a cycle but is not simple; whereas, `(` `(1,2), (2,4), (4,1)` `)` is a simple cycle.

Finding Simple Paths

• In this course, we will find it useful to have an algorithm which on inputs a graph `G=(V,E)` and two vertices `s`, `t`, computes a simple path from `s` to `t` -- provided, of course, that there is a simple path between these points.
• To do this we maintain a set `A` of active nodes and a set `S` of seen nodes.
  • Initialize `A={s}`, `S = emptyset`.
  • Repeat until either `t in A` or `A = emptyset`.
    • Pick an `x in A`, set `S:=S cup{x}`
    • Let `mbox(UnseenChild(x)) := {y | (x,y) in E ^^ y !in S}`
    • Set `A := (A cup mbox(UnseenChild(x))) - {x}`
  • If `A = emptyset` then output "there is no path s to t"
  • Otherwise, we can find a path in reverse order by looking in `S` for some `x` such that `(x,t) in E`, then looking in `S` for some `y` such that `(y,x) in E`, and so on until we get back to `s`. This will be a simple path.

Trees

• A tree is a graph without cycles, and that has one distinct vertex, called the root, such that there is exactly one path from the root to every other vertex.
• The root has no incoming edges.
• Any node without outgoing edges is called a leaf.
• If `(v,w)` is an edge in a tree, then `v` is called the parent of `w` and `w` is called the child of `v`.
• The level of a vertex is the number of edges in the path from the root to that vertex.
• The height of a tree is the largest level number of any vertex.
  

Definitions, Theorems, Proofs

• Definitions describe the objects and notions that we use. We want our definitions to be as precise as possible. Ideally, we might want something of the form: `mbox(mydefn(x)) iff P(x)`, where `P` is a first-order formula in whatever system we are working in.
• Once we have made some definitions we make mathematical statements involving them:
  • A proof is a convincing logical argument that a statement is true.
  • A theorem is a mathematical statement which has been proved true.
  • A lemma is a simple mathematical statement which has been proved true and which will be used in the proof of a theorem.
  • A proposition is a mathematical statement with an easy proof. One can view it like a warm-up result, which does not immediately lead to the proof of a theorem.
  • A corollary is a mathematical statement which can be proved easily once some theorem is known.

Strings

• For this class, we will define an alphabet to be some nonempty finite set `Sigma`. As examples, one could have:
  `Sigma = {0, 1}` or
  `Sigma = {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}`.
• The members of this set are called the symbols of the alphabet.
• A string over an alphabet is a finite sequence of symbols from that alphabet. For example, `0100`.
• The length of a string `w`, `|w|` is the number of symbols it contains.
• The empty string is written as `epsilon`.

Strings and Languages

• The reverse of a string `w`, `w^R` ,is the string consisting of the symbols of `w` in reverse order: `001^R = 100`.
• The concatenation of two strings `x` and `y`, `xy`, is the string consisting of the symbols in `x` followed by the symbols of `y`.
• We write `x^k` to denote `x` concatenated to itself `k` times.
• A language is a set of string.

Boolean Logic

• Is a logic based on two values TRUE or FALSE. (sometimes we use 1 and 0).
• These are called Boolean values.
• We also have boolean operations:
  • NOT (negation) `not x` is TRUE iff `x` is FALSE
  • AND (conjunction aka `cdot mod 2`) `x ^^ y` iff both `x` and `y` are TRUE
  • OR (disjunction) `x vv y` iff at least one of `x` or `y` is TRUE
  • XOR (exclusive OR, aka parity, aka `+ mod 2`) `x oplus y` iff exactly one of `x` or `y` is TRUE.
  • Equality `x iff y` iff `x` and `y` have the same value
  • Implications `x implies y` iff `x` is less than or equal to `y`

Finding proofs

• Example: For every graph G, the sum of the degrees of all the nodes in G is an even number.
  • Might approach problem by checking cases like when graph has a small number of vertices. One might then notice each edge contributes two to the total sum and that the sum of degrees = ` 2 times` number of edges in graph.
• Types of proofs:

  By Construction

  For example, There is a graph consisting of `n` nodes with only one cycle. Proof: let `V={1, ldots ,n}`, let `E={{1,2}, {2,3}, ...{n- 1,n}} cup {{1,n}}`.

  By Contradiction

  For example, `sqrt 2` cannot be expressed as `frac(m)(n)` where `m` and `n` are integers. i.e., it is irrationality. Proof Idea: If not can assume `sqrt 2 = frac(m)(n)` where `m` and `n` share no common factor. In which case, one is odd, the other even. Squaring both sides gives: `2n^2 = m^2`, so `m` is even because the square of an odd number is odd. So `m=2 cdot k`, so `2n^2 = (2k)^2= 4k^2`. So `n^2= 2k^2` implying `n` is also even, giving a contradiction.

  By induction

  `sum_(i = 1)^n i = frac(n(n+1))(2)`.
  Proof. When n =0, the left hand side is 0 and the right hand side is 0. So base case holds. Assume true of `n`, then `sum_(i = 1)^(n+1) i = n+1 + sum_(i = 1)^(n) i`. By the induction hypothesis, this last term is `frac(n(n+1))(2)`. So `sum_(i = 1)^(n+1) i = n+1 + frac(n(n+1))(2) = frac(2(n+1) + n(n+1))(2) = frac((n+1)(n+2))(2)`. So the induction step holds, thus, the statement holds for all `n`.