Outline

• Asymptotic Notation
• `Theta`, `O`, `Omega`, `omega`, `o` notation
• Notations when used in Equations
• Common Functions

Asymptotics

• We are interested in the behavior of algorithms as the input size `n` increases.
• When we argue for statements like: "For input sizes large enough only the order of growth of the running time is relevant", we are making arguments about the asymptotic efficiency of algorithms.
• That is, we are concerned about the behavior of algorithms in the limit where inputs sizes increase without bounds.
• Today, we study notations that can be used to simplify arguments concerning the asymptotic behavior of functions.

`Theta`-notation

To begin, let's be precise in what we mean by the notation `Theta(g(n))` for some function `g`.

Definition. For a given function `g(n)` (which we will assume is usually from `NN -> ZZ`), we denote by `Theta(g(n))` the set of functions:
`Theta(g(n)) = { f(n): text{there exists } c_1, c_2 > 0 text{and } n_0`
`text{ such that } 0 le c_1g(n) le f(n) le c_2 g(n) text{ for all } n ge n_0}.`

• Intuitively, a function `f` is in `Theta(g(n))` when for large enough `n` it can always be bounded above and below by some constant times `g(n)`.
• Since `Theta(g(n))` is a set of functions, we should write `f(n) in Theta(g(n))` to denote membership.
• It is a longstanding tradition in computer science to abuse notation and write `f(n) = Theta(g(n))`.
• Because `f(n)` is within a constant factor of `g(n)` both above and below, we say that `g(n)` is an asymptotically tight bound for `f(n)`.
• The definition above requires that every member `f(n) in Theta(g(n))` be asymptotically non-negative as `c_1g(n) ge 0` for `n > n_0`.
• So for the set `Theta(g(n))` to be nonempty `g(n)` itself must be asymptotically nonnegative.

Example - Showing something is `Theta(n^2)`

• Consider the function `1/2n^2 - 3n`. Let's prove carefully that it is in `Theta(n^2)`.
• To do this we must find positive `c_1`, `c_2`, `n_0` such that
  `c_1n^2 le 1/2n^2 - 3n le c_2n^2`
  for all `n ge n_0`.
• If we divide the above inequalities by `n^2`, we get:
  `c_1 le 1/2 - 3/n le c_2`.
• We can make the right hand inequality hold for `n ge 1` by choose `c_2 ge 1/2`.
• Similarly, we make the left-hand inequality hold for `n ge 7` by choosing `c_1 le 1/(14)`.
• Hence, if we choose `n_0 = 7` and use `c_1, c_2` as above we satisfy the definition of `Theta(n^2)`.

Example - Showing something is not `Theta(n^2)`

• Consider `f(n) = 6n^3`. Let's see it is not `Theta(n^2)`.
• We prove this by contradiction. Suppose `6n^3` were `Theta(n^2)`.
• So then there would be `c_2` and `n_0` such that `6n^3 le c_2n^2` whenever `n ge n_0`.
• Dividing both sides by `n^2`, yields `n le c_2/6`. But that are values of `n` bigger than the positive constant `c_2/6`.

Example -- Lead term is the most important

• Consider `f(n) = an^2 +bn +c`, where `a,b,c` are constants and `a >0` (doesn't matter how small `a` is -- it could be `1/10^(10000)`.)
• We can show `f(n)` is `Theta(n^2)`.
• Let `c_1 = a/4`, `c_2 = 7a/4` and `n_0 = max(|b|/a, sqrt(|c|/a))`
• One can check that `0 le c_1n^2 le an^2 +bn+c le c_2n^2` for all `n ge n_0`.
• In general, `p(n) = sum_(i=0)^d a_i n^i`, where `a_i` are constant and `a_d >0`, we have `p(n) = Theta(n^d)`.
• Since any constant is a degree-0 polynomial, we can express any constant function as `Theta(1)`.

`O`-notation and `Omega`-notation

• Notice in the definition of `f(n) in Theta(n)` notation we both lower and upper bound `f` by `g`.
• What if we had just used the lower bound? Or had just used the upper bound?
• It turns out both of these notions are useful.
• We write `O(g(n))` for the set of functions
  `{ f(n): text{there exists } c > 0 text{and } n_0`
  `text{ such that } 0 le f(n) le c g(n) text{ for all } n ge n_0}.`
• If `f(n)=O(g(n))` we say `g(n)` is an asymptotic upper bound for `f(n)`.
• For example any linear function `an +b` where `a > 0` is `O(n^2)`. just take `c = a + |b|` and `n_0 = max(1, -b/a)`.
• Such upper bounds are useful when describing the worst case behavior of algorithms.
• Similarly, we write `Omega(g(n))` for the set of functions
  `{ f(n): text{there exists } c > 0 text{and } n_0`
  `text{ such that } 0 le c g(n) le f(n) text{ for all } n ge n_0}.`
• If `f(n)=Omega(g(n))` we say `g(n)` is an asymptotic lower bound for `f(n)`.
• By unwinding our definitions one can check that `f(n) = Theta(g(n))` if and only if `f(n) = O(g(n))` and `f(n) = Omega(g(n))`.

Interpreting formulas involving Asymptotic notation.

• In computer science we often see expressions like `2n^2 + 3n +1 = 2n^2 + Theta(n)`.
• How should we interpret such a formula?
• When the asymptotic function stands alone on the right hand side such as `n = O(n^2)` we interpret it as set membership `n in O(n^2)`.
• When it appears in a formula, we interpret it as standing for some anonymous function that we do not care to name.
• For example `2n^2 + 3n +1 = 2n^2 + Theta(n)` means that `2n^2 + 3n +1 = 2n^2 + f(n)` for some function `f(n) in Theta(n)`. In this case we could take `f(n) = 3n+1` which is in `Theta(n)`.
• This allows us to reduce clutter in our analyses. We can write `T(n) = 2T(n/2) + Theta(n)` as the recurrence for merge sort without having to worry exactly about which function in `Theta(n)` we are dealing with.
• An expression like `2n^2 + Theta(n) = Theta(n^2)` should be interpereted as no matter how the anonmymous functions are chosen on the left hand side of the equal sign, there is a way to choose the anonymous function on the right hand side to make the equation valid.

`o` and `omega`-notation

• The bound `2n = O(n^2)` is not asymptotically tight.
• `n^2` grows faster the `2n`. We'd like to capture this with our notation.
• To do this we can use the following definition of `o(g(n))`. Notice it is a little o.
  `{ f(n): text{for all } c > 0 text{there exists } n_0`
  `text{ such that } 0 le f(n) le c g(n) text{ for all } n ge n_0}.`
• This says that `g(n)` is asymptotically bigger than `f(n)`.
• In particular, one can verify that `n in o(n^2)`.
• The opposite notion is asymptotically smaller. We can write this as `omega(g(n))` defined by
  `{ f(n): text{ for all } c > 0 text{ there exists } n_0`
  `text{ such that } 0 le c g(n) le f(n) text{ for all } n ge n_0}.`