Introduction

On Monday, we said that run time analysis of divide and conquer algorithms typically require one to solve recurrences.
So far, we've only seen one divide-and-conquer algorithm, MERGE-SORT, and that was the case... But maybe I'm lying.
To try to convince you otherwise, I will try to give a couple examples of divide and conquer algorithms each of whose analysis naturally yields a recurrence.
Then we will look at examples of how to solve recurrence relations using the three methods we described on Monday.

Maximum Subarray Problem

Suppose we are given an array listing out daily changes in stock prices for stock X.
So if stock X was worth $5 at the start of trading on Day 1 and and was worth $3 at the closing bell, then our array `A[1]` has the value -2.
We want to find locations `i< j` in this array such that if we bought stock X at the start of day i and sold at the close of day `j`, we maximize our return over any other pair choices in the array.
One thing you might think to try is to merely find the day `i` where the stock value was the lowest by a sweep through the array, then find the place it was the highest `j` and return those two days. If we did this `i` would necessarily be less that `j`. For example, see the stock graph above.
A brute force algorithm for this problem is to use a nested for loop cycling over each value of `i` then each value of `j >i`, computing the sum of the changes in stock price to determine the profit. While doing this, keep track of the maximum profit seen and for what values of `i` and `j` this occurred. Finally, when the loops were done we could return these values.
The runtime of this algorithm would be `c n` for some constant `c` when `i=1`, `c(n-1)` for when `n=2`, and so on. So would be
`c(n+(n-1) + ... +1) = sum_(i=1)^n c cdot i = Theta(n^2).`
Can we do better?

A Divide and Conquer Approach to Maximum Subarray

Image shows the three subcases of Maximum Subarray

Imagine we split the array `A` evenly into a LEFT HALF and a RIGHT HALF.
Then the maximal choice `i` and `j` are either:
1. Both in the LEFT HALF
2. Both in the RIGHT HALF
3. Or `i` is in the LEFT HALF and `j` is in the RIGHT HALF
So we could use recursion to solve step (1) and (2) above.

Below is a procedure to solve case (3) called FIND-MAX-CROSSING-SUBARRAY(A, low, mid, high). It finds the value of `i` such that the sum from `i` to the midpoint is the largest. Then it find the value `j` such that the sum from the midpoint to `j` is the largest. Adding these two together gives the largest sum that start before the midpoint and ends after it.

FIND-MAX-CROSSING-SUBARRAY(A, low, mid, high)
 1   left-sum = -infinity
 2   sum = 0
 3   for i = mid downto low
 4      sum = sum + A[i]
 5      if sum > left-sum
 6          left-sum = sum
 7          max-left = i
 8   right-sum = -infinity
 9   sum = 0
10   for j = mid + 1 to high
11      sum = sum + A[j]
12      if sum > right-sum
13         right-sum = sum
14         max-right = j
15   return (max-left, max-right, left-sum + right-sum)

Analyzing Maximum Subarray

Let `n` be high-low . FIND-MAX-CROSSING-SUBARRAY has two for loops each of which run through only half of this range. The bodies of these for-loops only consist of constant time operations. So the run time of FIND-MAX-CROSSING-SUBARRAY will be `Theta(n)`.

Given FIND-MAX-CROSSING-SUBARRAY we can solve the Maximum Subarray problem with the following procedure:

FIND-MAX-SUBARRAY(A, low, high)
 1   if high == low
 2      return (low, high, A[low]) // base case
 3   else mid = floor(low+high/2)
 4      (left-low, left-high, left-sum) =
           FIND-MAX-SUBARRAY(A, low, mid)
 5      (right-low, right-high, right-sum) =
           FIND-MAX-SUBARRAY(A, mid + 1, high)
 6      (cross-low, cross-high, cross-sum) =
           FIND-MAX-CROSSING-SUBARRAY(A, low, mid, high)
 7   if left-sum >= right-sum and left-sum >= cross-sum
 8      return (left-low, left-high, left-sum)
 9   elseif right-sum >=left-sum and right-sum >= cross-sum
10      return (right-low, right-high, right-sum)
11   else return (cross-low, cross-high, cross-sum)

The run time of line 4 and 5 will be the same so we get
`T(n) = Theta(1) +2T(n/2) + Theta(n) +Theta(1)`
`=2T(n/2) + Theta(n)`.
If we include the base case we get the recurrence relation:
`T(n)={(Theta(1),if n =1 ),(2 T(n/2) + Theta(n),text{n >1}):}`
This is the same recurrence relation as for MERGE-SORT so we know the run time will be `Theta(n log n)`.
In MERGE-SORT though we solved two sub-problems and merge their results together. The above algorithm was different in the solution had to satisfy one of three cases, two of which were given by subproblems, and where the third case was solvable in linear time.

Matrix Multiplication

From linear algebra, we should remember how to multiply matrices `A` and `B` to get a matrix `C`. If `A = [[a_(11),a_(12)],[a_(21),a_(22)]]`, and `B = [[b_(11),b_(12)],[b_(21),b_(22)]]`, then `C=A cdot B` is given by
`C = [[a_(11)b_(11) + a_(12)b_(21),a_(11)b_(12) + a_(12)b_(22)],[a_(21)b_(11) + a_(22)b_(21),a_(21)b_(12) + a_(22)b_(22)]]`
More generally, if `A` and `B` were `n times n` matrices, `C= [c_(ij)]` where
`c_(ij) = sum_(k=1)^(n)a_(ik)b_(kj)`.
To output `C` we need to ouput `n^2` matrix entries.
This could be done by cylcing over `i` and `j` in for loops and for each such pair have a for loop over `k` where we compute the sum above for `c_ij`.
This would be a `Theta(n^3)` algorithm.
It turns out this algorithm is not optimal. Starting with Strassen in 1969, better algorithms have been given for matrix multiplication.
These all rely on a divide and conquer technique at some point.

A Simple Divide and Conquer Algorithm for Matrices

Let's assume `n` is a power of 2. We can split the matrices `A` and `B` into four quadrants `A_(11)`, `A_(12)`, `A_(21)`, `A_(22)` and `B_(11)`, `B_(12)`, `B_(21)`, `B_(22)` each with `n/2 times n/2` entries. We can then multiply these quadrants to get `C` using matrix multiply for `2 times 2` matrices:
`A = [[A_(11),A_(12)],[A_(21),A_(22)]]`, `B = [[B_(11),B_(12)],[B_(21),B_(22)]]`, `C = [[C_(11),C_(12)],[C_(21),C_(22)]]`
where `C = A cdot B` is given by:
`[[C_(11),C_(12)],[C_(21),C_(22)]] = [[A_(11),A_(12)],[A_(21),A_(22)]][[B_(11),B_(12)],[B_(21),B_(22)]]`
`= [[A_(11)B_(11) + A_(12)B_(21),A_(11)B_(12) + A_(12)B_(22)],[A_(21)B_(11) + A_(22)B_(21),A_(21)B_(12) + A_(22)B_(22)]]`

We thus naturally get the following divide-and-conquer algorithm for matrix multiply:

SQUARE-MATRIX-MULTIPLY-RECURSIVE(A,B)
 1 n = A.rows
 2 let C be a new n x n matrix
  //we use little a,b,c for individual entries, big A,B,C for submatrices as above
 3 if n== 1
 4    c[1,1] = a[1,1]*b[1,1]
 5 else partition A, B, C as in the above equation
 6    C[1, 1] =  SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[1,1],B[1,1])
          + SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[1,2],B[2,1])
 7    C[1, 2] =  SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[1,1],B[1,2])
          + SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[1,2],B[2,2])
 8    C[2, 1]  =  SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[2,1],B[1,1])
          + SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[2,2],B[2,1])
 9    C[2, 2] =  SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[2,1],B[1,2])
          + SQUARE-MATRIX-MULTIPLY-RECURSIVE(A[2,2],B[2,2])
10 return C

Analysis of SQUARE-MATRIX-MULTIPLY-RECURSIVE

We have 8 SQUARE-MATRIX-MULTIPLY-RECURSIVE sub-calls above each of which take time `T(n/2)`.
The addition of two `n/2 times n/2` matrices takes `Theta(n^2)` time and we do this four times.
So for the non-base case
`T(n) = Theta(1) +8T(n/2) + Theta(n^2)`
`=8T(n/2) + Theta(n^2)`.
So our overall recurrence is:
`T(n)={(Theta(1),if n =1 ),(8 T(n/2) + Theta(n^2),text{n >1}):}`
This shows another example of a recurrence arising from a divide and conquer algorithm.
To solve it we could use the recursion tree method. At the first level, we do `c(n^2)` work for some constant `c >0`, at the second `8c(n/2)^2) = 2cn^2`, at the third `64c(n/4)^2 = 2^2cn^2` and in general at the `i` level we do `2^i c n^2` work.
There are at most `log n` levels, summing all the levels gives `cn^2(1+2+ 2^2 +... + 2^(log n)) = Theta(n^3)` work.

Strassen's Method

In order, to get a faster algorithm we need to make the recursion less bushy.
Rather than do 8 `n/2 times n/2` matrix multiplications we want to get away with just 7.
The cost to do this will be we perform more matrix additions.
The basic steps in Strassen's method are:
1. Divide the input matrices `A` and `B` and output matrix `C` into `n/2 times n/2` submatrices.
2. Create 10 matrices, `S_1, S_2, ... S_(10)` each of which is `n/2 times n/2`. Each of these will be the sum or difference of two of the matrices from step 1.
3. Using the submatrices of step 1 and the 10 matrices of step 2, recursively compute seven product matrices `P_1, P_2, ..., P_7`. Each matrix `P_i` is `n/2 times n/2`.
4. Compute the desired submatrices `C_(11),C_(12),C_(21),C_(22)` by adding and subtracting various combinations of the `P_i` matrices. This steps will also take `Theta(n^2)` time.
Step 3 is where we split into 7 sub-problems and so the whole procedure (if we spelled it out completely) would have a recurrence equation:
`T(n)={(Theta(1),if n =1 ),(7 T(n/2) + Theta(n^2),text{n >1}):}`
This could be solved either using the recursion tree technique we just saw or the master method which we will see later. The run time for the above is `Theta(n^(log 7))`

The Matrices Used in Strassen's Method

Using these matrices we one can check
`C_(11) = P_5 + P_4- P_2 +P_6`
`C_(12) = P_1 + P_2`
`C_(21) = P_3 + P_4`
`C_(22) = P_5 + P_1 - P_3 - P_7`
For example, to see the first equation, note:
From the above definition we can verify the truth of the statements about the run times of steps (1)-(4) on the last slide.
From our discussion, it is far from obvious how Strassen came up with these matrices, we have only shown that they work.
The how he came up with them would take us somewhat afield, so we return to recurrences.

The Substitution Method

There are two steps to the substitution method:
1. Guess the form of the solution
2. Use mathematical induction to find the constants and show that the solution works.
Consider `T(n) = 2T(lfloor n/2 rfloor) +n`
This is similar to recurrences we have solved before.
We often omit the base case when we state such recurrence and assume for small `n` that `T(n)` is `Theta(1)`.
So we might guess the solution `T(n) = O(n log n)`.
The substitution method would require we prove `T(n) le c nlog n` for an appropriate choice of `c >0`
This proof is typically by induction. Unfortunately, when `n = 1`, `c nlog n = 0`, so if `T(1) > 0` we could be out of luck.
It is legal to start the induction from any `n_0 >0` and for `n=2`, `2 log 2 = 2 > 0`. We can thus find a `c` such that `T(2) < c 2log 2= 2c`. And this might be the `c` we use for the rest of the induction. Let's pick `c ge 1`.
For the induction step we assume for `2 ge m < n`, `T(m) < cm log m` and try to show it is true of `n`.
By the induction hypothesis we have `T(lfloor n/2 rfloor) = clfloor n/2 rfloor log (lfloor n/2 rfloor)`. Substituting this into our recurrence gives:
`T(n) le 2(clfloor n/2 rfloor log (lfloor n/2 rfloor)) +n`
`le c nlog(n/2) +n`
`= c n log n - c n log 2 +n `
`= c n log n - cn + n` `le c n log n`.
The last inequality hold as we assumed `c ge 1`. Thus, we have established the induction step and so therefore for all `n>2`, `T(n) le cn log n`. Therefore, `T(n) in O(n log n)`.

Divide and Conquer Recurrences

Outline