Outline

• Finish LCS Problem
• Quiz
• Activity-Selector Problem

Introduction

• Last week we were talking about dynamic programming techniques.
• These typically involve optimization problems with overlapping subproblems.
• We looked at the rod cutting problem. Solving it using dynamic programming involved using an auxiliary array to hold solutions to sub-problems which had already been calculated.
• We said developing dynamic programming algorithms is often done by following four steps: (1) Characterize the structure of an optimal solution in terms of subproblems, (2) recursively define the value of an optimal solution, (3) compute the value of an optimal solution, typically bottom up, and (4) use that to construct the optimal solution.
• As a concrete example of going through these four steps we began looking at the longest common subsequence problem: Given two sequences `X` and `Y` find a longest length subsequence `Z` that they share in common.
• The very last slide we looked at last week dealt with Step (1) above, and we begin with it here today.

Step 1: Characterizing an LCS

• A brute force solution to the LCS problem enumerates all possible subsequences of `X` and checks each to see if it is also a subsequence of `Y`. If `X` has length `m`, there are `2^m` choices of subsets of indices `{1, ..., m}`. So this will be an exponential time algorithm.
• Let `X = langle x_1, x_2, ..., x_m rangle`, define the `i`th prefix of `X`, `X_i`, to be the sequence `langle x_1, ... x_i rangle`.
• For example, if `X = langle A,B, C, B, D, A, B rangle` then `X_4 = langle A, B, C, B rangle` and `X_0` is the empty sequence.
• The LCS problem has an optimal-substructure property given by the following theorem.
  Theorem. Let `X = langle x_1, x_2, ..., x_m rangle` and `Y = langle y_1, .. y_n rangle`, and let `Z=langle z_1, ... z_k rangle` be any LCS of `X` and `Y`. Then:
  1. If `x_m = y_n`, then `z_k = x_m = y_n` and `Z_(k-1)` is an LCS of `X_(m-1)` and `Y_(n-1)`.
  2. If `x_m ne y_n`, then `z_k ne x_m` implies that `Z` is an LCS of `X_(m-1)` and `Y`.
  3. If `x_m ne y_n`, then `z_k ne y_n` implies that `Z` is an LCS of `X` and `Y_(n-1)`.
• Notice `X_(m-1)` and `Y_(n-1)` are natural subproblems.

Step 2: A Recursive Solution

• The theorem of the last slide says we should examine either one ot two subproblems when finding an LCS of `X = langle x_1, x_2, ..., x_m rangle` and `Y = langle y_1, .. y_n rangle`...
• If `x_m = y_n`, we must find the LCS of `X_(m-1)` and `Y_(n-1)`. Appending `x_m = y_n` gives the LCS of `X` and `Y`.
• If `x_m ne y_n`, then we must solve two subproblems: `LCS(X_(m-1), Y)` and `LCS(X, Y_(n-1))`. Whichever of these is longer will solve `LCS(X, Y)`.
• Notice that to solve either `LCS(X_(m-1), Y)` or `LCS(X, Y_(n-1))` we might need to compute `LCS(X_(m-1), Y_(n-1))`, so we are in a situation where we might have overlapping subproblems and so dynamic programming is likely to help.
• To define a recursive solution for the optimal value (optimal length) of this problem, let `c[i,j]` denote the length of an LCS of the sequences `X_i` and `Y_j`.
• If either of these is of length `0`, the LCS will have length `0`. Otherwise, the remarks above give us:
  `c[i,j] = {(0,if i=0 or j =0),(c[i-1, j -1]+1,if i and j gt 0 text{ and } x_i = y_j),(max(c[i,j-1],c[i-1,j]),if i and j gt 0 text( and ) x_i ne y_j):}`

Step 3: Computing the length of an LCS

• Since the LCS problem has only `Theta(mn)` distinct subproblems (a polynomial number), it makes sense to use dynamic programming to compute the solutions bottom up.
• We use the procedure of the next slide to do that. It takes as input an `X` and `Y` as on the previous slide.
• It stores the `c[i,j]` values in a table `c[0..m,0..n]` and it computes entries in row-major order.
• That is, the procedure fills in the first row of `c` from left to right, then the second row, and so on.
• The procedure also maintains a table `b[1 ..m, 1 ..n]` to help construct an optimal solution.
• Intuitively, `b[i,j]` points to the table entry corresponding to the optimal subproblem solution chosen when computing `c[i,j]`.

LCS-Length

LCS-LENGTH(X,Y)
01 m = X.length
02 n = Y.length
03 let b[1..m, 1 ..n] and c[0..m, 0..n] be new tables
04 for i = 1 to m //initialize first column
05    c[i, 0] = 0
06 for j = 0 to n //initialize first row
07    c[0, j] = 0
08 for i = 1 to n
09    for j = 1 to m
10       if X[i] == Y[j]
11           c[i,j] = c[i-1, j -1] + 1
12           b[i,j] = "diagonal"
13       elseif c[i-1,j] ≥ c[i, j - 1]
14           c[i,j] = c[i-1,j]
15           b[i,j] = "up"
16       else c[i,j] = c[i, j-1]
17           b[i,j] = "left"
18 return c and b

This procedure will be `Theta(mn)` because of the doubly nested for loop each step of which takes `Theta(1)` time to compute.

Step 4: Constructing an LCS

• To construct an LCS using the table `b`, we begin at `b[m,n]` and trace through the table by following the directions given by the entries.
• Whenever we encounter a "diagonal" entry in `b[i,j]`, it implies `x_i = y_j` is an element of the LCS that LCS-LENGTH found.
• With this method, we encounter the elements of this LCS in reverse order.
• The recursive procedure below prints out an LCS of `X` and `Y` in forward order.
• The initial call to compute the LCS of the `X` and `Y` should be PRINT-LCS(b, X, X.length, Y.length).

PRINT-LCS(b, X, i, j)
1 if i == 0 or j == 0
2    return //empty sequence is an LCS
3 if b[i,j] == "diagonal"
4    PRINT-LCS(b, X, i - 1, j - 1)//print sub-case first so forward order
5    print x[i]
6 elseif b[i,j] == "up"
7    PRINT-LCS(b, X, i - 1, j)
8 else PRINT-LCS(b, X, i, j - 1)

Example

• The above image shows the `b` and `c` tables superimposed for a computation of the LCS of `X = langle A, B, C, B, D,A, B rangle` and `Y = langle B, D, C, A, B, A rangle`
• The darkened squares show the path followed during PRINT-LCS and the darkened squares with circles correspond to a row where a letter of `X` was output.
• Reading these off gives the LCS `langle B, C, B, A rangle`.

Quiz (Sec 5)

Quiz (Sec 6)

Greedy Algorithms

• We next look at another technique for solving optimization problems.
• An algorithm for an optimization problem typically goes through a sequence of steps with a set of choices at each step (for example where to cut the remaining rod next).
• A greedy algorithm always makes the choice that looks best at the moment. I.e., it makes a locally optimal choice in the hope that this will lead to a globally optimal solution.
• An example you might have seen in CS46B is the task of making change using the fewest coins possible from a set of quarters, dimes, nickels, and pennies. If the change amount is `x`, we pick the maximal coin value `c` such that `x gt c`, output that coin and recurse on the subproblem `x-c`.
• We now look at another problem where this greedy decision making actually yields a correct solution to the problem...

The Activity-Selection Problem

• Suppose we have several competing activities that require exclusive use of a common resource (for example, our time). Our goal is to select a maximum-size set of mutually compatible activities.
• To make this more precise, let `S = {a_1, a_2, ..., a_n}` be a set of proposed activities that wish to use a resource such as a lecture hall that can only serve one at a time.
• Each `a_i` has a start time `s_i` and a finish time `f_i`, where `0 le s_i lt f_i lt infty`.
• Activities `a_i` and `a_j` are compatible if the intervals `[s_i, f_i)` and `[s_j, f_j)` do not overlap. I.e., either `s_i ge f_j` or `s_j ge f_i`.
• The activity selector problem can be stated as follows:
  Input: A set `S = {a_1, a_2, ..., a_n}` of activities `a_i = (s_i, f_i)` ordered so that `f_1 le f_2 le ... le f_n`
  Output: A maximum-size subset of `S` of mutually compatible activities.

Example

• Suppose the activities in an instance of the activity selector problem had start and finish times according to the table above.
• Then the activities `{a_3, a_9, a_11}` would be mutually compatible.
• It is not a maximum sized subset though as an example largest mutually compatible subset is `{a_1, a_4, a_8, a_11}` is bigger.
• A maximum-sized subset need not be unique. For example, `{a_2, a_4, a_9, a_11}` also works.

Optimal Substructure

• Let `S_(ij)` be the set of activities that start after `a_i` finishes and that finish before `a_j` starts.
• Suppose we wish to find a maximum set of mutually compatible activities in `S_(ij)`.
• Let `A_(ij)` be such a maximal set and suppose it contains `a_k`.
• Including `a_k` in `A_(ij)` reduces the problem of computing `A_(ij)` to finding mutually compatible solutions in `S_(ik)` and `S_(kj)`.
• Let `A_(ik) = A_(ij) cap S_(ik)` and let `A_(kj) = A_(ij) cap S_(kj)`.
• Thus, we have `A_(ij) = A_(ik) cup {a_k} cup A_(kj)` and the maximal sized set of mutually compatible activities in `S_(ij)` is `|A_(ij)| = |A_(ik)| + |A_(kj)| + 1`.

Dynamic Programming Solution to Activity-Selector

• This characterization of the optimal substructure suggest that we could solve the activity-selector problem by dynamic programming.
• If we denote the size of an optimal solution for the set `S_(ij)` by `c[i, j]`, then we would have: `c[i,j] = { (0,if S_(ij) = emptyset),(max_(a_k in S_(ij)){c[i,k] + c[k,j] + 1},if S_(ij) ne emptyset):}`
• The max is to choose the best `a_k`.
• We could then develop a recursive algorithm, memoize it, or we could work bottom-up and fill in table entries.
• Such a dynamic programming approach though would be wasteful of memory and it also misses an important characteristic of the problem.

Making Greedy Choices

• To reduce the memory consumption of the algorithm so far, we want some way to avoid computing `max_(a_k in S_(ij)){c[i,k] + c[k,j] + 1}` on the last slide.
• Intuitively, we should choose an activity that leaves the resource available for as many other activities as possible.
• This means we should choose the activity that finishes first as it will leave the most time for the other activities
• Given this choice of activity what remains is to find the remaining activities which start after this activity finishes.
• The question is: Does choosing activities on this basis can in fact lead to an optimal solution?