Outline

• Parallel Maximal Independent Set
• Quiz

Introduction

• I put up one of the better student HWs as a solution for HW1.
• No one had a perfect Problem 3, so I wrote up my own solution for that.
• Last Wednesday, we went over a sorting algorithm for parallel random access machines (PRAMs) due to Reischuk 1985.
• It ran using linearly many processors in time `O(log n)` with high probability.
• There are links on last days lecture notes to the original article since I think its pretty well written and contains some of the proofs we glossed over.
• Above is a picture of the relationships between various parallel and non-parallel complexity classes, we've considered.
• In the above, `P` represents what languages can be recognized in polynomial time on a single processor.
• We begin today by looking at a problem which is on the boundary between problems which have polynomial time solutions and those which have efficient parallel algorithms.

Maximal Independent Set

• Let `G = (V,E)` be an undirected graph with `n` vertices and `m = Omega(n)` edges. A subset `I` of `V` is said to be independent in `G` if no edge in `E` has both ends in `I`.
• Equivalently, if `Gamma(v)` is the set of vertices connected to `v`, then `I` is independent if for all `v in I`, `Gamma(v) cap I = emptyset`.
• An independent set is maximal if it is not a proper subset of another independent set in `G`.
• The red nodes and the blues nodes in the graph above are two different maximal independent sets in the same graph. Notice the blue set has more nodes.
• The problem of finding a maximum independent set (the independent set with the most nodes) is NP-hard.
• In contrast the finding a maximal independent set is `O(m)` time:
  

  Greedy MIS:
  Input: Graph G(V,E) with V = {1,..,n}
  Output A maximal I contained in V.
  1. I := emptyset
  2. For v=1 to n do
  3.     If Gamma(v) intersect I = emptyset then I := I union {v}.
  

Analysis of Greedy MIS

• Greedy-MIS is very sequential in nature.
• For the graph on the last slide the algorithm outputs the Maximal Independent Set (MIS) {1,3,6}.
• Notice the two other independent we had previously drawn are {1,5} and {3, 4, 6}. According to dictionary (lexicographical) order {1, 3, 6} is before {1,5} is before {3, 4, 6}.
• It turns out Greedy-MIS always outputs the lexicographically first MIS (LFMIS).
• LFMIS is a P-complete problem (with respect to log-time poly- processor PRAM reductions) (Cook 1985).
• So it is known that an NC algorithm for LFMIS would imply P=NC. (This is nn open problem. In English, it asks does every poly-time algorithm have a good parallel one?)
• We will describe an RNC algorithm for MIS and later show how to derandomize it to an NC algorithm.
• The maximal set we output won't typically be the lexicographically first one.

Quiz

Which of the following statements is true?

1. We made use of the get_global_id(0) call in our Java thread example.
2. BoxSort makes use of the PRAM QuickSort algorithm as a subroutine.
3. BoxSort recurses down until it gets to instances of size `sqrt(n)` in the size of the original problem. At which point, SqrtSort is used.

Yet More on Greedy MIS

• Consider the following variant of Greedy MIS where we after setting `I = emptyset` in the for loop we do:

  1. Pick any vertex v
  2. Add v to I 
  3. Delete v and Gamma(v) from the graph.
  

• Choosing `v` to be the lowest numbered vertex present in the graph leads to the same outcome as Greedy MIS.
• The basic idea of our parallel algorithm is to generalize this to find an independent set `S`, add `S` to `I` and delete `S` and `Gamma(S)`.
• We want to choose an independent set such that `S cup Gamma(S)` is large to keep the number of iterations small.
• To do this we ensure the number of edges incident to `S cup Gamma(S)` is a large fraction of the total remaining edges.
• To find such an `S`, we pick a large random set of vertices `R` contained in `V`. `R` won't usually be independent. If we bias the sampling in favor of vertices with low degree, we can hope that few will have both endpoints in `R`. For those edges which have both endpoints in `R`, we delete the one of lower degree. This gives an independent set.

Parallel MIS

Input: G=(V,E)
Output: A maximal independent set I contained in V
1. I := emptyset
2. Repeat {
   a) For all v in V do in parallel
         If d(v) = 0 then add v to I and delete v from V.
         else mark v with probability 1/(2d(v)).
   b) For all (u,v) in E do in parallel
         if both u and v are marked
             then unmark the lower degree vertex.
   c) For all v in V do in parallel
         if v is marked then add v to S
   d) I := I union S
   e) Delete S union Gamma(S) from V and all incident edges from E 
   } Until V is empty.

Analysis of Parallel MIS

• The algorithm of the last slide and the analysis we give are due to (Luby 1986).
• Each iteration of the above takes `O(log n)` time on an EREW PRAM with `O(n+m)` processors.
• We want to bound the number of iterations we do.
• Call a vertex `v` good if it has at least `(d(v))/3` neighbors of degree no more the `d(v)`; otherwise, the vertex is bad. An edge is good if one of its endpoints is good and is bad otherwise.
• A good vertex is quite likely to have one of its lower degree neighbors in S and so is likely to be deleted from `V`.
• We argue that the number of good edges is large, and since good edges are likely to be deleted, a large number of edges will be deleted each iteration.

More Analysis of Parallel MIS

Lemma*. Let `v` in `V` be a good vertex with degree `d(v) > 0`. Then, the probability that some vertex `w in Gamma(v)` gets marks is at least `1- exp(-1/6)`.

Proof. Each vertex `w in Gamma(v)` is marked independently with probability `1/(2d(w))`. Since `v` is good, there exist `(d(v))/3` vertices in `Gamma(v)` with degree at most `d(v)`. Each of these is marked with probability at least `1/(2d(v))`. Thus, the probability none of these neighbors is marked is at most: `(1 - 1/(2d(v)))^((d(v))/3) le e^((-1)/6)`.

Here we are using that `(1 + a/n)^n <= e^(a)` and that the remaining neighbors of `v` can only help increase the probability under consideration.

Yet More Analysis of Parallel MIS

Lemma**. During any iteration, if a vertex `w` is marked then it is selected to be in `S` with probability at least `1/2`.

Proof. The only reason a marked vertex `w` becomes unmarked and hence not selected for `S` is if one of its neighbors of degree at least `d(w)` is also marked. Each such neighbor is marked with probability at most `1/(2d(w))`, and the number of such neighbors is at most `d(w)`. Hence, we get the probability that a marked vertex is selected to be in `S` is at least:
`1 - Pr{exists x in Gamma(w) mbox( such that ) d(x) ge d(w) mbox( and x is marked )}`
`ge 1 - |{x in Gamma(w)| d(x) ge d(w)}| times 1/(2d(w))`
`ge 1 - sum_(x in Gamma(w))1/(2(d(w))`
`= 1 - d(w) times 1/(2(d(w))`
`= 1/2`

Even More Analysis of Parallel MIS

Lemma#. The probability that a good vertex belongs to `S cup Gamma(S)` is at least `(1- exp(-1/6))/2`.

Proof. Let `v` be a good vertex with `d(v) > 0`, and consider the event `E` that some vertex in `Gamma(v)` does get marked. Let `w` be the lowest numbered marked vertex in `Gamma(v)`. By Lemma **, `w` is in `S` with probability at least `1/2`. But if `w` is in `S`, then `v` belongs `S cup Gamma(S)` as `v` is a neighbor of `w`. By Lemma *, the event `E` happens with probability `1- exp(-1/6)`. So the probability `v` is in `S cup Gamma(S)` is thus `(1- exp(-1/6))/2`.

Still More Analysis of Parallel MIS

Lemma## In a graph `G=(V,E)`, the number of good edges is at least `|E|/2`.

Proof. Our original graph was undirected. Direct the edges in `E` from the lower degree-point to the higher degree endpoint, breaking ties arbitrarily. Let `d_i(v)` be the indegree of `v` and `d_o(v)` be the out-degree. From the definition of goodness, we have for each bad vertex:
`d_o(v) - d_i(v) ge (d(v))/3 = (d_o(v) + d_i(v) )/3`
For all `S`, `T` contained in `V`, define the subset of the edges `E(S,T)` as those edges directed from vertices in `S` to vertices in `T`; further, let `e(S,T) = |E(S,T)|`. Let `V_G` and `V_B` be the sets of good and bad vertices respectively. The total degree of the bad vertices is given by:
`2e(V_B, V_B) + e(V_B, V_G) + e(V_G, V_B)`
`= sum_(v in V_B) (d_o(v) + d_i(v))`
`le 3 sum_(v in V_B)(d_o(v) - d_i(v))`
`= 3 sum_(v in V_G)(d_i(v) - d_o(v))`
`= 3[(e(V_B, V_G) + e(V_G, V_G)) - (e(V_G, V_B) + e(V_G, V_G))]`
`= 3[e(V_B, V_G) - e(V_G, V_B)]`
`le 3[e(V_B, V_G) + e(V_G, V_B)]`
The first and last expressions in this sequence of inequalities imply that
`e(V_B,V_B) <= e(V_B,V_G) + e(V_G,V_B)`.
Since every bad edge contributes edge contributes to the left side, and only good edges to the right side, the result follows.

Finishing up Parallel MIS

Theorem. The Parallel MIS algorithm has an EREW PRAM implementation running in expected time `O(log^2 n)` using `O(n+m)` processors.

Proof. Notice each round is `O(log n)` time on `O(n+m)` processors. Since a constant fraction of the edges are incident on good vertices and good vertices get eliminated with a constant probability, it follows that the expected number of edges eliminated during an iteration is a constant fraction of the current set of edges. So after `O(log n)` iteration we will have gotten down to the empty set. QED

Remark. By using pairwise independence rather than full independence in the above analysis one can show only `O(log n)` random bits are needed for the algorithm. From this one can derandomize the above algorithm to get an NC algorithm.