Outline

• Finish Up Parallel MIS
• Quiz
• Distributed Algorithms
• Choice Coordination Problem

Introduction

• Last week, we were looking at implementing parallel algorithms on PRAMs.
• We gave a ZNC algorithm for sorting, called BoxSort.
• We then gave the following RNC algorithm for Maximal Independent Set. Recall in a graph a set of vertices I is independent if for any edge `e=(v_i, v_j)` either `v_i` or `v_j` is not in `I`. Such a set is maximal if it is not a proper subset of another independent set.

  Input: G=(V,E)
  Output: A maximal independent set I contained in V
  1. I := emptyset
  2. Repeat {
     a) For all v in V do in parallel
           If d(v) = 0 then add v to I and delete v from V.
           else mark v with probability 1/(2d(v)).
     b) For all (u,v) in E do in parallel
           if both u and v are marked
               then unmark the lower degree vertex.
     c) For all v in V do in parallel
           if v is marked then add v to S
     d) I := I union S
     e) Delete S union Gamma(S) from V and all incident edges from E 
     } Until V is empty.
  

Introduction continued

• As each iteration has depth `O(log n)`, to show this is in RNC we show that the expected number of rounds till the above terminates is `O(log n)`. So we know there is some `k >0`, so if we ran above for `k (log n)^2` steps and look at the output, at least 1/2 the time it will be a maximal independent set.
• To do the analysis, we called a vertex `v` good if it has at least `(d(v))/3` neighbors of degree no more than `d(v)`; otherwise, the vertex is bad. An edge is good if one of its endpoints is good and is bad otherwise.
• Recall also `Gamma(v)` (resp. `Gamma(S)`) is used to denote the set of vertices connected with `v` (resp. `S`) by an edge.
• We have so far proven the following:

  Lemma*. Let `v` in `V` be a good vertex with degree `d(v) > 0`. Then, the probability that some vertex `w in Gamma(v)` gets marks is at least `1- exp(-1/6)`.

  Lemma**. During any iteration, if a vertex `w` is marked then it is selected to be in `S` with probability at least `1/2`.

  Lemma#. The probability that a good vertex belongs to `S cup Gamma(S)` is at least `(1- exp(-1/6))/2`.

Still More Analysis of Parallel MIS

Lemma## In a graph `G=(V,E)`, the number of good edges is at least `|E|/2`.

Proof. Our original graph was undirected. Direct the edges in `E` from the lower degree-point to the higher degree endpoint, breaking ties arbitrarily. Let `d_i(v)` be the indegree of `v` and `d_o(v)` be the out-degree. From the definition of goodness, we have for each bad vertex:
`d_o(v) - d_i(v) ge (d(v))/3 = (d_o(v) + d_i(v) )/3`
For all `S`, `T` contained in `V`, define the subset of the edges `E(S,T)` as those edges directed from vertices in `S` to vertices in `T`; further, let `e(S,T) = |E(S,T)|`. Let `V_G` and `V_B` be the sets of good and bad vertices respectively. The total degree of the bad vertices is given by:
`2e(V_B, V_B) + e(V_B, V_G) + e(V_G, V_B)`
`= sum_(v in V_B) (d_o(v) + d_i(v))`
`le 3 sum_(v in V_B)(d_o(v) - d_i(v))`
`= 3 sum_(v in V_G)(d_i(v) - d_o(v))`
`= 3[(e(V_B, V_G) + e(V_G, V_G)) - (e(V_G, V_B) + e(V_G, V_G))]`
`= 3[e(V_B, V_G) - e(V_G, V_B)]`
`le 3[e(V_B, V_G) + e(V_G, V_B)]`
The first and last expressions in this sequence of inequalities imply that
`e(V_B,V_B) <= e(V_B,V_G) + e(V_G,V_B)`.
Since every bad edge contributes to the left side, and only good edges to the right side, the result follows.

Finishing up Parallel MIS

Theorem. The Parallel MIS algorithm has an EREW PRAM implementation running in expected time `O(log^2 n)` using `O(n+m)` processors.

Proof. Notice each round is `O(log n)` time on `O(n+m)` processors. Since a constant fraction of the edges are incident on good vertices and good vertices get eliminated with a constant probability, it follows that the expected number of edges eliminated during an iteration is a constant fraction of the current set of edges. So after `O(log n)` iteration we will have gotten down to the empty set. QED

• How much randomness does this algorithm use? Notice to mark a vertice we need to use `log (2 cdot d(v))` coin flips, and `d(v)` could be as big as `n`. So in a given round we making use of up to `O(n log n)` coin flips in total across all processors.
• The standard way to try to derandomize an `RNC` algorithm is to try to reduce the randomness used to `O(log n)` and try all the `O(n) = 2^{O( log n)}` possibilities in parallel to find one that works.
• One way to reduce the randomness is to pick two random numbers `0 < a, b < n`, and when processor `i` needs random bits use `r_i = a cdot i +b mod n`.
• It turns out that these `r_i` are pairwise independent and Lemma * can be modified to work with slightly worse bounds and `r_i`, allowing us to derandomize the original RNC algorithm into NC.

Quiz

Which of the following statements is true?

1. All the boxes in the BoxSort algorithm are the same size.
2. If a box is smaller than size `log n`, BoxSort uses a different sorting strategy.
3. Parallel MIS always outputs the lexicographically first independent set.

Distributed Algorithms

• In the PRAM model all of the processors used the same clock so all the processors' computation steps were in sync.
• We will now consider the situation where we have n processors each with its own clock. So a step on one processor might be longer or shorter than some other processor.
• There is a global memory consisting of `m` registers.
• As several processors might attempt to simultaneously read/modify a register, we will assume before a processor accesses a global register it must first get a lock for it.
• While it has the lock of a register, no other processor can access that register and must wait.
• When the lock is released, all waiting processors are notified and can contend for the lock again.

The Choice Coordination Problem

• Memorably Disgusting Motivating story:
  • Have mites which survive by making colonies in the ears of moths.
  • If they infect both ears of the same moth, the moth can't hear bat sonar calls, and it and the mite colonies will be eaten.
  • How can the mites agree on only one ear to infect?
• We will be interested in a computer science variation on this problem called the Choice Coordination Problem.
• We will have `n` processors in our distributed setting.
• We want them to agree on a value between `1` and `m`.
• We will know an agreement has been met at the point when exactly one of the registers our processors have access to contains a special symbol #.
• An `Omega(n^(1/3))` time deterministic lower bound is known for this problem in the distributed model.
• We will show there is a expected constant time randomized algorithm.

Warm-Up Algorithm

• We first consider the simplified case of only two processors and in which these processors are synchronized.
• Let `P[i]` denote a processor, and `C[i]` denote its choice.
• Finally, let `B[i]` be a value which is local to processor `P[i]` only.

Synch-CCP:
Input: Registers C[0] and C[1] initialized to 0. 
Output: Exactly one of the register has the value #.
0 P[i] is initially scanning the register C[i] and 
  has its local variable B[i] initialized to 0.
1 Read the current register and obtain a bit R[i].
2 Select one of three cases: 
   (a) case [R[i] = #]: halt;
   (b) case [R[i] = 0, B[i] = 1]: write # into the current register and halt;
   (c) case [otherwise]: assign an unbiased random bit to B[i] 
       and write B[i] into the current register.
3 P[i] exchanges its current register with P[1-i] and returns to step 1.

Analysis of Synch-CCP

• Let's look at the correctness of Synch-CCP:
  • First notice, at most one register can ever have # written into it. Why? If both registers get the same value `#` then by 2(a) they must have both written `#` in the same iteration. Suppose this happens on the `k`th iteration. Let `B[i]_(k)` and `R[i]_(k)` denote the values used by `P[i]` just after Step 1 of the `k`th iteration. I.e., Just before case 2(b) could be applied in the `k`th round. The previous round must have used case 2(c), so we know `R[0]_(k) = B[1]_(k)` and `R[1]_(k) = B[0]_(k)`. The only way a `#` could be written is if `R[i] =0` and `B[i]=1;` but then `R[1-i] = 1` and `B[1-i]=0`, so `P[1-i]` can't write `#` in that iteration.
• Notice during each iteration, the probability that both `B[i]` have the same value is a `1/2`. If the two bits are ever different then within two stages the algorithm stops. So after `k` steps the odds the algorithm has not stop is `O(1/2^k)`.
• So with odds `1-O(1/(2^k))` the algorithm terminate in `k` steps.