Outline

• Sorting on PRAMs
• Quiz

Introduction

• On Wednesday, we started talking about parallel random access machines (PRAMs).
• In this parallel model, we have a collection of processors each using the same instruction set and clock but able to run in parallel and able to read and write a global memory.
• We considered different kind of conflict resolution for accessing the global memory: Exclusive Read/Exclusive Write (EREW), Concurrent Read/Exlusive Write(CREW), and Concurrent Read/Concurrent Write (CRCW).
• We also considered a few different complexity classes related to PRAMs:
  • NC - (Nick's Class) languages recognized on a PRAM using polylog time on polynomially many processors.
  • RNC - languages recognized by PRAMs with access to random number generators. I.e., such that if `x in L` the PRAM algorithm accepts at least 50% of the time, if it's not, the algorithm always rejects.
  • ZNC = RNC `cap` co-RNC.
• We said we can also create analogous function classes for each of these and that we'd abuse notation and call these NC, RNC, and ZNC as well.
• We begin today by looking at a ZNC algorithm for sorting

Sorting on a PRAM

We now work towards a ZNC algorithm for sorting which run in `O(log n)` time doing a total of `O(n log n)` operations on all processors (Reischuk 1985). Let `P_i` denote the `i`th processor.

We begin by considering a PRAM variant of Quicksort:

1. If `n=1` stop.
2. Otherwise, pick a splitter uniformly at random from the `n` input elements
3. Each processor determines whether its element is bigger or smaller than the splitter.
4. Let `j` denote the rank of the splitter. If `j` is not in `[n/4, (3n)/4]` the step is declared a failure and we go back to step 1. Otherwise, we move the splitter to processor `P_j`. Each element that is smaller than the splitter is moved to a distinct processor is `P_i` such that `i < j`. Each element that is larger than the splitter is moved to a distinct processor `P_k` where `k > j`.
5. We sort recursively the data in the processors `P_1` through `P_(j-1)` and the data in `P_(j+1)` through `P_(n)`.

Notice this algorithm uses randomness, but it has zero error -- we can detect if we are in a bad case in which case we rerun -- it is in this sense it is a ZNC algorithm for sorting.

Analysis

• The third step above is O(1) time.
• To do step 4, we assume we have a global input array A and a global output arrays `B`, `C` and `S`. Each processor `P_i` sets a bit `b_i` in `B` in one of the global registers to `0` if its element is greater than the splitter and to `1` otherwise.
• For all `i`, let `S_i=Sigma_(t<=i)b_t`. Using a `log`-depth tree of processors, all the `S_i`'s can be computed in parallel in `O(log n)` steps into `S`. This allows us to map into the output array those elements which are smaller than `j`. Namely, each processor in parallel checks, if `b_i` is 1 (where `i` is its global id), if so, it maps the element in `A_i` to `C_(S_i)`. Using a similar setting of bits and computing sums, we can also handle all those element which should be mapped to locations larger than `j`. From this we can do stage 4 in `O(log n)` steps.
• The expected number of time before step 1-4 succeeds is `1/(1/2) = 2` since half the `j`'s are in the interval `[n/4, (3n)/4]`. So the total runtime will be `O(log^2 n)`.

Quiz

Using More Splitters

Suppose we have n processors and `n` elements. Suppose the first `r` processors have values in sorted order.

• We can use these `r` elements as splitters to sort the remaining `n-r` elements.
• The goal is to insert the `n-r` unsorted elements among the splitters in the following sense:
  • Each processor should end up with a distinct input element.
  • Let `s_j` denote the `j`th largest splitter and `i_(s_j)` denote the index of the processor containing `s_j` after the insertion. Then for all `k < i_(s_j)`, processor `P_k` contains an element that is smaller than `s_j` and for all `k>i_(s_j)`, `P_k` contains an element that is larger than `s_j`.
• We hope we can do this in `O(log n)` time.

A Good Choice for `r`, a Complete Algorithm (called BoxSort)

1. If so, we could pick `n^(1/2)` elements at random. Then using all `n` processors sort them in `O(log n)` steps.
2. Then using these sorted elements insert the remaining elements among them in `O(log n)` steps .
3. Treat the remaining elements that are inserted between splitter as subproblems, recur on each subproblem whose size exceed `log n`, otherwise, use LogSort:
   • Compare each element in parallel with its neighbor first to its left, swap if necessary; then to its right, swap if necessary, do this O(log n) times.

Intuition on Splitting

• Doing splitting should take `O(log mbox((the size of box we need to split)))`.
• In a perfect world, at each level the expected size of the box goes down by a square root, so we get the sum
  `O(log n + log n^(1/2) + log n^(1/4) +...) = O(log n +1/2log n +1/4logn +...) = `
  `O((log n) cdot (1+ 1/2+1/4+ ...)) = O(log n)`
• We will argue even in a non perfect world that with high probability the sum of the log of the sizes of the boxes along any path is `O(log n)`, so the runtime will be `O(log n)`.

Analysis of Splitting

• To see the intuition of the last slide is true, partition the interval `[1,n]` into sub-intervals `I_0, I_1, ...` We will then bound the probability that a box whose size is in `I_k` has a child whose size is also in `I_k`.
• Fix `gamma` and `d` such that `1/2 < gamma < 1` and `1 < d < 1/gamma`. For positive integers `k`, let `tau_k=d^k`, `rho_k= n^(gamma^k)`. Define `I_k=[rho_(k+1), rho_k]`.
• `n = (log n)^{log n log_(log n)2}`. So if `gamma_k > 1/ log n loglog n^2`, then `rho_k= n^( gamma^k) < log n`. This will happen for some `k < c log log n`.
• So we will only be interested in `O(log log n)` many intervals `I_k`.
• For a box `B` in the tree, we let `alpha(B)=k` if `|B|` is in `I_k`.
• In terms of our notation, the time to split Box `B` is `O(log rho_(alpha(B)))` .
• For a root-leaf path, `P = (B_1, ..., B_t)`, the runtime is given by `sum_(j=1)^t log rho_(alpha(B_j))`.
• The total runtime of the algorithm will be O of this plus log n (to sort the leaves).
• Define the event `E_P` to be that the sequence `alpha(B_1), ..., alpha(B_t)` does not contain the value `k` more than `tau_k` times for `1<= k <= c log log n`.
• If `E_P` holds then the number of PRAM steps on path `P` will be:
  `O(log n + sum_(k=1)^infty log tau_k gamma^k log n))`