Outline

• More About Map Reduce
• HW Problem
• Map Reduce for Page Rank

Recalling Map Reduce

• So far we talked about map reduce as a model of parallel computation.
• As described a map reduce job had three phases: (a) a map phase, where key/value pairs are read from the input and the map function is applied to each of them individually, (b) a shuffle phase, where the pairs produced during the map phase are sorted by their key, and all values for the same key are grouped together, (c) a reduce phase where the reduce function is applied to each key and its values.
• We have machines (nodes) which either act as mappers. They apply the map function to a subset of the key value pairs.
• They then send the results to different reducer machines based on a hash of the key value for a key-value output by the map phase.
• Before the reducer begins reducing the data from all its inputs, the data is typically stored in a distributed file system, in case the reducer fails -- that way we don't have to recompute the map phase it depended on.
• Often mappers and reducers are replicated three ways to keep things operating smoothly.

Recalling Page Rank

The goal of page rank is to find a vector `\vec{r}` such that `\vec{r} = \vec{A}\vec{r}`. Here `A` is the so-called Google matrix which was a modified form of the adjacency matrix where we normalized the rows and tacked on some other conditions.
• These other conditions guaranteed that if we started with some guess `vec{r'}` for `vec(r)` and we computed powers `\vec{A}^{(n)}\vec{r'}` until
  `||\vec{A}^{(n+1)}\vec{r'} - \vec{A}^{(n)}\vec{r'}|| < \epsilon`
  then `\vec{A}^{(n)}\vec{r'} approx \vec{r}`.
• Why is computing these powers hard?
• Why would we even want to try to use Map Reduce to compute this?
• Powering seems to involve some kind of iteration. So we at least need to extend our basic map reduce to handle iterating the map and reduce operations.

The Scary Numbers Behind the Google Matrix

• At first blush because of the teleportation correction every single entry in the Google matrix is nonzero.
• How many entries are there? Well the number of rows is the number of web pages, and the number of columns is the number of web pages.
• So conservatively it is a `10^(10) times 10^(10)` matrix. i.e, it has `10^(20)` entries.
• Let `n` be the rank of the matrix (in this case `10^(10)`). Multiplying the matrix against `vec{r'}` takes time `O(n^2)` using the usual way to do matrix multiplication.
• So 50 iterations would take time `O(50n^2)` or order `5 times 10^(21)` operations!
• Currently, a typical processor does less than `10^(10)` ops per second. So it would take `5 times 10^(11)` seconds or about `15855` years to do on a single machine.
• There are fast matrix multiply techniques based on techniques originally due to Strassen (1969), but these tend to be hard to use in practice -- and it wouldn't save enough to be feasible on one machine.
• The key thing that makes computing `vec(r)` feasible is that the Google matrix is essentially, a normalized adjacency matrix (which only has a small number of nonzero entries), together with corrections which can be computed in linear time in the rank.
• So multiplying against `vec(r')` can be done in time proportional to the square of the nonzero entries in the adjacency matrix.
• If we assume any given page links to on average `50` or so other pages, this reduces the time of a single matrix to something like `O(50n)` and the total time to something like `O(2500n)` -- still huge if `n = 10^10`, but doable if we can split the job among several machines.

High-Level Parallel Algorithm for Page Rank

Let epsilon be the constant we used to decide if we stop
Compute initial current_r, 
do {
    Store in distributed file system (DFS) current_r as old_r
    Do map reduce job to compute A*current_r 
        where A is the normalized adjacency matrix
    Store result in DFS as current_r.
    Do map reduce job to compute dangling node correction to current_r
    Store result in DFS as current_r.
    Do map reduce job to compute teleporter correction to current_r
    Store result in DFS as current_r.
    Do map reduce job to compute len = || current_r - old_r||
} while(len > epsilon)
output current_r as page ranks

• As we can see, we iteratively apply map reduce jobs to data stored in the distributed file system
• The amount of space we need to store stuff in the DFS is proportional to the nonzero entries in `A` and `vec(r)`.
• Howard Karloff, Siddharth Suri, and Sergei Vassilvitskii (2010) studied what kinds of computations can be done by iteratively applying map reduce jobs `t` times , where each machine can store at most `n^(1 - epsilon)` of the input. They get the following intersting result:
  Any CREW PRAM algorithm using `O(n^(2 - 2 epsilon))` total memory, `O(n^(2 - 2 epsilon))` processors, and `t = t(n)` time can be run in `O(t)` rounds in DMRC.

HW Problem

Exercise 14.6. (a) Design a MapReduce that computes the average document length in a corpus of text.

Answer. The average document length is the total number of tokens divided by the total number of documents. Let our initial key value pairs be (docid, document). Our mapper will produce key value pairs of the form ("doc_count", 1) and ("token_count", 1) according to the following (we assume or arrange both keys go to same reducer):

map(docid, document) {
  output("doc_count", 1);
  for each token t in document {
     output ("token_count", 1);
  }
  return;
}

The reducer then adds these pairs as follows:

reduce(key, (v[1],... v[n])) { // remark v[i] will always be 1
    static total_docs = 0;
    static total_tokens = 0;
    for i = 1 to n {
        if(key == "doc_count") {
            total_docs += v[i];
        }
        if(key == "token_count") {
            total_tokens += v[i];
        }
    }
    if(total_docs > 0) {
       return total_tokens/total_docs;
    }
    return -1;
}

We assume if we see -1 we should wait for the second return value of the function

(b) Revise your reduce function so that it can be used as a combiner.

Answer. To make a combiner we should output ("doc_count", total_count) and ("token_count", total_tokens) in addition to the average. We assume things output from the combiner are sent to the reducer, and that the return value is "local" to the machine.

What do some of the map reduce jobs for page rank look like?

• The following example of computing A*current_r for the normalized adjacency matrix is from Lin and Dyer (2010):
  
• My hope is this gives a little bit of the flavor of how MapReduce is used.
• MapReduce can also be used to compute inverted indexes as part of indexing.