On Monday, we began talk about parallel and distributed databases.
We said one of the key differences between these systems is whether we think
communication cost will play or won't play a significant amount in data processing on multiple machines/core.
After talking about different kinds of parallel DB architectures (shared memory, shared disk, shared nothing),
we said how to implement tuple at a time and full relation, relational algebra operations in the parallel shared nothing
set-up.
We didn't look at the I/O cost for the last of these.
Today, we review full relation queries in the parallel set-up and then talk about its I/O cost.
We then look at the map reduce model of parallelism.
Parallel Algorithms for Full-Relation Operations - General Case
Similar techniques work for Union, Intersection, Difference, Grouping, and Join.
For these operations, each machine `M` runs the following code:
Determine a hash function `h` for the query operation at hand (each machine should compute the same hash function). For example, for the query of the previous slide we chose a hash function that depended only on SALARY. For join, you might choose a hash function that depends only on the join attributes.
Compute `h(t)` for each tuple in `R` (in the case of a binary operation like `R cup S` for each tuple in `R` and `S`)
The output of the hash function should be a machine number `i` for a machine `M_i` that should handle that particular tuple.
For each `i`, send all the tuples `t` with hash `i` to machine `M_i`.
After receiving all the tuple that other machines have sent to it, `M` computes the relational algebra operation on its received portion of the tuples and outputs the result.
Cost of Parallel Algorithm for Full-Relations
Suppose we wanted to use the algorithm of the last slide to compute in parallel on `N`
processors one binary operation on `R` and `S`. For concreteness, let's say $R \bowtie S$.
Step (1) takes no IOs.
Step (2) and (3) is applied to all the blocks that make up `R` and `S`, so will take `B(R) + B(S)` total IOs. If each machine has its own disk then time to do these IOs will be `(B(R) + B(S))/N`.
Step(4) involves sending over the network `(N-1)/(N)(B(R) + B(S))` blocks, which hope is of negligible cost, and then doing `(B(R) + B(S))/N` on each machine to write its received data to disk.
Step(5) using our algorithm from earlier in the semester would take `(3(B(R) + B(S)))/N` IOs.
So the total time is `(5(B(R) + B(S)))/N` disk IOs, which for `N ge 2` is faster than `3(B(R) + B(S))` we needed to do the operation on a single machine.
Map-Reduce Parallelism Framework
Map-Reduce is a framework developed at Google for doing massively parallel computations on very large datasets.
It is often used to compute properties of the web-graph such as Page Rank.
The original paper in which it was described is Dean and Ghemawat (2004).
The Basic Framework
Map-Reduce was inspired by the map and reduce functions found in functional programming languages such as Lisp.
The map function takes as its argument a function `f` and a list of elements `l = langle l_1, ..., l_n rangle`. It returns a new list
`map(f,l) = langle f(l_1), ..., f(l_n) rangle`.
The reduce function takes a function `g` and a list of elements `l = langle l_1, ..., l_n rangle`. It returns a new element `l'` such that
`l' = \r\e\d\u\c\e(g,l) = g(l_1,g(l_2, g(l_3, ... )))`.
From a high-level view, a Map-Reduce program creates a sequence of key/value pairs, performs some computations on them, and outputs another sequence of key/value pairs.
Keys and values are often strings, but may be of any data type.
Distinct Phases of a Map-Reduce Job
Map Phase, key/value pairs are read from the input and the map function is applied to each of them individually. The function is of the general form:
`map: langle k, v rangle |-> langle langle k_1, v_1 rangle, langle k_2, v_2 rangle, ...,rangle`
Shuffle Phase, the pairs produced during the map phase are sorted by their key, and all values for the same key are grouped together.
Reduce Phase, the reduce function is applied to each key and its values.
`\r\e\d\u\c\e: langle k, langle v_1, v_2, ... rangle rangle |-> langle k, langle v_1', v_2', ... rangle rangle`
That is, for each key the reduce function processes the list of associated values and outputs another list of values. The output values and their number may not be the same as the input values.
Example Map-Reduce Jobs for Counting
This program consist of two parts:
The map job:
/* imagine k is a document key (for example a URL or a table ID),
v is a document (such as a web page, or a text column of a table)
*/
map(k, v)
split v into tokens
for each token t do
output (t, 1)
return
and the reduce job:
reduce (k, [v[1], v[2], ..., v[n]])
count := 0
for i := 1 to n do
count += v //v might be not 1 depending on MR optimization in use
output (count)
return
One technique to reduce the network traffic for Map-Reduce jobs is to use a combiner.
A combiner modifies the map job so that each map worker also does the shuffle/reduce phase on its portion of the data before sending the result to the corresponding reduce worker.
I.e., before sending data to some reducer `j`, the combiner would first run the reduce job on this data then send it.
In-Class Exercise
Using the map reduce framework of the last couple slides, make a map-reduce job that computes average document size for documents in a corpus.
The difference between a distributed database system and a shared-nothing parallel database system is that for the former we assume network communication is expensive, and for the latter we don't.
Some situations where we might use a distributed database are:
A bank may have several branches. Each branch might have a database of its accounts, but customers can choose to bank at any branch. Each of these databases might be further replicated offsite so as to be recoverable in the case of a fire.
A department store chain has individual stores each with their own database of sales. We also have a central database used to keep track of chain-wide inventory.
A digital library might consist of a collection of universities each of which holds different on-line books and documents. We might want to be able to do a search across all of these collecte and Rd universities for a particular book.
Notice we can often view these system as being a big database partitioned across machines.
For example, each chain in the department store chain might have a table Sales(item, date, price, purchaser), so we can imagine they each store some tuples of a big global table.
We now look at how to handle transactions in this distributed set-up.
Distributed Query Processing
We have already said how to do query processing in the parallel shared nothing architecture setting.
Recall one step in computing a join was for each machine to send each of the other machines, its fraction of the join. If the network is relatively low-capacity this might be very slow.
To fix a simple example suppose we want to compute $R(X,Y) \bowtie S(Y, Z)$ and `R` and `S` are on different machines.
A naive way to compute the join might be:
Determine who has the smaller table.
Send a copy of the smaller table to the other site, then compute the join there.
Ideally, we'd like to send only the relevant part of the query to the other site.
Semi-join Reductions
Assume `R` is the smaller table
To facilitate only sending data needed to compute the join,
we introduce the notion of a semi-join, $R \ltimes S := R \bowtie (\pi_Y(S))$.
Notice the project of `\pi_Y(S)` will return precisely those values of `Y` which might join with $R$ in $R(X,Y) \bowtie S(Y, Z)$.
To minimize the communication between the two sites, we can then do
the following steps to compute the join:
The machine with `S` computes `\pi_Y(S)` sends it the machine with `R`.
The machine with `R` computes $R \ltimes S$ sends it to the machine with `S`.
The machine with `S` uses $R \ltimes S$ to compute $R(X,Y) \bowtie S(Y, Z)$.
For this to be an improvement over the naive algorithm, it should be the case that $R \ltimes S$ is smaller than `R`.
That is, `R` contains many tuples which are dangling (don't join with `S`).
In the case of joins of many relations, it becomes important to identify which semi-joins to compute and in what order. The book gives algorithms for this that we will skip.
Hopefully, though, the above example gives a flavor of how query processing might change in the distributed setting.
