Introduction

On Monday we introduced heaps.
Recall a heap is a binary tree with all levels except maybe the last filled in.
Each node of a heap has some value (and maybe some other auxiliary information) associated with it.
A heap can be stored in an array `A` by storing the value and other data for the root in `A[1]`. If we store the data of node `n_i` in `A[i]`, then we store the data of its left child in `A[2i]` and its right child in `A[2i+1]`
A heap enjoys the max-heap property if the value stored in the parent of a node `n` is always greater than or equal to the value stored in `n`.
A max-heap is a heap that satisfies the max-heap property. (We can define min-heap-property and min-heap analogously.)
We can use a max-heap to implement sorting if we can take an array then convert it into a heap, pluck the maximal element out of the heap (which will always be in `A[1]`), then re-heapify, and continue in this manner until we have taken all the elements out of the heap from biggest to least.
Besides sorting heaps can also be used as priority queues.

Maintaining the Max-Heap Property

Suppose we have a heap stored in array `A` and an index `i`.
Let LEFT(i) and RIGHT(i) be the roots of the binary trees corresponding to the left and right children of `i`.
Suppose both of these trees are max-heaps, but that `A[i]` might be smaller than one of its children.
We would like a procedure that quickly restores the max-heap property.

The idea is to "percolate down" this smaller value. Consider:

MAX-HEAPIFY(A,i)
 1  l = LEFT(i)
 2  r = RIGHT(i)
 3  if l <= A.heap_size and A[l] > A[i]
 4      largest = l
 5  else largest = i
 6  if r <= A.heap_size and A[r] > A[largest]
 7      largest = r
 8  if largest != i
 9      exchange A[i] with A[largest]
10      MAX-HEAPIFY(A, largest)

Remarks on MAX-HEAPIFY

Lines 3-8 ensure that `lar\g\est` has the index among `i`, LEFT(`i`) and RIGHT(`i`) such that `A[lar\g\est]` is as big as possible.
If `lar\g\est` is not `i` then we swap their positions in the tree and make a recursive call on the subtree.
The recursion will stop at or before a leaf node.
(a) above shows a starting configuration of a tree A for MAX-HEAPIFY(A, 2). This makes the recursive calls MAX-HEAPIFY(A, 4) and MAX-HEAPIFY(A, 9) before stopping.
Notice when the procedure is done, the tree rooted at A[2] is a max heap.

Runtime of MAX-HEAPIFY

Lines 1-9 above are all `Theta(1)` time.
The worst case is when the subtrees for the recursive call has size close to `(2n)/3`.
This can occur when the bottom level of the tree is exactly half full.
Therefore the run-time of MAX-HEAPIFY is given by the recurrence
`T(n) le T((2n)/3) + Theta(1)`.
This falls into case 2 of the Master Theorem and so `T(n) = O(log n)`. Alternatively, we can bound the run-time in terms of the height of the heap rooted at `i`. Since our heap is a complete binary tree except the last level, this give the same answer.

Building a Heap

Using MAX-HEAPIFY, we can convert any array of values A[1..n] into a heap with the following code:

BUILD-MAX-HEAP(A)
1 A.heap-size = A.length
2 for i = floor(A.length/2) downto 1
3     MAX-HEAPIFY(A, i)

Let `n = A`.length. So the above makes `n/2` calls to an `O(log n)` procedure. So the algorithm above will be `O(n log n)`.
You can actually show with a bit more work that it is `O(n)`, but we skip this as the overall sorter we will build in `O(n log n)`.
To see that the algorithm does create a heap, we can show the following is a loop invariant:
At the start of each iteration of the for loop of lines 2-3, each node `i+1`, `i+2`, ..., `n` is the root of a max-heap.
Proof.

Initialization
Prior to the first iteration of the loop `i = lfloor n/2 rfloor`. Each node `lfloor n/2 rfloor +1`,`lfloor n/2 rfloor +2`, ..., `n` is a leaf and is thus the root of a max-heap.

Maintenance
To see that each iteration maintains the loop invariant, notice that the children of `i` are `2i > i` and `2i+1 > i`. By the loop invariant, they are both roots of max-heaps. This is precisely the condition required for the call MAX-HEAPIFY(A, i) to make node `i` a max-heap root. Moreover, the MAX-HEAPIFY call preserves the property that `i+1`, `i+2`,..., `n` are roots of max-heaps. Decrementing `i` in the for loop update reestablishes the loop invariant for the next iteration.

Termination
At termination `i=0`, By the loop invariant, each node `1,2,..., n` is the root of a max-heap, so in particular, node 1 is.

BUILD-MAX-HEAP Example

A sequence of trees illustrating BUILD-MAX-HEAP

Heapsort

We now give the pseudo-code for a sorting algorithm based on heaps. On the next slide we give some intuition and estimate its runtime.

HEAPSORT(A)
1 BUILD-MAX-HEAP(A)
2 for i = A.length downto 2
3     exchange A[1] with A[i]
4     A.heap-size = A.heap-size - 1
5     MAX-HEAPIFY(A, 1)

Remarks on Heapsort

The heapsort algorithm takes as input an array `A[1..n]`
It then builds a heap from these in `O(n log n)` time.
At this point `A(1)` is the largest item. So we exchange it with `A[n]`.
We then reduce `A`.heap_size by 1.
The only element which could make the result not a heap would be then at `A[1]`. So we call MAX-HEAPIFY to restore the property.
So `A[1]` would now have the next to largest element and we could swap it with `A[n-1]` and continue in this fashion.
So we have an initial `O(nlog n)` start-up cost to make a max-heap, followed by a loop with at most `n-1` calls to MAX-HEAPIFY each of which will take `O(log n)` time. So the total run-time will be `O(n log n)`.

Priority Queues

Heapsort is a fast algorithm, but quicksort, which we will present next week, tends to beat it in practice.
Nevertheless, the heap data structure itself has many uses.
One of these is to serve as an efficient priority queue.
As with heaps, priority queues come in two flavors: There are max-priority queues and min-priority queues. We will mainly consider the former.
A priority queue is a data structure for maintaining a set `S` of elements, each with an associated value called a key.
A max-priority queue supports the following operations:

INSERT(S, x)
inserts the element `x` into the set `S`. This makes `S := S cup {x}`.

MAXIMUM(S)
returns the element of `S` with the largest key.

EXTRACT-MAX(S)
removes and returns the element of `S` with the largest key.

INCREASE-KEY(S, x, k)
increases the value of `x`'s key to the new value `k`, which is assumed to be at least as large as `x`'s current value.

Example use of a Max Priority Queue

Priority queues can be used to schedule jobs on a shared computer.
The max-priority queue keeps track of the jobs to be performed and their relative priorities.
When a job is finished or interrupted, the scheduler selects the highest-priority job from among those pending by calling EXTRACT-MAX.
The scheduler can add a new job to the queue at any time by calling INSERT.
INCREASE_KEY can be used to increase a job's priority.

Example use of a Min-Priority Queue

A min-priority queue supports the operations INSERT, MINIMUM, EXTRACT-MIN, and DECREASE KEY.
It might be used in an event driven simulator.
Items in the queue represent events to be simulated, each with an associated time of occurrence that serves as its key.
So the events need to be simulated in order of their time of occurrence.
The simulator calls EXTRACT-MIN at each step to choose the next event to simulate.
As new events are produced, the simulator inserts then into the min-priority queue using INSERT.

Auxiliary data

In both job scheduling and event simulation, elements of a priority queue correspond to objects in the application.
We often need to determine which application object corresponds to a given priority queue element and vice versa.
To do this, a handle corresponding to the application object is stored in each heap element.
the exact make up of the handle depends on the application.
Similarly, a handle such as an array index to the corresponding heap element might be stored in the application object.
Because heap elements often change locations after a given heap operation, we must have code to update the application object's handle as necessary when a heap operation occurs.

Using Heaps for Priority Queues

Below is max-heap pseudo-code for each of the priority queue operations. We give the run-times as comments.

HEAP-MAXIMUM(A)
// This is an O(1) algorithm
1 return A[1]

HEAP-EXTRACT-MAX(A)
// This is an O(log n) algorithm
1 if A.heap-size < 1
2     error "heap underflow"
3 max = A[1]
4 A[1] = A[A.heap-size]
5 A.heap-size = A.heap-size - 1
6 MAX-HEAPIFY(A,1)
7 return max

HEAP-INCREASE-KEY(A, i, key)
// This is an O(log n) algorithm
1 if key < A[i]
2     error "new key is smaller than current key"
3 A[i] = key
4 while i > 1 and A[parent(i)] < A[i]
5     exchange A[i] with A[Parent(i)]
6     i = Parent(i)

MAX-HEAP-INSERT(A, key)
// This is an O(log n) algorithm
1 A.heap-size = A.heap-size + 1
2 A[A.heap-size] = - infty
3 HEAP-INCREASE-KEY(A, A.heap-size, key)

Heapsort, Priority Queues

Outline