Outline

• Insertion Sort
• Quiz

Introduction

• Last week, we defined the term algorithm, said what we meant by a computational problem, and said what it means for an algorithm to solve a computational problem.
• We then compared how long it would take to sort 10 million items using insertion sort versus mergesort on a fast versus slow machine given the functions `2n^2` and `50nlog n` as the number of instructions it takes for each of these to sort `n` items.
• Today, we are going to introduce the basic framework which we will use for the rest of the semester for designing and analyzing algorithms.
• We start by looking at the Insertion Sort algorithm.

Pseudo-code

• Our first algorithm, insertion sort, solve the sorting problem that we discussed last week:
  Input: A sequence of `n` numbers `langle a_1, ... a_n rangle`.
  Output: A permutation `langle a'_1, ..., a'_n rangle` of the input sequence such that `a'_1 leq a'_2 leq ... leq a'_n.`
• The numbers we wish to sort are sometimes called keys.
• We will describe algorithms in pseudocode. This is similar to writing them in C, C++, Java, Python, etc.
• The difference between pseudocode and real code is that in pseudocode, we are allowed to say things in the clearest more concise way needed to specify the algorithm.
• Sometimes this will involves using a short English phrase or sentence in a section of code
• Pseudocode also does not concern itself with issues of software engineering such as data abstraction, modularity, and error handling.

Insertion Sort -- Intuition

• This is somewhat like the way some people sort a hand of playing cards.
• We start with an empty left hand and the cards face down on the table. We remove one card at a time from the table and insert it into the correct position in the left hand.
• To find the correct position for a card, we compare it with the cards from right to left until we find its correct position.
• At all times, in the process the cards held in the left hand are sorted.
• We will call our pseudocode procedure for this kind of sorting INSERTION-SORT.
• It takes as input an array`A[1..n]` containing the sequence of length `n` to be sorted. For such an array we use A.length to denote its length.
• Our algorithm sorts the input in place, it rearranges the numbers within `A` with at most a constant number of them stored outside the array at any time.
• When completed, the input array `A` contains the sorted output sequence.

Quiz

INSERTION-SORT(A) Pseudocode

1 for j=2 to A.length
2     key = A[j]
3     // Insert A[j] into the sorted sequence A[1..j-1]
4     i = j - 1
5     while i > 0 and A[i] > key
6         A[i+1] = A[i]
7         i = i - 1
8     A[i + 1] = key

Example

• Consider the input `A = langle 5, 2, 4, 6, 1, 3 rangle`.
• When `j=2`, `key = A[2] = 2`. In the while loop we compare `A[1] = 5` and `key` and set `A[2] = A[1] = 5`. Then outside the while loop we set A[1] = key. So we get the sequence `A = langle 2, 5, 4, 6, 1, 3 rangle`.
• The figure above shows what happens for `j=2,3,4,5,6`. The black square indicate the current value of `j`, the gray squares indicates whats sorted so far, the gray arrows indicate right moves, and the black arrow indicates where we put the key.
• At the beginning of the for loop, the subarray `A[1, .. j -1]` constitutes the currently sorted hand, and `A[j+1, .. n]` corresponds to the pile of cards still on the table.
• Moreover, the elements `A[1 .. j-1]` are the elements which were orginally in positions 1 to j-1, but now in sorted order.
• This can be stated as a loop invariant:
  

  At the start of each iteration of the for loop of lines 1-8, `A[1, .. j-1]` consists of the elements originally in `A[1 .. j-1]`, but in sorted order.

• Loop invariants used to help understand why an algorithm is correct.

Loop Invariants

• To show something is a loop invariant, we need to establish the following three things about it:

  Initialization

  It is true prior to the first iteration of the loop

  Maintenance

  If it is true before an iteration of the loop, it remains true before the next iteration.

  Termination

  When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct.

• Showing the above is a little like carrying out an induction argument. Initialization is like the base case of such an argument, and maintenance is like proving the induction step.
• If the first two properties hold, the loop invariant is true prior to every iteration of the loop.
• Often we use the loop invariant along with the condition that caused the loop to terminate to get the Termination property we want.

Proving our INSERTION-SORT Loop Invariant

Initialization

We need to show our loop invariant holds when `j=2`. In this case the subarray `A[1.. j-1]` consists of the single element `A[1]`, which is in fact the original element `A[1]`, and it is also trivially sorted.

Maintenance

We need to show that each iteration maintains the loop invariant. Informally, the body of the for loop works by moving `A[j-1]`, `A[j-2]`, ... and so on by one position to the right until it finds the proper position for `A[j]` (lines 4-7), at which point it inserts the value of `A[j]` (line 8). The subarray `A[1..j]` then consists of the elements originally in `A[1 ..j]`, but in sorted order. Incrementing `j` for the next iteration then preserves the loop invariant.

Termination

What happens when the loop terminates? This happens when `j > Atext(.length)=n`. At that time we have `j = n +1`. Substituting `n+1` for the loop invariant, we have the subarray `A[1..n]` consists of the elements originally in `A[1..n]`, but in sorted order. I.e., the whole array is sorted. So the algorithm is correct.

Remarks on Code

• Like Python, we indicate code blocks by the level of indentation -- each change in indention indicates a new code block or the resumption of an old one.
• We are using // for comments
• while, for, repeat-until and if-else behave like in most modern programming languages
• Like in Java, C, etc we allow multi assignments. `i=j=e`. This assigns `e` to `j` first and the output is `e`, which is then assigned to `i`.
• We use `A[j]` to indicate the `j`th element of the array. We write `A[1..j]` to indicate the subarray consisting of the elements `A[1], A[2], ..., A[j]`.
• We use `.` to indicate we are referencing an attribute of an object. For example `A text{.length}`. The dot notation can be nested: `x.f.g`
• Sometimes NIL will be used for a pointer to no object.
• We pass parameters by value. Changing values of parameters in a procedure does not affect the value of the variable seen by the calling procedure.
• Arrays are passed by pointer. So changing the contents of an array is seen by the calling procedure.
• A return statement transfers control back to the point of the call in the calling procedure
• Boolean operators have "short circuit" semantics. So if have (`x==1` and `y==2`), and x is in fact 6, so x==1 is false, then we never bother evaluating the rest of the expression as we already know its whole value will be false. Similarly, if we have `(x or y)` and `x` is true we don't evaluate `y`. Short circuit behavior is useful for expressions like `x ne NIL and x.f = y`. Since if `x==NIL` we don't have to worry about what happens when we evaluate `x.f=y`.
• The keyword error indicates that an error occurred because conditions were wrong for the procedure to have been called. It is returned to the calling procedure which would then handle it.