Outline

• Introduction
• Efficiency
• Algorithms and other Technologies

Introduction

• On Monday, we defined the terms algorithm, problem, and instance of a problem.
• We said what it meant for a problem to be solved by an algorithm.
• We described some problems which can be solved with algorithms.
• We said sometimes it was sufficient to get algorithms which worked on some but not all of the the inputs, or that gave the correct answer only with high probability.
• We said what a data structure is, and we gave an argument as to why we still need to study algorithms and data structures even if people already have good libraries for the ones we'll study this semester.
• We said sometimes there are problems which don't have efficient algorithms, so special case solution might be important.
• We mentioned the class of problems, NP, as being one such class for which efficient algorithms are not known in general.
• Today, we are going to start the process of quantifying what makes a good algorithm.

Constraints on Algorithms

• Imagine we lived in a world with arbitrarily fast computers with as much memory as needed.
• Would we still study algorithms?
• Yes, before computers existed people studied algorithms using essentially these kind of assumptions.
• As you will find out in a class like CS154, there are some problems which are not solvable at all by any algorithm.
• To solve a problem, you would still need to come up with an algorithm and show it stops on all inputs with the correct answer. So it would still be useful to build up toolboxes of techniques to design algorithms for which you could carry out such proofs of always stopping and correctness.
• In reality, computers are not arbitrarily fast and computer memory is finite and has a cost.
• Time and space are bounded resources, and we would like our algorithms to run within these constraints.

Efficiency

• Different algorithms for the same problem might differ a lot in how efficiently they use time and space resources.
• These differences can be much bigger than hardware or software difference in where the algorithms run.
• For example, insertion sort solves the sorting problem on n items in time `c_1n^2` where `c_1` is a constant not depending on `n`.
• Merge sort solves sorting in time `c_2 n log n` where in this class log is `log_2` and `c_2` is another constant.
• Typically, `c_1 < c_2`.
• We can rewrite the growth of these two functions as `c_1 n cdot n` and `(c_2 log n)n`. So which is faster depends on the growth of `c_1 n` versus `c_2 log n`.
• `log n` grows very slowly. For example when `n=1000` is it roughly 10 and for `n=1,000,000` it is roughly `20`. So no matter how much bigger `c_2` is than `c_1`, eventually as `n` grows `c_1 n` will be bigger than `c_2 log n`.

Fast versus Slow Machine

• Suppose we had a fast machine A running insertion sort versus a slow machine B running merge sort.
• They each need to sort `10` million numbers. (Even if the numbers were long's, this could be stored in 80 megabytes, so easily storable in ram on modern machines.)
• Suppose A executes `10` billion instructions/second and B executes `10` million instructions/second.
• Suppose as coded insertion sort take `2n^2` instructions to sort `n` numbers and merge sort takes `50nlog n` instructions to sort `n` numbers.
• On A, the run time to sort `10` million items would be:
  `(2cdot(10^7)^2 text(instructions))/(10^(10)text(instruction/second)) = 20,000 text(seconds) approx 5.6 text(hours).`
• On B, the run time to sort 10 million items would be:
  `(50cdot 10^7 log 10^7 text(instructions))/(10^7 text(instructions/second)) approx 1163 text(second)` less than 20 minutes!
• The speed difference is even more pronounced if we had to sort 100 million items.

Class Problem

For each of the following functions `f(n)`, and times `t`, determine the largest size of `n` of a problem that can be solved in time `t`, assuming that the algorithm to solve the problem take `f(n)` microseconds.

Algorithms and other technologies

• We should consider algorithms, like computer hardware, as a technology.
• Total system performance depends on choosing efficient algorithms as much as on choosing fast hardware.
• As with other technologies, advances in algorithms occur all the time.
• You might wonder if other technologies make the study of algorithms obsolete. For example,
  • advanced computer architectures
  • graphical user interfaces
  • Object-oriented systems
  • integrated Web technologies
  • fast networking, both wired and wireless
• The answer is no.
• Although some applications do not explicitly require algorithmic content at the application level many do.

Example Application Level Algorithms

• Consider a Web-based map service that can compute directions from one location to another.
• Not many of these exist. There is Google maps, AOL's Mapquest (probably the oldest, dating from the 1980s), NavTeq, TeleAtlas.
• Open source solutions based on Open Street Map data exists for maps, but route finding is hard enough that third parties only give partial support.
• Although Google, etc, have APIs, it is a dangerous proposition to rely on someone else's API for your business model. Cost structures of such services can change, the service provider might easily replicate your site, or maybe the provider abandons their API entirely. For example, Google has been known to shut down services: Google Reader, Wave, etc.
• If you want to create such a service, its implementation would rely on fast hardware, a graphical user interface, wide area networking, and would probably be coded in an object-oriented fashion.
• It would also require algorithms for operations like finding routes, rendering maps, and interpolating addresses.

Lower Levels Require Algorithms too

• Even an application that doesn't rely on algorithmic content at the application level often relies on algorithms.
• For example, if the application needs fast hardware. That hardware design probably used algorithms.
• Similarly, the design of GUIs othen relies on algorithms (for example, to render them to the screen).
• If the application relied on networking than it relies on algorithms by virtue of the fact that routing makes heavy use of algorithms.
• High level languages rely on algorithms to efficient convert their code into machine code.
• Algorithms are at the core of most technologies in contemporary computers.