Introduction

So far this semester we have been looking at search strategies for problem solving agents for a single category of problems.
We have been looking at problems where the environment is observable, deterministic, and known and where the solution is a sequence of action.
We now try to relax these assumptions in different ways...

Local Search

So far as part of the solution to a problem, we have been interested in the sequence of actions which take us to the goal state.
For some problems, this sequence of actions is not needed. For example, a solution to the 8-queens problem is eight non-attacking queens on a chess board -- we don't care so much how they were found.
This property holds of many problems. For example, IC Design, factory-floor layout, job scheduling, network optimization, vehicle routing, portfolio management, etc.
A local search algorithm operates using a single current node and generally moves to neighbors of that node. We don't keep track of the path history.
In addition to just finding goals, local search algorithms are often used in optimization problems: problems in which the aim is to find the best state with respect to a objective function.
The search process can be viewed in terms of the state space landscape. The landscape has both a "location" (which state we are currently at) and an "elevation" (what the objective function is for this state -- some real number).
Plotting this scalar-valued function `c:S -> RR` we can find local maxima which would be states so that no neighboring state has as large/small a value of `c`, we might find global maxima/minima which have larger/smaller values than any other values corresponding to states in the state space. There might be other phenomena, like shoulders, saddles, "flat" local maxima, etc.
We say a local search algorithm is complete if it always finds a goal if one exists. We say a local search algorithm is optimal, if it can always finds a global minimum/maximum.

Quiz

Which of the following is true?

Effective branching factor can help in determining which heuristic is better amongst a group of `A^star` heuristics.
Coroutines are functions which output a sequence of values when their next() method is called
Python does not support multiple inheritance of classes.

Hill Climbing

function HILL_CLIMBING(problem) returns a state that is a local maximum
    current := MAKE_NODE(problem, INITIAL-STATE)
    loop do
        neighbor := a highest-valued successor of current
        if(neighbor.value ≤ current.value) return current.state
        current := neighbor

The above is the so-called hill-climbing algorithm. It follows the strategy of looking at ones neighbors and always picking the neighbor which moves one in the most upward direction.
If no such neighbors exist, we return a solution. This will be a local maxima, but not necessarily a global maxima.
It has no memory of previous moves.
For example, we could apply this algorithm to the eight queens problem: A state would consist of eight queens on the board, one per column.
The successor state consist of all states which can be derived by moving one queen to a different square.
The heuristic cost function `h` is the number of pairs of queen that are attacking each other.

Hill Climbing

two 8-queens boards, one with h=17, one with h=1

The left board above has `h=17`, the right `h=1`.
Hill-climbing is sometimes called greedy-local search.
Hill-climbing will usually choose randomly between two sucessors if they have the same cost.
From a bad solution, hill-climbing often makes quite rapid progress to reach a close to optimal solution. For example, the second board on the right above is reached in five steps from the initial board on the left.
Unfortunately, it can get stuck on local maxima (the right board above is one). You can also have ridges and plateaux.

Variants of Hill-Climbing

To avoid local minima, and also to hopefully solve the plateaux problem is to allow moves that move to the same value as the current state. i.e., a sideways move.
This could lead to an infinite loop, depending on the plateaux, so typically one puts limit on the total number of sideways moves.
With a limit of a 100 consecutive sideways moves, the chance of solving the 8-puzzle given a random starting configuration goes from 14% to 94%.
The average solution though takes much longer now 21 moves on success, 64 on failure.
Another variant of hill-climbing is stochastic hill climbing: Choose an uphill move at random from the uphill moves with probability proportional to the steepness of the move. This typically converges to a solution more slowly than deterministic hill-climbing, but tends to find better solutions.
One can make hill-climbing complete by allowing restarts. In random-hill climbing with restarts one runs the hill climbing algorithm from a randomly chosen initial, see if one gets a solution, and if not, pick a new random initial state and try again.
If each hill-climbing search has a probability `p` of find a solution than the expected number of restarts is `1/p`.
For the 8-queen problem with no sideways moves `p approx 0.14` and roughly 7 restarts on average are needed to find the goal.
This technique can be used to solve instances of the `n`-queens problems into the millions in under a minute.

Simulated Annealing

Hill-climbing doesn't allow for downwards moves.
We next consider an algorithm that does, simulated annealing.
In metallurgy, annealing is the process used to harden glass or metal by heating a substance to a high temperature and then gradually cooling it. This allows the material to reach a lower crystalline energy state.
In this context we are looking at a minimization problem rather than a maximization problem: We imagine our cost is the energy of the system and we are trying to get the system into as low an energy state as possible.
For example, one could imagine the problem of getting a ping pong ball into the deepest crevasse in a bumby surface.
The way annealing tries to do this is to shake the surface a whole bunch initially, hoping the ball will go generally downloads and not "bump up" too often. Then one gradually slows down the shaking. Keeping some shaking allows us to climb out of local minima, but we gradually reduce the shaking as we get closer and closer to what we think is a minima.
Simulated annealing is a computer simulation of this idea. It has been used in VLSI layout since the 1980s.

Simulated Annealing Algorithm

function SIMULATED-ANNEALING(problem, schedule) returns a solution state
    inputs: problem, a problem
            schedule, a mapping from time "temperature"
    current := MAKE-NODE(problem, INITIAL_STATE)
    for t = 1 to infty do
        T := schedule(t) //typically as t gets larger schedule(t) is smaller
        if(T == 0) return current //when Temperature is 0 return state
        next := a randomly selected successor of current
        DeltaE := next.value - current.value //estimated change in energy
        if(DeltaE > 0) current := next 
        else current := next with probability e^(DeltaE/T)

Local Beam Search

Rather than keep track of only one state at a time, one could imagine keeping track of `k` states.
This is what local beam search does. It begins with `k` randomly generated states.
At each step the successors of all these states are generated.
If any is the goal, it stops.
Otherwise, the `k` best successors are chosen for the next round.
Notice this is different from doing `k` random restarts as the `k` best states in a given round might not come from the successors of all `k` different current states, but instead may come from only some of them.
As with hill-climbing, beam search can be made stochastic.
The `k` states can be viewed as a pool of candidates and their successors as offspring.

Local Search via Genetic Algorithms

A genetic algorithm is a stochastic beam search in which successor states are generated by combining two parent states rather than by modifying a single state.
Again, an analogy can be made with natural selection, but, whereas, in the stochastic local beam search case offspring are generated by asexual reproduction, in the genetic algorithm case they are generated by sexual reproduction.
GAs begin with a set of `k` randomly generated states called the population.
Each state (individual) is represented as a string over a finite alphabet. For example in the 8-queens, the strings might consist of the position in each column of a queen.
The initial states are ranked according to some fitness function which should be higher for better states. For 8-queens, one might use the number of non-attacking queens.
One selects individuals at random according to fitness to "breed".
A cross-over point is chosen randomly from the positions in the string to generate new strings. A mutation might then be applied.
Finally, this whole process is repeated on the new strings or until a goal or closeness to a goal is achieved.

Beyond Classical Search

Outline

Introduction

Local Search

Quiz

Hill Climbing

Hill Climbing

Variants of Hill-Climbing

Simulated Annealing

Simulated Annealing Algorithm

Local Beam Search

Local Search via Genetic Algorithms