Outline

• Planning Search Strategies
• Heuristics for Planning
• In-Class Exercise
• Planning Graphs

Introduction

• On Monday, we introduced the planning logic PDDL.
• We showed how problems that could in theory be solved by problem solving agents could be described in it.
• We then considered the complexity of planning.
• We begin today by considering some of the issues that might be involved in forward and backward search for solutions.
• Then we discuss some heuristics that might be helpful in performing this search for solutions.

Forward State-Space Search

• In forward search, we start with our initial state and we apply each possible action rule and search forward hoping to find a goal state.
• One problem with forward search is it is prone to considering irrelevant actions.
• For example, suppose the goal is to buy a textbook and we have an action schema `Buy(isbn)` with effect `Own(isbn)`. ISBNs are ten digits, so this action schema has 10 billion ground actions. An uninformed forward-search would have to enumerate through each.
• Planning problems often have large state spaces. Consider an air cargo problem with 10 airports, where each airport has 5 planes and 20 pieces of cargo. The goal is to move all the cargo at airport A to airport B. This could be done by loading 20 pieces of cargo at airport A into one plane, fly the plane to B, and unload the cargo.
• Finding the solution, however, might be hard because each of the 50 planes can fly to 9 other airports, and each of the 200 packages can be loaded or unloaded into any planes at its airport. The book estimates this problem might have as many as `2000^41` possible states.
• For forward planning to work we need to have good domain-independent heuristics (which we will see we do).

Backward (Regression) Relevant-States Search

• In backward (regression) search, we start from the goal, and try to search backward until we reach a state implied by the initial state.
• This kind of search is also called relevant-states search because we only consider actions that are relevant to the goal.
• At each step, there will be a set of relevant states to consider, not just a single state.
• For example, suppose the goal was `neg P\o\o\r ^^ Famous`. This describes those states in which Poor is false, Famous is true, and we don't care about the value of the other states.
• If there are `n` ground fluents in a domain, then there are `2^n` ground states, but `3^n` descriptions of sets of goals: positive, negative, and not mentioned.
• If a domain is expressed in PDDL, it is not too hard to do regression on it. Given a state `g`, action `a` (everything ground), the regression from `g` to `a` gives us a state `g'` defined by
  `g' = (g - ADD(a))cup Precond(a)`
• In addition to being able to handle ground states, we need to be able to handle partially un-instantiated actions and states.
• For example, suppose the goal is to deliver a specific piece of cargo to SFO: `At(C_2, SFO)`. This suggests the action `Unload(C_2, p', SFO)`:
  `Action(Unload(C_2, p', SFO)`,
  `\quad Precond: In(C_2, p') ^^ At(p',SFO) ^^ Cargo(C_2) ^^ Pla\n\e(p') ^^ Airp\o\r\t(SFO)`,
  `\quad Effect: At(C_2, SFO) ^^ neg In(C_2, p')).`

Heuristics for Planning

• One way to try to make planning problems easier is to try to come up with plans for relaxed problems.
• We can think of a planning search problem as a graph where the nodes are states and the edges are actions.
• We want to find a path from an initial state to a goal state.
• We can make this problem easier by adding more edges to the graph, or by grouping multiple nodes together and viewing them as one node (reducing the number of states).
• We next look at these approaches in more detail.

Ignore Preconditions Heuristic

• One way to try to implement the first strategy is to use the ignore preconditions heuristic. This heuristic drops all the preconditions from actions.
• It can still be the case that the number of steps to solve the resulting problem is more than the number of unsatisfied goals because some actions might undo the the effects of others.
• So to make this heuristic even more guaranteed, we also delete all effects from our actions except those that are literals in the goal.
• We then count the minimum number of actions required such that the union of those actions' effect satisfies the goal.
• An exact count is an NP-hard problem, but in p-time using a greedy algorithm one can approximate the true minimum to within a `log n` factor where `n` is the number of literals in the goal. (This isn't an admissible heuristic because we might overestimate the cost of a solution).
• Still, by using cost to a given state + this heuristic, we can search for solutions in an `A^star`-star like fashion.

In-Class Exercise

Ignore Delete Lists

• Another heuristic we can use in searching for solutions to planning problems is to remove all the delete lists from the effects of all actions.
• This means as one searches toward the goal, states monotonically increase in the number of true literals until eventually (hopefully) we find the goal.
• It turns out this way of relaxing planning problems still leaves one with an NP-hard search problem.
• Nevertheless, an approximate solution to this problem can be found in polynomial time using hill climbing.

State Abstraction

• The relaxed problems made by these two ignore heuristics leave us with a simplified but still expensive planning problem just to compute the value of the heuristic.
• This is because these methods do not reduce the total number of states of the problem, which for things like the cargo problem can easily have `10^(155)` states for a 10 airport, 50 plane, 200 pieces of cargo problem.
• So we want to consider relaxations that decrease the number of state by performing state abstraction -- a many-to-one mapping from states in the ground representation of the problem to the abstract representation.
• For instance, one thing we could do is assume that all of the packages are at one of 5 airports and all packages at a given airport have the same destination -- say a hub.

Problem Decomposition

• An important idea in defining heuristics is decomposition; dividing a problem into parts, solving each part independently, and then combining the parts.
• The subgoal independence assumption is the assumption that the cost of solving a conjunction of subgoals is approximated by the sum of the costs of solving each subgoal independently.

Planning Graphs 1

• A planning graph is a special data structure which can be used to give better heuristic estimates for the cost of a plan.
• It is a polynomial size approximation to the tree one would get by starting at the initial state expanding all possible next actions, then next actions of those actions, etc.
• A planning graph is organized into levels: first a level `S_0` for the initial state; then level `A_0` consisting of nodes for each ground action that might be applicable in `S_0`. Then alternating levels `S_i` followed by `A_i`; until we reach a termination condition.
• Roughly, `S_i` contains all the literals that could hold at time `i`. If it is possible that either `P` or `neg P` could hold, then both will be represented in` S_i`.
• Roughly, `A_i` contains all the actions that could have their preconditions satisfied at time `i`.
• Planning graphs only work for propositional planning problems.

Planning Graphs 2

• The above picture is an example planning graph. Each action at level `A_i` is connected to its preconditions at `S_i` and its effects at `S_(i+1)`.
• We also want to say that a literal can persist if no action negates it. To do this we have persistence actions (no-ops). For every literal `C`, we add to the problem a persistence action with precondition `C` and effect `C`. These are drawn with small square boxes in the picture above.
• We also indicate things that cannot simultaneously happen using mutex (mutual exclusion) links.
• We continue building the graph from `S_0` to `A_0` to `S_1`, ... until we get to two consecutive levels which are identical. We then say the graph has leveled off. In the above, this happens at `S_2`.
• For a planning graph with `l` literals and `a` actions, each `S_i` has no more than `l` nodes and `l^2` mutex links, and each `A_i` has no more than `a+l` nodes (including no-ops), `(a +l)^2` mutex links, and `2(al +l)` precondition and effect links. So a graph of `n` levels has size `O(n(a+l)^2)`, a polynomial. The construction time is the same.

Using Planning Graphs for Heuristic Estimation

• If any goal fails to appear in the final level of the planning graph of the problem, we know the problem is unsolvable.
• We can estimate the cost of achieving any goal `g_i` from state `s` as the level at which `g_i` first appears in the planning graph constructed with starting state `s`. This is called the level cost of `g_i`.
• For example `Have(Cake)` has level cost `0` and `Eaten(Cake)` has level cost `1` on the previous slide.
• To estimate the conjunction of goals, there are three common approaches: max-level, the largest level-cost amongst the goals; level-sum, the sum of the level costs, and set-level, the first level in graph in which all the literals in the conjunctive goal appear in the planning graph without any pair of them being mutually exclusive.

GraphPlan

The GraphPlan algorithm is as follows:

function GraphPlan(problem) returns solution or failure:
    graph := Initial-Planning-Graph(problem) 
    goals := Conjuncts(problem, Goal)
    nogoods := empty hash table
    for t= 0 to infty do:
        if goals all present and non-mutex in S_t of graph then
            solution := Extract-Solution(graph, goals, NumLevels(graph), nogoods)
            if solution != failure then return solution
        if graph  and nogoods have not changed since t-1 then return failure
        graph := Expand-Graph(graph, problem)

• Initial-Planning-Graph just computes and returns the `S_0` layer of the planning graph.
• Conjuncts takes the Goal for the problems and makes a list of fluents
• Expand-Graph computes `A_(i-1)` and `S_i` from `S_(i-1)` including mutex relations.
• We will describe algorithms for Extract-Solution on the next slide.

Two Approaches to Extract Solution

There are two common approaches to computing extract solution:

1. If all the goals are present and non-mutex, one can take the current graph and formulate it as a CSP. The variables for this CSP are the actions at each level, the values for each variable are either in or out of the plan, and the constraints are the mutexes and the need to satisfy each goal and precondition. One can then use CSP algorithms to solve this problem.
2. Another way to code Extract-Solution is as a backward search problem, where each state in the search contains a pointer to a level in the planning graph and a set of unsatisfied goals. This search problem can be defined as follows:
   • The initial state is the last level of the planning graph `S_n`, along with the set of goals from the planning problem.
   • The actions available at level `S_i` are to select any conflict-free subset of the actions in `A_(i-1)` whose effects cover the goals in the state. The resulting state has level `S_(i-1)` and has as its set of goals the preconditions for the selected set of actions. Here "conflict free" means a set of actions such that no two of them are mutex and no two of their preconditions are mutex.
   • The goal is to reach a state at level `S_0` such that all the goals are satisfied.
   • The cost of each action is 1.

GraphPlan - Spare Tire Example

• Notice the goal `At(Spare, Ax\l\e)` is not present in in `S_0` so we would call Expand-Graph adding into `A_0` the three actions whose preconditions exist at level `S_0`. Using these actions we get `S_1`.
• Again, `At(Spare, Ax\l\e)` is still not present in `S_1`, so Expand-Graph is called again, adding `A_1` and `S_2`.
• At this point we might be able to call Extract-Solution.
• Before we continue, let's classify the kinds of mutexes which we have in our graph:
  • Inconsistent effects: Remove(Spare, Trunk) is mutex with LeaveOvernight because one has the effect `At(Spare, Ground)` and the other its negation.
  • Interference: `Remove(Flat, Ax\l\e)` is mutex with LeaveOvernight because one has the precondition `At(Flat, Ax\l\e)` and the other has its negation as an effect. i.e., if we imagine trying to write these operation in serial order, if we did `LeaveOvernight` first, we couldn't do `Remove(Flat, Ax\l\e)`, on the other hand, if we did `Remove(Flat, Ax\l\e)` first, then `LeaveOvernight` has no effect.
  • Competing needs: `PutOn(Spare, Ax\l\e)` is mutex with `Remove(Flat, Ax\l\e)` because one has `At(Flat, Ax\l\e)` as a precondition and the other has its negation.
  • Inconsistent support: `At(Spare, Ax\l\e)` is mutex with `At(Flat, Ax\l\e)` in `S_2` because the only way of achieving `At(Spare, Ax\l\e)` is by `PutOn(Spare, Ax\l\e)` and that is mutex with the persistence action that is only way of achieving `At(Flat, Ax\l\e)`. Thus, the mutex relations relations detect the immediate conflict that arises from trying to put two objects in the same space at the same time