Outline

• Heuristics for Planning
• Quiz
• Planning Graphs

Introduction

• Last week, we introduced the planning logic PDDL.
• We showed how problems that could in theory be solved by problem solving agents could be described in it.
• We then considered the complexity of planning as well as some of the issues that might be involved in forward and backward search for solutions.
• We start today by considering some heuristics that might be helpful in performing this search for solutions

Heuristics for Planning

• One way to try to make planning problems easier is to try to come up with plans for relaxed problems.
• We can think of a planning search problem as a graph where the nodes are states and the edges are actions.
• We want to find a path from an initial state to a goal state.
• We can make this problem easier by adding more edges to the graph, or by grouping multiple nodes together and viewing them as one node (reducing the number of states).
• We next look at these approaches in more detail.

Ignore Preconditions Heuristic

• One way to try to implement the first strategy is to use the ignore preconditions heuristic. This heuristic drops all the preconditions from actions.
• It can still be the case that the number of steps to solve the resulting problem is more than the number of unsatisfied goals because some actions might undo the the effects of others.
• So to make this heuristic even more guaranteed, we also delete all effects from our actions except those that are literals in the goal.
• We then count the minimum number of actions required such that the union of those actions' effect satisfies the goal.
• An exact count is an NP-hard problem, but in p-time using a greedy algorithm one can approximate the true minimum to within a `log n` factor where `n` is the number of literals in the goal. (This isn't an admissible heuristic because we might overestimate the cost of a solution).
• Still, by using cost to a given state + this heuristic, we can search for solutions in an `A^star`-star like fashion.

Ignore Delete Lists

• Another heuristic we can use in searching for solutions to planning problems is to remove all the delete lists from the effects of all actions.
• This means as one searches toward the states monotonically increase in the number of true literals until eventually (hopefully) we find the goal.
• It turns out this way of relaxing planning problems still leaves one with an NP-hard search problem.
• Nevertheless, an approximate solution to this problem can be found in polynomial time using hill climbing.

State Abstraction

• The relaxed problems made by these two ignore heuristics leave us with a simplified but still expensive planning problem just to compute the value of the heuristic.
• This is because these methods do not reduce the total number of states of the problem, which for things like the cargo problem can easily have `10^(155)` states for a 10 airport, 50 plane, 200 pieces of cargo problem.
• So we want to consider relaxations that decrease the number of state by performing state abstraction -- a many-to-one mapping from states in the ground representation of the problem to the abstract representation.
• For instance, one thing we could do is assume that all of the packages are at one of 5 airports and all packages at a given airport have the same destination -- say a hub.

Problem Decomposition

• An important idea in defining heuristics is decomposition; dividing a problem into parts, solving each part independently, and then combining the parts.
• The subgoal independence assumption is the assumption that the cost of solving a conjunction of subgoals is approximated by the sum of the costs of solving each subgoal independently.

Quiz

Planning Graphs 1

• A planning graph is a special data structure which can be used to give better heuristic estimates for the cost of a plan.
• It is a polynomial size approximation to the tree one would get by starting at the initial state expanding all possible next actions, then next actions of those actions, etc.
• A planning graph is organized into levels: first a level `S_0` for the initial state; then level `A_0` consisting of nodes for each ground action that might be applicable in `S_0`. Then alternating levels `S_i` followed by `A_i`; until we reach a termination condition.
• Roughly, `S_i` contains all the literals that could hold at time `i`. If it is possible that either `P` or `neg P` could hold, then both will be represented in` S_i`.
• Roughly, `A_i` contains all the actions that could have their preconditions satisfied at time `i`.
• Planning graphs only work for propositional planning problems.

Planning Graphs 2

• The above picture is an example planning graph. Each action at level `A_i` is connected to its preconditions at `S_i` and its effects at `S_(i+1)`.
• We also want to say that a literal can persist if no action negates it. To do this we have persistence actions (no-ops). For every literal `C`, we add to the problem a persistence action with precondition `C` and effect `C`. These are drawn with small square boxes in the picture above.
• We also indicate things that cannot simultaneously happen using mutex (mutual exclusion) links.
• We continue building the graph from `S_0` to `A_0` to `S_1`, ... until we get to two consecutive levels which are identical. We then say the graph has leveled off. In the above, this happens at `S_2`.
• For a planning graph with `l` literals and `a` actions, each `S_i` has no more than `l` nodes and `l^2` mutex links, and each `A_i` has no more than `a+l` nodes (including no-ops), `(a +l)^2` mutex links, and `2(al +l)` precondition and effect links. So a graph of `n` levels has size `O(n(a+l)^2)`, a polynomial. The construction time is the same.

Using Planning Graphs for Heuristic Estimation

• If any goal fails to appear in the final level of the planning graph of the problem, we know the problem is unsolvable.
• We can estimate the cost of achieving any goal `g_i` from state `s` as the level at which `g_i` first appears in the planning graph constructed with starting state `s`. This is called the level cost of `g_i`.
• For example `Have(Cake)` has level cost `0` and `Eaten(Cake)` has level cost `1` on the previous slide.
• To estimate the conjunction of goals, there are three common approaches: max-level, the largest level-cost amongst the goals; level-sum, the sum of the level costs, and set-level, the first level in graph in which all the literals in the conjunctive goal appear in the planning graph without any pair of them being mutually exclusive.

GraphPlan

The GraphPlan algorithm is as follows:

function GraphPlan(problem) returns solution or failure:
    graph := Initial-Planning-Graph(problem) 
    goals := Conjuncts(problem, Goal)
    nogoods := empty hash table
    for t= 0 to infty do:
        if goals all present and non-mutex in S_t of graph then
            solution := Extract-Solution(graph, goals, NumLevels(graph), nogoods)
            if solution != failure then return solution
        if graph  and nogoods have not changed since t-1 then return failure
        graph := Expand-Graph(graph, problem)

• Initial-Planning-Graph just computes and returns the `S_0` layer of the planning graph.
• Conjuncts takes the Goal for the problems and makes a list of fluents
• Expand-Graph computes `A_(i-1)` and `S_i` from `S_(i-1)` including mutex relations.
• We will describe algorithms for Extract-Solution on the next slide.

Two Approaches to Extract Solution

There are two common approaches to computing extract solution:

1. If all the goals are present and non-mutex, one can take the current graph and formulate it as a CSP. The variables for this CSP are the actions at each level, the values for each variable are either in or out of the plan, and the constraints are the mutexes and the need to satisfy each goal and precondition. One can then use CSP algorithms to solve this problem.
2. Another way to code Extract-Solution is as a backward search problem, where each state in the search contains a pointer to a level in the planning graph and a set of unsatisfied goals. This search problem can be defined as follows:
   • The initial state is the last level of the planning graph `S_n`, along with the set of goals from the planning problem.
   • The actions available at level `S_i` are to select any conflict-free subset of the actions in `A_(i-1)` whose effects cover the goals in the state. The resulting state has level `S_(i-1)` and has as its set of goals the preconditions for the selected set of actions. Here "conflict free" means a set of actions such that no two of them are mutex and no two of their preconditions are mutex.
   • The goal is to reach a state at level `S_0` such that all the goals are satisfied.
   • The cost of each action is 1.

GraphPlan - Spare Tire Example

• Notice the goal `At(Spare, Ax\l\e)` is not present in in `S_0` so we would call Expand-Graph adding into `A_0` the three actions whose preconditions exist at level `S_0`. Using these actions we get `S_1`.
• Again, `At(Spare, Ax\l\e)` is still not present in `S_1`, so Expand-Graph is called again, adding `A_1` and `S_2`.
• At this point we might be able to call Extract-Solution.
• Before we continue, let's classify the kinds of mutexes which we have in our graph:
  • Inconsistent effects: Remove(Spare, Trunk) is mutex with LeaveOvernight because one has the effect `At(Spare, Ground)` and the other its negation.
  • Interference: `Remove(Flat, Ax\l\e)` is mutex with LeaveOvernight because one has the precondition `At(Flat, Ax\l\e)` and the other has its negation as an effect. i.e., if we imagine trying to write these operation in serial order, if we did `LeaveOvernight` first, we couldn't do `Remove(Flat, Ax\l\e)`, on the other hand, if we did `Remove(Flat, Ax\l\e)` first, then `LeaveOvernight` has no effect.
  • Competing needs: `PutOn(Spare, Ax\l\e)` is mutex with `Remove(Flat, Ax\l\e)` because one has `At(Flat, Ax\l\e)` as a precondition and the other has its negation.
  • Inconsistent support: `At(Spare, Ax\l\e)` is mutex with `At(Flat, Ax\l\e)` in `S_2` because the only way of achieving `At(Spare, Ax\l\e)` is by `PutOn(Spare, Ax\l\e)` and that is mutex with the persistence action that is only way of achieving `At(Flat, Ax\l\e)`. Thus, the mutex relations relations detect the immediate conflict that arises from trying to put two objects in the same space at the same time

Backward Search and Spare Tire Problem

• If we used backward search to try to solve the spare tire problem, then we start at `S_2` with the goal `At(Spare, Ax\l\e)`.
• The only choice for achieving this goal set is `PutOn(Spare, Ax\l\e)`. That brings us to a search state at `S_1` with goals `At(Spare, Ground)` and `neg At(Flat, Ax\l\e)`.
• The former can be achieved only by `Remove(Spare, Trunk)`, and the latter by either `Remove(Flat, Ax\l\e)` or `LeaveOvernight`.
• `LeaveOvernight` is mutex with `Remove(Spare, Trunk)`, so we choose `Remove(Spare, Trunk)` and `Remove(Flat, Ax\l\e)`
• This brings us to a search of state `S_0` with the goals `At(Spare, Trunk)` and `At(Flat, Ax\l\e)`. As both of these are present in `S_0` we have found a solution: `Remove(Spare, Trunk)`, `Remove(Flat, Ax\l\e)`, and `PutOn(Spare, Ax\l\e)`.
• In the case where Extract-Solution fails to find a solution for a set of goals at a given level, we record the pair (level, goals) pairs as a no-good. If Extract-Solution is called again with the same level and goals, we just immediately return failure.

GraphPlan Heuristics

• We know planning is PSPACE-complete and making the planning graph is in polynomial time.
• So the backward search can still be hard, so heuristics are still needed for choosing among actions.
• People often use a greedy algorithm based on the level cost of literals.
• For any set of goals, we proceed in the following order:
  1. Pick first the literal with the highest level cost.
  2. To achieve that literal, prefer actions whose sum of the level costs of its preconditions is smallest.
• Although we won't show it in class, one can show that the GraphPlan Algorithm terminates.