Up till now our agents all operated under the assumption that everything is true, false, or unknown.
We would like agents that can operate given some probability that something is true.
For example, we would like to make plans and say a plan will succeed provided certain conditions are met at time of execution. Then estimate probability those conditions will be met to do reasoning.
As a particular instance of this: We might reason that "the having fun in park plan" will succeed if it is not raining. We
then might estimate this probability to try to figure out if we want to execute plan at a given time in future.
The right thing to do in a given situation -- the rational decision --depends both on the relative importance of various goals and the likelihood that, and degree to which, these goals can be achieved.
We will now spend some time developing the probability needed to talk about rational decision making and later about statistical learning.
How to make a logical model of uncertainty.
Consider:
`forall p, Sym\p\t\om(p, \T\o\othache) => Disease(p, Cavity)`
This is not always true.
For example, gum disease might have been the cause of the symptom.
To make a valid statement we might try to enumerate the possibilities:
`forall p Sym\p\t\om(p, \T\o\othache) => Disease(p, Cavity) vv Disease(p, GumDisease) vv ...`
This kind of enumerations is quite hard: We are often too lazy or don't know all the possibilities.
The point of above is modeling uncertainty using just first-order logic or propositional logic is both awkward and potentially hard to do -- We want to use a formalism more geared to modeling these things.
We will use probability theory to do this.
We will keep a notion of propositional and 1st order sentence, but now add a notion of degree of belief in that sentence.
Uncertainty and Rational Decisions
Before we go into probability let's briefly look at how we would like to modify our agents to work with them.
First, to make a decision as to what to do next the agent needs to have preferences between it possible goals.
This is called a utility theory. For example, I prefer a cup of coffee twice as much as a cup of tea. This is relative to the agent.
A Decision Theory takes this utility and adds to it some calculation based on the probability of achieving each of the possible goals.
The agent should choose an action which yields the highest expected payoff among the available choices. This is called the principle of maximum expected utility (MEU).
Below is some pseudo-code for an agent acting in this way:
function DT-Agent(percept) returns an action
persistent belief-state, probabilistic beliefs about
the current state of the world
action, the agent's action
update belief-state based on action and percept
calculate outcome probabilities for actions
given action descriptions and current belief-state
select amongst these actions the one with highest expected utility
given probabilities of outcomes and utility information
return action
What probabilities are about
In probability theory, we will have a notion of possible worlds, which we will call the sample space.
These are the sets of things that could be true. Elements/possible worlds of this space are mutually exclusive and they exhaust the space.
For example, for a pair of dice, the sample space might be all possible ordered pairs of the numbers 1-6: (1,1), (1,2), ... (2,1) ..., (6,6).
Often `Omega` is used to refer to a sample space and `omega` to an element of this space.
A probability model/probability distribution associates a numerical number `P(omega)` between `0` and `1` with elements of the sample .
We require that `sum_(omega in Omega) P(omega) = 1`.
Probabilistic assertions and queries are not usually about particular worlds, but about sets of them. For example,
what's the odd that two dice add up to 11.
In probability theory, these sets of possible worlds are called events and these events are almost always described by propositions in a formal language.
For any proposition `\phi`, `P(phi) = sum_(omega in Omega, phi mbox( in ) omega mbox( holds)) P(omega)`. For example P(Total = 11) = P((5,6)) + P((6,5)) = 1/36 + 1/36 = 1/18, assuming the die are fair.
Conditional versus Unconditional Probabilities
Probabilities such as P(Total = 11) and P(doubles) are called unconditional or prior probabilities; they refer to degrees of belief in propositions in the absence of any other information.
Most times we have some information, usually called evidence.
For example, we might ask what's the odd of getting 11 given that we know one dice has already landed as a 5?
This is called a conditional probability. We would write this probability as `P(\T\o\tal = 11 | Die_1 = 5)`
Mathematically, conditional probabilities can be computed using the formula:
`P(a|b) = frac(P(a^^b))(P(b))`
which holds whenever `P(b) > 0`.
For example,
`P(doubl\es | Die_1 =5) = frac(P(doubl\es ^^ Die_1 =5))(P(Die_1=5)) = frac(1/36)(1/6) = 1/6`
Quiz
Which of the following is true?
GraphPlan always expands the complete planning graph.
You can't express in the event calculus that a fluent ceases to hold at time `t`.
The GraphPlan algorithm does a backward search when computing Extract-Solution.
A Language for Probability Assertions
In propositional logic or first-order logic we build up statements with variables or atomic formula. Let's see how to do the analogous thing for probabilities.
Recall when we talked about CSP or first order logic we had variables which we write with an initial capital letter. For example, Weather. These variables take on some values in some domain. For example: {sunny, rainy, cloudy, snowy}
We will write atomic expressions like Weather=sunny to mean: We have chosen the sunny value out of Weather's domain/sample space.
We can write a probability distribution on Weather's domain then by specifying the value probability of the different values weather could map to. For example,
P(Weather = sunny) = .4, P(Weather = rainy) = .2, P(Weather = cloudy) =.38, P(Weather = snowy) = 0.02.
Assuming an ordering on the elements of our domain, we can simplify the writing of the distribution as a vector `vec P` (Weather) = (0.4, 0.2, 0.38, 0.02).
We can characterize the kind of variable referred to above by its type of sample space: For example, boolean if its domain is {true, false}, discrete if it consists of separated points like a finite set or the integers, or continuous such as points in space.
Given two domains/sample spaces it is often interesting to take their Cartesian product to make a new domain/sample space. We can often create a new probability distribution on the resulting set called a joint probability distribution. For example, `vec(P)(Weather, Cavity)`.
This starts to allow us to talk about probability for sentences because we can write things like:
`P(Weather = su\n\n\y ^^ Cavity = true)`
So joint distributions give us a notion of probabilities for `^^`.
Random Variables
A random variable `X` is a function `X:Omega -> RR`. For example, we might bet $3 on whether a coin flip is heads or tails. So `X(HEADS) =3` (I win), `X(TAILS) = -3` (I lose).
A random variable is not the same thing as a variable in the sense of the last slide: The variables on the last slide were used to pick out values from the sample space/domain.
If we want to emphasize that we are talking about the previous kind of variable, we will say domain variable.
Given a probability distribution `P` (for example, P(HEADS) = 0.2, P(TAILS) = 0.8), the expected value of random variable `X`, `E(X)`, with respect to this distribution is defined as:
`E[X] = sum_(omega in Omega) X(omega)P(omega)`.
So for the distribution `P` and random variable `X` above, `E[X] = 3 times 0.2 + (-3) times 0.8 = -1.8`.
One can verify from the definition that expectations are linear. So if `Y` and `Z` are random variables `E[Y+Z] = E[Y] + E[Z]`.
We can classify random variables in a similar fashion to how we classified domain variables by looking at sets of points that they map to:
A boolean random variable (indicator variable) is a random variable that maps each point in its sample space to one of two values. For example, one value mapped to might be 0 (which we could treat as false) the other 1 (which we could treat as one). If the random variable's sample space has size 2, then we will have a Boolean random variable. There are other cases, though: Consider the function from `F: RR ->RR` which is 0 if `x < 10` and is 1 otherwise.
A discrete random variable is a random variable such that any point mapped to in the range is contained in some interval of the real line that contains no other image points of the random variable. Typical examples of this would occur is the sample space is finite or if the image of the random variable is a subset of the integers.
A continuous random variable which has domain usually consisting of an infinite number of states and where the function is continuous on this domain (pre-image of open set is open). For example, the real interval (18, 26) might be the range. The sample space might be points in this room. The random function might compute for a point in the sample space its temperature in Celcius. A probability distribution over such a sample space would be called a probability density function. To compute the expected value of the temperature in this case would typically involve computing an integral.
Probabilities for Propositional Formulas
Given an event `a`. The probability that `a` did not happen is the sum over all the situations where `a` did not occur. By our earlier definitions, this and the situation where `a` did occur must sum to 1. Hence.
`P(neg a) + P(a) = 1` and so `P(neg a) = 1 - P(a)`.
We can use the principle of inclusion exclusion to get a formula for the probability of the OR of two things as:
`P(a vv b) = P(a) + P(b) - P(a ^^ b)`.
We will talk about how to use probabilities to do inference.
Probabilistic Inference
Consider a domain consisting of the three Boolean variables: Toothache, Cavity, Catch (dentist probes catches in mouth):
Notice the probabilities in the full distribution sum to 1.
There are six possible worlds in which `cavity vv t\o\othache` holds so it probability is: 0.108 + 0.012 + 0.008 + 0.016 + 0.072+ 0.064 = 0.28
Often one wants to extract the distribution over some subset of variables or a single variable. For example, adding the entries in the first row gives the unconditional or marginal probability of cavity: 0.108+ 0.012 + 0.072 +0.008 = 0.2
This process is called marginalization or summing out.
The general rule for summing out any sets of variables `Y` and `Z` is:
`vec P(Y) = sum_(z in Z) vec P(Y,z).`
For example, we just calculated `\vec{P}(Cavity) = sum_(z in \{Catch, T\o\othache\})\vec{P}(Cavity,z)`.
I am writing `\vec{P}` when we are talking about the distribution rather than a probability.
If we are dealing with conditional probabilities instead of joint probabilities, we get the following rule called
conditioning: `vec{P}(Y) = sum_z vec{P}(Y|z)P(z)`.
Both of these rules are useful for derivations involving probability expressions.
Example of Computing Probabilities using Conditioning
Using our definition for conditional probabilities we can compute:
`P(cavity|t\o\othache) = frac(P (cavity ^^ t\o\othache) )(P(t\o\othache)) = frac(0.108 + 0.012)(0.108 + 0.012 + 0.016 +0.064) = 0.6` and
`P(neg cavity|t\o\othache) = frac(P (neg cavity ^^ t\o\othache) )(P(t\o\othache)) = frac(0.016 + 0.064)(0.108 + 0.012 + 0.016 +0.064) = 0.4`
As you expect, these sum to 1. Notice that `frac(1)(P(\t\o\othache))` appears in both of these. We can treat it as a normalization constant `\alpha` for the distribution `\vec P(Cavity |t\o\othache)` ensuring that it adds up to 1
So we can calculate `\vec(P)(Cavity |t\o\othache)` even if we don't know the value of `P(\t\o\othache)`, we just need to divide the last vector by the sum 0.12 + 0.08.
Using this we can extract a general inference procedure: Begin with the case in which the query involves a single variable, `X` (Cavity in our example). Let `\vec(E)` be the list of evidence variables (Toothache in our example). Let `vec(e)` be the list of observed values for them, and let `Y` be the remaining unobserved variables (Catch in our case). The query is `\vec(P)(X | \vec(e))` and can be evaluated as:
`vec(P)(X|vec(e)) = \alpha vec(P)(X, vec(e)) = alpha sum_vec(y) vec(P)(X,vec(e),\vec(y))`
Given the full joint distribution, this equation can answer probabilitic queries for discrete variables. It doesn't scale though: It require a table of `O(2^n)` size and so it take `O(2^n)` time to compute.
Independence
Suppose we added to our three variables a fourth variable Weather to get the full joint distribution
`vec(P)(T\o\othache, Catch, Cavity, Weather)`. This table now has `2 times 2 times 2 times 4 = 32 entries`, four "editions"
of the table of last day, one for each kind of weather.
What relationship do these editions have to each other and to the original three-variable table?
For example, how are `P(t\o\othache, catch, cavity, cloudy)` and `P(t\o\othache, catch, cavity)` related?
Using the product rule we know:
`P(t\o\othache, catch, cavity, cloudy) = P(cloudy | t\o\othache, catch, cavity) P(t\o\othache, catch, cavity)`
It is likely that the weather does not influence the dental variables. So it is safe to say:
`P(cloudy | t\o\othache, catch, cavity) = P(cloudy)`.
A similar equation exists for every entry in
`\vec(P)(T\o\othache, Catch, Cavity, Weather)` so we get:
`\vec(P)(T\o\othache, Catch, Cavity, Weather) = \vec(P)(T\o\othache, Catch, Cavity)\vec(P)(Weather)`
This property is called independence. In our case, the weather is independent of our dental problems
Independence means a full joint distribution can often be factored into separate disjoint distributions. Independence can
often help reduce the size of the domain representation and complexity of the inference problem.
Baye's Rule and Its Use
We already defined the product rules: `P(a ^^ b) = P(a | b)P(b)` and `P(a ^^ b) = P(b | a)P(a)`.
Equating the two right hand sides gives us:
`P(b|a) = frac(P(a|b)P(b))(P(a))`
This is called Baye's rule.
Baye's rule is used quite often in AI systems. The reason is that although `P(b|a)` might be hard to directly calculate the three terms on the right are often easier to determine
For example, we want to know the most likely cause of some effect. We could consider each cause and estimate:
`P(cause | effect) = frac(P(effect|cause)P(cause))(P(effect)).`
`P(effect | cause)` quantifies the relationship in the causal direction, whereas `P(cause | effect)` describes the diagnostic direction.
For example, a doctor often knows `P(symp\t\oms | disease)` but wants to calculate the disease that is causing the symptoms.
So if the doctor knows that 70% of people with meningitis have stiff necks and the odds of meningitis are 1/50000 and the odds
of a stiff neck are 1/100. Then the odds of meningitis given a stiff neck are (.7 * 1/50000)/0.01 = 0.0014.
Probabilistic Agent Example
Recall the Wumpus World game is played on a 4x4 grid starting at the lower left square.
We want to find the gold, avoid pits and the Wumpus (neither of which move), and leave the world with the gold.
Our knowledge about the world is given by sensors which can detect a breeze, a glitter, a stench, or a scream (if wumpus dies) in the adjacent square.
In one turn, we can move forward, turn left/right, grab gold, or shoot.
The above possible world might represent a situation that could cause a purely logical agent to get stuck the three available squares that have not been
visited all might contain a pit. So which should we choose?
We would like to use probabilities to choose the one least likely to contain a pit.
Modeling the Problem
As in the logical case we will use the following variables:
`P_(ij)` - represents square `(i,j)` has a pit.
`B_(ij)` - represents `(i,j)` is breezy.
To figure out what to do next we need to specify the full joint distribution
`vec P (P_(11), ..., P_(44), B_(11), B_(12), B_(21))`.
This can be calculated using
`vec P (P_(11), ..., P_(44), B_(11), B_(12), B_(21)) = vec P(B_(11), B_(12), B_(21)|P_(11), ..., P_(44)) vec P(P_(11), ..., P_(44))`
Each square contains a pit with probability 0.2 independently of all the other squares.
So we have:
`vec P(P_(11), ..., P_(44)) = prod_(i,j =1,1)^(4,4) vec P (P_(ij))`.
This vector is `\langle mbox(prob have 0 pits), mbox(prob have 1 pit), ..., mbox(prob have 16 pits) rangle`.
If the wumpus world has exactly `n` pits this gives:
`P(P_(11), ..., P_(44)) = (0.2)^n(0.8)^(16-n)`.
More Modeling the Problem
We abbreviate by `b` the observed breeze information `neg b_(11) ^^ b_(12) ^^ b_(21)`.
We abbreviate by `known` the squares we know pit info about `neg p_(11) ^^ neg p_(12) ^^ neg p_(21)`
We are interested in queries such as `vec P(P_(13) | known, b)` : How likely is that (1,3) contains a
pit given the observations so far?
Answering the Question
To answer the query we want to sum over entries from the full joint distribution.
Let `Unknown` be the set of `P_(ij)` variables for squares other than `Known` squares and the query square.
So as in our conditioning example earlier we have:
`vec P(P_(13) | known, b) = alpha sum_(unknown) vec P(P_(13), unknown, known, b)`
There are 12 unknown squares, so the above sum contains 4096 terms.
Not all squares are relevant to the probability on the left. For example `(4,4)` does not affect whether `(1,3)` has a pit.
Let Frontier denote the pit variables that are adjacent to visited squares. i.e., (2,2) and (3,1).
Let Other be the pit variables for the other unknown squares (10 in this case).
We can manipulate the sum above to make use of the fact that the observed breezes are conditionally independent of the other variables, given the known, frontier,
and query variables:
`vec P(P_(13) | known, b)`
`= alpha sum_(unknown) vec P(P_(13), known, b, unknown)`
`= alpha sum_(unknown) vec P(b|P_(13), known, unknown) vec P(P_(13), known, unknown)`
`= alpha sum_(\f\rontier)sum_(other) vec P(b|known,P_(13), \f\rontier, other) vec P(P_(13), known, \f\rontier, other)`
`= alpha sum_(\f\rontier)sum_(other) vec P(b|known,P_(13), \f\rontier) vec P(P_(13), known, \f\rontier, other)`
The last step uses the conditional independence.
The first term above does not depend on the Other variables, so we can move the summation inwards giving:
`vec P(P_(13) | known, b)`
`= alpha sum_(\f\rontier)vec P(b|known,P_(13), \f\rontier)sum_(other)vec P(P_(13), known, \f\rontier, other)`
By the independence of pit variables, we can factor the right hand sum and reorder things:
`vec P(P_(13) | known, b)`
`= alpha sum_(\f\rontier)vec P(b|known,P_(13), \f\rontier)sum_(other)vec P(P_(13))P(known)P(\f\rontier)P(other)`
`= alpha P(known) P(P_(13)) sum_(\f\rontier)vec P(b|known,P_(13), \f\rontier)P(\f\rontier)sum_(other)P(other)`
`= alpha' P(P_(13)) sum_(\f\rontier)vec P(b|known,P_(13), \f\rontier)P(f\rontier)`
In the last step we use `sum_(other) P(other) = 1`.
The above sum has just four terms.
`vec P(b|known,P_(13), \f\rontier)P(f\rontier)` is 1 when the frontier is consistent with the breeze observation and 0 otherwise.
So we sum over the logical models that are consistent with the known facts.
This gives us the models shown in the figure above.
Working it out, we have:
`vec P(P_(13) | known, b) = alpha' langle 0.2 (0.04 +.16 +.16), 0.8 (0.04 +0.16) rangle approx langle 0.31, 0.69 rangle`
