Outline

• Agents in Uncertain Domains
• Quiz
• Basic Probability

Introduction

• Up till now our agents all operated under the assumption that everything is true, false, or unknown
• Would like agents that can operate given some probability that something is true.
• For example, we would like to make plans and say a plan will succeed provided certain conditions are met at time of execution. Then estimate probability those conditions will be met to do reasoning.
• As a particular instance of this: We might reason that "the having fun in park plan" will succeed if it is not raining. We then might estimate this probability to try to figure out if we want to execute plan at a given time in future.
• The right thing to do in a given situation -- the rational decision --depends both on the relative importance of various goals and the likelihood that, and degree to which, these goals can be achieve.
• We will now spend some time developing the probability needed to talk about rational decision making and later about statistical learning.

How to make a logical model of uncertainty.

• Consider:
  `forall p, Sym\p\t\om(p, \T\o\othache) => Disease(p, Cavity)`
  This is not always true.
• For example, gum disease might have been the cause of the symptom.
• To make a valid statement we might try to enumerate the possibilities:
  `forall p Sym\p\t\om(p, \T\o\othache) => Disease(p, Cavity) vv Disease(p, GumDisease) vv ...`
• This kind of enumerations is quite hard: We are often too lazy or don't know all the possibilities.
• The point of above is modeling uncertainty using just first-order logic or propositional logic is both awkward and potentially hard to do -- We want to use a formalism more geared to modeling these things.
• We will use probability theory to do this.
• We will keep a notion of propositional and 1st order sentence, but now add a notion of degree of believe in that sentence.

Uncertainty and Rational Decisions

• Before we go into probability let's briefly look at how we would like to modify our agents to work with them.
• First, to make a decision as to what to do next the agent needs to have preferences between it possible goals. This is called a utility theory. For example, I prefer a cup of coffee twice as much as a cup of tea. This is relative to the agent.
• A Decision Theory takes this utility and adds to it some calculation based on the probability of achieving each of the possible goals.
• The agent should choose an action which yields the highest expected payoff among the available choices. This is called the principle of maximum expected utility (MEU).
• Below is some pseudo-code for an agent acting in this way:

  function DT-Agent(percept) returns an action
      persistent belief-state, probabilistic beliefs about 
                               the current state of the world
                 action, the agent's action
  
      update belief-state based on action and percept
      calculate outcome probabilities for actions
         given action descriptions and current belief-state
      select amongst these actions  the one with highest expected utility
          given probabilities of outcomes and utility information
      return action
  

Quiz

What probabilities are about

• In probability, we will have a notion of possible worlds, which we will call the sample space.
• These are the sets of things that could be true. Elements/possible worlds of this space are mutually exclusive and they exhaust the space.
• For example, for a pair of dice, the sample space might be all possible ordered pairs of the numbers 1-6: (1,1), (1,2), ... (2,1) ..., (6,6).
• Often `Omega` is used to refer to a sample space and `omega` to an element of this space.
• A probability model/probability distribution associates a numerical number `P(omega)` between `0` and `1` with elements of the sample .
• We require that `sum_(omega in Omega) P(omega) = 1`.
• Probabilistic assertions and queries are not usually about particular worlds, but about sets of them. For example, what's the odd that two dice add up to 11.
• In probability theory, these sets of possible worlds are called events and these events are almost always described by propositions in a formal language.
• For any proposition `\phi`, `P(phi) = sum_(omega in Omega, phi mbox( in ) omega mbox( holds)) P(omega)`. For example P(Total = 11) = P((5,6)) + P((6,5)) = 1/36 + 1/36 = 1/18, assuming the die are fair.

Conditional versus Unconditional Probabilities

• Probabilities such as P(Total = 11) and P(doubles) are called unconditional or prior probabilities; they refer to degrees of belief in propositions in the absence of any other information.
• Most times we have some information, usually called evidence.
• For example, we might ask what's the odd of getting 11 given that we know one dice has already landed as a 5?
• This is called a conditional probability. We would write this probability as `P(\T\o\tal = 11 | Die_1 = 5)`
• Mathematically, conditional probabilities can be computed using the formula:
  `P(a|b) = frac(P(a^^b))(P(b))`
  which holds whenever `P(b) > 0`.
• For example,
  `P(doubl\es | Die_1 =5) = frac(P(doubl\es ^^ Die_1 =5))(P(Die_1=5)) = frac(1/36)(1/6) = 1/6`

A Language for Probability Assertions

• In propositional logic or first-order logic we build up statements with variables or atomic formula. Let's see how to do the analogous thing for probabilities.
• Recall when we talked about CSP or first order logic we had variables which we write with an initial capital letter. For example, Weather. These variables take on some values in some domain. For example: {sunny, rainy, cloudy, snowy}
• We will write atomic expressions like Weather=sunny to mean: We have chosen the sunny value out of Weather's domain/sample space.
• We can write a probability distribution on Weather's domain then by specifying the value probability of the different values weather could map to. For example, P(Weather = sunny) = .2, P(Weather = rainy) = .2, P(Weather = cloudy) =.38, P(Weather = snowy) = 0.2.
• Assuming an ordering on the elements of our domain, we can simplify the writing of the distribution as a vector `vec P` (Weather) = (0.4, 0.2, 0.38, 0.02).
• We can characterize the kind of variable referred to above by its type of sample space: For example, boolean if its domain is {true, false}, discrete if it consists of separated points like a finite set or the integers, or continuous such as points in space.
• Given two domains it is often interesting to take their Cartesian product to make a new domain. We can often create a new probability distribution on the resulting set called a joint probability distribution. For example, P(Weather, Cavity).
• This start to allow us to talk about probability for sentences because we can write things like:
  `P(Weather = su\n\n\y ^^ Cavity = true)`
• So joint distributions give us a notion of probabilities for `^^`.

Random Variables

• A random variable `X` is a function `X:Omega -> RR`. For example, we might bet $3 on whether a coin flip is heads or tails. So `X(HEADS) =3` (I win), `X(TAILS) = -3` (I lose).
• A random variable is not the same thing as a variable in the sense of the last slide: The variables on the last slide were used to pick out values from the sample space/domain.
• If we want to emphasize that we are talking about the previous kind of variable, we will say domain variable.
• Given a probability distribution `P` (for example, P(HEADS) = 0.2, P(TAILS) = 0.8), the expected value of random variable `X`, `E(X)`, with respect to this distribution is defined as:
  `E[X] = sum_(omega in Omega) X(omega)P(omega)`.
• So for the distribution `P` and random variable `X` above, `E[X] = 3 times 0.2 + (-3) times 0.8 = -1.8`.
• One can verify from the definition that expectations are linear. So if `Y` and `Z` are random variables `E[X+Y] = E[Y] + E[Z]`.
• We can classify random variables in a similar fashion to how we classified domain variables by looking at sets of points that they map to:
  • A boolean random variable (indicator variable) is a random variable that maps each point in its sample space to one of two values. For example, one value mapped to might be 0 (which we could treat as false) the other 1 (which we could treat as one). If the random variables sample space has size 2, then we will have Boolean random variable. There are other cases, though: Consider the function from `F: RR ->RR` which is 0 if `x < 10` and is 1 otherwise.
  • A discrete random variable is a random variable such that any point mapped to in the range is contained in some interval of the real line that contains no other image points of the random variable. Typical examples of this would occur is the sample space is finite or if the image of the random variable is a subset of the integers.
  • A continuous random variable which has domain usually consisting of an infinite number of states and where the function is continuous on this domain (pre-image of open set is open). For example, the real interval [18 to 26] might be the range. The sample space might be points in this room the function computes the temperature of that point in Celcius. A probability distribution over such a sample space would be called a probability density function. To compute the expected value of the temperature in this case would typically involve computing an integral.

Probabilities for Propositional Formulas

• Given an event `a`. The probability that `a` did not happen is the sum over all the situations where `a` did not occur. By our earlier definitions, this and the situation where `a` did occur must sum to 1. Hence. `P(neg a) + P(a) = 1` and so `P(neg a) = 1 - P(a)`.
• We can use the principle of inclusion exclusion to get a formula for the probability of the OR of two things as:
  `P(a vv b) = P(a) + P(b) - P(a ^^ b)`.
• Next day, we will talk about how to use probabilities to do inference.