Let's return to our Naive Bayes Classifier now, with rain as the class either yes or
no, and grass being wet as the only feature that we're going to measure.
But suppose we have the conditional probability that grass is wet, given that
there is rain. So obviously, if there is rain, the chance
the grass is wet is very high. Otherwise, it's still there but not too
much. And if there's no rain, there is a very
low chance that the grass is wet. And of course, if there's no rain, the
chance that the grass is not wet is very high.
Notice again that this is a conditional probability table.
So, the values for a particular value of rain being yes, these have to add up to
one. The prior or the a priori probability that
it rains regardless of whether the grass is wet or not is a twenty percent of the
time it rains in this locality and 80% it doesn't rain.
Now, if we just have this one feature, our Bayes rule gives us the fact that the
joint probability of R and W can be factored in two ways as the probability of
R given W times the probability of W, Or this conditional probability times the
prior probability of R. Now, suppose we are given some evidence
that we actually observe that the grass is wet,
That is W equal to yes. Now, we can condition this joint
probability by restricting it to the case where W is yes.
So, we get probability of R given W equal to yes times the probability that W equal
to yes. And this side gets probability of W equal
to yes given R which is a likelihood times the prior which doesn't have anything to
do with W so we don't have to condition it.
This element probability that of R given W equal to yes, can now be written as the
right hand side here divided by the probability of the evidence.
The inverse of probability W equal to yes, we'll just write it as sigma.
It's, it's simply the probability of the evidence and we'll use the sigma wherever
we need to refer to one over the probability of the evidence that we are
observing. Pardon the use of sigma for this purpose
here, Only are we use sigma as a selection
operator but now we will use sigma only as the inverse of the evidence probability.
Study this carefully so that what we want is the a posterior probability of R given
the evidence, which is simply proportional to the likelihoods multiplied by the
prior. In sequel, we can write this as select sum
of P times P from these two tables where W equal to yes, and R equal to R because we
have to make sure that we're joining these tables on the common attribute R and
finally grouping by R. This is what we've stated before, how to
multiple two probability tables. So, the result is what we get by
restricting our case, cases to the, the rows where W is yes,
Multiplying the P values and adding them up.
But it doesn't add up since there are only two values,
Each of them having two rows having distinct values of R.
So we simply multiply 0.9 by 0.2 to get the first row R equal to yes,
And 0.2 by 0.8 to get the second row. This is the product of these two
potentials or probability tables. Now, we need to normalize so that the
total probability that, of R is one. So, the sum has to be one.
So essentially, we were, we're taking the probability of R being yes as 0.18 divided
by the sum of these two values which we get as 0.18 divided by the sum, and you
get 53% of 0.53. This is the chance that, or our belief that it's raining once we
see the grass as being wet. The reason it's so small, one might expect
it's a little bit higher, Is that there are cases where the grass
can be wet without there being any rain. In fact, there are cases of twenty percent
where the grass is wet when there's no rain.
In addition, it hardly ever rains. So, combining these two things together
essentially gives us only a 53% chance of saying that it's actually raining if the
grass is wet. Now, let's see what happens when we have
more than one feature as we normally do in a Baysian classifier.
We're going to have another feature called thunder, which will say whether or not we
are hearing thunder. And, let's for the moment, assume that we
don't really know whether we heard thunder at all.
But the probability of hearing thunder, given, that it's raining, is 0.8, not
hearing it is 0.2 and so on. So we have a conditional table even for
this variable thunder, but in this case let us assume that we haven't actually
observed thunder. We could be asking our neighbor over the
phone whether the grass is wet, and then trying to conclude whether or not it's
raining there. But, we didn't ask our neighbor whether they're hearing thunder.
So now, the probability of RT given W equal to yes, because that's all we know,
is by the same equation that we had earlier, probability of W equal to yes
given R, probability of T given R times the probability of R. This is our Baysian
formula. Again, we have the probability of the
evidence over here, but this time we don't have anything that says T equal to yes or
no. So, we need to sum out T in this product
if we want to understand or, or know only the probability of rain.
The way we sum that out is that we, again, do sequel.
So, we select R and the sum of the products of all three of these columns
from these three tables where W is yes, and all the common variables, which are
only R over here. The only common variable between these
tables is R, And then you group by R.
This effectively sums out T because we're only selecting R and summing up the values
for different values of T. Now you can verify that you could do this
by first joining T1 and T3, that is this table and this table that was the PW given
R and P of R, just like we did earlier, and then joining the result of that with
the T given R table. So, we're just going to take the result we
had earlier and join it with this new table.
But, notice that we get the same result because when we multiplied this element by
0.8, for R equal to yes, and again by 0.2 for R equal to no. And then, sum them up,
we'll get the same result because 0.8 plus 0.2 is one.
Similarly, for 0.1 and 0.9, so it doesn't change anything.
This is to be expected since there was no new evidence as compared to earlier.
Just by including something new in our diagram, we couldn't expect to change our
belief in R. Another important point is that if you
remember in one of the homeworks, we asked whether ignoring some of the features
changes our belief. Well, it does, change our belief as
compared to if we had actually observed the features.
But it's in some sense, equally correct, because suppose we didn't observe the
feature. So this is actually the same as a Bayesian
classifier with only two features, but only partially being observed, that only
one of them is being observed. So you could have millions of features and
only observe one or two, and you'll still get the same result just by putting them
in your classifier. The summing out process make sure that
this process always works and you don't get any wrong results.
Go back to that example where we asked whether partial evidence changes things
and, in fact, it doesn't. It is simply as if the feature didn't exist.
Of course, if we are observing the feature, it's better to include it.
But, by ignoring it, we're not saying that we had a wrong result, it's just that we
didn't have that feature to measure, that's all.
Now, let's see what happens if we actually do have evidence about T equal to yes or
no and that suppose we have T equal to yes. So now, we're looking for the
probability of rain given W and T are both yes. And now, in this case, we restrict
our multiplication by the PT given R table to the case where T equal to yes only.
In this case, again, we use the same sequel, only we have another select
statement which restricts us to use only those rows of this table.
Again, joining with the prior join that we had of T1 and T3.
But with the restriction that T = to yes, we'll now get a different result because
now we'll multiply this by 0.8 and the second row by 0.1, and these rows don't
count, so we get different results. And normalizing gives us the probability
of rain equal to yes, given the evidence is now 90%.
In a sense, our belief has undergrown a revision from the earlier value of 53% to
90%.. So, new evidence has changed our belief.
In classical logic, once you asserted that say, rain occurs or doesn't occur, you
can't change our belief. But, in probabilistic reasoning, belief
can be revised. It's very important.