Let's now revisit machine learning, like we studied during the listen chapter and
the naive based classifier is one method for the, machine learning.
Let's try to do this a little bit formally.
In a manner so as to make some of the material that will come later fall
naturally into the same formal framework. We have a bunch of data.
In this case, features X1 to XN which are. We call X, capital X a, a vector, a
collection of features. Earlier we had different examples, you
know, the query terms that we used, the words in a comment.
And in general, anything that we observe about instances which we then want to
classify into different. Classes and then we have some output
variables which are the labels that we would like to assign to a class, whether a
positive sentiment or a negative sentiment or a buyer, or a browser, and.
In this case, we'll say the output variable is zero.
If it's a buyer, a one. If it's a browser, a zero.
If it's a negative, a one. If it's positive, and so on.
In general, we might have many different output variables.
Each of them 01, and. The points in the space, X, could be shown
as, you know, points in, in some high dimensional space.
I've shown them here as just points in the plane.
But in general each of these will have many coordinates.
Like N for example. And some of them are red which means they
have zero assigned to them. And some of them are blue.
That means they have a one assigned to them, in terms of the Y variable.
And the,, the goal of classification is to figure out, given a new point whether it's
likely to bem red or blue. In general, this need not to be a binary
task and you may want to classified in to three, four many classes, but the moment
we stick to binary classification, as it makes things a bit simpler.
Now, to formally model the classification problem, if you have a bunch of features
in a space, we define a function, let's say, F of X which is the expected value Of
the output variables y, given the input variable, x.
Now, this is a little bit different than what we had earlier, which was the
probability. Of getting some variable Y equal to zero
or one, given a particular combination of X's but as we shall soon see it really its
the same thing, at least for classification, but for other types of
machine learning we'll have different values of F and that's why this framework
is useful so that it unifies our whole concept of machine learning.
So, what we're trying to do is figure out what is the expected value of y.
Is it a zero or a one? Is it closer to zero or closer to one
given a particular combination x? Well, the expected value is nothing but.
And clearly, Y is zero one, then the expected value is.
One times the probability of Y equal to one, given X, plus zero times the
probability that Y equal to zero, given X. So.
Since you multiply by zero, this term disappears, and we simply get the
probability that Y equal to one, given X. And this is exactly what we had earlier
estimated using various assumptions, like independence and the bye, Bayes rule.
And we will, could figure out that then we, all we needed was a training set were
we could compute the required likelihoods that would enable us to compute the
probability of y equal to one given a combination x.
Now that's, that's fairly basic. But this formulation tells us a few
things. First.
It's not always necessary to compute this probability y equal to one given x.
Directly. We did that through some approximation in
ninth phase. We might instead want to compute this
function F of X. In this case it turns out to be just the
same as this probability. In other cases it might not be the same.
For example if you have a classifier where you didn't have two classes you might have
a different combination of F. The second thing it tells you is that.
The probability. It's one way of figuring out where a
particular combination X belongs. Another way might be to somehow estimate
this F and estimate the boundary. Where F of X is more than or less than a
half. For example, in this case, if you could
figure out that the boundary lay somewhere along this line over here.
So that we minimize the number of false positives and true negatives.
We could find a much easier way to classify the points as being positive or
negative than actually computing this rather complex animal, which is the a
posteriori probability. Directly trying to find estimates of F is
what more sophisticated machine learning techniques, like support vector machines,
end up doing. They do other complicated things like,
changing the nature of the space. So instead of dealing with the space at
its given, they use, various combinations of the x's, sometimes squaring them,
sometimes doing, other, funny things to them to, project this space to a
different, space, and then find, a line or a plane which, nicely separates, the,
instances of positive and negative. So what they're really doing is trying e-,
estimate the function boundary F directly, without actually computing the function
directly. Now, that's another long-winded way of
defining machine learning, but it, it will serve to.
Unify the different types of learning that we'll see as we go along this week.
Let's take a look at some examples using this new notation that we just introduced.
Recall our machine learning of whether somebody's a, buyer or a browser, based on
the queries that they, issue in a search box.
In this case the queries that we had last time were, red, flower, gift, or cheap.
And we had various instances of people querying, and then possibly buying or not
buying. So our x was essentially the set of query
words, y was whether or not one buys. And so we have y, x as our.
Space, B, R, F, G and C in the notation we had last time.
Similarly, we had machine learning of positive or negative comments.
In this case, the sentiment was the y. The set of all words formed our X, which
is extremely high dimensional space. A word could either be present or absence
in a comment, and all of these would still binary variables.
We turn to another example now. Imagine a baby observing various animals,
And wants to figure out which animal is of what name.
So, we have various features, the size of the animal, the size of the head, the
noise the animal makes, the number of legs it has and the animal itself.
And the baby observes many instances and somehow it is able to discern these
features in terms of whether or not the size of the head is large or small or
medium. Or whether the animal itself is large or
small. And the noise that the animal makes, etc.
The machine learning task is then to classify a new animal into appropriate
category. Lion, cat, elephant, etc.
So here Y comma X is now the animal itself is Y, size of the animal, it's head size,
it's noise, number of legs or the features, it's a fixed set.
So here on the fore features of multi valued categorical variable so they take.
Values in specific categories. They don't, they're not real numbers for
example. They're, they take one of, of few
categories: small, medium, large for example.
And lastly we consider another example where we have customers going to
supermarkets and buying bunches of products together in transactions A
transaction might consists of milk, diapers and some Cola.
An other one might consists of some diapers and beer.
Yet more transactions will have different products.
And here we have. Interestingly no output variable, but only
features which are just the items that people buy and the items are again multi
valued categorical variables but they're a variables set just like the comments that
we had earlier that we could, we could consider the set of all possible items and
have binary variables or we could have a variable number of features and.
Multi-valued categorical variables. These examples are all cases where one
could do, various kinds of machine learning.
In the first three examples, that is queries, comments and animals, there was a
clear output variable, which indicated the class of the particular instance.
And so one could. Imagine a supervised machine learning
scenario, using something like a naive based classifier, to compute the
likelihoods and estimate the A plus GRI probability, or as we have just seen, the
expected value of the class, given X. In the last example transactions, there
was no output variable, so the task there is a little bit different, which, which
we, which we shall come to, very soon. Do go back over the formalism, and make
sure that you're able to figure out how. Classification happens in each of these
cases. That is exactly how the Y and the X are
formulated so that we get a formula, a formal representation of the problem in
terms of features and an output variable. In this particular case of course there is
no output variable as we have already mentioned.
And let's now get into the reason why we had that.