We've seen a number of techniques, some in
previous lectures and some we talked about
in this lecture.
We've seen the Naive Bayes classifier,
fairly in detail.
We've seen some Probabilistic Graphical
Models like Bayesian Networks.
We've seen linear regression in detail,
this time, we also heard about logistic
regression neural networks and support
vector machines, at least as to what they
are.
And, let's look at the problem and see
which kind of techniques one would need to
consider depending on the nature of the
problem.
We classify the problem in terms of the
kind of features it has, whether they are
numerical or categorical, that means
numbers or classes.
And the target variable, which is what we
are trying to predict.
We might be predicting a value, then it
becomes a prediction problem where the
value is numerical.
We might be predicting a class in which,
in which case it's a classification
problem which is part of learning theory.
Techniques can be used interchangeably
across these two different types of
prediction based on the kinds of features,
of course, some techniques are more
applicable than others.
So, in the most straightforward case,
If we have numerical features and a
numerical target we want to predict,
The correlation is stable and fairly
linear, we'd use linear regression.
Now, when I say stable and fair, fairly
linear, even in situations like this,
One would still prefer to use linear
regression rather than some complicated
non-linear function.
Because using high order functions we'd,
we'd say squares or cubes and sines and
cos's, will tend to over fit the data and
will not generalize to situations which
may come up, arise in the future.
So, right now you might have a great fit
to the training data, but it really
doesn't work in practice.
So, linear regression is preferred unless
you have some real reason to not use
linear techniques.
Similarly, even if your futures are
categorical and your target is numerical,
you can still use linear regression but
you have to code the features. So, if the
feature, for example, takes five different
values or eight different values, you
replace that feature with eight
categorical variables, binary ones taking
zero and one depending on whether, which
value, which category value that feature
took.
It's better to do with, with binary coding
as opposed to say, numerical coding
because there's no reason why red being
coded as five, blue being coded as six,
and green being coded as seven.
There's no reason to believe that red and
blue are closer than red and green.
So, using five, six, and seven is
misleading and can make the regression
technique go haywire.
So, using three different features, eight,
zero, one, to figure out whether something
is red, blue, or green is better than
using numbers.
When we have categorical variables and
numerical target,
Neural networks can also be used just like
they can be used for normal linear
regression as well.
But, they've sort of waned in their
popularity except for certain situations
which we will talk about in the next
section.
Now, let's come to the case where we have
unstable or severely non-linear
situations, which might look something
like this, as we have seen before.
There is no way one can fit a straight
line to this para, this parabolic curve.
and therefore, it's better to use a neural
network which has non-linear elements,
multi-level and hidden layers.
So, a more complicated function can be
learned, at the same time, one is not
pre-supposing that it's going to be a
problem.
Because that, that would be kind of
counter-productive because one is sort of
pre-supposing the nature of f, rather than
letting a neural network with many
different possibilities discovered..
Next, we come to the classification
situations where the target variable is
categorical.
Of course, when we have categorical
features and categorical targets, when we
have seen how to use naive-based and other
probabilistic graphical models.
These days, SVM's or Support Vector
Machines are also very popular for even
classification.
Of course, for catagorical variables, one
does have to do feature coding to a
certain extent. So, we, we,
We do need to do the same trick that we
did for categorical features in linear
regression because SVM essentially
requires numerical inputs.
Of course, if you have numerical inputs
and categorical classification, SVMs are
perfect. They are designed especially for
those situations where you have unstable
and severely non-linear correlations, and
that's what they essentially do well, very
well. On the other hand, if you have
fairly stable linear correlations and you
do have a classification problem,
Then rather than using linear regression,
as we have seen, one should use a logistic
regression where one is bumping up or
bumping down the, the difference form the
separating line using the logistic
function.
So, take a look at this table. It will
guide you, definitely in the problem set
or the homework assign, the programming
assignment for prediction.
But, in general also, it's something that
you should learn something from.
We have not covered many techniques yet,
we've only taken a very few techniques.
Further, we've only talked about
classification prediction, optimization,
control, we haven't talked about those and
we won't have time to get into those in
this course.