In this video, I would like
to talk about how to
evaluate a hypothesis that has
been learned by your algorithm.
In later videos,
we will build on this
to talk about how to
prevent in the problems of
overfitting and underfitting as well.
When we fit the parameters
of our learning algorithm
we think about choosing the parameters
to minimize the training error.
One might think that
getting a really low value of
training error
might be a good thing,
but we have already seen that
just because a hypothesis
has low training error,
that doesn't mean it is
necessarily a good hypothesis.
And we've already seen the
example of how a hypothesis can overfit.
And therefore fail to generalize the
new examples not in the training set.
So how do you tell if
the hypothesis might be overfitting.
In this simple example 
we could plot the hypothesis h of x
and just see what was going on.
But in general for problems with
more features than just one feature,
for problems with a large
number of features like these
it becomes hard or
may be impossible
to plot what the hypothesis looks like
and so we need some other way
to evaluate our hypothesis.
The standard way to evaluate
a learned hypothesis is as follows.
Suppose we have
a data set like this.
Here I have just shown 
10 training examples,
but of course usually we may have
dozens or hundreds or maybe
thousands of training examples.
In order to make sure we
can evaluate our hypothesis,
what we are going to do is split
the data we have into two portions.
The first portion is going 
to be our usual training set
and the second portion 
is going to be our test set,
and a pretty typical split of this
all the data we have into a
training set and test set
might be around
say a 70%, 30% split.
Worth more today to
grade the training set
and relatively less to the test set.
And so now, if
we have some data set,
we run a sine of say 70%
of the data to be our
training set where here "m"
is as usual our
number of training examples
and the remainder of our data
might then be assigned
to become our test set.
And here, I'm going to use
the notation m subscript test
to denote the number of test examples.
And so in general, this
subscript test is going to denote
examples that come
from a test set so that
x1 subscript test, 
y1 subscript test is my first
test example which 
I guess in this example
might be this example over here.
Finally, one last detail
whereas here I've drawn this
as though the first 70%
goes to the training set and
the last 30% to the test set.
If there is any sort of
ordinary to the data.
That should be better
to send a random 70%
of your data to
the training set and a
random 30% of
your data to the test set.
So if your data were
already randomly sorted,
you could just take
the first 70% and last 30%
that if your data
were not randomly ordered,
it would be better to randomly shuffle or
to randomly reorder
the examples in your training set.
Before you know sending
the first 70% in the training set
and the last 30% of the test set.
Here then is a
fairly typical procedure
for how you
would train and test
the learning algorithm
and the learning regression.
First, you learn the parameters
theta from the training set
so you minimize the usual 
training error objective j of theta,
where j of theta 
here was defined using that
70% of all the data you have.
There is only the training data.
And then you would
compute the test error.
And I am going to denote
the test error as j subscript test.
And so what you do is
take your parameter theta
that you have learned from 
the training set, and plug it in here
and compute your test set error.
Which I am going to write as follows.
So this is basically
the average squared error
as measured on your test set.
It's pretty much what you'd expect.
So if we run every test
example through your hypothesis
with parameter theta and 
just measure the squared error
that your hypothesis has on
your m subscript test, test examples.
And of course, this is
the definition of the
test set error if 
we are using linear regression
and using
the squared error metric.
How about if we were doing
a classification problem
and say using
logistic regression instead.
In that case,
the procedure for training
and testing say
logistic regression is pretty similar
first we will do the parameters
from the training data,
that first 70% of the data.
And it will compute
the test error as follows.
It's the same objective function
as we always use but
we just logistic regression,
except that now is define using
our m subscript test,
test examples.
While this definition of
the test set error
j subscript test is
perfectly reasonable.
Sometimes there is an alternative
test sets metric that
might be easier to interpret,
and that's the misclassification error.
It's also called the zero one
misclassification error,
with zero one denoting that
you either get an example right
or you get an example wrong.
Here's what I mean.
Let me define
the error of a prediction.
That is h of x.
And given the label y as
equal to one
if my hypothesis
outputs the value
greater than equal to five
and Y is equal to zero
or if my hypothesis outputs 
a value of less than 0.5
and y is equal to one,
right, so both of these
cases basic respond
to if your hypothesis
mislabeled the example
assuming your threshold at an 0.5.
So either thought it was more
likely to be 1, but it was actually 0,
or your hypothesis
stored was more likely
to be 0, but the
label was actually 1.
And otherwise, we define
this error function to be zero.
If your hypothesis basically
classified the example y correctly.
We could then
define the test error,
using the misclassification
error metric to be
one of the m tests
of sum from i equals one
to m subscript test of the
error of h of x(i) test
comma y(i).
And so that's just my way of
writing out that this is exactly
the fraction of
the examples in my test set
that my hypothesis has mislabeled.
And so that's the definition of
the test set error using the
misclassification error
of the 0 1 
misclassification metric.
So that's the standard
technique for evaluating
how good a learned hypothesis is.
In the next video, 
we will adapt these ideas
to helping us do things
like choose what features
like the degree polynomial
to use with the learning algorithm
or choose the regularization
parameter for learning algorithm.