[MUSIC]
So having finished the preceding modules,
I'm feeling pretty confident that I come in, I can specify a model, and
I can also specify an algorithm for how to fit that model.
In doing that, I come in and I get some fitted function, and
I know how to use that function to make predictions.
So I go, I make predictions about the value of my house.
I go to sell my house, and I make money.
And I'm happy, right?
I did a good job.
Well, maybe, maybe not.
Maybe my predictions weren't that good.
And so, as a result, the value that I list my house for was inaccurate.
And maybe I end up losing money as a result of that.
So what we can think about,
is a measure of how much are we losing when we make a certain prediction?
So for example in the housing application, if we list the house value as too low,
then maybe we get low offers.
And that's a cost to me relative to having made a better prediction.
Or if I list the value as too high, maybe people don't come see the house and
I don't get any offers.
Or maybe people notice that not many people are showing up
to look at the house and they make me a very low offer.
So, again, I'm in the situation of being in a worse financial state
having made a poor prediction of the value of my house.
So a question is,
how much am I losing compared to having made perfect predictions?
Of course we can never make perfect predictions, the way in which the world
works is really complicated, and we can't hope to perfectly model that as well as
the noise that's adherent in the process of any observations we might see.
But let's just imagine that we could perfectly predict the value,
then we'd say, in that case, our loss is 0.
We're not losing any money because we did perfectly.
So a question is, how do we formalize this notion of how much we're losing?
And in machine learning, we do this by defining something called a loss function.
And what the loss function specifies is the cost incurred
when the true observation is y, and I make some other prediction.
So, a bit more explicitly, what we're gonna do,
is we're gonna estimate our model parameters.
And those are w hat.
We're gonna use those to form predictions.
So, this notation here,
f sub w hat is something we've equivalently written as f hat, but
for reasons that we'll see later in this module, this notation is very convenient.
And what it is, is it's our predicted value at some input x.
And y is the true value.
And this loss function, L,
is somehow measuring the difference between these two things.
And there are a couple ways in which we could define loss function.
Well, there's actually many, many ways, but
I'm just gonna go through a couple examples.
And in particular, these examples that I'm gonna go through assume that the cost you
incur by doing an overestimate, relative to an underestimate, are exactly the same.
So there's no difference in listing my house as $1,000 too high,
relative to $1,000 too low.
Okay, so we're assuming what's called a symmetric loss function in these examples.
And very common choices include assuming something that's called absolute error,
which just looks at the absolute value of the difference between your
true value and your predicted value.
And another common choice is something called squared error, where, instead of
just looking at the absolute value, you look at the square of that difference.
And so that means that you have a very high cost if that difference is large,
relative to just absolute error.
So as we're going through this module,
it's useful to keep in the back of your mind this quote by George Box.
Which says that, Remember that all models are wrong;
the practical question is how wrong do they have to be to not be useful.
Okay, so we have spent a lot of time defining different models, and
now we're gonna have tools to assess the performance of these methods,
to think about these questions of whether they can be useful to us in practice.
[MUSIC]