[MUSIC] And I wanna talk about this notion of
overfitting because this is something that we've talked about before in the course. I wanna formalize it, and we're gonna discuss it a lot more
in the remainder of this course. Okay, so the notion of overfitting
is if you have some model, let's say a model here
with parameters W hat, so this model has some complexity and some
associated estimated parameters, W hat. Well, this model is overfit, if there exists a model with estimated parameters, I'll just call them w prime. So let's just say some other point here. Let's say these have parameters w prime such that two conditions hold. The training error, so one is training,
can't spell right now. Training error of w hat is less than the training error of w prime. But on the other hand, the true error of w hat is greater than the true error of w prime. Okay, so
this might not seem that intuitive, but let me go through it in
terms of this picture here, which is exactly what these points,
one and two, are saying. Which is there are a wide
range of models that have true error larger than for
example, this w prime here. But the ones that are overfit are the ones
that have smaller training error. These are the ones that are really, really
highly fit to the training data set but don't generalize well. Whereas the other points on the other
half of this space are the ones that are not really well fit to the training
data and also don't generalize well. Okay, so this is formally our notion
of what an overfitted model is. [MUSIC]