[MUSIC]
We've talked at great length about models where you specify some set of features.
And what that results in is a model with fixed flexibility.
There's a finite capacity to how flexible the model can
be based on the specified features.
But now we're gonna take a totally different approach.
There are approaches that are called nonparametric approaches.
And the one that we're gonna look at in this module
is called k-nearest neighbors and kernel regression.
And what these methods allow you to do is be extremely flexible.
The implementations are very, very simple and they allow
the complexity of the models you infer to increase, as you get more data.
This really simple approach is surprisingly hard to beat.
But as we're gonna see,
all of this relies on us having enough data to use these types of approaches.
To start with,
let's talk about this idea of fitting a function fit globally versus fit locally.
So let's imagine that we have the following data that represent
observations of houses with a given square feet and their associated sales price.
And maybe just for the sake of argument in this module,
let's imagine that for small houses there tends to
be a linear relationship between square feet and house value.
But then that relationship tends to taper off and
there's not much change in house price for square feet.
But then you get into this regime of these really large houses where as
you increase the square feet,
the prices can just go way up because maybe they become very luxurious houses.
In the types of models that we've described so far, we've talked
about fitting some function across the whole input space of square feet.
So for example, if we assume a really simple model,
like just a constant fit, we might get the following fit to our data.
Or if we assume that we're gonna fit some line to the data,
we might get the following or a quadratic function.
And we might say, well, this data kinda looks like there's some cubic fit to it.
And so the cubic fit looks pretty reasonable, but
the truth is that this cubic fit is kind of a bit too flexible,
a bit too complicated for the regions where we have smaller houses.
It looks like it fits very well for our large houses but
it's a little bit too complex for lower values of square feet.
Because, really for this data you could describe it
as having a linear relationship like, I talked about for low square feet value.
And then, just a constant relationship between square feet and value for
some other region.
And then, having this maybe quadratic relationship between square feet and
house value when you get to these really, really large houses.
So this motivates this idea of wanting to fit our function locally to
different regions of the input space.
Or have the flexibility to have a more local description of what's
going on then our models which did these global fits allowed.
So what are we gonna do?
So we want to flexibly define our f(x) the relationship in this
case between square feet and house value to have this type of local structure.
But let's say we don't want to assume that there is what are called structural
breaks that there are certain change points where the structure of our
regression is gonna change.
In that case, you'd have to infer where those break points are.
Instead let's consider a really simple approach that works well when you have
lots of data.
[MUSIC]