[MUSIC]
How are defining distance?
Well, in 1-d it's really straightforward because our distance
on continuous space is just gonna be Euclidean distance.
Where we take our input-xi and
our query x-q and look at the absolute value between these numbers.
So, these might represent square feet for two houses and
we just look at the absolute value of their difference.
But when we get to higher dimensions,
there's lots of interesting distance metrics that we can think about.
And let's just go through one that tends to be pretty useful in practice,
where we're going to simply Weight the different dimensions differently but
use standard Euclidian distance otherwise.
So, it looks just like Euclidian distance, but
we're going to have different weightings on our different dimensions.
So, just to motivate this, going back to our housing application,
you could imagine that you have some set of different inputs,
which are Attributes of the house, like how many bedrooms it has.
How many bathrooms, square feet.
All our standard inputs that we've talked about before.
But when we think about saying which house is most similar to my house.
Well, some of these inputs might matter more
than others when I think about this notion of similarity.
So, for example number of bedrooms, number of bathrooms, square feet of the house.
Might be very relevant, much more so
than what year the house was renovated when I'm going to assess the similarity.
So, to account for this, what we can do is we can define what's called a scaled
Euclidean distance, where we take the distance between
now this vector Of inputs, let's call it x,j.
And this vector of inputs associated with our query house x,q and
we're gonna component wise look at their difference squared.
But then we're gonna scale it by some number.
And then we're gonna sum this over all our different dimensions, okay?
So, in particular I'm using this letter a to denote the scaling.
So, a sub d
is the scaling on our dth input, and what this is capturing is
the relative importance of these different inputs in computing this similarity.
And after we take the sum of all these squares we're gonna take the square root
and if all these a values were exactly equal to 1, meaning that all our inputs
had the same importance then this just reduces to standard Euclidean distance.
So, this is just one example of a distance metric we can define at multiple
dimensions, there's lots and
lots of other interesting choices we might look at as well But lets visualize what
impact different distance metrics have on our resulting nearest neighbor fit.
So, if we just use standard Euclidean distance on the data shown here.
We might get this image, which is shown on the right where the different
colors indicate what the predicted value is in each one of these regions.
Remember each region you're gonna assume any point in that region,
the predicted value is exactly the same because it has the same nearest neighbor.
So, that's why we get these different regions of constant color.
But if we look at the plot on the left hand side, where we're using a different
distance metric, what we see is we're defining different regions where
again those regions mean that any point within that region is closer to
the one data point lying in that region, than any of the other data points
in our training data set, but the way this distance is defined is different so
thus the region looks different, so for example, with this Manhattan distance
what this is saying just think of New York and driving along the streets of New York.
It's measuring distance along this axis-aligned directions, so
it's distance along the x direction plus distance along the y direction
which is a different difference than our standard Euclidean distance.
[MUSIC]