[MUSIC]
So this leads us straight into a discussion of how are we going to compute
this distance between two given articles.
Well, when we're in 1D one really simple measure that we can use is
just Euclidean distance.
And hopefully, this should be fairly familiar to you, but
this really isn't going to be something of interest to us because this would be
assuming that we just have, in our example, just one word in our vocabulary.
And in almost all the scenarios that we're going to think about in this
specialization, we're going to assume that we have multiple features, or
multiple different dimensions that we want to consider.
And in this case, things get really interesting.
There are lots of different distance functions we can think about using.
And just one example of ways in which we can do interesting things in
multiple dimensions,
is we could think about weighting the different dimensions differently.
So we could think about putting different weights on
different words in the vocabulary, or different features that we might have.
So for example, if you go back to course two when we we're talking about
regression and we're talking about predicting the price of a house,
we looked at using the nearest neighbor regression to predict that house value and
we said, we can put different weight on a different attributes of the house.
So maybe if we're thinking about what features are really important for
predicting house value, is things like number of bedrooms, number of bathrooms,
square footage of that house.
Those are really important but maybe other things like number of floors or
years renovated are less important to assessing that value.
Well, in our document example there's this very similar analogy.
So maybe when we're going to compute the similarity between two different articles,
maybe we want to weight more heavily on the title, maybe that's really,
really informative.
And much more so than the body of the article that can have a lot
of noise in these words that are kind of hard to account for.
Likewise, if there's an article that has an abstract like a scientific article,
that might also be more informative than the main body of the article.
So these are both examples where you might want to specify weights
that are different across the different features that you have.
Another case where you might want to weight different features differently,
is in scenarios where one of your features varies just a little bit
across the different observations you have, but the other feature varies widely.
So this can be because one of the features is in a different unit than the other
feature or could just be that there's a lot of variance in that dimension.
So in this cases what would happen is if you go to just compute something like
including distance weighing both of this features equally, well,
the one where you get these big changes might dominate these little changes.
But really, in practice, it might be that these little changes in Feature 1 that we
have here are as important as a larger change in Feature 2,
the feature that varies more wildly across the different observations.
So in this case, there are a couple things that people tend to do.
And they both relate to scaling the feature by some measure of the spread of
observations.
So one way that you could account for the spread is simply to take for
feature J, take every observation in that column.
So you take here every row, remember it's a different observation.
Each column has a different feature.
So you take an entire column of your data matrix and you scale it by the maximum
overall values in that column minus the minimum overall values in that column.
And you do that for every observation in that column.
An alternative is to scale by one over the variants of all
observations of that feature.
So all of these are cases where we introduce
weights across our different features when we're going to computer distance.
So formally, we can think about computing what's called Scaled Euclidean distance.
Where it looks very much like standard Euclidean distance in multiple dimensions.
But now across each one or our different dimensions we have a different weight
on that dimension which I'm indicating here with it's ai.
So a1 all the way to ad and so these are weights on a different features,
and what they represent is the relative importance of these different features.
And one example of how you could think about setting the weights is just as
binary weights, 0s and 1s.
That would be a special case of scaled Euclidian distance computation.
And what that's equivalent to is feature selection, because if you set a weight
equal to 0, you're knocking out that feature altogether.
And it's not getting incorporated into the computation of the distance and
so you're saying that future just not matter for
the sake of assessing similarity or distance between two different articles.
But remember here in contrast the one we talked about things like lasso or
other notions or feature selection, here, we're pretty specifying what these weights
are or in this binary case which features including which ones are excluded.
But overall the thing that I really want to emphasize here is the fact that
how we specify our data representation and
compute this distance is really, really, really important.
And it's a very challenging thing to do.
So this idea of feature engineering or feature selection is very important,
but it's also a fundamentally hard task and it's a task that there is
literature on how to think about going about this feature engineering.
But it really is an area in machine learning where a lot of tweaking comes in
and this is one of the places where there's a knob to turn, and
a lot of domain knowledge often comes in and thinking about how
to think about setting these weights, or defining these distances.
So I just want to emphasize that it really matters.
There is no one solution for how to go about this.
But think about it don't just compute some distance and assume that, that represents
a distance that's of importance in the application without thinking about
what the data is and what's happening when you're going to compute that distance.
[MUSIC]