[MUSIC] So this leads directly into this
discussion that clustering is an unsupervised task. So let's talk a little bit
about what this means. Because so far in this specialization
we've assumed that we've been presented with a set of outputs
associated with our inputs. So for example in the regression course
when we're talking about predicting house prices, in our training data set, we assume that not only do we have the
features of the house which are the input, but we also had the price, the sales
price of the house, which was output. In the classification course,
we assume that we had restaurant reviews, we have a text of those restaurant
reviews which are our input and now we also had the output, whether it
is a positive or a negative review for the reviews in our training set. So, if we were to imagine that we
had labels in this application, maybe we would have labels of
articles in term of topics. So these articles where the input again
would be the text of the article, just like with reviews but
the output would be a topic like sports or world news or entertainment or science. And so in this case,
we can think of problem as a multi class classification problem,
where imagine you get some new article and you want to label what topic
that article is about. Well then you just go
into your training set and based on the classifier that you've
learned from that training set where labels were provided, you can then think
about labeling this document as one of these five different
topics presented here. So this is an example of a supervised
learning problem, because outputs are provided or these class labels
are provided on our training examples. But when we're thinking about doing
clustering we're assuming there are no labels provided, or that
the labels are really really unreliable. And the goal here is to uncover
this grouping structure directly from the data,
the input values alone. So in particular,
here we might imagine that we have documents with just two
words in the vocabulary. So let's say here's word one, counts, and here's word two, counts, and each one of these points is a given document, document Xi. And so in particular, our input to the algorithm is going to be
documents represented as vectors, Xi. But then the output are going
to be cluster labels. So in particular,
maybe we'll label this red cluster as cluster 1,
this green cluster as cluster 2, and this blue cluster as cluster 3,
then associate it with Xi. The output might be zi = 1,
that it's associated with cluster 1, and we are going to output this for
every observation in our data set. Just to be clear, maybe this red cluster
corresponds to articles about world news, this green cluster corresponds
to articles about science, and this blue cluster corresponds
to articles about entertainment. So we're not provided with those labels,
but we're going to group articles, color them,
as we're doing in this picture, based on similarities in
the observed input space. So in this simple 2D example, this
would be similarities between articles, based on a simple two word vocabulary and counting how many times those words
appear in each of these articles. Okay, well this is an example of
an unsupervised learning task because all we're presenting our
algorithm with are the inputs and the algorithm is supposed to output
a set of labels as the output. Well, how can we think
about learning such labels? Well a key component of that question is
thinking about what defines a cluster? And once we've defined
what a cluster means, then we need to define algorithms for
inferring these clusterings. Okay, but
a cluster is defined by its center, which is often called a centroid, as well as the shape of the cluster. And in this application here,
the clusters are defined by ellipses. So here, this red circle is
an example of an ellipse, and this green cluster has this elongated
ellipse that's at one angle and the blue cluster has another smaller
ellipse at a different rotation. So these are the family of shapes
that we're exploring, our ellipses, different rotations and
stretchings of these ellipses, but that's what we're using
to define a cluster. And then,
when we have a given observation, again, observation xi,
which is a specific document. The way we think about assigning that
document to a given cluster is based on something we'll call
the score under each cluster. So often we just think about defining
this score as the distance to the cluster center, so we can think about,
let me choose another color just for fun to make this stand out
a little bit more on this slide. So the distance to
the cluster centers are these distances here, And what we see is that the smallest distance
is the distance to this red cluster. So if the distance to the cluster
alone is what defines the score, then this document Xi would be
assigned to this red cluster. [MUSIC]