[MUSIC]
And when we're thinking about using mixture models to do clustering,
note that they can also be used just to do what's called density estimations.
So estimate those types of curves over the histograms that we drew earlier.
But in our case, we're going to focus in on the clustering application.
And there, there's a really important other variable that we're going to
introduce, and that is the cluster indicator,
the assignment variable for every one of our observations.
So we have, this is the cluster
assignment for observation Xi.
So this is exactly the same variable that we had in k-means that was assigning
observations to clusters, but in that case just using the cluster center.
Okay, so let's step back and think about what our model is saying.
And the first question we can think about is, what's the probability that the ith
data point in our data set is associated with the kth cluster?
So for example, when we're talking about our images we could say,
what's the probability that the ith image I see is,
let's say, in the cluster of clouds images?
And let's talk about this before we've actually observed what the image is.
All we know is it's the ith index in our data set.
Okay, well this is fully specified by the mixture weight pi k,
because that tells us how prevalent cloud images are in our data set.
So that's given right here.
And if we don't observe the content of the image, then we just are caring about how
many cloud images do we have relative to forest images relative to sunset images.
So we say that the prior probability,
that the ith image is assigned to cluster k, is given by pi k.
Another question is, what if I know that an image comes from cluster k?
So I'm going to fix that.
I already know that it's a clouds image.
Now I can say,
what's the likelihood of observing the RGB vector associated with this image?
So Xi, given that the image came from the kth cluster,
this cluster of cloud images.
And in this case what we do is we simply go to, this is the distribution of
cloud images, or distribution of blue for
cloud images.
And we say okay, let's take this one image I have.
This is my Xi image.
And I look at its blue intensity and I say, under this distribution for clouds,
how likely is it?
Well, it's pretty likely.
So it's reasonable to say that this was a clouds image.
But I can also look at this probability under,
remember this was the forest category.
And I could say,
well what's the likelihood of this image under the forest category?
Well, it's not that high.
But on the other hand, what we know is that
there are many more forest images in our data set than cloud images.
So what we're going to be doing when we're going to form our soft assignments,
which we'll talk about in the next section,
is we're going to be thinking about weighting these two terms.
Saying, well what's the prior probability that this image
is form any one of these different classes?
So in this case, it's most likely a forest image.
But the I say, okay, well now I've observed the content of this image,
the RGB vector for this image, and I want to say, I need to weight that in.
And under the sunset category, it's extremely extremely unlikely.
There is basically zero probability of observing
this blue intensity value under that category.
So I can rule it out regardless of what the weight is on that category.
But for these other categories, these other clusters,
there's going to be some competition between how much I'm likely to just see
images of that type versus how likely it is under that category.
And we're going to use both of these things to
represent our uncertainty about the cluster assignment.
So just to circle back and make sure we're very clear when we're looking at
the probability of an observed RGB vector.
RGB for image i, given that it's in cluster k,
then this is just a single Gaussian with mean mu k and covariance sigma k.
And this is referred to as the likelihood term,
whereas before we called this the prior term.
And I want to point out that this image, indeed there should be
uncertainty about whether it's assigned to the clouds cluster or
the forest cluster, because here we see some trees.
And here we see some clouds.
So it'd be natural to have uncertainty on the assignment of this image.
[MUSIC]