But before we continue our story about the Algorithm,
I wanted to spend a little bit of time showing how these soft assignments
are actually an application of something called Bayes' rule.
And this is advanced material, so we're going to make this an optional video.
And for those who are happy with the level of description
of the responsibility equation in the previous video, then that's fine.
Skip ahead to the next video.
But for those who are interested in a little bit more detailed derivation and
explanation of the equation, feel free to watch this video and
see a few more details.
Okay, so remember that our responsibilities were the probability
of an assignment of observation i to cluster k,
given our cluster parameters and given the observed data value xi.
Well let's just call all those cluster parameters just params.
And then what we see is that we can rewrite this equation and say pi k,
so the prior probability of an assignment of zi=k is just what's written here.
Probability zi=k given our parameters,
where in this case the only parameter we actually need is pi k.
And likewise for this likelihood term,
we can write it generically as probability of our observed data point,
xi given an assignment to cluster k, zi = k.
And given our model parameters where again here in this case,
the only parameters that we're going to need are mu k and sigma k.
Okay, but we're trying to write things more generically so
that we can show how this is an application of Bayes' rule.
But to do this, we're going to say that the event zi = k,
we're going to call that event A.
And the event that we observe a value xi for our ith data point,
we're going to call event B.
Now using this, we can rewrite the terms in our numerator as probability
of A given our parameters and probability of B given A and our parameters.
Then we can do this same thing in the denominator where now we're going to
call the event zi = j, an assignment of observation i to cluster j.
We're going to call that sum event C.
And so we can rewrite the numerator as a probability of C given our
parameters times the probability of B, still the same observed valued.
Now given this event C and again our model parameters, and
in the denominator we're going to sum over all these possible events C.
And then putting all these pieces together, we have the following.
We have that our responsibilities, our probability of A given B, and
our parameters, and we rewrote that on a previous slide in the following form.
Where here in the denominator, if we look at this thing here inside,
we get that this is by definition equivalent to
probability of events B and C given our model parameters.
And so when we look at this sum event,
we have sum over event C, probability of
events B and C given our parameters.
So this means just to be
clear, events B and C.
Then again, if we sum over one of these events,
if we're looking at every time B occurs with a specific example of C, and
then we're summing over all possible values of C.
That's exactly equivalent to just the probability of that single event,
B, given our model parameters.
So then using this fact, this is how we get the denominator here,
and now what we see is we have just a much general form of the standard Bayes' rule.
In particular, the standard way
we would write Bayes' rule is that probability of A given B
is equal to probability of A probability of B
given A divided by probability of B.
And we see exactly this form, except the only difference is we're also conditioning
on something, our model parameters.
And we're doing so for every term in this Bayes' rule.
Okay, so in summary, the derivation of the equation for
the soft assignments is just an application of Bayes' rule.
So that we can utilize known quantities from our model,
like the prior probabilities of cluster assignments and the likelihood of data
points, given a cluster assignment, to form this responsibility.
[MUSIC]