In this video, I'll talk about why it's difficult to learn sigmoid belief nets.
And then in the following two videos, I'll describe two different methods we
discovered that allow us to do the learning.
The good news about learning in sigmoid belief nets is that unlike Boltzmann
machines, we don't need two different phases.
We just need what in a Boltzmann machine would be the positive phase.
That's because sigmoid belief nets are what is called locally normalized models.
So we don't have to deal with a partition function or its derivatives.
Another piece of good news about Sigma belief nets, is that if we could get
unbiased samples from the posterior distribution over the hidden units, given
the data vector, Then learning would be easy.
That is, we could follow the gradient specified by maximum likelihood learning,
in a mini batch stochastic kind of way. The problem is that it's hard to get
unbiased samples, from the posterior distribution over the hidden units.
This is largely due to a phenomenon that Judeo Po calls explaining away.
And I'll explain, explaining away in this video and it's important to understand it.
Now, I'm going to talk about why it's difficult to learn sigmoid belief nets.
As we've seen, it's easy to generate an unbiased sample, once you've done the
learning. That is, once we've decided on the weights
and the network, we can easily see the kinds of things the network believes in by
generating samples from this model. This is done top down, one layer at a
time. It's easy, because it's a causal model.
However, even if we know the weights, it's hard to infer the posterior distribution
over hidden causes when we observe the visible effects.
The reason for this is that the number of possible patterns of hidden causes is
exponential in the number of hidden nodes. It's hard even to get a sample from the
posterior, which is what we need if we're going to do stochastic gradient descent.
So given this difficulty in sampling from the posterior.
It's hard to see how we can learn sigmoid belief nets with millions of parameters,
Which is what we'd like to do. This is a very different regime from the
one normally used with graphical models. There they have interpretable models, and
they're trying to learn dozens or maybe hundreds of parameters.
They're not typically trying to learn millions of parameters.
Now before I go into ways in which we can try and get samples from the posterior
distribution. I just want to tell you what the learning
rule is, if we could get those samples. So, if we can get an unbiased sample from
the posterior distribution of hidden states, given the observed data, then
learning is easy. So here's part of a sigmoid belief net,
and we're going to suppose that for every node we have a binary value.
So, for node J, that binary value is SJ. And that vector binary value is a global
configuration for the node, which is the sample from the posterior distribution.
In order to do maximum likelihood learning, all we have to do is maximize
the law of probability, that the inferred binary state of unit, I would be generated
from the inferred binary states of its parents.
So the learning rule is local and simple. The probability that the parents of I
would turn I on, is given by a logistic function that involves the binary states
of the parents. And what we need to do is make that
probability be similar to the actually observed binary value of I and although
I'm not going to derive it here, The maximum likelihood learning rule for
the weight WJI is simply to change it in proportion to the state of J times the
difference between the binary state of I and the probability that the binary states
of I's parents would turn it on. So to summarize,
If we have an assignment of binary states to all the hidden nodes, then it's easy to
do maximum likelihood learning in our typical stochastic way.
Where we sample from the posterior, and then we update the weights based on that
sample. And we average that update over a mini
batch of samples. So, let's go back to the issue of why it's
hard to sample from the posterior. The reason it's hard to get an unbiased
sample from the posterior over the hidden nodes, given an observed data factor at
the leaf nodes, is a phenomenon called explaining away.
So if you look at this little sigma B leaf net chair, it has two hidden causes and
one observed effect. And if you look at the biases, you'll see
that the observed effect of a high stress jumping is very unlikely to happen unless
one of those causes is true. But if one of those causes has happened,
the twenty cancels the minus twenty, and neither house will jump with the
probability of a half. Each of the causes is itself rather
unlikely but not nearly as unlikely as a host spontaneously jumping.
So if you see the house jump, one plausible explanation is that a truck hit
the house. A different plausible explanation, is that
it was an earthquake. And each of those has a probability of
about E to the minus ten. Whereas the house jumping spontaneously
has a probability of about E to the minus twenty.
However, if you assume both hidden causes, that has a probability of E to the
-twenty, so that's extremely unlikely, even if the house did jump.
So assuming there was an earthquake, reduces the probability that the house
jumped because the truck hit it. And we get an anti-correlation between the
two hidden causes when we've observed the house jumping.
Notice in the model itself, in the prior for the model, these two hidden causes are
quite independent. So if the house jumps.
This basically an even chunks that was because of the truck or because of the
earthquake. The posterior actually look something like
this. There's four possible patterns of hidden
causes, given that the house jumped. Two of them are extremely unlikely.
Namely that the truck hit the house, and there was an earthquake.
Or that neither of those things happened. The other two combinations are equally
probable, and you'll notice they form an exclusive all.
We have two likely patterns of causes which are just the opposites of each
other. That's explaining away.
Now that we've understood explaining away, let's consider,
Let's go back to the issue of learning a deep sigmoid belief net.
So we're going to have multiple layers of hidden variables.
They're going to give rise to some data in our causal model.
And we want to learn those weights, W, between the first layer of hidden
variables in the data. And let's see what it takes to learn W.
First of all, the posterior distribution over the first layer of hidden variables
is not going to be a factorial. They're not independent in the posterior.
And that's because of explaining away. So, even if we just had that layer of
hidden variables, once we've seen the data, they wouldn't be independent of one
another. But because we have higher layers of
hidden variables, they're not even independent in the prior.
This hidden variables in the laser bath created prior, and that prior itself will
cause correlations between the hidden variables in first layer.
To learn W, we need to learn the posteria in the first hidden layer were least in
the approximation to it. And even if you are only approximating it
we need to know all of the weights in higher layers in order to compute that
prior term. In fact it's even worse than that.
Because to compute that prior term, we need to integrate out all the hidden
variables and higher layers. That is we need to consider all possible
patterns of activity in these higher layers.
And combine them all to compute the prior that the higher levels create for the
first hidden layer. Computing that prior is a very complicated
thing. So these three problems suggest that it's
gonna be extremely difficult to learn those weights W.
And in particular, we're not gonna be able to learn them without doing a lot of work
in the higher layers to compute the prior. So now we're gonna consider some methods
for learning deep belief mets. The first one is the Monte Carlo method
used by Radford Neal. And that Monte Carlo method basically does
all the work. That is, if we go back to the previous
slide, it considers patents activity over all of the hidden variables.
And it runs a mark off chain that takes a long time to settle down, given the data
factor. And once it's settled down, to thermal
equilibrium, you get a sample from the posterior, but it's a lot of work.
So, in large deep belief nets, this methods pretty slow.
In the 1990's people developed much faster methods for learning deep belief nets,
which we call variational methods. In fact this is where variational methods
came from at least the artificial intelligence community.
The variational methods give up on getting unbiased sound post from the posterior and
they content themselves with just getting approximate samples that is samples from
some other distribution that approximates the posterior.
Now as we saw before, if we have samples from the posterior, maximum likelihood
learning is simple. If we have samples from some other
distribution, we could still use the maximum likelihood learning rule, but it's
not clear what will happen. On the face of it, crazy things might
happen if we're using the wrong distribution to get our samples.
There doesn't seem to be any guarantee that things will improve.
In fact there is a guarantee that something will improve.
It's not the log probability that the model would generate the data.
But it is related to that. In fact it's a lower band on that log
probability. And by pushing up the lower band, we can
usually push up the log probability.