In this video, I'll go into more detail about how we can speed up the Boltzmann
Machine Learning Algorithm by using cleverer ways of keeping Markov chains
near the equilibrium distribution, or by using what are called mean field methods.
The material is quite advanced and so it's not really part of the course.
There won't be any quizzes on it and it's not on the final test.
You can safely skip this video. It's included for people who are really
interested in how to get deep Boltzmann machines to work well.
There are better ways of collecting the statistics than the method that Terry
Snofsky and I originally came up with. If we start from a random state, it may
take a long time to reach thermal equilibrium.
Also, there's no easy tests for whether you've reached thermal equilibrium, so we
don't know how long we need to run for. So, the idea is why not start from
whatever state you ended up in last time you saw that particular data vector?
So, we remember the interpretation of the data vector in the hidden units, and we
start from there. This stored state, the interpretation of
the data vector, is called a particle. Using particles that persist gives us a
warm start and it has a big advantage. If we were at equilibrium before and we
only updated the weights a little bit, it'll only take a few updates of the units
in a particle to bring it back to equilibrium.
We can use particles for both the positive phase, where we have a clamp data vector,
and for the negative phase, when nothing is clamped.
So, here's the method for directing statistics introduced by Radford Neal in
1992. In the positive phase, you have a set of
data specific particles, one or a few per training case. And each particle has a
current value that's the configuration of the hidden units plus which data vector it
goes with. You sequentially update all the hidden
units a few times in each particle with the relevant data vector clamped.
And then, for every connected pair of units, you average the probability of the
two units being on over all these particles.
In the negative phase, you keep a set of fantasy particles.
That is, these are global configurations. And again, after each weight update, you
sequentially update all the units in each fantasy particle a few times.
Now, you're updating the visible units as well.
And for every connected pair of units, your average, SiSj, over all the fantasy
particles. The learning rule is then the change in
the weights is proportional to the average you got with data, averaged over all
training data, and the average you got with the fantasy particles when nothing
was clamped. This works better than the learning rule
that Terry Snofsky and I introduced, at least for full batch learning.
However, it's difficult to apply this approach to mini batches.
And the reason is, that by the time we get back to the same data vectorn if we're
using mini batch learning, the weights would have been updated many times.
Son the stored data specific particle for that data vector won't be anywhere near
thermal equilibrium anymore. The hidden units won't be in thermal
equilibrium with the visible units of the particle given the new weights.
And again, we don't know how long we're going to have to run for, before we get
close to equilibrium again. So, we can overcome this by making a
strong assumption about how we understand the world.
It's a kind of a epistemological assumption.
We're going to assume that when a data vector is clamped, the set of good
explanations, that is states of the hidden units, that act as interpretations of that
data vector is uni-modal. That means we're saying that, for a given
data vector, there aren't two very different explanations for that data
vector. We assume that for sensory input, there is
one correct explanation. And if we have a good model of the data,
our model will give us one energy minimum for that data point.
This is a restriction on the kinds of models we're willing to learn.
We're going to use a learning algorithm that's incapable of learning models in
which a data vector has many very different interpretations.
Provided we're willing to make this assumption, we can use a very efficient
method for approaching thermal equilibrium or an approximation to thermal
equilibrium, with the data. It's called a mean field approximation.
So, if we want to get the statistics right, we need to update the units
statistically and sequentially. And the update rule is the probability of
turning on unit, i, is the logistic function of the total input it receives
from the other units in its bias. Where Sj, the state of another unit, is a
stochastic binary thing. Now, instead of using that rule, we could
say, we're not going to keep binary states for unit i, we're going to keep a real
value between zero and one which we call a probability.
And that probability at time t + one is going to be the output of the logistic
function. The more you put in is the bias, and the
sum of all the probabilities at time t times the weights.
So, we're replacing this stochastic binary thing by a real value probability.
And that's not quite right because this stochastic binary thing is inside a
non-linear function. If it was a linear function, things would
be fine. But because the logistics non-linear, we
don't get the right answer when we put probabilities instead of fluctuating
binary things inside. However, it works pretty well.
It can go wrong by giving us biphasic oscillations because now we're going to be
updating everything in parallel. And we can normally deal with those by
using what's called damped mean field where we compute that pi of t1.
+ one. But, we don't go all the way there.
We go to a point in between where we are now, and where this update wants us to go.
So, in damped mean field, we'll go to lambda times the place we are now, plus
one minus lambda times the place the update rule tells us to go to.
And that will kill oscillations. Now, we can get an efficient mini batch
learning procedure for both the machines, and this is what Russ Salakhutdinov
realized. In the positive phase, we can initialize
all probabilities at 0.5. We can clamp a data vector on the visible
units, and we can update all the hidden units in parallel using mean field until
convergence. And for mean field, you can recognize
convergence is when the probability stop changing.
And once we converged, we record PiPj for every connected pair of units.
In the negative phase, we do what we were doing before.
We keep a set of fantasy particles, each of which has a value that's a global
configuration. And after each weight update, we
sequentially update all the units in each fantasy particles a few times.
And then, for every connected pair of units, we average SiSj, these stochastic
binary things, over all fantasy particles. And the difference in those averages is
the learning rule. That is, we change the weights by an
amount proportional to that difference. If we want to make the updates for the
fantasy particles more parallel, we can change the architecture of the Boltzmann
machine. So, we're going to have a special
architecture that allows alternating parallel updates for the fantasy
particles. We have no connections within a layer, and
we have no skip-layer connections, but we allow ourselves lots of hidden layers.
So, the architecture looks like this. We call it a Deep Boltzmann Machine.
And, it's really a general Boltzmann machine with lots of missing connections.
All those skipped layer connections, if they were present.
We wouldn't really have layers at all, it would just be a general Boltzmann machine.
But, in this special architecture, there's something nice we can do.
We can update the states for example the first hidden layer and the third hidden
layer, given the current states of the visible units and the second hidden layer.
And then, we can update the states of the visible units in the second hidden layer.
And then, we can go back and update the other states,
And we can go backwards and forwards like this.
And so, we can update half the states of all the units in parallel and that'll be a
correct update. So, one question is, if we have a deep
Boltzmann machine like that trained by using mean field for the positive phase
and updating fantasy particles by alternating between even layers and odd
layers for the negative phase, can we learn good models of things like the MNIST
digits, or indeed, a more complicated things?
So, one way to tell whether you've learned a good model is after learning, you remove
all the input and you just generate samples from your model.
So, you run the Markov chain for a long time until it's burned in, and then you
look at the samples you get. So, Russ Salakhutdinov used a eep
Boltzmann machine to model MNIST digits using mean field for the positive phase,
And alternating updates of the layers of the particles for the negative phase.
And the real data looks like this. And the data that he got from his model
looks like this. You can see, they're actually fairly
similar. The model is producing things very like
the MNIST digits so it's learned a pretty good model.
So here's a puzzle. When he was learning that, he was using
mini-batches with 100 data examples and also he was using 100 fantasy particles,
the same 100 fantasy particles for every mini-batch.
And the puzzle is, how can we estimate the negative statistics with only 100 negative
examples to characterize the whole space? For all interesting problems, the global
configurations base is going to be highly multi model.
And how do we manage to find and represent all the nodes with only 100 particles?
There's an interesting answer to this. The learning interacts with the Markov
chain that's being used to gather the negative statistics, either one that's
used to update the fantasy particles, and it interacts with it to make it have a
much higher effective mixing rate. That means, we cannot analyze the learning
by thinking of it being an outer loop that updates the weights,
And an inner loop that gathers statistics with a fixed set of weights.
The learning is affecting how effective that inner loop is.
The reason for this is that whenever the fantasy particles outnumber the positive
data, the energy surface is raised, and this has an effect on the mixing rate of
the Markov chain. It makes the fantasies rush around
hyper-actively, And they move around much faster than the
mixing rate of the mark of chain to find better current static weights.
So, here's a picture that shows you what's going on.
If there's a mode in the energy surface that has more fantasy particles than data,
the energy surface will be raised until the fantasy particles escape from that
mode. So, the mode on the left has four fantasy
particles and only two data points. So, the effect of the learning is going to be
to raise the energy there. And that energy barrier might be much too
high for a Markov chain to be able to cross, so the mixing rate will be very
slow. But, the learning will actually spill
those red particles out of that energy minimum by raising the minimum.
And we get filled up and the fantasy particles will escape and go off somewhere
else, to some other deep minimum. So, we can get out of minima that the
Markov chain would not be able to get out of, at least, not in a reasonable time.
So, what's going on here is the energy surface is really being used for two
different purposes. The energy surface represents our model,
but it's also being manipulated by the learning algorithm to make the Markov
chain mix faster. Or rather, to have the effect of a
faster-mixing Markov chain. Once the fantasy particles have filled up
one hole, they'll rush off to somewhere else and deal with the next problem.
An analogy for them is that their like investigative journalists who rush in to
investigate some nasty problem. As soon as the publicity has caused that
problem to be fixed, instead of saying, okay, everything is okay now.
They rush off to find the next nasty problem.