In the previous video,
I showed how a Boltzmann machine can be
used a probabilistic model of a set of
binary data vectors.
In this video we're finally going to get
around to the Boltzmann machine learning
algorithm.
This is a very simple learning model which
has an elegant theoretical justification,
but it turned out in practice, it was
extremely slow and noisy, and just wasn't
practical.
And for many years, people thought that
Boltzmann machines would never be
practical devices.
Then we found several different ways of
greatly speeding up the learning
algorithm.
And now the algorithm is much more
practical, and has, in fact, been used as
part of the winning entry for a million
dollar machine learning competition, which
I'll talk about in a later video.
The Bolton machine learning algorithm is
an unsupervised learning algorithm.
Unlike the typical user back propagation,
where we have a input vector and we
provide it with a desired output. In
Boltzmann machine learning we just give it
the input vector.
There are q labels. What the algorithm is
trying to do is build a model of a set of
input vectors, though it might be better
to think of them as output vectors.
What we want to do is maximize the product
of the probabilities, that the Boltzmann
machine assigns to a set of binary
vectors,
The ones in the training set.
This is equivalent to maximizing the sum
of the log probabilities that the
Boltzmann machine assigns to the training
vectors.
It's also equivalent to maximizing the
probability that we'd obtain exactly the
end training cases, if we ran the
Boltzmann machine in the following way.
First, we let it settle to its stationary
distribution, and different times, with no
external input.
Then we sample the visible vector once.
Then we let it settle again, and sample
the visible vector again.
And so on.
Now the main reasons why the learning
could be difficult.
This is probably the most important
reason.
If you consider a chain of units,
A chain of hidden units here, with visible
units attached to the two ends,
And if we use a training set that consist
of one, zero and zero, one.
In other words, we want the two visible
units to be in opposite states.
Then the way to achieve that is by making
sure that the product of all those weights
is negative.
So, for example, if all of the weights are
positive, turning on W1 will tend to turn
on the first hidden unit.
And that will tend to turn on the second
hidden unit, and so on.
And the fourth hidden unit will tend to
turn on the other visible unit.
If one of those weights is negative, then
we'll get an anti-correlation between the
two visible units.
What this means is, that if we're thinking
about learning weight W1, we need to know
other weights.
So there's W1.
To know how to change that weight, we need
to know W3.
We need to have information about W3,
because if W3 is negative what we want to
do with W1 is the opposite of what we want
to do with W1 if W3 is positive.
So given that one weight needs to know
about other weights in order to be able to
change even in the right direction, it's
very surprising that there's a very simple
learning algorithm, and that the learning
algorithm only requires local information.
So it turns out that everything that one
weight needs to know about all the other
weights and about the data is contained in
the difference of two correlations.
Another way of saying that is that if you
take the log probability that the
Boltzmann machine assigns to a visible
vector V.
And ask about the derivative of that log
probability with respect to a weight, WIJ.
It's the difference of the expected value
of the products of the states of I and J.
When the networks settle to thermal
equilibrium with v clamped on the visible
units.
That is how often are INJ on together when
V is clamped in visible units and the
network is at thermal equilibrium, minus
the same quantity.
But when V is not clamped on visible
units, so because the derivative of the
log probability of a visible vector is
this simple difference of correlations we
can make the change in the weight be
proportional to the expected product of
the activities average over all visible
vectors in the training set, that's what
we call data.
Minus the product of the same two
activities when your not clamping anything
and the network has reached thermal
equilibrium with no external interference.
So this is a very interesting learning
rule.
The first term in the learning rule says
raise the weights in proportion to the
product of the activities the units have
when you're presenting data.
That's the simplest form of what's known
as a Hebbian learning rule.
Donald Hebb, a long time ago, in the 1940s
or 1950s, suggested that synapses in the
brain might use a rule like that.
But if you just use that rule, the synapse
strengths will keep getting stronger.
The weights will all become very positive,
and the whole system will blow up.
You have to somehow keep things under
control, and this learning algorithm is
keeping things under control by using that
second term.
It's reducing the weights in proportion to
how often those two units are on together,
when you're sampling from the model's
distribution.
You can also think of this as the first
term is like the storage term for a
Hopfield Net.
And the second term is like the term for
getting rid of spurious minima.
And in fact this is the correct way to
think about that.
This rule tells you exactly how much
unlearning to do.
One obvious question is why is the
derivative so simple.
Well, the probability of a global
configuration at thermal equilibrium, that
is once you've let it settle down, is an
exponential function of its energy.
The probability is related to E to the
minus energy.
So when we settle to equilibrium we
achieve a linear relationship between the
log probability and the energy function.
Now, the energy function is linear in the
weights.
So, we have a linear relationship between
the weights and the log probability.
And since we're trying to manipulate log
probabilities by manipulating weights,
that's a good thing to have.
It's a log linear model.
In fact, the relationship's very simple.
It's that the derivative of the energy
with respect to a particular weight WIJ is
just the product of the two activities
that, that weight connects.
So what's happening here?
Is the process of settling to thermal
equilibrium is propagating information
about weights?
We don't need an explicit back propagation
stage.
We do need two stages.
We need to settle with the data.
And we need to settle with no data.
But notice that the networks behaving in
pretty much the same way in those two
phases.
The unit deep within the network is doing
the same thing, just with different
boundary conditions.
With back prop the forward pass and the
backward pass are really rather different.
Another question you could ask is what's
that negative phase for.
I've already said it's like the unlearning
we do in a Hopfield net to get rid of
spurious minima.
But let's look at it in more detail.
The equation for the probability of a
visible vector, is that it's a sum overall
hidden vectors of E to the minus the
energy of that visible and hidden vector
together.
Normalized by the same quantity, summed
overall visible vectors.
So if you look at the top term, what the
first term in the learning rule is doing
is decreasing the energy of terms in that
sum that are already large and it finds
those terms by settling to thermal
equilibrium with the vector V clamped so
that it can find an H that goes nicely
with V, that is gives a nice low energy
with V.
Having sampled those vectors H, it then
changes the weights to make that energy
even lower.
The second phase in the learning, the
negative phase, is doing the same thing,
but for the partition function.
That is, the normalizing term on the
bottom line.
It's finding global configurations,
combinations of visible and hidden states
that give low energy,
And therefore, are large contributors to
the partition function.
And having find those global
configurations, it tries to raise their
energy so that the can contribute less.
So the first term is making the top line
big, and the second term is making the
bottom line small.
Now in order to run this learning rule,
you need to collect those statistics.
You need to collect what we call the
positive statistics, those are the ones
when you have data clamped on the visible
units, and also the negative statistics,
those are the ones when you don't have
data clamped and that you're going to use
for unlearning.
An inefficient way to track these
statistics was suggested by me and Terry
Sejnowski in 1983.
And the idea is, in the positive phase you
clamp a data vector on the visible units,
you set the hidden units to random binary
states,
And then you keep updating the hidden
units in the network, one unit at a time,
until the network reaches thermal
equilibrium at a temperature of one.
We actually did that by starting at a high
temperature and reducing it, but that's
not the main point here.
And then once you reach thermal
equilibrium, you sample how often two
units are on together.
So you're measuring the correlation of INJ
with that visible vector clamped.
You then repeat that, over all the visible
vectors, so that, that correlation you're
sampling is averaged over all the data.
Then in the negative phase, you don't
clamp anything.
The network is free from external
interference.
So, you set all of the units, both visible
and hidden, to random binary states.
And then you update the units, one at a
time, until the network reaches thermal
equilibrium, at a temperature of one.
Just like you did in the positive phase.
And again, you sample the correlation of
every pair of units INJ,
And you repeat that many times.
Now it's very difficult to know how many
times you need to repeat it, but certainly
in the negative phase you expect the
energy landscape to have many different
minima, but are fairly separated and have
about the same energy.
The reason you expect that is we're going
to be using Boltzmann machines to do
things like model a set of images.
And you expect there to be reasonable
images, all of which have about the same
energy.
And then very unreasonable images, which
have much higher energy.
And so you expect a small fraction of the
space to be these low energy states.
And a very large fraction of the space to
be these bad high energy states.
If you have multiple modes, it's very
unclear how many times you need to repeat
this process to be able to sample those
modes.