In this video, I'm going to explain how adding noise can help systems escape from
local minima. And, I'm going to show what you have to do
to the units in Hopfield net to add noise in the appropriate way.
I'm not going to introduce the idea that we confined better minima by using noise.
So, Hopfield net always makes decisions that reduce the energy, or if it doesn't
state of the unit, the energy stays the same.
This makes it impossible to climb out of a local minimum.
So, if you look at the landscape here. If we get into the local minimum A,
there's no way we're going to get over the energy barrier to get to the better
minimum B because we can't go uphill in energy.
If we add random noise, we can escape from poor minima, especially minima that is
shallow, that is, ones that don't have big energy barriers around them.
It turns out, rather than using a fixed noise level, the most effective strategy
is to start with a lot of noise which allows you to explore the space on a
coarse scale and find the generally good regions of the space, and then to decrease
the noise level. With a lot of noise, you can cross big
barriers. As you decrease the noise level, you start
concentrating on the best nearby minima. If you slowly reduce the noise, so the
system ends up in a deep minimum, that's called simulated annealing.
And this ideal was, propogated by Kirkpatrick at around the same time as
Hopfield nets were proposed. So, the reason for simulated annealing is
because the temperature, in a physical system, or in a simulated system with a
energy function, Affects the transition probabilities.
So, in a high temperature system, the probability of going uphill from B to A is
lower than the probability of going downhill from A to B.
But it's not much lower. In effect, the temperature flattens the
energy landscape, and so the little black dots are meant to be particles.
And what we are imagining is particles moving about according to the transition
probabilities that you get with an energy function and a temperature.
And this might be a typical distribution if you're on the system of high
temperature where it's easier to cross barriers, but it's also hard to stay in a
deep minimum once you've got that. If you are in the system of much lower
temperature, Then your probability of crossing barriers
gets much smaller but your ratio gets much better.
So, the ratio of the probability of going from A to B versus the probability of
going from B to A is much better in the low temperature system.
And so, if we run it long enough, we would expect all of the particles to end up in
B. But if we just run it for a long time at
low temperature, it will take a very long time for particles to escape from A.
And it turns out a good compromise is to start at a high temperature and gradually
reduce the temperature. The way we get noise in to Hopfield net is
to replace the binary threshold units by binary stochastic units and make biased
random decisions. And the amount of noise is controlled by
something called temperature, Which you'll see in a minute in the
equation. Raising the noise level is equivalent to
decreasing all the energy gaps between configurations.
So, this is our normal logistic equation. But with the energy gap scaled by a
temperature. If the temperature is very high, that
exponent will be roughly zero, so the right hand side will be one over one plus
one. And so, the probability of the unit turning on will be about a half.
It'll be in it's on and off states, more or less equally off.
As we lower the temperature, Depending on the sign of delta E, the unit
will become either more and more firmly on and more and more firmly off.
At zero temperature, which is what we're be using in a Hopfield net,
Then the sign of delta E determines whether the right hand side goes to zero
or goes to one. But, with T zero, it will either be zero
or one on the right hand side. And so, the unit will behave
deterministically and that's a binary threshold unit.
It will always adopt whatever of the two states is the lowest energy.
So, the energy gap we saw on a previous slide, and it's just the difference in the
energy of the whole system depending on whether unit I is off, or the unit I is
on. Although simulated annealing is a very
powerful method for improving searches that get stuck in local optima, and
although it was influential in leading Terry Sejnowski and I to the ideas behind
Boltzmann machines, it's actually a big distraction from understanding Boltzmann
machines. So, I'm not going to talk about it anymore
in this course even though it's a very interesting idea.
And, from now on, I'm going to use binary stochastic units that have a temperature
of one. That is, it's the standard logistic
function in the energy gap. So, one concept that you need to
understand in order to understand the learning procedure for both the machines,
is the concept of thermal equilibrium. And because we're setting the temperature
to one, this the concept of thermal equilibrium at a fix temperature.
It's a difficult concept. Most people think that it means the system is settled
down and isn't changing anymore. That's normally what equilibrium means. But it's
not the states of the individual units that are settled down.
The individual units are still rattling around at thermal equilibrium, and less
temperature zero. The thing that settles down is the probability distribution over
configurations. That's a difficult concept the first time you meet it, and so I'm
going to give you an example. The probability distribution settles to a
particular distribution called the Stationary Distribution.
The stationary distribution is determined by the energy function of the system.
And, in fact, in the stationary distribution, the probability of any
configuration is proportional to each of the minus its energy.
A nice intuitive way to think about thermal equilibrium is to imagine a huge
ensemble of identical systems that all have exactly the same energy function.
So, imagine a very large number of stochastic Hopfield nets all with the same
weights. Now, in that huge ensemble, we can define
the probability of configuration as the fraction of the systems that are in that
configuration. So, now we can understand what's happening
as we approach thermal equilibrium. We can start with any distribution we like
over all these identical systems. We could make them all, be in the same
configuration. So, that's the distribution with a property of one on one
configuration, and zero on everything else. Or we could start them off, with an
equal number of systems in each possible configuration.
So that's a uniform distribution. And then, we're going to keep applying our
stochastic update rule. Which, in the case of a stochastic
Hopfield net would mean, You pick a unit, and you look at its
energy gap. And you make a random decision based on
that energy gap about whether to turn it on or turn it off.
Then, you go and pick another unit, and so on.
We keep applying that stochastic rule. And after we've run systems stochastically
in this way, We may eventually reach a situation where
the fraction of the systems in each configuration remains constant.
In fact, that's what will happen if we have symmetric connections.
That's the stationary distribution that physicists call thermal equilibrium.
Any given system keeps changing its configuration.
We apply the update rule, And the states of its units will keep
flipping between zero and one. But, the fraction of systems in any
particular configuration doesn't change. And that's because we have many, many more
systems than we have configurations. So, here's an analogy kust to help with
the concept. Imagine a very large casino in Las Vegas
with lots of card dealers. And, in fact, we have many more than 52 factorial card
dealers. We start with all the card packs in the standard order that they come from
the manufacturer. Let's suppose that has the ace of spades, and the king of spades,
and the queen of spades. And then, the dealers all start shuffling.
And they do random shuffles, they don't do fancy shuffles that bring them back to the
same order again. After a few shuffles, there's still a good
chance that the king of spades will be next to the queen of spades in any given
pack. So, the packs have not yet forgotten where
they started. Their initial order is still influencing
their current order. If we keep shuffling, eventually the
initial order will be irrelevant. The packs will have forgotten where they
started. And, in fact, in this example, there will
be an equal number of packs in each of the 52 factorial possible orders.
Once this has happened, if we carry on shuffling,
There'll still be an equal number of packs in each of the 52 factorial orders.
That's why it's called equilibrium. It's because the fraction in any one
configuration doesn't change, Even though the individual systems are
still changing. The thing that's wrong with this analogy
is that once we've each equilibrium here, all configurations have equal energy.
And so, they all have the same probability.
In general, we're interested in reaching equilibrium for systems where some
configurations have lower energy than others.