Now that we have the preliminaries out the
way, we can get back to the central issue,
which is how to learn multiple layers of
features.
So in this video, I'm finally going to
describe the back propagation algorithm
which was the main advance in the 1980s
that led to an explosion of interest in
neural networks.
Before I describe back propagation, I'm
going to describe another very obvious
algorithm that does not work nearly as
well, but is something that many people
think of.
Now that we know how to learn the weights
of the logistic units, we're going to
return to the central issue, which is how
to learn the weights of hidden units.
If you have neural networks without hidden
units, they are very limited in the
mappings they can model.
If you add a layer of hand coded features
as in a perceptron, you make the net much
more powerful but the difficult bit for a
new task is designing the features.
The learning won't solve the hard problem;
you have to solve it by hand.
What we'd like is a way of finding good
features without requiring insights into
the tasks or repeated trial and error,
where we guess some features and see how
well they work.
In effect, what we need to do is automate
the loop of designing features for a task
and seeing how well they work.
We'd like the computer to do that loop,
instead of having a person in that loop.
So the thing that occurs to everybody who
knows about evolution is to learn by
perturbing the weights.
You randomly perturb one weight.
So that's meant to be like a mutation, and
you see if it improves performance.
And if it improves performance of the net,
you save that change in the weight.
You can think of this as a form of
reinforcement learning.
Your action consists of making a small
change.
And then you check whether that pays off,
and if it does, you decide to perform that
action.
The problem is it's very inefficient.
Just to decide whether to change one
weight, we need to do multiple forward
passes on a representative set of training
cases.
We have to see if changing that weight
improves things, and you can't judge that
by one training case alone.
Relative to this method of randomly
changing weight, and seeing if it helps,
back propagation is much more efficient.
It's actually more efficient by a factor
of the number of weights in the network,
which could be millions.
An additional problem with randomly
changing weights and seeing if it helps is
that towards the end of learning, any
large change in weight will nearly always
make things worse, because the weights
have to have the right relative values to
work properly.
So towards the end of learning not only do
you have to do a lot of work to decide
whether each of these changes helps but
the changes themselves have to be very
small.
There are slightly better ways of using
perturbations in order to learn.
One thing we might try is to perturb all
the weights in parallel and then correlate
the performance gain with the weight
changes.
That actually doesn't really help at all.
The problem is that we need to do lots and
lots of trials with different random
perturbation of all the weights, in order
to see the effect of changing one weight,
through the noise created by changing all
the other weights.
So it doesn't help to do it all in
parallel.
Something that does help, is to randomly
perturb the activities of the hidden
units, instead of perturbing the weight.
Once you've decided that perturbing the
activity of a hidden unit on a particular
training case is going to make things
better.
You can then compute how to change the
weights.
Since there's many fewer activities than
weights, there's less things that you're
randomly exploring.
And this makes the algorithm more
efficient.
But it's still much less efficient than
backpropagation.
Backpropagation still wins by a factor of
the number of neurons.
So the idea behind back propagation is
that we don't know what the hidden units
ought to be doing.
They're called hidden units because
nobody's telling us what their states
ought to be.
But we can compute how fast the error
changes as we change a hidden activity on
a particular training case.
So instead of using activities of the
hidden units as our desired states, we use
the error derivatives with respect to our
activities.
Since each hidden unit can affect many
different output units, it can have many
different effects on the overall error if
we have many output units.
These affects have to be combined.
And we can do that efficiently.
So that allows us to compute error
derivatives for all of the hidden units
efficiently at the same time.
Once we've got those error derivatives for
the hidden units, that is, we know how
fast the error changes as we changed the
hidden activity on that particular
training case, it's easy to convert those
error derivatives for the activities into
error derivatives for the weights coming
into a hidden unit.
So here's a sketch of how backpropagation
works, for a single training case.
First we have to define the error, and
here we'll use the error being the square
difference between the target values of
the output unit J and the actual value
that the net produces for the output unit
J.
And we're gonna imagine there are several
output units in this case.
We differentiate that, and we get a
familiar expression for how the error
changes as you change the activity of an
output unit J.
And I'll use a notation here where the
index on a unit will tell you which layer
it's in.
So the output layer has a typical index of
J, and the layer in front of that, the
hidden layer below it in the diagram, will
have a typical index of I.
And I won't bother to say which layer
we're in because the index will tell you.
So once we've got the aeroderivative with
respect to the output of one of these
output units, we then want to use all
those aeroderivatives in the output layer
to compute the same quantity in the hidden
layer that comes before the output layer.
So back propagation, the core of back
propagation is taking error derivatives in
one layer and from them computing the
error derivatives in the layer that comes
before that.
So we want to compute DE by DY, I.
Now obviously, when we change the output
of unit I, it'll change the activities of
all three of those output units, and so we
have to sum up all those effects.
So we're going to have an algorithm that
takes error derivatives we've already
computed for the top layer here.
And combines them using the same weights
as we use in the forward pass to get error
derivatives in the layer below.
So, this slide is going to explain the
backpropagation algorithm.
And you really need to understand this
slide.
And the first time you see it, you may
have to study it for a long time.
This is how you backpropagate the error
derivative with respect to the output of a
unit.
So we'll consider an output unit J on a
hidden unit I.
The output of the hidden unit I will be
YI.
The output of the output unit J will be
YJ.
And the total input received by the output
unit J will be ZJ.
The first thing we need to do is convert
the error derivative with respect to Y J,
into an error derivative with respect to Z
J.
To do that we use the chain rule.
So we say DE by DZJ, equals DYJ by DZJ,
times DE by DYJ.
And af, as we've seen before, when we were
looking at logistic units, that's just YJ
into one minus YJ times the error
derivative with respect to the output of
unit J.
So now we've got the error derivative with
respect to the total input received by
unit J.
Now we can compute the error derivative
with respect to the output of unit I.
It's going to be the sum over all of the
three outgoing connections of unit I, of
this quantity, DZJ by DYI times DE by DZJ.
So the first term there is how the total
input to unit J changes as we change the
output of unit I.
And then we have to multiply that by how
the error root of changes as we change the
total input to unit J which we computed on
the line above.
And as we saw before when studying the
logistic unit dzj by dyi is just the
weight on the connection wij.
So what we get is that the error
derivative.
We respect to the output of unit I is the
sum over all the outgoing connections to
the layer above of the weight wij on that
connection times a quantity we would have
already computed which is de by dzj for
the layer above.
And so you can see the computation looks
very like what we do on the forward pass,
but we're going in the other direction.
What we do for each unit in that hidden
layer that contains I, is we compute the
sum of a quantity in the layer above the
weights on the connections.
Once we've got to E by DZJ, which we
computed on the first line here, it's very
easy to get the error derivatives for all
the weights coming into unit J.
To E by DWIJ is simply D, E, by DZJ, which
we computed already, times how ZJ changes.
As we change the weight on the connection.
And that's simply the activity of the unit
in the layer below YI.
So the rule for changing the weight is
just you multiply, this quantity you've
computed at a unit, to E by DZJ, by the
activity coming in from the layer below.
And that gives you the error of derivative
with respect to weight.
So on this slide we have seen how we can
stop with DE by DYJ and back propagate to
get DE by DYI we'll come backwards through
one layer and computed the same quantity
the derivative of the error with respect
to the output in the previous layer.
So we can clearly do that for as many
layers as we like.
And after we've done that for all these
layers, we can compute how the error
changes as you change the weights on the
connections.
That's the backpropagation algorithm.
It's an algorithm for taking one training
case, and computing, efficiently, for
every weight in the network, how the error
will change as, on that particular
training case, as you change the weight.