In this video, I'm going to talk about the back propagation through time algorithm.
It's the standard way to train or recurrence your own network.
The algorithm is really quite simple once you have seen the equivalents between a
recurrent neural network and a feed forward neural network that has one layer
for each time step. I'll also talk about ways of providing
input, and desired outputs, to recurrent neural networks.
So the diagram shows a simple recurrent net with three interconnected neurons.
We're going to assume there's a time delay of one in using each of those connections
and that the network runs in discrete time, so the clock that has integer ticks.
The key to understanding how to train a recurrent network is to see that a
recurrent network is really just the same as a feed forward network, where you've
expanded the recurrent network in time. So the recurrent network starts off in
some initial state. Shown at the bottom there, times zero.
And then uses the way some of these connections to get a new state, shown at
time one. You then uses the same weights again to
get another new state, and it uses the same weights again to get another new
state and so on. So it's really just a lead feed forward
network, where the weight is a constraint to be the same at every layer.
Now backprop is good at learning when there are weight constraints.
We saw this for convolutional nets and just to remind you, we can actually
incorporate any linear constraint quite easily in backprop. So we compute the
gradients as usual, as if the weights were not constrained.
And then we modify the gradients, so that we maintain the constraints.
So if we want W1 to equal W2, we start off with an equal and then we need to make
sure that the changing W1 is equal to the changing W2.
And we do that by simply taking the derivative of the area with respect to W1,
the derivative with respect to W2, and adding or averaging them, and then
applying the same quantity for updating both W1 and W2.
So if the weights started off satisfying the constraints they'll continue to
satisfy the constraints. The backpropagation through time algorithm
is just the name for what happens when you think of a recurrent net as a lead feet
forward net with shared weights, and you train it with backpropagation.
So, we can think of that algorithm in the time domain.
The forward pass builds up a stack of activities at each time slice.
And the backward pass peels activities off that stack and computes error derivatives
each time step backwards. That's why it's called back propagation
through time. After the backward pass we can add
together the derivatives at all the different time step for each particular
weight. And then change all the copies of that
weight by the same amount which is proportional to the sum or average of all
those derivatives. There is an irritating extra issue.
If we don't specify the initial state of the all the units, for example, if some of
them are hidden or output units, then we have to start them off in some particular
state. We could just fix those initial states to
have some default value like 0.5, but that might make the system work not quite as
well as it would otherwise work if it had some more sensible initial value.
So we can actually learn the initial states.
We treat them like parameters rather than activities and we learn them the same way
as learned the weights. We start off with an initial random guess
for the initial states. That is the initial states of all the
units that aren't input units And then at the end of each training sequence we back
propagate through time all the way back to the initial states.
And that gives us the gradient of the error function with respects to the
initial state. We then just, adjust the initial states by
following, that gradient. We go downhill in the gradient, and that
gives us new initial states that are slightly different.
There's many ways in which we can provide the input to a recurrent neural net.
We could, for example, specify the initial state of all the units.
That's the most natural thing to do when we think of a recurrent net, like a feed
forward net with constrained weights. We could specify the initial state of just
a subset of the units or we can specify the states at every time stamp of the
subset of the units and that's probably the most natural way to input sequential
data. Similarly, there's many way we can specify
targets for a recurrent network. When we think of it as feed forward
network with constrained weights, the natural thing to do is to specify the
desired final states for all of the units. If we're trying to train it to settle to
some attractor, we might want to specify the desired states not just for the final
time steps but for several time steps. That will cause it to actually settle down
there, rather than passing through some state and going off somewhere else.
So by specifying several states of the end, we can force it to learn attractors
and it's quite easy as we back propagate to add in derivatives that we get from
each time stamp. So the back propegation starts at the top,
with the derivatives for the final time stamp.
And then as we go back through the line before the top we add in the derivatives
for that man, and so on. So it's really very little extra effort to
have derivatives at many different layers. Or we could specify the design activity of
a subset of units which we might think of as output units.
And that's a very natural way to train a recurrent neural network that is meant to
be providing a continuous output.