[MUSIC] Now, let's actually summarize the entire
gradient descent algorithm for multiple regression. Stepping through, very carefully,
every step of this algorithm. So in particular at first, what we're
gonna do, is we're just gonna initialize all of our different parameters to
be zero at the first iteration. Or you could initialize them randomly or
you could do something a bit smarter. But let's just assume that they're
all initialized to zero and we're gonna start our
iteration counter at one. And then what we're doing is we're
saying while we're not converged, and what was the condition we talked about
before in our simple regression module for not being converged? We said while the gradient of
our residual sum of squares, the magnitude of that gradient
is sufficiently large. Larger than some tolerance epsilon
then we were going to keep going. So what is the magnitude
of residual sum of squares? Here this thing just to be very
explicit is the square root of the square, so what are the elements of
the gradient of residual sum of squares? Well, it's a vector where every element is the partial derivative with
respect to some parameter. I'm gonna refer to that as partial of j,
okay? So, when I take the magnitude
of the vector I multiply the vector times its transposed,
take the square root. That's equivalent to saying I'm
gonna sum up the partial derivative with respect to the first
feature squared plus all the way up to the partial derivative
of the capital Dth feature. Sorry, I guess I should start, the
indexing really starts with zero, squared, and then I take the square root. So if the result of this is greater than epsilon then I'm gonna continue
my gradient descent iterates. If it's less than epsilon
then I'm gonna stop. But let's talk about what
the actual iterates are. Well, for every feature in my
multiple regression model, first thing I'm going to do is I'm going
to calculate this partial derivative, with respect to the jth feature. And I'm going to store that,
because that's going to be useful in both taking the gradient step as well as
monitoring convergence as I wrote here. So this jth partial,
we derived it on the previous slide. It has this formed and then my gradient
step takes that jth coefficient at timed t and subtracts my step size
times that partial derivative. And then once I cycle through
all the features in my model, then I'm gonna increment this t counter. I'm gonna check whether I've
achieved convergence or not. If not I'm gonna loop through,
and I'm gonna do this until this condition, this magnitude of my
gradient is less than epsilon. Okay, so I wanna take a few moments to talk
about this gradient descent algorithm. Because we presented it specifically in
the context of multiple regression, and also for the simple regression case. But this algorithm is really,
really important. It's probably the most widely used
machine learning algorithm out there. And we're gonna see it when
we talk about classification, all the way to talking
about deep learning. So even though we presented this in
the context of multiple regression, this is a really really useful algorithm,
actually an extremely useful algorithm, as the title of this slide shows. [MUSIC]