[MUSIC]
Okay, so now we're onto the final important step of the derivation,
which is taking the gradient.
Because as we saw in the simple regression case, the gradient was important both for
our closed form solution as well as, of course, for
the gradient descent algorithm.
So what's the gradient of our residual sum of squares in this multiple
regression case?
Well, it's the gradient of this matrix notation that we use for
representing the residual sum of squares.
And if you know gradients of vectors and matrices,
which we're not assuming you do, so please don't think that you need to know this,
but the result is -2H transpose, so
taking that big grain matrix and turning it on its side, times y-Hw,
which again is that vector, of residuals.
And why is this the result?
Well, I'm not gonna give a complete proof of this.
I'm just gonna give some motivation.
I'm going to walk through an analogy to 1D case, and we'll see some patterns, and
maybe you'll believe that that's the result of the matrix case.
So, in particular, if we think about taking the derivative with respect to w
of a function that is y-hw times
y-hw where these things are all scalars.
So this is the 1D analog to this equation here,
where the gradient is just this derivative of this one parameter w.
That arrow is not quite pointing to w.
Well what's the derivative of this?
It's equivalent to the derivative with respect to w of Y minus hw squared.
And, like we've done multiple times in this course now,
when I take the derivative with respect to w of some function raised to the power,
by the chain rule, I bring that power down.
Then I'm gonna multiply by the function Hw raised to the power minus 1.
And then I'm gonna take the derivative of the inside.
And what's the derivative of this function with respect to w?
It's minus h.
And so the result here is -2h(y-Hw).
So we have the -2 in both cases,
this little scalar H is this big matrix in our case,
and y- Hw in the scalar case, this big vector matrix notation here.
Okay, so just believe that this is the gradient.
We didn't wanna bog you down in too much linear algebra, or
too much in terms of derivatives.
But if we have this notation, then we can derive everything we need to for
our two different solutions to fitting this model.
[MUSIC]