[MUSIC]
So the first approach that we're gonna talk about is just a closed form solution
where we take our gradient, and simply set it equal to zero, and solve for w.
Just like we did in the simple regression case.
Okay, here's my gradient of our residual sum of squares, and
I've set it equal to zero.
And now let's solve for w.
So, let's do out this multiplication here where we're gonna get -2H transpose, y.
And then we're gonna get a +2H transpose,
H times w, and we're setting this equal to zero.
And so these matrix multiplies act very similarly to if these had been scalars,
but we have to keep the order.
The order is very important.
We couldn't have switched it around to be y h transposed, okay.
So, now let's solve this equation.
First of all, the two cancels out, we can just divide both sides by two.
And so what I'm gonna end up with is h transpose
hw equals, I bring this to the other side, h transpose y.
Then I'm gonna multiply both sides by h transpose h inverse.
So H transpose Hw and now I'm gonna multiple,
pre-multiply the other said by the same.
H transpose H inverse, H transpose y and so this is a little aside.
What is a matrix A inverse times the same matrix a.
Well, that's the definition of,
so, by definition of a matrix inverse that is the identity matrix.
And another aside, so that's aside number one another aside if I take
a the identity matrix and multiply by any vector,
so I'll call the vector v, I just get v back.
Or if I take the matrix and multiply it by any matrix, big V,
I'm gonna get that matrix back.
Okay, so what we see if we apply these two identities here is that together,
these two terms, h transpose h is like our big A matrix here.
So we have a matrix times it's inverse, this is just gonna be the identity.
Then we have the identity matrix times a vector w,
that's just gonna be the w vector.
So we have w, and let me put hats on, because remember,
that once we set this equal to zero and solve for w this is our
estimated set of coefficients so I'm going to put a hat on w and
what we see is w hat is simply equal to h transpose h inverse HTy.
So this is one of those aha moments.
Maybe you missed it.
Cuz aha moment is, we have a whole collection of different parameters,
w0 all the way up to wd.
These are the things multiplying all the features we're using in our multiple
regression model.
And in one line I've written the solution to the fit.
I've fit all of w 0, w 1, all the way up to D just by doing this matrix multiply.
Okay?
So this.
Motivates why we went through all this work to write things in this matrix
notation because it allows me to have this nice closed form solution for
all my parameters written very compactly.
[MUSIC]