1
00:00:00,147 --> 00:00:04,662
[MUSIC]

2
00:00:04,662 --> 00:00:08,064
>> So the simple alternative
approach is gradient descent.

3
00:00:08,064 --> 00:00:12,960
Where, remember that
the gradient descent algorithm,

4
00:00:12,960 --> 00:00:20,314
we just initialize our vector of parameter
somewhere and take these gradient steps.

5
00:00:20,314 --> 00:00:25,146
And eventually, we will converge

6
00:00:25,146 --> 00:00:29,650
to the optimum of this problem.

7
00:00:30,960 --> 00:00:34,580
Okay, so what does this algorithm
look like for multiple regression?

8
00:00:34,580 --> 00:00:38,800
Well, it looks very similar to
our simple linear regression,

9
00:00:38,800 --> 00:00:44,980
where we say while not converged,
we're gonna take our w parameters.

10
00:00:44,980 --> 00:00:52,470
And we're gonna update them by subtracting
sum step size atta times the gradient

11
00:00:52,470 --> 00:00:58,080
of our residual sum of squares,
at our previous set of parameters wt.

12
00:00:58,080 --> 00:01:00,169
So what is our residual sum of squares?

13
00:01:01,880 --> 00:01:04,750
Sorry, the gradient of the residual sum
of squares, I'm writing right here,

14
00:01:04,750 --> 00:01:08,560
so this update is w at iteration t.

15
00:01:08,560 --> 00:01:15,120
The minus sign and this minus
sign will turn into a plus sign.

16
00:01:15,120 --> 00:01:20,424
Two eta times this h matrix,

17
00:01:20,424 --> 00:01:26,841
h transpose y- Hw at iteration t.

18
00:01:26,841 --> 00:01:29,830
And what is this here?

19
00:01:29,830 --> 00:01:35,840
Well, h times w at iteration t is
my predicted set of observations,

20
00:01:35,840 --> 00:01:37,200
the whole vector of them.

21
00:01:38,970 --> 00:01:44,250
Assuming that I use w at iteration
t performing those predictions.

22
00:01:45,990 --> 00:01:50,610
Okay, so what this version of
the algorithm is doing is it's taking our

23
00:01:50,610 --> 00:01:55,076
entire w vector, all the regression
coefficients in our model, and

24
00:01:55,076 --> 00:01:59,316
updating them all at once using
this matrix notation shown here.

25
00:01:59,316 --> 00:02:03,409
[MUSIC]