1
00:00:00,000 --> 00:00:04,424
[MUSIC]

2
00:00:04,424 --> 00:00:09,114
Okay, so let's take a moment to compare
the two approaches that we've gone over,

3
00:00:09,114 --> 00:00:13,760
either setting the gradient equal
to zero or doing gradient descent.

4
00:00:13,760 --> 00:00:16,840
Well, in the case of minimizing
residual sum of squares,

5
00:00:16,840 --> 00:00:20,660
we showed that both were
fairly straight forward to do.

6
00:00:20,660 --> 00:00:24,160
But in a lot of the machine learning
method's that we're interested in

7
00:00:24,160 --> 00:00:26,240
taking the gradient and
setting it equal to zero,

8
00:00:26,240 --> 00:00:30,410
well there's just no close
form solution to that problem.

9
00:00:30,410 --> 00:00:35,280
So, often we have to turn to
method's like gradient descent.

10
00:00:35,280 --> 00:00:41,421
And likewise, as we're gonna see in the
next module, where we turn to having lots

11
00:00:41,421 --> 00:00:46,680
of different inputs, lots of
different features in our regression.

12
00:00:46,680 --> 00:00:51,474
Even though there might be a close form
solution to setting the gradient equal to

13
00:00:51,474 --> 00:00:54,979
zero, sometimes in practice
it can be much more efficient

14
00:00:54,979 --> 00:00:58,860
computationally to implement
the gradient descent approach.

15
00:01:00,410 --> 00:01:04,080
And finally one thing that I should
mention about the gradient descent

16
00:01:04,080 --> 00:01:08,220
approach is the fact that in that case,
we had to choose a stepsize and

17
00:01:08,220 --> 00:01:09,470
a convergence criteria.

18
00:01:10,830 --> 00:01:13,090
Whereas, of course,
if we take the gradient and

19
00:01:13,090 --> 00:01:17,350
are able to set it to zero, we don't
have to make any of those choices.

20
00:01:17,350 --> 00:01:20,140
So that is a downside to
the gradient descent approach,

21
00:01:20,140 --> 00:01:24,820
is having to specify these
parameters of the algorithm.

22
00:01:26,340 --> 00:01:31,257
But sometimes we're relying on these
types of optimization algorithms to

23
00:01:31,257 --> 00:01:33,685
solve our optimization objective.

24
00:01:33,685 --> 00:01:38,309
[MUSIC]