[MUSIC] Congratulations for getting through
this really challenging course. We covered a lot, a lot of ground
discussing many ideas relating to regression, as well as more general ideas
that are foundational to machine learning. So let's just recap where we've been, and then look ahead to what's
next in a specialization. Okay, I know, I love to start slides
with okay, but that's how it is. So okay, what have we learned
in this regression course? Well in the first module we talked about
something called simple regression. Let's just remember what
simple regression was. This is where we assume that we just had
a single input and we're just fitting a very simple line as a relationship
between the input and the output. Then we discussed what the cost
was of using a given line, and for this we define something we called
the residual sum of squares. Then once we've defined our
residual sum of squares, this allows us to assess the different
fits of different lines. And then with that we can choose which
best fits our specific training data set. In particular, to find the best fitting
line, we talked about our objective being an objective over two variables, which
represented our slope and our intercept. And the specific algorithm that we
went through in this module was gradient descent, which we discussed
was an iterative algorithm that moves in the direction of
the negative gradient. And for convex functions,
converges to the optimum. We then turn to multiple regression. And multiple regression allowed us to fit more complicated relationships between
our single input and our output. We talked about polynomial regression,
seasonality, and lots of different features of our single input that we
could use in a multiple regression model. But then we talked more generically
about incorporating different inputs. Like in our housing applications,
square feet, number of bathrooms, number of bedrooms, lot size, year built,
as well as features of these inputs. And so generically, we wrote our
multiple regression model as follows. So it's just a weighted collection of
our features h of our inputs xi for our ith house. Plus some epsilon term, which is our error representing
the noise in our observations. Then for this case of multiple regression
we have some more complicated relationship between, now,
a whole bunch of features and our output. We talked about how we define
our residual sum of squares. And then, in order to derive both
a closed form solution as well as a gradient descent algorithm, we talked about taking the gradient
of this residual sum of squares. So it's pretty cool how, even when dealing
with a large collection of features, we can derive a closed-form solution
which says, just given all of our data, we just have to compute
the term shown here. And that will give us our estimated set
of coefficients for all of our features. But we also talked about the fact that
this could be computationally intensive. It's operations are cubic and the number of features that we have in
addition to the fact that this matrix, H transpose, H that we might have
to invert, might not be invertible. And so now we know that this is where
ridge regression can be so useful. In the cases where we might not
have a matrix that we can invert, the ridge regression is just a very simple
modification to this closed-form solution, and leads to forms that
are always invertible. And thus allows us to handle cases where
we have lots and lots of features, even more features than
we have observations. But instead of this closed-form
solution which we mentioned could be very computationally intensive,
we talked about a gradient descent algorithm which is an iterative procedure
for solving this optimization objective. Where we can start at any point
in our space, so anywhere. And compute the gradient,
and take these steps. Where, remember,
there was a step size that we had to set. Where is our step size, here it is. And that determined a lot
of the properties of how quickly we converged to the optimal. But we showed that,
in our well-behaved cases, that we would converge to
this optimal solution. And this idea of gradient descent, or
this optimization algorithm, is a very general purpose tool that we're gonna
see again later in this specialization. [MUSIC]