[MUSIC] We've now seen the basics of decision
trees, which are an amazing type of classifier than can be used for
when the range of different types of data. However, decision trees
are highly prone to overfitting. And so, let's dig in a little bit in this
module on how we can avoid overfitting in the context of decision trees. And as a reminder, we're going to continue
to use our loan application evaluation system as a running example
where loan data will come in and we'll be able to predict whether that's
a safe loan or a risky loan application. And so, that's the decision
making we're trying to make. And from that loan application,
we're going to learn the decision tree that allows us to traverse
down the tree and make a prediction as to whether
a particular loan is risky or safe. And so, the input is going to be xi and the output is going to be this y hat i
that we're going to predict from data. Let's first, spend a quick minute
reviewing overfitting and then dig in as to how it happens in decision trees, which
hint, hint is going to be really bad. As we all recall, overfitting is the fact
that separates the training error with then goes down to zero as we make our
models more and more complex and the true error, which goes down with the complexity
of the model, but then spikes backup. And so more specifically, overfitting
happens when we end up model w hat, which has low training_error,
but high true_error. But there was some other model or
model parameters, w*, which had maybe higher training_error,
but definitely lower true_error. And so, that's the overfitting problem. And when somehow, pick a model that's less
complex to avoid that kind of overfitting. So we saw this effect in logistic
regression quite pronouncedly where as we increase the degree of the polynomial,
we got this crazier and crazier decision boundaries where we saw overfitting,
which was bad overfitting over here. But overfitting for polynomials of
degree six and then polynomials of degree 20 here for features,
this is a technical term that I use. I think I use crazy decision boundary,
but let's call it crazy overfitting. So really, bad stuff. And so, we're trying to
avoid overly complex models. And as well see with decision trees, models can get overly
complex very quickly. [MUSIC]