[MUSIC] >> Very good. Now, we learn this abstract concept of squeezing the score into the interval 0,
1. And we call it generalized linear models. It's pretty abstract. Logistic regression is
a specific case of that, where we use what's called the logistic
function to squeeze minus infinity to plus infinity into the interval 0, 1 so we can
predict probabilities for every class. For logistic regression,
the link function we'll use, it's called the logistic function. Sometimes called sigmoid,
sometimes called logit. And it's a slightly scary function over
here which takes the score as input, and says that the score, sorry the output, of a sigmoid is 1
divided by 1 plus e to the minus Score. So it shows up over here. And it's e to the power of something. I learned about that
function as an undergrad. I didn't think it was that
interesting a function, but turns out to be extremely useful. And here we'll see
an example how it's useful. So, at the bottom here I'm plotting
the score, which can range from minus infinity to plus infinity. And let's see what happens
when you take that score and push it through the sigmoid. So, for example, if you take the score
at zero, it actually hits 0.5. Which is cool because this is
exactly what we're hoping for. The score is 0,
the probability should be 0.5. So, actually let's do that explicitly. So, if I compute the sigmoid,
The sigmoid of 0, that is 1 divided by 1 plus e to the 0. And as a little cheat sheet here at
the bottom, e to the 0 is exactly 1. So, that's good. So, this is 1 divided by 1
plus 1 which is equal to 0.5. QED. So, now we have that if it score of
0's input, you get output of 0.5. That's super exciting. So, let's look at the positive
end of the spectrum. You see that the curve keeps going up and
up and up, and eventually it hits 1. So, the score of plus infinity,
which is somewhere out here, turns out to be, sorry,
the sigmoid positivity, turns out to be 1. Which, again, what we wanted. If the score's plus infinity we want for
probability 1. So, let's actually do that. So, let's see what happens
sigmoid of plus infinity. That's 1 over 1 plus e to
the power of minus infinity. And cheat sheet here, e to the minus infinity is equal to 0. And so, this thing here
directly gets you the output 1. Okay, let's go to the other extreme. Each of them, lets look at minus infinity. So, if the score is minus infinity,
as you can see down here, it looks like you hit 0 there,
and that's exactly what you want. If the score is very negative, then you want the probability
that y equals plus 1 to be 0. And we can plug it into here. 1 divided by 1 plus e to the minus
minus infinity is e to the infinity. In the cheat sheet down here,
e to the infinity is infinity. And so,
this is equal to 1 over 1 plus infinity. 1 over 1 plus infinity is 0. Exactly what we'd want. So, the sigmoid has this
property that it goes from 0 to 0.5 to 1 really in the way we want. Now, what really is important
here is the places in between. So, for example, if the score is 2, we'll see that we'll
hit the 0.88 over here. And if the score were minus 2, [COUGH] We have 0.12. It's a symmetric function
that ranges from 0 to 1. And so,
it provides exactly the mapping from minus infinity to infinity to the interval 0, 1. [MUSIC]