[MUSIC]
>> Very good.
Now, we learn this abstract concept of
squeezing the score into the interval 0, 1.
And we call it generalized linear models.
It's pretty abstract.
Logistic regression is a specific case of that,
where we use what's called the logistic function to squeeze minus infinity to plus
infinity into the interval 0, 1 so we can predict probabilities for every class.
For logistic regression, the link function we'll use,
it's called the logistic function.
Sometimes called sigmoid, sometimes called logit.
And it's a slightly scary function over here which takes the score as input,
and says that the score,
sorry the output, of a sigmoid is 1 divided by 1 plus e to the minus Score.
So it shows up over here.
And it's e to the power of something.
I learned about that function as an undergrad.
I didn't think it was that interesting a function,
but turns out to be extremely useful.
And here we'll see an example how it's useful.
So, at the bottom here I'm plotting the score, which can range from minus
infinity to plus infinity.
And let's see what happens when you take that score and
push it through the sigmoid.
So, for example, if you take the score at zero, it actually hits 0.5.
Which is cool because this is exactly what we're hoping for.
The score is 0, the probability should be 0.5.
So, actually let's do that explicitly.
So, if I compute the sigmoid, The sigmoid of 0,
that is 1 divided by 1 plus e to the 0.
And as a little cheat sheet here at the bottom, e to the 0 is exactly 1.
So, that's good.
So, this is 1 divided by 1 plus 1 which is equal to 0.5.
QED.
So, now we have that if it score of 0's input, you get output of 0.5.
That's super exciting.
So, let's look at the positive end of the spectrum.
You see that the curve keeps going up and up and up, and eventually it hits 1.
So, the score of plus infinity, which is somewhere out here,
turns out to be, sorry, the sigmoid positivity, turns out to be 1.
Which, again, what we wanted.
If the score's plus infinity we want for probability 1.
So, let's actually do that.
So, let's see what happens sigmoid of plus infinity.
That's 1 over 1 plus e to the power of minus infinity.
And cheat sheet here,
e to the minus infinity is equal to 0.
And so, this thing here directly gets you the output 1.
Okay, let's go to the other extreme.
Each of them, lets look at minus infinity.
So, if the score is minus infinity, as you can see down here,
it looks like you hit 0 there, and that's exactly what you want.
If the score is very negative,
then you want the probability that y equals plus 1 to be 0.
And we can plug it into here.
1 divided by 1 plus e to the minus minus infinity is e to the infinity.
In the cheat sheet down here, e to the infinity is infinity.
And so, this is equal to 1 over 1 plus infinity.
1 over 1 plus infinity is 0.
Exactly what we'd want.
So, the sigmoid has this property that it goes from
0 to 0.5 to 1 really in the way we want.
Now, what really is important here is the places in between.
So, for example, if the score is 2,
we'll see that we'll hit the 0.88 over here.
And if the score were minus 2,
[COUGH] We have 0.12.
It's a symmetric function that ranges from 0 to 1.
And so, it provides exactly the mapping from minus
infinity to infinity to the interval 0, 1.
[MUSIC]