1
00:00:00,000 --> 00:00:04,637
[MUSIC]

2
00:00:04,637 --> 00:00:09,060
We're going to start by defining this idea
of precision a little bit more formally.

3
00:00:09,060 --> 00:00:16,130
So precision is, after showed my website,
which fraction were actually positive?

4
00:00:16,130 --> 00:00:21,920
In general, it's the fraction of positive
predictions that are actually positive.

5
00:00:21,920 --> 00:00:27,150
Precision then is the fraction of positive
predictions that were actually positive.

6
00:00:27,150 --> 00:00:31,030
So let's say that my algorithm predicted
that the six sentences were positive.

7
00:00:31,030 --> 00:00:35,310
So it predicted y hat = + 1 for
the six sentences.

8
00:00:35,310 --> 00:00:39,950
But in reality, only four out of
those six were truly positive.

9
00:00:39,950 --> 00:00:46,110
So we got four truly positive ones,
and two false positives in the mix.

10
00:00:46,110 --> 00:00:48,500
So it's precision was four, six.

11
00:00:50,430 --> 00:00:55,710
So in general, we have a set of
data points we're calling positive,

12
00:00:55,710 --> 00:00:58,360
that we're predicting
to be positive y hat.

13
00:00:58,360 --> 00:01:02,710
In this case y = + 1,
some of them are truly positive.

14
00:01:02,710 --> 00:01:05,190
The yi is + 1.

15
00:01:05,190 --> 00:01:09,320
But some of them were actually not
positive so that yi was actually- 1.

16
00:01:09,320 --> 00:01:10,480
And the question is,

17
00:01:10,480 --> 00:01:13,840
how big a fraction of those are the ones
that actually truly positive?

18
00:01:16,300 --> 00:01:20,460
So here is where we can review that notion
of true positives and true negatives.

19
00:01:20,460 --> 00:01:24,975
And so we can look at this table where,
when in the rows we have the true label,

20
00:01:24,975 --> 00:01:25,550
yi.

21
00:01:25,550 --> 00:01:29,568
While in the columns we have
the predicted label, y hat.

22
00:01:29,568 --> 00:01:33,580
And so if the truth is positive and

23
00:01:33,580 --> 00:01:36,370
the predictor is positive,
we call that a true positive.

24
00:01:36,370 --> 00:01:37,710
It was positive and it was true.

25
00:01:38,740 --> 00:01:46,220
If the true label is- 1 and the prediction
is- 1, we call that true negative.

26
00:01:46,220 --> 00:01:48,980
It was negative, and
we predicted negative,- 1.

27
00:01:48,980 --> 00:01:51,430
So both of those are correct.

28
00:01:51,430 --> 00:01:53,750
But there are two types
of mistakes you can make.

29
00:01:53,750 --> 00:01:57,170
The first type,
those are called false negatives.

30
00:01:57,170 --> 00:02:01,390
It was truly a positive review, but
we predicted it to be negative.

31
00:02:01,390 --> 00:02:04,870
So yi was + 1, y hat was- 1.

32
00:02:04,870 --> 00:02:10,100
And finally, a false positive
is one where the true label

33
00:02:10,100 --> 00:02:13,150
was- 1, but the prediction was + 1.

34
00:02:13,150 --> 00:02:17,460
So the truth was negative would predict
that it's going to be positive, so

35
00:02:17,460 --> 00:02:19,140
it's false positive.

36
00:02:20,530 --> 00:02:23,650
I find it very helpful to ground
these ideas of false positive and

37
00:02:23,650 --> 00:02:29,080
false negative in the context of
an example, to really feel it and

38
00:02:29,080 --> 00:02:32,960
really understand what the impact
of those mistakes can be.

39
00:02:32,960 --> 00:02:35,730
So let's look at this matrix here again.

40
00:02:35,730 --> 00:02:39,050
If you look at the top left, we have

41
00:02:39,050 --> 00:02:43,990
a truly positive sentence,
so it was a plus 1 sentence.

42
00:02:45,290 --> 00:02:48,730
And we got it right,
we had a + 1 prediction.

43
00:02:49,890 --> 00:02:53,350
So that's no mistake, that's great.

44
00:02:53,350 --> 00:02:55,440
Similarly for the bottom right,
we didn't make a mistake.

45
00:02:55,440 --> 00:02:58,730
We had a- 1 sentence,

46
00:02:58,730 --> 00:03:03,320
so a negative sentence, and
we made a negative prediction.

47
00:03:04,780 --> 00:03:07,510
Now the problematic was, I did only two.

48
00:03:07,510 --> 00:03:10,880
So let's look first at the top one.

49
00:03:10,880 --> 00:03:16,890
So what happened here, was that I had
a positive sentence, but a- 1 prediction.

50
00:03:18,150 --> 00:03:20,020
So, what does this actually means?

51
00:03:20,020 --> 00:03:21,700
Those are positive sentence in the word.

52
00:03:21,700 --> 00:03:23,570
Did they fall with negative?

53
00:03:23,570 --> 00:03:25,820
So, I missed the sentences
to show my website.

54
00:03:27,490 --> 00:03:30,980
Maybe this is not too bad, you know, there
might be some positive things I've said.

55
00:03:30,980 --> 00:03:34,070
So, maybe missing one is not that bad but
it's still a problem.

56
00:03:35,430 --> 00:03:37,400
But let's look at the other quadrant here.

57
00:03:37,400 --> 00:03:42,950
The other quadrant is when
we have- 1 sentence but

58
00:03:42,950 --> 00:03:45,810
I made a + 1 prediction.

59
00:03:45,810 --> 00:03:46,340
In other words,

60
00:03:46,340 --> 00:03:50,880
it was a negative sentence in the world in
a review and I thought it was positive.

61
00:03:50,880 --> 00:03:53,580
So that means I showed a bad thing.

62
00:03:53,580 --> 00:03:55,020
I showed a bad review on my website.

63
00:03:58,460 --> 00:03:59,936
So this is quite problematic.

64
00:03:59,936 --> 00:04:04,104
I showed a bad review on the website,
maybe said the sushi sucked,

65
00:04:04,104 --> 00:04:08,720
everybody read it,
nobody comes to my restaurant anymore.

66
00:04:08,720 --> 00:04:11,210
Big, big, big, big trouble.

67
00:04:11,210 --> 00:04:12,490
With these definitions,

68
00:04:12,490 --> 00:04:16,010
we can now talk about precision
in a little bit more precise way.

69
00:04:16,010 --> 00:04:21,290
So, precision is the fraction
of the true positives

70
00:04:21,290 --> 00:04:26,390
to all my positive predictions, so
the true positives to the false positives,

71
00:04:26,390 --> 00:04:29,060
and it has best possible value 1.

72
00:04:29,060 --> 00:04:33,820
That means that everything I predict will
be positive is actually positive, and

73
00:04:33,820 --> 00:04:34,700
worst possible value 0.

74
00:04:34,700 --> 00:04:38,970
Everything I predict will be
positive Turned out to be negative.

75
00:04:38,970 --> 00:04:44,620
In our example here,
we had four true positives,

76
00:04:44,620 --> 00:04:49,100
so four things that I predicted to be
positive, and were actually positive.

77
00:04:49,100 --> 00:04:53,490
And then we had four positives and

78
00:04:53,490 --> 00:04:59,250
two negatives in there, so we had a grand
total here of four-sixths or two-thirds.

79
00:04:59,250 --> 00:05:00,980
Just like I said in the beginning.

80
00:05:00,980 --> 00:05:02,470
So, two mistakes, four correct.

81
00:05:03,590 --> 00:05:06,970
Now, in the context of our application,
what would happen would be,

82
00:05:06,970 --> 00:05:10,010
I'm going to show the six
sentences of my website.

83
00:05:10,010 --> 00:05:13,390
Unfortunately, I ended up showing two
negative sentences on my website.

84
00:05:13,390 --> 00:05:19,340
So, for example,
I'll show that this one here, which says

85
00:05:19,340 --> 00:05:25,820
the seaweed salad was just okay,
the vegetable salad was just ordinary and

86
00:05:25,820 --> 00:05:31,050
so basically the salad sucked in my
restaurant and so that might not be good.

87
00:05:31,050 --> 00:05:34,010
I don't want to show bad
stuff on my website.

88
00:05:34,010 --> 00:05:38,021
So, I want to make sure that
I'm high precision which means

89
00:05:38,021 --> 00:05:42,450
things that are predicted to be
positive are actually positive.

90
00:05:42,450 --> 00:05:47,879
[MUSIC]