1
00:00:00,120 --> 00:00:01,020
In some of the earlier videos,

2
00:00:01,690 --> 00:00:03,300
I was talking about PCA as

3
00:00:03,410 --> 00:00:05,270
a compression algorithm where you

4
00:00:05,330 --> 00:00:06,760
may have say, a thousand dimensional

5
00:00:07,270 --> 00:00:08,760
data and compress it

6
00:00:09,100 --> 00:00:10,850
to a hundred dimensional feature back

7
00:00:11,010 --> 00:00:12,360
there, or have three dimensional

8
00:00:12,810 --> 00:00:14,980
data and compress it to a two dimensional representation.

9
00:00:16,360 --> 00:00:17,430
So, if this is a

10
00:00:17,620 --> 00:00:19,040
compression algorithm, there should

11
00:00:19,360 --> 00:00:20,440
be a way to go back from

12
00:00:20,660 --> 00:00:22,930
this compressed representation, back to

13
00:00:23,030 --> 00:00:25,560
an approximation of your original high dimensional data.

14
00:00:26,340 --> 00:00:28,070
So, given z(i), which maybe

15
00:00:28,780 --> 00:00:30,250
a hundred dimensional, how do

16
00:00:30,320 --> 00:00:31,710
you go back to your original

17
00:00:32,050 --> 00:00:34,720
representation x(i), which was maybe a thousand dimensional?

18
00:00:35,760 --> 00:00:36,820
In this video, I'd like to

19
00:00:36,930 --> 00:00:40,350
describe how to do that.

20
00:00:40,500 --> 00:00:43,620
In the PCA algorithm, we may have an example like this.

21
00:00:43,940 --> 00:00:45,670
So maybe that's my example x1

22
00:00:45,910 --> 00:00:47,810
and maybe that's my example x2.

23
00:00:48,110 --> 00:00:49,340
And what we do

24
00:00:49,570 --> 00:00:51,010
is, we take these examples and

25
00:00:51,120 --> 00:00:54,160
we project them onto this one dimensional surface.

26
00:00:55,150 --> 00:00:56,280
And then now we need

27
00:00:56,450 --> 00:00:57,750
to use only a real number,

28
00:00:58,350 --> 00:01:00,500
say z1, to specify the

29
00:01:00,600 --> 00:01:01,950
location of these points after

30
00:01:02,300 --> 00:01:04,640
they've been projected onto this one dimensional surface. So

31
00:01:04,890 --> 00:01:06,930
, given a point

32
00:01:07,730 --> 00:01:08,730
like this, given a point z1,

33
00:01:08,980 --> 00:01:10,840
how can we go back to

34
00:01:11,000 --> 00:01:12,580
this original two-dimensional space?

35
00:01:13,290 --> 00:01:15,380
And in particular, given the

36
00:01:15,510 --> 00:01:16,510
point z, which is an

37
00:01:16,660 --> 00:01:17,840
r, can we map

38
00:01:18,160 --> 00:01:19,660
this back to some approximate

39
00:01:20,440 --> 00:01:22,060
representation x and r2

40
00:01:22,370 --> 00:01:24,970
of whatever the original value of the data was?

41
00:01:26,520 --> 00:01:28,090
So, whereas z equals 0

42
00:01:28,400 --> 00:01:29,570
reduced transverse x, if you

43
00:01:29,680 --> 00:01:30,640
want to go in the opposite

44
00:01:30,930 --> 00:01:33,620
direction, the equation for

45
00:01:33,780 --> 00:01:35,150
that is, we're going

46
00:01:35,290 --> 00:01:38,220
to write x approx equals

47
00:01:40,470 --> 00:01:43,570
U reduce times z.

48
00:01:44,020 --> 00:01:44,880
Again, just to check the dimensions,

49
00:01:45,950 --> 00:01:47,760
here U reduce is

50
00:01:47,970 --> 00:01:48,990
going to be an n by k

51
00:01:49,680 --> 00:01:51,260
dimensional vector, z is

52
00:01:51,370 --> 00:01:53,270
going to be a k by 1 dimensional vector.

53
00:01:54,030 --> 00:01:56,280
So, we multiply these out and that's going to be n by one.

54
00:01:56,720 --> 00:01:58,270
So x approx is going

55
00:01:58,450 --> 00:01:59,990
to be an n dimensional vector.

56
00:02:00,310 --> 00:02:01,320
And so the intent of PCA,

57
00:02:01,390 --> 00:02:03,320
that is, the square projection error

58
00:02:03,620 --> 00:02:04,510
is not too big, is that

59
00:02:04,730 --> 00:02:06,050
this x approx will be

60
00:02:06,500 --> 00:02:08,640
close to whatever was

61
00:02:08,960 --> 00:02:10,090
the original value of x

62
00:02:10,270 --> 00:02:13,140
that you had used to derive z in the first place.

63
00:02:14,080 --> 00:02:16,630
To show a picture of what this looks like, this is what it looks like.

64
00:02:16,870 --> 00:02:17,820
What you get back of this

65
00:02:17,970 --> 00:02:19,640
procedure are points that lie

66
00:02:19,920 --> 00:02:22,860
on the projection of that onto the green line.

67
00:02:23,500 --> 00:02:24,580
So to take our early example,

68
00:02:24,920 --> 00:02:26,400
if we started off with

69
00:02:26,610 --> 00:02:28,570
this value of x1, and got

70
00:02:28,850 --> 00:02:29,690
this z1, if you plug

71
00:02:30,310 --> 00:02:32,760
z1 through this formula to get

72
00:02:33,440 --> 00:02:35,510
x1 approx, then this

73
00:02:35,730 --> 00:02:37,040
point here, that will be

74
00:02:37,590 --> 00:02:40,110
x1 approx, which is

75
00:02:40,260 --> 00:02:41,990
going to be r2 and so.

76
00:02:42,780 --> 00:02:44,060
And similarly, if you

77
00:02:44,180 --> 00:02:45,640
do the same procedure, this will

78
00:02:45,760 --> 00:02:47,840
be x2 approx.

79
00:02:49,640 --> 00:02:50,630
And you know, that's a pretty

80
00:02:50,780 --> 00:02:53,160
decent approximation to the original data.

81
00:02:53,670 --> 00:02:54,870
So, that's how you

82
00:02:55,060 --> 00:02:56,190
go back from your low dimensional

83
00:02:56,630 --> 00:02:58,350
representation z back to

84
00:02:58,700 --> 00:03:00,720
an uncompressed representation of

85
00:03:00,760 --> 00:03:01,990
the data we get back an

86
00:03:02,240 --> 00:03:03,480
the approxiamation to your original

87
00:03:03,690 --> 00:03:05,400
data x, and we

88
00:03:05,500 --> 00:03:07,210
also call this process reconstruction

89
00:03:08,220 --> 00:03:08,900
of the original data.

90
00:03:09,530 --> 00:03:10,950
When we think of trying to reconstruct

91
00:03:11,310 --> 00:03:13,630
the original value of x from the compressed representation.

92
00:03:16,770 --> 00:03:18,370
So, given an unlabeled data

93
00:03:18,610 --> 00:03:19,850
set, you now know how to

94
00:03:19,990 --> 00:03:21,710
apply PCA and take

95
00:03:21,970 --> 00:03:23,800
your high dimensional features x and

96
00:03:24,130 --> 00:03:25,440
map it to this

97
00:03:25,560 --> 00:03:27,200
lower dimensional representation z, and

98
00:03:27,400 --> 00:03:28,630
from this video, hopefully you now

99
00:03:28,910 --> 00:03:29,670
also know how to take

100
00:03:30,260 --> 00:03:31,690
these low representation z and

101
00:03:31,860 --> 00:03:32,810
map the backup to an approximation

102
00:03:33,700 --> 00:03:35,780
of your original high dimensional data.

103
00:03:37,300 --> 00:03:38,180
Now that you know how to

104
00:03:38,460 --> 00:03:40,280
implement in applying PCA, what

105
00:03:40,470 --> 00:03:41,290
we will like to do next is to

106
00:03:41,390 --> 00:03:42,250
talk about some of the

107
00:03:42,290 --> 00:03:43,460
mechanics of how to

108
00:03:43,990 --> 00:03:45,240
actually use PCA well,

109
00:03:45,530 --> 00:03:46,670
and in particular, in the next

110
00:03:46,890 --> 00:03:47,610
video, I like to talk

111
00:03:48,090 --> 00:03:49,730
about how to choose K, which is,

112
00:03:49,910 --> 00:03:51,140
how to choose the dimension

113
00:03:51,560 --> 00:03:53,570
of this reduced representation vector z.