1
00:00:00,170 --> 00:00:01,720
Neural Networks are a pretty old

2
00:00:02,070 --> 00:00:03,950
algorithm that was originally motivated

3
00:00:05,030 --> 00:00:07,330
by the goal of having machines that can mimic the brain.

4
00:00:08,260 --> 00:00:09,330
Now in this class, of course

5
00:00:09,620 --> 00:00:11,000
I'm teaching Neural Networks to you

6
00:00:11,170 --> 00:00:12,170
because they work really well

7
00:00:12,460 --> 00:00:14,070
for different machine learning problems and

8
00:00:14,260 --> 00:00:16,910
not, certainly not because they're logically motivated.

9
00:00:18,050 --> 00:00:19,260
In this video, I'd like

10
00:00:19,440 --> 00:00:21,640
to give you some of the background on Neural Networks.

11
00:00:22,510 --> 00:00:25,430
So that we can get a sense of what we can expect them to do.

12
00:00:26,200 --> 00:00:27,170
Both in the sense of

13
00:00:27,330 --> 00:00:28,320
applying them to modern day

14
00:00:28,440 --> 00:00:30,470
machinery problems, as well as for

15
00:00:30,650 --> 00:00:32,000
those of you that might

16
00:00:32,190 --> 00:00:33,870
be interested in maybe the

17
00:00:34,000 --> 00:00:35,130
big AI dream of someday

18
00:00:35,710 --> 00:00:36,680
building truly intelligent machines.

19
00:00:37,790 --> 00:00:40,650
Also, how Neural Networks might pertain to that.

20
00:00:42,400 --> 00:00:44,260
The origins of Neural Networks was

21
00:00:44,900 --> 00:00:46,320
as algorithms that try

22
00:00:46,600 --> 00:00:47,880
to mimic the brain and

23
00:00:47,900 --> 00:00:49,020
those a sense that if we

24
00:00:49,140 --> 00:00:50,750
want to build learning systems

25
00:00:51,310 --> 00:00:53,110
while why not mimic perhaps the

26
00:00:53,180 --> 00:00:54,960
most amazing learning machine we

27
00:00:55,080 --> 00:00:56,070
know about, which is perhaps the

28
00:00:56,840 --> 00:00:58,710
brain. Neural Networks came to

29
00:00:58,820 --> 00:01:00,720
be very widely used throughout the

30
00:01:00,960 --> 00:01:03,260
1980's and 1990's and

31
00:01:03,750 --> 00:01:04,840
for various reasons as popularity

32
00:01:05,640 --> 00:01:06,680
diminished in the late

33
00:01:06,890 --> 00:01:08,410
90's. But more recently,

34
00:01:09,170 --> 00:01:10,520
Neural Networks  have had a

35
00:01:10,770 --> 00:01:12,060
major recent resurgence.

36
00:01:13,350 --> 00:01:14,530
One of the reasons for this

37
00:01:14,770 --> 00:01:16,640
resurgence is that Neural Networks

38
00:01:17,540 --> 00:01:19,010
are computationally some what

39
00:01:19,130 --> 00:01:20,590
more expensive algorithm and

40
00:01:20,960 --> 00:01:22,110
so, it was only, you know,

41
00:01:22,290 --> 00:01:23,830
maybe somewhat more recently that

42
00:01:24,440 --> 00:01:26,190
computers became fast enough

43
00:01:26,510 --> 00:01:27,540
to really run large scale

44
00:01:27,900 --> 00:01:29,610
Neural Networks and because of

45
00:01:29,690 --> 00:01:30,950
that as well as a

46
00:01:30,980 --> 00:01:32,940
few other technical reasons which we'll

47
00:01:33,080 --> 00:01:34,840
talk about later, modern Neural

48
00:01:35,110 --> 00:01:36,390
Networks today are the state

49
00:01:36,620 --> 00:01:38,620
of the art technique for many applications.

50
00:01:39,820 --> 00:01:41,000
So, when you think about mimicking

51
00:01:41,440 --> 00:01:42,600
the brain while one of

52
00:01:42,630 --> 00:01:44,860
the human brain does tell me same things, right?

53
00:01:45,030 --> 00:01:46,660
The brain can learn to

54
00:01:46,750 --> 00:01:48,170
see process images than to

55
00:01:48,400 --> 00:01:50,330
hear, learn to process our sense of touch.

56
00:01:50,570 --> 00:01:51,360
We can, you know, learn to

57
00:01:51,520 --> 00:01:52,380
do math, learn to do

58
00:01:52,710 --> 00:01:53,970
calculus, and the brain

59
00:01:54,110 --> 00:01:55,560
does so many different and amazing things.

60
00:01:55,930 --> 00:01:56,730
It seems like if you want

61
00:01:57,000 --> 00:01:58,280
to mimic the brain it seems

62
00:01:58,410 --> 00:01:59,630
like you have to write lots of different

63
00:01:59,960 --> 00:02:01,350
pieces of software to mimic all

64
00:02:01,620 --> 00:02:03,540
of these different fascinating, amazing things

65
00:02:03,820 --> 00:02:05,580
that the brain tell us, but does

66
00:02:05,820 --> 00:02:07,780
is this fascinating hypothesis that the

67
00:02:08,090 --> 00:02:09,100
way the brain does all of

68
00:02:09,170 --> 00:02:10,410
these different things is not

69
00:02:10,780 --> 00:02:12,080
worth like a thousand different programs,

70
00:02:13,070 --> 00:02:14,810
but instead, the way the

71
00:02:14,940 --> 00:02:16,020
brain does it is worth

72
00:02:16,440 --> 00:02:18,890
just a single learning algorithm.

73
00:02:19,310 --> 00:02:20,840
This is just a hypothesis but

74
00:02:21,080 --> 00:02:22,240
let me share with you

75
00:02:22,660 --> 00:02:24,440
some of the evidence for this.

76
00:02:24,750 --> 00:02:25,840
This part of the brain, that little

77
00:02:26,060 --> 00:02:27,270
red part of the brain, is

78
00:02:27,520 --> 00:02:29,150
your auditory cortex and

79
00:02:29,240 --> 00:02:30,620
the way you're understanding my voice

80
00:02:30,990 --> 00:02:33,340
now is your ear is

81
00:02:33,500 --> 00:02:34,940
taking the sound signal and routing

82
00:02:35,230 --> 00:02:36,940
the sound signal to your auditory

83
00:02:36,980 --> 00:02:38,180
cortex and that's what's

84
00:02:38,430 --> 00:02:40,100
allowing you to understand my words.

85
00:02:41,330 --> 00:02:42,590
Neuroscientists have done the

86
00:02:42,750 --> 00:02:44,560
following fascinating experiments where

87
00:02:44,790 --> 00:02:46,300
you cut the wire from

88
00:02:46,510 --> 00:02:47,440
the ears to the auditory

89
00:02:47,630 --> 00:02:49,100
cortex and you re-wire,

90
00:02:50,140 --> 00:02:52,010
in this case an animal's brain, so

91
00:02:52,200 --> 00:02:53,310
that the signal from the eyes

92
00:02:53,620 --> 00:02:56,890
to the optic nerve eventually gets routed to the auditory cortex.

93
00:02:58,040 --> 00:02:59,140
If you do this it turns out,

94
00:02:59,350 --> 00:03:00,620
the auditory cortex will learn

95
00:03:02,130 --> 00:03:02,400
to see.

96
00:03:02,730 --> 00:03:04,000
And this is in every single sense

97
00:03:04,430 --> 00:03:06,270
of the word see as we know it.

98
00:03:06,390 --> 00:03:07,470
So, if you do this to the animals,

99
00:03:07,770 --> 00:03:09,790
the animals can perform visual discrimination

100
00:03:10,310 --> 00:03:12,260
task and as they can

101
00:03:12,460 --> 00:03:13,570
look at images and make appropriate

102
00:03:14,100 --> 00:03:15,190
decisions based on the

103
00:03:15,460 --> 00:03:16,460
images and they're doing

104
00:03:16,780 --> 00:03:18,300
it with that piece of brain tissue.

105
00:03:19,590 --> 00:03:20,150
Here's another example.

106
00:03:21,270 --> 00:03:23,430
That red piece of brain tissue is your somatosensory cortex.

107
00:03:24,070 --> 00:03:26,680
That's how you process your sense of touch.

108
00:03:27,440 --> 00:03:29,020
If you do a similar re-wiring process

109
00:03:30,070 --> 00:03:32,740
then the somatosensory cortex will learn to see.

110
00:03:33,370 --> 00:03:34,710
Because of this and other

111
00:03:35,150 --> 00:03:36,670
similar experiments, these are

112
00:03:36,760 --> 00:03:38,200
called neuro-rewiring experiments.

113
00:03:39,470 --> 00:03:40,550
There's this sense that if

114
00:03:40,670 --> 00:03:41,670
the same piece of physical

115
00:03:42,180 --> 00:03:44,020
brain tissue can process sight

116
00:03:44,500 --> 00:03:45,970
or sound or touch then

117
00:03:46,190 --> 00:03:47,480
maybe there is one learning

118
00:03:47,780 --> 00:03:48,870
algorithm that can process

119
00:03:49,280 --> 00:03:50,520
sight or sound or touch.

120
00:03:51,450 --> 00:03:52,660
And instead of needing to

121
00:03:52,840 --> 00:03:54,530
implement a thousand different programs

122
00:03:55,120 --> 00:03:56,520
or a thousand different algorithms to

123
00:03:56,660 --> 00:03:58,430
do, you know, the thousand wonderful things

124
00:03:58,780 --> 00:04:00,510
that the brain does, maybe what

125
00:04:00,670 --> 00:04:01,980
we need to do is figure out

126
00:04:02,220 --> 00:04:04,900
some approximation or to whatever

127
00:04:05,160 --> 00:04:07,220
the brain's learning algorithm is and

128
00:04:07,340 --> 00:04:08,470
implement that and that the

129
00:04:08,690 --> 00:04:10,130
brain learned by itself how to

130
00:04:10,240 --> 00:04:11,860
process these different types of data.

131
00:04:13,000 --> 00:04:14,180
To a surprisingly large extent,

132
00:04:14,640 --> 00:04:15,730
it seems as if we can

133
00:04:16,270 --> 00:04:17,440
plug in almost any sensor

134
00:04:17,890 --> 00:04:19,020
to almost any part of

135
00:04:19,080 --> 00:04:21,030
the brain and so, within the

136
00:04:21,100 --> 00:04:22,990
reason, the brain will learn to deal with it.

137
00:04:25,730 --> 00:04:26,470
Here are a few more examples.

138
00:04:26,660 --> 00:04:28,680
On the upper left is

139
00:04:29,010 --> 00:04:31,220
an example of learning to see with your tongue.

140
00:04:32,100 --> 00:04:33,630
The way it works is--this is

141
00:04:33,830 --> 00:04:35,700
actually a system called BrainPort undergoing

142
00:04:36,500 --> 00:04:37,700
FDA trials now to help

143
00:04:37,870 --> 00:04:39,380
blind people see--but the

144
00:04:39,470 --> 00:04:41,300
way it works is, you strap

145
00:04:42,080 --> 00:04:43,360
a grayscale camera to your

146
00:04:43,580 --> 00:04:45,320
forehead, facing forward, that takes

147
00:04:45,620 --> 00:04:47,680
the low resolution grayscale image

148
00:04:48,120 --> 00:04:49,230
of what's in front of you

149
00:04:49,530 --> 00:04:50,630
and you then run a wire

150
00:04:51,750 --> 00:04:52,710
to an array of electrodes

151
00:04:53,420 --> 00:04:54,540
that you place on your tongue

152
00:04:55,090 --> 00:04:56,370
so that each pixel gets mapped

153
00:04:56,730 --> 00:04:58,720
to a location on your

154
00:04:58,830 --> 00:05:00,300
tongue where maybe a

155
00:05:00,430 --> 00:05:01,850
high voltage corresponds to a

156
00:05:02,020 --> 00:05:03,620
dark pixel and a low voltage

157
00:05:04,160 --> 00:05:05,780
corresponds to a bright

158
00:05:06,140 --> 00:05:08,320
pixel and, even as

159
00:05:08,480 --> 00:05:09,580
it does today, with this sort

160
00:05:09,880 --> 00:05:10,840
of system you and I will

161
00:05:10,900 --> 00:05:12,240
be able to learn to see, you know,

162
00:05:12,490 --> 00:05:15,120
in tens of minutes with our tongues.

163
00:05:15,270 --> 00:05:16,790
Here's a second example of human

164
00:05:17,210 --> 00:05:18,600
echo location or human sonar.

165
00:05:19,790 --> 00:05:20,990
So there are two ways you can do this.

166
00:05:21,330 --> 00:05:22,810
You can either snap your fingers,

167
00:05:24,490 --> 00:05:27,600
or click your tongue.

168
00:05:28,120 --> 00:05:28,570
I can't do that very well.

169
00:05:29,430 --> 00:05:30,480
But there are blind people

170
00:05:30,760 --> 00:05:31,970
today that are actually being

171
00:05:32,140 --> 00:05:33,420
trained in schools to do this

172
00:05:33,910 --> 00:05:35,600
and learn to interpret the pattern

173
00:05:36,040 --> 00:05:38,380
of sounds bouncing off your environment - that's sonar.

174
00:05:39,190 --> 00:05:39,860
So, if after you search

175
00:05:39,940 --> 00:05:42,310
on YouTube, there are

176
00:05:42,420 --> 00:05:44,040
actually videos of this amazing kid who

177
00:05:44,320 --> 00:05:45,770
tragically because of cancer

178
00:05:46,410 --> 00:05:49,020
had his eyeballs removed, so this is a kid with no eyeballs.

179
00:05:49,890 --> 00:05:51,740
But by snapping his fingers, he

180
00:05:51,890 --> 00:05:53,660
can walk around and never hit anything.

181
00:05:54,440 --> 00:05:55,390
He can ride a skateboard.

182
00:05:56,320 --> 00:05:57,480
He can shoot a basketball into a

183
00:05:57,550 --> 00:05:59,360
hoop and this is a kid with no eyeballs.

184
00:06:00,510 --> 00:06:02,120
Third example is the

185
00:06:02,370 --> 00:06:05,000
Haptic Belt where if

186
00:06:05,240 --> 00:06:07,250
you have a strap

187
00:06:07,510 --> 00:06:08,900
around your waist, ring up

188
00:06:09,060 --> 00:06:11,710
buzzers and always have the northmost one buzzing.

189
00:06:12,090 --> 00:06:13,450
You can give a human a

190
00:06:13,560 --> 00:06:14,780
direction sense similar to

191
00:06:15,240 --> 00:06:18,760
maybe how birds can, you know, sense where north is.

192
00:06:19,570 --> 00:06:21,530
And, some of the bizarre example, but

193
00:06:21,680 --> 00:06:22,820
if you plug a third eye

194
00:06:23,110 --> 00:06:24,080
into a frog, the frog

195
00:06:24,460 --> 00:06:25,830
will learn to use that eye as well.

196
00:06:27,410 --> 00:06:29,220
So, it's pretty amazing to

197
00:06:29,440 --> 00:06:31,300
what extent is as if

198
00:06:31,360 --> 00:06:32,640
you can plug in almost any sensor

199
00:06:32,830 --> 00:06:34,150
to the brain and the brain's

200
00:06:34,570 --> 00:06:35,940
learning algorithm will just figure

201
00:06:36,170 --> 00:06:37,520
out how to learn from that

202
00:06:37,710 --> 00:06:39,170
data and deal with that data.

203
00:06:40,290 --> 00:06:41,280
And there's a sense that

204
00:06:41,560 --> 00:06:42,840
if we can figure out what

205
00:06:43,060 --> 00:06:45,360
the brain's learning algorithm is, and,

206
00:06:45,510 --> 00:06:46,780
you know, implement it or implement some approximation

207
00:06:47,550 --> 00:06:49,400
to that algorithm on a computer, maybe

208
00:06:49,700 --> 00:06:50,760
that would be our best shot

209
00:06:51,190 --> 00:06:52,060
at, you know, making real progress

210
00:06:52,680 --> 00:06:54,320
towards the AI, the

211
00:06:54,380 --> 00:06:55,920
artificial intelligence dream of

212
00:06:55,990 --> 00:06:58,060
someday building truly intelligent machines.

213
00:06:59,370 --> 00:07:00,410
Now, of course, I'm not

214
00:07:00,830 --> 00:07:02,310
teaching Neural Networks, you know,

215
00:07:02,410 --> 00:07:03,590
just because they might give us

216
00:07:03,710 --> 00:07:04,740
a window into this far-off

217
00:07:05,200 --> 00:07:06,180
AI dream, even though I'm

218
00:07:06,290 --> 00:07:07,500
personally, that's one of the things

219
00:07:07,760 --> 00:07:09,890
that I personally work on in my research life.

220
00:07:10,650 --> 00:07:11,680
But the main reason I'm

221
00:07:11,840 --> 00:07:12,890
teaching Neural Networks in this

222
00:07:13,140 --> 00:07:14,520
class is because it's actually a

223
00:07:14,670 --> 00:07:15,830
very effective state of the

224
00:07:16,050 --> 00:07:18,340
art technique for modern day machine learning applications.

225
00:07:18,990 --> 00:07:20,340
So, in the next

226
00:07:20,630 --> 00:07:22,160
few videos, we'll start diving into

227
00:07:22,460 --> 00:07:23,830
the technical details of Neural

228
00:07:24,130 --> 00:07:25,280
Networks so that you

229
00:07:25,460 --> 00:07:26,460
can apply them to modern-day

230
00:07:26,490 --> 00:07:28,570
machine learning applications and get

231
00:07:28,730 --> 00:07:30,860
them to work well on problems.

232
00:07:31,160 --> 00:07:32,180
But for me, you know, one

233
00:07:32,430 --> 00:07:33,720
of the reasons the excite me  is

234
00:07:33,850 --> 00:07:35,450
that maybe they give

235
00:07:35,550 --> 00:07:37,000
us this window into

236
00:07:37,550 --> 00:07:38,660
what we might do if

237
00:07:38,890 --> 00:07:41,700
we're also thinking of

238
00:07:41,920 --> 00:07:43,600
what algorithms might someday be

239
00:07:43,730 --> 00:07:46,000
able to learn in a manner similar to humankind.