Neural Networks are a pretty old
algorithm that was originally motivated
by the goal of having machines that can mimic the brain.
Now in this class, of course
I'm teaching Neural Networks to you
because they work really well
for different machine learning problems and
not, certainly not because they're logically motivated.
In this video, I'd like
to give you some of the background on Neural Networks.
So that we can get a sense of what we can expect them to do.
Both in the sense of
applying them to modern day
machinery problems, as well as for
those of you that might
be interested in maybe the
big AI dream of someday
building truly intelligent machines.
Also, how Neural Networks might pertain to that.
The origins of Neural Networks was
as algorithms that try
to mimic the brain and
those a sense that if we
want to build learning systems
while why not mimic perhaps the
most amazing learning machine we
know about, which is perhaps the
brain. Neural Networks came to
be very widely used throughout the
1980's and 1990's and
for various reasons as popularity
diminished in the late
90's. But more recently,
Neural Networks  have had a
major recent resurgence.
One of the reasons for this
resurgence is that Neural Networks
are computationally some what
more expensive algorithm and
so, it was only, you know,
maybe somewhat more recently that
computers became fast enough
to really run large scale
Neural Networks and because of
that as well as a
few other technical reasons which we'll
talk about later, modern Neural
Networks today are the state
of the art technique for many applications.
So, when you think about mimicking
the brain while one of
the human brain does tell me same things, right?
The brain can learn to
see process images than to
hear, learn to process our sense of touch.
We can, you know, learn to
do math, learn to do
calculus, and the brain
does so many different and amazing things.
It seems like if you want
to mimic the brain it seems
like you have to write lots of different
pieces of software to mimic all
of these different fascinating, amazing things
that the brain tell us, but does
is this fascinating hypothesis that the
way the brain does all of
these different things is not
worth like a thousand different programs,
but instead, the way the
brain does it is worth
just a single learning algorithm.
This is just a hypothesis but
let me share with you
some of the evidence for this.
This part of the brain, that little
red part of the brain, is
your auditory cortex and
the way you're understanding my voice
now is your ear is
taking the sound signal and routing
the sound signal to your auditory
cortex and that's what's
allowing you to understand my words.
Neuroscientists have done the
following fascinating experiments where
you cut the wire from
the ears to the auditory
cortex and you re-wire,
in this case an animal's brain, so
that the signal from the eyes
to the optic nerve eventually gets routed to the auditory cortex.
If you do this it turns out,
the auditory cortex will learn
to see.
And this is in every single sense
of the word see as we know it.
So, if you do this to the animals,
the animals can perform visual discrimination
task and as they can
look at images and make appropriate
decisions based on the
images and they're doing
it with that piece of brain tissue.
Here's another example.
That red piece of brain tissue is your somatosensory cortex.
That's how you process your sense of touch.
If you do a similar re-wiring process
then the somatosensory cortex will learn to see.
Because of this and other
similar experiments, these are
called neuro-rewiring experiments.
There's this sense that if
the same piece of physical
brain tissue can process sight
or sound or touch then
maybe there is one learning
algorithm that can process
sight or sound or touch.
And instead of needing to
implement a thousand different programs
or a thousand different algorithms to
do, you know, the thousand wonderful things
that the brain does, maybe what
we need to do is figure out
some approximation or to whatever
the brain's learning algorithm is and
implement that and that the
brain learned by itself how to
process these different types of data.
To a surprisingly large extent,
it seems as if we can
plug in almost any sensor
to almost any part of
the brain and so, within the
reason, the brain will learn to deal with it.
Here are a few more examples.
On the upper left is
an example of learning to see with your tongue.
The way it works is--this is
actually a system called BrainPort undergoing
FDA trials now to help
blind people see--but the
way it works is, you strap
a grayscale camera to your
forehead, facing forward, that takes
the low resolution grayscale image
of what's in front of you
and you then run a wire
to an array of electrodes
that you place on your tongue
so that each pixel gets mapped
to a location on your
tongue where maybe a
high voltage corresponds to a
dark pixel and a low voltage
corresponds to a bright
pixel and, even as
it does today, with this sort
of system you and I will
be able to learn to see, you know,
in tens of minutes with our tongues.
Here's a second example of human
echo location or human sonar.
So there are two ways you can do this.
You can either snap your fingers,
or click your tongue.
I can't do that very well.
But there are blind people
today that are actually being
trained in schools to do this
and learn to interpret the pattern
of sounds bouncing off your environment - that's sonar.
So, if after you search
on YouTube, there are
actually videos of this amazing kid who
tragically because of cancer
had his eyeballs removed, so this is a kid with no eyeballs.
But by snapping his fingers, he
can walk around and never hit anything.
He can ride a skateboard.
He can shoot a basketball into a
hoop and this is a kid with no eyeballs.
Third example is the
Haptic Belt where if
you have a strap
around your waist, ring up
buzzers and always have the northmost one buzzing.
You can give a human a
direction sense similar to
maybe how birds can, you know, sense where north is.
And, some of the bizarre example, but
if you plug a third eye
into a frog, the frog
will learn to use that eye as well.
So, it's pretty amazing to
what extent is as if
you can plug in almost any sensor
to the brain and the brain's
learning algorithm will just figure
out how to learn from that
data and deal with that data.
And there's a sense that
if we can figure out what
the brain's learning algorithm is, and,
you know, implement it or implement some approximation
to that algorithm on a computer, maybe
that would be our best shot
at, you know, making real progress
towards the AI, the
artificial intelligence dream of
someday building truly intelligent machines.
Now, of course, I'm not
teaching Neural Networks, you know,
just because they might give us
a window into this far-off
AI dream, even though I'm
personally, that's one of the things
that I personally work on in my research life.
But the main reason I'm
teaching Neural Networks in this
class is because it's actually a
very effective state of the
art technique for modern day machine learning applications.
So, in the next
few videos, we'll start diving into
the technical details of Neural
Networks so that you
can apply them to modern-day
machine learning applications and get
them to work well on problems.
But for me, you know, one
of the reasons the excite me  is
that maybe they give
us this window into
what we might do if
we're also thinking of
what algorithms might someday be
able to learn in a manner similar to humankind.