In this video, I'm gonna tell you a little bit about real neurons on the real brain
which provide the inspiration for the artificial neural network that we're gonna
learn about in this course. In most of the course, we won't talk much
about real neurons but I wanted to give you a quick overview of the beginning.
There's several different reasons to study how networks of neurons can compute
things. The first is to understand how the brain
actually works. You might think we could do that just by
experiments on the brain. But it's very big and complicated, and it
dies when you poke it around. And so we need to use computer simulations
to help us understand what we're discovering in empirical studies.
The second is to understand the style of parallel computation, this inspired by the
fact that the brain can compute with a big parallel network, a world of relatively
slow neurons. If you can understand that style of
parallel computation we might be able to make better parallel computers.
It's very different from the way computation is done on a conventional
serial processor. It should be very good for things that
brains are good at like vision, and it should also be bad for things that brains
are bad at by multiplying two numbers together.
A third reason, which is the relevant one for this course, is to solve practical
problems by using novel learning algorithms that were inspired by the
brain. These algorithms can be very useful even
if they're not actually how the brain works.
So in most of this course we won't talk much about how the brain actually works.
It's just used as a source of inspiration to tell us the big, parallel networks of
neurons can compute very complicated things.
I'm gonna talk more in this video though about how the brain actually works.
A typical cortical neuron has a gross physical structure that consists of a cell
body, and an axon where it sends messages to other neurons, and a denditric tree
where it receives messages from other neurons.
Where an axon from one neuron contacts a dendritic tree of another neuron, there's
a structure called a synapse. And a spike of activity traveling along
the axon, causes charge to be injected into the post synaptic neuron at a
synapse. A neuron generates spikes when it's
received enough charge in its dendritic tree to depolarize a part of the cell body
called the axon hillock. And when that gets depolarized, the neuron
sends a spike out along its axon. And the spike's just a wave of
depolarization that travels along the axon.
Synapses themselves have interesting structure.
They contain little vesicles of transmitter chemical and when a spike
arrives in the axon it causes these vesicles to migrate to the surface and be
released into the synaptic cleft. There's several different kinds of
transmitter chemical. There's one that implement positive
weights and ones that implement negative weights.
The transmitter molecules diffuse across the synaptic clef and bind to receptor
molecules in the membrane of the post-synaptic neuron, and by binding to
these big molecules in the membrane they change their shape, and that creates holes
in the membrane. These holes are like specific ions to flow
in or out of the post-synaptic neuron and that changes their state of
depolarization. Synapses adapt, and that's what most of
learning is, changing the effectiveness of a synapse.
They can adapt by varying the number of vesicles that get released when a spike
arrives. Or by varying the number of receptor
molecules that are sensitive to the released transmitter molecules.
Synapses are very slow compared with computer memory.
But they have a lot of advantages over the random access memory on a computer,
they're very small and very low power. And they can adapt.
That's the most important property. They use locally available signals to
change their strengths, and that's how we learn to perform complicated computations.
The issue of course is how do they decide how to change their strength?
What is the, what are the rules for how they should adapt.
So, all on one slide this is how the brain works.
Each neuron receives inputs from other neurons.
A few of the neurons receive inputs from the receptors.
It's a large number of neurons, but only a small fraction of them.
And, the neurons communicate with each other within in the cortex by sending
these spikes of activity. The effective in input line on a neuron is
controlled by synaptic weight, which can be positive or negative.
And these synaptic weights adapt. And by adapting these weights the whole
network learns to perform different kinds of computation.
For example recognizing objects, understanding language, making plans,
controlling the movements of your body. You have about ten to the eleven neurons,
each of which has about ten to the four weights.
So you probably ten to the fifteen or maybe only about ten to the fourteen
synaptic weights. And a huge number of these weights, quite
a large fraction of them, can affect the ongoing computation in a very small
fraction of a second, in a few milliseconds.
That's much better bandwidth to stored knowledge than even a modern workstation
has. One final point about the brain is that
the cortex is modular, at least it learns to be modular.
Different bits of the cotex end up doing different things.
Genetically, the inputs from the senses go to different bits of the cortex.
And that determines a lot about what they end up doing.
If you damage the brain of an adult, local damage to the brain causes specific
effects. Damage to one place might cause you to
lose your ability to understand language. Damage to another place might cause you to
lose your ability to recognize objects. We know a lot about how functions are
located in the brain because when you use a part of the brain for doing something it
requires energy, and so it demands more blood flow, and you can see the blood flow
in a brain scanner. That allows you to see which bits of the
brain you're using for particular tasks. But the remarkable thing about cortex is
it looks pretty much the same all over, and that strongly suggests that it's got a
fairly flexible universal learning algorithm in it.
That's also suggested by the fact that if you damage the brain early on, functions
will relocate to other parts of the brain. So it's not genetically predetermined, at
least not directly, which part of the brain will perform which function.
There's convincing experiments on baby ferrets that show that if you cut off the
input to the auditory cortex that comes from the ears, and instead, reroute the
visual input to auditory cortex, then the auditory cortex that was destined to deal
with sounds will actually learn to deal with visual input, and create neurons that
look very like the neurons in the visual system.
This suggest the cortex is made of general purpose stuff that has the ability to turn
into special purpose hardware for particular tasks in response to
experience. And that gives you a nice combination of,
rapid parallel computation once you have learnt, plus flexibility, so you can put,
you can learn new functions, so you are learning, to do the parallel computation.
Its quiet like a FPGA, where you build standard parallel hardware, then after its
built, you put in information that tells it what particular parallel computation to
do. Conventional computers get their
flexibility by having a stored sequential program.
But this required very fast central processors to access the lines in the
sequential program and perform long sequential computations.