You have learned a lot about ConvNets,
everything ranging from the architecture of the ConvNet to
how to use it for image recognition, to object detection, to face
recognition and neural-style transfer. And even though most of
the discussion has focused on images, on sort of 2D data,
because images are so pervasive. It turns out that many of the ideas
you've learned about also apply, not just to 2D images but
also to 1D data as well as to 3D data. Let's take a look. In the first week of this course,
you learned about the 2D convolution, where you might input a 14 x 14 image and
convolve that with a 5 x 5 filter. And you saw how 14 x 14
convolved with 5 x 5, this gives you a 10 x 10 output. And if you have multiple channels,
maybe those 14 x 14 x 3, then it would be 5 x 5
that matches the same 3. And then if you have multiple filters, say
16 filters, you end up with 10 x 10 x 16. It turns out that a similar idea
can be applied to 1D data as well. For example, on the left is an EKG signal,
also called an electrocardioagram. Basically if you place an electrode
over your chest, this measures the little voltages that vary across
your chest as your heart beats. Because the little electric waves
generated by your heart's beating can be measured with a pair of electrodes. And so
this is an EKG of someone's heart beating. And so each of these peaks
corresponds to one heartbeat. So if you want to use EKG signals
to make medical diagnoses, for example, then you would have 1D
data because what EKG data is, is it's a time series showing
the voltage at each instant in time. So rather than a 14 x
14 dimensional input, maybe you just have
a 14 dimensional input. And in that case, you might want to
convolve this with a 1 dimensional filter. So rather than the 5 by 5,
you just have 5 dimensional filter. So with 2D data what a convolution will
allow you to do was to take the same 5 x 5 feature detector and apply it across at
different positions throughout the image. And that's how you wound up
with your 10 x 10 output. What a 1D filter allows you to do is
take your 5 dimensional filter and similarly apply that in lots of different
positions throughout this 1D signal. And so if you apply this convolution, what you find is that a 14
dimensional thing convolved with this 5 dimensional thing, this would
give you a 10 dimensional output. And again, if you have multiple channels,
you might have in this case you can use just 1 channel, if you have 1 lead
or 1 electrode for EKG, so times 5 x 1. And if you have 16 filters,
maybe end up with 10 x 16 over there, and this could be one
layer of your ConvNet. And then for the next layer of your
ConvNet, if you input a 10 x 16 dimensional input and you might convolve
that with a 5 dimensional filter again. Then these have 16 channels,
so that has a match. And we have 32 filters,
then the output of another layer would be 6 x 32,
if you have 32 filters, right? And the analogy to the the 2D data, this is similar to all of
the 10 x 10 x 16 data and convolve it with a 5 x 5 x 16,
and that has to match. That will give you a 6
by 6 dimensional output, and you have 32 filters,
that's where the 32 comes from. So all of these ideas apply also to
1D data, where you can have the same feature detector, such as this,
apply to a variety of positions. For example, to detect the different
heartbeats in an EKG signal. But to use the same set of features to
detect the heartbeats even at different positions along these time series, and
so ConvNet can be used even on 1D data. For along with 1D data applications, you
actually use a recurrent neural network, which you learn about in the next course. But some people can also try
using ConvNets in these problems. And in the next course on sequence models,
which we will talk about recurring neural networks and
LCM and other models like that. We'll talk about the pros and cons of
using 1D ConvNets versus some of those other models that are explicitly
designed to sequenced data. So that's the generalization
from 2D to 1D. How about 3D data? Well, what is three dimensional data? It is that, instead of having a 1D list
of numbers or a 2D matrix of numbers, you now have a 3D block, a three
dimensional input volume of numbers. So here's the example of that
which is if you take a CT scan, this is a type of X-ray scan that gives
a three dimensional model of your body. But what a CT scan does is it takes
different slices through your body. So as you scan through a CT
scan which I'm doing here, you can look at different slices of
the human torso to see how they look and so this data is fundamentally
three dimensional. And one way to think of this data is
if your data now has some height, some width, and then also some depth. Where this is the different
slices through this volume, are the different slices
through the torso. So if you want to apply a ConvNet
to detect features in this three dimensional CAT scan or CT scan,
then you can generalize the ideas from the first slide to three
dimensional convolutions as well. So if you have a 3D volume, and for the sake of simplicity let's
say is 14 x 14 x 14 and so this is the height, width,
and depth of the input CT scan. And again, just like images
they'll all have to be square, a 3D volume doesn't have to
be a perfect cube as well. So the height and
width of a image can be different, and in the same way the height and width and
the depth of a CT scan can be different. But I'm just using 14 x 14 x 14
here to simplify the discussion. And if you convolve this with
a now a 5 x 5 x 5 filter, so you're filters now
are also three dimensional then this would give you
a 10 x 10 x 10 volume. And technically, you could also have by 1,
if this is the number of channels. So this is just a 3D volume, but
your data can also have different numbers of channels,
then this would be times 1 as well. Because the number of channels here and
the number of channels here has to match. And then if you have 16 filters did
a 5 x 5 x 5 x 1 then the next output will be a 10 x 10 x 10 x 16. So this could be one layer of your
ConvNet over 3D data, and if the next layer of the ConvNet convolves this again
with a 5 x 5 x 5 x 16 dimensional filter. So this number of channels has
to match data as usual, and if you have 32 filters then similar to
what you saw was ConvNet of the images. Now you'll end up with a 6 x 6
x 6 volume across 32 channels. So 3D data can also be learned on, sort of directly using
a three dimensional ConvNet. And what these filters do is really
detect features across your 3D data, CAT scans, medical scans as
one example of 3D volumes. But another example of data, you could
treat as a 3D volume would be movie data, where the different slices could be
different slices in time through a movie. And you could use this to detect motion or
people taking actions in movies. So that's it on generalization
of ConvNets from 2D data to also 1D as well as 3D data. Image data is so pervasive that the vast
majority of ConvNets are on 2D data, on image data, but I hope that these other
models will be helpful to you as well. So this is it,
this is the last video of this week and the last video of this course on ConvNets. You've learned a lot about ConvNets and I hope you find many of these
ideas useful for your future work. So congratulations on
finishing these videos. I hope you enjoyed this
week's exercise and I look forward also to seeing you in
the next course on sequence models.