In this video we talk about
matrix, matrix multiplication or
how to multiply two matrices together.
When we talk about the method
in linear regression for how
to solve for the parameters,
theta zero and theta one, all in one shot.
So, without needing an iterative algorithm like gradient descent.
When we talk about that algorithm,
it turns out that matrix, matrix
multiplication is one of the key steps that you need to know.
So, let's, as usual, start with an example.
Let's say I have two matrices
and I want to multiply them together.
Let me again just reference this
example and then I'll tell you in a little bit what happens.
So, the first thing
I'm gonna do is, I'm going
to pull out the first
column of this matrix on the right.
And I'm going to take this
matrix on the left and
multiply it by, you know, a vector.
That's just this first column, OK?
And it turns out if I
do that I am going to get the vector 11, 9.
So, this is the same matrix
vector multiplication as you saw in the last videos.
I worked this out in advance so, I know it's 11, 9.
And, then, the second thing
I'm going to do is, I'm going
to pull out the second column,
this matrix on the right and
I am then going to
take this matrix on the left,
right, so, it will be that matrix,
and multiply it by
that second column on the right.
So, again, this is a matrix
vector multiplication set, which
you saw from the previous video, and
it turns out that if you
multiply this matrix and this
vector, you get 10,
14 and by
the way, if you want to practice
your matrix vector multiplication, feel
free to pause the video and check this product yourself.
Then, I'm just going
to take these two results and
put them together, and that will be my answer.
So, turns out the
outcome of this product is going
to be a 2 by 2 matrix, and
The way I am going to fill
in this matrix is just by
taking my elements 11,
9 and plugging them here, and
taking 10, 14 and plugging
them into the second column.
Okay?
So, that was the mechanics of
how to multiply a matrix by
another matrix.
You basically look at the
second matrix one column at a time, and you assemble the answers.
And again, we will step
through this much more carefully in
a second, but I just
want to point out also, this
first example is a 2x3 matrix matrix.
Multiplying that by a
3x2 matrix, and the
outcome of this product, it
turns out to be a 2x2
matrix.
And again, we'll see in a second why this was the case.
All right.
That was the mechanics of the calculation.
Let's actually look at the
details and look at what
exactly happened.
Here are details.
I have a matrix A and
I want to multiply that
with a matrix B, and the result
will be some new matrix C. And
it turns out you can only
multiply together matrices whose
dimensions match so A
is an m by n matrix,
so m columns, n columns and
I am going to multiply
that with an n by o
and it turns out this n
here must match this n
here, so the number of columns
in first matrix must equal to the number of rows in second matrix.
And the result of this
product will be an M
by O matrix, like the the matrix C here.
And, in the previous
video, everything we did corresponded
to this special case of OB
equal to 1.
Okay?
That was, that was in case of B being a vector.
But now, we are going to
view of the case of values of O larger than 1.
So, here's how you
multiply together the two matrices.
In order to get, what
I am going to do is
I am going to take the
first column of B
and treat that as a vector,
and multiply the matrix A,
with the first column of B,
and the result of that will
be a M by 1 vector,
and we're going to put that over here.
Then, I'm going to
take the second column
of B, right, so,
this is another n by
one vector, so, this column
here, this is right, n
by one, those are n dimensional
vector, gonna multiply this
matrix with this n by one vector.
The result will be
a M dimensional vector,
which we'll put there.
And, so on.
Okay?
And, so, you know, and then
I'm going to take the third
column, multiply it by
this matrix, I get a M dimensional vector.
And so on, until you get
to the last column times,
the matrix times the
lost column gives you
the lost column of C.
Just to say that again.
The ith column of the
matrix C is attained
by taking the matrix A and
multiplying the matrix A with
the ith column of the
matrix B for the values
of I equals 1, 2
up through O. Okay ?
So, this is just a summary
of what we did up there
in order to compute the matrix
C.  Let's look at just one more example.
Let 's say, I want to multiply together these two matrices.
So, what I'm going to
do is, first pull
out the first column
of my second matrix, that
was matrix B, that was
my matrix B on the previous slide.
And, I therefore, have this
matrix times my vector and
so, oh, let's do this calculation quickly.
There's going to be equal to,
right, 1, 3 times 0,
3 so that gives 1
times 0, plus 3 times 3.
And, the second element
is going to be 2,
5 times 0, 3 so, that's going to
be two times 0 plus 5
times 3 and that is
9,15, actually didn't
write that in green, so this
is nine fifteen, and then mix.
I am going to pull out
the second column of this,
and do the corresponding
calculation so there's this
matrix times this vector 1, 2.
Let's also do this
quickly, so that's one times
one plus three times two.
So that deals with that
row, let's do the
other one, so let's see,
that gives me two times
one plus times two,
so that is going to
be equal to, let's see,
one times one plus three times
one is four and two
times one plus five times two
is twelve.
So now I have these two
you, and so my
outcome, so the product
of these two matrices is going
to be, this goes
here and this
goes here, so I
get nine fifteen and
four twelve and you
may notice also that the result
of multiplying a 2x2 matrix
with another 2x2 matrix.
The resulting dimension is going
to be that first two times
that second two, so the result
is itself also a two by two matrix.
Finally let me show you
one more neat trick you can
do with matrix matrix multiplication.
Let's say as before that we
have four houses whose
prices we want to predict,
only now we have three
competing hypothesis shown here
on the right, so if
you want to So apply all
3 competing hypotheses to
all four of the houses, it
turns out you can do that
very efficiently using a
matrix matrix multiplication so here
on the left is my usual
matrix, same as from the
last video where these values
are my housing prices and I put ones there on the left as well.
And, what I'm going to
do is construct another matrix, where
here these, the first
column, is this minus
40 and two five and
the second column is this two
hundred open one and so
on and it
turns out that if you
multiply these two matrices
what you find is that, this
first column, you know,
oh, well how do you get this first column, right?
A procedure from matrix
matrix multiplication is the way
you get this first column, is
you take this matrix and you
multiply it by this
first column, and we
saw in the previous video that this
is exactly the predicted
housing prices of the
first hypothesis, right?
Of this first hypothesis here.
And, how about a second column?
Well, how do setup the second column?
The way you get the second column
is, well, you take this
matrix and you multiply by this second column.
And so this second column turns
out to be the predictions of
the second hypothesis of
the second hypothesis up there,
and similarly for the third column.
And so, I didn't step
through all the details but hopefully
you just, feel free to
pause the video and check
the math yourself and check
that what I just claimed really is true.
But it turns out that by
constructing these two matrices, what
you can therefore do is very
quickly apply all three
hypotheses to all four
house sizes to get,
you know, all twelve predicted
prices output by your
three hypotheses on your four houses.
So one matrix multiplications
that you manage to make 12
predictions and, even
better, it turns out that
in order to do that matrix
multiplication and there are
lots of good linear algebra libraries
in order to do this
multiplication step for you,
and no matter so pretty
much any reasonable programming language that you might be using.
Certainly all the top ten
most popular programming languages will have great linear algebra libraries.
And they'll be good thing are
highly optimized in order
to do that, matrix matrix
multiplication very efficiently, including
taking, taking advantage of
any parallel computation that
your computer may be capable
of, when your computer has multiple
calls or lots of
multiple processors, within a processor sometimes
there's there's parallelism as well called symdiparallelism [sp].
The computer take care of
and you should, there are
very good free libraries
that you can use to do
this matrix matrix multiplication very
efficiently so that you
can very efficiently, you
know, makes lots of predictions of lots of hypotheses.