In this video, I want to
tell you about a couple of special
matrix operations, called the
matrix inverse and the matrix transpose operation.
Let's start by talking about matrix
inverse, and as
usual we'll start by thinking about
how it relates to real numbers.
In the last video, I said
that the number one plays the
role of the identity in
the space of real numbers because
one times anything is equal to itself.
It turns out that real numbers
have this property that very
number have an, that
each number has an inverse,
for example, given the number
three, there exists some
number, which happens to
be three inverse so that
that number times gives you
back the identity element one.
And so to me, inverse of course this is just one third.
And given some other number,
maybe twelve there is
some number which is the
inverse of twelve written as
twelve to the minus one, or
really this is just one twelve.
So that when you multiply these two things together.
the product is equal to
the identity element one again.
Now it turns out that in
the space of real numbers, not everything has an inverse.
For example the number zero
does not have an inverse, right?
Because zero's a zero inverse, one over zero that's undefined.
Like this one over zero is not well defined.
And what we want to
do, in the rest of this
slide, is figure out what does
it mean to compute the inverse of a matrix.
Here's the idea: If
A is a n by
n matrix, and it
has an inverse, I will say
a bit more about that later, then
the inverse is going to
be written A to the
minus one and A
times this inverse, A to
the minus one, is going to
equal to A inverse times
A, is going to
give us back the identity matrix.
Okay?
Only matrices that are
m by m for some the idea of M having inverse.
So, a matrix is
M by M, this is also
called a square matrix and
it's called square because
the number of rows is equal to the number of columns.
Right and it turns out
only square matrices have inverses,
so A is a square
matrix, is m by m,
on inverse this equation over here.
Let's look at a concrete example,
so let's say I
have a matrix, three, four,
two, sixteen.
So this is a two by
two matrix, so it's
a square matrix and so this
may just could have an and
it turns out that I
happen to know the inverse
of this matrix is zero point
four, minus zero point
one, minus zero point
zero five, zero zero seven five.
And if I take this matrix
and multiply these together it
turns out what I get
is the two by
two identity matrix, I,
this is I two by two.
Okay?
And so on this slide,
you know this matrix is
the matrix A, and this matrix is the matrix A-inverse.
And it turns out
if that you are computing A
times A-inverse, it turns out
if you compute A-inverse times
A you also get back the identity matrix.
So how did I
find this inverse or how
did I come up with this inverse over here?
It turns out that sometimes
you can compute inverses by hand
but almost no one does that these days.
And it turns out there is
very good numerical software for
taking a matrix and computing its inverse.
So again, this is one of
those things where there are lots
of open source libraries that
you can link to from any
of the popular programming languages to compute inverses of matrices.
Let me show you a quick example.
How I actually computed this inverse,
and what I did was I used software called Optive.
So let me bring that up.
We will see a lot about Optive later.
Let me just quickly show you an example.
Set my matrix A to
be equal to that matrix on
the left, type three four
two sixteen, so that's my matrix A right.
This is matrix 34,
216 that I have down
here on the left.
And, the software lets me compute
the inverse of A very easily.
It's like P over A equals this.
And so, this is right,
this matrix here on my
four minus, on my one, and so on.
This given the numerical
solution to what is the
inverse of A. So let me
just write, inverse of A
equals P inverse of
A over that I
can now just verify that A
times A inverse the identity
is, type A times the
inverse of A and
the result of that is
this matrix and this is
one one on the diagonal
and essentially ten to
the minus seventeen, ten to the
minus sixteen, so Up to
numerical precision, up to
a little bit of round off
error that my computer
had in finding optimal matrices
and these numbers off the
diagonals are essentially zero
so A times the inverse is essentially the identity matrix.
Can also verify the inverse of
A times A is also
equal to the identity,
ones on the diagonals and values
that are essentially zero except
for a little bit of round
dot error on the off diagonals.
If a definition that the inverse
of a matrix is, I had
this caveat first it must
always be a square matrix, it
had this caveat, that if
A has an inverse, exactly what
matrices have an inverse
is beyond the scope of this
linear algebra for review that one
intuition you might take away
that just as the
number zero doesn't have an
inverse, it turns out
that if A is say the
matrix of all zeros, then
this matrix A also does
not have an inverse because there's
no matrix there's no A
inverse matrix so that this
matrix times some other
matrix will give you the
identity matrix so this matrix of
all zeros, and there
are a few other matrices with properties similar to this.
That also don't have an inverse.
But it turns out that
in this review I don't
want to go too deeply into what
it means matrix have an
inverse but it turns
out for our machine learning
application this shouldn't be
an issue or more precisely
for the learning algorithms where
this may be an to namely
whether or not an inverse matrix
appears and I will tell when
we get to those learning algorithms
just what it means for an
algorithm to have or not
have an inverse and how to fix it in case.
Working with matrices that don't
have inverses.
But the intuition if you
want is that you can
think of matrices as not
have an inverse that is somehow
too close to zero in some sense.
So, just to wrap
up the terminology, matrix that
don't have an inverse Sometimes called
a singular matrix or degenerate
matrix and so this
matrix over here is an
example zero zero zero matrix.
is an example of a matrix that is singular, or a matrix that is degenerate.
Finally, the last special
matrix operation I want to
tell you about is to do matrix transpose.
So suppose I have
matrix A, if I compute
the transpose of A, that's what I get here on the right.
This is a transpose which is
written and A superscript T,
and the way you compute
the transpose of a matrix is as follows.
To get a transpose I am going
to first take the first
row of A one to zero.
That becomes this first column of this transpose.
And then I'm going to take
the second row of A,
3 5 9, and that becomes the second column.
of the matrix A transpose.
And another way of
thinking about how the computer transposes
is as if you're taking this
sort of 45 degree axis
and you are mirroring or you
are flipping the matrix along that 45 degree axis.
so here's the more formal
definition of a matrix transpose.
Let's say A is a m by n matrix.
And let's let B equal A
transpose and so BA transpose like so.
Then B is going to
be a n by m matrix
with the dimensions reversed so
here we have a 2x3 matrix.
And so the transpose becomes a
3x2 matrix, and moreover,
the BIJ is equal to AJI.
So the IJ element of this
matrix B is going to be
the JI element of that
earlier matrix A. So for
example, B 1 2
is going to be equal
to, look at this
matrix, B 1 2 is going to be equal to
this element 3 1st row, 2nd column.
And that equal to this, which
is a two one, second
row first column, right, which
is equal to two and some
of the example B 3
2, right, that's B
3 2 is this element 9,
and that's equal to
a two three which is
this element up here, nine.
And so that wraps up
the definition of what it
means to take the transpose
of a matrix and that
in fact concludes our linear algebra review.
So by now hopefully you know
how to add and subtract
matrices as well as
multiply them and you
also know how, what are
the definitions of the inverses
and transposes of a matrix
and these are the main operations
used in linear algebra
for this course.
In case this is the first time you are seeing this material.
I know this was a lot
of linear algebra material all presented
very quickly and it's a
lot to absorb but
if you there's no need
to memorize all the definitions
we just went through and if
you download the copy of either
these slides or of the
lecture notes from the course website.
and use either the slides or
the lecture notes as a reference
then you can always refer back
to the definitions and to figure
out what are these matrix
multiplications, transposes and so on definitions.
And the lecture notes on the course website also
has pointers to additional
resources linear algebra which
you can use to learn more about linear algebra by yourself.
And next with these new tools.
We'll be able in the next
few videos to develop more powerful
forms of linear regression that
can view of a lot
more data, a lot more
features, a lot more training
examples and later on
after the new regression we'll actually
continue using these linear
algebra tools to derive more
powerful learning algorithims as well