[MUSIC] When specifying a Gaussian distribution
over two random variables, we assign probability density for the pair
of random variables everywhere over the range of values that each
of these variables can take. And as in 1D, there's just a single
hump to this distribution. So for example,
here this would be the place where there's most probability. So it's most likely that the blue and
green random variables. So the intensities of blue and green in the images
fall within this range. Whatever, if you were to drop down this
point and look at the blue value and the green value, that would represent the mean of this distribution in
the most probable region because in a Gaussian the mean is also equivalent
to the mode of the distribution. And when we walk away from that mode, we are walking to regions
of lower probability. And another way to view a Gaussian
distribution in two dimensions, and the one that's more commonly used because
of being able to see it on 2D plots, is what's called a contour plot. And a contour plot is just a bird's
eye view of this mesh plot, this three-dimensional mesh plot. So you take this three-dimensional mesh,
you look down at it from the very top and you plot these curves,
these ellipses of equal probability. Where the coloring
represents the intensity so how high it was in probability. So again this center point is
the region of highest probability. And this blue shading represents low
probability just like in the mesh plot. So this ring would be lower probability. And we see lower probability as
we walk out in this direction or in this direction. Or in this direction, or
any direction moving away from the peak. The question is just how
rapidly it drops off. And it drops off most
rapidly in this direction, least rapidly in this direction here. And somewhere in between, between these
two different directions that we've shown. Okay, so this contour plot is going to be
our standard representation of a Gaussian in two dimensions. In two dimensions a Gaussian is fully
specified by a mean of vector and the covariance matrix. So this mean vector has elements
that center the distribution along every dimension. So for example in this case,
mu1 centers the distribution along the blue axis so the blue intensity. And mu2 centers the distribution
along the green intensity. And then the crosshairs pinpoint
the center of the distribution jointly in the blue and green intensity space. Then the covariance matrix
specifies the spread and orientation of the distribution. Along the diagonal of this covariance
matrix we have the variance terms for each of the dimensions. So the top left hand corner we just use
sigma blue squared, so that's the variance along just the blue intensity direction or
dimension of the observation vector. And likewise for sigma green squared, that's the variance just
along that one dimension. But then we have this other parameter,
sigma blue, green, or green, blue and actually those are the same
value because it's a symmetric matrix, that's not actually a detail
you really need to know. But anyway those off-diagonal
terms specify is the correlation structure of this distribution, so
what's the orientation of these ellipses. So let's look at a few examples
of covariance structures that we could specify. So one is a diagonal covariance with
equal elements along that diagonal. First, by specifying
a diagonal covariance, what we're seeing is that
there's no correlation between our two random variables, because
there's zeroes in the off diagonals. And what this means is that having
one random variable like this blue intensity, being high or
low doesn't relate to whether the other random variable like the green
intensity is high or low. And furthermore by having equal values of
the variances along the diagonal where we end up where this is circular shape to
the distribution because we are saying that the spread along each one of these
two dimensions is exactly the same. In contrast, if we were to specify
different variances along the diagonal, what this means is that the spread in
each of these dimensions is different and so what we end up with are these
axis-aligned ellipses. And finally, if we consider a full
covariance that allows for correlation between our two random variables we can
provide these non-access aligned ellipses. So in this example that we're showing
here, if the blue intensity is high, it's more likely that the green
intensity is also high. So these would be what are called
positively correlated random variables. And if the blue intensity is low, it's more likely that the green
intensity if also low. Okay well, we can define a Gaussian in any
number of dimensions, not just one or two. And in this case, we have a mean vector
where the number of elements in that vector is exactly the same as
the number of dimensions that we're in, the dimensionality of the random
vector x that we're looking at. And this covariance matrix has dimension. Let's say we're looking at a random
vector x that has d different dimensions then our covariance matrix is
going to be a d by d matrix. They're going to be d variance parameter
along the diagonal of that matrix. And then these off-diagonals are going to
capture this correlation structure amongst these d different random variables. The matrix is also something that's
called positive semidefinite. But again, if you don't know
what that means, that's okay. I just want to put that out there as
a word of warning is that there are some constraints on what the elements
of this matrix can be. Okay, but notationally, we're going to use
a very similar notation to what we used in the one-dimensional case. So we have this N of x, and then this bar. But now we have mu and this capital
sigma instead of just the variance term, little sigma, squared. [MUSIC]