In this video I'm going to talk about the issue of viewpoint invariance.
Each time we look at an object in the seen, we typically have a different view
point. So the object shares up on different
pixel. This makes okay recognition very unlike
most machine learning tasks and I'm going to talk about various ways of trying to do
with that issue. And number of different ways have been
suggested for coping with view point variations.
We're so good at it that we don't really appreciate how difficult it is.
It's one of the main difficulties in making computers perceive, and there still
aren't generally accepted solutions, either in engineering or in psychology.
The first approach is to use redundant invariant features.
The second approach is to put a box round the object so that you can normalize the
pixels. The third approach is to use replicated
features, and pool them. This is called convolutional neural nets.
I'll go to that in great detail. And the first approach we shall talk about
at the end of the lecture is to use a hierarchy of parts and to explicitly
represent the places of the parts relative to the camera or retina.
So, the invariant feature approach says you should extract a large and redundant
set of features. And they should be features that are
invariant under the transformations like translation and rotational scaling.
So here's an example of an invariant feature.
It's a pair of roughly parallel lines, with a red dot between them.
That's actually being suggested as the feature the baby herring gulls use for
knowing what to peck for food. If you paint that feature on a piece of
wood, they'll peck at the appropriate place on the piece of wood.
With enough invariant features, there's only one way to assemble them into an
object or an image. You don't actually need to represent the
relationships between features directly because those relationships are captured
by other features. This has been pointed out for strings of
letters by psychologist called Wayne [UNKNOWN] it's been pointed out in vision
by Shimon Ullman. And, it's a sort of acute point that all
we need is a big bag of features, because with overlapping and redundant features.
One feature will tell you how two other features are related.
Unfortunately, if you're doing recognition, you're going to get a whole
bunch of features that are composed of parts of different objects.
And they'll be very misleading for recognition.
So you'd like to avoid forming features from parts of different objects.
A second approach is what I call judicious normalization.
So if you look at that upside down capital letter R on the right, I put a box around
it. Not very well, in fact.
And I've labeled a top and a front for that box.
And relative to that box, the R has for example a vertical stroke at the back, and
it has a loop facing forwards at the top. So if we describe features of the r
relative to that box, they're going to be invariant.
This is assuming it's a rigid shape. Putting a box around a rigid shape solves
the dimension hopping problem. It gets rid of the effect of changes in
viewpoint. If we choose the box correctly, the same
part of an object would always occur on the same normalized pixels.
It doesn't have to be a rectangular box. We can provide invariance to not only
translation and rotation scale but also things like sheer and stretch.
Unfortunately, choosing the box is difficult.
It's difficult because we might have segmentation errors.
We might have occlusion so you can't just shrink a box around things.
We might have unusual orientations. That example of the upside down R makes it
clear that we have to use our knowledge of what the shape is to help us decide what
the box is. If, for example, we had a character that
was like a lowercase D, but with an extra stroke coming out of the loop of the D.
We would see that as an upright one of those characters.
So it's a chicken and egg problem. In order to get the box right, we need to
recognize the shape. In order to recognize the shape, we need
to get the box right. An aside here for psychologists.
Many psychologists think we do mental rotation to deal with shapes that aren't
oriented right. This is complete nonsense.
That capital letter R you recognize perfectly well before you do any mental
rotation. Indeed, you need to recognize that it's an
R and it's upside down, in order to know how to rotate it.
You use mental rotation for dealing with judgments like handedness.
That is, is it a correct R or mirror image R?
You can't tell that without doing mental rotation.
The mental rotation is not used for dealing with the fact that it's upside
down when we want to recognize it. The brute force normalization approach
works like this. You use well segmented, upright images
that you can judiciously put a box around when you train the recognizer, and then at
test time, when you have to deal with cluttered images, you try all possible
boxes in a whole range of positions and scales.
This approach is widely used in computer vision.
Particularly for detecting upright things like faces or house numbers in unsegmented
images. It's much more efficient if they recognize
they can cope with some variation in the position and scale so that we can use a
course grid when trying on possible boxes.