Still, we, I think there is something missing.
Even if we have a neural architecture which appears to predict time series very
well. Even if we have a variety of techniques
for prediction classification, On the one hand, reasoning and rules on
the other. The link between these is still missing.
For example, If you're trying to predict how other
players or pedestrians on the road will move, or if you're trying to predict the
consequences of a decision cuz when one takes a decision, one imagines the future.
If I did x, then y will happen, if I did z, then a will happen.
And we continuously imagining the future by playing things out in our head.
The missing element is that symbolic reasoning, optimization, planning.
These features, or these sort of techniques appear very different from the
regressions, or the neural learning, or sequence prediction, or naive-based
classification which essentially data driven predictions that we have seen.
Reasoning requires one to learn rules. One requires one to learn classes, and
then reason in a symbolic way about these things. And the link between how did a
driven bottom-up techniques, eventually give rise to higher level symbolic
reasoning in an architecture like the brain, is, is the missing link.
We still don't know how that happens. The hierarchical temple of memory promises
that, yes we'll, we're going to learn about this, but that's not been
demonstrated yet. So, in the absence of that link being
there, there are other ways to put these different techniques together in practical
systems. The most popular one is called a
blackboard architecture and it's a very old technique going back to the 50's.
And is now starting to get used increasingly in complex AI systems which
require, which need to use many different techniques.
Some bottom up data driven, some top down symbolic reasoning oriented,
And this is how the black board architecture works.
The blackboard architecture consists of a blackboard where knowledge or, or that
what one learns about the world is posted. And this, this knowledge is posted by
knowledge sources. Now, knowledge sources can be of many
types. They could be bottom-up feature learning,
clustering, sequence miners like HTM, classifiers.
So things which learn from the data directly, or there could be symbolic rule
engines or decision engines which do planning or, or reasoning,
And they operate on a common blackboard.. So, the lower level data driven knowledge
sources might learn something about the world, like what are the features to look
for, what are the classes, what are the rules.
And then, higher level rule engines might operate on these rules to perform
reasoning, do planning, and take decisions.
And a controller looks at the blackboard and tries to figure out based on what is
available on the black board, what kinds of knowledge sources would be most
applicable to the kinds of stuff which are on the blackboard..
So, This is a way of putting different types
of machine learning techniques, reasoning techniques together in one architecture.
It's a hierarchical system, and some blackboard systems are also Bayesian in
the sense that if the two different elements are on the blackboard,.
Placing a third element might make the probability that one of the older
elements, which was already deemed to be true, become less true through something
like the explaining away effect. So, those are called Bayesian blackboards.
Some examples of blackboards are the earliest, one of the earliest examples is
speech recognition. The first speech recognition systems used
blackboard reasoning. So, the lower levels of the blackboard
would detect things like of phonemes, And then higher levels would detect words,
And even higher levels would talk about sentences.
And at each level, One is not only going bottom up, but one
is using the predictions at the higher level, layers to drive the reasoning at,
or the classification at the lower layer. So, the likelihood of the next word being
a particular one is driven by what the previous word is and that, as we have seen
in, in, in a few lectures back, Well that also drives what phonemes to
look for. So, lower level classifiers are adjusted
based on what possible words are most likely in this particular higher level
context. So, that's how speech recognition systems
have used this hierarchical reasoning fairly effectively.
There are other systems which are do, deal with analogical reasoning which are
essentially ways of trying to mimic analogy, like who is the Dhoni of USA.
So, how do you map different frames of reference to different contexts through
analogies? I'd like to show you an example of an
analogical reasoning system, or at least one that tries to mimic analogical
reasoning. This one is due to a student of Hofstadter
called Melanie Mitchell.. Hofstadter, if you remember was the author
of the Pulitzer Prize winning book Godel, Escher, Bach, which many of you might have
read. It's an old book, about more than 30 years
old. Melanie Mitchell, his student, has
recently written a book called Complexity which is also a very interesting
exposition of variety of areas in artificial intelligence and complex
systems. Well,
Let's look what analogical systems reasoning works in the copycat program of
Melanie Mitchell. The analogy one is trying to mimic is, if
you're given a transformation between a, b, c which takes a, b, c to a,
B, d. Its like a puzzle. What would you deem as the analogous
transformation of i, j, k? Well, Think about it.
Let's see what the system does. It's reasoning.
It's trying to find out what the analogy is between these two and apply that same
analogy to this particular strain. And it figures out that analogy is
replaced the letter category of the right most letter by it's successo, and it came
out with i, J, l. Let's try it again.
This time we'll give it a problem, a, b, c goes to b, b, c and see what it comes up
with. The blackboard architecture is reasoning,
different types of rules are being applied in a hierarchy and each rule is affecting
what to look at next. And it comes up with j,
J, k. Replace the category of the left most letter by its successor.
It has learned the analogy. So, as we can see, the blackboard systems
are extremely powerful. And they do form a way of marrying the bottom-up data driven
reasoning with the top-down symbolic reasoning, and allowing both of these to
influence each other just as Bayesian networks and hierarchical temporal memory,
all also include this element of top-down, bottom-up reasoning working together.
So, we will now end the course with a recap.
I hope you've enjoyed this lecture. I've tried to cover many exciting things,
at least things which I find extremely exciting and promising.
A few things in a little detail, like linear regression and the ability to, to
predict values using regression and maybe even other techniques like logistic and
SVM, if you use packages. And then, some more speculative AI
aspects, and how they come together for big data analytics