So, perhaps if we now return to the
question that we began this course with,
what does data have to do with
intelligence?
Any fool can know the point is to
understand and the goal of understanding
is to predict.
The brain, as we have seen, is largely a
prediction machine.
It also controls our bodies, but it
doesn't through prediction.
And we've learned how business systems,
web applications can use the techniques.
For predictive intelligence, that we've
learned in this course, which are looking,
listening, learning, connecting,
predicting and of course, correcting,
which we haven't covered.
So it's worthwhile recapping what we've
learned in each of these elements, just so
that one realizes that we've actually
covered a lot of ground, and to highlight
the most important points that we might
have gone through.
In Look, we talked about search, a very
important technique called locality
sensitive hashing.
In relationship of search, page rank,
etcetera, to memory, and touched upon
associative memories in response to
shuffled memories which we just realized
are crucial to things like hierarchical
and temporal memory, which is the latest
in predictive intelligence.
Went on to listen, we talked, we learned
about the naive base classifier and the
role of mutual information in figuring out
which features are better features and
which ones are not.
We went on to learn.
We looked at a unified frame work for
classification as well as clustering, rule
mining.
And then talked about how latent or hidden
models can be used to learn features and
classes together.
In coonect we covered reasoning, the
symantic web vision, how rules can be
learned from large volumes of text.
Base your networks for reasoning
uncertainty.
And finally in predict, we've talked about
linear regression, linear prediction, dual
networks, anarchical temporal memory, a
black board architecture.
And, of course, in the end an intelligent
system also has to translate these
predictions into actions through
corrections, which involves optimization
and planning, which we haven't had time to
cover this time.
Maybe next time.
Along the way we also learned about the
load element.
How large volumes of data and processing
can be executed on modern computer systems
emerging from the web.
We learned about map produce and the
evolution of data bases.
So, congratulations!
We've, we've really covered a lot and
learned a lot in this course, even if at a
high level.
I hope you all see how things fit
together.
My goal in this course has been to try to
convey the big picture.
How many different techniques are
different ways of looking at the same
thing.
Which techniques work well in which
situations, and hopefully you'll be ready
for some interesting challenges, which
I'll point to in a minute.
Before that let me point out some deep
research problems.
First one is looking.
It seems very simple, searching.
And indexing.
But then think about what's involved in
looking at data.
When I looked at a piece of data, I need
to figure out a lot of things.
What features am I going to extract from
this data?
What techniques am I going to use?
What insights do I think I can get out of
this?
These are all reasoning.
Elements.
Looking at data involves reason.
Today all that is done by people.
There are very few assistive decision
support systems to help people look at
data.
On a more abstract level,
I've already mentioned this.
How symbolic reasoning arises from
bottom-up, data driven techniques.
Be they neural or predictive,
classification, etcetera,
Clustering.
Where do the symbols emerge and where does
the rules and reasoning emerge?
Where does logic emerge?
These things are mysteries which we simply
haven't understood adequately.
And lastly, any web intelligence system.
Real intelligent system like the human
being or any business intelligence system
requires a purpose.
I'm certainly not suggesting that a
machine would acquire its own purpose.
I'm not suggesting for a moment that free
will is anywhere near our grasp in terms
of coding it.
But at least the level at which we code
our purpose is today, very low.
We're actually telling the system the real
pieces of the puzzle.
We're telling it how to look, how to
listen, how to learn.
How all the pieces fit together.
We're giving it the architecture.
We're not anywhere near, systems which
learn how to put these pieces together by
themselves, if given a go.
For example, if you were to design a
system which controls all the traffic in a
city along with self driving cars.
How would the system evolve?
How would the system make use of better
techniques to achieve its stated goals?
The stated goals are very clear.
It's given to, we give them, give, we give
the system its goals, but the system
itself is hard coded in terms of how it
puts, it puts different techniques
together to achieve them.
It's not able to reason about how it's
reasoning.
So it comes to my first problem about
reasoning about looking.
Can we give systems a higher level
purpose, and then let the systems figure
out the sub-goals, and figure out how
different techniques are put together?
I think these three problems are deep,
extremely difficult to, to tackle.
They can be looked at as pointers to very
specific research problems in thousands of
different ways.
And for graduate students looking at deep
problems in big deal analytics or
Artificial Intelligence, I think these are
good pointers.
For those of you looking for more
practical challenges I'd like to point you
to Kaggle, which is a site where
organizations, both governments and
companies, post data and problems about
that data and invite people to compete for
prizes if one can do some great data
analytics using that data.
There are lots of competitions up there,
for example.
There's this one about online product
sales, predicting the online sales or
consumer product based on its features.
So that data is used in the latest
programming assignment, there are others
about different topics such as very large
scale data mining using big data, posted
by Best Buy. We're talking about mobile,
web data, and figuring out which products,
user will be most interested in.
So, I think that you know, with all the
techniques that you've learned or at least
been exposed to in this course, you're
well versed to solve many of these data
mining challenges probably with a little
extra reading and a little extra, extra
work but you should be able to approach
this problems, know which algorithms to
use, which packages to look for, which
kinds of techniques to apply.
So finally, do remember that all remaining
quizzes, homeworks and programming
assignments included are due on the ninth
of November, when the course ends at
eleven:5959 PST.
The final exam is on Friday, the November
ninth.
The time will be announced on the on the
site.
And it will be open, until 23:5959 that
day.
Though it might be closed for a short
period of time after this particular
interval, so I can extract the specific
grades for IIT and Triple IT.
Finally, thanks for being such a great
class.
I hope you enjoyed the course.
I know there have been some, lapses in
terms of errors in homework assignments.
The arena we've covered is also rather
vast, and some of you may have lost track
along the way.
Do write about this course on say
coursedoc for example, so that the next
version of the course can be made much,
much better.
Contact me if anybody wants to research
any of the deep problems or enjoy yourself
doing great data challenges on Kaggle.
Best wishes and goodbye.