In fact, in many ways artificial intelligence at web scale is already here.
Think of machine translation between human languages as a feat that was thought to be
impossible after the failures in the early days of AI in the sixties.
Think of recognizing images such as in Google Goggles.
Or in recognizing faces such as happens with Picasa or Facebook every day.
Last but not least, think about Watson the IBM program which defeated world
champions, or rather at least US national champions, at the Jeopardy quiz game in
2009. Certainly these tasks are worthy of being
called. Successful artificial intelligence
applications. In fact, if you think about the jeopardy
example, is the human participants had been behind a screen, like Watson, and
communicated using computer generated language.
Then, could we have actually made out the difference between the two?
And in that sense, hasn't the Turing test been successfully passed?
To a certain extent, their full web scale, artificial intelligence is already here.
Now what about data? What is all this about big data?
Well, there are lots of webpages now. There are a billion Facebook users and
many more Facebook pages, hundreds of millions of twitter accounts, hundreds of
millions of tweets per day, billions of Google queries per day, millions of
servers, terabytes of data powering all this.
And clearly driving this explosion is More's Law where computing power doubles
every eighteen months. In Criner's Law, which is that this
capacity is growing even faster than Moore's Law.
In contrast with all these massive growth and data, on the web.
Typical large enterprises like banks, retail companies.
Or hospitals have. Far fewer servers.
Maybe the largest banks have a few thousand or tens of thousands of servers.
And terabytes of data rather than petabytes, and only a few million
transactions a day, nowhere near the billions.
As a result, the technology used by large enterprises which, pretty much looks
something like this. Where we have a bunch of databases where.
Data is collected, cleaned, put into things called data warehouses.
And then taken out into further databases on which some statistics or reporting is
performed. This approach simply does not work.
And Google, Facebook, LinkedIn, Ebay, Amazon which needed a process large
volumes of big data on the web did not use traditional databases.
In fact they could not, later in this course we'll study why.
As well as, what they. Replace this technology with.
In short. They used massive parallelism, and a new
programming paradigm for data processing called map reduce, which is essentially
the heart of what is today called big data technology.