1
00:00:00,000 --> 00:00:09,000
In fact, in many ways artificial
intelligence at web scale is already here.

2
00:00:09,420 --> 00:00:17,576
Think of machine translation between human
languages as a feat that was thought to be

3
00:00:17,576 --> 00:00:24,120
impossible after the failures in the early
days of AI in the sixties.

4
00:00:24,420 --> 00:00:30,142
Think of recognizing images such as in
Google Goggles.

5
00:00:30,142 --> 00:00:37,984
Or in recognizing faces such as happens
with Picasa or Facebook every day.

6
00:00:37,984 --> 00:00:45,932
Last but not least, think about Watson the
IBM program which defeated world

7
00:00:45,932 --> 00:00:54,410
champions, or rather at least US national
champions, at the Jeopardy quiz game in

8
00:00:54,410 --> 00:01:00,705
2009.
Certainly these tasks are worthy of being

9
00:01:00,705 --> 00:01:04,736
called.
Successful artificial intelligence

10
00:01:04,736 --> 00:01:09,244
applications.
In fact, if you think about the jeopardy

11
00:01:09,244 --> 00:01:15,622
example, is the human participants had
been behind a screen, like Watson, and

12
00:01:15,622 --> 00:01:19,620
communicated using computer generated
language.

13
00:01:19,980 --> 00:01:25,298
Then, could we have actually made out the
difference between the two?

14
00:01:25,298 --> 00:01:30,460
And in that sense, hasn't the Turing test
been successfully passed?

15
00:01:31,360 --> 00:01:36,800
To a certain extent, their full web scale,
artificial intelligence is already here.

16
00:01:37,100 --> 00:01:42,340
Now what about data?
What is all this about big data?

17
00:01:42,620 --> 00:01:48,255
Well, there are lots of webpages now.
There are a billion Facebook users and

18
00:01:48,255 --> 00:01:54,115
many more Facebook pages, hundreds of
millions of twitter accounts, hundreds of

19
00:01:54,115 --> 00:01:59,675
millions of tweets per day, billions of
Google queries per day, millions of

20
00:01:59,675 --> 00:02:02,981
servers, terabytes of data powering all
this.

21
00:02:02,981 --> 00:02:08,917
And clearly driving this explosion is
More's Law where computing power doubles

22
00:02:08,917 --> 00:02:13,932
every eighteen months.
In Criner's Law, which is that this

23
00:02:13,932 --> 00:02:18,640
capacity is growing even faster than
Moore's Law.

24
00:02:19,500 --> 00:02:25,640
In contrast with all these massive growth
and data, on the web.

25
00:02:25,900 --> 00:02:32,820
Typical large enterprises like banks,
retail companies.

26
00:02:33,420 --> 00:02:38,544
Or hospitals have.
Far fewer servers.

27
00:02:38,544 --> 00:02:44,980
Maybe the largest banks have a few
thousand or tens of thousands of servers.

28
00:02:45,260 --> 00:02:50,670
And terabytes of data rather than
petabytes, and only a few million

29
00:02:50,670 --> 00:02:54,305
transactions a day, nowhere near the
billions.

30
00:02:54,305 --> 00:03:00,523
As a result, the technology used by large
enterprises which, pretty much looks

31
00:03:00,523 --> 00:03:05,450
something like this.
Where we have a bunch of databases where.

32
00:03:05,450 --> 00:03:10,778
Data is collected, cleaned, put into
things called data warehouses.

33
00:03:10,778 --> 00:03:17,583
And then taken out into further databases
on which some statistics or reporting is

34
00:03:17,583 --> 00:03:21,273
performed.
This approach simply does not work.

35
00:03:21,273 --> 00:03:27,011
And Google, Facebook, LinkedIn, Ebay,
Amazon which needed a process large

36
00:03:27,011 --> 00:03:32,340
volumes of big data on the web did not use
traditional databases.

37
00:03:32,340 --> 00:03:37,260
In fact they could not, later in this
course we'll study why.

38
00:03:37,560 --> 00:03:42,500
As well as, what they.
Replace this technology with.

39
00:03:43,120 --> 00:03:48,369
In short.
They used massive parallelism, and a new

40
00:03:48,369 --> 00:03:54,429
programming paradigm for data processing
called map reduce, which is essentially

41
00:03:54,429 --> 00:03:58,520
the heart of what is today called big data
technology.