1 00:00:02,270 --> 00:00:05,000 Welcome, Yuanqing. I'm really glad you could join us today. 2 00:00:05,000 --> 00:00:05,710 Sure. 3 00:00:05,710 --> 00:00:13,200 Today you are the head of IT research and when the Chinese government, 4 00:00:13,200 --> 00:00:16,420 the government of China, was looking for someone to start 5 00:00:16,420 --> 00:00:20,040 up and build a National Deep Learning Research Lab, 6 00:00:20,040 --> 00:00:23,255 they tapped you to help start this thing. 7 00:00:23,255 --> 00:00:25,953 So, you know, arguably, I think maybe you're 8 00:00:25,953 --> 00:00:31,075 the number one deep learning person in the entire country of China. 9 00:00:31,075 --> 00:00:34,295 I'd like to ask you a lot questions about your work, 10 00:00:34,295 --> 00:00:36,955 but before that, I want to hear about your personal story. 11 00:00:36,955 --> 00:00:41,075 So how did you end up getting to do this work that you do? 12 00:00:41,075 --> 00:00:46,020 Yeah, so, actually, before my Ph.D. program, 13 00:00:46,020 --> 00:00:48,450 my major was in Optics, 14 00:00:48,450 --> 00:00:50,730 it's more like in Physics. 15 00:00:50,730 --> 00:00:55,050 I think I had a fairly good kind of background, 16 00:00:55,050 --> 00:00:58,922 a very good background, on maths. 17 00:00:58,922 --> 00:00:59,940 After I came to the US, 18 00:00:59,940 --> 00:01:02,010 then I was thinking, 19 00:01:02,010 --> 00:01:05,835 what kind of major can I take for my Ph.D. program? 20 00:01:05,835 --> 00:01:07,110 I was thinking, well, 21 00:01:07,110 --> 00:01:11,870 I guess I could go for Optics or go for something else. 22 00:01:11,870 --> 00:01:14,100 Back to like early 2000, 23 00:01:14,100 --> 00:01:17,640 I think nanotechnology was very very hot. 24 00:01:17,640 --> 00:01:24,087 But I was thinking that probably I should look at something even more exciting. 25 00:01:24,087 --> 00:01:27,150 And that leaves a good chance that I was taking 26 00:01:27,150 --> 00:01:30,740 some classes at UPenn and I got to know Dan Lee. 27 00:01:30,740 --> 00:01:33,906 So later, he become my Ph.D. adviser. 28 00:01:33,906 --> 00:01:38,170 I was thinking, machine learning was a great thing to do. 29 00:01:38,170 --> 00:01:42,000 And I got really excited and I changed my major. 30 00:01:42,000 --> 00:01:46,245 So, I did my Ph.D at UPenn. 31 00:01:46,245 --> 00:01:48,030 I majored in machine learning. 32 00:01:48,030 --> 00:01:55,245 I was there for five years and it was kind of 33 00:01:55,245 --> 00:01:59,290 very exciting and I 34 00:01:59,290 --> 00:02:03,015 learned a lot of things from scratch and lots of algorithms, even like PCAs. 35 00:02:03,015 --> 00:02:05,228 I didn't know all those before. 36 00:02:05,228 --> 00:02:08,305 I feel like I was learning new things every day. 37 00:02:08,305 --> 00:02:10,440 So it was a very, very exciting experience for me. 38 00:02:10,440 --> 00:02:12,571 This was one of those things of a lot of starts. 39 00:02:12,571 --> 00:02:15,140 Although, you know, you just did a lot of 40 00:02:15,140 --> 00:02:18,390 work and it was an underappreciated for its time. 41 00:02:18,390 --> 00:02:21,857 Right, that's right. Yes. So I think NEC was exciting place, 42 00:02:21,857 --> 00:02:24,480 and I was there at the beginning as a researcher. 43 00:02:24,480 --> 00:02:27,617 Again, I also like to feel like, whoa. 44 00:02:27,617 --> 00:02:30,370 I learned lots of things. 45 00:02:30,370 --> 00:02:34,132 Actually, later at NEC, 46 00:02:34,132 --> 00:02:36,150 I started working on computer visions. 47 00:02:36,150 --> 00:02:43,110 I actually started working on computer vision very late, relatively late. 48 00:02:43,110 --> 00:02:45,304 And the first thing I did was 49 00:02:45,304 --> 00:02:48,970 I participated in ImageNet Challenge. That's the first year of ImageNet Challenge. 50 00:02:48,970 --> 00:02:56,420 I was kind of managing a team to work on a project. 51 00:02:56,420 --> 00:03:00,950 It was lucky, we were quite lucky that we were quite strong, 52 00:03:00,950 --> 00:03:02,575 and in the end, 53 00:03:02,575 --> 00:03:05,460 we actually got the number one place. 54 00:03:05,460 --> 00:03:08,985 Overwhelming, number one place in the contest. 55 00:03:08,985 --> 00:03:13,185 So you are the first ever in the world ImageNet Competition winner? 56 00:03:13,185 --> 00:03:17,890 Right. Yeah, and I was the person that did a presentation at that workshop. 57 00:03:17,890 --> 00:03:20,840 It was a really nice experience for me, 58 00:03:20,840 --> 00:03:28,620 and that actually get me into Lisa [inaudible] computer vision tasks. 59 00:03:28,620 --> 00:03:34,115 I had been working on Liza [inaudible] probably since then. 60 00:03:34,115 --> 00:03:42,930 When New York Times head paper came out, 61 00:03:42,930 --> 00:03:45,030 and also later on AlexNet came out, 62 00:03:45,030 --> 00:03:47,328 it really blow my mind. 63 00:03:47,328 --> 00:03:51,610 I thought, wow, deep learning is so powerful. 64 00:03:51,610 --> 00:03:58,270 Since then, I put a lot of effort to work on those. 65 00:03:58,270 --> 00:04:00,705 So, as a head of China's National Lab, 66 00:04:00,705 --> 00:04:02,535 National Research Lab on deep learning, 67 00:04:02,535 --> 00:04:05,933 there must be a lot of exciting activities going on there. 68 00:04:05,933 --> 00:04:08,715 So, for the global audience, who are watching this, 69 00:04:08,715 --> 00:04:12,070 what should they know about what's happening with this National Lab? 70 00:04:12,070 --> 00:04:14,753 The mission of this National Engineering Lab is to 71 00:04:14,753 --> 00:04:18,275 build a really large deep learning platform, 72 00:04:18,275 --> 00:04:20,865 and hopefully be the biggest one, 73 00:04:20,865 --> 00:04:23,780 or at least the biggest one in China. 74 00:04:23,780 --> 00:04:29,900 And this platform would offer people deep learning framework like Pelo Pelo. 75 00:04:29,900 --> 00:04:34,573 And we offer people really large scale computing resource, 76 00:04:34,573 --> 00:04:39,687 and we also offer people really large, really big kind of data, 77 00:04:39,687 --> 00:04:43,132 and if people are able to develop 78 00:04:43,132 --> 00:04:47,330 a research or develop good technology on these platform, 79 00:04:47,330 --> 00:04:50,580 we also offer them big applications. 80 00:04:50,580 --> 00:04:54,370 So for example, the technology can be proved into some big applications in Baidu 81 00:04:54,370 --> 00:04:58,650 so the level of technology could get integrate and improve it. 82 00:04:58,650 --> 00:05:06,565 So, we believe that combining those resources altogether, 83 00:05:06,565 --> 00:05:10,225 I think, is going to be a really powerful platform. 84 00:05:10,225 --> 00:05:14,130 You can also get one example on each. 85 00:05:14,130 --> 00:05:18,805 For example, right now, if we publish a paper, 86 00:05:18,805 --> 00:05:20,740 and someone want to reproduce it, 87 00:05:20,740 --> 00:05:27,162 the best thing to do is to provide a code to somewhere, 88 00:05:27,162 --> 00:05:28,745 and you could download the code to 89 00:05:28,745 --> 00:05:33,460 your computer and that you also try and find that the data sets somewhere. 90 00:05:33,460 --> 00:05:42,060 And you probably also need to get good [inaudible] for your computing resources to run smoothly. 91 00:05:42,060 --> 00:05:46,440 So this will easily take you some effort at least. 92 00:05:46,440 --> 00:05:51,135 National Lab things will become much easier. 93 00:05:51,135 --> 00:05:54,760 So if someone's using these platform to write the paper, 94 00:05:54,760 --> 00:05:56,440 to do that work and write a paper, 95 00:05:56,440 --> 00:06:00,715 and the lab who have the code on these platform 96 00:06:00,715 --> 00:06:06,340 and the computing structure is already set up for this code, and data's allowed too. 97 00:06:06,340 --> 00:06:09,670 So basically, you just need a common line to lift the database up. 98 00:06:09,670 --> 00:06:19,570 So, this is a huge relief for loss of reproducibility issues in combination with science. 99 00:06:19,570 --> 00:06:23,430 So you easily, just few seconds, 100 00:06:23,430 --> 00:06:26,020 you can start learning something that you see in a paper. 101 00:06:26,020 --> 00:06:28,610 Yeah. So this is extremely powerful. 102 00:06:28,610 --> 00:06:33,970 So this is just one example that we are working on to make sure that we 103 00:06:33,970 --> 00:06:39,690 are providing a really powerful platform to the community and to the industry. 104 00:06:39,690 --> 00:06:43,395 That's amazing. That really speeds up deep learning research. 105 00:06:43,395 --> 00:06:44,985 Yeah. 106 00:06:44,985 --> 00:06:48,605 Can you give a sense of how much resources 107 00:06:48,605 --> 00:06:53,700 the Chinese government is putting to back you for this Deep Learning National Lab? 108 00:06:53,700 --> 00:06:57,275 So, I think that for this National Engineering Lab, 109 00:06:57,275 --> 00:07:04,165 I think government can invest some funding here to build up infrastructure. 110 00:07:04,165 --> 00:07:05,320 But I think more importantly, 111 00:07:05,320 --> 00:07:07,675 these are going to be 112 00:07:07,675 --> 00:07:13,820 a flagship in China that are going to lead a lots of deepening the efforts, 113 00:07:13,820 --> 00:07:17,255 including like national project, 114 00:07:17,255 --> 00:07:21,190 and the laws of policies, and things. 115 00:07:21,190 --> 00:07:23,835 So this is actually extremely powerful, 116 00:07:23,835 --> 00:07:25,110 and I think by doing that, 117 00:07:25,110 --> 00:07:28,360 we are really honored to get this lab. 118 00:07:28,360 --> 00:07:32,238 You are somewhat at the heart of deep learning in China. 119 00:07:32,238 --> 00:07:34,855 So, there's a lot of activity in China that 120 00:07:34,855 --> 00:07:38,850 the global audience isn't aware of yet and hasn't seen yet, 121 00:07:38,850 --> 00:07:43,545 so what should people outside China know about deep learning in China? 122 00:07:43,545 --> 00:07:46,420 Yeah. I think in China, 123 00:07:46,420 --> 00:07:48,575 especially in the past few years, 124 00:07:48,575 --> 00:07:53,790 I think deepening empowered a product, 125 00:07:53,790 --> 00:07:57,084 so it's really booming, 126 00:07:57,084 --> 00:08:00,470 ranging from search engines, 127 00:08:00,470 --> 00:08:03,890 to, like, a phrase recognition, 128 00:08:03,890 --> 00:08:09,320 surveillance, to the e-commerce, lots of place. 129 00:08:09,320 --> 00:08:14,270 I think that they are investing big effort into deep learning and 130 00:08:14,270 --> 00:08:22,515 also really make use of technology to make the business much more powerful. 131 00:08:22,515 --> 00:08:32,810 And this actually is very important for developing a high technology in general. 132 00:08:32,810 --> 00:08:34,300 I think for myself, 133 00:08:34,300 --> 00:08:37,175 and also lots of people share this, 134 00:08:37,175 --> 00:08:42,730 we believe that actually it's really important to this, what's often called [inaudible] loop. 135 00:08:42,730 --> 00:08:46,990 For example, when we start out to 136 00:08:46,990 --> 00:08:51,185 think of building some technologies that will have some initial data, 137 00:08:51,185 --> 00:08:53,960 and which we try to do with some initial algorithm, it will launches 138 00:08:53,960 --> 00:08:56,965 our initial product for that service. 139 00:08:56,965 --> 00:09:00,310 Then, after, later, that will get the data for users. 140 00:09:00,310 --> 00:09:03,070 And the others will get more data, 141 00:09:03,070 --> 00:09:09,085 so we would develop a better algorithms. 142 00:09:09,085 --> 00:09:12,910 Because we see more data, we know what would be the better algorithm. 143 00:09:12,910 --> 00:09:14,760 So we have more data and a better algorithm, 144 00:09:14,760 --> 00:09:19,604 we will be able to have better technology for product service, 145 00:09:19,604 --> 00:09:23,600 and then definitely we hope that we will be able attract more users using the product. 146 00:09:23,600 --> 00:09:25,200 The technology is better. 147 00:09:25,200 --> 00:09:26,870 And then we will get more data. 148 00:09:26,870 --> 00:09:29,309 So this is a very good, positive move. 149 00:09:29,309 --> 00:09:30,750 And this is very special, 150 00:09:30,750 --> 00:09:34,745 especially for AI related technologies, 151 00:09:34,745 --> 00:09:38,685 for traditional technology like a laser. 152 00:09:38,685 --> 00:09:41,335 I was working on that before. 153 00:09:41,335 --> 00:09:45,955 So the course of technology is going to be more linear. 154 00:09:45,955 --> 00:09:47,505 But before, this AI technology, 155 00:09:47,505 --> 00:09:49,415 because of this positive loop, 156 00:09:49,415 --> 00:09:56,330 you can imagine that definitely, [inaudible] come with really fast growth of technology. 157 00:09:56,330 --> 00:10:03,235 And [inaudible] is actually super important when we design a research into them, 158 00:10:03,235 --> 00:10:04,862 when we design our ND. 159 00:10:04,862 --> 00:10:12,550 We work on the direction that we're able to get to this quick improvement period. 160 00:10:12,550 --> 00:10:21,415 But if our whole business were not able to fund these positive loop, 161 00:10:21,415 --> 00:10:30,350 if we are not able to fund this strong positive loop, 162 00:10:30,350 --> 00:10:32,090 this will not work out because someone else will 163 00:10:32,090 --> 00:10:34,160 have a strong vision to fund this strong loop 164 00:10:34,160 --> 00:10:38,620 and they will get to this place much more quickly than you are. 165 00:10:38,620 --> 00:10:44,190 So this [inaudible] an important logic for us when we're looking at, 166 00:10:44,190 --> 00:10:45,630 say, hey, you need 167 00:10:45,630 --> 00:10:47,030 a company, what direction should we work on, and what direction we should not work on. 168 00:10:47,030 --> 00:10:51,140 This is definitely a really important factor to look at. 169 00:10:51,140 --> 00:10:53,540 Today, both in China, 170 00:10:53,540 --> 00:10:55,353 in the U.S., and globally, 171 00:10:55,353 --> 00:11:00,420 there are a lot of people wanting to enter deep learning and wanting to enter AI. 172 00:11:00,420 --> 00:11:04,655 What advice would you have for someone that wants to get into this field? 173 00:11:04,655 --> 00:11:10,790 So now, there's definitely people who will start with open source frameworks. 174 00:11:10,790 --> 00:11:16,080 I think that's extremely powerful to many starters. 175 00:11:16,080 --> 00:11:20,265 I think when I was studying my deep learning study, 176 00:11:20,265 --> 00:11:23,530 there was not much open source resources. 177 00:11:23,530 --> 00:11:27,490 I think nowadays, in AI, especially in Deep learning, 178 00:11:27,490 --> 00:11:29,850 it's a very good community, 179 00:11:29,850 --> 00:11:36,010 and there are multiple really good people in the frameworks. 180 00:11:36,010 --> 00:11:40,055 It's always [inaudible] , a cafe. 181 00:11:40,055 --> 00:11:43,010 Now they call it cafe too, right? In China we have a really good Pelo-pelo. 182 00:11:43,010 --> 00:11:50,415 And even in most [inaudible] online, 183 00:11:50,415 --> 00:11:55,970 they have lots of courses to teach you how to use those. 184 00:11:55,970 --> 00:11:57,840 And also, nowadays, there's also 185 00:11:57,840 --> 00:12:02,285 many publically available benchmark and the people will see, 186 00:12:02,285 --> 00:12:06,740 hey, really skillful, really experienced people like, 187 00:12:06,740 --> 00:12:09,430 how well they could do on those benchmark. 188 00:12:09,430 --> 00:12:13,402 So basically, time to get familiar with deep learning. 189 00:12:13,402 --> 00:12:15,725 I think those are really good starting point. 190 00:12:15,725 --> 00:12:18,245 How did you gain that understanding? 191 00:12:18,245 --> 00:12:20,555 Actually, I do it the opposite way. 192 00:12:20,555 --> 00:12:25,658 I learned PCA, I learned LDA, 193 00:12:25,658 --> 00:12:30,100 all of those before I learned deep learning actually. 194 00:12:30,100 --> 00:12:35,144 So, basically, it's also a good way, I feel. 195 00:12:35,144 --> 00:12:38,850 We kind of lay down lots of foundations. 196 00:12:38,850 --> 00:12:41,495 We learned graphic model. 197 00:12:41,495 --> 00:12:43,390 So these are all important. 198 00:12:43,390 --> 00:12:48,950 Although right now, deep learning is beyond [inaudible]. 199 00:12:48,950 --> 00:12:51,150 But knowing laws, will actually give you very good intuition 200 00:12:51,150 --> 00:12:52,985 about how deep learning works. 201 00:12:52,985 --> 00:12:54,330 And then one day, 202 00:12:54,330 --> 00:13:01,260 probably leads connection of deep learning into laws like a framework or approach. 203 00:13:01,260 --> 00:13:03,745 I think there are already lots of connections. 204 00:13:03,745 --> 00:13:07,815 And the laws actually make deep learning richer. 205 00:13:07,815 --> 00:13:11,410 I mean, there are richer ways of doing Deep Learning. 206 00:13:11,410 --> 00:13:18,390 Yeah. So, I feel like it's good to start with lots of open source, 207 00:13:18,390 --> 00:13:21,690 and those are extremely powerful resource. 208 00:13:21,690 --> 00:13:27,510 I will also suggest that you also do learn those basic things about machine learning. 209 00:13:27,510 --> 00:13:29,025 So, thank you. That was fascinating. 210 00:13:29,025 --> 00:13:30,600 Even though I've known you for a long time, 211 00:13:30,600 --> 00:13:32,525 there are a lot of details you're thinking that I 212 00:13:32,525 --> 00:13:34,760 didn't realize until now. So, thank you very much. 213 00:13:34,760 --> 00:13:36,480 Thanks so much for having me.