CMU scholars criticize Google: only recognize money! Machine learning has not improved for 20 years

"Machine learning is an amazing engineering achievement, but it is not science.”

"What this technology is doing now is no different from 1990. At most, it is bigger, but it does not give us a deeper insight than 20 years ago."

"Google is more powerful than many universities. The only reason is that each of their researchers is able to hire more than ten times more graduate students than others."

"At Google, everything is a business plan."

"You will be at the forefront of machine learning, but this is just an engineering discipline. All the basic goals are set by big companies, you can't be a scientist."

Not long ago, a CMU scholar, Simon DeDeo, slammed the Google Brain team on Twitter and ridiculed the for-profit machine learning industry. (In fact, it is also a ruthless attack on Facebook's research institutions.)

He criticized these big companies for focusing on artificial intelligence, just to get more money and profits, not to promote scientific progress. These views have caused widespread concern.

In the words of the famous media TNW, this is an epic discussion.

There are voices in favor and rebuttal. But Jeff Dean, head of Google AI at the Storm Center, did not speak. However, Facebook's chief AI scientist Yann LeCun stood up.

Fight

The cause of the incident is this.

Originally a bunch of people (again) criticized Facebook. Or Facebook should be split, including WhatsApp, Instagram, etc., should not be in the hands of Zuckerberg.

During the discussion, Simon DeDeo said that the biggest difference between AT&T and Facebook is that Bell Labs invented C language, UNIX, transistors, quantum Hall effect, cosmic microwave background radiation, etc.; and Facebook Research invented a new social network that makes you addicted. method.

Hearing this, someone interjected and said: You criticized Facebook's research institutions, then how do you evaluate Google Brain, or other scientists gathered in the lab?

It’s just like this, the war is on.

                                                                 △Simon DeDeo

Simon DeDeo seems to have cleared his throat and said: Since you ask, tell the truth. after all. I don't need Google's sponsorship, and I don't have to tie them up in my career.

Subsequently, he continued to send more than 30 tweets to form a essay. Mainly said these words:

I grew up in the Bayesian era. At that time scientists used some simple, theoretically driven equations to change our perception of the world. I also used this as a standard.

Around 2010, deep learning has become a must.

This is exciting. In the Institute, we heard interviewers explain the decision tree, the random forest, and so on. I have also tried these new methods, but to be honest, they are not very attractive, because other simpler tools can handle very, very many tasks.

Later, I joined Indiana University, and we were eager to recruit someone who was engaged in deep learning. I took all the candidates to have breakfast and wanted to understand what deep learning is.

The basic conclusion is: there is nothing good to answer. No matter what you do, basically it is a person sitting there, various adjustment parameters.

Machine learning is an amazing engineering achievement. But this is not science, far from it. What this technology is doing now is no different from 1990. At most, it is bigger, but it does not give us a deeper insight than 20 years ago.

Google is better than many universities, the only reason is that each of their researchers is able to hire more than ten times more graduate students than others. But research at the university is more academic.

They don't know what they are doing. Google has enough people to apply deep learning to everything, and the goal is to find the areas that can make the most impact.

I have visited about 50 universities and I have gained new things everywhere. When I visited the company's "research" lab, the situation was completely different.

Can you do some cool research in Google Brain? Honestly, no. You will be at the forefront of machine learning, but this is just an engineering discipline. All the basic goals are set by big companies, you can't be a scientist.

If you want to build a machine that monitors others and sells more ads to them, then go to the corporate lab.

If you want to make money, you can earn a lot. Not in the academic world.

But if your mind and soul want to be the driving force behind the progress of human wisdom, you better not go to Google, of course, don't go to Facebook.

I suggest that you come to school to study and become a doctor. You won't get a good income, but you will get a mentor who really cares about you.

It is shameful that an excellent doctoral tutor will not interfere with the students' thoughts and thoughts. At Google, everything is a business plan.

This is not a joke. It is my ten-year experience summary. In the fall, it is time to start a new postgraduate admission application. I suggest you think about where you are going.

Finally tell a story. We had previously visited Google Research, and the people there were incredibly clever. We brainstormed a lot of wonderful academic research ideas. On the last day of the meeting, college scholars said: OK! Let's go to the bar and talk about it!

Google’s people say: These days are equivalent to vacations for us, the actual work on hand has fallen, and this weekend must work overtime.

Yes, talking about academics is like a holiday for them.

Counterattack

This large stone has stirred up thousands of waves.

Oren Etzioni, CEO of the Allen Institute for Artificial Intelligence (AI2), commented that DeDeo's sniper simplifies things and is offensive, but it is also thought-provoking.

"Interesting. A lot of people have this kind of sharp and provocative remarks. They are very interesting," retorted Charles Sutton, a visiting scholar at Google Brain. Subsequently, he continued to defend the word "engineering" and pointed out that there have been many advances in machine learning.

On reddit, netizens also launched a hot discussion.

It has been said that DeDeo only encountered some research-oriented people in Google Research and summarized all the laboratories in the industry. Then take DeepMind as an example, only companies like Google can spend money on technology after decades.

Netizen jbcraigs also held objections. He said that Google has a large number of parts that are doing pure academic research. Since 2013, Geoffrey Hinton has been working for Google. There is no "leader" telling him what to study, as are other Google researchers.

Some people are puzzled: What is the motivation for companies to invest heavily in basic research?

“One of the important factors is PR. When the company reaches a certain scale, the cost of supporting 10 scholars to conduct research at random is almost insignificant, but it can attract more talents and improve their reputation.” User NichG gives his own opinion from this angle. view.

Despite the controversy surrounding the outside world, Jeff Dean, head of Google AI (including Google Research), has yet to respond.

Instead, Facebook’s chief AI scientist Yann LeCun couldn’t help but counterattack. He said to DeDeo:

You are seriously misled.

Yes, Google Brain has a lot of work focused on engineering and applications.

However, FAIR, DeepMind, and many of Google Brain's research-oriented departments are indeed doing actual scientific research.

In fact, the above-mentioned engineering and academic debates have a long history, both inside and outside.

Do you still remember? Last year, Google Brain team scientist Denny Britz once wrote a deep learning: the bottleneck is too much to pay attention to engineering!

When I was studying NLP and information extraction, most of the time, I used to turn scientific research ideas into code. At that time, 95% of graduate students and mentors were not willing to touch the code. When I ask a question about a question, the response is usually: "This is just an engineering problem, let it pass first."

Later I learned that the subtext of this rhetoric is: I don't think a paper related to this can be peer reviewed. This mentality seems to be widespread among people in academia. But as an engineer, I can't help but notice that there is a lack of obstacles in engineering practice.

Of course, there is no shortage of other opinions.

During the NIPS period last year, Ali Rahimi, who won the “Time of Value” award for “Test of Time”, gave a speech and made a shot to the entire deep learning community: machine learning has become an alchemy.

I hope that in the world I live in, these systems are built on rigorous, thorough, verifiable knowledge, not "alchemy."

I don't exclude the use of technologies that I don't understand. For example, I am flying. I don't fully understand how it works. But I know that the entire aviation industry is studying this technology.

But Ali said that he said that many of the basic tools for building deep neural networks "we know almost nothing about it." He criticized machine learning for engineering and lacked theoretical support.

Of course, LeCun jumped out as usual.

Details of that matter can be reviewed through this portal.

What opinions do you have in this debate?

And, what exactly is Google Brain studying?

Follow Me

Link:Tenco

                                                              ——END——