Machine Learning (Theory)

6/4/2018

When the bubble bursts…

Tags: AI,Machine Learning jl@ 2:39 pm

Consider the following facts:

  1. NIPS submission are up 50% this year to ~4800 papers.
  2. There is significant evidence that the process of reviewing papers in machine learning is creaking under several years of exponentiating growth.
  3. Public figures often overclaim the state of AI.
  4. Money rains from the sky on ambitious startups with a good story.
  5. Apparently, we now even have a fake conference website (https://nips.cc/ is the real one for NIPS).

We are clearly not in a steady-state situation. Is this a bubble or a revolution? The answer surely includes a bit of revolution—the fields of vision and speech recognition have been turned over by great empirical successes created by deep neural architectures and more generally machine learning has found plentiful real-world uses.

At the same time, I find it hard to believe that we aren’t living in a bubble. There was an AI bubble in the 1980s (before my time), a techbubble around 2000, and we seem to have a combined AI/tech bubble going on right now. This is great in some ways—many companies are handing out professional sports scale signing bonuses to researchers. It’s a little worrisome in other ways—can the field effectively handle the stress of the influx?

It’s always hard to say when and how a bubble bursts. It might happen today or in several years and it may be a coordinated failure or a series of uncoordinated failures.

As a field, we should consider the coordinated failure case a little bit. What fraction of the field is currently at companies or in units at companies which are very expensive without yet justifying that expense? It’s no longer a small fraction so there is a chance for something traumatic for both the people and field when/where there is a sudden cut-off. My experience is that cuts typically happen quite quickly when they come.

As an individual researcher, consider this an invitation to awareness and a small amount of caution. I’d like everyone to be fully aware that we are in a bit of a bubble right now and consider it in their decisions. Caution should not be overdone—I’d gladly repeat the experience of going to Yahoo! Research even knowing how it ended. There are two natural elements here:

  1. Where do you work as a researcher? The best place to be when a bubble bursts is on the sidelines.
    1. Is it in the middle of a costly venture? Companies are not good places for this in the long term whether a startup or a business unit. Being a researcher at a place desperately trying to figure out how to make research valuable doesn’t sound pleasant.
    2. Is it in the middle of a clearly valuable venture? That could be a good place. If you are interested we are hiring.
    3. Is it in academia? Academia has a real claim to stability over time, but at the same time opportunity may be lost. I’ve greatly enjoyed and benefited from the opportunity to work with highly capable colleagues on the most difficult problems. Assembling the capability to do that in an academic setting seems difficult since the typical maximum scale of research in academia is a professor+students.
  2. What do you work on as a researcher? Some approaches are more “bubbly” than others—they might look good, but do they really provide value?
    1. Are you working on intelligence imitation or intelligence creation? Intelligence creation ends up being more valuable in the long term.
    2. Are you solving synthetic or real-world problems? If you are solving real-world problems, you are almost certainly creating value. Synthetic problems can lead to real-world solutions, but the path is often fraught with unforeseen difficulties.
    3. Are you working on a solution to one problem or many problems? A wide applicability for foundational solutions clearly helps when a bubble bursts.

Researchers have a great ability to survive a bubble bursting—a built up public record of their accomplishments. If you are in a good environment doing valuable things and that environment happens to implode one day the strength of your publications is an immense aid in landing on your feet.

13 Comments to “When the bubble bursts…”
  1. […] When the bubble bursts… Consider the following facts: NIPS submission are up 50% this year to ~4800 papers. There is significant evidence that the process of reviewing papers in machine learning is creaking under several years of exponentiating growth. Public figures often overclaim the state of AI. Money rains from the sky on ambitious startups with a good story. Apparently, we now even have a fake conference website (https://nips.cc/ is the real one for NIPS). We are clearly not in a steady-state situation. Is this a bubble or a revolution? The answer surely includes a bit of revolution—the fields of vision and speech recognition have been turned over by great empirical successes created by deep neural architectures and more generally machine learning has found plentiful real-world uses. At the same time, I find it hard to believe that we aren’t… Original Post: When the bubble bursts… […]

  2. […] When the bubble bursts… 3 […]

  3. David says:

    Data Science is a growing field with many real world problems to solve for nearly every industry.

  4. Phil Winder says:

    Hi John,
    Thanks for the post. I’ve seen a few articles recently suggesting that we are in a bubble about to pop. I agree with your observations, but I would urge caution regarding the timing of such a prediction. If we assume that technology hype cycles are somewhat like the stock market (in a way they are linked, even in academic circles), then it has been shown time and time again that it is very difficult to predict stock market crashes. And like you said when they happen they happen fast.

    But I agree with your advice. If you are in industry, are you and is your business working on something that solves a valuable problem? If yes, you’ve got nothing to worry about. I believe that most of the hype is coming from the media and marketing departments. If we can just get marketing to stop saying that their product has AI… 😉

    Phil

  5. Alan says:

    Unfortunately, doing valuable works does not necessarily save us from downturn. When the dot-com bubble burst, many excellent companies went belly up. Think it this way, during a irrational exuberance, bad companies may also earn a lot of money. It should not be a surprise to see excellent companies decimated when the downturn became irrational.

  6. […] a put up on his personal blog, Dr. Langford goes on to […]

  7. […] an article on his personal blog Dr. Langford […]

  8. […] a post on his deepest weblog, Dr. Langford goes on to […]

  9. […] a post on his private weblog, Dr. Langford goes on to […]

  10. […] a post on his personal blog, Dr. Langford goes on to […]

  11. […] a post on his personal blog, Dr. Langford goes on to […]

  12. […] a post on his personal blog, Dr. Langford goes on to […]

  13. […] a post on his personal blog, Dr. Langford goes on to […]

Leave a Reply


Powered by WordPress