Research Styles in Machine Learning

Machine Learning is a field with an impressively diverse set of reseearch styles. Understanding this may be important in appreciating what you see at a conference.

  1. Engineering. How can I solve this problem? People in the engineering research style try to solve hard problems directly by any means available and then describe how they did it. This is typical of problem-specific conferences and communities.
  2. Scientific. What are the principles for solving learning problems? People in this research style test techniques on many different problems. This is fairly common at ICML and NIPS.
  3. Mathematical. How can the learning problem be mathematically understood? People in this research style prove theorems with implications for learning but often do not implement (or test algorithms). COLT is a typical conference for this style.

Many people manage to cross these styles, and that is often beneficial.

Whenver we list a set of alternative, it becomes natural to think “which is best?” In this case of learning it seems that each of these styles is useful, and can lead to new useful discoveries. I sometimes see failures to appreciate the other approaches, which is a shame.

3 Replies to “Research Styles in Machine Learning”

  1. In some ways, ML at its best is like physics. One can see the same 3 distinct branches:
    1) Experiment Physicists
    2) Theoretical Physicists
    3) Mathematical Physicists

    This interaction and mutual hard-work and respect seems to have served physics extremely well. Again, there are people who
    cross (2?) disciples and the boundaries often smear. It seems like an important question to ask is how we can achieve that same level of interaction? My guess is that it’s an education problem– our 3 levels don’t always work from the same basic understandings.

  2. I agree—it’s like physics, and it seems like there is not nearly enough understanding (and consequently respect) between the branches in learning. This is one of the reasons why I like colocation of learning conferences a lot.

    I wish we had an ICML, COLT, or NIPS colocation with vision conferences, a language learning conference, or KDD (not one separated by several miles as in Washington DC in 2003).

  3. I’d even be happy to see more cross-over between NIPS and COLT, for instance. Workshops have some potential to do this,
    although it’s hard. A good thing to do would be to help run workshops devoted to applications at the theoretical/mathematical conferences (NIPS/COLT) and really recruit hard the serious practitioners. (Not as good as co-located conferences, but probably still useful if you can get high density of smart people from all camps.)

Comments are closed.