NIPS is getting big. If you think of each day as a conference crammed into a day, you get a good flavor of things. Here are some of the interesting things I saw.
- Grammar as a foreign language. Essentially, attention model + LSTM + a standard dataset = good parser.
- A New View of Predictive State Methods for Dynamical System Learning. Predicting future from past and future+1 from past allows you to form an estimate of system dynamics.
- Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing (Much) better labeling by better incentives.
- Hidden Technical Debt in Machine Learning Systems. A somewhat less vivid title than the earlier one which is entirely worth reading if you worry about ML systems.
- Bandits with Unobserved Confounders: A Causal Approach. In systems where a ‘default action’ exists, the act of intervening is not so simple.
- The Self-normalized Estimator for Counterfactual Learning. A good idea for reducing variance contextual bandit situations.
- Character-level Convolutional Networks for Text Classification. Extensive empirical experiments showing that character alphabets can be effective for NLP tasks.
- Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning. A yet-tighter form of RL MDP learning.
- On Elicitation Complexity. How many questions do you need to ask to get answers to questions about distributions? This has strong implications on learning algorithm design.
- End-to-End Memory Networks. There are not many algorithms for coherently forming pools of memory and using them to answer questions.
- Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets. Another way to add memory to a learning system.
- Scalable Semi-Supervised Aggregation of Classifiers. Better results for classifier aggregation in transductive settings.
Two other notable events happened during NIPS.
- The Imagenet challenge and MS COCO results came out. The first represents a significant improvement over previous years (details here).
- The Open AI initiative started. Concerned billionaires create a billion dollar endowment to advance AI in a public(NOT Private) way. What will be done better than NSF (which has a similar(ish) goal)? I can think of many possibilities.
See also Seb’s post.