NIPS is getting big. If you think of each day as a conference crammed into a day, you get a good flavor of things. Here are some of the interesting things I saw.
- Grammar as a foreign language. Essentially, attention model + LSTM + a standard dataset = good parser.
- A New View of Predictive State Methods for Dynamical System Learning. Predicting future from past and future+1 from past allows you to form an estimate of system dynamics.
- Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing (Much) better labeling by better incentives.
- Hidden Technical Debt in Machine Learning Systems. A somewhat less vivid title than the earlier one which is entirely worth reading if you worry about ML systems.
- Bandits with Unobserved Confounders: A Causal Approach. In systems where a ‘default action’ exists, the act of intervening is not so simple.
- The Self-normalized Estimator for Counterfactual Learning. A good idea for reducing variance contextual bandit situations.
- Character-level Convolutional Networks for Text Classification. Extensive empirical experiments showing that character alphabets can be effective for NLP tasks.
- Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning. A yet-tighter form of RL MDP learning.
- On Elicitation Complexity. How many questions do you need to ask to get answers to questions about distributions? This has strong implications on learning algorithm design.
- End-to-End Memory Networks. There are not many algorithms for coherently forming pools of memory and using them to answer questions.
- Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets. Another way to add memory to a learning system.
- Scalable Semi-Supervised Aggregation of Classifiers. Better results for classifier aggregation in transductive settings.
Two other notable events happened during NIPS.
- The Imagenet challenge and MS COCO results came out. The first represents a significant improvement over previous years (details here).
- The Open AI initiative started. Concerned billionaires create a billion dollar endowment to advance AI in a public(NOT Private) way. What will be done better than NSF (which has a similar(ish) goal)? I can think of many possibilities.
See also Seb’s post.
I think the first entry should be commented as “attention + LSTM + a standard dataset + a SotA parser = a good parser”…
Not that impressive from an NLP point of view.
That was pretty true with last year’s paper, but this paper looks better to me unless they left some result out.
Looking at Table 1, it looks like they are about 1.3% behind the best-known result using just WSJ data on WSJ 23. That’s a good-if-not SotA parser.
Adding in the SotA parser(s) to label a high confidence set, they end up .4% better than the best result listed.