John Platt, who is PC-chair for NIPS 2006 has organized a NIPS paper evaluation criteria document with input from the program committee and others.
The document contains specific advice about what is appropriate for the various subareas within NIPS. It may be very helpful, because the standards of evaluation for papers varies significantly.
This is a bit of an experiment: the hope is that by carefully thinking about and stating what is important, authors can better understand whether and where their work fits.
Update: The general submission page and Author instruction including how to submit an appendix.
I think the real headline, which I found only by reading the linked document, is “NIPS Switches to Double-Blind Reviewing!”
I found the document itself to be mostly common sense. I guess it’s nice to have the criteria I was already applying in my reviewing spelled out explicitly (assuming I am asked to review again), but I don’t feel like I’d do anything different after reading this document. Also, I don’t feel like the standards he suggests are any different from what I use when reviewing for journals.
I think double blind reviewing is generally a positive idea. There is however one caveat — it is difficult to squeeze more theoretical papers in 8 pages (even considering the new format) and there is no mechanism for providing a supporting technical report. Reviewers have no way to check the validity of claims made in the paper.
misha b
In general, double blind implies that theory authors must work harder to include the proof.
Nevertheless, this remains a concern in the (short) NIPS format. We are trying to setup some system where papers have can have an appendix. The semantics of appendix is “the author can include anything, and the reviewer can optionally look at it”.
Good job, I’ve just heard that appendices are now allowed. I think this will strengthen the review process.