NeurIPS Is Once Again Amid Controversies Due To “Terrible” Reviews

This year, the thirty-fourth annual conference on Neural Information Processing Systems, NeurIPS 2020 is going to be held virtually from 6th to 12th December. The paper submissions for this year is 38% more than last year, i.e. a total of 1,903 papers were accepted, compared to 1,428 last year. 

The review period of the papers began in July, and in August, the popular AI conference, NeurIPS has sent out the paper reviews for this year’s conference. This has brought the popular machine learning event once again amid the controversies as it has been claimed that the reviews of the papers are “terrible”, i.e. either they are not clear, or the sentences were incomplete by the reviewers, among others.

In a tweet, Director of AI Research at NVIDIA and California Institute of Technology Professor, Anima Anandkumar shed light on the paper reviews. Anandkumar said that she has been witnessing terrible paper reviews for the “nth time” and it is worthy of banning the reviewers from submitting as authors “if they can’t write a review in good faith”.


Sign up for your weekly dose of what's up in emerging technology.

Replying to this tweet, an alumnus of IISc Bangalore, Suraj Krishnamurthy stated that the reviews of his paper were worse, and there were “absolutely no takeaway messages from the reviewers.”

Google AI researcher, Rohan Anil mentioned in a tweet that the reviews are poor in nature with incomplete sentences by the reviewers.

Some of the machine learning researchers suggested a few solutions that can help in solving the “terrible” reviews. Christian Szegedy, an AI research scientist, suggested that the system should propose a different review process. One suggested way that can improvise the system is by decoupling the task of judging and analysing the papers. 

Szegedy added, “I would suggest a review process in which in the first phase, the (fewer) reviewers’ task is to analyse the paper. The reviewers should have the opportunity to give an enthusiastic recommendation of acceptance or reject if they feel strongly, but are not forced to rate at all.”

Replying to his own tweet, Szegedy further mentioned that in the second phase, the reviewers should stack rank the paper based on the first round’s analysis. This would be useful for borderline papers, for which the reviews are lukewarm.

While Victor Zakhary, SWE at Oracle tweeted that the system should have something similar to Arxiv with public reviews in a decentralised manner or utilise a blockchain-like solution where papers reviews are public and tied to their writers.  

However, some of the machine learning enthusiasts had a different take for the review system. In a comment, a Reddit user mentioned that the “NeurIPS reviews received by the user were “actually really fair and detailed for the most part.” The user also mentioned that the problem is NeurIPS wants publications to be new and not published elsewhere (except Arxiv).

Also, Graham Neubig, Associate professor at CMU, mentioned in a tweet that 70-80% of the reviews at NeurIPS2020 were reasonable and fixing any complaints will make the papers better.  

This is not the first time that controversies have scarred the reputation of the conference. In other words, it can be said that controversies are not a new thing for this popular machine learning conference. 

In 2018, the organisers of the “NIPS” (Neural Information Systems Processing) conference had changed the event’s name from NIPS to NeurIPS after heading into a controversy about whether “NIPS” is an offensive name or not. According to sources, signed a letter calling on NIPS to be rebranded following reports of inappropriate behaviour. The letter stated that the “acronym of the conference is prone to unwelcome puns.” 

More Great AIM Stories

Ambika Choudhury
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM