Key takeaways
- Anonymity policies can hinder researchers' ability to publish their work on platforms like arXiv.
- Recent rejections at the ACL conference highlight frustrations with strict submission deadlines.
- Many researchers feel that these policies make ACL less appealing for natural language processing submissions.
- The debate continues on the balance between anonymity and open science principles.
How would you feel if you spend months working on a research and when you try to publish it for the world, the people in authority pull it down because of some "anonymity policy"? This heartbreaking scenario has been playing out for some time with researchers submitting their papers to the Association of Computational Linguistics (ACL) conference because of its review and citation policy.
Naomi Saphra, a PhD researcher at University of Edinburgh, recently took to X to share her experience with this policy. “Just got a desk rejection, post-rebuttals, for a paper being submitted to arXiv <30 min late for the anonymity deadline,” she said. Sapra posted this after the ACL conference rejected her because she was 30 minutes too late late in changing the anonymity, which means removing her name from the paper published on open-access archive platform, arXiv.
Vitthal Bhandari also shared the same reason for getting his paper rejected in May, which he submitted for the ACL conference. Undoubtedly, people on X say that this is making the ACL venue less and less desirable for NLP work.
Sebastian Raschka from LightningAI replied to the post, “Wait, first they don’t allow people to talk about their arXiv manuscripts online and now they also restrict when someone can and cannot upload to arXiv?” It seems like ACL is saying that your paper is a very valuable contribution for the scientific community, "but since you published it on another website 30 minutes after us, your work is not worthy to appear in our prestigious venue".
Let’s get this clear that ACL believes that if you want to publish your paper for its conference, you have to be anonymous, get your paper reviewed, and not publish on any other website such as arXiv, as long as the committee tells you to. ACL wants to have complete control over the science and research community. This whole conversation is directed against arXiv.
Ethicists getting jealous
ACL is not the only platform that is against arXiv. The conversation actually got interesting as soon as Emily M Bender, a professor and member of the Distributed Artificial Intelligence Research Institute (DAIR), which is founded by Timnit Gebru, decided to put in her thoughts. Bender said, “arXiv is a cancer that promotes dissemination of “junk science” in a format indistinguishable from real publication.”
Obviously, this take from an academic researcher was not received well by the whole research community. “Coming from a person at a position of considerable academic power, this is an unbelievably careless & elitist statement that goes against a lot of what (I hope) you believe in terms of equity & level playing field for people outside of prestigious institutions,” replied Stanislav Fort to Bender.
Members of the DAIR community have been the most vocal when talking about making AI more ethical and democratising the field by making it more inclusive. Interestingly, this comment is antithetical to the “inclusive philosophy” of their organisation.
It seems that the organisation holds itself on a higher pedestal and aims to decide who can actually contribute to open source. Moreover, the organisation wishes to promote peer reviewed publications, but do not realise it actually kills the open source spirit and affects junior researchers the most.
To prove the point and rebut, Bender took on Medium and published a blog – Scholarship should be open, inclusive and slow, where admittedly she discusses a few points and merits about the policy she proposes. “Peer review, when it is working well, doesn’t guarantee truth or correctness, but it does mean: This was examined critically and thoughtfully by 2–4 independent people with relevant knowledge who found it sufficiently solid (given their own knowledge) and worthy of others’ attention.”
She further tries to explain how sloppy work by famous researchers often gets promoted because it is done by famous researchers, which shouldn’t be the case. Further, bringing anonymity levels the playing field for outside researchers. This seems like a fair point but later, continuing with the elitist view of being part of the ACL’s Executive Committee, and next year’s president, Bender says that arXiv promotes biases, while also calling research papers that would not be relevant in a few months, as “not all that interesting".
Research is about reproducibility, not only reviewing
There is merit in the conversation that anyone publishing on a website which is believed to be the truest source of information for science needs some assessment. But arXiv has been handling it pretty well by having a single URL all this while post publication for as much peer review as possible. Moreover, pushing anonymity and embargoing papers that need to be reviewed by an “expert committee” is contrary to the principles of open and inclusive science.
“It sounds like a gatekeeper who is mad that people found a path to publishing that bypasses their gate,” said Thomas Steinke.
There is not even an argument against the fact that arXiv has contributed more to the science and engineering community and research than any other publishing portal. The open access nature of the website is possibly pushing every field in the best forward direction that there is. Boaz Barak, a theoretical computer science professor, reiterates, “I've said it before, arXiv has done much more to advance science, and expand participation in it, than all the anonymity interventions ever will.”
Even Yann LeCun, the Meta AI chief, agrees that any policy that obstructs arXiv is just silly.
All in all, it seems like ACL and DAIR are just afraid of how much arXiv is praised as being the best platform for scientific research, and they only want to be the gatekeepers of the research, obsessed with control, more than making it better, not actually making it inclusive. There are valid concerns about the publishing culture for sure, but arXiv is not the culprit, thus not needing ACL or DAIR’s opinion.
