Now Reading
How Participants Are Manipulating ML Hackathons

How Participants Are Manipulating ML Hackathons

Rohit Chatterjee
machine learning hackathons
W3Schools

In recent years, Hackathons are the go-to place for many aspiring participants and data scientists who want to explore their skills against thousands of competitors in terms of coming out with real-word solutions using latest technology and devices. It also provides an individual with the chance to collaborate with a group of other programmers to work on a project. Not to mention, the prize money does play an important role, which gives a boost to the team to fast track their ideas or projects.

The prize money is the solution to a lot of programmer’s dreams but also has turned into a hiccup as several participants have manipulated and foul-played in hackathons to get their hands on it, creating a stir in the community.

Previous Cases of Cheating

A prize amount of $10,000 and an SV job went out of the window for a Kaggle grandmaster who was found to win the first position by cheating. In a recent Google-backed competition, the participants were required to develop a machine learning software that would help abandoned animals to find a shelter turned out to be ugly. 



The contest was devised by PetFinder.my that aimed to save animals with the assistance of AI solutions. Team Bestpetting lead by Pavel Pleskov, who is ranked third in the leaderboard of all Kaggle competitions was caught when Bestpetting’s code was put into final production.

When Benjamin Minihoxefer, who was part of the competition and ranked sixth joined PetFinder.my, he chose Bestpetting’s code into production mode, which was considered the best model. However, while poking around, Minihoxefer found that the function of ‘majority voting’ was concealing Python code that helped Bestpetting to cheat.

In 2014, two teams of the final eight of HackMIT were disqualified for they wrongfully represented their accomplishment. The participants implemented an algorithm, which originally belonged to people from Microsoft research. Another major incident goes back to the year 2013, which involved a massive prize amount of $1 million. Participants in the Dreamforce 2013 were directed to create a mobile-based app within a stipulated time. However, the winning team was found to be promoting the app even before the competition was flagged off.

How? 

Often participants are found to be manipulating commit logs to hide activity. Many teams copy and paste codes to combine and make it their own. In many instances, participants have simply copy and paste code from a sole source and named it as their own.

There are a number of other cheating forms, which participants use to manipulate the hackathon’s result. In certain cases, hackathons might require a video submission in which the demonstration videos can be faked by a team. In such a scenario, videos are edited in a way, which shows a hack to perform a function that it actually cannot perform. The second way of cheating is the most common one as is the case with the winner of Dreamforce 2013. The modus operandi remains the same with a team completing the coding by starting before the stipulated time frame provided by the hackathon organiser. 

See Also

Teams participating in hackathons also violate the rules by adding more teammates remotely. Although adding teammate remotely is allowed, a team cannot have more members than what the organising committee may have laid down in the rules.

Putting A Stop

Although there are a number of ways participants are using to disrupt the hackathons, several measures can be taken by organising committees to put a stop on the malicious tactic. Organisations can begin by asking teams to submit their source-code of the projects. Indeed, many who would like to use the code to create a business or another project would like to keep the ownership to themselves, this step is still a viable option to maintain transparency. Moving forward, organising committees can prevent cheating by bringing a group of such as past winners or experienced programmers who will take a deep look at the manipulations. Furthermore, an Audit team can be created, which could review the teams reaching the semi-final and final berth.

Final Words

When looked at the recent case of cheating at hackathons, the massive prize amount triggers participants who are willing to go to any length to achieve the winning prize. On the other hand, it is also the obsession of a few to become the best in the community, which leads to such scenarios as was the case with Pavel Peskov, who quoted saying, “For me, it was never about the money but rather about the Kaggle points: a constant struggle of becoming #1 in rating had compromised my judgement.” Preventive measures can be taken to avoid such circumstances, but it is in the hands of the participants to understand the real spirit of hackathons. 

For those who are looking to take part in hackathons, hit the link and take a look at MachineHack, which allows a user to practice machine learning skills and participate against hundreds of data scientists.

What Do You Think?

If you loved this story, do join our Telegram Community.


Also, you can write for us and be one of the 500+ experts who have contributed stories at AIM. Share your nominations here.

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top