Active Hackathon

The  “Test of Time” research that advanced our interpretation of Adversarial ML

Contemporary R&D progress shows that researchers have come up with ‘reactive’ and ‘proactive’ measures to secure ML algorithms.
Listen to this story

The 39th International Conference on Machine Learning is currently being held at the Baltimore Convention Centre in Maryland, USA and their ‘Test of Time’ award was awarded to a research work published in 2012 titled, ‘Poisoning attacks against Support Vector Machines’. 

This research work was undertaken to demonstrate that not only can an intelligent adversary predict a change in the decision-making function of a Support Vector Machine (SVM) due to malicious input but can also use this prediction to construct malicious data. 


Sign up for your weekly dose of what's up in emerging technology.

Conducted by Battista Biggio, Department of Electrical and Electronic Engineering, University of Cagliari along with Blaine Nelson and Pavel Laskov from the Wilhelm Schickard Institute of Computer Science, University of Tubingen—this is one of the earliest research works ever conducted on the poisoning attacks against SVMs.

(Image source: Twitter)

ICML’s ‘Test of Time’ is awarded to research works presented ten years from the current year in recognition of the impact that the works have caused since their publication to the current research and practice in the field of machine learning.

The research 

The research work successfully demonstrates how an intelligent adversary can, to some extent, predict the change of a Support Vector Machine’s (SVM) ‘decision function’ due to malicious input and use this ability to then construct malicious data.

SVMs are supervised machine learning algorithms that can be used for the classification and regression analysis of data groups and can even detect outliers. They are capable of both linear classification and non-linear classification. For non-linear classification, SVMs use a kernel trick.

In the course of the study, the research team made certain assumptions about the attacker’s familiarity with the learning algorithm and their access to underlying data distribution and the training data that the learner may be using. However, this may not be the case in real-world situations where the attacker is more likely to use a surrogate training set drawn from the same distribution. 

Based on these assumptions, the researchers were able to demonstrate a technique that any attacker can deploy to create a data point that can dramatically lower classification accuracy in SVMs. 

To simulate an attack on the SVM, the researchers used a technique called ‘gradient ascent strategy’, where the gradient is computed based on the properties of the optimal solution of the SVM training problem. 

Since it is possible for an attacker to manipulate the optimal SVM solution by interjecting specially crafted attack points, the research demonstrates that it is possible to find such attack points while retaining an optimal solution of the SVM training problem. In addition, it illustrates that the gradient ascent procedure significantly increases the classifier’s test error.

Significance of the research 

When this research was published in 2012, contemporary research works related to poisoning attacks were largely focused on detecting simple anomalies. 

This work, however, proposed a breakthrough that optimised the impact of data-driven attacks against kernel-based learning algorithms and emphasised the need to consider resistance against adversarial training data as an important factor in the design of learning algorithms.

The research presented in the paper inspired several subsequent works in the space of adversarial machine learning such as adversarial examples for deep neural networks, various attacks on machine learning models and machine learning security. 

It is noteworthy that the research in this domain has evolved since then—from focusing on the security of non-deep learning algorithms to understanding the security properties of deep learning algorithms in the context of computer vision and cybersecurity tasks. 

Contemporary R&D progress shows that researchers have come up with ‘reactive’ and ‘proactive’ measures to secure ML algorithms. While reactive measures are taken to counter past attacks, proactive measures are preventive in nature. 

Timely detection of novel attacks, frequent classifier retraining and verifying the consistency of classifier decisions against training data are considered reactive measures.

Security-by-design defences against ‘white-box attacks’, where the attacker has perfect knowledge of the attacked system and security-by-obscurity against ‘black-box attacks’, where the attacker has no information about the structure or parameter of the system are considered proactive measures.

The importance of employing such measures in present-day research highlights the significance of this paper as the pivotal step in the direction to secure ML algorithms.

By the same token, industry leaders too became increasingly aware of the different kinds of adversarial attacks like poisoning, model stealing and model inversion and recognised that these attacks can inflict significant damage to businesses by breaching data privacy and compromising intellectual property. 

Consequently, institutional vigilance about adversarial machine learning is prioritised. Tech giants like Microsoft, Google and IBM have explicitly committed to securing their traditional software systems against such attacks. 

Many organisations are however already ahead of the curve in systematically securing their ML assets. Organisations like ‘ISO’ are coming up with rubrics to assess the security of ML systems across industries. 

Governments are also signalling industries to build secure ML systems. For instance, the European Union released a checklist to assess the trustworthiness of ML systems.

Amid these concerns, machine learning techniques help detect underlying patterns in large datasets, adapt to new behaviours and aid in decision-making processes, and have thus gained significant momentum in the mainstream discourse. 

ML techniques are routinely used to solve big data challenges such as various security-related issues like detecting spam, frauds, worms or other malicious intrusions. 

Identifying poisoning as an attack on ML algorithms and the disastrous implications it may have for many businesses and industries like the medical sector, aviation sector, road safety or cyber security concretised the contribution of this paper as one of the first research works that paved the way for adversarial machine learning research. 

The authors challenged themselves with the task of finding if such attacks were possible against complex classifiers. Their objective was to identify an optimal attack point that maximised the classification error.  

In their work, the research team not only paved the way for adversarial machine learning research, a technique that tricks ML models by providing deceptive data, but also laid the foundation for any research that may help defend against growing threat in AI and ML. 

More Great AIM Stories

Zinnia Banerjee
Zinnia loves writing and it is this love that has brought her to the field of tech journalism.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM

The curious case of Google Cloud revenue

Porat had earlier said that Google Cloud was putting in money to make more money, but even with the bucket-loads of money that it was making, profitability was still elusive.

Global Parliaments can do much more with Artificial Intelligence

The world is using AI to enhance the performance of its policymakers. India, too, has launched its own machine learning system NeVA, which at the moment is not fully implemented across the nation. How can we learn and adopt from the advancement in the Parliaments around the world? 

Why IISc wins?

IISc was selected as the world’s top research university, trumping some of the top Ivy League colleges in the QS World University Rankings 2022