AWS Launches AuditLLM, a Multiprobe Approach Tool for LLMs

AWS has recently unveiled a tool named AuditLLM. By announcing their demo paper at CIKM 2024, they introduce a novel tool for auditing LLMs using a multi-probe approach.

This solution is designed to streamline the auditing process for LLMs by focusing on activity-based auditing.

“AuditLLM is set to redefine the way auditing is performed in the digital age,” said Aman Chadha, researcher from Stanford University & AWS Leadership Member. 

Authored by Maryam Amirizaniani, Elias Martin, Tanya Roosta,  Chirag Shah, and Chadha this is a collaboration with the University of Washington, where the primary objective of AuditLLM is to provide a comprehensive audit trail of all activities performed by LLMs. 

The tool leverages advanced technology to meticulously scrutinise the activities of LLMs, ensuring transparency, accuracy, and compliance with established standards. 

Despite its sophisticated functionality, AuditLLM is easily navigable, allowing auditors to perform tasks without extensive technical knowledge.

As LLMs gain prominence across sectors, effective auditing tools like AuditLLM are increasingly crucial. With its launch, it addresses this need, setting a new benchmark for digital-era auditing.

Meet the AI Expert Building Indic LLMs with IITs

AIM got in touch with Chadha, who was working on building a medical large language model for India on top of Sarvam AI’s OpenHathi, and released a research paper around it.  

Given the amount of Indic languages speakers all over the world, Chadha expressed his happiness that models like Bharat GPT, Sarvam AI and Kissan AI are coming up. “But there’s nothing on the healthcare or the medical side,” he added, saying that he has been tracking all the recent announcements.