|
Listen to this story
|
In today’s fast-paced world, where innovation and technological advancements are rapid, the clout of responsible AI is too big to ignore. The downside of this neck-and-neck race is companies hurriedly releasing new products and services, often without thorough testing or consideration of potential consequences.
As Microsoft integrated OpenAI’s GPT models into its product suite, Google responded with its own line of generative AI-powered products to compete. Nevertheless, critics contend that in the frantic AI race, responsible AI considerations have been sidelined.
Some companies are now deploying models publicly and only addressing issues after assessing the potential damage, sparking concerns about ethical AI practices.
However, Krishnaram Kenthapadi, Chief AI Officer and Chief Scientist at Fiddler AI, believes that the likes of Microsoft, Google and OpenAI releasing a not-so-perfect model is not as much of a concern. He believes these companies have the expertise and resources to incorporate responsible AI during the development of the models as well as even after the models have been released.
In fact, he believes it may not be possible to have a large language model (LLM) that is perfect on day one. Take Google Search, for example, it was far from perfect when it launched two decades ago. However, over the years, Google has incorporated user feedback to improve the model powering the search engine. Along similar lines, Microsoft, Google and OpenAI have been learning from unforeseen issues observed after deploying their LLMs and improving their offerings over time.
Responsible AI by design
“However, I am concerned about those who make use of open source models or OpenAI APIs and build applications by fine-tuning with their proprietary data or leveraging retrieval augmented generation. These enterprises may not have the resources or the expertise to stress test or even monitor these models and applications once they’re launched. I think that’s where there is a greater need for the right set of responsible tools to help them address these needs,” Kenthapadi told AIM.
“It might appear counterintuitive at first glance, given the frequent news coverage of the big players in the AI domain. But the bigger concern is the second world where there may be a large number of enterprise companies which may not even have the expertise. They may be in some other domain like healthcare, finance, hiring or manufacturing. I believe we need to provide them with the right set of tools so that they can deploy AI models responsibly.”
One way for enterprises to ensure responsible deployment of AI is to consider it an integral part of innovation, rather than something separate from it.
“It’s a beneficial perspective to view them together. Typically, it’s akin to not treating privacy or security as an afterthought or neglecting fairness or explainability. Instead, it’s wise to adopt a mindset of designing AI responsibly from the outset,” Kenthapadi said.
Hallucinations still remain
Although an increasing number of enterprises are utilising these models, the issue of hallucinations associated with LLMs still persists. It’s a challenge that the entire industry is striving to address but hasn’t successfully resolved yet.
“We’ve encountered various approaches to tackle the issue of hallucinations. At Fiddler, we’ve been exploring some of these methods as well. One approach entails employing an additional model to verify the responses generated by the LLMs.”
For example, if you are searching for places to explore in Paris, the responses can be easily vetted. “Recently, we developed and launched our chatbot based on Fiddler’s documentation pages. What we observed is that when a query was made about ingesting data from a training data source not covered in our documentation, the model began creating non-existent APIs. Detecting such fabrications can be challenging because they appear highly convincing to the user.”
When questioned about the level of risk enterprises face when utilising these models in light of the persisting hallucination issue, Kenthapadi replied that the extent of risk largely depends on the specific application in question.
For instance, if an enterprise employs the model for internal purposes or by customer support agents to enhance responses to customer inquiries, there might be a certain level of tolerance for hallucinations. In such cases, these errors could be deemed acceptable since the model acts as an aid for the enterprise and isn’t directly used by end-users.
However, if the model is being used by a doctor to diagnose a patient, there should be a much higher bar, when it comes to the level of tolerance for hallucinations.
“Therefore, I would suggest that depending on the criticality as well as the level of reputational risk involved, particularly if the model is intended for end-users, companies should carefully plan the gradual deployment of LLM-based applications.”
It’s crucial to learn from mistakes and continuously enhance these applications. In this context, having tools to assess these models and applications across various responsibility dimensions, along with tools for continuous monitoring, is of paramount importance.
Ethical pitfalls to avoid
While LLMs might have caught everyone’s attention, due to their mastery of language, and their ability to generate text and write code, there are also several ethical pitfalls associated with the use of these models, according to Kenthapadi.
Enterprises leveraging these models, for instance, need to test them not just before deploying, but also monitor them post-deployment.
“A Stanford research paper found that the performance of models like GPT 3.5, and GPT-4 tend to vary substantially over time. Over the span of three or four months, they noticed that the performance of the model for certain maths, visual analysis or coding tests varied – improving in some cases, and substantially degrading in others.”
Therefore, as enterprises or startups harness these models, it’s imperative to consider these aspects. While this primarily concerns performance, there are additional responsible AI considerations, such as robustness.
“It’s not enough to solely evaluate LLMs based on specific prompts and responses. We must also assess their robustness by introducing slight alterations to prompts to see how robust the responses are. Moreover, it is critical to ensure that the text generated by these models are bias-free.”
Kenthapadi underscores the importance of preventing model responses from exhibiting bias based on factors like gender, race, religion, caste, or creed. Equally vital is the need to ensure that the models do not inadvertently disclose any Personal Identifiable Information (PII).
“This should not happen, especially once the model is deployed as it can jeopardise the company’s reputation and lead to potential legal consequences. Additionally, there are other considerations such as explaining the model’s outputs – can it provide comprehensible explanations, and how trustworthy are these explanations? These are all aspects that enterprises must take into account.”
The key to prevention, according to Kenthapadi, is to proactively measure the various responsible AI dimensions, in addition to solely assessing performance. Furthermore, this process should not be a one-time effort but an ongoing monitoring of these metrics, even after the application has been deployed.
“This is where tools like those offered by Fiddler AI can prove to be valuable,” Kenthapadi concluded.
