Artificial intelligence is quickly and irreversibly changing the translation and localisation sector. The need for quick, affordable translation and numerous technical advances fuel innovations of all kinds. From translation to quality control, AI can be employed at every stage of a localisation project’s workflow. In reality, localisation and AI complement each other well. The localisation industry has made significant improvements in speech recognition, AI, speech synthesis, and NLP technology. However, many believe that the human brain will always be the same, regardless of how advanced algorithmic technology grows.
Karthik Ramesh, VP – Client Partner – US Provider and Lifesciences at Emids, moderated the Roundtable session along with panellists Rishi Swami, Head of Data Science at MPL, Ashwin Swarup, Vice President – Data Science at Digite, Satish Grampurohit, Co-Founder and Chief Evangelist at Cogniquest and Lavi Nigam, Lead Data Scientist at Google Cloud AI Ecosystem.
Demystifying Localisation
Localisation involves tailoring a system or product to meet the specific needs of the local geography in which it will be sold. This requires consideration of factors such as language (verbal and non-verbal), geography, culture, processes and practices, preferences, and regulations. Interestingly, localisation has become increasingly important due to increasing globalisation. While globalisation tends to standardise products and services across the world, the need for localisation has grown more urgent as businesses seek to meet the diverse needs of local markets while going global.
— Satish Grampurohit, Co-Founder and Chief Evangelist at Cogniquest
The Approach of Private Companies
There are many products, one of them is ‘AnthroKrishi’. This is more from a sustainable point of view. They’re trying to see the satellite data that we have on Google Maps and see how we can leverage that for the benefit of agriculture. They do similar programmes in other countries, but this is with Indian governments and many NGOs. In this project, they’re trying to gather data and segment it manually through the local government for a specific crop. AI/ML helps them do this by segmenting or making the information available, which is quite easy for everybody to filter through.
— Lavi Nigam, Lead Data Scientist at Google Cloud AI Ecosystem.
Relevance of Localisation in the AI/ML industry
What AI has done in terms of natural language processing, image translation, the manual tasks of translating things from English, where a lot of content was being generated, to regional languages has become very easy. And that enables any company to localise to the specific target keeping the example of India. So, it’s much easier to convert your content, app and website into regional languages. AI has played a huge role in reducing this manual task and automating it.
— Rishi Swami, Head of Data Science at MPL
Aspects of localisation
We’ve broken down localisation into two parts. One is processing localisation, that is, you’re adhering to the standards or mechanisms and processes of a specific region. And second one is language localisation, that is, whether they are able to access whatever standardised processes you have in their regional language. Process localisation is slightly more challenging. It needs to be encoded for every state because those established there keep evolving. There is only one way for a system to adapt to a process-based system, if it sees sufficient examples. Conversely, language localisation has received a major boost lately. With the advent of these current light, large language models, there is a scope for us to generate that kind of reinforcement learning feedback. So, it’s easier if we have enough APIs to deal with. The challenge would be our dialects and, within the context of India, it will be the dialects and the sheer differences in the same spoken language.
— Ashwin Swarup, Vice President – Data Science at Digite
Make AI in India and Make AI work for India
The role of the government is basically to create the dataset. All these models will keep evolving and we’ll have access to them. But how do we know how the masses access them? That’s where we’ll need much more training data in accordance with Indian dialects and languages and that’s where someone needs to take the initiative. This will be a huge process because it is available in English and other languages. But, for local languages, the government can come in. They can help create this dataset, which helps train our models. All the language models will need that. That’s also where the centre of excellence will help. In terms of startups, it will help make things more accessible. If our language models improved in text or speech recognition for regional languages, it would make more apps which will start translating for users. A large chunk of the population needs to access the apps because they are primarily in English or Hindi. With AI coming in and easy translation and transition being available, we can make all the services available to an even larger section of the population.
— Rishi Swami, Head of Data Science at MPL
It is encouraging to see the government’s focus on Digital India. India offers a unique opportunity and challenge for AI adoption, due to the diverse demographics – range of languages, rural-urban divide, literate-illiterate population. Government has taken several citizen-centric initiatives using AI. For instance, ‘Airavat’ provides citizens with access to all government initiatives at their fingertips, ‘Bhashini’ offers services in vernacular languages and ‘AI for Health’ uses local health information that is specific to India. Our government is driving and leveraging AI with the slogan of “AI for All”, and by pushing for leveraging India talent. There are significant developments happening in both industry and academia, with improving interfaces and increasing investments in the field. The effort is focused on leveraging India technology and leadership to address the challenges and opportunities that arise from the country demographics. Exciting times lie ahead in terms of the possibilities that can arise from technology and localisation efforts.
— Satish Grampurohit, Co-Founder and Chief Evangelist at Cogniquest
5G Services
5G will open up many possibilities that we have not seen before. Similar to what we had seen when we moved from 3G to 4G. Most of the Internet things we nowadays take for granted happened because of that 4G boom. They opened up opportunities for companies to build amazing AI. We will see the same scenario playing out with 5G. We need to find out what those use cases are. These companies and many other new companies benefit society when they reach their billion users because they’ll build these models in AI, which will then come to the open world. I’m very optimistic about 5G or 6G, whatever comes in that direction, because it opens up a huge market for new AI. Once we have AI as part of our day-to-day life, localisation can be created based on the local requirements and use cases.
— Lavi Nigam, Lead Data Scientist at Google Cloud AI Ecosystem.
The theory of concept localization highlights a major challenge in comprehending unfamiliar ideas when they are presented without sufficient context. As human beings, we tend to rely on metaphorical explanations to make sense of complex concepts. We learn best when someone provides us with a metaphor that allows us to grasp the underlying meaning of the concept. With the advent of advanced language models , one can take a given concept and translate it into metaphors that are tailored to an individual’s unique background, making it easier for them to understand and absorb the idea.
— Ashwin Swarup, Vice President – Data Science at Digite
Localisation As a Service for Startups
I think one of the key things is that we can come up with a lot of local data centres. It’s important to do that from a policy perspective because most organisations, whether government or private, don’t prefer putting their datasets outside. That’s one of the greatest moves by all three cloud vendors, Google, AWS and Azure as they’re trying to localise all the data centres in India. The second is all three of the clouds have services specifically for startups. There’s a push that is given to them, which is like a seed fund that is not money but essentially the credits. I see that as a great programme because that means startups, specifically people with great ideas, can quickly build things using all the great APIs at all three clouds. The clouds are providing those APIs for free training.
— Lavi Nigam, Lead Data Scientist at Google Cloud AI Ecosystem.
Conclusion
Localisation is a kind of unity in diversity in terms of making everything so local and specific to the region we come from. India is diverse in terms of culture, in terms of languages, the number of states, the kind of people. So, it’s going to be exciting to see how this future unfolds. Let’s also remember the role of humans as we live in a chatGPT(& other LLM) dialogue driven future world diverse input world. Still, linguists also have many roles to play in validating the output and checking the accuracy generated to localised models or output content streaming.
— Karthik Ramesh, VP – Client Partner – US Provider and Lifesciences at Emids
This article is written by a member of the AIM Leaders Council. AIM Leaders Council is an invitation-only forum of senior executives in the Data Science and Analytics industry. To check if you are eligible for a membership, please fill out the form here
