Indian-origin researcher Prady Modukuru has made a remarkable tool that effortlessly translates any video into the language of your choice. All you need is a YouTube link!
Modukuru posted a video on Twitter translating a conversation between Meta CEO Zuckerberg and Lex Fridman into Hindi. Here is a glimpse of the sorcery that's unfolding.
A serial entrepreneur, Modukuru is currently working on a stealth startup. Some of his previous ventures include marketing startup SŪNYA and mental wellness startup Oneself (inactive). He has previously worked with companies like Microsoft, Cisco, and Qknow among others.
How it works?
Modukuru leveraged the power of GPT-4 and prompt engineering techniques for seamless translation. To ensure accurate voice representation in different languages while preserving the original speaker's voice, he collaborated with ElevenLabs for voice training and text-to-speech capabilities.
In order to achieve precise lip syncing, Modukuru utilised wav2lip-2, which harnesses AI for generating lifelike lip movements. This breakthrough in the AI industry opens up new possibilities, allowing users to easily comprehend informative content in their preferred language.
In the near future, this technology can revolutionise dubbing, and can be used by major OTT platforms to translate content including series and movies to any other language.
- Microsoft Outage: Fix Fails to Restore Outlook, Teams, Copilot Across 74 Countries
- In a Stunning Fall, Jensen Huang Resigns as NVIDIA Chief After 33 Years
- Space Tech Startup Astrome Bags ₹64 Cr RDI Fund for AI Satellite Network
- Siemens Wants to Continue NVIDIA Partnership Even If Jensen Huang is Ousted
- LTM Integrates AI Tool to Fix Quantum Computing Errors on IBM
- Chariot Launches Multilingual TTS Model with Support for 11 Indian Languages