Indian-origin researcher Prady Modukuru has made a remarkable tool that effortlessly translates any video into the language of your choice. All you need is a YouTube link!
Modukuru posted a video on Twitter translating a conversation between Meta CEO Zuckerberg and Lex Fridman into Hindi. Here is a glimpse of the sorcery that's unfolding.
A serial entrepreneur, Modukuru is currently working on a stealth startup. Some of his previous ventures include marketing startup SŪNYA and mental wellness startup Oneself (inactive). He has previously worked with companies like Microsoft, Cisco, and Qknow among others.
How it works?
Modukuru leveraged the power of GPT-4 and prompt engineering techniques for seamless translation. To ensure accurate voice representation in different languages while preserving the original speaker's voice, he collaborated with ElevenLabs for voice training and text-to-speech capabilities.
In order to achieve precise lip syncing, Modukuru utilised wav2lip-2, which harnesses AI for generating lifelike lip movements. This breakthrough in the AI industry opens up new possibilities, allowing users to easily comprehend informative content in their preferred language.
In the near future, this technology can revolutionise dubbing, and can be used by major OTT platforms to translate content including series and movies to any other language.
- Tech Mahindra Partners With SpaceXSI to Bring AI Into Software Engineering
- Infosys AI Revenue Rises 29% in Q2, FY27 Growth Guidance Cut
- LTM Launches BlueVerse Industrial AI Lab in Chennai
- Chariot Launches Speech Recognition Model for 10 Indian Languages
- Meta Expects Up To $145 Billion In Capex As AI Investments Expand
- Cyient's Technology Order Intake Rises 39.7% as It Expands AI Capabilities