In the global entertainment landscape, TV show and film production stretches far beyond Hollywood or Bollywood — it's a worldwide phenomenon.

However, while streaming platforms have broadened the reach of content, dubbing and translation technology still has plenty of room for growth.

Deepdub acts as a digital bridge, providing access to content by using generative AI to break down language and cultural barriers.

On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz spoke with the Israel-based startup’s co-founder and CEO, Ofir Krakowski. Deepdub uses AI-driven dubbing to help entertainment companies boost efficiency and cut costs while increasing accessibility.

The company is a member of NVIDIA Inception, a free program that offers startups go-to-market support, expertise and technological assistance.

Traditional dubbing is slow, costly and often missing the mark, Krakowski says. Current technology struggles with the subtleties of language, leaving jokes, idioms or jargon lost in translation.

Deepdub offers a web-based platform that enables people to interact with sophisticated AI models to handle each part of the translation and dubbing process efficiently. It translates the text, generates a voice and mixes it into the original music and audio effects.

But as Krakowkski points out, even the best AI models make mistakes, so the platform involves a human touchpoint to verify translations and ensure that generated voices sound natural and capture the right emotion.

Deepdub is also working on matching lip movements to dubbed voices.

Ultimately, Krakowski hopes to free the world from the restrictions placed by language barriers.

“I believe that the technology will enable people to enjoy the content that is created around the world,” he said. “It will globalize storytelling and knowledge, which are currently bound by language barriers.”

https://blogs.nvidia.com/blog/2023/08/30/deepdub/