“The face is the uncanny valley. The nuance and detail in the face and the way it moves is a very difficult thing to animate,” Jonathan Bronfman, CEO of MARZ explains on SlatorPod.

Canada-based MARZ, which stands for Monsters Aliens Robots Zombies, was founded in 2018 to leverage AI and bring quantifiable metrics like time and cost to the business of visual effects (VFX).

Bronfman recounts how their research for Vanity AI, a solution that creates effects such as de-aging, cosmetic, wig, and prosthetic fixes, led to them becoming an “expert at the face” and launching LipDub AI, a solution to solve the “mismatch between spoken word and visuals for viewers”.

According to Bronfman, users can take an audio stem (the dialogue track), the original piece of footage, and push both through LipDub and have a lip-synced visual in 30 seconds. Bronfman also explains that their USP is that LipDub is Hollywood-grade and works on 4K and ProRes files.

The two also discuss how actors feel about the emergence of lip-syncing visual effects and why Bronfman does not think an expansion into the creator market is on the cards for MARZ.

Florian shares that as a life-long consumer of dubbed content he sees the massive potential in a solution that improves the viewer experience and makes it more natural.