“The face is the uncanny valley. The nuance and detail in the face and the way it moves is a very difficult thing to animate,” Jonathan Bronfman, CEO of MARZ explains on SlatorPod.

Canada-based MARZ, which stands for Monsters Aliens Robots Zombies, was founded in 2018 to leverage AI and bring quantifiable metrics like time and cost to the business of visual effects (VFX).

Bronfman recounts how their research for Vanity AI, a solution that creates effects such as de-aging, cosmetic, wig, and prosthetic fixes, led to them becoming an “expert at the face” and launching LipDub AI, a solution to solve the “mismatch between spoken word and visuals for viewers”.

According to Bronfman, users can take an audio stem (the dialogue track), the original piece of footage, and push both through LipDub and have a lip-synced visual in 30 seconds. Bronfman also explains that their USP is that LipDub is Hollywood-grade and works on 4K and ProRes files.

The two also discuss how actors feel about the emergence of lip-syncing visual effects and why Bronfman does not think an expansion into the creator market is on the cards for MARZ.

Florian shares that as a life-long consumer of dubbed content he sees the massive potential in a solution that improves the viewer experience and makes it more natural.

Florian and Esther discuss the language industry news of the week, where amid ChatGPT's new enterprise launch, Florian tested ChatGPT's translation quality and came across mistranslations and awkward phrasing.

Florian highlights the launch of the Slator Pro Guide: Subtitling and Captioning, providing an overview of its content and use cases. The discussion shifted to Meta's latest release SeamlessM4T which handles various translation-related tasks, including speech recognition, speech-to-speech translation, and more, across 100+ languages.

Meanwhile, Appen shares price dropped considerably, so they aim to reposition themselves more toward generative AI but may face stiff competition from companies like Scale AI which recently announced a strategic partnership with OpenAI.

The duo give their view on a Twitter post by a freelancer who was asked to provide discounts for typos by an LSP. When posed this question in Slator’s newsletter, the community's response was divided, with the majority opposing discounts for typos.

Georg Ell, CEO of Phrase, joins SlatorPod to talk about key moments since he joined the localization platform in May 2022, merging Memsource into the Phrase brand, and the opportunities of AI-driven growth of content requiring localization.

Georg discusses Phrase’s product offering, which includes machine translation (MT) aggregator Phrase Translate and proprietary MT engine Phrase NextMT, where they provide customers with flexibility, neutrality, and solutions that align with their localization needs and varying maturity levels.

Georg unveils Phrase's upcoming pricing structure revamp, set to launch in September, which aims to provide customers with streamlined access to the entire suite of capabilities across Phrase TMS, Phrase Strings, Phrase Orchestrator, and other features.

Georg shares a perspective on the future of a hyper-personalized internet enabled by generative AI. He discusses the importance of AI-driven workflows, complex custom configurations, and dynamically evaluated machine learning models.

Georg advises on how LSPs should embrace AI and MT. He also touches on funding in the language technology sector, and how they intend to invest in areas that align with the company's growth strategy.

Joining SlatorPod this week is Longyue Wang, a Research Scientist at Tencent AI Lab, where he is involved in the research and practical applications of machine translation (MT) and natural language processing (NLP).

Longyue Longyue expands on Tencent’s approach to language technology where they integrate MT with Tencent Translate (TranSmart). He highlights how Chinese-to-English MT has made significant advancements, thanks to improvements in technology and data size. However, translating Chinese to non-English languages has been more challenging.

Recent research by Longyue explores large language models’ (LLMs) impact on MT, demonstrating their superiority in tasks like document-level translation. He emphasized that GPT-4 outperformed traditional MT engines in translating literary texts like web novels.

Longyue discusses various promising research directions for MT using LLMs, including stylized MT, interactive MT, translation memory-based MT, and a new evaluation paradigm. His research suggests LLMs can enhance personalized MT, adapting translations to users' preferences.

Longyue also sheds light on how Chinese researchers are focusing on building Chinese-centric MT engines, directly translating from Chinese to other languages. There's an effort to reduce reliance on English as a pivot language.

Looking ahead, Longyue's research will address challenges related to LLMs, including handling hallucination and timeless information issues.

Florian and Esther discuss the language industry news from over the past month, where NWN Carousel, a Massachusetts-based cloud communications company, secured a USD 6.5 million contract with Baltimore, Maryland, to provide AI-powered multilingual emergency 9-1-1 services.

Meta has unveiled Llama 2, its latest large language model, trained on 40% more data. Despite its extensive training, the model's focus on English data and limited coverage of other languages might restrict its suitability for non-English language applications.

In a financial results round-up, Straker Translations reported a 2% revenue growth with increased revenues from IBM and strong demand in Europe. After a successful 2022, ZOO Digital is now facing headwinds from Hollywood’s writers and actors strike. The pod also checks in on Keywords Studios’ and Honyaku Center’s financials.

Esther’s M&A corner features Belgian LSP ElaN Languages acquiring Lingua-World, expanding into the German market. Further north, LanguageWire, one of the largest LSPs in the Nordics, acquired Finnish LSP Delingua to further strengthen its position in the region.

Meanwhile, in funding, Phrase raised EUR 12m in debt financing from CIBC Innovation Banking, to support further development of the Phrase Localization Suite. Over in Wyoming, Language I/O has secured USD 8m in a Series A1 funding round to enhance its platform with generative AI technology and for hiring.

In this week’s SlatorPod, we are joined by Florian Stègre, Founder of Checksub, to discuss how the SaaS platform powers subtitling and dubbing for SMEs and enterprise clients.

Florian's entrepreneurial journey began after graduating from business school and founding video production company, ROOM Agency, which laid the foundation for Checksub.

Checksub primarily serves SMEs and enterprises, where they have the option to work with external providers in the video editor.

Florian expands on the platform's unique features, including animated subtitles and voice cloning, which analyzes the emotion and type of voice and provides a better match for the original speaker.

Florian shares why he took a bootstrapping approach to company building, focusing on achieving product market fit with an eye on long-term sustainability rather than rapid scaling.

When it comes to building tech in the fast-evolving age of AI, Florian keeps up to date with recent research, attends conferences and workshops, and collaborates with industry professionals.

Checksub ensures they remain ahead in the competitive AI market by focusing on skills development in their team. On the roadmap, Florian plans to enhance user experience, improve the translation editor, and add more capabilities to their AI dubbing platform.

Rocio Martinez, VP of Language Services at Jellyfish, joins SlatorPod to talk about the role of creative content localization at the digital marketing agency.

Rocio explains that while the language services team at Jellyfish function as an LSP, their positioning in the market is broader, and they act as an internal solution provider that supports the various divisions within Jellyfish. She emphasizes the importance of connecting every part of the creative process, from creation to adaptation, and monitoring performance and optimization.

Rocio provides an overview of J+ Creative, a proprietary set of tools that assist their teams in the production and localization process, including workflow management, deliveries, approvals, and version control.

Rocio talks about the use of back translations in transcreation, primarily for creative content that requires significant adaptation from the source material. Rocio shares how finding the right copywriting talent and linguists can be a challenge, especially for rare languages or specific sectors.

Rocio discusses the potential of large language models and generative AI tools for content generation and localization. She mentions that Jellyfish is actively testing and integrating AI tools, focusing on improving workflows, and considering prompt engineering to enhance content generation.

Regarding the future, Jellyfish's initiatives revolve around AI, with specialist teams being trained to identify opportunities and implement them across marketing capabilities. Rocio also mentions the potential of Pencil, a generative AI tool for content and asset creation.

In this week’s SlatorPod, we are joined by Graham Neubig, Associate Professor of Computer Science at Carnegie Mellon University, to discuss his research on multilingual natural language processing (NLP) and machine translation (MT).

Graham discusses the research at Neulab, where they focus on various areas of NLP, including incorporating broad knowledge bases into NLP models and code generation.

Graham expands on his Zeno GPT-MT Report comparing large language models (LLMs) with special-purpose machine translation models like Google Translate, Microsoft Translate, and DeepL. He revealed that GPT-4 was competitive from English to other languages, but struggled with very long sentences.

When it comes to cost comparison, Graham highlights that GPT-3.5 Turbo (the model behind the free version of ChatGPT) is significantly cheaper than Google Translate and Microsoft Translator, but GPT-4 (available via OpenAI’s subscription) is more expensive.

Graham predicts that companies will likely move towards using general-purpose LLMs and fine-tuning them for specific tasks like translation. The discussion also covers the recent flurry of speech-to-speech machine translation system releases.

Graham talks about his startup, Inspired Cognition, which aims to provide tools for building and improving AI systems, particularly in text and code generation. Graham concludes the pod with advice for new graduates in the NLP field and his plans for Zeno and the Zeno report.

Florian and Esther discuss the language industry news over the past month, with ChatGPT traffic hitting its peak in May 2023 and large language models being released left, right, and center.

The EU Commission (EC) unveiled the launch of a pilot program that will see the EC use its own machine translation tool to translate certain press releases. In mid-June, media localization company Dotsub announced it had ceased operations, due to its current liquidity situation.

In more positive news, Microsoft's beloved Language Portal has made a comeback after the company announced that it would be shut down at the end of June.

Over in M&A, XL8 has acquired AHT to diversify and grow its customer base to US government agencies. Also branching out is RWS as it bought legal and legislative content management system Propylon for a total purchase price of USD 47.3m.

Meanwhile, in funding, Czech localization startup Localazy announced at the end of June that they raised EUR 1m. Voice generation platform, Eleven Labs, announced it had raised USD 19m in a Series A round. 

Deepdub Co-founder and CEO, Ofir Krakowski, joins SlatorPod to talk about the AI dubbing platform’s goal to make content more accessible through automation and human collaboration.

Ofir’s experience in the Israeli Air Force's machine learning and innovation department led him to use his knowledge in deep learning and generative AI to co-found Deepdub with his brother, Nir.

The CEO outlines the importance of human creativity in translation and how they have built a platform that incorporates both AI solutions and human curation. When it comes to AI dubbing, Deepdub prioritizes authentic and natural-sounding voices and incorporates domain-specific technologies to improve translation accuracy.

Ofir recognizes the impact of generative AI, exemplified by ChatGPT, and emphasizes the importance of building a product rather than just a technology. The company actively engages with organizations such as the BFFS, a union for artists, to find ways to combine human talent with technology.

While Deepdub serves enterprise customers in the entertainment industry, they also cater to professional audiovisual creators by making the dubbing process more readily accessible through their recently launched SaaS product Deepdub GO.

Looking ahead, Deepdub plans further language expansion, with a focus on addressing the diverse linguistic landscape of countries like India.