Tool lets creators reach global audiences by automatically dubbing, syncing videos across languages
A sign with Meta’s logo on it. SOURCE: REUTERS
Meta, the company behind Facebook and Instagram, has expanded its AI capabilities in a move designed to make short-form video content accessible across language barriers.
They have rolled out enhanced AI-powered translations and lip-syncing for Reels, its short video format, supporting a wider range of languages, including Bengali, Tamil, Telugu, Marathi and Kannada alongside previously available English, Hindi, Portuguese and Spanish.
According to the blog post, the update uses neural machine translation models as well as on-device and cloud-based AI models to generate dubbed audio and real-time captions, respectively, which can automatically be suggested to viewers when the language of a Reel differs from that set on their device. The technology goes beyond subtitles and recreates a creator’s voice in the translated language, matching mouth movements in the video and making it appear as though the creator is speaking the new language.
In a statement, Meta said the feature is intended to foster cross-cultural engagement, allowing users to discover content “no matter where it was created or what language it was filmed in.” Early reactions among creators suggest the expanded language support has already broadened the reach of their videos.
The translation and lip-sync tools are free to use, available to all public Instagram accounts in regions where Meta AI is active and to Facebook creators with at least 1,000 followers.
“Many creators want to reach global audiences, and their feedback helped shape our work on translation features,” Meta said in their blog post. “Translations with Meta AI is just one way we’re bringing the best content to audiences and finding innovative new ways to connect people around the world.”
Critics, however, warn about the challenges of accurately capturing context, emotion and cultural nuance in AI translations. It is also possible to intentionally misrepresent what a person says if the audio dub or translation is inaccurate.
