Meta has introduced a groundbreaking AI model named SeamlessM4T, designed to revolutionize language translation and transcription. With the ability to understand and translate text and speech across nearly 100 languages, SeamlessM4T is a multifaceted tool catering to various communication needs.
Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model.
— Meta AI (@MetaAI) August 22, 2023
This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task.
Details ⬇️
The model's features include translating speech into text for almost 100 input and output languages, translating speech into speech with 36 output languages, as well as converting text into speech for 35 output languages. Notably, SeamlessM4T is attuned to instances where speakers switch languages mid-conversation, enabling seamless translation for multilingual discussions.
SeamlessM4T represents a significant breakthrough in the field of speech-to-speech & speech-to-text by addressing the challenges of limited language coverage & a reliance on separate systems.
— Meta AI (@MetaAI) August 23, 2023
More details ➡️ https://t.co/BIQk48gDcc pic.twitter.com/A21CWQ4kiu
By merging speech and text translation in a single system, SeamlessM4T improves translation efficiency, accuracy, and quality. Released under a research license, this model empowers AI researchers and translators.
Bradley Rhodes
Moreover, Meta's publication of the SeamlessAlign dataset, containing over 270,000 hours of speech and text, contributes to the development of AI-driven translation tools.