Meta just announced its own media-focused AI model, called Movie Gen, which can be used to generate realistic video and audio clips.
The company shared several 10-second clips generated with Movie generationincluding a swimming Moo Deng-style baby hippopotamus, to demonstrate its capabilities. While the tool is not yet available for use, this announcement from Movie Gen comes shortly after its Meta Connect event, which showed off new, updated hardware and the latest version of its big language model, Llama 3.2.
Going beyond generating simple text-to-video clips, the Movie Gen model can make specific edits to an existing clip, such as adding an object to someone’s hands or changing the appearance of a surface. In one of Meta’s example videos, a woman wearing a virtual reality headset was transformed to look like she was wearing steampunk binoculars.
Audio snippets can be generated alongside videos with Movie Gen. In sample clips, an AI man stands near a waterfall with audible splashes and the hopeful sounds of a symphony; A sports car engine purrs and tires screech as it races down the track, and a snake slithers across the jungle floor, accompanied by suspenseful horns.
Meta shared more details about Movie Gen in a research paper published on Friday. Movie Gen Video consists of 30 billion parameters, while Movie Gen Audio consists of 13 billion parameters. (A model’s parameter count roughly corresponds to its capacity; by contrast, the largest variant of Llama 3.1 has 405 billion parameters.) Movie Gen can produce high-definition videos up to 16 seconds long, and Meta claims it outperforms competing models. in the overall quality of the video.
Earlier this year, CEO Mark Zuckerberg demonstrated Meta AI’s Imagine Me feature, where users can upload a photo of themselves and render their face in multiple scenarios, posting an AI image of themselves. drowning in gold chains in Threads. A video version of a similar feature is possible with the Movie Gen model; Consider it a kind of ElfYourself on steroids.
What information has Movie Gen been trained on? Details are unclear in Meta’s announcement post: “We have trained these models on a combination of licensed and publicly available datasets.” Training data sources and what’s fair to pull from the web remain a contentious topic for generative AI tools, and it’s rarely public knowledge what text, video, or audio clips were used to create any of the models. main.
It will be interesting to see how long it takes for Meta to make Movie Gen widely available. The announcement blog vaguely notes a “possible future release.” For comparison, OpenAI announced its AI video model, called Sora, earlier this year and has yet to make it available to the public or share any upcoming release dates (although WIRED did receive some exclusive Sora clips from company for an investigation into bias). ).
Considering Meta’s legacy as a social media company, it’s possible that Movie Gen-powered tools will eventually begin to appear within Facebook, Instagram, and WhatsApp. In September, competitor Google shared plans to make aspects of its Veo video model available to creators within its YouTube Shorts sometime next year.
While larger tech companies are still holding off on fully releasing video models to the public, right now you can experiment with AI video tools from smaller, upcoming startups like Clue and pika. Try Pikaffects if you’ve ever been curious about what it would be like to see yourself. cartoonishly crushed with a hydraulic press or suddenly melting into a puddle.