Google DeepMind’s Genie 4 builds AI worlds that talk and change on command

Google DeepMind has released Genie 4, an AI model that creates realistic-looking worlds in real time. These worlds include sound and characters that can speak. On 27 October, the research lab said in a blog post that Genie 4 was available to selected, trusted testers as an early research preview.

Users can guide a simulation with written instructions while it is running. They can change the weather, move the camera, or add a person or object. DeepMind says Genie 4 is its first world model that creates audio directly as part of the process. This includes music, background sounds, and dialogue that responds to what the user does.

“Understanding and simulating the world is a crucial aspect of building AGI,” the DeepMind team wrote.

Gemini helps guide the simulation

Genie 4 builds on Google's Gemini models. Gemini Omni helps determine how the worlds look and behave. Gemini 3.8 Flash acts as an agent that guides the process. It watches the scene, breaks a goal into smaller steps, and checks each new frame to see whether a step worked.

In one experiment, Gemini 4 Argon, the model Google released publicly this month, planned a task for Genie 4 to simulate: Apptronik's Apollo humanoid robot playing checkers. DeepMind says this could create an endless range of situations for training robots.

Scenes can now stay consistent for up to two minutes, twice as long as Genie 3 allows, Rebecca Bellan reported for TechCrunch. DeepMind says the model also handles physics, such as how air affects movement and how objects collide, more realistically. People in the simulations also move more naturally.

“What we’ll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction,” Tejas Kulkarni, a research director at Google, said in a briefing, according to TechCrunch.

Who can use it, and when?

According to DeepMind, testers working in film, journalism, education, gaming, and engineering have used Genie 4 to plan scenes before making them and to build early versions of games. Photojournalists have also tested ways to turn old photographs into simulations. Next, the model will become available to Google AI Ultra subscribers through Project Genie. Early partner organizations will be able to use it through the Gemini API and Google AI Studio. DeepMind has opened a waiting list but has not announced a release date.

Google AI Ultra plans start at $100 per month, Ryan Whitwam wrote for Ars Technica. Ars also noted that the announcement of Genie 3 last year caused gaming stocks to fall sharply.

DeepMind has listed several limits. As a simulation runs for longer or users change it with several new prompts, details may shift and objects may not stay consistent. Making readable text and accurate audio is still a work in progress. All video and audio created by Genie 4 includes SynthID, Google's invisible watermark.

The release comes as companies in the world-model field are joining forces or buying one another. AMD agreed to buy World Labs. Amazon has been in talks to buy Decart, and Rivian bought Third Dimension to help create driving simulations.