Google DeepMind Genie 4 World Model Adds Audio

Google DeepMind’s Genie 4

Google DeepMind’s newest world model uses a Gemini model to watch what happens one frame at a time and decide what should happen next. Genie 4, released on Tuesday as a research preview for trusted testers, combines this planning ability with realistic-looking scenes that users can change by typing instructions. It also creates sounds and spoken dialogue as the world develops.

Key Takeaways

• Genie 4 creates interactive worlds with built-in sound, including music, background sounds and dialogue. Users can change the weather, camera angles, people and objects while the simulation is running.
• Gemini models plan events that involve several steps and check new frames to see what happens. In a robotics test, Gemini 4 Argon guided a simulated human-shaped robot through a game of checkers.
• Trusted testers can use Genie 4 now. Google AI Ultra subscribers using Project Genie and early partners using the Gemini API and Google AI Studio are expected to get access next.

A Director in the Loop

Genie 4 creates the world one pixel at a time, while Gemini models do the thinking. Gemini Omni helps each scene look and behave like the real world. During a session, Gemini 3.8 Flash watches the scene, breaks a goal into steps, checks the frames to see whether each step worked, and changes its plan when the world changes or the user gives a new instruction.

Users move through scenes with standard keyboard controls and guide events by typing instructions. They can change the weather, move the camera, or add a person or object. Gemini adjusts each instruction to match what is already on the screen. Google says a scene can remain consistent for up to two minutes.

Tejas Kulkarni, a research director at Google, said at a briefing that the future belongs to “hybrid world models, where you have a combination of Gemini writing code and text.” Genie then turns that output into a simulated world in real time.

Sound, Physics and People

Genie 4 is DeepMind’s first world model that creates audio as part of the process in real time. Wind, waves and music appear alongside the visuals. Characters also speak and respond to what users do as events happen.

The model can also simulate physical effects, such as how air moves around objects and what happens when objects collide. DeepMind says its biggest improvement is how it simulates people. Characters move naturally, react to their surroundings and follow instructions in real time.

In a test connected to Gemini Robotics, a Gemini 4 Argon model broke a task into steps and sent them to Genie 4 as captions while the scene was being created. The demonstration showed Apptronik’s Apollo humanoid robot playing checkers. This suggests that robot actions could be tested in a simulation before a real-world setup is built.

Testers Treat It as a Storyboard

Trusted testers work in creative fields, journalism, education, gaming and engineering. Filmmakers use Genie 4 to make early scene drafts before filming. They can try several versions while changing camera angles, lighting, dialogue and action. Photojournalists are testing how to turn a single old photograph into a scene people can explore. Game developers can turn an idea into a playable prototype before writing code for a game engine.

Google DeepMind engineer Diego Rivas said at the briefing that real-time changes are most helpful when creators want to explore ideas quickly, especially during early concept work and storyboarding. He added that Gemini Omni might one day turn these prototypes into finished creative materials. However, he said that this is still some way off and that the final product experience has not yet been settled.

Known Gaps and the Rollout

DeepMind openly lists the model’s weaknesses. Details and objects can change or lose consistency as a session gets longer, after complicated edits or while users move through a scene. Creating accurate audio in real time, showing readable text inside the world, getting spatial details right and keeping scenes consistent over long periods are still research challenges.

The company says it carried out internal safety tests before releasing the preview. All video and audio output includes its SynthID watermark. Access begins with trusted testers, followed by Google AI Ultra subscribers using Project Genie and early-access partners using the Gemini API and Google AI Studio. A waiting list is available.

A Crowded, Consolidating Field

Genie 4 arrives as world models move from demonstrations toward real products. Runway released its playable GWM Worlds 2 model with audio a couple of months ago. AMD is buying Fei-Fei Li’s World Labs, Amazon is discussing a possible purchase of Decart, and Rivian bought Third Dimension to run self-driving tests in simulations. Google has made a big step forward of its own: Genie 3, introduced in August 2025, ran at 720p resolution and 24 frames per second and produced no sound.

Robotics may offer the biggest business opportunity. Simulated worlds can provide endless situations for training robots. This is the same need that drives work on computer-generated training data for humanoid robots and spatial cameras designed for robot makers. However, some people doubt that these models can predict what would actually happen in the physical world. Until scenes remain consistent for more than a couple of minutes, Genie 4 seems more useful as a place to rehearse and test ideas than as a simulator people can fully rely on.

Written by Alius Noreika