A new AI tool, named Muse, has been launched by Microsoft in partnership with Teachable AI Experiences and Ninja Theory, an Xbox Games Studios developer. This AI is designed to help game developers do their work faster.
Muse’s primary function is to produce in-game visuals and the corresponding player actions, using a 3D understanding of the game’s environment. To achieve this, the Muse model went through extensive training, utilizing 1.6 billion parameters, along with 1 billion images and 1 billion controller actions, which equates to roughly 7 years of continuous human gameplay. The training data was derived from Ninja Theory’s game, Bleeding Edge.
Muse was trained on user-consented gameplay data. Specifically, the model processes visual and controller input at a resolution of 300 x 180 pixels. This allows Muse to generate sequences spanning up to two minutes and, using a world model mode, predict future game evolution.
Demos from Microsoft reveal that Muse effectively manages camera transitions, path selection, and even character creation, all while adhering to the game’s established physics. In terms of practical application, Microsoft confirms that Muse is already being leveraged to produce real-time playable AI models in other games.
Beyond its expected development uses, Muse can also be used for bringing classic video games back to life. This AI model could streamline the process of adapting older titles for contemporary hardware.
To expand its accessibility, Microsoft has made Muse’s model weights, sample datasets, and the WHAM (World and Human Action Model) available through Azure AI Foundry. Additionally, later this year, Copilot Labs will offer brief, interactive demonstrations of AI-driven gameplay.
