AI Has Been Mastering Games; Now It Can Also Create Them

Games have always been used to train AI, but it seems the tables have turned now. Researchers have found a way to generate Super Mario levels using large language models, opening the door for generative AI to influence game design in a big way. 

In a research paper published late last week, a group of researchers have detailed a method to use GPT-2 to generate tile-based game levels for old platforming games like Mario and Sonic. According to them, this method not only performs better than techniques like procedural content generation, but can also provide ‘controllable level generation’ through text prompting. 

Mario Gets A GPT Twist

Dubbed as the world’s first ‘text-to-level’ model, MarioGPT is an algorithm that uses GPT-2 to create text-based level blueprints. The model is first trained on a textual representation of Super Mario levels, which substitutes common ‘tiles’ or blocks for an ASCII symbol. For example, a ‘coin’ tile would be denoted by the character ‘o’. These converted levels are then fed into an LLM, powered by GPT-2, as training data. 

As to why the researchers picked GPT-2, Shyam Sudhakaran, the lead author on the paper, said, “I think with small datasets in general, GPT2 is better suited than GPT3, while also being much more lightweight and easier to train. However, in the future, with bigger datasets and more complicated prompts, we may need to use a more sophisticated model like GPT3.”

A visual representation of how the model ‘sees’ a level

The model is then paired with a frozen text encoder based on BART, which functions as an input for the model. GPT-2’s layers are then combined with a cross-attention mechanism, which passes information between an encoder and decoder for more reliable outputs. Using this mechanism, level blueprints are generated using a textual depiction, which can then be converted to a normal level using the tile key. 

This method enables them to prompt certain characteristics using text. Prompts such as ‘many pipes, many enemies, little blocks’, or ‘no pipes, no enemies, many blocks’ will result in the LLM creating levels which reflect these characteristics, as it is still relying on GPT-2’s token prediction system. What’s more, the LLM is also able to extract a path from the level using the same system, as explained by the researchers in their paper. 

This combination of factors allows the agent to generate playable levels still discrete from each other, with the added benefit of user input. This is where this method differs from traditional methods such as procedural generation, which commonly encounters issues such as a lack of meaningful content and a close-ended approach to generation.

Procedural content generation is a method of generating large amounts of content algorithmically for a game. Proc-gen usually combines human-created assets with computer-generated randomness to create new levels for the player to explore. Some examples of games using proc-gen include Minecraft, Dwarf Fortress, and No Man’s Sky, which generate entire worlds using a ‘seed’ of random numbers. 

Proc-gen comes with its own host of issues, leading it to be adopted either by indie games or for content generation in a AAA title. For example, Minecraft uses proc-gen to create a world roughly 60 million metres in diameter, while the Borderlands series uses proc-gen to generate up to 1 billion guns. However, these are an extremely narrow use-case of proc-gen, as using it in a widespread manner will result in a game feeling unfinished or too random. 

Generative AI, on the other hand, holds the potential to change game design for big studios and indie game development alike. Used in conjunction with other techniques, such as zero-shot AI voice generation and AI-generated art, machine learning can form the basis for the next generation of gaming technology. 

Replacing Busy Work

After the first wave of despair that accompanied generative AI, many artists and creative professionals began to realise that it was a good tool to accompany their existing workloads. The same goes for the gaming industry, with many artists finding out that assets can easily be generated with AI. For example, this video game artist on Twitter generated an asset pack of potions using only Stable Diffusion, later turning it into a company called Scenario.gg.

In a time before GenAI, these assets would have been painstakingly drawn and animated by a group of designers, but the current technology allows them to automate this task and move on to the more constructive ones. These kinds of processes are already in practice in the gaming industry, with pre-created assets being one of the biggest ancillary businesses of the gaming industry. 

Similarly, voice acting for video games is undergoing a sea change as well. With the rise of technologies like Eleven Labs’ Speech Synthesis technology, game designers can take a 1-minute sample of their preferred voice actor, feed it into a generative AI algorithm, and receive a voice-over of the text of their choice. Game modders have already begun to use this technology to breathe life back into some older games.

Two modders, going by the names Kezyma and Tewlwolow on Nexus Mods, have dedicated their time and resources to make Morrowind, a 20-year-old action RPG game, fully voice acted. The original game had a lot of written dialogue that wasn’t voice-acted, but using samples of the original voice actors, the modders were able to replicate their voice using AI and inject it into the game. They now plan to voice the entirety of the game using this technology. 

Similarly, a game released recently, titled ‘High on Life’ made gratuitous use of AI art to provide background assets for the game. However, using AI art for game assets has largely been met by vitriolic reactions from both the gaming and artistic communities. The gaming community expressed disdain that the game was created with AI, and the artistic community echoed their already-strong feelings on AI art. On the other hand, the Morrowind voice-over project has been met with resounding support from the community, with many claiming that this is the ‘best AI voice mod’ they’ve used. 

As public sentiment rises against generative AI used in new creative undertakings, one mustn’t forget that this technology can be used to breathe new life into age-old games. Apart from this, smaller developers who don’t have access to the money and resources of big studios can use generative AI to undertake tasks that previously would not be possible. As with any emerging technology, generative AI is proving to be a double-edged sword, leaving its users to decide the ethics behind its usability.