Active Hackathon

Researchers Introduce Enhanced Deep RL Model For Automated Playtesting

Our focus is on combining DRL and MCTS to predict pass and churn rates as measures of game difficulty and engagement.

The development of a game involves rounds of playtesting to study players’ behaviours and experiences before shaping the final product. However, human playtesting tasks are arduous and repetitive, costly, and can significantly slow down the design and development process.

Automated playtesting minimises the need for human intervention. Existing Deep Reinforcement Learning (DRL) game-playing agents can predict both game difficulty and player engagement.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

Now, researchers from Aalto University and Rovio Entertainment have introduced a novel method for automated playtesting by enhancing DRL with Monte Carlo Tree Search (MCTS). “Our focus is on combining DRL and MCTS to predict pass and churn rates as measures of game difficulty and engagement, and to model the relationship between these two game metrics,” as per the paper.

The researchers took an earlier model presented in the paper titled ‘Predicting Game Difficulty and Churn Without Players’ as the base. The intent behind the model is to integrate AI gameplay with a simulation of how the player population evolves over the stages using DRL.

Method & results

“In this paper, we extend Roohi et al.’s original approach by complementing their DRL gameplay with MCTS and by improving the features extracted from the AI gameplay data. By utilising the same high-level population simulation, reward function, and DRL agents trained for each level, we allow for a direct comparison with their original work.” 

Researchers used MCTS for the pass and churn rate prediction. However, the first thing required was to identify which variant of the algorithm allows an AI agent to play even the game’s hardest levels successfully. Four MCTS algorithms were tested to try the best one:

  • Vanilla MCTS
  • DRL MCTS
  • Myopic MCTS
  • DRL-Myopic MCTS

To select the best MCTS variant, researchers ran each candidate 20 times on each level. One run used 16 parallel instances of the game. This process took approximately 10 hours per level on a 16-core 3.4 GHz Intel Xeon CPU.

By comparing against the original DRL-based approach, the paper demonstrated that combining MCTS and DRL can improve the prediction accuracy while requiring much less computation in solving hard levels than MCTS alone. “Although combinations of MCTS and DRL have been utilised in many state-of-the-art game-playing systems, our work is the first to measure its benefits in predicting ground-truth human player data,” as per the paper.

However, games aren’t just for fun. We can learn how to optimise diverse processes in a range of distinct and intriguing subfields by training a virtual agent to outperform human players. This is what Google DeepMind accomplished with their popular AlphaGo, which defeated the world’s best Go player at the time. From training, game testing agents to evaluating RL agents, recent research works have enhanced the gaming ecosystem.

Recent developments

Google AI came out with an ML-based approach to train game testing agents and found bugs in games. Imitationation learning (IL) – inspired by the DAgger algorithm– informs ML policies by watching professionals play the game. 

A group of researchers from Microsoft Research, the University of Nottingham and UC Berkeley have created a novel methodology for testing the human-AI collaboration of RL agents in the Overcooked two-player gaming environment.

Researchers have effectively used deep reinforcement learning (deep RL) to train agents that can perform well in a variety of contexts in recent years. However, deploying a reinforcement learning agent in a real-world setting necessitates a high level of robustness.

More Great AIM Stories

kumar Gandharv
Kumar Gandharv, PGD in English Journalism (IIMC, Delhi), is setting out on a journey as a tech Journalist at AIM. A keen observer of National and IR-related news.

Our Upcoming Events

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR

Ouch, Cognizant

The company has reduced its full-year 2022 revenue growth guidance to 8.5% – 9.5% in constant currency from the 9-11% in the previous quarter

The curious case of Google Cloud revenue

Porat had earlier said that Google Cloud was putting in money to make more money, but even with the bucket-loads of money that it was making, profitability was still elusive.

[class^="wpforms-"]
[class^="wpforms-"]