AUTOMATED VIDEO GAME TESTING USING REINFORCEMENT LEARNING AGENTS

Download

automated_video_game_testing_using_reinforcement_learning_agents.pdf

Date

2022-9-21

Author

Arıyürek, Sinan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

472
views

530
downloads

In this thesis, several methodologies are introduced to automate and improve video game playtesting. These methods are based on Reinforcement Learning (RL) agents. First, synthetic and human-like tester agents are proposed to automate video game testing. The synthetic agent uses test goals generated from game scenarios, and the human-like agent uses test goals extracted from tester trajectories. Tester agents are derived from Sarsa and Monte Carlo Tree Search (MCTS) but focus on finding defects, while traditional game-playing agents focus on maximizing game scores. Second, various MCTS modifications are proposed to enhance the bug-finding capabilities of MCTS. Third, to improve playtesting \textit{developing persona} is introduced, enabling development in an agent's personality and more robust playtesting. RL algorithms aim to find paths that maximize the total accumulated reward. However, discovering various alternative paths that achieve the same objective is equally essential for playtesting. Consequently, Alternative Path Finder (APF) is introduced to let RL agents discover these alternative paths. We experiment with our proposed methodologies using the General Video Game Artificial Intelligence (GVG-AI) and VizDoom frameworks. The experiments reveal that human-like and synthetic agents compete with human testers' bug-finding performances. Furthermore, the experiments indicate that MCTS modifications improve bug-finding performance. Moreover, the experiments show that developing personas provide better insight into the game and how different players would play. Lastly, the alternative paths found by APF are presented and reasoned why traditional RL agents cannot discover those paths.

Subject Keywords

Automated Game Testing, Automated Playtesting, Reinforcement Learning, Monte Carlo Tree Search, Player Modeling, Graph Coverage

URI

https://hdl.handle.net/11511/99066

Collections

Graduate School of Informatics, Thesis

Suggestions

OpenMETU
Core

Using Generative Adversarial Nets on Atari Games for Feature Extraction in Deep Reinforcement Learning Aydın, Ayberk; Sürer, Elif (2020-04-01) Deep Reinforcement Learning (DRL) has been suc-cessfully applied in several research domains such as robotnavigation and automated video game playing. However, thesemethods require excessive computation and interaction with theenvironment, so enhancements on sample efficiency are required.The main reason for this requirement is that sparse and delayedrewards do not provide an effective supervision for representationlearning of deep neural networks. In this study, Proximal Policy...
Automated Video Game Testing Using Synthetic and Human-Like Agents Arıyürek, Sinan; Betin Can, Aysu; Sürer, Elif (2019-06-01) In this paper, we present a new methodology that employs tester agents to automate video game testing. We introduce two types of agents -synthetic and human-like- and two distinct approaches to create them. Our agents are derived from Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents, but focus on finding defects. The synthetic agent uses test goals generated from game scenarios, and these goals are further modified to examine the effects of unintended game transitions. The human-like ag...
Enhancing the Monte Carlo Tree Search Algorithm for Video Game Testing Arıyürek, Sinan; Betin Can, Aysu; Sürer, Elif (2020-03-01) In this paper, we study the effects of several Monte Carlo Tree Search (MCTS) modifications for video game testing. Although MCTS modifications are highly studied in game playing, their impacts on finding bugs are blank. We focused on bug finding in our previous study where we introduced synthetic and human-like test goals and we used these test goals in Sarsa and MCTS agents to find bugs. In this study, we extend the MCTS agent with several modifications for game testing purposes. Furthermore, we present a...
Using Generative Adversarial Nets on Atari Games for Feature Extraction in Deep Reinforcement Learning Aydın, Ayberk; Sürer, Elif (2020-10-05) Deep Reinforcement Learning (DRL) has been successfully applied in several research domains such as robot navigation and automated video game playing. However, these methods require excessive computation and interaction with the environment, so enhancements on sample efficiency are required. The main reason for this requirement is that sparse and delayed rewards do not provide an effective supervision for representation learning of deep neural networks. In this study, Proximal Policy Optimization (PPO) algo...
Automated Video Game Testing Using Synthetic and Human-Like Agents Arıyürek, Sinan; Betin Can, Aysu; Sürer, Elif (2019-10-01) We present a new methodology that employs tester agents to automate video game testing. We introduce two types of agents —synthetic and human-like. Our agents are derived from Sarsa and MCTS but focus on finding defects while traditional game-playing agents focus on maximizing game scores. The synthetic agent uses test goals generated from game scenarios, and these goals are further modified to examine the effects of unintended game transitions. The human-like agent uses test goals extracted by our proposed...

Citation Formats

S. Arıyürek, “AUTOMATED VIDEO GAME TESTING USING REINFORCEMENT LEARNING AGENTS,” Ph.D. - Doctoral Program, Middle East Technical University, 2022.