Hierarchical temporal memory based autonomous agent for partially observable video game environments

Download

index.pdf

Date

2017

Author

Sungur, Ali Kaan

Metadata

Show full item record

Item Usage Stats

97
views

224
downloads

Believable non-player characters (NPC) can have a profound impact on the experience that a video game provides. This thesis presents an online, unsupervised and lifelong learning autonomous agent that the player can interact with. It has an architecture utilizing a combination of Hierarchical Temporal Memory and Temporal Difference Learning Lambda with the guidance of neurobiological research. The agent has a visual sensor with an online data stream. Input from this sensor feeds the architecture to model the surrounding environment. The goal of the agent is to learn rewarding sequences of behavior based on the stimulation it receives caused by its actions. It navigates in a procedurally generated three-dimensional environment and is in a continuous learning state adapting the synapses of its neural connectome. The architecture is also capable of being stored and loaded at any point allowing for persistent learning through multiple simulation sessions. The study presents the learning characteristics of the agent on a video game related learning task. We compared the data collected from the experiments with varying parameters along with providing the runtime and serialization performance. The proposed methodology results in an autonomous NPC that can learn rewarding behaviors without any supervision. Moreover, it is also capable of learning specific action sequences via player guidance. The result is a promising and novel NPC architecture that is also relatively open to incremental improvements through the relevant neurobiological studies and the advancements on the theory of Hierarchical Temporal Memory.

Subject Keywords

Video games., Computer games., Computer graphics.

URI

http://etd.lib.metu.edu.tr/upload/12621275/index.pdf
https://hdl.handle.net/11511/26667

Collections

Graduate School of Informatics, Thesis

Suggestions

OpenMETU
Core

Designing and implementing a game development framework for interactive stories and role playing games Bayyurt, Ozan Emirha; Sahillioğlu, Yusuf; Department of Multimedia Informatics (2019) Video games are a great medium for storytelling. Taking advantage of this quality is, however, harder than it needs to be. Making role-playing games takes too much time and work on the part of game developers. Existing tools to develop such games, unfortunately, focus more on already proven design features and mostly duplicate the same game mechanics. This thesis proposes a game development framework to offer a solution to help game developers to build role-playing games in a much easier and quicker fashion...
Applying Computational Aesthetics to a Video Game Application Using Machine Learning ERDEM, Ali Naci; Halıcı, Uğur (2016-07-01) The authors have developed a novel approach to evaluating the aesthetic quality of the camera direction in video game scenes rendered in real time while the game is being played. Their goal was to improve the visual aesthetic quality of computer-generated images using a computational aesthetics approach via a regression machine learning model. Considering the challenges and limitations involved, the proposed approach yielded promising prediction performance. The results show that near-real-time aesthetic an...
Data-driven and anisotropic tearing for cloth simulation Karaöz, Mustafa Mert; İşler, Veysi; Department of Game Technologies (2015) Cloth simulations improve realism of video games by extending interactivity of virtual environment and characters. Cloth is a heterogeneous material and it has anisotropic features which can increase immersion especially in a tearing scenario. Today, most video games ignore this fact mainly due to its complexity. However, style of the final output is highly valuable for video games and addition of anisotropic tearing can improve output quality. Shape of cloths can change, meaning that relative distance of t...
Improving the sight efficiency of visually impaired people with games that can be played with motion capture systems Konya, Oğuz; Çağıltay, Kürşat; Department of Game Technologies (2014) This thesis researches whether the life skills and eye-body coordination of the visually impaired children can be improved through the use of video games which use motion capture system as input. It also researches what kind of video games can be developed and how they can be used in order to provide benefit to visually impaired children. This study was conducted as a mix of qualitative and quantitative studies. It was carried out in six phases. In the first phase, the psychomotor development areas of the v...
Generalised audio synthesis algorithm for simulating firearm and subsonic/supersonic projectiles Saka, Teksin; Hacıhabiboğlu, Hüseyin; Alemdaroğlu, Hüseyin Nafiz; Department of Game Technologies (2015) One of the single largest challenges in today’s game production is the simulation and rendering of realistic hair in real time. In most games, hair and fur are usually covered or simplified with textured meshes. TressFX real-time GPU hair framework, which is used in Tomb Raider (2013) game, includes realistic hair by utilizing parallel nature of GPUs. This framework, however, lacks one of the most distinctive properties of hair: Inter-hair interaction. Even though calculating this interaction in real-time i...

Citation Formats

A. K. Sungur, “Hierarchical temporal memory based autonomous agent for partially observable video game environments,” M.S. - Master of Science, Middle East Technical University, 2017.