Using Transitional Bottlenecks to Improve Learning in Nearest Sequence Memory Algorithm

Date

2017-11-08

Author

Aydın, Hüseyin
Cilden, Erkin
Polat, Faruk

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

17
views

0
downloads

Cite This

Instance-based methods are proven tools to solve reinforcement learning problems with hidden states. Nearest Sequence Memory (NSM) is a widely known instance-based approach mainly based on k-Nearest Neighbor algorithm. It keeps the history of the agent in terms of action-observation-reward tuples and uses it to vote for the best upcoming action. In this work, an improving heuristic is proposed for the NSM algorithm which provides the agent an additional prior information, namely transitional bottlenecks, on the way to goal. Additionally, a tuple extension pattern is shown to further improve the heuristic by means of ambiguity reduction due to the nature of transitional bottlenecks, thus increase the learning speed. Empirical results indicate a significant improvement in learning performance, in terms of number of steps to goal.

Subject Keywords

Reinforcement learning, Nearest sequence memory, Bottleneck state, Task decomposition

URI

https://hdl.handle.net/11511/41896

Collections

Department of Computer Engineering, Conference / Seminar