Memory Efficient Factored Abstraction for Reinforcement Learning

Date

2015-06-26

Author

Sahin, Coskun
Cilden, Erkin
Polat, Faruk

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

104
views

0
downloads

Classical reinforcement learning techniques are often inadequate for problems with large state-space due to curse of dimensionality. If the states can be represented as a set of variables, it is possible to model the environment more compactly. Automatic detection and use of temporal abstractions during learning was proven to be effective to increase learning speed. In this paper, we propose a factored automatic temporal abstraction method based on an existing temporal abstraction strategy, namely extended sequence tree algorithm, by taking care of state differences via state variable changes. The proposed method has been shown to provide significant memory gain on selected benchmark problems.

Subject Keywords

Reinforcement learning, Factored MDP, Learning abstractions, Extended sequence tree

URI

https://hdl.handle.net/11511/52779

Collections

Department of Computer Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

Recursive Compositional Reinforcement Learning for Continuous Control Sürekli Kontrol Uygulamalari için Özyinelemeli Bileşimsel Pekiştirmeli Öǧrenme Tanik, Guven Orkun; Ertekin Bolelli, Şeyda (2022-01-01) Compositional and temporal abstraction is the key to improving learning and planning in reinforcement learning. Modern real-world control problems call for continuous control domains and robust, sample efficient and explainable control frameworks. We are presenting a framework for recursively composing control skills to solve compositional and progressively complex tasks. The framework promotes reuse of skills, and as a result quickly adaptable to new tasks. The decision-tree can be observed, providing insi...
Toward Generalization of Automated Temporal Abstraction to Partially Observable Reinforcement Learning Cilden, Erkin; Polat, Faruk (2015-08-01) Temporal abstraction for reinforcement learning (RL) aims to decrease learning time by making use of repeated sub-policy patterns in the learning task. Automatic extraction of abstractions during RL process is difficult but has many challenges such as dealing with the curse of dimensionality. Various studies have explored the subject under the assumption that the problem domain is fully observable by the learning agent. Learning abstractions for partially observable RL is a relatively less explored area. In...
Abstraction in Model Based Partially Observable Reinforcement Learning using Extended Sequence Trees Cilden, Erkin; Polat, Faruk (2012-12-07) Extended sequence tree is a direct method for automatic generation of useful abstractions in reinforcement learning, designed for problems that can be modelled as Markov decision process. This paper proposes a method to expand the extended sequence tree method over reinforcement learning to cover partial observability formalized via partially observable Markov decision process through belief state formalism. This expansion requires a reasonable approximation of information state. Inspired by statistical ran...
Improving reinforcement learning using distinctive clues of the environment Demir, Alper; Polat, Faruk; Department of Computer Engineering (2019) Effective decomposition and abstraction has been shown to improve the performance of Reinforcement Learning. An agent can use the clues from the environment to either partition the problem into sub-problems or get informed about its progress in a given task. In a fully observable environment such clues may come from subgoals while in a partially observable environment they may be provided by unique experiences. The contribution of this thesis is two fold; first improvements over automatic subgoal identifica...
Attention mechanisms for semantic few-shot learning Baran, Orhun Buğra; Cinbiş, Ramazan Gökberk; İkizler-Cinbiş, Nazlı; Department of Computer Engineering (2021-9-1) One of the fundamental difficulties in contemporary supervised learning approaches is the dependency on labelled examples. Most state-of-the-art deep architectures, in particular, tend to perform poorly in the absence of large-scale annotated training sets. In many practical problems, however, it is not feasible to construct sufficiently large training sets, especially in problems involving sensitive information or consisting of a large set of fine-grained classes. One of the main topics in machine learning...

Citation Formats

C. Sahin, E. Cilden, and F. Polat, “Memory Efficient Factored Abstraction for Reinforcement Learning,” 2015, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/52779.