A Concept Filtering Approach for Diverse Density to Discover Subgoals in Reinforcement Learning

Date

2017-11-08

Author

DEMİR, ALPER
Cilden, Erkin
Polat, Faruk

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

246
views

0
downloads

In the reinforcement learning context, subgoal discovery methods aim to find bottlenecks in problem state space so that the problem can naturally be decomposed into smaller subproblems. In this paper, we propose a concept filtering method that extends an existing subgoal discovery method, namely diverse density, to be used for both fully and partially observable RL problems. The proposed method is successful in discovering useful subgoals with the help of multiple instance learning. Compared to the original algorithm, the resulting approach runs significantly faster without sacrificing the solution quality. Moreover, it can effectively be employed to find observational bottlenecks of problems with perceptually aliased states.

Subject Keywords

Reinforcement learning, Subgoal discovery, Diverse density

URI

https://hdl.handle.net/11511/48528

DOI

https://doi.org/10.1109/ictai.2017.00012

Collections

Department of Computer Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

Local Roots A Tree Based Subgoal Discovery Method to Accelerate Reinforcement Learning Demir, Alper; Polat, Faruk; Cilden, Erkin (2016-12-04) Subgoal discovery in reinforcement learning is an effective way of partitioning a problem domain with large state space. Recent research mainly focuses on automatic identification of such subgoals during learning, making use of state transition information gathered during exploration. Mostly based on the options framework, an identified subgoal leads the learning agent to an intermediate region which is known to be useful on the way to goal. In this paper, we propose a novel automatic subgoal discovery meth...
Using chains of bottleneck transitions to decompose and solve reinforcement learning tasks with hidden states Aydın, Hüseyin; Çilden, Erkin; Polat, Faruk (2022-08-01) Reinforcement learning is known to underperform in large and ambiguous problem domains under partial observability. In such cases, a proper decomposition of the task can improve and accelerate the learning process. Even ambiguous and complex problems that are not solvable by conventional methods turn out to be easier to handle by using a convenient problem decomposition, followed by the incorporation of machine learning methods for the sub-problems. Like in most real-life problems, the decomposition of a ta...
EMDD-RL: faster subgoal identification with diverse density in reinforcement learning Sunel, Saim; Polat, Faruk; Department of Computer Engineering (2021-1-15) Diverse Density (DD) algorithm is a well-known multiple instance learning method, also known to be effective to automatically identify sub-goals and improve Reinforcement Learning (RL). Expectation-Maximization Diverse Density (EMDD) improves DD in terms of both speed and accuracy. This study adapts EMDD to automatically identify subgoals for RL which is shown to perform significantly faster (3 to 10 times) than its predecessor, without sacrificing solution quality. The performance of the proposed method na...
A visual interactive approach for scenario-based stochastic multi-objective problems and an application Balibek, E.; Köksalan, Mustafa Murat (2012-12-01) In many practical applications of stochastic programming, discretization of continuous random variables in the form of a scenario tree is required. In this paper, we deal with the randomness in scenario generation and present a visual interactive method for scenario-based stochastic multi-objective problems. The method relies on multi-variate statistical analysis of solutions obtained from a multi-objective stochastic problem to construct joint confidence regions for the objective function values. The decis...
A Modal Superposition Method for the Analysis of Nonlinear Systems Ferhatoglu, Erhan; Ciğeroğlu, Ender; Özgüven, Hasan Nevzat (2016-01-28) In the determination of response of nonlinear structures, computational burden is always a major problem even if frequency domain methods are used. One of the methods used to decrease the computational effort is the modal superposition method for nonlinear systems where the modes of the linear system are used in the calculation. However, depending on the type of the nonlinearity, in order to obtain an accurate response, the number of modes retained in the response calculations needs to be increased, which i...

Citation Formats

A. DEMİR, E. Cilden, and F. Polat, “A Concept Filtering Approach for Diverse Density to Discover Subgoals in Reinforcement Learning,” 2017, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/48528.