Date

2022-7-21

Author

Bal, Melis İlayda

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

627
views

149
downloads

Cite This

In a Reinforcement Learning task, a learning agent needs to extract useful information about its uncertain environment in an efficient way during the interaction process to successfully complete the task. Through strategic exploration, the agent acquires sufficient information to adjust its behavior to act intelligently as it interacts with the environment. Therefore, efficient exploration plays a key role in the learning efficiency of Reinforcement Learning tasks. Due to the delayed-feedback nature of Reinforcement Learning settings with sparse explicit reward structure, the required time for learning becomes the main cause of learning inefficiency. This problem is exacerbated particularly in complex tasks with large state and action spaces. Decomposing the task or modifying the reward structure to provide frequent feedback to the agent has been shown to accelerate learning. This thesis proposes two methods with a reward shaping mechanism to address the aforementioned problems. To attack the efficient exploration problem, a framework called population-based repulsive reward shaping mechanism using eligibility traces is proposed under the scope of tabular RL representation. The computational study on benchmark problem domains showed that efficient exploration is achieved with a significant improvement in learning and state-space coverage with the proposed framework. Furthermore, to accelerate learning, the thesis also proposes an approach called potential-based reward shaping using state-space segmentation with the extended segmented Q-Cut algorithm. Experimental results on sparse-reward benchmark domains showed that the proposed method indeed speeds up the learning of the agent without sacrificing computation time.

Subject Keywords

reinforcement learning, coordinated exploration, eligibility traces, potential-based reward shaping, state-space segmentation

URI

https://hdl.handle.net/11511/98151

Collections

Graduate School of Natural and Applied Sciences, Thesis