Multiagent reinforcement learning using function approximation

2000-11-01
Abul, O
Polat, Faruk
Alhajj, R
Learning in a partially observable and nonstationary environment is still one of the challenging problems In the area of multiagent (MA) learning. Reinforcement learning is a generic method that suits the needs of MA learning in many aspects. This paper presents two new multiagent based domain independent coordination mechanisms for reinforcement learning; multiple agents do not require explicit communication among themselves to learn coordinated behavior. The first coordination mechanism Is perceptual coordination mechanism, where other agents are included in state descriptions and coordination information is Learned from state transitions. The second is observing coordination mechanism, which also includes other agents in state descriptions and additionally the rewards of nearby agents are observed from the environment. The observed rewards and agent's own reward are used to construct an optimal policy. This way, the latter mechanism tends to increase region-wide joint rewards. The selected experimented domain is adversarial food-collecting world (AFCW), which can be configured both as single and multiagent environments, Function approximation and generalization techniques are used because of the huge state space. Experimental results show the effectiveness of these mechanisms.
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART C-APPLICATIONS AND REVIEWS

Suggestions

Positive impact of state similarity on reinforcement learning performance
Girgin, Sertan; Polat, Faruk; Alhaj, Reda (Institute of Electrical and Electronics Engineers (IEEE), 2007-10-01)
In this paper, we propose a novel approach to identify states with similar subpolicies and show how they can be integrated into the reinforcement learning framework to improve learning performance. The method utilizes a specialized tree structure to identify common action sequences of states, which are derived from possible optimal policies, and defines a similarity function between two states based on the number of such sequences. Using this similarity function, updates on the action-value function of a st...
Free gait generation with reinforcement learning for a six-legged robot
Erden, Mustafa Suphi; Leblebicioğlu, Mehmet Kemal (Elsevier BV, 2008-03-31)
In this paper the problem of free gait generation and adaptability with reinforcement learning are addressed for a six-legged robot. Using the developed free gait generation algorithm the robot maintains to generate stable gaits according to the commanded velocity. The reinforcement learning scheme incorporated into the free gait generation makes the robot choose more stable states and develop a continuous walking pattern with a larger average stability margin. While walking in normal conditions with no ext...
A pattern classification approach for boosting with genetic algorithms
Yalabık, Ismet; Yarman Vural, Fatoş Tunay; Üçoluk, Göktürk; Şehitoğlu, Onur Tolga (2007-11-09)
Ensemble learning is a multiple-classifier machine learning approach which produces collections and ensembles statistical classifiers to build up more accurate classifier than the individual classifiers. Bagging, boosting and voting methods are the basic examples of ensemble learning. In this study, a novel boosting technique targeting to solve partial problems of AdaBoost, a well-known boosting algorithm, is proposed. The proposed system finds an elegant way of boosting a bunch of classifiers successively ...
FRACTAL SET-THEORETIC ANALYSIS OF PERFORMANCE LOSSES FOR TUNING TRAINING DATA IN LEARNING-SYSTEMS
Erkmen, Aydan Müşerref (1992-08-28)
This paper focuses on the evaluation of learning performance in intelligent dynamic processes with supervised learning. Learning dynamics are characterized by basins of attraction generated by state transitions in control space (statespace + parameter space). State uncertainty is modelled as a cellular control space, namely the cell space. Learning performance losses are related to nonseparable basins of attractions with fuzzy boundaries and to their erosions under parameter changes. Basins erosions are ana...
GENERATING EFFECTIVE INITIATION SETS FOR SUBGOAL-DRIVEN OPTIONS
DEMİR, ALPER; Cilden, Erkin; Polat, Faruk (World Scientific Pub Co Pte Lt, 2019-03-01)
Options framework is one of the prominent models serving as a basis to improve learning speed by means of temporal abstractions. An option is mainly composed of three elements: initiation set, option's local policy and termination condition. Although various attempts exist that focus on how to derive high-quality termination conditions for a given problem, the impact of initiation set generation is relatively unexplored. In this work, we propose an effective goal-oriented heuristic method to derive useful i...
Citation Formats
O. Abul, F. Polat, and R. Alhajj, “Multiagent reinforcement learning using function approximation,” IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART C-APPLICATIONS AND REVIEWS, pp. 485–497, 2000, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/36505.