A comparative study of deep reinforcement learning methods and conventional controllers for aerial manipulation

Ünal, Kazım Burak
Aerial manipulation with unmanned aerial vehicles is increasingly becoming a necessity in many applications. In this thesis, we analyze the controller approaches for a bi-rotor aerial manipulator for a pick and place operation. First of all, we compare a classical control approach with a minimum snap trajectory generation and Deep Reinforcement actor-critic algorithms for the control of the aerial manipulator. Furthermore, we examine the effects of degrees of freedom of the manipulator for the Deep Reinforcement Learning approaches and analyze how the change of goal position of the object that the aerial manipulator needs to carry affects the training of the learning approaches. Moreover, to obtain a faster convergence for the learning approaches we have added informative states in which the aerial manipulator starts with the object it needs to carry is already grasped. Our results of the 2D simulation environment for the aerial manipulation suggest that all of the actor-critic algorithms yield valuable results with off-policy algorithms being more sample efficient. Still, these algorithms have stability issues that fail in the task for some cases. On the other hand, the classical controller approach does not have this problem but finishes the task slower than the Deep Reinforcement Learning approaches.


A simulation study of ad hoc networking of UAVs with opportunistic resource utilization networks
Lilien, Leszek T.; BEN OTHMANE, Lotfi; Angın, Pelin; DECARLO, Andrew; Salih, Raed M.; BHARGAVA, Bharat (Elsevier BV, 2014-02-01)
Specialized ad hoc networks of unmanned aerial vehicles (UAVs) have been playing increasingly important roles in applications for homeland defense and security. Common resource virtualization techniques are mainly designed for stable networks; they fall short in providing optimal performance in more dynamic networks such as mobile ad hoc networks (MANETs)-due to their highly dynamic and unstable nature. We propose application of Opportunistic Resource Utilization Networks (Oppnets), a novel type of MANETs, ...
A flexible reference point-based multi-objective evolutionary algorithm: An application to the UAV route planning problem
DAŞDEMİR, ERDİ; Köksalan, Mustafa Murat; TEZCANER ÖZTÜRK, DİCLEHAN (Elsevier BV, 2020-02-01)
We study the multi-objective route planning problem of an unmanned air vehicle (UAV) moving in a continuous terrain. In this problem, the UAV starts from a base, visits all targets and returns to the base in a continuous terrain that is monitored by radars. We consider two objectives: minimizing total distance and minimizing radar detection threat. This problem has infinitely many Pareto-optimal points and generating all those points is not possible. We develop a general preference-based multi-objective evo...
Autonomous Fruit Picking With a Team of Aerial Manipulators
Köse, Tahsincan; Ertekin Bolelli, Şeyda; Department of Computer Engineering (2021-9-7)
Manipulation is the ultimate capability for autonomous micro unmanned aerial vehicles (MAVs), which would enable a substantial number of novel use-cases. Precision agriculture is such a domain with plenty of practical problems that could utilize aerial manipulation, which is faster with respect to ground manipulation. Apple harvesting is the most prominent use case with ever-growing percentages in the overall apple production costs due to increasing imbalance between labor supply and demand. Moreover, conte...
Integrating navigation surveillance of unmanned air vehicles into the civilian national airspaces by using ADS-B applications
Pahsa, Alper; Kaya, Pınar; Alat, Gökçen; Baykal, Buyurman (null; 2011-05-12)
A novel flight control algorithm for multicopters
Tekinalp, Ozan (null; 2016-09-29)
A new nonlinear control algorithm is proposed for multi-copter type vertical takeoff vehicles. The algorithm uses linear controllers for the position control in the outer loop. The inner loop attitude control is carried out using quaternion representation. The necessary thrust vector requirements of the position control are converted to inner loop as attitude control commands. The inner loop uses a Lyapunov function based attitude controller utilizing the to-go quaternion calculated from thrust vector comma...
Citation Formats
K. B. Ünal, “A comparative study of deep reinforcement learning methods and conventional controllers for aerial manipulation,” M.S. - Master of Science, Middle East Technical University, 2021.