Reinforcement Learning versus Conventional Control for Controlling a Planar Bi-rotor Platform with Tail Appendage

Date

2021-08-01

Author

Ugurlu, Halil Ibrahim
Kalkan, Sinan
Saranlı, Afşar

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

248
views

0
downloads

In this paper, we study the conventional and learning-based control approaches for multi-rotor platforms, with and without the presence of an actuated “tail” appendage. A comprehensive experimental comparison between the proven control-theoretic approaches and more recent learning-based ones is one of the contributions. Furthermore, an actuated tail appendage is considered as a deviation from the typical multi-rotor morphology, complicating the control problem but promising some useful applications. Our study also explores, as another contribution, the impact of such an actuated tail on the overall position control for both the conventional as well as learning-based controllers. For the conventional control part, we used a multi-loop architecture where the inner loop regulates the attitude while the outer loop controls the position of the platform. For the learning controller, a multi-layer neural network architecture is used to learn a nonlinear state-feedback controller. To improve the learning and generalization performance of this controller, we adopted a curricular learning approach which gradually increases the difficulty of training samples. For the experiments, a planar bi-rotor platform is modeled in a 2D simulation environment. The planar model avoids mathematical complications while preserving the main attributes of the problem making the results more useful. We observe that both types of controllers achieve reasonable control performance and can solve the position control task. However, neither one shows a clear advantage over the other. The learning-based controller is not intuitive and the system suffers from long training times. The architecture of the multi-loop controller is handcrafted (not required for the learning-based controller) but provides a guaranteed stable behavior.

URI

https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85109734699&origin=inward
https://hdl.handle.net/11511/91466

Journal

Journal of Intelligent and Robotic Systems: Theory and Applications

DOI

https://doi.org/10.1007/s10846-021-01412-3

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

Optimal control of a half-circular compliant legged monopod AYDIN, Yasemin Ozkan; Saranlı, Afşar; Yazıcıoğlu, Yiğit; Saranlı, Uluç; Leblebicioğlu, Mehmet Kemal (2014-12-01) This paper investigates an optimal control strategy for the dynamic locomotion of a simplified planar compliant half-circular legged monopod model. We first present a novel planar leg model which incorporates rolling kinematics and a new compliance model, motivated by the use of similar leg designs on existing platforms. Two locomotion tasks, moving at a prescribed horizontal velocity and a one-shot jump to maximum possible height or length, are then investigated within this model. The designs of two high-l...
Feedback motion planning of a novel fully actuated unmanned surface vehicle via sequential composition of random elliptical funnels Özdemir, Oğuz; Ankaralı, Mustafa Mert; Department of Electrical and Electronics Engineering (2022-12-27) This thesis proposes and analyzes a motion planning and control schema for unmanned surface vehicles that fuses sampling-based approaches’ probabilistic completeness with closed-loop approaches’ robustness. The Proposed schema is based on the sequential composition of elliptical funnels, and it consists of two stages: tree generation and motion control. For validation of the approach, we carried out experiments using both simulation and physical setup besides the mathematical analysis. In order to have a co...
Evolutionary topology optimization of a folding missile wing for stiffness and frequency Ürün, Ata; Şahin, Melin; Gürses, Ercan; Department of Aerospace Engineering (2023-1-25) This thesis presents a study on the topology optimization of a folding wing structure for a cruise missile with the aim of minimizing the weight of the wing while maximizing its stiffness and/or maximizing the selected natural frequency values. The weight of the folding wing has a significant impact on the performance of the opening mechanism and the overall dynamic behavior of the missile. The Bidirectional Evolutionary Structural Optimization (BESO) method, a widely-used topology optimization technique, i...
Reinforcement learning control for helicopter landing in autorotation Kopsa, Kadircan; Kutay, Ali Türker (2018-01-01) This study presents an application of an actor-critic reinforcement learning method to the nonlinear problem of helicopter guidance during autorotation in order to achieve safe landing following engine power loss. A point mass model of an OH-58A helicopter in autorotation was built to simulate autorotation dynamics. The point-mass model includes equations of motion In vertical plane. The states of the point-mass model are the horizontal and vertical velocities, the horizontal and vertical positions, the rot...
Nonlinear Dynamic Inversion Autopilot Design for an Air Defense System with Aerodynamic and Thrust Vector Control Bıyıklı, Rabiya; Yavrucuk, İlkay; Tekin, Raziye; Department of Aerospace Engineering (2022-2) The study proposes complete attitude and acceleration autopilots in all three channels of a highly agile air defense missile by utilizing a subcategory of nonlinear feedback linearization methods Nonlinear Dynamic Inversion (NDI). The autopilot design includes cross-coupling effects enabling bank-to-turn (BTT) maneuvers and a rarely touched topic of control in the boost phase with hybrid control which consists of both aerodynamic fin control and thrust vector control. This piece of work suggests solut...

Citation Formats

H. I. Ugurlu, S. Kalkan, and A. Saranlı, “Reinforcement Learning versus Conventional Control for Controlling a Planar Bi-rotor Platform with Tail Appendage,” Journal of Intelligent and Robotic Systems: Theory and Applications, pp. 0–0, 2021, Accessed: 00, 2021. [Online]. Available: https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85109734699&origin=inward.