A novel deep reinforcement learning algorithm for online antenna tuning

Date

2019-12-01

Author

Balevi, Eren
Andrews, Jeffrey G.

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

73
views

0
downloads

© 2019 IEEE.The interactions between the cells, most notably due to their coupled interference and the large number of users, render the optimization of antenna parameters prohibitively complex. To cope with this problem, we propose a novel practical deep learning (DL) based reinforcement learning (RL) algorithm to jointly optimize antenna tilt angle and vertical and horizontal half-power beamwidths of the macrocells in a heterogeneous cellular network (HetNet). In the proposed algorithm, DL is used to extract the features by learning the locations of users, and mean field RL is used to learn the average interference values for different antenna settings. Our results illustrate that the proposed deep RL algorithm can approach the optimum weighted sum rate with hundreds of online trials, as opposed to millions of trials for standard Q-learning, assuming relatively low environmental dynamics. Furthermore, the proposed algorithm is compact and implementable, and empirically appears to provide a performance guarantee regardless of the amount of environmental dynamics.

Subject Keywords

Deep reinforcement learning, HetNets, Online antenna tuning, Q-learning

URI

https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85081947782&origin=inward
https://hdl.handle.net/11511/100200

DOI

https://doi.org/10.1109/globecom38437.2019.9013308

Conference Name

2019 IEEE Global Communications Conference, GLOBECOM 2019

Collections

Department of Electrical and Electronics Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

Online Antenna Tuning in Heterogeneous Cellular Networks With Deep Reinforcement Learning Balevi, Eren; Andrews, Jeffrey G. (2019-12-01) We aim to jointly optimize antenna tilt angle, and vertical and horizontal half-power beamwidths of the macrocells in a heterogeneous cellular network (HetNet). The interactions between the cells, most notably due to their coupled interference render this optimization prohibitively complex. Utilizing a single agent reinforcement learning (RL) algorithm for this optimization becomes quite suboptimum despite its scalability, whereas multi-agent RL algorithms yield better solutions at the expense of scalabilit...
A clustering algorithm that maximizes throughput in 5G heterogeneous F-RAN networks Balevi, Eren; Gitlin, Richard D. (2018-07-27) © 2018 IEEE.In this paper, a clustering algorithm is proposed that dynamically determines the locations of fog nodes in 5G wireless networks, which are upgraded from small cells, in order to maximize throughput assuming the number of fog nodes and small cells are given as a priori information. The proposed algorithm dynamically clusters the small cells around the fog nodes. The approach is based on a soft clustering model where one small cell can be connected to many fog nodes. The numerical results demonst...
Modified neural multiple source tracking algorithm in the presence of mutual coupling Caylar, Selcuk; Leblebicioğlu, Mehmet Kemal; Dural, Guelbin (2007-06-15) In smart antenna systems, mutual coupling between elements can significantly degrade the processing algorithms [1]. In this paper mutual coupling effects on Modified Neural Multiple Source Tracking Algorithm (MN-MUST) has been studied. MN-MUST algorithm applied to the Uniform Circular Array (UCA) geometry for the first time. The validity of MN-MUST algorithm in the presence of mutual coupling has been proved for both Uniform Linear Array (ULA) and UCA. Simulation results of MN-MUST algorithm are provided fo...
A Novel Input Impedance Computation Method for Coaxial Probe Fed Microstrip Antennas by Utilizing Characteristic Modes Cetin, Metehan; Alatan, Lale (2017-07-14) A method to efficiently compute the input impedance of the coaxial probe fed microstrip antennas by using characteristic modes is proposed in this paper. The efficiency is achieved by defining a discontinuous source current at the feed location that models the current injected by the probe. The input impedance is simply expressed in terms of modal excitation coefficients and eigenvalues of each mode.
A Novel Broadband Multilevel Fast Multipole Algorithm With Incomplete-Leaf Tree Structures for Multiscale Electromagnetic Problems Takrimi, Manouchehr; Ergül, Özgür Salih; Erturk, Vakur B. (2016-06-01) An efficient and versatile broadband multilevel fast multipole algorithm (MLFMA), which is capable of handling large multiscale electromagnetic problems with a wide dynamic range of mesh sizes, is presented. By invoking a novel concept of incomplete-leaf tree structures, where only the overcrowded boxes are divided into smaller ones for a given population threshold, versatility of using variable-sized boxes is achieved. Consequently, for geometries containing highly overmeshed local regions, the proposed me...

Citation Formats

E. Balevi and J. G. Andrews, “A novel deep reinforcement learning algorithm for online antenna tuning,” presented at the 2019 IEEE Global Communications Conference, GLOBECOM 2019, Hawaii, Amerika Birleşik Devletleri, 2019, Accessed: 00, 2022. [Online]. Available: https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85081947782&origin=inward.