Reinforcement Learning Based Adaptive Blocklength and MCS Selection for Minimization of Age Violation Probability

Download

AysenurOzkaya_Thesis.pdf

Date

2022-1

Author

Özkaya, Ayşenur

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

291
views

94
downloads

As a measure of data freshness, Age of Information (AoI) is an important semantic performance metric in systems where small status update packets need to be delivered to a monitor in a timely manner. This study aims to minimize the age violation probability (AVP), which is defined as the probability that instantaneous age exceeds a certain threshold. The AVP can be considered as one of the key performance indicators in emerging 5G and beyond technologies such as massive machine-to-machine communications (mMTC) and ultra-reliable low latency communications (URLLC). This thesis focuses on two main problems regarding the adaptive transmission of short packets in time-sensitive systems. Firstly, we propose two methods for choosing the optimal blocklength for coding in short packet transmissions. We utilize finite blocklength theory approximations along with dynamic programming (DP) and reinforcement learning (RL) methods. Adopting state-aggregated value iteration and Q-learning algorithms, we present adaptive policies that dynamically select the optimal blocklength according to the state of the system. Our second problem focuses on choosing the appropriate modulation and coding scheme (MCS) for minimization of age violation probability. We construct a deep reinforcement learning (DRL) framework and employ deep Q networks (DQN) to exploit a policy for the dynamic selection of MCS among available MCSs defined in 5G standards. The performances of the proposed approaches are demonstrated in different scenarios and compared with the performances of benchmark policies and state-of-the-art algorithms.

Subject Keywords

Age of information, Dynamic programming, Reinforcement learning, Adaptive modulation and coding, Finite blocklength

URI

https://hdl.handle.net/11511/99608

Collections

Graduate School of Natural and Applied Sciences, Thesis

Suggestions

OpenMETU
Core

Optimizing age of information on real-life TCP/IP connections through reinforcement learning Sert, Egemen; Sonmez, Canberk; Baghaee, Sajjad; Uysal, Elif (2018-07-05) Age of Information (AoI) has emerged as a performance metric capturing the freshness of data for status-update based applications ( e.g. , remote monitoring) as a more suitable alternative to classical network performance indicators such as throughput or delay. Optimizing AoI often requires distinctly novel and sometimes counter-intuitive networking policies that adapt the rate of update transmissions to the randomness in network resources. However, almost all previous work on AoI to data has been theoretic...
Measuring age of information on real-life connections Beytur, Hasan Burhan; Baghaee, Sajjad; Uysal, Elif (2019-04-01) Age of Information (AoI) is a relatively new metric to measure freshness of networked application such as real-time monitoring of status updates or control. The AoI metric is discussed in the literature mainly in a theoretical way. In this work, we want to point out the issues related to the measuring AoI-related values, such as synchronization and calculation of the values. We discussed the effect of synchronization error in the measurement and a solution for calculating an estimate of average AoI without ...
Hierarchical Coding for Cloud Storage: Topology-Adaptivity, Scalability, and Flexibility Yang, Siyi; Hareedy, Ahmed; Calderbank, Robert; Dolecek, Lara (2022-06-01) In order to accommodate the ever-growing data from various, possibly independent, sources and the dynamic nature of data usage rates in practical applications, modern cloud data storage systems are required to be scalable, flexible, and heterogeneous. The recent rise of the blockchain technology is also moving various information systems towards decentralization to achieve high privacy at low costs. While codes with hierarchical locality have been intensively studied in the context of centralized cloud stor...
Age Minimization of Multiple Flows using Reinforcement Learning Beytur, Hasan Burhan; Uysal, Elif (2019-04-11) Age of Information (AoI) is a recently proposed performance metric measuring the freshness of data at the receiving side of a flow. This metric is particularly suited to status-update type information flows, like those occurring in machine-type communication (MTC), remote monitoring and similar applications. In this paper, we consider the problem of AoI-optimal scheduling of multiple flows served by a single server. The performance of scheduling algorithms proposed in previous literature has been shown unde...
Delay and Peak-Age Violation Probability in Short-Packet Transmissions Devassy, Rahul; Durisi, Giuseppe; Ferrante, Guido Carlo; Simeone, Osvaldo; Uysal, Elif (2018-06-22) This paper investigates the distribution of delay and peak age of information in a communication system where packets, generated according to an independent and identically distributed Bernoulli process, are placed in a single-server queue with first-come first-served discipline and transmitted over an additive white Gaussian noise (AWGN) channel. When a packet is correctly decoded, the sender receives an instantaneous error-free positive acknowledgment, upon which it removes the packet from the buffer. In ...

Citation Formats

A. Özkaya, “Reinforcement Learning Based Adaptive Blocklength and MCS Selection for Minimization of Age Violation Probability,” M.S. - Master of Science, Middle East Technical University, 2022.