Age Minimization of Multiple Flows using Reinforcement Learning

Date

2019-04-11

Author

Beytur, Hasan Burhan
Uysal, Elif

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

138
views

0
downloads

Age of Information (AoI) is a recently proposed performance metric measuring the freshness of data at the receiving side of a flow. This metric is particularly suited to status-update type information flows, like those occurring in machine-type communication (MTC), remote monitoring and similar applications. In this paper, we consider the problem of AoI-optimal scheduling of multiple flows served by a single server. The performance of scheduling algorithms proposed in previous literature has been shown under limited assumptions, due to the analytical intractability of the problem. The goal of this paper is to apply reinforcement learning methods to achieve scheduling decisions that are resilient to network conditions and packet arrival processes. Specifically, Policy Gradients and Deep Q-Learning methods are employed. These can adapt to the network without a priori knowledge of its parameters. We study the resulting performance relative to a benchmark, the MAF algorithm, which is known to be optimal under certain conditions.

Subject Keywords

Scheduling, Age of information, DQN, Policy gradients, Reinforcement learning, Multiple flows, Machine type communication, Multisensor

URI

https://hdl.handle.net/11511/40144

DOI

https://doi.org/10.1109/iccnc.2019.8685524

Conference Name

International Conference on Computing, Networking and Communications (ICNC)

Collections

Department of Electrical and Electronics Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

Reinforcement Learning Based Adaptive Blocklength and MCS Selection for Minimization of Age Violation Probability Özkaya, Ayşenur; Ceran Arslan, Elif Tuğçe; Department of Electrical and Electronics Engineering (2022-1) As a measure of data freshness, Age of Information (AoI) is an important semantic performance metric in systems where small status update packets need to be delivered to a monitor in a timely manner. This study aims to minimize the age violation probability (AVP), which is defined as the probability that instantaneous age exceeds a certain threshold. The AVP can be considered as one of the key performance indicators in emerging 5G and beyond technologies such as massive machine-to-machine communications (mM...
Optimizing age of information on real-life TCP/IP connections through reinforcement learning Sert, Egemen; Sonmez, Canberk; Baghaee, Sajjad; Uysal, Elif (2018-07-05) Age of Information (AoI) has emerged as a performance metric capturing the freshness of data for status-update based applications ( e.g. , remote monitoring) as a more suitable alternative to classical network performance indicators such as throughput or delay. Optimizing AoI often requires distinctly novel and sometimes counter-intuitive networking policies that adapt the rate of update transmissions to the randomness in network resources. However, almost all previous work on AoI to data has been theoretic...
Non-Linear Filtering based on Observations from Student's t Processes Saha, Saikat; Orguner, Umut; Gustafsson, Fredrik (2012-03-10) We consider measurements from possibly zero-mean stochastic processes in a nonlinear filtering framework. This is a challenging problem, since it is only the second order properties of the measurements that bear information about the unknown state vector. The covariance function of the measurements can have both spatial and temporal correlation that depend on the state. Recently, a solution to this problem was presented for the case of Gaussian processes. We here extend the theory to Student's t processes. ...
Age-of-Information in Practice: Status Age Measured over TCP/IP Connections through WiFi, Ethernet and LTE Sonmez, Canberk; Baghaee, Sajjad; Ergisi, Abdussamed; Uysal, Elif (2018-06-07) The Age of Information (AoI) has gained importance as a Key Performance Indicator (KPI) measuring freshness of information in information-update systems and time-critical applications. Almost all the previous literature on AoI has been theoretical. Most of these theoretical studies assumed knowledge of the statistics of network delays and/or link service times. However, in real-life networks, various factors contribute to these parameters, resulting in a complicated effect on age. This paper reports the res...
Activity Learning from Lifelogging Images Belli, Kader; Akbaş, Emre; Yazıcı, Adnan (2019-01-01) The analytics of lifelogging has generated great interest for data scientists because big and multi-dimensional data are generated as a result of lifelogging activities. In this paper, the NTCIR Lifelog dataset is used to learn activities from an image point of view. Minute definitions are classified into activity classes using images and annotations, which serve as a basis for various classification techniques, namely SVMs and convolutional neural network structures (CNN), for learning activities. The perf...

Citation Formats

H. B. Beytur and E. Uysal, “Age Minimization of Multiple Flows using Reinforcement Learning,” presented at the International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, 2019, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/40144.