Age Minimization of Multiple Flows using Reinforcement Learning

Beytur, Hasan Burhan
Uysal, Elif
Age of Information (AoI) is a recently proposed performance metric measuring the freshness of data at the receiving side of a flow. This metric is particularly suited to status-update type information flows, like those occurring in machine-type communication (MTC), remote monitoring and similar applications. In this paper, we consider the problem of AoI-optimal scheduling of multiple flows served by a single server. The performance of scheduling algorithms proposed in previous literature has been shown under limited assumptions, due to the analytical intractability of the problem. The goal of this paper is to apply reinforcement learning methods to achieve scheduling decisions that are resilient to network conditions and packet arrival processes. Specifically, Policy Gradients and Deep Q-Learning methods are employed. These can adapt to the network without a priori knowledge of its parameters. We study the resulting performance relative to a benchmark, the MAF algorithm, which is known to be optimal under certain conditions.


Optimizing age of information on real-life TCP/IP connections through reinforcement learning
Sert, Egemen; Sonmez, Canberk; Baghaee, Sajjad; Uysal, Elif (2018-07-05)
Age of Information (AoI) has emerged as a performance metric capturing the freshness of data for status-update based applications ( e.g. , remote monitoring) as a more suitable alternative to classical network performance indicators such as throughput or delay. Optimizing AoI often requires distinctly novel and sometimes counter-intuitive networking policies that adapt the rate of update transmissions to the randomness in network resources. However, almost all previous work on AoI to data has been theoretic...
Non-Linear Filtering based on Observations from Student's t Processes
Saha, Saikat; Orguner, Umut; Gustafsson, Fredrik (2012-03-10)
We consider measurements from possibly zero-mean stochastic processes in a nonlinear filtering framework. This is a challenging problem, since it is only the second order properties of the measurements that bear information about the unknown state vector. The covariance function of the measurements can have both spatial and temporal correlation that depend on the state. Recently, a solution to this problem was presented for the case of Gaussian processes. We here extend the theory to Student's t processes. ...
Age-of-Information in Practice: Status Age Measured over TCP/IP Connections through WiFi, Ethernet and LTE
Sonmez, Canberk; Baghaee, Sajjad; Ergisi, Abdussamed; Uysal, Elif (2018-06-07)
The Age of Information (AoI) has gained importance as a Key Performance Indicator (KPI) measuring freshness of information in information-update systems and time-critical applications. Almost all the previous literature on AoI has been theoretical. Most of these theoretical studies assumed knowledge of the statistics of network delays and/or link service times. However, in real-life networks, various factors contribute to these parameters, resulting in a complicated effect on age. This paper reports the res...
Activity Learning from Lifelogging Images
Belli, Kader; Akbaş, Emre; Yazıcı, Adnan (2019-01-01)
The analytics of lifelogging has generated great interest for data scientists because big and multi-dimensional data are generated as a result of lifelogging activities. In this paper, the NTCIR Lifelog dataset is used to learn activities from an image point of view. Minute definitions are classified into activity classes using images and annotations, which serve as a basis for various classification techniques, namely SVMs and convolutional neural network structures (CNN), for learning activities. The perf...
Measuring age of information on real-life connections
Beytur, Hasan Burhan; Baghaee, Sajjad; Uysal, Elif (2019-04-01)
Age of Information (AoI) is a relatively new metric to measure freshness of networked application such as real-time monitoring of status updates or control. The AoI metric is discussed in the literature mainly in a theoretical way. In this work, we want to point out the issues related to the measuring AoI-related values, such as synchronization and calculation of the values. We discussed the effect of synchronization error in the measurement and a solution for calculating an estimate of average AoI without ...
Citation Formats
H. B. Beytur and E. Uysal, “Age Minimization of Multiple Flows using Reinforcement Learning,” presented at the International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, 2019, Accessed: 00, 2020. [Online]. Available: