A reinforcement learning based algorithm for personalization of digital, just-in-time, adaptive interventions

Date

2021-05-01

Author

Gönül, Suat
Namlı, Tuncay
Coşar, Ahmet
Toroslu, İsmail Hakkı

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

231
views

0
downloads

Suboptimal health related behaviors and habits; and resulting chronic diseases are responsible for majority of deaths globally. Studies show that providing personalized support to patients yield improved results by preventing and/or timely treatment of these problems. Digital, just-in-time and adaptive interventions are mobile phone-based notifications that are being utilized to support people wherever and whenever necessary in coping with their health problems. In this research, we propose a reinforcement learning-based mechanism to personalize interventions in terms of timing, frequency and preferred type(s). We simultaneously employ two reinforcement learning models, namely intervention-selection and opportune-moment-identification; capturing and exploiting changes in people's long-term and momentary contexts respectively. While the intervention-selection model adapts the intervention delivery with respect to type and frequency, the opportune-moment-identification model tries to find the most opportune moments to deliver interventions throughout a day. We propose two accelerator techniques over the standard reinforcement learning algorithms to boost learning performance. First, we propose a customized version of eligibility traces for rewarding past actions throughout an agent's trajectory. Second, we utilize the transfer learning method to reuse knowledge across multiple learning environments. We validate the proposed approach in a simulated experiment where we simulate four personas differing in their daily activities, preferences on specific intervention types and attitudes towards the targeted behavior. Our experiments show that the proposed approach yields better results compared to the standard reinforcement learning algorithms and successfully capture the simulated variations associated with the personas.

Subject Keywords

Intervention delivery optimization, Just-in-time adaptive interventions, Personalized health interventions

URI

https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85103689016&origin=inward
https://hdl.handle.net/11511/90838

Journal

Artificial Intelligence in Medicine

DOI

https://doi.org/10.1016/j.artmed.2021.102062

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

An expandable approach for design and personalization of digital, just-in-time adaptive interventions Gonul, Suat; Namli, Tuncay; Huisman, Sasja; Erturkmen, Gokce Banu Laleci; Toroslu, İsmail Hakkı; Coşar, Ahmet (2019-03-01) Objective: We aim to deliver a framework with 2 main objectives: 1) facilitating the design of theory-driven, adaptive, digital interventions addressing chronic illnesses or health problems and 2) producing personalized intervention delivery strategies to support self-management by optimizing various intervention components tailored to people's individual needs, momentary contexts, and psychosocial variables.
A Framework for design and personalization of digital, just-in-time, adaptive interventions Gönül, Suat; Coşar, Ahmet; Department of Computer Engineering (2018) Adverse and suboptimal health behaviors and chronic diseases are responsible from a substantial majority of deaths globally. Studies show that personalized support programs yield better results in overcoming these undesired behaviors and diseases. Digital, just-in-time, adaptive interventions are mobile phone-based notifications that are being used to support people wherever and whenever needed in coping with the health problem. In this study, a framework is proposed for design and personalization of such i...
An investigation of school walkability in different neighborhood contexts in Ankara with the proposed GIS tool Reyhan, Simge; Severcan, Yücel Can; Akyürek, Sevda Zuhal; Department of Geodetic and Geographical Information Technologies (2022-2-01) Children walking to school contributes to the daily required physical activity recommended by health guidelines and makes them less prone to obesity and mental health problems. Urban form characteristics such as distance, density, diversity that have the potential to encourage children to walk to school are essential for the public health and livability of cities. The main aim of this study is to analyze and compare the school walkability in the different neighborhoods to understand the role of neighborhood...
A comprehensive model for obsessive-compulsive disorder symptoms: a cross-cultural investigation of cognitive and other vulnerability factors Yorulmaz, Orçun; Gençöz, Tülin; Department of Psychology (2007) The current coginitive models of the Obsessive-Compulsive Disorder (OCD) symptoms focuses on the different cognitive factors. Like other nonspecific and noncognitive variables, these factors may also function as vulnerability factors. However, they have been mostly studied separately and majority of the findings in the literature come from the Western samples. Accordingly, the studies examining these factors together and the impact of the culture in these studies are sparse in number. The present study sugg...
An investigation of adaptive and maladaptive dimensions of perfectionism in relation to adult attachment and big five personality traits Ulu, İnci Pınar; Tezer, Esin; Department of Educational Sciences (2007) The present study investigated the role of anxiety and avoidance dimensions of attachment and big five personality traits in adaptive and maladaptive dimensions of perfectionism. A pilot study was carried out with 408 (260 males and 148 females) preparatory school students of Middle East Technical University (METU) for the adaptation studies of Almost Perfect Scale-Revised (APS-R). The results of the exploratory and confirmatory factor analyses yielded a 21 item scale with three factors; Standards, Discrepa...

Citation Formats

S. Gönül, T. Namlı, A. Coşar, and İ. H. Toroslu, “A reinforcement learning based algorithm for personalization of digital, just-in-time, adaptive interventions,” Artificial Intelligence in Medicine, pp. 0–0, 2021, Accessed: 00, 2021. [Online]. Available: https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85103689016&origin=inward.