Batch mode reinforcement learning for controlling gene regulatory networks and multi-model gene expression data enrichment framework

Download

index.pdf

Date

2013

Author

Şirin, Utku

Metadata

Show full item record

Item Usage Stats

129
views

86
downloads

Over the last decade, modeling and controlling gene regulation has received much attention. In this thesis, we have attempted to solve (i) controlling gene regulation systems and (ii) generating high quality artificial gene expression data problems. For controlling gene regulation systems, we have proposed three control solutions based on Batch Mode Reinforcement Learning (Batch RL) techniques. We have proposed one control solution for fully, and two control solutions for partially observable gene regulation systems. For controlling fully observable gene regulation systems, we have proposed a method producing approximate control policies directly from gene expression data without making use of any computational model. Results show that our proposed method is able to produce approximate control policies for gene regulation systems of several thousands of genes just in seconds without loosing significant performance; whereas existing studies get stuck even for several tens of genes. For controlling partially observable gene regulation systems, firstly, we have proposed a novel Batch RL framework for partially observable environments, Batch Mode TD(λ). Its idea is to produce approximate stochastic control policies mapping observations directly to actions probabilistically without estimating actual internal states of the regulation system. Results show that Batch Mode TD(λ) is able to produce successful stochastic policies for regulation systems of several thousands of genes in seconds; whereas existing studies cannot produce control solution for regulation systems of several tens of genes. To our best knowledge, Batch Mode TD(λ) is the first framework for solving non-Markovian decision tasks with limited number of experience tuples. For controlling partially observable gene regulation systems, secondly, we have proposed a method to construct a Partially Observable Markov Decision Process (POMDP) directly from gene expression data. Our novel POMDP construction method calculates approximate observation-action values for each possible observation, and applies hidden state identification techniques to those approximate values for building the ultimate POMDP. Results show that our constructed POMDPs perform better than existing solutions in terms of both time requirements and solution quality. For generating high quality artificial gene expression data, we have proposed a novel multi-model gene expression data enrichment framework. We have combined four gene expression data generation models into one unified framework, and tried to benefit all of them concurrently. We have sampled from each generative models separately, pooled the generated samples, and output the best ones based on a multi-objective selection mechanism. Results show that our proposed multi-model gene expression data generation framework is able to produce high quality artificial samples from which inferred regulatory networks are better than the regulatory networks inferred from original datasets.

Subject Keywords

Gene regulatory networks., Gene expression., Genetic regulation., Markov processes.

URI

http://etd.lib.metu.edu.tr/upload/12616020/index.pdf
https://hdl.handle.net/11511/22841

Collections

Graduate School of Natural and Applied Sciences, Thesis

Suggestions

OpenMETU
Core

Batch Mode TD(lambda) for Controlling Partially Observable Gene Regulatory Networks Sirin, Utku; Polat, Faruk; Alhajj, Reda (2017-11-01) External control of gene regulatory networks (GRNs) has received much attention in recent years. The aim is to find a series of actions to apply to a gene regulation system making it avoid its diseased states. In this work, we propose a novel method for controlling partially observable GRNs combining batch mode reinforcement learning (Batch RL) and TD(lambda) algorithms. Unlike the existing studies inferring a computational model from gene expression data, and obtaining a control policy over the constructed...
Efficient partially observable markov decision process based formulation of gene regulatory network control problem Erdoğdu, Utku; Polat, Faruk; Alhajj, Reda; Department of Computer Engineering (2012) The need to analyze and closely study the gene related mechanisms motivated the research on the modeling and control of gene regulatory networks (GRN). Di erent approaches exist to model GRNs; they are mostly simulated as mathematical models that represent relationships between genes. Though it turns into a more challenging problem, we argue that partial observability would be a more natural and realistic method for handling the control of GRNs. Partial observability is a fundamental aspect of the problem; ...
Mathematical Modeling and Approximation of Gene Expression Patterns Yılmaz, Fatih; Öktem, Hüseyin Avni (2004-09-03) This study concerns modeling, approximation and inference of gene regulatory dynamics on the basis of gene expression patterns. The dynamical behavior of gene expressions is represented by a system of ordinary differential equations. We introduce a gene-interaction matrix with some nonlinear entries, in particular, quadratic polynomials of the expression levels to keep the system solvable. The model parameters are determined by using optimization. Then, we provide the time-discrete approximation of our time...
Integer linear programming based solutions for construction of biological networks Eren Özsoy, Öykü; Can, Tolga; Department of Health Informatics (2014) Inference of gene regulatory or signaling networks from perturbation experiments and gene expression assays is one of the challenging problems in bioinformatics. Recently, the inference problem has been formulated as a reference network editing problem and it has been show that finding the minimum number of edit operations on a reference network in order to comply with perturbation experiments is an NP-complete problem. In this dissertation, we propose linear programming based solutions for reconstruction o...
Inference of Gene Regulatory Networks Via Multiple Data Sources and a Recommendation Method Ozsoy, Makbule Gulcin; Polat, Faruk; Alhajj, Reda (2015-11-12) Gene regulatory networks (GRNs) are composed of biological components, including genes, proteins and metabolites, and their interactions. In general, computational methods are used to infer the connections among these components. However, computational methods should take into account the general features of the GRNs, which are sparseness, scale-free topology, modularity and structure of the inferred networks. In this work, observing the common aspects between recommendation systems and GRNs, we decided to ...

Citation Formats

U. Şirin, “Batch mode reinforcement learning for controlling gene regulatory networks and multi-model gene expression data enrichment framework,” M.S. - Master of Science, Middle East Technical University, 2013.