Brain-inspired learning for face analysis in artificial neural networks: A multitask and continual learning framework

2023-1
Okcu, Sefa Burak
Show full item record
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
168
views
158
downloads
The phenomenon known as catastrophic forgetting is common in connectionist models while learning from a sequence of data from different distributions. On the other hand, the human brain has the ability to learn from a sequence of experiences continually while retaining old information. Recent studies utilize different brain-inspired methods such as regularization, parameter isolation, and replay to alleviate this problem in artificial systems. Following the previous studies, we investigated different continual learning methods on face analysis tasks involving age estimation, binary gender recognition, emotion recognition, and face recognition. Neurological findings implicate that there are different specialized functional and neural areas in the brain for the perception of faces. Similarly, we analyzed faces in two stages, very common in artificial neural networks: face detection and face attributes analysis. Firstly, experiments for learning face detection and facial landmark detection were conducted by studying multitask learning. Secondly, some continual learning methods inspired by biological systems were leveraged to overcome catastrophic interference in artificial models. In the first experiments, our proposed model was able to learn both face and facial landmark detection efficiently, along with a performance boost. In later experiments, we observed that the utilized continual learning methods performed better on task incremental scenarios than class incremental scenarios. Nevertheless, a combination of two different continual learning methods resulted in remarkable performance improvement in class incremental scenarios. As a result, the combination of different alternative neuroscience-inspired methods is required for mitigating forgetting and approaching multitask performance.
Continual learning, Multitask learning, Catastrophic forgetting, Face detection, Face analysis
https://hdl.handle.net/11511/102081
Graduate School of Informatics, Thesis

Suggestions

On numerical optimization theory of infinite kernel learning
Ozogur-Akyuz, S.; Weber, Gerhard Wilhelm (2010-10-01)
In Machine Learning algorithms, one of the crucial issues is the representation of the data. As the given data source become heterogeneous and the data are large-scale, multiple kernel methods help to classify "nonlinear data". Nevertheless, the finite combinations of kernels are limited up to a finite choice. In order to overcome this discrepancy, a novel method of "infinite" kernel combinations is proposed with the help of infinite and semi-infinite programming regarding all elements in kernel space. Look...
Pattern formation in time series systems due to viscoelastic behavior: Case studies in uniform distribution, normal distribution, stock market index, and music
Gündüz, Güngör (World Scientific Pub Co Pte Lt, 2018-09-01)
A new methodology was introduced to investigate the pattern formation in time series systems due to their viscoelastic behavior. Four stochastic processes, uniform distribution, normal distribution, Nasdaq-100 stock market index, and a melody were studied within this context. The time series data were converted into vectorial forms in a scattering diagram. The sequential vectors can be split into its in-line (or conservative) and out-of-line (or dissipative) components. Thus, one can define the storage and ...
Integrated nonlinear regression analysis of tracer and well test data
Akın, Serhat (Elsevier BV, 2003-08-01)
One frequent observation from conventional pressure transient test analysis is that field data match mathematical models derived for homogeneous systems. This observation suggests that pressure data as presently interpreted may not contain details concerning certain reservoir heterogeneities. On the other hand, tracer tests may be more sensitive to heterogeneous elements present in the reservoir because of the convective nature of the flow test. In this study, a possible improvement of conventional pressure...
Temporal clustering of time series via threshold autoregressive models: application to commodity prices
Aslan, Sipan; Yozgatlıgil, Ceylan; İyigün, Cem (2018-01-01)
The primary aim in this study is grouping time series according to the similarity between their data generating mechanisms (DGMs) rather than comparing pattern similarities in the time series trajectories. The approximation to the DGM of each series is accomplished by fitting the linear autoregressive and the non-linear threshold autoregressive models, and outputs of the estimates are used for feature extraction. Threshold autoregressive models are recognized for their ability to represent nonlinear feature...
Comparison of non-deterministic search techniques in the optimum design of real size steel frames
Hasançebi, Oğuzhan; Doğan, E.; Erdal, F.; Saka, M.P. (Elsevier BV, 2010-9)
There is a noticeable increase in the emergence of non-deterministic search techniques that simulate natural phenomena into a numerical optimization technique in recent years. These techniques are used for developing structural optimization algorithms that are particularly effective for obtaining solutions to discrete programming problems. In this study amongst these techniques genetic algorithms, simulated annealing, evolution strategies, particle swarm optimizer, tabu search, ant colony optimization and h...
Citation Formats
S. B. Okcu, “Brain-inspired learning for face analysis in artificial neural networks: A multitask and continual learning framework,” M.S. - Master of Science, Middle East Technical University, 2023.
METU IIS - Integrated Information Service open@metu.edu.tr