Pain Detection from Facial Videos Using Two-Stage Deep Learning

Date

2019-01-01

Author

Menchetti, Guglielmo
Chen, Zhanli
Wilkie, Diana J.
Ansari, Rashid
Yardimci, Yasemin
Cetin, A. Enis

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

233
views

0
downloads

A new method to objectively measure pain using computer vision and machine learning technologies is presented. Our method seeks to capture facial expressions of pain to detect pain, especially when a patients cannot communicate pain verbally. This approach relies on using Facial muscle-based Action Units (AUs), defined by the Facial Action Coding System (FACS), that are associated with pain. It is impractical to use human FACS coding experts in clinical settings to perform this task as it is too labor-intensive and recent research has sought computer-based solutions to the problem. An effective automated system for performing the task is proposed here in which we develop an end-to-end deep learning-based Automated Facial Expression Recognition (AFER) that jointly detects the complete set of pain-related AUs. The facial video clip is processed frame by frame to estimate a vector of AU likelihood values for each frame using a deep convolutional neural network. The AU vectors are concatenated to form a table of AU values for a given video clip. Our results show significantly improved performance compared with those obtained with other known methods.

Subject Keywords

Facial Action Coding System (FACS), Pain, Action Unit (AU), Deep Learning, Convolutional Neural Network

URI

https://hdl.handle.net/11511/68277

DOI

https://doi.org/10.1109/globalsip45357.2019.8969274

Collections

Graduate School of Informatics, Article

Suggestions

OpenMETU
Core

Neural information retrieval: at the end of the early years Onal, Kezban Dilek; Zhang, Ye; Altıngövde, İsmail Sengör; Rahman, Md Mustafizur; Karagöz, Pınar; Braylan, Alex; Dang, Brandon; Chang, Heng-Lu; Kim, Henna; McNamara, Quinten; Angert, Aaron; Banners, Edward; Khetan, Vivek; McDonnell, Tyler; An Thanh Nguyen, An Thanh Nguyen; Xu, Dan; Wallace, Byron C.; de Rijke, Maarten; Lease, Matthew (Springer Science and Business Media LLC, 2018-06-01) A recent "third wave'' of neural network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this ...
Position estimation for timing belt drives of precision machinery using structured neural networks KILIÇ, Ergin; DOĞRUER, CAN ULAŞ; Dölen, Melik; Koku, Ahmet Buğra (2012-05-01) This paper focuses on a viable position estimation scheme for timing-belt drives using artificial neural networks. In this study, the position of a carriage (load) is calculated via a structured neural network topology accepting input from a position sensor on the actuator side of the timing belt. The paper presents a detailed discussion on the source of transmission errors. The characteristics of the error in different operation regimes are exploited to construct different network topologies. That is, a re...
Representing temporal knowledge in connectionist expert systems Alpaslan, Ferda Nur (1996-09-27) This paper introduces a new temporal neural networks model which can be used in connectionist expert systems. Also, a Variation of backpropagation algorithm, called the temporal feedforward backpropagation algorithm is introduced as a method for training the neural network. The algorithm was tested using training examples extracted from a medical expert system. A series of experiments were carried out using the temporal model and the temporal backpropagation algorithm. The experiments indicated that the alg...
Mobile Robot Heading Adjustment Using Radial Basis Function Neural Networks Controller and Reinforcement Learning BAYAR, GÖKHAN; Konukseven, Erhan İlhan; Koku, Ahmet Buğra (2008-10-28) This paper proposes radial basis function neural networks approach to the Solution of a mobile robot heading adjustment using reinforcement learning. In order to control the heading of the mobile robot, the neural networks control system have been constructed and implemented. Neural controller has been charged to enhance the control system by adding some degrees of strength. It has been achieved that neural networks system can learn the relationship between the desired directional heading and the error posi...
Crack detection with deep learning: an exemplary study of data design in architecture Özgenel, Çağlar Fırat; Sorguç, Arzu; Department of Building Science in Architecture (2018) Dramatic increase of available data in the last 20 years transformed the role of data in artificial intelligence algorithms for problem solving. Deep learning embodies potentials for both finding novel correlations within data, and improvement in decision making process in its massiveness. Thus, this approach is prominent in processing such massive data by removing the necessity of explicitly determining features relevant to the solution. Reformulation of the problem in terms of determining which data repre...

Citation Formats

G. Menchetti, Z. Chen, D. J. Wilkie, R. Ansari, Y. Yardimci, and A. E. Cetin, “Pain Detection from Facial Videos Using Two-Stage Deep Learning,” pp. 0–0, 2019, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/68277.