Pain Detection from Facial Videos Using Two-Stage Deep Learning

Menchetti, Guglielmo
Chen, Zhanli
Wilkie, Diana J.
Ansari, Rashid
Yardimci, Yasemin
Cetin, A. Enis
A new method to objectively measure pain using computer vision and machine learning technologies is presented. Our method seeks to capture facial expressions of pain to detect pain, especially when a patients cannot communicate pain verbally. This approach relies on using Facial muscle-based Action Units (AUs), defined by the Facial Action Coding System (FACS), that are associated with pain. It is impractical to use human FACS coding experts in clinical settings to perform this task as it is too labor-intensive and recent research has sought computer-based solutions to the problem. An effective automated system for performing the task is proposed here in which we develop an end-to-end deep learning-based Automated Facial Expression Recognition (AFER) that jointly detects the complete set of pain-related AUs. The facial video clip is processed frame by frame to estimate a vector of AU likelihood values for each frame using a deep convolutional neural network. The AU vectors are concatenated to form a table of AU values for a given video clip. Our results show significantly improved performance compared with those obtained with other known methods.


Neural information retrieval: at the end of the early years
Onal, Kezban Dilek; Zhang, Ye; Altıngövde, İsmail Sengör; Rahman, Md Mustafizur; Karagöz, Pınar; Braylan, Alex; Dang, Brandon; Chang, Heng-Lu; Kim, Henna; McNamara, Quinten; Angert, Aaron; Banners, Edward; Khetan, Vivek; McDonnell, Tyler; An Thanh Nguyen, An Thanh Nguyen; Xu, Dan; Wallace, Byron C.; de Rijke, Maarten; Lease, Matthew (Springer Science and Business Media LLC, 2018-06-01)
A recent "third wave'' of neural network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this ...
Position estimation for timing belt drives of precision machinery using structured neural networks
KILIÇ, Ergin; DOĞRUER, CAN ULAŞ; Dölen, Melik; Koku, Ahmet Buğra (2012-05-01)
This paper focuses on a viable position estimation scheme for timing-belt drives using artificial neural networks. In this study, the position of a carriage (load) is calculated via a structured neural network topology accepting input from a position sensor on the actuator side of the timing belt. The paper presents a detailed discussion on the source of transmission errors. The characteristics of the error in different operation regimes are exploited to construct different network topologies. That is, a re...
Representing temporal knowledge in connectionist expert systems
Alpaslan, Ferda Nur (1996-09-27)
This paper introduces a new temporal neural networks model which can be used in connectionist expert systems. Also, a Variation of backpropagation algorithm, called the temporal feedforward backpropagation algorithm is introduced as a method for training the neural network. The algorithm was tested using training examples extracted from a medical expert system. A series of experiments were carried out using the temporal model and the temporal backpropagation algorithm. The experiments indicated that the alg...
Mobile Robot Heading Adjustment Using Radial Basis Function Neural Networks Controller and Reinforcement Learning
BAYAR, GÖKHAN; Konukseven, Erhan İlhan; Koku, Ahmet Buğra (2008-10-28)
This paper proposes radial basis function neural networks approach to the Solution of a mobile robot heading adjustment using reinforcement learning. In order to control the heading of the mobile robot, the neural networks control system have been constructed and implemented. Neural controller has been charged to enhance the control system by adding some degrees of strength. It has been achieved that neural networks system can learn the relationship between the desired directional heading and the error posi...
Multi-task Deep Neural Networks in Protein Function Prediction
Rifaioğlu, Ahmet Süreyya; Doğan, Tunca; Martin, Maria Jesus; Atalay, Rengül; Atalay, Mehmet Volkan (2017-05-01)
In recent years, deep learning algorithms have outperformed the state-of-the art methods in several areas thanks to the efficient methods for training and for preventing overfitting, advancement in computer hardware, the availability of vast amount data. The high performance of multi-task deep neural networks in drug discovery has attracted the attention to deep learning algorithms in bioinformatics area. Here, we proposed a hierarchical multi-task deep neural network architecture based on Gene Ontology (GO...
Citation Formats
G. Menchetti, Z. Chen, D. J. Wilkie, R. Ansari, Y. Yardimci, and A. E. Cetin, “Pain Detection from Facial Videos Using Two-Stage Deep Learning,” pp. 0–0, 2019, Accessed: 00, 2020. [Online]. Available: