"Read That Article": Exploring Synergies between Gaze and Speech Interaction

Vieira, Diogo
Freitas, Joao Dinis
Acartürk, Cengiz
Teixeira, Antonio
Sousa, Luis
Silva, Samuel
Candeias, Sara
Dias, Miguel Sales
Gaze information has the potential to benefit Human-Computer Interaction (HCI) tasks, particularly when combined with speech. Gaze can improve our understanding of the user intention, as a secondary input modality, or it can be used as the main input modality by users with some level of permanent or temporary impairments. In this paper we describe a multimodal HCI system prototype which supports speech, gaze and the combination of both. The system has been developed for Active Assisted Living scenarios.


A Multi-perspective Analysis of Social Context and Personal Factors in Office Settings for the Design of an Effective Mobile Notification System
ÇAVDAR, ŞEYMA; Taşkaya Temizel, Tuğba; Musolesi, Mirco; Tino, Peter (2020-03-01)
In this study, we investigate the effects of social context, personal and mobile phone usage on the inference of work engagement/challenge levels of knowledge workers and their responsiveness to well-being related notifications. Our results show that mobile application usage is associated to the responsiveness and work engagement/challenge levels of knowledge workers. We also developed multi-level (within- and between-subjects) models for the inference of attentional states and engagement/challenge levels w...
Trust attribution in collaborative robots: An experimental investigation of non-verbal cues in a virtual human-robot interaction setting
Ahmet Meriç, Özcan; Şahin, Erol; Acartürk, Cengiz; Department of Bioinformatics (2021-6)
This thesis reports the development of non-verbal HRI (Human-Robot Interaction) behaviors on a robotic manipulator, evaluating the role of trust in collaborative assembly tasks. Towards this end, we developed four non-verbal HRI behaviors, namely gazing, head nodding, tilting, and shaking, on a UR5 robotic manipulator. We used them under different degrees of trust of the user to the robot actions. Specifically, we used a certain head-on neck posture for the cobot using the last three links along with the gr...
The IRIS Project A liaison between industry and academia towards natural multimodal communication
Freıtas, Joao; Sara, Candeıas; Mıguel, Sales Dıas; Eduardo, Lleıda; Alfonso, Ortega; Antonıo, Teıxeıra; Samuel, Sılva; Acartürk, Cengiz; Veronıca, Orvalho (null; 2014-11-30)
his paper describes a project with the overall goal of providing a natural interaction communication platform accessible and adapted for all users, especially for people with speech impairments and elderly, by sharing knowledge between Industry and Academia. The platform will adopt the princi-ples of natural user interfaces such as speech, silent speech, gestures, picto-grams, among others, and will provide a set of services that allow easy access to social networks, friends and remote family members, thus ...
Gestures production under instructional context The role of mode of instruction
Melda, Coşkun; Acartürk, Cengiz (Cognitive Science Society ; 2015-09-25)
We aim at examining how communication mode influences the production of gestures under specific contextual environments. Twenty-four participants were asked to present a topic of their choice under three instructional settings: a blackboard, paper-and-pencil, and a tablet. Participants’ gestures were investigated in three groups: deictic gestures that point to entities, representational gestures that present picturable aspects of semantic content, and beat gestures that are speech-related rhythmic hand move...
Eye tracking in multimodal comprehension of graphs
Acartürk, Cengiz (2012-07-31)
Eye tracking methodology has been a major empirical research approach for the study of online comprehension processes in reading and scene viewing. The use of eye tracking methodology for the study of diagrammatic representations, however, has been relatively limited so far. The investigation of specific types of diagrammatic representations, such as statistical graphs is even scarce. In this study, we propose eye tracking as an empirical research approach for a systematic analysis of multimodal comprehensi...
Citation Formats
D. Vieira et al., ““Read That Article”: Exploring Synergies between Gaze and Speech Interaction,” 2015, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/31172.