Demonstratives in the context of a collaborative problem solving task

Date

2018

Author

Büyüktekin, Faruk

Metadata

Show full item record

Item Usage Stats

5
views

1
downloads

Multimodal user interfaces have become increasingly important. They are being used in various applications such as computer games, online education, simulations, and assistive technology. Such interfaces now can make use of inputs coming from different modalities such as speech and eye gaze. This multimodality enables situated dialogue between users and artificial agents, encouraging collaborative language use. Naturally, people need to refer to entities in the surrounding environment using demonstratives during the conversation. Multimodal interfaces must resolve these referring expressions for successful communication. However, demonstrative reference is still a challenging problem due to the complex nature of the phenomenon. Traditional accounts of demonstratives are individual and speaker-centric, suggesting a distance contrast. On the contrary, recent findings reveal it is a joint and multimodal action. Consequently, there is a need to study demonstratives in a situated and distributed framework to better understand the cognitive processes behind reference generation and resolution. To this end, this study investigated the relationship between demonstrative use and eye gaze of the participants collaborating in a situated environment. Our findings suggest demonstrative reference does not occur randomly, rather follows certain cognitive principles, which questions the prevailing speaker-centric view. They clearly show it is a joint activity in which both the speaker and the listener play active roles. They also indicate eye gaze proves to be a significant visual cue, which is temporally linked with demonstrative use.

Subject Keywords

Visual perception., Human-computer interaction., User interfaces (Computer systems)., Grammar, Comparative and general

URI

http://etd.lib.metu.edu.tr/upload/12622508/index.pdf
https://hdl.handle.net/11511/27504

Collections

Graduate School of Informatics, Thesis