Show/Hide Menu
Hide/Show Apps
Logout
Türkçe
Türkçe
Search
Search
Login
Login
OpenMETU
OpenMETU
About
About
Open Science Policy
Open Science Policy
Open Access Guideline
Open Access Guideline
Postgraduate Thesis Guideline
Postgraduate Thesis Guideline
Communities & Collections
Communities & Collections
Help
Help
Frequently Asked Questions
Frequently Asked Questions
Guides
Guides
Thesis submission
Thesis submission
MS without thesis term project submission
MS without thesis term project submission
Publication submission with DOI
Publication submission with DOI
Publication submission
Publication submission
Supporting Information
Supporting Information
General Information
General Information
Copyright, Embargo and License
Copyright, Embargo and License
Contact us
Contact us
Multimodal communication in instructional settings: An investigation of the functional roles of gestures and arrows
Download
index.pdf
Date
2021-12-01
Author
Acartürk, Cengiz
Emil, Serap
Metadata
Show full item record
This work is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
.
Item Usage Stats
143
views
41
downloads
Cite This
In daily communication and instructional settings, arrows and spontaneous gestures usually convey similar spatial information to interlocutors. They can visualize spatial information in 2D or 3D space based on shared conceptual and spatial representations. The present study investigates the relationship between spontaneous gestures and arrows from the perspective of generation and comprehension in instructional settings. We introduce a corpus of arrows in written multimodal text and a corpus of gestures generated during communication through multimodal instructional material. We also report an experimental investigation that used corpus material to study the impact of gestures on comprehension learning. A comparison of the corpora reveals functional similarities and differences between arrows and gestures and their complementary role in multimodal communication. The findings suggest that arrows and gestures may share a common conceptual space during communication, having a significant impact on comprehension and learning.
Subject Keywords
Modality
,
Deixis
,
Arrows, Gestures
,
Multimodality
URI
https://hdl.handle.net/11511/95212
Journal
Revista Signos
DOI
https://doi.org/10.4067/s0718-09342021000300867
Collections
Graduate School of Informatics, Article
Suggestions
OpenMETU
Core
Gestures in communication through line graphs
Özge, Alaçam; Christopher, Habel; Acartürk, Cengiz (null; 2012-08-31)
Line graphs are widely used in communication settings, for conveying information about states and processes that unfold in time. The communication is achieved by the contribution of other modalities than graphs, such as language and gestures. In a set of experimental investigations, we analyzed the production and comprehension of gestures during communication through line graphs. The findings reveal a systematic use of gestures as well as the limitations of cognitive resources due to the split of attention ...
Multimodal comprehension of graphics with textual annotations The role of graphical means relating annotations and graph lines
Acartürk, Cengiz; Çağıltay, Kürşat (2008-09-30)
Graphs are often accompanied by text, i.e. linguistically coded information. augmenting information presented diagrammatically. Thus graph comprehension by human Often Constitutes comprehension and integration of information provided by different representational modalities, namely graphical elements and Verbal Constituents. In this Study We focus on text annotations to line graphs providing information about events, processes and their temporal properties its well it temporal relations about the events and...
Bimodal visualisation: A financial trading case study
Taşkaya Temizel, Tuğba (2003-01-01)
A visualisation system that deals with two modalities of information - numerical and textual - is presented. The current application domain is that of prediction in financial trading. The system synthesises news stories and time series data to generate buy/sell signals for financial instruments.
Multichannel Audio Coding Based on Analysis by Synthesis
Elfitri, Ikhwana; Günel Kılıç, Banu; AHMET, KONDOZ (2011-04-01)
Spatial hearing enables translation of an auditory scene into a perceived 3-D image by interpreting the acoustic cues related to the sounding objects, their locations, and the physical characteristics of the space. Spatial audio production requires multichannel audio signals in order to convey this information and increase the realism of a real or virtual environment for applications such as the home entertainment, virtual reality, and remote collaboration. As demand to spatial audio continues to expand, ef...
Multi-modal stereo-vision using infrared / visible camera pairs
Yaman, Mustafa; Kalkan, Sinan; Department of Computer Engineering (2014)
In this thesis, a novel method for computing disparity maps from a multi-modal stereo-vision system composed of an infrared-visible camera pair is introduced. The method uses mutual information as the basic similarity measure where a segmentation based adaptive windowing mechanism is proposed along with a novel mutual information computation surface for greatly enhancing the results. Besides, the method incorporates joint prior probabilities when computing the cost matrix in addition to negative mutual info...
Citation Formats
IEEE
ACM
APA
CHICAGO
MLA
BibTeX
C. Acartürk and S. Emil, “Multimodal communication in instructional settings: An investigation of the functional roles of gestures and arrows,”
Revista Signos
, vol. 54, no. 107, pp. 867–892, 2021, Accessed: 00, 2022. [Online]. Available: https://hdl.handle.net/11511/95212.