Cohesion in multimodal documents Effects of cross referencing

Date

2013-08-01

Author

Acartürk, Cengiz
CHRISTOPHER, HABEL

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

126
views

0
downloads

In multimodal documents, different types of cohesive or cross-reference link (i.e., signaling) are used in the text to link verbally coded content with the graphical material. In this study, we identify three types of reference, within the framework of previous work on cohesion (Halliday & Hasan, 1976): directive signaling, descriptive signaling, and no signaling in the text to introduce the figure. In an experimental study, we use eye tracking to investigate how those three reference types influence the processing of the material by humans. The results reveal differences between the reference types both in terms of eye movement parameters and retention of the material

Subject Keywords

Multimodal documents, Cross-reference, Text-figure relations, Cohesion

URI

https://hdl.handle.net/11511/31938

Journal

Information Design Journal

DOI

https://doi.org/10.1075/idj.20.2.01aca

Collections

Graduate School of Informatics, Article

Suggestions

OpenMETU
Core

On Fuzzy Extensions to Energy Ontologies for Text Processing Applications Kucuk, Dilek; Kucuk, Dogan; Yazıcı, Adnan (2014-10-28) Ubiquitous application areas of domain ontologies include text processing applications like categorizing related documents of the domain, extraction of information from these documents, and semantic search. In this paper, we focus on the utilization of two energy ontologies, one for electrical power quality and the second for wind energy, within such applications. For this purpose, we present fuzzy extensions to these domain ontologies as fuzziness is an essential feature of the ultimate forms of the ontolo...
Bimodal automatic speech segmentation based on audio and visual information fusion Akdemir, Eren; Çiloğlu, Tolga (2011-07-01) Bimodal automatic speech segmentation using visual information together with audio data is introduced. The accuracy of automatic segmentation directly affects the quality of speech processing systems using the segmented database. The collaboration of audio and visual data results in lower average absolute boundary error between the manual segmentation and automatic segmentation results. The information from two modalities are fused at the feature level and used in a HMM based speech segmentation system. A T...
Performance-based parametric design explorations: A method for generating appropriate building components Ercan, Burak; Elias Özkan, Soofia Tahira (2015-05-01) Performance-based parametric design explorations depend on formulating custom-designed workflows that require reading, writing, interpreting and manipulating databases, as part of the design process. The possibilities of customization and parameterization offered by the user-friendly interfaces of advanced building-performance simulation software and digital design tools have now enabled architects to carry out performance-based design explorations without the help of simulation experts. This paper presents...
Investigation of haptic line graph comprehension through co production of gesture and language Deniz, Ozan; Mehmetcan, Fal; Acartürk, Cengiz (null; 2013-06-30) In communication settings, statistical graphs accompany language by providing visual access to various aspects of domain entities, such as conveying information about trends. A similar and comparable means for providing perceptual access is to provide haptic graphs for blind people. In this study, we present the results of an experimental study that aimed to investigate visual line graphs and haptic line graphs in time domain by means of gesture production as an indicator of event conceptualization. The par...
On reciprocal improvement in multimodal generation Co-reference by text and information graphics Chrıstopher, Habel; Acartürk, Cengiz (2007-01-31) In this paper, we argue that in the production of complex documents combining text and information graphics, improvement—seen as a specific revision process—has to be considered as an independent module in a multi-pass architecture. Inside improvement, graph comprehension and text comprehension modules interact in building up a common content representation based on a conceptual inventory specified via topological and geometrical concepts. During concurrent comprehension the reciprocal improvement module in...

Citation Formats

C. Acartürk and H. CHRISTOPHER, “Cohesion in multimodal documents Effects of cross referencing,” Information Design Journal, pp. 98–110, 2013, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/31938.