Defining Image Memorability Using the Visual Memory Schema

Download

index.pdf

Date

2020-09-01

Author

Akagündüz, Erdem
Evans, Karla K.

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

84
views

29
downloads

Memorability of an image is a characteristic determined by the human observers' ability to remember images they have seen. Yet recent work on image memorability defines it as an intrinsic property that can be obtained independent of the observer. The current study aims to enhance our understanding and prediction of image memorability, improving upon existing approaches by incorporating the properties of cumulative human annotations. We propose a new concept called the Visual Memory Schema (VMS) referring to an organization of image components human observers share when encoding and recognizing images. The concept of VMS is operationalised by asking human observers to define memorable regions of images they were asked to remember during an episodic memory test. We then statistically assess the consistency of VMSs across observers for either correctly or incorrectly recognised images. The associations of the VMSs with eye fixations and saliency are analysed separately as well. Lastly, we adapt various deep learning architectures for the reconstruction and prediction of memorable regions in images and analyse the results when using transfer learning at the outputs of different convolutional network layers.

URI

https://hdl.handle.net/11511/93622

Journal

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE

DOI

https://doi.org/10.1109/tpami.2019.2914392

Collections

Graduate School of Informatics, Article

Suggestions

OpenMETU
Core

Representing images and regions for object recognition Buzcu, İlker; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2015) We can represent images in entirely different ways, in order to fulfill different purposes. For object recognition, power of a representation comes from its discriminative ability. In this thesis work, handcrafted representations that dominated the last decade of computer vision are evaluated against the current paradigm of Deep Learning, to try and pinpoint the reasons behind why and how the fairly old Artificial Neural Network (ANN) framework suddenly emerged as the state of the art in discriminative repr...
Tracing microtubules in live cell images Sargin, Mehmet Emre; Altinok, Alphan; Kiriş, Erkan; Feinstein, Stuart C; Wilson, Leslie; Rose, Kenneth; Manjunath, B S (2007-06-15) Microtubule (MT) dynamics are traditionally analyzed from time lapse images by manual techniques that are laborious, approximate and often limited. Recently, computer vision techniques have been applied to the problem of automated tracking of MTs in live cell images. Aside of very low signal to noise ratios, live cell images of MTs exhibit severe clutter for accurate tracing of MT body. Moreover, intersecting and overlapping MT regions appear brighter due to additive fluorescence. In this paper, we present ...
VISIBLE AND INFRARED IMAGE FUSION USING ENCODER-DECODER NETWORK Ataman, Ferhat Can; Akar, Gözde (2021-01-01) © 2021 IEEE.The aim of multispectral image fusion is to combine object or scene features of images with different spectral characteristics to increase the perceptual quality. In this paper, we present a novel learning-based solution to image fusion problem focusing on infrared and visible spectrum images. The proposed solution utilizes only convolution and pooling layers together with a loss function using no-reference quality metrics. The analysis is performed qualitatively and quantitatively on various da...
A NOVEL SHADOW RESTORATION ALGORITHM BASED ON ATMOSPHERIC EFFECTS FOR AERIAL IMAGES Aytekin, Caglar; Alatan, Abdullah Aydın (2010-09-29) In aerial images, the performance of the segmentation and object recognition algorithms could suffer due to shadows in the scene. This effort describes a novel shadow restoration algorithm based on atmospheric effects and characteristics of sun light for aerial images. Firstly, shadow regions are detected exploiting the Rayleigh scattering phenomena and the well-known fact related to the low illumination intensity in the shadow regions. After detection, shadow restoration is achieved by first restoring part...
Modeling Voxel Connectivity for Brain Decoding Onal, Itir; Ozay, Mete; Yarman Vural, Fatoş Tunay (2015-06-12) The massively dynamic nature of human brain cannot be represented by considering only a collection of voxel intensity values obtained from fMRI measurements. It has been observed that the degree of connectivity among voxels provide important information for modeling cognitive activities. Moreover, spatially close voxels act together to generate similar BOLD responses to the same stimuli. In this study, we propose a local mesh model, called Local Mesh Model with Temporal Measurements (LMM-TM), to first estim...

Citation Formats

E. Akagündüz and K. K. Evans, “Defining Image Memorability Using the Visual Memory Schema,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 42, no. 9, pp. 2165–2178, 2020, Accessed: 00, 2021. [Online]. Available: https://hdl.handle.net/11511/93622.