Defining Image Memorability Using the Visual Memory Schema

Download
2020-09-01
Akagündüz, Erdem
Evans, Karla K.
Memorability of an image is a characteristic determined by the human observers' ability to remember images they have seen. Yet recent work on image memorability defines it as an intrinsic property that can be obtained independent of the observer. The current study aims to enhance our understanding and prediction of image memorability, improving upon existing approaches by incorporating the properties of cumulative human annotations. We propose a new concept called the Visual Memory Schema (VMS) referring to an organization of image components human observers share when encoding and recognizing images. The concept of VMS is operationalised by asking human observers to define memorable regions of images they were asked to remember during an episodic memory test. We then statistically assess the consistency of VMSs across observers for either correctly or incorrectly recognised images. The associations of the VMSs with eye fixations and saliency are analysed separately as well. Lastly, we adapt various deep learning architectures for the reconstruction and prediction of memorable regions in images and analyse the results when using transfer learning at the outputs of different convolutional network layers.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE

Suggestions

Representing images and regions for object recognition
Buzcu, İlker; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2015)
We can represent images in entirely different ways, in order to fulfill different purposes. For object recognition, power of a representation comes from its discriminative ability. In this thesis work, handcrafted representations that dominated the last decade of computer vision are evaluated against the current paradigm of Deep Learning, to try and pinpoint the reasons behind why and how the fairly old Artificial Neural Network (ANN) framework suddenly emerged as the state of the art in discriminative repr...
Tracing microtubules in live cell images
Sargin, Mehmet Emre; Altinok, Alphan; Kiriş, Erkan; Feinstein, Stuart C; Wilson, Leslie; Rose, Kenneth; Manjunath, B S (2007-06-15)
Microtubule (MT) dynamics are traditionally analyzed from time lapse images by manual techniques that are laborious, approximate and often limited. Recently, computer vision techniques have been applied to the problem of automated tracking of MTs in live cell images. Aside of very low signal to noise ratios, live cell images of MTs exhibit severe clutter for accurate tracing of MT body. Moreover, intersecting and overlapping MT regions appear brighter due to additive fluorescence. In this paper, we present ...
A NOVEL SHADOW RESTORATION ALGORITHM BASED ON ATMOSPHERIC EFFECTS FOR AERIAL IMAGES
Aytekin, Caglar; Alatan, Abdullah Aydın (2010-09-29)
In aerial images, the performance of the segmentation and object recognition algorithms could suffer due to shadows in the scene. This effort describes a novel shadow restoration algorithm based on atmospheric effects and characteristics of sun light for aerial images. Firstly, shadow regions are detected exploiting the Rayleigh scattering phenomena and the well-known fact related to the low illumination intensity in the shadow regions. After detection, shadow restoration is achieved by first restoring part...
Modeling Voxel Connectivity for Brain Decoding
Onal, Itir; Ozay, Mete; Yarman Vural, Fatoş Tunay (2015-06-12)
The massively dynamic nature of human brain cannot be represented by considering only a collection of voxel intensity values obtained from fMRI measurements. It has been observed that the degree of connectivity among voxels provide important information for modeling cognitive activities. Moreover, spatially close voxels act together to generate similar BOLD responses to the same stimuli. In this study, we propose a local mesh model, called Local Mesh Model with Temporal Measurements (LMM-TM), to first estim...
Automatic target recognition and detection in infrared imagery under cluttered background
GÜNDOĞDU, ERHAN; KOÇ, AYKUT; Alatan, Abdullah Aydın (2017-09-14)
Visual object classification has long been studied in visible spectrum by utilizing conventional cameras. Since the labeled images has recently increased in number, it is possible to train deep Convolutional Neural Networks (CNN) with significant amount of parameters. As the infrared (IR) sensor technology has been improved during the last two decades, labeled images extracted from IR sensors have been started to be used for object detection and recognition tasks. We address the problem of infrared object ...
Citation Formats
E. Akagündüz and K. K. Evans, “Defining Image Memorability Using the Visual Memory Schema,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 42, no. 9, pp. 2165–2178, 2020, Accessed: 00, 2021. [Online]. Available: https://hdl.handle.net/11511/93622.