The learning of adjectives and nouns from affordance and appearance features

Yürüten, Onur
Şahin, Erol
Kalkan, Sinan
We study how a robot can link concepts represented by adjectives and nouns in language with its own sensorimotor interactions. Specifically, an iCub humanoid robot interacts with a group of objects using a repertoire of manipulation behaviors. The objects are labeled using a set of adjectives and nouns. The effects induced on the objects are labeled as affordances, and classifiers are learned to predict the affordances from the appearance of an object. We evaluate three different models for learning adjectives and nouns using features obtained from the appearance and affordances of an object, through cross-validated training as well as through testing on novel objects. The results indicate that shape-related adjectives are best learned using features related to affordances, whereas nouns are best learned using appearance features. Analysis of the feature relevancy shows that affordance features are more relevant for adjectives, and appearance features are more relevant for nouns. We show that adjective predictions can be used to solve the odd-one-out task on a number of examples. Finally, we link our results with studies from psychology, neuroscience and linguistics that point to the differences between the development and representation of adjectives and nouns in humans.
Adaptive Behavior


Learning adjectives and nouns from affordances on the iCub humanoid robot
Yürüten, Onur; Uyanik, Kadir Firat; Çalişkan, Yiǧit; Bozcuoǧlu, Asil Kaan; Şahin, Erol; Kalkan, Sinan (2012-09-14)
This article studies how a robot can learn nouns and adjectives in language. Towards this end, we extended a framework that enabled robots to learn affordances from its sensorimotor interactions, to learn nouns and adjectives using labeling from humans. Specifically, an iCub humanoid robot interacted with a set of objects (each labeled with a set of adjectives and a noun) and learned to predict the effects (as labeled with a set of verbs) it can generate on them with its behaviors. Different from appearance...
Formation of adjective, noun and verb concepts through affordances
Yürüten, Onur; Kalkan, Sinan; Şahin, Erol; Department of Computer Engineering (2012)
In this thesis, we study the development of linguistic concepts (corresponding to a subset of nouns, verbs and adjectives) on a humanoid robot. To accomplish this goal, we use affordances, a notion first proposed by J.J. Gibson to describe the action possibilities offered to an agent by the environment. Using the affordances formalization framework of Sahin et al., we have implemented a learning system on a humanoid robot and obtained the required data from the sensorimotor experiences of the robot. The sys...
Unsupervised Learning of Affordance Relations on a Humanoid Robot
Akgun, Baris; Dag, Nilguen; Bilal, Tahir; Atil, Ilkay; Şahin, Erol (2009-09-16)
In this paper, we study how the concepts learned by a robot can be linked to verbal concepts that humans use in language. Specifically, we develop a simple tapping behaviour on the iCub humanoid robot simulator and allow the robot to interact with a set of objects of different types and sizes to learn affordance relations in its environment. The robot records its perception, obtained from a range camera, as a feature vector, before and after applying tapping on an object. We compute effect features by subtr...
The Second language processing of nominal compounds: a masked priming study
Çelikkol Berk, Nurten; Kırkıcı, Bilal; Department of English Language Teaching (2018)
The primary purpose of the present study was to understand the workings of the cognitive mechanisms underlying L2 morphological processing, and more particularly, to explore how noun-noun compounds in L2 English are processed by native speakers of Turkish in the earliest stages of word recognition. Furthermore, the study investigated the role of constituent morphemes in the processing of compound words and examined whether or not a compound word primes its first and second constituents equally. The final pu...
Integrating Spatial Concepts into a Probabilistic Concept Web
Celikkanat, Hande; Şahin, Erol; Kalkan, Sinan (2015-07-31)
In this paper, we study the learning and representation of grounded spatial concepts in a probabilistic concept web that connects them with other noun, adjective, and verb concepts. Specifically, we focus on the prepositional spatial concepts, such as "on", "below", "left", "right", "in front of" and "behind". In our prior work (Celikkanat et al., 2015), inspired from the distributed highly-connected conceptual representation in human brain, we proposed using Markov Random Field for modeling a concept web o...
Citation Formats
O. Yürüten, E. Şahin, and S. Kalkan, “The learning of adjectives and nouns from affordance and appearance features,” Adaptive Behavior, pp. 437–451, 2013, Accessed: 00, 2020. [Online]. Available: