Show/Hide Menu
Hide/Show Apps
Logout
Türkçe
Türkçe
Search
Search
Login
Login
OpenMETU
OpenMETU
About
About
Open Science Policy
Open Science Policy
Communities & Collections
Communities & Collections
Help
Help
Frequently Asked Questions
Frequently Asked Questions
Guides
Guides
Thesis submission
Thesis submission
MS without thesis term project submission
MS without thesis term project submission
Publication submission with DOI
Publication submission with DOI
Publication submission
Publication submission
Supporting Information
Supporting Information
General Information
General Information
Copyright, Embargo and License
Copyright, Embargo and License
Contact us
Contact us
Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?
Date
2019-09-12
Author
Bilge, Yunus Can
İkizler Cinbiş, Nazlı
Cinbiş, Ramazan Gökberk
Metadata
Show full item record
Item Usage Stats
72
views
0
downloads
Cite This
We introduce the problem of zero-shot sign language recognition (ZSSLR), where the goal is to leverage models learned over the seen sign class examples to recognize the instances of unseen signs. To this end, we propose to utilize the readily available descriptions in sign language dictionaries as an intermediate-level semantic representation for knowledge transfer. We introduce a new benchmark dataset called ASL-Text that consists of 250 sign language classes and their accompanying textual descriptions. Compared to the ZSL datasets in other domains (such as object recognition), our dataset consists of limited number of training examples for a large number of classes, which imposes a significant challenge. We propose a framework that operates over the body and hand regions by means of 3D-CNNs, and models longer temporal relationships via bidirectional LSTMs. By leveraging the descriptive text embeddings along with these spatio-temporal representations within a zero-shot learning framework, we show that textual data can indeed be useful in uncovering sign languages. We anticipate that the introduced approach and the accompanying dataset will provide a basis for further exploration of this new zero-shot learning problem.
URI
https://bmvc2019.org/wp-content/uploads/papers/0122-paper.pdf
https://hdl.handle.net/11511/86256
Collections
Department of Computer Engineering, Conference / Seminar
Suggestions
OpenMETU
Core
Towards Zero-shot Sign Language Recognition
Bilge, Yunus Can; Cinbiş, Ramazan Gökberk; Ikizler-Cinbis, Nazli (2022-01-01)
This paper tackles the problem of zero-shot sign language recognition (ZSSLR), where the goal is to leverage models learned over the seen sign classes to recognize the instances of unseen sign classes. In this context, readily available textual sign descriptions and attributes collected from sign language dictionaries are utilized as semantic class representations for knowledge transfer. For this novel problem setup, we introduce three benchmark datasets with their accompanying textual and attribute descrip...
Real-time traffic sign detection and recognition on FPGA
Yalçın, Hüseyin; Bulut, Mehmet Mete; Akar, Gözde; Department of Electrical and Electronics Engineering (2013)
In this thesis, an embedded system for traffic sign detection and recognition is proposed. Proposed system is first designed in MATLAB and optimized. After optimization process, system design is transferred to FPGA and Virtex-V FX70 FPGA is selected for implementation platform. 640x480 sized image in RGB format is sent to FPGA system via computer interface. This image is segmented for red, blue, and yellow colors. Red and blue color maps are divided into 8x8 sub-blocks. Yellow color map is divided into 32x3...
Positive impact of state similarity on reinforcement learning performance
Girgin, Sertan; Polat, Faruk; Alhaj, Reda (Institute of Electrical and Electronics Engineers (IEEE), 2007-10-01)
In this paper, we propose a novel approach to identify states with similar subpolicies and show how they can be integrated into the reinforcement learning framework to improve learning performance. The method utilizes a specialized tree structure to identify common action sequences of states, which are derived from possible optimal policies, and defines a similarity function between two states based on the number of such sequences. Using this similarity function, updates on the action-value function of a st...
Optical character recognition for cursive handwriting
Arica, N; Yarman Vural, Fatoş Tunay (2002-06-01)
In this paper, a new analytic scheme, which uses a sequence of segmentation and recognition algorithms, is proposed for offline cursive handwriting recognition problem. First, some global parameters, such as slant angle, baselines, and stroke width and height are estimated. Second, a segmentation method finds character segmentation paths by combining gray scale and binary information. Third, Hidden Markov Model (HMM) is employed for shape recognition to label and rank the character candidates. For this purp...
Random Matrix Based Extended Target Tracking with Orientation: A New Model and Inference
Tuncer, Barkın; Özkan, Emre (2021-02-01)
In this study, we propose a novel extended target tracking algorithm which is capable of representing the extent of dynamic objects as an ellipsoid with a time-varying orientation angle. A diagonal positive semi-definite matrix is defined to model objects' extent within the random matrix framework where the diagonal elements have inverse-Gamma priors. The resulting measurement equation is non-linear in the state variables, and it is not possible to find a closed-form analytical expression for the true poste...
Citation Formats
IEEE
ACM
APA
CHICAGO
MLA
BibTeX
Y. C. Bilge, N. İkizler Cinbiş, and R. G. Cinbiş, “Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?,” 2019, Accessed: 00, 2021. [Online]. Available: https://bmvc2019.org/wp-content/uploads/papers/0122-paper.pdf.