Automated shopping system using computer vision

Date

2020-11-01

Author

Odeh, Nemer
Direkoglu, Cem

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

270
views

0
downloads

The shopping experiment made by amazon go in USA is one of the most interesting applications of computer vision recently. They allow you to shop and automatically charge your virtual card for whatever goods you purchased using cameras and wireless systems, so no checkouts or waiting lines are required. However, amazon didn't reveal yet the details of how their system components are implemented. In this paper, we introduce a complete system for computer vision based automated shopping. The proposed system contains barcode scanning of objects, data registration, image capturing for offline training stage, motion (change) detection, CNN and SVM for object classification and charging/discharging customers. Our system can be integrated with the wireless data transmission to do the whole shopping process. First, the proposed method extracts the objects' barcodes to register their details, and take sample images of objects for classifier training. We employ a pre-trained CNN (i.e. ResNet50) for feature extraction and a multi-class SVM for training. After training our classifier, we have a real-time operation stage (i.e. test stage). We assume that a camera is embedded above products on each shelf to capture videos of the products. We employ a change detector to understand any added or removed items. If the item is removed from or added to the shelve, the moving object is input to CNN feature extractor, and then SVM classifier for identification and pricing. Results show that the proposed system is fast and effective.

Subject Keywords

Media Technology, Computer Networks and Communications, Hardware and Architecture, Software

URI

https://hdl.handle.net/11511/65095

Journal

MULTIMEDIA TOOLS AND APPLICATIONS

DOI

https://doi.org/10.1007/s11042-020-09481-6

Collections

Engineering, Article

Suggestions

OpenMETU
Core

SWARM-based data delivery in Social Internet of Things Hasan, Mohammed Zaki; Al-Turjman, Fadi (Elsevier BV, 2019-03-01) Social Internet of Things (SIoTs) refers to the rapidly growing network of connected objects and people that are able to collect and exchange data using embedded sensors. To guarantee the connectivity among these objects and people, fault tolerance routing has to be significantly considered. In this paper, we propose a bio-inspired particle multi-swarm optimization (PMSO) routing algorithm to construct, recover and select k-disjoint paths that tolerates the failure while satisfying quality of service (QoS) ...
Graph-based multilevel temporal video segmentation Sakarya, Ufuk; TELATAR, ZİYA (Springer Science and Business Media LLC, 2008-11-01) This paper presents a graph-based multilevel temporal video segmentation method. In each level of the segmentation, a weighted undirected graph structure is implemented. The graph is partitioned into clusters which represent the segments of a video. Three low-level features are used in the calculation of temporal segments' similarities: visual content, motion content and shot duration. Our strength factor approach contributes to the results by improving the efficiency of the proposed method. Experiments sho...
Path planning for mobile-anchor based wireless sensor network localization: Static and dynamic schemes Erdemir, Ecenaz; Tuncer, Temel Engin (Elsevier BV, 2018-08-01) In wireless sensor networks, node locations are required for many applications. Usually, anchors with known positions are employed for localization. Sensor positions can be estimated more efficiently by using mobile anchors (MAs). Finding the best MA trajectory is an important problem in this context. Various path planning algorithms are proposed to localize as many sensors as possible by following the shortest path with minimum number of anchors. In this paper, path planning algorithms for MA assisted loca...
Cognitive-Node Architecture and a Deployment Strategy for the Future WSNs Al-Turjman, Fadi (Springer Science and Business Media LLC, 2019-10-01) The advent of sensing and communication technologies represents the next step in the evolution of wireless sensor networks (WSNs) and future applications. Future WSNs systems demand that connected devices could be able to work autonomously, while surfing on-line generated data and process them for self-decision making. Accordingly, we propose a cognitive Information-Centric Sensor Network (ICSN) framework. The fundamentals of cognition in ICSN can be recognized as a promising direction in addressing opportu...
Multimodal concept detection in broadcast media: KavTan SOYSAL, Medeni; Alatan, Abdullah Aydın; TEKİN, Mashar; ESEN, Ersin; SARACOĞLU, Ahmet; Acar, Banu Oskay; Ozan, Ezgi Can; Ates, Tugrul K.; SEVİMLİ, Hakan; SEVİNÇ, Muge; ATIL, Ilkay; Ozkan, Savas; Arabaci, Mehmet Ali; TANKIZ, Seda; KARADENİZ, Talha; ÖNÜR, Duygu; SELÇUK, Sezin; Alatan, A. Aydin; Çiloğlu, Tolga (Springer Science and Business Media LLC, 2014-10-01) Concept detection stands as an important problem for efficient indexing and retrieval in large video archives. In this work, the KavTan System, which performs high-level semantic classification in one of the largest TV archives of Turkey, is presented. In this system, concept detection is performed using generalized visual and audio concept detection modules that are supported by video text detection, audio keyword spotting and specialized audio-visual semantic detection components. The performance of the p...

Citation Formats

N. Odeh and C. Direkoglu, “Automated shopping system using computer vision,” MULTIMEDIA TOOLS AND APPLICATIONS, pp. 30151–30161, 2020, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/65095.