Automated shopping system using computer vision

Date

2020-11-01

Author

Odeh, Nemer
Direkoglu, Cem

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

311
views

0
downloads

The shopping experiment made by amazon go in USA is one of the most interesting applications of computer vision recently. They allow you to shop and automatically charge your virtual card for whatever goods you purchased using cameras and wireless systems, so no checkouts or waiting lines are required. However, amazon didn't reveal yet the details of how their system components are implemented. In this paper, we introduce a complete system for computer vision based automated shopping. The proposed system contains barcode scanning of objects, data registration, image capturing for offline training stage, motion (change) detection, CNN and SVM for object classification and charging/discharging customers. Our system can be integrated with the wireless data transmission to do the whole shopping process. First, the proposed method extracts the objects' barcodes to register their details, and take sample images of objects for classifier training. We employ a pre-trained CNN (i.e. ResNet50) for feature extraction and a multi-class SVM for training. After training our classifier, we have a real-time operation stage (i.e. test stage). We assume that a camera is embedded above products on each shelf to capture videos of the products. We employ a change detector to understand any added or removed items. If the item is removed from or added to the shelve, the moving object is input to CNN feature extractor, and then SVM classifier for identification and pricing. Results show that the proposed system is fast and effective.

Subject Keywords

Media Technology, Computer Networks and Communications, Hardware and Architecture, Software

URI

https://hdl.handle.net/11511/65095

Journal

MULTIMEDIA TOOLS AND APPLICATIONS

DOI

https://doi.org/10.1007/s11042-020-09481-6

Collections

Engineering, Article

Suggestions

OpenMETU
Core

SWARM-based data delivery in Social Internet of Things Hasan, Mohammed Zaki; Al-Turjman, Fadi (Elsevier BV, 2019-03-01) Social Internet of Things (SIoTs) refers to the rapidly growing network of connected objects and people that are able to collect and exchange data using embedded sensors. To guarantee the connectivity among these objects and people, fault tolerance routing has to be significantly considered. In this paper, we propose a bio-inspired particle multi-swarm optimization (PMSO) routing algorithm to construct, recover and select k-disjoint paths that tolerates the failure while satisfying quality of service (QoS) ...
Cognitive-Node Architecture and a Deployment Strategy for the Future WSNs Al-Turjman, Fadi (Springer Science and Business Media LLC, 2019-10-01) The advent of sensing and communication technologies represents the next step in the evolution of wireless sensor networks (WSNs) and future applications. Future WSNs systems demand that connected devices could be able to work autonomously, while surfing on-line generated data and process them for self-decision making. Accordingly, we propose a cognitive Information-Centric Sensor Network (ICSN) framework. The fundamentals of cognition in ICSN can be recognized as a promising direction in addressing opportu...
Multimodal concept detection in broadcast media: KavTan SOYSAL, Medeni; Alatan, Abdullah Aydın; TEKİN, Mashar; ESEN, Ersin; SARACOĞLU, Ahmet; Acar, Banu Oskay; Ozan, Ezgi Can; Ates, Tugrul K.; SEVİMLİ, Hakan; SEVİNÇ, Muge; ATIL, Ilkay; Ozkan, Savas; Arabaci, Mehmet Ali; TANKIZ, Seda; KARADENİZ, Talha; ÖNÜR, Duygu; SELÇUK, Sezin; Alatan, A. Aydin; Çiloğlu, Tolga (Springer Science and Business Media LLC, 2014-10-01) Concept detection stands as an important problem for efficient indexing and retrieval in large video archives. In this work, the KavTan System, which performs high-level semantic classification in one of the largest TV archives of Turkey, is presented. In this system, concept detection is performed using generalized visual and audio concept detection modules that are supported by video text detection, audio keyword spotting and specialized audio-visual semantic detection components. The performance of the p...
Feature-based software asset modeling with domain specific kits Altıntaş, Nesip İlker; Doğru, Ali Hikmet; Department of Computer Engineering (2007) This study proposes an industrialization model, Software Factory Automation, for establishing software product lines. Major contributions of this thesis are the conceptualization of Domain Specific Kits (DSKs) and a domain design model for software product lines based on DSKs. The concept of DSK has been inspired by the way other industries have been successfully realizing factory automation for decades. DSKs, as fundamental building blocks, have been deeply elaborated with their characteristic properties a...
Surveillance with wireless sensor networks in obstruction: Breach paths as watershed contours Onur, Ertan; Delic, Hakan; Akarun, Lale (Elsevier BV, 2010-02-01) For surveillance applications of wireless sensor networks, analysis of sensing coverage and quality of sensing is crucial. For rough terrains where obstacles block the sensing capability, region-based approaches must be employed to determine the sensing quality. In this paper, we present a method to determine the breach paths and the deployment quality defined as the minimum of the maximum detection probabilities on the breach paths in the presence of obstacles. We propose the utilization of watershed segme...

Citation Formats

N. Odeh and C. Direkoglu, “Automated shopping system using computer vision,” MULTIMEDIA TOOLS AND APPLICATIONS, pp. 30151–30161, 2020, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/65095.