Smart self-checkout carts based on deep learning for shopping activity recognition

Hong Chuan Chi, Muhammad Atif Sarwar, Yousef Awwad Daraghmi, Kuan Wen Lin, Tsi-Ui Ik*, Yih-Lang Li

*此作品的通信作者

研究成果: Conference contribution同行評審

7 引文 斯高帕斯(Scopus)

摘要

Fast and reliable communication plays a major role in the success of smart shopping applications. In a 'Just Walk Out' shopping scenario, a video camera is installed on the cart to monitor shopping activities and transmit images to the cloud for processing so that items in the cart can be tracked and checked out. This paper proposes a prototype of a smart shopping cart based on image-based action recognition. Firstly, deep learning networks such as Faster R-CNN, YOLOv2, and YOLOv2-Tiny are utilized to analyze the content of each video frame. Frames are classified into three classes: No Hand, Empty Hand, and Holding Items. The classification accuracy based on Faster R-CNN, YOLOv2, or YOLOv2-Tiny is between 93.0% and 90.3%, and the processing speed of the three networks can be up to 5 fps, 39 fps, and 50 fps, respectively. Secondly, based on the sequence of frame classes, the timeline is divided into No Hand intervals, Empty Hand intervals, and Holding Items intervals. The accuracy of action recognition is 96%, and the time error is 0.119s on average. Finally, we categorize the events into four cases: No Change, placing, Removing, and Swapping. Even including the correctness of the item recognition, the accuracy of shopping event detection is 97.9%, which is higher than the minimal requirement to deploy such a system in a smart shopping environment. A demo of the system and a link to download the data set used in the paper are in Smart Shopping Cart Prototype or found at this URL: https://hackmd.io/abEiC83rQoqxz7zpL4Kh2w.

原文English
主出版物標題APNOMS 2020 - 2020 21st Asia-Pacific Network Operations and Management Symposium
主出版物子標題Towards Service and Networking Intelligence for Humanity
發行者Institute of Electrical and Electronics Engineers Inc.
頁面185-190
頁數6
ISBN(電子)9788995004388
DOIs
出版狀態Published - 22 9月 2020
事件21st Asia-Pacific Network Operations and Management Symposium, APNOMS 2020 - Daegu, Korea, Republic of
持續時間: 22 9月 202025 9月 2020

出版系列

名字APNOMS 2020 - 2020 21st Asia-Pacific Network Operations and Management Symposium: Towards Service and Networking Intelligence for Humanity

Conference

Conference21st Asia-Pacific Network Operations and Management Symposium, APNOMS 2020
國家/地區Korea, Republic of
城市Daegu
期間22/09/2025/09/20

指紋

深入研究「Smart self-checkout carts based on deep learning for shopping activity recognition」主題。共同形成了獨特的指紋。

引用此