A Framework for Fusing Video and Wearable Sensing Data by Deep Learning

Ting Hui Chiang, Po Yi Kuo, Huan Ruei Shiu, Yu Chee Tseng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Both cameras and IoT devices have their particular capabilities in tracking human behaviors and statuses. Their correlations are, however, unclear. In this work, we propose a framework for integrating video and wearable sensing data for smart surveillance, such as person identification and tracking. Using biometric features such as fingerprint, iris, gait, and face may lead to good recognition results. However, these approaches all have their limitations in distance and privacy concerns. In this work, we present a data fusion framework based on deep learning for fusing the aforementioned data. Here, using deep learning is to help adaptively learn the hidden bindings of these data. We demonstrate how to retrieve data of interest from IoT devices, which are attached on human objects, and correctly tag them on the human objects captured by a camera, thus correlating video and IoT data. Potential applications of this framework include smart surveillance and friendly visualization. We then show a case study, including integrating video data with body movement and physiological data.

Original languageEnglish
Title of host publicationProceedings of the 2022 8th International Conference on Applied System Innovation, ICASI 2022
EditorsShoou-Jinn Chang, Sheng-Joue Young, Artde Donald Kin-Tak Lam, Liang-Wen Ji, Stephen D. Prior
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages134-139
Number of pages6
ISBN (Electronic)9781665496506
DOIs
StatePublished - 2022
Event8th International Conference on Applied System Innovation, ICASI 2022 - Nantou, Taiwan
Duration: 21 Apr 202223 Apr 2022

Publication series

NameProceedings of the 2022 8th International Conference on Applied System Innovation, ICASI 2022

Conference

Conference8th International Conference on Applied System Innovation, ICASI 2022
Country/TerritoryTaiwan
CityNantou
Period21/04/2223/04/22

Keywords

  • computer vision
  • data fusion
  • deep learning
  • IoT
  • surveillance
  • tracking

Fingerprint

Dive into the research topics of 'A Framework for Fusing Video and Wearable Sensing Data by Deep Learning'. Together they form a unique fingerprint.

Cite this