Semi-Automated Quadrotor Teleoperation by Hierarchical Gesture Control and Visual Perception

Jyi Shane Liu, Ting Yi Chiu, Chen Tao Hsu, Jen Hui Chuang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This study proposes a human-robot collaboration framework for semiautomatic control of quadrotor unmanned aerial vehicles (UAVs) for beyond visual line of sight applications. The framework integrates partially teleoperated flight control with partially autonomous task execution by developing a hierarchical control architecture by multiple levels of instruction. The flight control teleoperation is performed by an operator's hand gesture to a mounted camera on a monitoring device. We extract the abduction and adduction of fingers and hand pointing direction as features for gesture recognition using Mediapipe. Equipped with visual processing capability, the quadrotor can also automate a subtask execution process when a high-level instruction gesture is given. We provide a customizable target perception architecture based on ORB-SLAM2 and Detectron2 to estimate the volume and coordinates of 3D objects through instance segmentation. The combination of gesture control and autonomous visual perception provides advantage of a loosely engaged operation process over the tightly-coupled joystick control. Based on the experimental results, we show that our proposed approach improves the task effectiveness of quadrotor UAV space exploration and object inspection in an indoor environment.

Original languageEnglish
Title of host publication2022 7th International Conference on Robotics and Automation Engineering, ICRAE 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-6
Number of pages6
ISBN (Electronic)9781665489188
DOIs
StatePublished - 2022
Event7th International Conference on Robotics and Automation Engineering, ICRAE 2022 - Singapore, Singapore
Duration: 18 Nov 202220 Nov 2022

Publication series

Name2022 7th International Conference on Robotics and Automation Engineering, ICRAE 2022

Conference

Conference7th International Conference on Robotics and Automation Engineering, ICRAE 2022
Country/TerritorySingapore
CitySingapore
Period18/11/2220/11/22

Keywords

  • hierarchical gesture control
  • UAV teleoperation control
  • visual perception

Fingerprint

Dive into the research topics of 'Semi-Automated Quadrotor Teleoperation by Hierarchical Gesture Control and Visual Perception'. Together they form a unique fingerprint.

Cite this