Pose-Aware Placement of Objects with Semantic Labels - Brandname-based Affordance Prediction and Cooperative Dual-Arm Active Manipulation

Yung Shan Su, Lap Fai Yu, Hsueh-Cheng Wang, Shao Huang Lu, Po Sheng Ser, Wei Ting Hsu, Wei Cheng Lai, Biao Xie, Hong Ming Huang, Teng Yok Lee, Hung Wen Chen

研究成果: Conference contribution同行評審

摘要

The Amazon Picking Challenge and the Amazon Robotics Challenge have shown significant progress in object picking from a cluttered scene, yet object placement remains challenging. It is useful to have pose-aware placement based on human and machine readable pieces on an object. For example, the brandname of an object placed on a shelf should be facing the human customers. The robotic vision challenges in the object placement task: A) the semantics and geometry of the object to be placed need to be analysed jointly; b) and the occlusions among objects in a cluttered scene could make it hard for proper understanding and manipulation. To overcome these challenges, we develop a pose-aware placement approach by spotting the semantic labels (e.g., brandnames) of objects in a cluttered tote and then carrying out a sequence of actions to place the objects on a shelf or on a conveyor with desired poses. Our major contributions include 1) providing an open benchmark dataset of objects and brandnames with multi-view segmentation for training and evaluations; 2) carrying out comprehensive evaluations for our brandname-based fully convolutional network (FCN) that can predict the affordance and grasp to achieve pose-aware placement, whose success rates decrease along with clutters; 3) showing that active manipulation with two cooperative manipulators and grippers can effectively handle the occlusion of brandnames. We analyzed the success rates and discussed the failure cases to provide insights for future applications. All data and benchmarks are available at https://text-pick-n-place.github.io/TextPNP/

原文English
主出版物標題2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
發行者Institute of Electrical and Electronics Engineers Inc.
頁面4760-4767
頁數8
ISBN(電子)9781728140049
DOIs
出版狀態Published - 11月 2019
事件2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019 - Macau, China
持續時間: 3 11月 20198 11月 2019

出版系列

名字IEEE International Conference on Intelligent Robots and Systems
ISSN(列印)2153-0858
ISSN(電子)2153-0866

Conference

Conference2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
國家/地區China
城市Macau
期間3/11/198/11/19

指紋

深入研究「Pose-Aware Placement of Objects with Semantic Labels - Brandname-based Affordance Prediction and Cooperative Dual-Arm Active Manipulation」主題。共同形成了獨特的指紋。

引用此