Multi-step Pick-and-Place Tasks Using Object-centric Dense Correspondences

This paper presents an object-centric method for efficiently performing two types of challenging pick-and-place tasks, namely sequential pick and place and object sorting. We propose multiclass dense object nets (MCDONs) for learning object-centric dense descriptors that maintain not only intra-class variations but also inter-class separation. Intra-class consistency is also inherently learned and is useful for our pick-and-place tasks. All the tasks only require a single demonstration from users, which can then be generalized to all class instances. A dataset containing eight classes and a total of 52 objects was provided in this study. We obtained a task success rate of 93.33% on a five-block stacking task and 97.41% on a three-class object sorting task.