On-line Path Planning and Visual Tracking of Fast-Moving Objects by Robot Manipulators with High-Speed Camera Arrays

In this paper, we investigate the problem of tracking a fast-moving object by a robot manipulator. The tracking task considered in this paper includes visually locating and predicting the trajectory of an object, approaching and physically contacting the object with the robot arm. Since the object moves with high and varying speeds, tight integration of a high-speed vision system and the robot motion controller is critical for accomplishing this tracking task. We combine multiple cameras and embedded computers to construct a camera array with low latency and an equivalent frame rate as high as the sampling rate of the motion control law. Furthermore, the camera array is triggered and synchronized by the motion controller. The path of the robot is re-planned on-line at every sampling time whenever the camera array updates the latest position of the object. Experiments were conducted and the results showed that the proposed robotic tracking system can accurately predict the trajectory of a fast-moving object, and successfully contact it with the tip of the robot arm.