Visual servo control of a mobile manipulator using one-dimensional windows

A mobile manipulator system is developed to work as a guard robot in a modern building. Several prototypes of advanced task-oriented service robots have been reported. However, a breakthrough to mobile manipulation is still required on the fact that most mobile robots can only work on a single floor of the building. It is desirable that the robot can perform various tasks in every floor of a modern building. Consequently, the robot must be able to take conventional elevators. It is well recognized that techniques of visual servo are key factors for successful mobile manipulation. This paper presents a neural network for real-time clustering of objects in an image for tracking control. The system can successfully recognize each individual button on the elevator panel. Then a novel window-based visual tracking control scheme is proposed for guiding the mobile manipulator to push the correct panel button. The new method uses two one-dimensional windows for visual servo control. Experimental results reveal that this method performs much better than the standard techniques.