Multi-camera Marker-based Real-time Head Pose Estimation System

Estimating human head pose is a common task in many applications such as human attention modeling, fitting model to video, and visual reality... In this work, we implement a complete fiducial marker based human head pose estimation system. Our system uses fiducial markers as the key points to do Perspective n-Point, solves the 2D to 3D correspondence problem, and 3D points to 3D points registration. We also apply the depth data from RGB-D cameras as extra information to find the associated rotation and translation of the object head in the camera coordinate. Besides that, to overcome occlusion situation and also increase the tracking accuracy, we surround the user by multiple cameras. Our system will then fuse all pose results and introduce the final output as the most reliable and stable pose of the current object's head. At the end, our system could quickly detect the movement of object's head, and achieve high output frame rate that usually required in the real-time controlling applications.