3D printing on freeform surface: Real-time and accurate 3D dynamic dense surface reconstruction with HoloLens and displacement measurement sensors

This paper attempts to print irregular 3D objects on a freeform surface. Notably, it is inevitable to cause errors when creating a specific object shape in simulated and real-life scenarios. This is mainly attributed to the unique characteristics of the materials adopted, which tend to develop different deformations properties. Nevertheless, since the simulated model provides a clue regarding the coordinates of the deformed shape, it is adopted as an essential indicator in estimating accurate real-world coordinates. Concretely, this paper presents a complete system that is capable of reconstructing the detailed surface coordinates in real-time conditions. To verify the effectiveness of the proposed method, a metal board is used as the primary material to create different curvatures physically and virtually. In particular, the real-time performance of the overall 3D surface reconstruction has been experimentally evaluated using several tools, such as KUKA KR90 R3100 robots, HoloLens 2, displacement sensor, and ArUco markers. Consequently, both the quantitative and qualitative results are presented to demonstrate the feasibility of the proposed method. Furthermore, the experimental data obtained manifest that there are significant differences between the simulated and the real-world coordinates. Thus, the findings of this study provide an insightful outlook and lay down several important implications for future practice.