MPVF: 4D Medical Image Inpainting by Multi-Pyramid Voxel Flows

Tzu Ti Wei, Chin Kuo, Yu Chee Tseng, Jen Jee Chen

研究成果: Article同行評審


Generating a detailed 4D medical image usually accompanies with prolonged examination time and increased radiation exposure risk. Modern deep learning solutions have exploited interpolation mechanisms to generate a complete 4D image with fewer 3D volumes. However, existing solutions focus more on 2D-slice information, thus missing the changes on the <inline-formula><tex-math notation="LaTeX">$z$</tex-math></inline-formula>-axis. This paper tackles the 4D cardiac and lung image interpolation problem by synthesizing 3D volumes directly. Although heart and lung only account for a fraction of chest, they constantly undergo periodical motions of varying magnitudes in contrast to the rest of the chest volume, which is more stationary. This poses big challenges to existing models. In order to handle various magnitudes of motions, we propose a <italic>Multi-Pyramid Voxel Flows (MPVF)</italic> model that takes multiple multi-scale voxel flows into account. This renders our generation network rich information during interpolation, both globally and regionally. Focusing on periodic medical imaging, MPVF takes the maximal and the minimal phases of an organ motion cycle as inputs and can restore a 3D volume at any time point in between. MPVF is featured by a Bilateral Voxel Flow (BVF) module for generating multi-pyramid voxel flows in an unsupervised manner and a Pyramid Fusion (PyFu) module for fusing multiple pyramids of 3D volumes. The model is validated to outperform the state-of-the-art model in several indices with significantly less synthesis time. The code and models are available at <uri></uri>

頁(從 - 到)1-11
期刊IEEE Journal of Biomedical and Health Informatics
出版狀態Accepted/In press - 2023


深入研究「MPVF: 4D Medical Image Inpainting by Multi-Pyramid Voxel Flows」主題。共同形成了獨特的指紋。