Clear and deep temporal focusing multiphoton microscopy imaging using deep prediction with PhyCell and ConvLSTM

Temporal focusing multiphoton excitation microscopy (TFMPEM) can rapidly provide 3D imaging in neuroscience; however, due to the widefield illumination and the use of camera detector, the strong scattering of emission photons through biotissue will degrade the image quality and reduce the penetration depth. As a result, TFMPEM images suffers from poor spatial resolution and low signal-to-noise ratio (SNR), burying weak fluorescent signals of small structures such as neurons in calyx part, especially for deep layers under fast acquisition rate. In the study, we present a prediction learning model with depth information to overcome. First, a point-scanning multiphoton excitation microscopy (PSMPEM) image as the gold standard was precisely registered to the corresponding TFMPEM image via a linear affine transformation and an unsupervised VoxelMorph network. Then, a multi-stage 3D U-Net model with cross-stage feature fusion mechanism and self-supervised attention module has been developed to restore shallow layers of drosophila mushroom body under cross-modality training. Furthermore, a convolutional long short-term memory (ConvLSTM)-based network with PhyCell, which is designed to forecast the deeper information according to previous 3D information, is introduced for the prediction of depth information.