Clear and deep temporal focusing multiphoton microscopy imaging using deep prediction with PhyCell and ConvLSTM

Hao Chung Chi, Anupama Nair, Yvonne Yuling Hu, Feng Chun Hsu, Chia Wei Hsu, Chun Yu Lin, Shean Jen Chen*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Temporal focusing multiphoton excitation microscopy (TFMPEM) can rapidly provide 3D imaging in neuroscience; however, due to the widefield illumination and the use of camera detector, the strong scattering of emission photons through biotissue will degrade the image quality and reduce the penetration depth. As a result, TFMPEM images suffers from poor spatial resolution and low signal-to-noise ratio (SNR), burying weak fluorescent signals of small structures such as neurons in calyx part, especially for deep layers under fast acquisition rate. In the study, we present a prediction learning model with depth information to overcome. First, a point-scanning multiphoton excitation microscopy (PSMPEM) image as the gold standard was precisely registered to the corresponding TFMPEM image via a linear affine transformation and an unsupervised VoxelMorph network. Then, a multi-stage 3D U-Net model with cross-stage feature fusion mechanism and self-supervised attention module has been developed to restore shallow layers of drosophila mushroom body under cross-modality training. Furthermore, a convolutional long short-term memory (ConvLSTM)-based network with PhyCell, which is designed to forecast the deeper information according to previous 3D information, is introduced for the prediction of depth information.

Original languageEnglish
Title of host publicationAdvances in Microscopic Imaging IV
EditorsEmmanuel Beaurepaire, Adela Ben-Yakar, YongKeun Park
ISBN (Electronic)9781510664692
StatePublished - 2023
EventAdvances in Microscopic Imaging IV 2023 - Munich, Germany
Duration: 28 Jun 2023 → …

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X


ConferenceAdvances in Microscopic Imaging IV 2023
Period28/06/23 → …


  • bioimaging
  • cross modality
  • deep learning
  • deep prediction
  • Multiphoton microscopy
  • temporal focusing


Dive into the research topics of 'Clear and deep temporal focusing multiphoton microscopy imaging using deep prediction with PhyCell and ConvLSTM'. Together they form a unique fingerprint.

Cite this