Ambulatory Phonation Monitoring Using Wireless Headphones With Deep Learning Technology

Ji Yan Han, Chi Te Wang*, Jia Hui Li, Ying Hui Lai

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


This study developed a real-time ambulatory phonation monitoring (APM) system that integrates wireless headphones, mobile apps, and deep neural network (DNN) technologies. It aims to overcome the limitations of taping and wiring a contact microphone on the neck, as in the current APM design. We first set up an individualized DNN model to differentiate the voice signals of the target user from those of other speakers and environmental noise. The DNN models were trained using the recordings of each user reading a standard passage. These recordings were mixed with 20 types of noise for data augmentation, which were then used to train the individualized DNN model. Nine participants were invited to use the APM system while teaching a class. The study results show that the DNN model achieved over 92% detection accuracy in the simulated noisy conditions, outperforming the baseline system (genetic algorithm model) in this system. Furthermore, we measured the phonation ratio, volume, and fundamental frequency, and all were compatible with the existing literature. The results suggest that the proposed system is a user-friendly and practical APM system for further academic and clinical application.

Original languageEnglish
Pages (from-to)4752-4762
Number of pages11
JournalIEEE Systems Journal
Issue number3
StatePublished - 1 Sep 2023


  • Background noise
  • deep learning
  • noise reduction
  • voice disorder


Dive into the research topics of 'Ambulatory Phonation Monitoring Using Wireless Headphones With Deep Learning Technology'. Together they form a unique fingerprint.

Cite this