Ambulatory Phonation Monitoring Using Wireless Headphones With Deep Learning Technology

This study developed a real-time ambulatory phonation monitoring (APM) system that integrates wireless headphones, mobile apps, and deep neural network (DNN) technologies. It aims to overcome the limitations of taping and wiring a contact microphone on the neck, as in the current APM design. We first set up an individualized DNN model to differentiate the voice signals of the target user from those of other speakers and environmental noise. The DNN models were trained using the recordings of each user reading a standard passage. These recordings were mixed with 20 types of noise for data augmentation, which were then used to train the individualized DNN model. Nine participants were invited to use the APM system while teaching a class. The study results show that the DNN model achieved over 92% detection accuracy in the simulated noisy conditions, outperforming the baseline system (genetic algorithm model) in this system. Furthermore, we measured the phonation ratio, volume, and fundamental frequency, and all were compatible with the existing literature. The results suggest that the proposed system is a user-friendly and practical APM system for further academic and clinical application.