A Smartphone-Based Multi-Functional Hearing Assistive System to Facilitate Speech Recognition in the Classroom

In this paper, we propose a smartphone-based hearing assistive system (termed SmartHear) to facilitate speech recognition for various target users, who could benefit from enhanced listening clarity in the classroom. The SmartHear system consists of transmitter and receiver devices (e.g., smartphone and Bluetooth headset) for voice transmission, and an Android mobile application that controls and connects the different devices via Bluetooth or WiFi technology. The wireless transmission of voice signals between devices overcomes the reverberation and ambient noise effects in the classroom. The main functionalities of SmartHear include: 1) configurable transmitter/receiver assignment, to allow flexible designation of transmitter/receiver roles; 2) advanced noise-reduction techniques; 3) audio recording; and 4) voice-to-text conversion, to give students visual text aid. All the functions are implemented as a mobile application with an easy-to-navigate user interface. Experiments show the effectiveness of the noise-reduction schemes at low signal-to-noise ratios in terms of standard speech perception and quality indices, and show the effectiveness of SmartHear in maintaining voice-to-text conversion accuracy regardless of the distance between the speaker and listener. Future applications of SmartHear are also discussed.