Mandarin TONE RECOGNITION FROM BRAIN DYNAMICS

The development of technologies that enable patients with speech disorders to communicate is critical. Understanding the neural processing of natural speech processing is an important first step for designing Brain-Computer Interface (BCI) based speech enhancement and speech recognition systems. This study explored the neurophysiological correlates of Mandarin pronunciation to develop a BCI-based system. Previous research has mainly focused on English articulation; however, Mandarin is the most widely spoken language worldwide and quite distinct from English. Tone has an essential part of Mandarin pronunciation. This study created a new BRCSpeech database, which uses “Focus Group Interview” approach to build the database designed by linguists (speech, multimedia signal, and biomedical signal processing experts) and neurologists. The database allows us to select frequently used vocabulary in our group discussions. Subjects were instructed to articulate these words/sentences while their Electroencephalography (EEG) was collected. EEG data were analyzed by short-time Fourier transform. This pilot study focuses on the differences in EEG activities between Mandarin spoken with and without tone. Specifically, this study applied Linear discriminant analysis (LDA) and five-fold cross-validation to the EEG features to classify the speech with versus without tones. Study results showed that EEG Delta-Theta and Beta data could be used to distinguish “tone” and “no tone” sentences with good accuracy.