Hierarchical prosody modeling for Mandarin spontaneous speech

Cheng Hsien Lin, Chung Long You, Chen Yu Chiang, Yih-Ru Wang, Sin-Horng Chen

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


In this paper, a hierarchical prosody model (HPM)-based method for Mandarin spontaneous speech is proposed. First, an HPM is designed for describing relations among acoustic features of utterances, linguistic features of texts, and prosodic tags representing the underlying hierarchical prosodic structures of utterances. Subsequently, a sequential optimization algorithm is employed to train the HPM based on a large conversational speech corpus, the Mandarin Conversational Dialogue Corpus (MCDC), which features orthographic transcriptions and prosodic event annotations. In this unsupervised training method, all utterances of the MCDC are labeled with two types of prosodic tags, namely, break and prosodic states, automatically and simultaneously. After training, the HPM parameters are examined to identify critical prosodic properties of Mandarin spontaneous speech, which are then compared with their counterparts in the read-speech HPM. The prosodic tags on the studied utterances enable mapping of various prosodic events onto the hierarchical prosodic structures of the utterances. Prosodic analyses of some disfluent events are conducted using the prosodic tags affixed to the MCDC. Finally, an application of the HPM to assist in Mandarin spontaneous-speech recognition is discussed. Significant relative error rate reductions of 9.0%, 9.2%, 15.6%, and 7.3% are obtained for base-syllable, character, tone, and word recognition, respectively.

Original languageEnglish
Pages (from-to)2576-2596
Number of pages21
JournalJournal of the Acoustical Society of America
Issue number4
StatePublished - 1 Apr 2019


Dive into the research topics of 'Hierarchical prosody modeling for Mandarin spontaneous speech'. Together they form a unique fingerprint.

Cite this