Predicting neuropsychiatric symptoms of persons with dementia in a day care center using a facial expression recognition system

Liang Yu Chen*, Tsung Hsien Tsai, Andy Ho, Chun Hsien Li, Li Ju Ke, Li Ning Peng, Ming Hsien Lin, Fei Yuan Hsiao, Liang Kung Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Background: Behavioral and psychological symptoms of dementia (BPSD) affect 90% of persons with dementia (PwD), resulting in various adverse outcomes and aggravating care burdens among their caretakers. This study aimed to explore the potential of artificial intelligence-based facial expression recognition systems (FERS) in predicting BPSDs among PwD. Methods: A hybrid of human labeling and a preconstructed deep learning model was used to differentiate basic facial expressions of individuals to predict the results of Neuropsychiatric Inventory (NPI) assessments by stepwise linear regression (LR), random forest (RF) with importance ranking, and ensemble method (EM) of equal importance, while the accuracy was determined by mean absolute error (MAE) and root-mean-square error (RMSE) methods. Results: Twenty-three PwD from an adult day care center were enrolled with ≥ 11,500 FERS data series and 38 comparative NPI scores. The overall accuracy was 86% on facial expression recognition. Negative facial expressions and variance in emotional switches were important features of BPSDs. A strong positive correlation was identified in each model (EM: r = 0.834, LR: r = 0.821, RF: r = 0.798 by the patientwise method; EM: r = 0.891, LR: r = 0.870, RF: r = 0.886 by the MinimPy method), and EM exhibited the lowest MAE and RMSE. Conclusions: FERS successfully predicted the BPSD of PwD by negative emotions and the variance in emotional switches. This finding enables early detection and management of BPSDs, thus improving the quality of dementia care.

Original languageEnglish
Pages (from-to)1280-1291
Number of pages12
Issue number3
StatePublished - 2022


  • Artificial intelligence
  • Behavioral and psychological symptoms of dementia
  • Dementia
  • Facial expression recognition system
  • Machine learning


Dive into the research topics of 'Predicting neuropsychiatric symptoms of persons with dementia in a day care center using a facial expression recognition system'. Together they form a unique fingerprint.

Cite this