Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network

Cheng Jie Yang; Wei Chih Li; Meng Teen Wan; Wai-Chi Fang

doi:10.1109/ISCAS51556.2021.9401588

Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network

Cheng Jie Yang, Wei Chih Li, Meng Teen Wan, Wai-Chi Fang^*

^*Corresponding author for this work

Institute of Electronics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

In this paper, we presented an affective computing engine using the Long-term Recurrent Convolutional Network (LRCN) on electroencephalogram (EEG) physiological signal with 8 emotion-related channels. LRCN was chosen as the emotional classifier because compared to the traditional CNN, it integrates memory units allowing the network to discard or update previous hidden states which are particularly adapted to explore the temporal emotional information in the EEG sequence data. To achieve a real-time AI-edge affective computing system, the LRCN model was implemented on a 16nm Fin-FET technology chip. The core area and total power consumption of the LRCN chip are respectively 1.13x1.14 mm² and 48.24 mW. The computation time was 1.9µs and met the requirements to inference every sample. The training process cost 5.5µs per sample on clock frequency 125Hz which was more than 20 times faster than 128µs achieved with GeForce GTX 1080 Ti using python. The proposed model was evaluated on 52 subjects with cross-subject validation and achieved average accuracy of 88.34%, and 75.92% for respectively 2-class, 3-class.

Original language	English
Title of host publication	2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Number of pages	5
ISBN (Electronic)	9781728192017
DOIs	https://doi.org/10.1109/ISCAS51556.2021.9401588
State	Published - May 2021
Event	53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Daegu, Korea, Republic of Duration: 22 May 2021 → 28 May 2021

Publication series

Name	Proceedings - IEEE International Symposium on Circuits and Systems
Volume	2021-May
ISSN (Print)	0271-4310

Conference

Conference	53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021
Country/Territory	Korea, Republic of
City	Daegu
Period	22/05/21 → 28/05/21

Keywords

Affective computing
Deep learning chip
Electroencephalogram
Long-term recurrent convolutional network
On-chip learning

Access to Document

10.1109/ISCAS51556.2021.9401588

Cite this

Yang, C. J., Li, W. C., Wan, M. T., & Fang, W.-C. (2021). Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network. In 2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings Article 9401588 (Proceedings - IEEE International Symposium on Circuits and Systems; Vol. 2021-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCAS51556.2021.9401588

Yang, Cheng Jie ; Li, Wei Chih ; Wan, Meng Teen et al. / Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network. 2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2021. (Proceedings - IEEE International Symposium on Circuits and Systems).

@inproceedings{3e4d0252e1814f9ca856bfd5cc23ab9b,

title = "Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network",

abstract = "In this paper, we presented an affective computing engine using the Long-term Recurrent Convolutional Network (LRCN) on electroencephalogram (EEG) physiological signal with 8 emotion-related channels. LRCN was chosen as the emotional classifier because compared to the traditional CNN, it integrates memory units allowing the network to discard or update previous hidden states which are particularly adapted to explore the temporal emotional information in the EEG sequence data. To achieve a real-time AI-edge affective computing system, the LRCN model was implemented on a 16nm Fin-FET technology chip. The core area and total power consumption of the LRCN chip are respectively 1.13x1.14 mm2 and 48.24 mW. The computation time was 1.9µs and met the requirements to inference every sample. The training process cost 5.5µs per sample on clock frequency 125Hz which was more than 20 times faster than 128µs achieved with GeForce GTX 1080 Ti using python. The proposed model was evaluated on 52 subjects with cross-subject validation and achieved average accuracy of 88.34%, and 75.92% for respectively 2-class, 3-class.",

keywords = "Affective computing, Deep learning chip, Electroencephalogram, Long-term recurrent convolutional network, On-chip learning",

author = "Yang, {Cheng Jie} and Li, {Wei Chih} and Wan, {Meng Teen} and Wai-Chi Fang",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE; 53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021 ; Conference date: 22-05-2021 Through 28-05-2021",

year = "2021",

month = may,

doi = "10.1109/ISCAS51556.2021.9401588",

language = "English",

series = "Proceedings - IEEE International Symposium on Circuits and Systems",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings",

address = "United States",

}

Yang, CJ, Li, WC, Wan, MT & Fang, W-C 2021, Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network. in 2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings., 9401588, Proceedings - IEEE International Symposium on Circuits and Systems, vol. 2021-May, Institute of Electrical and Electronics Engineers Inc., 53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021, Daegu, Korea, Republic of, 22/05/21. https://doi.org/10.1109/ISCAS51556.2021.9401588

Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network. / Yang, Cheng Jie; Li, Wei Chih; Wan, Meng Teen et al.
2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2021. 9401588 (Proceedings - IEEE International Symposium on Circuits and Systems; Vol. 2021-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network

AU - Yang, Cheng Jie

AU - Li, Wei Chih

AU - Wan, Meng Teen

AU - Fang, Wai-Chi

PY - 2021/5

Y1 - 2021/5

N2 - In this paper, we presented an affective computing engine using the Long-term Recurrent Convolutional Network (LRCN) on electroencephalogram (EEG) physiological signal with 8 emotion-related channels. LRCN was chosen as the emotional classifier because compared to the traditional CNN, it integrates memory units allowing the network to discard or update previous hidden states which are particularly adapted to explore the temporal emotional information in the EEG sequence data. To achieve a real-time AI-edge affective computing system, the LRCN model was implemented on a 16nm Fin-FET technology chip. The core area and total power consumption of the LRCN chip are respectively 1.13x1.14 mm2 and 48.24 mW. The computation time was 1.9µs and met the requirements to inference every sample. The training process cost 5.5µs per sample on clock frequency 125Hz which was more than 20 times faster than 128µs achieved with GeForce GTX 1080 Ti using python. The proposed model was evaluated on 52 subjects with cross-subject validation and achieved average accuracy of 88.34%, and 75.92% for respectively 2-class, 3-class.

AB - In this paper, we presented an affective computing engine using the Long-term Recurrent Convolutional Network (LRCN) on electroencephalogram (EEG) physiological signal with 8 emotion-related channels. LRCN was chosen as the emotional classifier because compared to the traditional CNN, it integrates memory units allowing the network to discard or update previous hidden states which are particularly adapted to explore the temporal emotional information in the EEG sequence data. To achieve a real-time AI-edge affective computing system, the LRCN model was implemented on a 16nm Fin-FET technology chip. The core area and total power consumption of the LRCN chip are respectively 1.13x1.14 mm2 and 48.24 mW. The computation time was 1.9µs and met the requirements to inference every sample. The training process cost 5.5µs per sample on clock frequency 125Hz which was more than 20 times faster than 128µs achieved with GeForce GTX 1080 Ti using python. The proposed model was evaluated on 52 subjects with cross-subject validation and achieved average accuracy of 88.34%, and 75.92% for respectively 2-class, 3-class.

KW - Affective computing

KW - Deep learning chip

KW - Electroencephalogram

KW - Long-term recurrent convolutional network

KW - On-chip learning

UR - http://www.scopus.com/inward/record.url?scp=85109031715&partnerID=8YFLogxK

U2 - 10.1109/ISCAS51556.2021.9401588

DO - 10.1109/ISCAS51556.2021.9401588

M3 - Conference contribution

AN - SCOPUS:85109031715

T3 - Proceedings - IEEE International Symposium on Circuits and Systems

BT - 2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021

Y2 - 22 May 2021 through 28 May 2021

ER -

Yang CJ, Li WC, Wan MT, Fang WC. Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network. In 2021 IEEE International Symposium on Circuits and Systems, ISCAS 2021 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2021. 9401588. (Proceedings - IEEE International Symposium on Circuits and Systems). doi: 10.1109/ISCAS51556.2021.9401588

Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this