A VR-based Training and Intelligent Assessment System Integrated with Multi-modal Sensing for Children with Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) exhibits social communication and social interaction disorders, and abnormal restrictive and repetitive behaviors. However, symptoms of infants less than 1-year-old are difficult to reliably predict subsequent diagnosis. Patients with mild ASD may not be discovered until school age, because schools have more opportunities for social activities. In addition, the therapist also needs to consider the labor cost. To provide effective treatment, it also needs to consume more resources. The current situation in Taiwan is that outlying islands and remote areas often have insufficient manpower for therapists. If VR technology can be applied, some of the problems may be solved. However, due to the global pandemic, COVID-19, early treatments or group treatments in many countries have been forced to stop. If VR technology can provide interpersonal interaction scenes, the training of ASD children can hardly be affected.This research uses Virtual Reality (VR) technology, combined with wearable multi-model sensing technology, including EEG, eye tracking, heart rate variability (HRV), and breath-sensing strap. Physiological signals and game performance data are collected while users are training, and integrate multiple evaluation scales such as ADOS, SRS, and CBCL. Statistical analysis of these data is performed to classify them through machine learning models to develop a VR assistance system that can be used to evaluate the diagnosis, severity, and social behavior treatment of ASD. This system presents assessment and therapy in a game-oriented way. In addition to enhancing the incentives for users to participate, it provides better training results than traditional training. It is also an effective and convenient tool for the therapist to use during evaluation and training.