Accurate diagnosis of endoscopic mucosal healing in ulcerative colitis using deep learning and machine learning

Background: In clinical applications, mucosal healing is a therapeutic goal in patients with ulcerative colitis (UC). Endoscopic remission is associated with lower rates of colectomy, relapse, hospitalization, and colorectal cancer. Differentiation of mucosal inflammatory status depends on the experience and subjective judgments of clinical physicians. We developed a computer-aided diagnostic system using deep learning and machine learning (DLML-CAD) to accurately diagnose mucosal healing in UC patients. Methods: We selected 856 endoscopic colon images from 54 UC patients (643 images with endoscopic score 0-1 and 213 with score 2-3) from the endoscopic image database at Tri-Service General Hospital, Taiwan. Endoscopic grading using the Mayo endoscopic subscore (MES 0-3) was performed by two reviewers. A pretrained neural network extracted image features, which were used to train three different classifiers—deep neural network (DNN), support vector machine (SVM), and k-nearest neighbor (k-NN) network. Results: DNN classified MES 0 to 1, representing mucosal healing, vs MES 2 to 3 images with 93.8% accuracy (sensitivity 84.6%, specificity 96.9%); SVM had 94.1% accuracy (sensitivity 89.2%, specificity 95.8%); and k-NN had 93.4% accuracy (sensitivity 86.2%, specificity 95.8%). Combined, ensemble learning achieved 94.5% accuracy (sensitivity 89.2%, specificity 96.3%). The system further differentiated between MES 0, representing complete mucosal healing, and MES 1 images with 89.1% accuracy (sensitivity 82.3%, specificity 92.2%). Conclusion: Our DLML-CAD diagnosis achieved 94.5% accuracy for endoscopic mucosal healing and 89.0% accuracy for complete mucosal healing. This system can provide clinical physicians with an accurate auxiliary diagnosis in treating UC.