Defects to popular two-dimensional (2D) transition metal dichalcogenides (TMDs) seriously lower the efficiency of field-effect transistor (FET) and depress the development of 2D materials. These atomic defects are mainly identified and researched by scanning tunneling microscope (STM) because it can provide precise measurement without harming the samples. The long analysis time of STM for locating defects in images has been solved by combining feature detection with convolutional neural networks (CNN). However, the low signal-noise ratio, insufficient data, and a large amount of TMDs members make the automatic defect detection system hard to be applied. In this study, we propose a deep learning-based atomic defect detection framework (DL-ADD) to efficiently detect atomic defects in molybdenum disulfide (MoS2) and generalize the model for defect detection in other TMD materials. We design DL-ADD with data augmentation, color preprocessing, noise filtering, and a detection model to improve detection quality. The DL-ADD provides precise detection in MoS2 (F2-scores is 0.86 on average) and good generality to WS2 (F2-scores is 0.89 on average).