The presence of cranial and facial bone fractures is an important finding on non-enhanced head computed tomography (CT) scans from patients who have sustained head trauma. Some prior studies have proposed automatic cranial fracture detections, but studies on facial fractures are lacking. We propose a deep learning system to automatically detect both cranial and facial bone fractures. Our system incorporated models consisting of YOLOv4 for one-stage fracture detection and improved ResUNet (ResUNet++) for the segmentation of cranial and facial bones. The results from the two models mapped together provided the location of the fracture and the name of the fractured bone as the final output. The training data for the detection model were the soft tissue algorithm images from a total of 1,447 head CT studies (a total of 16,985 images), and the training data for the segmentation model included 1,538 selected head CT images. The trained models were tested on a test dataset consisting of 192 head CT studies (a total of 5,890 images). The overall performance achieved a sensitivity of 88.66%, a precision of 94.51%, and an F1 score of 0.9149. Specifically, the cranial and facial regions were evaluated and resulted in a sensitivity of 84.78% and 80.77%, a precision of 92.86% and 87.50%, and F1 scores of 0.8864 and 0.8400, respectively. The average accuracy for the segmentation labels concerning all predicted fracture bounding boxes was 80.90%. Our deep learning system could accurately detect cranial and facial bone fractures and identify the fractured bone region simultaneously.