Hemophagocytic lymphohistiocytosis (HLH) is a heterogeneous group of hyperinflammatory statuses that are difficult to diagnose and can be life-threatening. Bone marrow (BM) hemophagocytosis is one of the diagnostic criteria according to HLH 2004 diagnostic criteria and HS score. Limited studies have focused on the prognostic factors of BM hemophagocytosis and its association with hematologic malignancies. We aimed to analyze the clinical significance of BM hemophagocytosis. Patients with BM hemophagocytosis, either by cytology or pathology, were enrolled at Taipei Veterans General Hospital from January 2002 to July 2021. Relevant clinical and laboratory data were extracted from medical records. Of 119 patients with BM hemophagocytosis, 57 were diagnosed with hematologic malignancies. The median age of the patients was 58, ranging from 21 to 90. Splenomegaly (adjusted odds ratio [aOR] 2.96; 95% confidence interval [CI] 1.13–7.79) was a risk factor for hematologic malignancies, while autoimmune disease (aOR 0.07; 95% CI 0.01–0.39) and increased D-dimer (aOR 0.25; 95% CI 0.07–0.92) were protective factors. Risk factors for mortality in patients with BM hemophagocytosis were hematologic malignancies (adjusted hazard ratio [aHR] 2.34; 95% CI 1.24–4.44), Eastern Cooperative Oncology Group score ≥3 (aHR 2.42; 95% CI 1.20–4.89) and thrombocytopenia (aHR 3.09; 95% CI 1.04–9.16). In conclusion, among patients with BM hemophagocytosis, splenomegaly was a predictor of hematologic malignancies. Patients with hematologic malignancies, poor performance status, or thrombocytopenia had a higher mortality risk. Further validation studies are warranted.