Background: The prognosis of unfavorable cancer of unknown primary is extremely poor. This is the first report to compared the treatment results between generations of CUP and examined prognostic factors. Methods: This retrospective single-center cohort study enrolled 68 patients with newly diagnosed unfavorable cancer of unknown primary at Taipei Veteran General Hospital from 2017 to 2020 as study cohort and 167 patients from 2000 to 2009 as historical cohort. Results: The median overall survival was 4.3 months in the study cohort (95% CI, 2.7–6.2 months) and 4.5 months in the historical cohort (95% CI, 3.0–5.5 months; p = 0.858). Eleven patients in the study cohort received immunotherapy. The disease control rates were 45%. Multivariate analysis showed that an Eastern Cooperative Oncology Group score > 1 and a C-reactive protein level > 1 correlated with poor survival. A new prognostic stratification model was constructed by using Eastern Cooperative Oncology Group score and C-reactive protein values. The good-, intermediate-, and poor-risk groups had distinct median overall survival of 18.3, 7.0 and 1.2 months, respectively (area under the curve, 0.817; p < 0.001). Conclusion: The outcome of unfavorable cancer of unknown primary has not changed much over the last 20 years. The application of a new prognostic stratification model can further stratify unfavorable cancer of unknown primary.