Background This study explored the relationship between the glycated hemoglobin (HbA1c) level in patients with or without diabetes mellitus and future risks of cardiovascular disease and death. Methods Based on a national representative cohort, a total of 5277 participants (7% with diabetes) were selected from Taiwan's Triple High Survey in 2002. The comorbidities, medication usages, and outcomes of cardiovascular disease and death, were extracted from the Taiwan's National Health Insurance Research Database and National Death Registry. Results After a median follow-up of 9.7 years, participants with diabetes had higher incidence of new onset cardiovascular disease (17.9 versus 3.16 cases per 1000 person-years) and death (20.1 versus 4.96 cases per 1000 person-years) than those without diabetes (all P < 0.001). Diabetes showed increased risk of all-cause death after adjusting for all confounders (adjusted hazard ratio [HR]: 2.29, 95% confidence interval [CI]: 1.52-3.45). Every 1% increment of HbA1c was positively associated with the risk of total cardiovascular disease (HR: 1.2, 95% CI: 1.08-1.34) and the risk of death (HR: 1.14, 95% CI: 1.03-1.26) for all participants. As compared to the reference group with HbA1c below 5.5%, participants with HbA1c levels ≥7.5% had significantly elevated future risks of total cardiovascular disease (HR: 1.82, 95% CI: 1.01-3.26) and all-cause death (HR: 2.45, 95% CI: 1.45-4.14). Conclusions/Interpretation Elevated HbA1C levels were associated with increased risks of cardiovascular disease and death, the suboptimal glycemic control with HbA1c level over 7.5% (58.5 mmol/mol) was strongly associated with increased risks of cardiovascular disease and all-cause death.