Background Health care improvements have led to increased survival among stroke patients; however, the disability level remains high. These patients require assistance from caregivers, particularly in the first year after stroke. Longitudinal studies of quality of life (QoL) and the factors associated with QoL for both patients and caregivers are limited. Objectives To describe the changes in QoL and determine the factors associated with QoL for both stroke patients and their informal caregivers in the first year after stroke. Methods This multicenter prospective study was conducted in public hospitals in Mongolia. In this study, 155 first-time stroke patients and their 88 informal caregivers were followed up for 1 year. The WHOQOL-BREF questionnaire was used to assess QoL. The Barthel Index was administered to the patients at the baseline and after 1 year. A generalized estimating equation analysis was used to determine the factors associated with QoL. Results The QoL of stroke patients in the domains of physical and environmental aspects improved significantly (p < 0.05) after 1 year; however, social relationship and psychological health declined, but the decline was not significant. Among caregivers, psychological health and social relationship domains improved significantly. Factors associated with low QoL among stroke patients were advancing age, male patients, being single and less improvements on BI score, and among caregivers were poor physical health and financial difficulties. Conclusions Efficient rehabilitation therapy for poststroke patients can improve their QoL. Disability training and financial support for caregivers of poststroke patients might be helpful; however, further research is required.