Objectives: Poor sleep quality is a common issue among older adults; it can lead to a poor quality of life and impairments in cognitive function and physical health. This study aimed to conduct a systematic review and meta-analysis of the effect of listening to music on sleep quality in older adults. Design: Systematic review and meta-analyses. Setting: Five databases, including Embase, Ovid Medline, Cochrane Library, Scopus, and the Index to Taiwan Periodical Literature System, were searched to identify studies assessing the efficacy of music therapy in older adults aged 60 years and older published through February 20, 2021. Participants: Adults aged 60 years and older. Measurements: We searched English- and Chinese-language studies of randomized control trials. All studies were reviewed by two independent investigators. The primary sleep outcome was the Pittsburgh sleep quality index. The Cochrane Collaboration tool was used to assess the risk of bias, and Review Manager 5.3 software was used to conduct the meta-analysis. Results: Five randomized control trials were included in the meta-analysis. Older adults who listened to music experienced significantly better sleep quality than those who did not listen to music [mean difference (MD): −1.96, 95% CI −2.23 to −1.73, P = 0.003]. The subgroup analysis revealed that older adults who listened to sedative music obtained a more effective improvement in sleep quality than those who listened to rhythm-centered music (MD: −2.35, 95% CI –3.59 to −1.10, P = 0.0002). Furthermore, listening to music for longer than 4 weeks (MD: −2.61, 95% CI −4.72 to −0.50, P = 0.02) was to be effective at improving sleep quality. Conclusions: Music therapy is safe and easy to administer and can effectively improve sleep quality among older adults, particularly those listening to more sedative music for at least a four-week duration.