The effects of air pollution on sleep and dementia remain unclear. The objective of this study was to investigate the effects of air pollution on cognitive function as mediated by the sleep cycle. A cross-sectional study design was conducted to recruit 4866 subjects on which PSG had been performed. Fifty of them were further given a cognitive function evaluation by the MMSE and CASI as well as brain images by CT and MRI. Associations of 1-year air pollution parameters with sleep parameters, cognitive function, and brain structure were examined. We observed that O3 was associated with a decrease in arousal, an increase in the N1 stage, and a decrease in the N2 stage of sleep. NO2 was associated with an increase in the N1 stage, a decrease in the N2 stage, and an increase in REM. PM2.5 was associated with a decrease in the N1 stage, increases in the N2 and N3 stages, and a decrease in REM. The N1 and N2 stages were associated with cognitive decline, but REM was associated with an increase in cognitive function. The N1 stage was a mediator of the effects of PM2.5 on the concentration domain of the MMSE. O3 was associated with an increase in the pars orbitalis volume of the left brain. NO2 was associated with increases in the rostral middle frontal volume, supramarginal gyrus volume, and transverse temporal volume of the left brain, and the pars opercularis volume of the right brain. PM2.5 was associated with increases in the pars triangularis volume of the left brain and the fusiform thickness of the right brain. In conclusion, we observed that air pollution was associated with cognitive decline by mediating effects on the sleep cycle with changes in the brain structure in controlling executive, learning, and language functions in adults.