Organic photodetectors (OPDs) with spectral response extending from ultraviolet to near-infrared domains are of great interest for many applications. Morphological impact on the performance of photomultiplication (PM)-type OPDs, however, is still rarely investigated so far. Herein, a non-fullerene acceptor, Y6-Se-HD, is synthesized, in which heavier selenium atoms substitute the sulfur atoms in the core of Y6, and used for fabricating PM-OPDs. The resulting devices exhibit remarkable PM effects in a very broadband spectral range covering from 320 to 1090 nm. A maximum EQE value of ≈6500% at 860 nm is achieved at a low bias of −1.0 V. As compared with the device prepared with Y6 molecules, the Y6-Se-HD OPDs exhibited much enhanced performance. From the morphological analysis, we infer that Y6-Se-HD almost covers the entire active layer and avoids the direct contact of the hole-trapping donor polymers with the Ag electrode, thereby resulting in stronger charge trapping and preventing possible charge recombination and/or quenching. Furthermore, finer phase separation between the donor and acceptor molecules also facilitates hole trapping and strengthens the PM effects. This research highlights the importance of morphological effects on the PM-OPDs and demonstrates one approach for controlling the device morphology.