Thanks to the nature of molecular orbitals, the absorption spectra of organic semiconductors are not continuous like those in traditional inorganic semiconductors, which offers a unique application of organic photovoltaics (OPVs): semitransparent OPVs. Recently, the exciting progress of materials design has promoted the development of semitransparent OPVs. However, in the perspective of device engineering, almost all reported works reduce the thickness of back/reflected electrode to obtain high average visible transmittance (AVT), which is a trade-off between power conversion efficiency (PCE) and the transmittance of the whole solar spectrum (visible and infrared), and therefore limit the further development. Herein, a unique strategy of "transparent hole-transporting frameworks" is proposed. A hole-transporting large-bandgap polymer (poly[bis(4-phenyl)(2,4,6-trimethylphenyl)amine (PTAA)) is employed to partially replace polymer donors in the active layer of PBDB-T/Y1. PTAA is a p-type polymer with a large bandgap of 2.9 eV; the partial substitution of PBDB-T by PTAA reduces the absorption of the active layer only in the visible region, keeping the hole-transporting pathways as well as the optimized film morphology. As a result, semitransparent OPVs with PCEs of 12% and AVTs of 20% are achieved, both on rigid and flexible substrates. To demonstrate the generality, this strategy is also used in three different active layers.