A new zwitterionic small molecule 4-(dimethyl(pyridin-2-yl)ammonio)butane-1-sulfonate (PAS), synthesized from 2-dimethylaminopyrindine (2-DMAP), was developed for the ITO cathode modifier. PAS and 2-DMAP dissolved in methanol can form a thin layer on ITO cathode by a simple spin-coating process. The heteroatom moieties in 2-DMAP (sp2 and sp3 nitrogen) and PAS (sp2 nitrogen and sulfonate ion) can coordinate to the ITO surface and decrease the ITO work function by the induced surface dipoles. The fullerene-based (PBDTT-FTTE:PC71BM) inverted OSCs using PAS and 2-DMAP interlayer can achieve PCEs of 8.95 and 8.26%, respectively, which are superior to the devices without a modifier (PCE = 3.25%) and comparable to the corresponding ZnO-based device (PCE = 8.57%). Nevertheless, 2-DMAP, like other nitrogen-containing polymer interlayer materials, turns out to be not applicable to inverted organic solar cells (I-OSCs) with IT-4F as the n-type electron acceptor because the amino group of 2-DMAP can act as a nucleophile to attack the end-group of IT-4F at the interface. The decomposition of IT-4F by 2-DMAP was carefully proved to be via retro-aldol condensation. As a result, the device (PBDBT-F:IT-4F) modified with 2-DMAP displayed a low PCE of 7.34%. The zwitterionic PAS with reduced nucleophilicity and basicity can modify the ITO surface without decomposing IT-4F. The PBDBT-F:IT-4F-based device modified with PAS maintained a high PCE of 11.41%. Most importantly, the PAS-based device using the well-known Y6 acceptor (PBDBT-F:Y6) can achieve a PCE of 13.82%. This new interfacial material can be universally applied to I-OSCs employing various A-D-A-type acceptors installed with the electrophilic 1,1-dicyanamethylene-5,6-difluoro-3-indanone (FIC) end-group.