With the incorporation of nitrogen (N) into InAs quantum dots (QDs), the carrier distribution near the QD displays electron emissions from a localized N-induced defect state at 0.34 eV and a weak emission at 0.15 eV from the QD. This defect state causes drastic carrier depletion in the neighboring GaAs bottom layer near the QD, which can effectively suppress tunneling emission for the QD excited states. As a result, electrons escape from the QD ground state through thermal emission to near the GaAs conduction band, rather than through thermal emission to the QD first excited state and a subsequent tunneling to the GaAs conduction band, as observed in InAs QDs without N incorporation. Thermal annealing can weaken the defect emission and enhance the QD emission, suggesting a removal of the defect state and a recovery of carriers in the QD. Increasing annealing temperature can significantly decrease the emission time and energy of the QD emission, which is explained by a weakening of tunneling suppression due to the removal of the defect state.