CeOs4Sb12 (COS) and PrOs4Sb12 (POS) are two representative compounds that provide the ideal vantage point to systematically study the physics of multi-f-electron systems. COS with Ce 4f1, and POS with Pr 4f2 configurations show distinct properties of Kondo insulating and heavy fermion superconductivity, respectively. We unveiled the underlying microscopic origin by angle-resolved photoemission spectroscopy studies. Their eV-scale band structure matches well, representing the common characters of conduction electrons in ROs4Sb12 systems (R=rare earth). However, f electrons interact differently with conduction electrons in COS and POS. Strong hybridization between conduction electrons and f electrons is observed in COS with band dependent hybridization gaps, and the development of a Kondo insulating state is directly revealed. Although the ground state of POS is a singlet, finite but incoherent hybridization exists, which can be explained by the Kondo scattering with the thermally excited triplet crystalline electric field state. Our results help us to understand the intriguing properties in COS and POS, and provide a clean demonstration of the microscopic differences in heavy fermion systems with 4f1 and 4f2 configurations.