Two-dimensional (2D) nanosheets (NSs) of a Ni-S coordination polymer have been successfully synthesized with the use of 2D Ni(OH)2 NSs grown on conductive carbon cloth as the template and 1,4-benzenedithiol as the ligand. In situ X-ray absorption spectroscopy revealed that the as-prepared catalyst was entirely transformed into ultrathin Ni NSs under alkaline reductive conditions. The in-situ-generated catalysts exhibited superior activity toward the hydrogen evolution reaction (HER) with an overpotential of 80 mV to reach 10 mA cm−2. Studies revealed that the large-area ultrathin Ni NSs served as active sites for H2 generation, and the trace sulfur adsorbed on the Ni surface promoted water dissociation. This work has developed a templating approach for preparing highly active HER electrocatalysts and identified the real active sites under electrocatalytic conditions.