We integrate lidar point clouds and large-scale vector maps to perform building modeling. The proposed scheme comprises three major steps: (a) the preprocessing of lidar point clouds and vector maps, (b) roof analysis, and (c) building reconstruction. During the preprocessing stage, the building polygons are first obtained from the polylines, followed by the selections of lidar points in the building polygons. An irregular triangulated network is then built to represent the facets. The segmentation of planar facets for roof analysis is implemented by examining the patch size and the facet orientation. The interior 3D roof edges are then determined from the intersection of the roof planes. Finally, the building models are reconstructed through regularization. Two sample sites are tested for the purposes of validation. The experimental results indicate that the proposed scheme allows for high fidelity and accuracy, provided that the point cloud density is enough.