Frequency response function (FRF) data can provide considerably more information on damage in the desired frequency range as compared to modal data extracted from a very limited range around resonances. Among structural health monitoring techniques, FRF-based methods have the potential to locate structural damage. Conventional structural damage detection technology collects structural response data using contact systems, such as displacement or acceleration transducers. However, installing these contact systems can be costly in terms of labor, cost, and time. Several noncontact measurement technologies, such as optical, laser, radar, and GPS, have been developed to overcome these obstacles. Given the rapid advances in optical imaging hardware technology, the use of digital photography in structural monitoring systems has attracted considerable attention. This study develops a displacement FRF-based approach to locate damage to building structures. The proposed damage location index, CurveFRFDI, improves the sensitivity of SubFRFDI, which is a substructure FRF-based damage location index proposed by Lin et al. (2012). Moreover, the feasibility of applying the proposed approach to locate damage to building structures using displacement measured by a digital camera combined with digital image correlation techniques is also investigated in this study. A numerical example and an experimental example are presented to demonstrate the feasibility of using the proposed approach to locate damage to building structures for single and multiple nonadjacent damage locations.