The passively reconstructed Impulse Response Function (IRF) (or Green’s function) between two points of a medium can reveal valuable details about the dynamic behavior of a system, including the detection of structural damage. This kind of “passive” inspection simplifies the Structural Health Monitoring (SHM) task by reducing or eliminating the active excitation sources. In the monitoring of civil infrastructure, one method that utilizes concepts of passive sensing is Seismic Interferometry (SI). This paper analyzes theoretical aspects of passive reconstruction of the system’s IRF, and particularly the outcome of normalized cross-correlations with segmental averages vs. cross-correlations with time averages, as well as the role of causal vs. acausal IRF reconstructions. Where the traditional SI has been applied to monitor civil infrastructures (buildings and bridges), in the low frequency range, this work focuses on the transient high-frequency range that offers increased sensitivity to localized damage. Moreover, aspects of wave beamforming will be utilized to image the location of damage in a “passive” manner. Results will be presented to detect beam-column damage in a laboratory-scale multistory steel frame, and to image defects in aluminum pieces and composite panels through passive beamforming.