This paper presents a multi-damage identification approach for three-dimensional frame structures that combines the modal strain energy (MSE)-based method with the particle swarm optimization (PSO) algorithm. Firstly, the potential damage locations are identified by the modal strain energy-based index (MSEBI). This index is calculated from the difference in the MSE values of the elements corresponding to the two states prior to and after the damage’s occurrence. In order to improve accuracy and reduce the limitations of the noise elements in determining damage location, the MSEBI index is determined from the first six vibration mode shapes. Secondly, the PSO with a function of objective variables based on the vibration modal strain energy (MSE) values determines the damage level of the elements identified in the first step. The accuracy and reliability of the proposed method are evaluated by analyzing a four-story three-dimensional frame structure with 63 elements, considering three different damage scenarios. The obtained results confirm that the proposed method accurately identifies the location and severity of multi-damage in the three-dimensional frame structures.