RC frame structure is a commonly used structural system in China, in which the frame joint connects the beams and columns and is the key part of force transmission. Various damage patterns have been inspected in past studies, which are caused by the coupled effect of beams, columns and joints in the damage developing. In order to realize the performance-based design targets of RC frame structures, it is necessary to accurately identify the seismic damage pattern of an RC frame structure, which is also a prerequisite to suggest the repair procedure effectively. According to the test data of 147 RC beam-column joint assemblies collected from 68 Chinese literatures, it is found that the final failure modes of RC beam-column joint assemblies can be classified into bending failure (B), bending shear failure (BJ) and shear failure (J). The failure mode is determined by various mechanical and geometric properties, and damage development processes. In this study, the characteristic crack pattern is employed to classify damage states (DS), and further to establish the fragility curves of RC beam-column joint assemblies with B, BJ and J failure modes, respectively. The displacement angle was taken as the engineering demand parameter (EDP). It is found that obvious differences were observed in the damage phenomena of different types of joint assemblies, which are closely related to the bearing capacity characteristic points. According to the database and determination of damage status, the seismic damage of RC frame structures can be easily assessed in the field reconnaissance, and subsequent repair and reinforcement method could be correctly suggested.