The growing popularity of inverter-based resources (IBRs) within the modern power transmission networks presents a major challenge to traditional fault detection and fault classification schemes owing to the minimal contribution by fault currents, being controlled by the inbuilt control scheme, and non-stationary signal response. To overcome these difficulties, this paper will suggest a two-level protection scheme for IBR-concentrated transmission systems. In the first stage, a voltage trajectory deformation-based fault detection index is developed in the stationary \(\alpha - \beta\)reference frame using Clarke transformed voltage signals. The proposed analytical index utilizes the geometric distortion of the voltage trajectories and is independent of the magnitude of fault current, which ensures reliable fault detection in the situation of high fault resistance and weak grid. In the second stage, Intrinsic Time-Scale Decomposition (ITD) is employed to extract fault-relevant transient components from post-fault voltage signals, followed by statistical feature extraction and fault classification using an Extreme Gradient Boosting (XGBoost) classifier. The proposed methodology is validated on a modified IEEE 14-bus test system using detailed electromagnetic transient simulations in PSCAD. Rigorous simulations are performed under different types of faults, fault location (10% to 90%), fault impedance (0 to 100 Ω), fault-initiation angle (0°–180°), noise levels (up to 40 dB), and IBR operating conditions. The simulation results shows that the proposed scheme detect the faults within a quarter cycle of a voltage signal and the scheme classifies the faults with an overall accuracy of 99.65%. The accuracy of the classification even under high impedance fault and noisy conditions is above 95% indicating the resilience and practical viability of the proposed framework in the modern IBR-rich transmission networks context.