This paper presents an enhanced, fast distributed observer for higher order multi-agent systems subjected to Byzantine attacks. The observer is designed to enhance the speed of convergence for the state estimation trajectories for the followers under the influence of Byzantine adversaries. By augmenting a robust switching law, the system ensures finite-time zero-error convergence of all followers’ state estimates to the leader’s state. The Lyapunov-based analysis is provided to prove the faster rate of convergence compared with the existing approaches. Rigorous simulation-based analysis is performed using diverse directed graph topologies with Byzantine behavior to confirm the efficacy of the proposed strategy.