<p>The advancement of drone technology has unlocked numerous possibilities, but it has also led to a surge in incidents of damage and violation caused by drones. This leads to the increment of the need for technologies preventing drone misuse and ensuring their safe operation. Recent progress in edge devices and computer vision has enabled the development of lightweight and efficient tracking systems. While fixed-camera systems have been proposed for drone detection and tracking, such configurations are inherently limited in their ability to reliably track fast and small drones in dynamic environments. To overcome these limitations, Pan-Tilt-Zoom (PTZ) cameras can be used to track the detected drone with pan and tilt movements. While the PTZ camera is a viable solution for detecting and tracking drones, real-world problems such as network latency, detection accuracy and control stability have rarely been systematically investigated. In this paper, we conduct a comprehensive analysis of a lightweight real-time drone tracking system that integrates image capture, YOLOv8-based detection, and PTZ control via Proportional-Integral-Differential (PID) control feedback, focusing on its performance under the realistic operational constraints. Quantitative analysis reveals the impact of each factor on tracking performance and suggests directions for system optimization. The findings offer practical guidance for the deployment of drone tracking systems and highlight design considerations essential for ensuring robust real-world operation.</p>