UAV-assisted surveillance is currently gaining tremendous popularity. In standard scenarios, the implementation of real-time detection of objects through UAV-based real-time video data is quite challenging due to the large computing resource requirement. The proposed work addresses this issue and uses an Internet of Drone Things framework with a lightweight edge-dew-fog combination of computing philosophies. In the proposed framework, we presented an edge-assisted UAV-based video data collection and lightweight detection system on a fog-dew-enabled real-life test bed. The video data were captured by the Parrot Anafi UAV node and sent to the base station via an RTSP link with a gstreamer codec via the h264 encoding method. The location data for the drone were transferred through the MQTT protocol with a QoS of 2. At the base station, we implemented a CNN-based single-shot detection framework. The results show nearly 1590 ms of latency for MQTT-QoS2 and a maximum throughput benchmark of 1400 kbps for RTSP streaming. A total of 98.05% accuracy is reported for the single-shot detection approach with 66% CPU and 16% memory utilization.