The method of fusing visual and inertial information for localization has been shown to deliver stable navigational outcomes when the unmanned aerial vehicle (UAV) undergoes large attitude changes. However, when the UAV’s motion state changes such as sudden stopping and prolonged hovering to identify the search target, the drift noise of the IMU and insufficient constraints of the visual features results in diminished precision of localization. Furthermore, if there is interference from dynamic objects in the environment currently, incorrect visual measurement information will be introduced. To tackle the issue, this research advances a method for visual-inertial navigation anchored in the motion feature constraints, which introduces a zero-velocity update factor into optimization without increasing the number of sensors to detect the zero-velocity state of the UAV and improve the localization accuracy. By designing UAV experiments, it is proved that the method outlined in this paper indeed improves the precision of positioning in dynamic scenes where the UAV’s trajectory appears a long-time hovering compared to VINS-Fusion.