Given the unique characteristics of USUV (Unmanned Surface and Underwater Vehicle) navigation, a low-cost multi-source information fusion navigation method is proposed. This method divides the USUV’s navigation state into two stages: passive and calibration, through the coordination of different navigation stages, the system’s navigation performance can be enhanced. It effectively utilizes information from various sources, including MIMU, DVL, Magnetometer, and GNSS, and employs the factor graph algorithm to achieve multi-source information fusion and positioning. Experimental results demonstrate that, with the combined use of DVL and Magnetometer, there is no significant difference in attitude and velocity errors for the USUV whether it is navigating on the water surface or underwater. However, regarding position errors, when the USUV is operating underwater, these errors accumulate over time. Once it transitions back to surface navigation, where GNSS is operational, position errors can be quickly calibrated. The method proposed in this paper can provide support for the USUV’s continuous operational capabilities on and under water.