Amphibious climbing robots have the potential to replace manual labor in repairing hull surfaces, significantly enhancing safety standards. However, achieving amphibious cross-media localization poses a key challenge for these robots when navigating ship surfaces. Therefore, this paper proposes a cross-media localization method for amphibious climbing robots, leveraging pseudo-depth maps and visual-inertial odometry (VIO) with a binocular camera and an IMU. By exploiting disparities in pseudo-depth, a classification model is created to differentiate between above-water and underwater RGB images. Additionally, utilizing a refraction model, underwater RGB images are corrected alongside left and right images. Subsequently, based on binocular imaging principles, underwater depth maps are generated to facilitate underwater localization. These components, including the image classifier and corrected RGB and depth images, are integrated into the visual-inertial odometry framework. Furthermore, feature points with insufficient tracking times are filtered to mitigate large odometry drift caused by water fluctuations. Validation of the proposed amphibious cross-media localization method was conducted through experiments in a 3 m × 1.5 m × 1 m pool using a handheld camera test stand, yielding a maximum trajectory error percentage of 1.22%.