In recent years, visual-inertial odometry (VIO) has been widely applied in localization, navigation, and autonomous driving due to its lower hardware cost and robust feature coupling mechanism. However, deep learning-based VIO methods still face challenges such as large parameter sizes and high computational cost. In this paper, we propose a fast VIO method that discards heavy FlowNet encoder and adopts a lightweight optical flow encoder as the visual frontend, which use concise head feature pooling pyramid and efficient correlation computation layers. Furthermore, an adaptive feature selection gate is introduced to reduce computational redundancy by dynamically disabling unstable visual modalities. Lastly, we propose a multi-state pose joint loss function based on geometric consistency to reduce multi-frame cumulative pose estimation biases. The experiment results on the KITTI Odometry dataset demonstrate that our method achieves a 57.8% reduction in the number of parameters and a 78.3% reduction in computational complexity, all while maintaining outstanding performance. Compared to existing VIO methods, our method has higher computational efficiency and accuracy.