Optical navigation is currently the most widely used relative navigation method. Systematic errors, such as optical camera installation errors and image plane translation, affect the navigation accuracy of non-cooperative space targets. This paper uses only optical images to analyze the relative autonomous navigation system. Obviously, in this case, the weak observability of the system and the high computational complexity of the traditional augmented Kalman filter algorithm are the problems to be faced. A unified modelling and estimation method for systematic errors of image-based relative navigation systems is proposed to address the above problems. The relationship between the installation error, the image plane translation and the optical camera measurement is deduced, and a unified dimensionality-reduction model for the optical camera systematic errors is constructed. The augmented Kalman filtering algorithm, based on the unified model, is designed to achieve state estimation and error compensation. Simulation experiments confirm the effectiveness and feasibility of this method.