This article presents an adaptive orbit control system for spacecraft around irregularly-shaped rotating asteroids. It is assumed that the asteroid’s mass and inertia matrix are not known. Unlike the published articles, an adaptive orbit control system is designed based on a logarithmic manifold. In the closed-loop system, first the spacecraft trajectory converges to this nonlinear manifold. Subsequently, it slides on this manifold towards the equilibrium point. In contrast to linear sliding surfaces, the motion on the logarithmic manifold is faster due to the high gain of the nonlinear function in the vicinity of the equilibrium point. Based on the Lyapunov theory, asymptotic stability of the closed-loop system and boundedness of the parameter error are established. Simulation results for 433 Eros and Ida asteroids are presented. These results show that the adaptive control law accomplishes precise reference orbit tracking as well as hovering control over the fixed equilibrium position in the vicinity of the asteroid, despite the uncertainties in the model parameters.