This paper proposes a model-free incremental adaptive optimal control scheme for spacecraft hovering over an asteroid. To leverage the robustness of incremental control, first, the nonlinear system is linearized near the state of the previous time step to get an incremental representation of the system. Subsequently, the relevant matrix parameters of the incremental representation are identified through the recursive least squares (RLS) algorithm. By constructing a quadratic cost function, an incremental adaptive dynamic programming (IADP) algorithm is utilized to design an approximate optimal control law. The algorithms for cost evaluation and policy improvement are proposed to solve the optimal problem, enabling spacecraft hovering control near an asteroid. Finally, numerical simulations demonstrate the effectiveness of the proposed method.