<p>Online optimal control is essential for improving the autonomy and reliability of dynamical systems. Recently, data-driven supervised learning has emerged as a lightweight paradigm for online optimal control, where control policies are learned offline and executed online through efficient neural network inference. Nevertheless, for infinite-time optimal control problems, optimality is characterized by asymptotic convergence rather than explicit terminal conditions, rendering existing dataset generation methods fundamentally infeasible or highly inefficient. This paper aims to enhance the applicability of supervised learning to infinite-time nonlinear optimal control. We propose a comprehensive framework to enhance supervised learning for infinite-time nonlinear optimal control. First, by decomposing the infinite-time problem, we extend the Backward Generation of Optimal Examples (BGOE) method to efficiently generate optimal solutions. Second, a state-transition-matrix-guided data generation strategy is proposed to achieve uniform coverage of a specified state region. Third, a Lyapunov-stable neural network architecture is incorporated to guarantee the local stability of the learned policy. Simulations on three nonlinear systems demonstrate that our approach achieves an advantageous balance, improving computational efficiency while preserving near-optimal control performance. The source code is available at https://github.com/wong-han/PaperNORC.</p>