<p>For solving the optimal control problem of nonlinear systems, conventional adaptive dynamic programming (ADP) methods always exploit neural networks to approximate the nonlinear functions in the Hamilton-Jacobi-Bellman (HJB) equation. However, the selection of basis functions in neural networks remains a challenging task, especially when the model dynamics are unknown. This article develops a novel data-driven ADP scheme for the optimal control problem of nonlinear polynomial systems with totally unknown dynamics. We utilize the Carleman operator to lift the nonlinear model into an infinite-dimensional bilinear system and the original optimal control problem is transformed into solving the infinite-dimensional Riccati equation. Then, by using state and input measurements, we present a value iteration-based algorithm to learn the near-optimal control protocol which comes from a finite-dimensional truncated Riccati equation. Besides, we prove that, under routine conditions, the obtained controller stabilizes the closed-loop system and its robustness property is analyzed. It should be noted that the proposed methodology significantly circumvents the issue of specifying basis functions in the existing ADP-based optimal control schemes and gives rise to computational improvement. Finally, three application examples are provided to demonstrate the feasibility of our method.</p>