Harnessing Deterministic Chaos for Adaptive Gradient Optimization
摘要
This paper systematically investigates deterministic chaos as a source of structured gradient perturbations for modern optimisers. Seven chaotic maps—including cubic, logistic, single- and multiparametric tent, Hénon, Ikeda and Baker—are projected to bounded, zero-mean signals and injected at principled loci within SGD, Adam and RMSProp. We log each optimiser’s internal state—momentum_buffer, first and second moments, and square_avg—to explain how chaos reshapes exploration and variance control. Experiments on a synthetic regression task show that cubic-map noise accelerates momentum build-up in SGD, Hénon noise yields faster damping of RMSProp’s variance estimate, and tent-map perturbations modulate Adam’s learning rate without destabilising convergence. Parameter sweeps reveal that small chaos scales can even reduce adaptive-variance inflation below the clean baseline. Our analysis highlights the importance of map choice, invariant-measure bias and injection locus, providing concrete design guidelines and opening a path toward chaos-aware optimisation on complex deep-learning benchmarks.