Learning and deploying inexpensive replacements for computationally expensive subroutines “on-the-fly” (OTF) during a dynamic simulation offers potential advantages and unique drawbacks. In OTF learning, a machine learned surrogate function is trained to replace a target subroutine in a simulation as the dynamics evolve. The advantages of OTF learning include reducing simulation error and model training costs, but the weaknesses include the possibility of introducing artifacts when adaptively updating the physics of the simulation over time. Here, we enhance an existing control system, Proxima, which ensures that surrogates are used appropriately in time-independent state-sampling-based simulations, to time-dependent dynamical simulations by introducing a “blending” procedure that hides discontinuities when transitioning between original subroutine and surrogate. Our new control system, Proxima+Blend, produces a blend of the surrogate and target functions according to the relationship between error and an uncertainty signal observed as the dynamic simulation evolves. We show that while the original control system can shorten application runtime and accurately capture some macro-scale observables of a molecular dynamics simulation, the addition of blending is necessary to avoid unphysical dynamics at shorter time and length scales and to correct observables derived from these dynamics. Proxima+Blend delivers a 1.5x speedup over use of the target subroutine while producing solutions within 5% error in dynamical quantities, while the original Proxima algorithm has up to 80% error. Our implementation of Proxima+Blend can be deployed by simply replacing the existing subroutine with a wrapper that includes a machine learning approach for surrogate training along with specifying control and uncertainty signals for the simulation of interest.