To capture and understand animal behavior, engineers and biologists seek to develop biologically accurate neuromechanical models of muscle dynamics and neural control. However, demand-driven enhancement of complex neuromechanics, such as the multifunctional Aplysia californica feeding apparatus, can be challenging due to the multidimensional biomechanical and neural models involved. We propose an analysis pipeline that enables reinforcement learning (RL) to classify which aspects of an engineered neuromechanical model can accurately capture animal behavior. As an example, prioritizing where demand-driven enhancement of a biomechanical and neural model is needed, the neural model of a published neuromechanical model of Aplysia swallowing during feeding was replaced with an RL controller and their performances were compared and correlated with in vivo swallowing behavior. By comparing the performance of the neural model and the learned model to in vivo animal behavior, we can pinpoint areas for improvement. The analysis pipeline identified that the neuromechanical model confidently captured force performance with no significant difference from animal swallowing force behavior. It most usefully also indicated that the biomechanical model will need to be improved in future iterations to better capture motor neuron activity. Future work should explore the accuracy of the RL-enabled analysis pipeline with a more advanced biomechanical model.