By now it has been a while that long short-term memoryLong short-term memory neural networks [6, 9] have been successfully used to process time-varying sequences [5]. While they have been very useful in implementations their properties as discrete-time dynamical systems expressed in terms of difference equations are still quite a bit of a mystery. One may stipulate that this is, at least partially, due to the use of specific sigmoid functions and their concatenations with multiple weight matrices and biases, as they appear in the long short-term memory neural network dynamical model. This model carries some inherent difficulties where even most basic questions like the existence and the number of equilibria, are complex to answer due to the transcendental nature of the corresponding fixed-point algebraic equation, which they are solutions to. Therefore, the heuristic implementations of the long short-term memory neural networks have been successful yet there is a lack of technical analysis for their complex dynamic behaviors. In this chapter we recall the analysis provided in [15] and we continue in the same direction of finding connections between the nonlinear autonomous long short-term memory neural network dynamical model and its linearization. More sophisticated simulations are provided to support the idea that these two discrete-time dynamical systems behave similarly in terms of the equilibria stability, yet unlike in [15], we were able to determine some discrepancies, at least in simulations.