The non-periodicity and unpredictability of fractional-order memristor chaotic circuits make them have a broader application prospect in many fields compared to general chaotic circuits. The majority of prior research has concentrated on the utilization of chaotic circuits or integer-order memristor chaotic circuits for image encryption, with relatively few studies on generalized memristive chaotic systems of fractional order in image encryption. This paper adopts the hyperbolic sine nonlinear characteristic curve as the model of magnetic control memristor, and based on this, a new four-dimensional integer-order chaotic circuit is constructed. It can be extended to order in fractional using the Adomian decomposition approach, yielding a four-dimensional memristive chaotic system in fractional order. The application of the chaotic system in pixel encryption, encryption methods, and encryption effects are analyzed. The dynamical characteristics of this new memristor chaotic circuit are analyzed to conform to the characteristics of chaotic systems and be in a chaotic state using the phase trajectory method, bifurcation diagram, Lyapunov exponent method, and analysis of system dissipativity. In image encryption, the sensitivity of the key, histogram, and correlation of adjacent pixels are analyzed, showing that this new four-dimensional fractional-order memristor chaotic system can effectively enhance the security of image encryption.