The utilization of frequency domain information, which offers a compact and compressed portrayal of inherent patterns, has recently made significant strides in time-series forecasting. However, real-world time series usually contain dynamic non-stationary characteristics which change over time. The dynamics make non-stationary time series prediction rather challenging for existing frequency-based methods, as they generally assume the frequency information within look-back window remains unchanged. In this article, we propose a novel Multi-Perspective Frequency Transformer (MPFT) to model input frequency variation for non-stationary time series forecasting. Instead of the whole input window, MPFT employs Fast Fourier Transform (FFT) on unequal-length patches to model frequency token shifts from multi-perspective views. To facilitate learning in the frequency domain, we propose a novel Real-Imaginary Exchange (RIE) structure that can learn both real and imaginary parts while balancing their association. A multi-head attention mechanism is used to enhance the learning of the correlation among different views. Extensive experimental results across seven real-world long-term time series datasets demonstrate the superiority of our method over other state-of-the-art methods, in terms of both forecasting performance and computational efficiency.