Predicting pedestrian trajectories in the future is a key task to enhance the decision-making and planning capabilities of automated vehicles. This is challenging because human motion is influenced by their unique movement pattern and complicated interactions. Previous data-driven methods have attempted to address this problem with massive data and complex models, but this high-cost approach has limited the application of pedestrian trajectory prediction in autonomous driving. In this work, we aimed to improve modeling efficiency and reduce computing consumption to bridge the gap between theoretical research and practical applications. First, a new trajectory prediction metric was proposed to well weigh the relationship between computational consumption and model performance. In addition, by using efficient graph attention mechanisms to understand the spatio-temporal information, spatial interactions and motion patterns of pedestrians were represented as implied vectors. Finally, a four-layer time-extrapolator convolution neural network was used to predict the future trajectories of pedestrians. Experimental results on two openly accessible pedestrian trajectory prediction datasets demonstrate competitive performance compared with state-of-the-art methods.