Recently, image compression methods based on rate distortion autoencoder (RDAE) have achieved advanced performance. However, these methods have high decoding complexity, which limits their application on low-power devices. To address this issue, Implicit Neural Representations (INR) represents images as neural networks that map coordinates to signal values and forms INR-based image compression method. Despite with low decoding complexity, there is a significant performance gap between INR-based approaches and RDAE-based approaches. In this paper, we propose an image compression method with hybrid neural representation (HNRC) to improve compression performance of INR-based approaches while keeping decoding lightweight. Specifically, we design a Groupwise Feature Aggregation module to aggregate feature of different groups, develop a Pointwise Local Modulation module to enhance the representation of local details, and employ a Gaussian Mixture Model to improve the accuracy of rate estimation. Extensive experiments demonstrate that our method achieves an approximate 1.1 dB improvement in terms of PNSR over INR-based approaches on the Kodak dataset while reducing decoding complexity by 88.9%.