Digital Images with a substantial amount of missing pixels, commonly referred to as sparse images, are frequently encountered in real-world applications. For achieving missing pixel estimation, the factorization of tensor based methods are used to reconstruct sparse digital images. However, these typical estimation methods often suffer from low accuracy and color distortion. To address this issue, we propose a novel missing pixel estimation method based on latent representation network combined with adaptive mean filter, which is characterized by the following properties: 1) Leveraging CP decomposition, we design a novel instance-frequency attention-based latent representation network to reconstruct sparse images. 2) With considering of the imbalance missing rates of different channels, an instance frequency attention mechanism is established to account for such imbalance, thereby improving color preservation. 3) An adaptive mean filter is employed to remove the noise arising from unstable parameter optimization in network training, thereby enhancing the texture details of the reconstructed images. Experimental results on three real-world datasets demonstrate that our proposed method achieves better performance in terms of estimation accuracy and computational efficiency.