Implicit neural representations (INRs) have emerged as a powerful paradigm for continuous signal representation across 3D scenes, images, and videos. However, existing video INR methods struggle to achieve optimal performance due to their ineffective exploitation of spatio-temporal redundancies in video sequences. To address these challenges, we propose a novel neural video representation model E-NeRV. The motion-aided quality enhancement module based on deformable convolution is incorporated into our model to better utilize temporal dependence between video frames. Moreover, we introduce an efficient architecture based on channel attention mechanism, which strengthens the representation ability with minimal parameter cost. Experimental results demonstrate that E-NeRV shows superior video representation ability compared to previous video INR work, and our ablation studies further validate that all proposed components contribute significantly to the performance gains.