<p>Existing implicit neural representation (INR) methods typically aim to compress individual videos by eliminating redundancy among images. However, as 3D vision technologies has driven the demand for multi-view immersive videos, existing INR frameworks struggle to handle complex geometric deformations across different view effectively in a single unified compression model. Two core limitations emerge: 1) Duplicated parameters across views not only waste memory but prevent the learning of unified scene understanding; 2) The lack of a shared 3D representation compels the model to reduce inherently 3D spatial relationships to 2D approximations, weakening cross-view geometric correlations. In contrast, this paper introduces a spatially unified neural representation M-NeRV that models entire view scenes as fundamental units, which exploits richer 3D spatial information in high-dimensional variations of the multi-view scene to enhance representation learning. Specifically, the proposed M-NeRV first constructs a global point cloud and encodes it into a contextual space as the 3D structure condition. Guided by the condition, a Dynamic Grid Adaptation (DGA) module eliminates parametric duplication in view-specific embeddings by dynamically generating features conditioned on localized 3D scene primitives, maintaining coherent 3D geometric relationships across multiple viewpoints. Besides, we introduce a 3D Spatial-Consistency Aggregation (3D-SCA) module that utilizes 3D structural relationships in real-world scenes rather than 2D feature matching to maintain the semantic coherence and enhance detailed geometries of the scene, which preserves more structural details in decoded images. Experimental results demonstrate that our method achieves up to 3.00 BD-PSNR improvement compared to the latest MPEG immersive video standards, delivering state-of-the-art coding performance.</p>