This paper targets three persistent bottlenecks in LiDAR–Visual–Inertial (LVI) SLAM for real, complex environments—trajectory continuity, incremental mapping efficiency, and degeneracy-robust fusion—and presents a fast, direct framework addressing them in a unified manner. First, under an IMU-centric time base, non-uniform B-splines are introduced to model cross-modal time drift and local jitter, thereby restoring temporal consistency for LiDAR deskewing and photometric alignment and improving linearization fidelity. Second, an incremental voxel index is employed to confine point-to-plane assembly and local plane statistics to a constant neighborhood with amortized constant complexity; a unified voxel map jointly serves geometric and photometric queries, markedly reducing dual-stream data association overhead. Third, a condition-number–aware numerical scheme lifts the spectral floor and applies structured damping along weakly observable directions, suppressing oscillation and divergence in semi-degenerate regimes. The system adopts a sequential ESIKF for measurement-level tight coupling, augmented with online exposure-time estimation and on-demand voxel ray-casting to enhance convergence and visibility reuse under strong illumination variation and near-field blind zones. The design aligns with and substantiates the efficient direct-fusion paradigm exemplified by FAST-LIVO2. Extensive evaluations on public benchmarks- NTU VIRAL demonstrate absolute trajectory errors on par with state-of-the-art tightly coupled direct methods while significantly reducing end-to-end latency and tail jitter, validating the synergistic benefits of continuous-time priors, incremental voxel assembly, and mild, degeneracy-aware regularization in single-plane, low-texture, low-light, and low-parallax scenarios.