Fusing LiDAR and camera data for 3D object detection remains a key challenge in autonomous driving. While most methods adopt dual-branch frameworks to extract BEV features from both modalities before fusion, progress in point cloud feature extraction still lags behind that of image-based networks, limiting overall fusion effectiveness. To address this gap, we propose BEV-SI, a novel multi-modal detection framework featuring a lightweight yet expressive LiDAR branch. At its core is the Split-Inception Block, which enhances point cloud representation by applying diverse channel-wise operations and expanding the receptive field. Furthermore, we introduce the Split-Neck module, which performs efficient multi-scale feature fusion through adaptive downsampling and Branch Attention, allowing the network to dynamically reweight spatial features across different scales. Extensive experiments on the nuScenes benchmark demonstrate that BEV-SI achieves competitive accuracy with significantly improved inference speed.