VSLAM (Visual Simultaneous Localization And Mapping) systems have shown superior accuracy in GPS-denied indoor parking navigation. How-ever, existing VSLAM systems often face scale ambiguity issues in complex conditions, leading to potential localization drift during navigation. Although VI-SLAM (Visual-Inertial SLAM) systems have improved localization accuracy by additional inertial measurements, they still encounter challenges in system robustness and scale consistency, particularly when IMU measurements are un- available or its excitation is insufficient. Moreover, traditional SLAM systems typically rely on monocular or stereo cameras with limited FoV (Field of View), making them prone to unstable tracking in environments with sparse textures. Addressing these challenges, this work develops a tightly-coupled semantic SLAM system utilizing large-FOV fisheye cameras in a surround-view configuration, namely the VCASurroundSLAM system, in cases where an IMU is not incorporated. In VCASurroundSLAM, robust strategies for both initialization and optimization are proposed with the assistance of a virtual camera capturing view-independent ground semantics, parking-slot for instance. Specifically, the initialization module offers a generic solution for launching any surround-view SLAM system. By solving a surround-virtual motion alignment problem, it is the first of its kind to effectively overcome the scale ambiguity issue in scenarios lacking reliable IMU measurements. Additionally, the joint optimization framework resolves cumulative scale drift by fusing surround-view and scale-aware virtual constraints, achieving a 41.5% improvement in localization accuracy. Qualitative and quantitative evaluations show the superiority of our VCASurroundSLAM system for autonomous indoor parking.