Vector retrieval technologies are essential for applications such as search engines and recommendation systems. However, with rapidly growing data volumes, traditional CPU/GPU-based approaches encounter significant “memory wall” and “power wall” bottlenecks, leading to high energy consumption and latency due to extensive data movement. Compute-in-Memory (CIM) has emerged as an effective solution by enabling computation directly within memory, efficiently performing critical operations like Matrix-Vector Multiplication (MVM). Nevertheless, current CIM chip manufacturing processes impose limitations on data precision, adversely affecting vector retrieval accuracy. To overcome this challenge, we introduce a novel Fixed-Point Splitting Quantization (FPSQ) algorithm tailored for CIM-based vector retrieval systems. Our approach strategically trades spatial resources for increased precision, converting full-precision data into finite integer representations using fixed-point splitting and quantization methods. This allows FPSQ to preserve fine-grained data characteristics more effectively, significantly enhancing retrieval accuracy without sacrificing CIM performance advantages. Experimental evaluations demonstrate that FPSQ achieves a 17%–21% accuracy improvement compared to baseline quantization techniques, highlighting its practical efficacy and potential for deployment in next-generation CIM-based retrieval architectures.