While information retrieval (IR) systems have made significant strides in high-resource languages, their application to low-resource languages like Vietnamese remains a challenge. This is primarily due to data scarcity, limited parallel corpora, and the absence of standardised datasets. Vietnamese also presents unique difficulties, including tonal variation, compound word formations, and ambiguous segmentation. These challenges hinder the effectiveness of traditional retrieval approaches. In this work, we propose a retrieval framework tailored specifically for Vietnamese. The framework combines two key innovations: (1) a contextualised late-interaction retriever, which computes fine-grained token-level semantic similarity and improves retrieval accuracy through contrastive alignment with minimal bilingual data; and (2) a knowledge-distilled cross-encoder reranker, which transfers semantic knowledge from a larger teacher model to a smaller, more efficient student model, preserving accuracy while reducing inference time. This enables fast inference while maintaining strong semantic discrimination. We evaluate our system on multiple Vietnamese retrieval benchmarks spanning news, Wikipedia, and special domain (legal) documents. The framework consistently achieves high retrieval quality, with recall@100 exceeding 90% across all domains. These results demonstrate the effectiveness and robustness of our method in diverse settings. Our work addresses key limitations in low-resource similarity search by integrating efficient retrieval architectures with scalable training strategies. The proposed framework contributes a practical solution for real-world information retrieval in Vietnamese and offers transferable insights for other low-resource languages.