Mobile devices are integral to daily life, necessitating secure authentication methods like speaker verification for enhanced security and convenience. While deep neural networks have improved speaker verification performance, deploying these models on resource-constrained devices with low latency remains challenging. We address these issues by applying knowledge distillation to compress a speaker verification model for mobile deployment. Additionally, systems must handle long non-speech segments in audio streams. Therefore, we propose a voice activity detection (VAD) model to resilient the silence segments in real-world scenarios. Experiments on Vietnamese voice datasets show that the MobileNetV2 architecture with relational knowledge distillation achieves competitive performance, reducing model parameters by 3x compared to the original. The lightweight model, when deployed on a mid-range smartphone, achieves a latency of 28.5ms, 25x faster than the uncompressed model. Furthermore, the lightweight VAD model enhances verification performance in real-time on mobile devices, demonstrating the practical viability and effectiveness of our approach in various environmental conditions, especially for Vietnamese speech.