Vision Language Models (VLMs) have transformed the field of computer vision and natural language processing, playing a vital role in tasks such as image captioning, visual question answering, and autonomous navigation. However, these advanced models require substantial computational resources, which poses challenges for deployment in resource-constrained environments. To address this, our research introduces a hybrid and partial quantization approach designed to enhance efficiency while minimizing performance degradation. Using entropy-based layer partitioning, our method retains FLOAT-16 precision for high-entropy layers and applies INT-8 quantization to low-entropy layers, thereby reducing memory usage and computational cost. The experimental results demonstrate a 2.3 \(\times \) reduction in model size and a 1.35 \(\times \) increase in inference speed, making it suitable for real-time applications in autonomous systems, medical imaging, and interactive AI. Despite a 10.4% drop in accuracy, this trade-off enables effective deployment on edge devices, advancing the practical usability of large-scale VLMs such as Florence-2.