Scholarlink

We present SigLIP-Gemma-2.4B, a lightweight multimodal model that fuses a frozen SigLIP-400M vision encoder with a 2.4B-parameter Gemma decoder-only LLM. The model integrates visual features via a learned projection, enabling joint vision-language understanding. Without extensive training, SigLIP-Gemma-2.4B achieves strong performance on image captioning (CIDEr 141.9) and visual question answering (VQA 83.19%), despite its compact size. We further evaluate the model in retrieval-augmented generation and agent reasoning tasks. Our approach offers a practical and efficient path toward accessible multimodal AI systems.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Lightweight Multimodal Vision-Language Model with a Frozen Vision Encoder and 2.4B-Parameter LLM SigLIP-Gemma-2.4B

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Sandip Shinde,
  • Arya Pathak,
  • Shreya Mohite,
  • Sanchitsai Nipanikar,
  • Keyur Pande

摘要

We present SigLIP-Gemma-2.4B, a lightweight multimodal model that fuses a frozen SigLIP-400M vision encoder with a 2.4B-parameter Gemma decoder-only LLM. The model integrates visual features via a learned projection, enabling joint vision-language understanding. Without extensive training, SigLIP-Gemma-2.4B achieves strong performance on image captioning (CIDEr 141.9) and visual question answering (VQA 83.19%), despite its compact size. We further evaluate the model in retrieval-augmented generation and agent reasoning tasks. Our approach offers a practical and efficient path toward accessible multimodal AI systems.