Scholarlink

This paper presents a fully offline Retrieval-Augmented Generation (RAG) system designed for document-based question answering. Built with a user-friendly Streamlit interface and powered by local large language models (LLMs) via the Ollama framework, it eliminates the need for external APIs. Documents are parsed using PDF loaders and chunked recursively. Semantic embeddings are generated using the nomic-embed-text model. A Flashrank-based reranking component ensures high-relevance passage retrieval. Compared with cloud-based Gemini-powered RAG systems, the offline model provides superior contextual understanding and answer quality, making it ideal for privacy-focused, constrained environments. A comparison with cloud-based Gemini-powered RAG systems is conducted. The offline approach offers better contextual understanding. It also provides higher answer quality. These results highlight the potential of local LLMs. They are ideal for high-quality, privacy-focused applications in constrained environments.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Privacy-First AI: Building an Offline RAG System with Streamlit and Local LLMs

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Pratik Chakraborty,
  • Debabrata Ghosh,
  • Shivanshu Srivastav,
  • Sayan Hait,
  • Srijita Chakraborty

摘要

This paper presents a fully offline Retrieval-Augmented Generation (RAG) system designed for document-based question answering. Built with a user-friendly Streamlit interface and powered by local large language models (LLMs) via the Ollama framework, it eliminates the need for external APIs. Documents are parsed using PDF loaders and chunked recursively. Semantic embeddings are generated using the nomic-embed-text model. A Flashrank-based reranking component ensures high-relevance passage retrieval. Compared with cloud-based Gemini-powered RAG systems, the offline model provides superior contextual understanding and answer quality, making it ideal for privacy-focused, constrained environments. A comparison with cloud-based Gemini-powered RAG systems is conducted. The offline approach offers better contextual understanding. It also provides higher answer quality. These results highlight the potential of local LLMs. They are ideal for high-quality, privacy-focused applications in constrained environments.