The tourism domain demands AI systems that can integrate rich travel knowledge with language understanding. Retrieval-Augmented Generation (RAG) has shown promise in adapting large language models to specialized domains, but travel applications in Vietnamese are largely unexplored. We present ViTravelRAG, a comprehensive framework tailored for Vietnamese travel tasks, including itinerary planning, destination QA, landmark identification. Our system ingests a multimodal travel corpus, text from Vietnamese travel blogs, guides, and booking sites, plus images of attractions, charts and builds a joint embedding index for retrieval. At query time, ViTravelRAG embeds user questions possibly with an image of a landmark or map and retrieves relevant documents and images, which are then fed to an language model to generate contextually grounded travel responses. To evaluate this approach, we curate a VietnamTravel-VQA Benchmark of 1,000 human-annotated query–document pairs in the tourism domain. Experimental results demonstrate that ViTravelRAG significantly outperforms the Gemini 2.5 Pro baselines, achieving an 8.6% gain in generation quality.