We present XROI-GS, an immersive XR framework enabling real-time exploration and interaction with complex real-world scenes. The system addresses the challenge of delivering photorealistic detail for user-specified objects within large-scale scenes while maintaining real-time XR performance. Our demonstration showcases a cultural heritage scene from the Hôtel de la Marine museum, where users can freely explore the scene space and naturally interact with historical artifacts using controller-free virtual hands. Our contribution is an automatic mechanism that significantly improves the visual quality of objects: when a user approaches or interacts with an artifact, the system enhances its resolution to increase rendering quality. In addition, we provide physically based manipulation for realistic and intuitive object handling. This framework democratizes access to fragile cultural treasures by enabling close inspection and interaction impossible in traditional museum settings, opening new possibilities for digital heritage preservation and public engagement.