Machine Learning (ML) models often rely on datasets, but acquiring real-world data can be impractical due to ethical, safety, or cost constraints. Simulators have emerged as viable alternatives for generating controlled, high-quality training data, particularly in high-risk applications such as autonomous driving, robotic surgery, and emergency response. In this context, virtual reality (VR) presents a promising medium for training AI systems, especially when dealing with human-computer interaction (HCI) scenarios that require physiological signal interpretation. This paper introduces Project Xavier XR, a VR-based system integrating eye-tracking and EEG for multimodal interaction. Originally conceived as a control interface for motorized wheelchairs, the project evolved into a VR simulator designed to collect high-quality physiological data for training AI guidance systems, particularly in augmented reality (AR) applications. The system was built using the HTC Vive Pro Eye for eye-tracking and the EMOTIV EPOC Flex for EEG signal acquisition, with Unity serving as the development platform. Throughout development, several technical and methodological challenges were encountered, including SDK integration issues, limitations of consumer-grade EEG devices, gaze-based UI design trade-offs, and the impact of system latency on interaction quality. User testing revealed critical insights into interface usability, the effectiveness of gaze and neural command-based interactions, and the constraints of current hardware solutions. This paper documents the development process, discusses key lessons learned, and outlines future improvements, including enhanced multimodal feedback mechanisms, better hardware synchronization, and more robust machine learning training approaches. The findings serve as both a technical roadmap and a cautionary guide for researchers and developers aiming to integrate physiological signals into VR and AR applications.