Visually impaired individuals face significant challenges in everyday tasks, while traditional aids like white canes and guide dogs offer essential support but lack adaptability to complex environments. Advancements in computer vision, machine learning, and mobile technologies offer new possibilities for assistive tools that enhance autonomy. Smartphone-based solutions, particularly those leveraging depth sensors like LiDAR, provide a promising approach by integrating real-time spatial awareness and object recognition. This study explores the design considerations and evaluation of an AI-driven assistive application. As a case study, we present the LetSee mobile app, which utilizes the phone’s sensors and can deliver multimodal feedback through audio and haptic cues, complementing the user’s sensory inputs without overriding them. We present the design methodology, which incorporates computational efficiency, real-time responsiveness, and ease of use to ensure seamless integration into daily activities. The functions of the LetSee app do not require network connection; they rely solely on on-device computation. Evaluation of the system was conducted through Cybathlon 2024’s Vision Assistance Race, benchmarking performance against real-world-inspired tasks. Results demonstrate the app’s effectiveness in aiding users across a variety of tasks and environments. This research underscores the importance of user-centered design in advancing assistive technology, integrating cutting-edge approaches with practical usability.