Despite significant progress in autonomous driving, detecting and segmenting obstacles under poor conditions remains a major challenge. This paper reviews deep learning models that tackle real-world difficulties like occlusion, fog, motion blur, and uneven road surfaces such as potholes and broken speed bumps—factors that heavily impact safety and detection accuracy. Over ten recent models are analyzed, including prompt-based approaches like Semantic-SAM and EPCFormer, memory-augmented ones like OOSIS and XMem, and task-specific detectors such as DR-YOLO, D-YOLO, and motion-aware YOLO variants. Each model type comes with trade-offs: prompt-based systems are flexible but depend on large vision-language datasets, while memory-based methods offer temporal consistency at the cost of increased computation. A key focus is on handling unstructured and uncertain road conditions, especially common in countries like India, where irregular infrastructure and unpredictable traffic are everyday challenges. Models trained solely on structured data often fail in these environments. To address this, the survey includes detailed comparisons and benchmark tests under difficult traffic and weather conditions. These insights inform the design of XenSense-V.1, a real-time deep learning framework using optical f low, temporal reasoning, and efficient segmentation to handle occlusion, weather issues, and complex road scenarios—particularly suited to Indian driving conditions.