Conformal Prediction in the Age of Multimodal Foundation Models: A Survey
摘要
Foundation Models and their multimodal extensions have transformed AI, enabling flexible reasoning and generation across diverse modalities such as text, vision, audio, and video. However, their black-box nature, large output spaces, and susceptibility to hallucinations raise concerns about reliability, particularly in high-stakes applications. Conformal Prediction offers a principled, distribution-free framework for uncertainty quantification, but its traditional formulations are challenged by the complexity of FM pipelines, multimodal interactions, and streaming or partially supervised scenarios. This chapter provides a detailed survey of recent efforts to adapt CP to deep neural networks and multimodal foundation models. We review innovations spanning generative modeling, predictive pipelines, reasoning and control, and retrieval-augmented generation, highlighting techniques such as sampling-aware calibration, trust-score conditioning and input-adaptive error control. The chapter also synthesizes design principles for robust conformal prediction in multimodal contexts, including per-modality normalization, dependency-aware calibration, and representative evaluation strategies. We conclude by outlining open research directions that can shape reliable and interpretable uncertainty quantification for the next generation of foundation models.