Sign up to save your podcasts
Or
Share EP173: AI models diagnosing diseases from blank scans
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
EP173: AI models diagnosing diseases from blank scans
Paper Link: https://arxiv.org/abs/2603.21687
Summary:
The paper "Mirage: The Illusion of Visual Understanding" explores a phenomenon called the mirage effect, where multimodal AI systems generate highly detailed descriptions and reasoning traces for images that were never actually provided. This behavior creates a "false epistemic frame," allowing models to simulate a perceptual process that isn't grounded in real visual input.
Key findings and contributions of the research include:
• High Mirage Scores: Frontier models (such as GPT-5, Gemini 3 Pro, and Claude Opus 4.5) retain 70–80% of their reported accuracy on standard visual benchmarks even when the images are removed. In one extreme case, a text-only "super-guesser" model outperformed both human radiologists and large multimodal models on a chest X-ray benchmark without ever seeing an image.
• Pathology Bias: In medical contexts, these "mirages" are not neutral; they are heavily biased toward pathology, with models frequently fabricating sensitive clinical findings like strokes, tumors, or fractures for non-existent images.
• Distinction from Guessing: When models are explicitly instructed to "guess" without an image, their performance declines, suggesting that the "mirage regime" allows them to exploit hidden textual cues and benchmark structures more effectively than simple deduction.
• B-Clean Framework: The authors introduce B-Clean, a principled evaluation method that identifies and removes compromised, vision-independent questions from benchmarks. Applying this method reduced some benchmarks by over 75%, revealing that original model rankings were often inflated by non-visual inference.
Ultimately, the paper argues that high benchmark performance is not a reliable indicator of genuine visual understanding and calls for more rigorous, vision-grounded evaluation standards to ensure safety in high-stakes deployments.
View all episodes
By Yun WuPaper Link: https://arxiv.org/abs/2603.21687
Summary:
The paper "Mirage: The Illusion of Visual Understanding" explores a phenomenon called the mirage effect, where multimodal AI systems generate highly detailed descriptions and reasoning traces for images that were never actually provided. This behavior creates a "false epistemic frame," allowing models to simulate a perceptual process that isn't grounded in real visual input.
Key findings and contributions of the research include:
• High Mirage Scores: Frontier models (such as GPT-5, Gemini 3 Pro, and Claude Opus 4.5) retain 70–80% of their reported accuracy on standard visual benchmarks even when the images are removed. In one extreme case, a text-only "super-guesser" model outperformed both human radiologists and large multimodal models on a chest X-ray benchmark without ever seeing an image.
• Pathology Bias: In medical contexts, these "mirages" are not neutral; they are heavily biased toward pathology, with models frequently fabricating sensitive clinical findings like strokes, tumors, or fractures for non-existent images.
• Distinction from Guessing: When models are explicitly instructed to "guess" without an image, their performance declines, suggesting that the "mirage regime" allows them to exploit hidden textual cues and benchmark structures more effectively than simple deduction.
• B-Clean Framework: The authors introduce B-Clean, a principled evaluation method that identifies and removes compromised, vision-independent questions from benchmarks. Applying this method reduced some benchmarks by over 75%, revealing that original model rankings were often inflated by non-visual inference.
Ultimately, the paper argues that high benchmark performance is not a reliable indicator of genuine visual understanding and calls for more rigorous, vision-grounded evaluation standards to ensure safety in high-stakes deployments.