Interpretable AI for Radiology

Interpretable AI in radiology requires more than accurate predictions. In chest x-ray analysis, models should also indicate whether they are relying on the same kinds of evidence that radiologists use and whether their decisions are grounded in anatomically meaningful image patterns. Our work addresses this through two complementary directions: radiologist eye tracking and counterfactual image generation. Eye-tracking data collected during routine report dictation provide scalable, label-specific localization signals that help train models to highlight abnormalities more faithfully, while counterfactual methods generate disease-related image changes that make model evidence easier to inspect than standard saliency maps alone. Together, these approaches aim to make radiology AI more transparent, clinically grounded, and better suited for trust, validation, and bias discovery.

Technically, we combine weak supervision from human behavior with generative models of disease evidence. In one line of work, gaze traces are aligned with timestamped report dictation to extract label-specific eye-tracking maps, which are then used to supervise localization-aware chest x-ray models without hurting classification accuracy. In another, adversarial generative models create counterfactual transformations that reveal what image changes correspond to differences in disease presence or severity, either through additive effect maps or deformation fields that better capture anatomical shape change. This project focuses on radiology, but the broader interpretability theme also extends to pathology, where related work such as HistoEM ranks and visualizes discriminative gland-level features to relate predictions back to tissue morphology.