End-to-end multimodal pathology foundation model with clinical dialogue

Key Points:

• PRISM2, a computational pathology foundation model trained to generate clinically relevant text, achieves diagnostic-grade performance on cancer detection and subtyping tasks without task-specific training, outperforming or matching specialist commercial models like Paige Prostate and Paige Breast.
• PRISM2 produces two types of embeddings—base and diagnostic—that generalize well to various downstream pathology tasks, including pan-cancer detection, tumor grading, and cancer subtyping, often outperforming other models such as PRISM, TITAN, COBRA, and Prov-GigaPath.
• The model’s survival embeddings, fine-tuned on a large dataset of over 225,000 cases, outperform a survival specialist model in predicting recurrence-free survival and disease-specific survival, demonstrating strong prognostic capabilities.
• PRISM2 base embeddings also generalize effectively to biomarker prediction tasks across multiple cancer types, achieving performance comparable to or better than other state-of-the-art models, suggesting potential to complement or replace genetic sequencing in some contexts.
• Beyond classification, PRISM2 enables prompt-based inference for detailed pathology report completion following CAP guidelines, showing promising capabilities in generating consistent, clinically relevant diagnostic text and supporting dialogue-based interactions without further training.