Generating dermatopathology reports from gigapixel whole slide images with HistoGPT
- Manuel Tran
- Paul Schmidle
- Ruifeng Ray Guo
- Sophia J. Wagner
- Valentin Koch
- Valerio Lupperger
- Brenna Novotny
- Dennis H. Murphree
- Heather D. Hardway
- Marina D’Amato
- Judith Lefkes
- Daan J. Geijs
- Annette Feuchtinger
- Alexander Böhner
- Robert Kaczmarczyk
- Tilo Biedermann
- Avital L. Amir
- Antien L. Mooyaart
- Francesco Ciompi
- Geert Litjens
- Chen Wang
- Nneka I. Comfere
- Kilian Eyerich
- Stephan A. Braun
- Carsten Marr
- Tingying Peng
2025-05-27
Histopathology is the reference standard for diagnosing the presence and nature of many diseases, including cancer. However, analyzing tissue samples under a microscope and summarizing the findings in a comprehensive pathology report is time-consuming, labor-intensive, and non-standardized. To address this problem, we present HistoGPT, a vision language model that generates pathology reports from a patient’s multiple full-resolution histology images. It is trained on 15,129 whole slide images from 6705 dermatology patients with corresponding pathology reports. The generated reports match the quality of human-written reports for common and homogeneous malignancies, as confirmed by natural language processing metrics and domain expert analysis. We evaluate HistoGPT in an international, multi-center clinical study and show that it can accurately predict tumor subtypes, tumor thickness, and tumor margins in a zero-shot fashion. Our model demonstrates the potential of artificial intelligence to assist pathologists in evaluating, reporting, and understanding routine dermatopathology cases.