Vision-Language Contrastive Learning Approach to Robust Automatic Placenta Analysis Using Photographic Images

Yimu Pan (1), Alison D. Gernand (1), Jeffery A. Goldstein (2), Leena Mithal (3), Delia Mwinyelle (4), and James Z. Wang (1)

(1) The Pennsylvania State University, University Park, Pennsylvania, USA
(2) Northwestern University, Chicago, Illinois, USA
(3) Lurie Children’s Hospital, Illinois, USA
(4) The University of Chicago, Illinois, USA
Abstract:

The standard placental examination helps identify adverse pregnancy outcomes but is not scalable since it requires hospital-level equipment and expert knowledge. Although the current supervised learning approaches in automatic placenta analysis improved the scalability, those approaches fall short on robustness and generalizability due to the scarcity of labeled training images. In this paper, we propose to use the vision-language contrastive learning (VLC) approach to address the data scarcity problem by incorporating the abundant pathology reports into the training data. Moreover, we address the feature suppression problem in the current VLC approaches to improve generalizability and robustness. The improvements enable us to use a shared image encoder across tasks to boost efficiency. Overall, our approach outperforms the strong baselines for fetal/maternal inflammatory response (FIR/MIR), chorioamnionitis, and sepsis risk classification tasks using the images from a professional photography instrument at the Northwestern Memorial Hospital; it also achieves the highest inference robustness to iPad images for MIR and chorioamnionitis risk classification tasks. It is the first approach to show robustness to placenta images from a mobile platform that is accessible to low-resource communities.

Full Paper
(PDF, 0.9MB)

More information


Citation: Yimu Pan, Alison D. Gernand, Jeffery A. Goldstein, Leena Mithal, Delia Mwinyelle and James Z. Wang, ``Vision-Language Contrastive Learning Approach to Robust Automatic Placenta Analysis Using Photographic Images,'' Proceedings of the International Conference on Medical Image Computing and Computer Assisted Interventions, Lecture Notes in Computer Science, vol. ??, ?? et al. (eds.), pp. ???-???, Singapore, September 2022.

© 2022 MICCAI. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the MICCAI.

Last Modified: September 14, 2022
© 2022