AI-SAM: Automatic and Interactive Segment Anything Model

Yimu Pan (1), Sitao Zhang (1), Alison D. Gernand (1), Jeffery A. Goldstein (2), and James Z. Wang (1)

(1) The Pennsylvania State University, University Park, Pennsylvania, USA
(2) Northwestern University, Chicago, Illinois, USA
Abstract:

Semantic segmentation is a core task in computer vision, traditionally approached as either automatic or interactive. Interactive approaches, exemplified by the Segment Anything Model, have shown promise as pre-trained models, but current adaptation strategies tend to favor either automatic or interactive methods. Interactive approaches rely on user prompts, whereas automatic methods bypass interactive promptability entirely. We introduce the Automatic and Interactive Segment Anything Model (AI-SAM), a novel paradigm that addresses these limitations. At its core is the Automatic and Interactive Prompter (AI-Prompter), which automatically generates initial prompts while allowing user input. AI-SAM achieves state-of-the-art performance in both medical and non-medical applications and offers flexibility to further enhance results through user interaction. Code is available at https://github.com/ymp5078/AI-SAM.

Full Paper
(PDF, 1.6MB)

NIH project information

More information


Citation: Yimu Pan, Sitao Zhang, Alison D. Gernand, Jeffery A. Goldstein and James Z. Wang, ``AI-SAM: Automatic and Interactive Segment Anything Model,'' Proceedings of the 9th International Workshop on Health Intelligence, in conjunction with the Annual AAAI Conference on Artificial Intelligence, pp. -, Philadelphia, Pennsylvania, March 2025.

© 2025 AAAI. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the AAAI.

Last Modified: February 10, 2025
© 2025