Toward Robust In-Context Segmentation via Concept Guidance

In-context segmentation asks a model to identify target regions in new images using only a handful of labeled reference examples — no retraining required. Current approaches work by matching low-level visual features between references and queries, making them brittle when references vary in viewpoint, lighting, or appearance. CG-ICS instead extracts high-level semantic concepts from references using a multimodal language model, then uses these concepts alongside a spatial grounding route to guide a frozen SAM3 segmentation backbone. It achieves state-of-the-art accuracy and substantially reduced variance across diverse reference choices. Applications span medical image annotation, few-shot industrial inspection, and rapid domain adaptation in computer vision pipelines with limited labeled data. Authors: Zhigang Chen, Xiawu Zheng, Rongrong Ji Paper: https://arxiv.org/abs/2606.28149v1