A2LC: Active and Automated Label Correction for Semantic Segmentation

Youjin Jeon*, Kyusik Cho*, Suhan Woo, Euntai Kim†
Yonsei University
Paper arXiv Code

Abstract

Active Label Correction (ALC) has emerged as a promising solution to the high cost and error-prone nature of manual pixel-wise annotation in semantic segmentation, by actively identifying and correcting mislabeled data. Although recent work has improved correction efficiency by generating pseudo-labels using foundation models, substantial inefficiencies still remain. In this paper, we introduce A²LC, an Active and Automated Label Correction framework for semantic segmentation, where manual and automatic correction stages operate in a cascaded manner. Specifically, the automatic correction stage leverages human feedback to extend label corrections beyond the queried samples, thereby maximizing cost efficiency. In addition, we introduce an adaptively balanced acquisition function that emphasizes underrepresented tail classes, working in strong synergy with the automatic correction stage. Extensive experiments on Cityscapes and PASCAL VOC 2012 demonstrate that A²LC significantly outperforms previous state-of-the-art methods. Notably, A²LC exhibits high efficiency by outperforming previous methods with only 20% of their budget, and shows strong effectiveness by achieving a 27.23% performance gain under the same budget on Cityscapes.

A2LC Framework

A2LC is a semi-automated label correction framework for semantic segmentation, built on cascading stages of manual and automatic correction. The figure below illustrates the overview of A²LC framework.

A2LC framework

We execute Grounded SAM on unlabeled images to generate initial pseudo-labels. For each of the R rounds, the model trained with pseudo-labels selects top-B masks via the ABC acquisition function to query for manual correction. Then, the model in Label Correction Module (LCM) is trained using the queried masks and corrects the labels of unqueried masks. After both manual and automatic corrections, the pseudo-labels and model are updated, completing a single round of the correction cycle.

Label Correction Module

Label Correction Module (LCM) performs automatic correction by propagating human-corrected labels beyond the queried samples. LCM operates in two steps: first, the model is trained using masks queried in the current round, and second, it corrects masks that may be mislabeled but still remain unqueried. The figure below illustrates the LCM pipeline.

Label Correction Module (LCM)

Here, a traffic light (orange) mask, mislabeled as a traffic sign (yellow), was corrected by the annotator. This manually corrected mask is then used to train the model, enabling automatic refinement of similar cases. As a result, three additional similar masks were automatically corrected.

Adaptively Balanced Confidence in Label

Adaptively Balanced Confidence in Label (ABC) guides both correction stages toward class-balanced correction by incorporating pixel-wise adaptive class weight into the acquisition function. Adaptive class weight consists of two components: (1) class rarity score, that prioritizes tail class pixels during sampling, and (2) dataset imbalance score, that adjusts this weighting based on label statistics. The figure below illustrates the class distribution of sampled data.

ABC acquisition function

The x-axis shows classes sorted by pseudo-label frequency, and the y-axis shows the number of sampled masks. Unlike the baseline, which largely concentrates on head classes, our proposed ABC acquisition function substantially increases the sampling of tail classes.

BibTeX

@misc{jeon2025a2lcactiveautomatedlabel,
      title={A$^2$LC: Active and Automated Label Correction for Semantic Segmentation}, 
      author={Youjin Jeon and Kyusik Cho and Suhan Woo and Euntai Kim},
      year={2025},
      eprint={2506.11599},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.11599}, 
}