Projects per year
Abstract
Most existing weakly supervised semantic segmentation (WSSS) methods rely on class activation mapping (CAM) to extract coarse class-specific localization maps using image-level labels. Prior works have commonly used an off-line heuristic thresholding process that combines the CAM maps with off-the-shelf saliency maps produced by a general pretrained saliency model to produce more accurate pseudo-segmentation labels. We propose AuxSegNet<inline-formula> <tex-math notation="LaTeX">$+$</tex-math> </inline-formula>, a weakly supervised auxiliary learning framework to explore the rich information from these saliency maps and the significant intertask correlation between saliency detection and semantic segmentation. In the proposed AuxSegNet<inline-formula> <tex-math notation="LaTeX">$+$</tex-math> </inline-formula>, saliency detection and multilabel image classification are used as auxiliary tasks to improve the primary task of semantic segmentation with only image-level ground-truth labels. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps. In particular, we propose a cross-task dual-affinity learning module to learn both pairwise and unary affinities, which are used to enhance the task-specific features and predictions by aggregating both query-dependent and query-independent global context for both saliency detection and semantic segmentation. The learned cross-task pairwise affinity can also be used to refine and propagate CAM maps to provide better pseudo labels for both tasks. Iterative improvement of segmentation performance is enabled by cross-task affinity learning and pseudo-label updating. Extensive experiments demonstrate the effectiveness of the proposed approach with new state-of-the-art WSSS results on the challenging PASCAL VOC and MS COCO benchmarks.
Original language | English |
---|---|
Pages (from-to) | 1-15 |
Number of pages | 15 |
Journal | IEEE Transactions on Neural Networks and Learning Systems |
DOIs | |
Publication status | E-pub ahead of print - 13 Mar 2024 |
Fingerprint
Dive into the research topics of 'Auxiliary Tasks Enhanced Dual-Affinity Learning for Weakly Supervised Semantic Segmentation'. Together they form a unique fingerprint.-
Intelligent Virtual Human Companions
Bennamoun, M. (Investigator 01), Laga, H. (Investigator 02) & Boussaid, F. (Investigator 03)
ARC Australian Research Council
31/12/21 → 30/12/25
Project: Research
-
Fine-grained Human Action Recognition with Deep Graph Neural Networks
Wang, Z. (Investigator 01), Bennamoun, M. (Investigator 02), Hagenbuchner, M. (Investigator 03), Tsoi, A. C. (Investigator 04) & Lewis, S. (Investigator 05)
ARC Australian Research Council
4/01/21 → 31/12/24
Project: Research