Projects per year
This paper proposes a new transformer-based framework to learn class-specific object localization maps as pseudo labels for weakly supervised semantic segmentation (WSSS). Inspired by the fact that the attended regions of the one-class token in the standard vision transformer can be leveraged to form a class-agnostic localization map, we investigate if the transformer model can also effectively capture class-specific attention for more discriminative object localization by learning multiple class tokens within the transformer. To this end, we propose a Multi-class Token Transformer, termed as MCTformer, which uses multiple class tokens to learn interactions between the class tokens and the patch tokens. The proposed MCTformer can successfully produce class-discriminative object localization maps from the class-to-patch attentions corresponding to different class tokens. We also propose to use a patchlevel pairwise affinity, which is extracted from the patchto-patch transformer attention, to further refine the localization maps. Moreover, the proposed framework is shown to fully complement the Class Activation Mapping (CAM) method, leading to remarkably superior WSSS results on the PASCAL VOC and MS COCO datasets. These results underline the importance of the class token for WSSS.(-1)
|Title of host publication||2022 IEEE/CVF Conference on computer vision and pattern recognition (CVPR 2022)|
|Place of Publication||USA|
|Publisher||IEEE, Institute of Electrical and Electronics Engineers|
|Number of pages||10|
|Publication status||Published - 2022|
|Event||IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) - New Orleans, Lao People's Democratic Republic|
Duration: 18 Jun 2022 → 24 Jun 2022
|Conference||IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)|
|Country/Territory||Lao People's Democratic Republic|
|Period||18/06/22 → 24/06/22|
FingerprintDive into the research topics of 'Multi-class Token Transformer for Weakly Supervised Semantic Segmentation'. Together they form a unique fingerprint.
- 2 Active
Intelligent Virtual Human Companions
Bennamoun, M., Laga, H. & Boussaid, F.
31/12/21 → 30/12/25
Fine-grained Human Action Recognition with Deep Graph Neural Networks
Wang, Z., Bennamoun, M., Hagenbuchner, M., Tsoi, A. C. & Lewis, S.
4/01/21 → 31/12/23