Fully Convolutional Network-Based Self-Supervised Learning for Semantic Segmentation

Zhengeng Yang, Hongshan Yu, Yong He, Wei Sun, Zhi Hong Mao, Ajmal Mian

Research output: Contribution to journalArticlepeer-review

13 Citations (Web of Science)

Abstract

Although deep learning has achieved great success in many computer vision tasks, its performance relies on the availability of large datasets with densely annotated samples. Such datasets are difficult and expensive to obtain. In this article, we focus on the problem of learning representation from unlabeled data for semantic segmentation. Inspired by two patch-based methods, we develop a novel self-supervised learning framework by formulating the jigsaw puzzle problem as a patch-wise classification problem and solving it with a fully convolutional network. By learning to solve a jigsaw puzzle comprising 25 patches and transferring the learned features to semantic segmentation task, we achieve a 5.8% point improvement on the Cityscapes dataset over the baseline model initialized from random values. It is noted that we use only about 1/6 training images of Cityscapes in our experiment, which is designed to imitate the real cases where fully annotated images are usually limited to a small number. We also show that our self-supervised learning method can be applied to different datasets and models. In particular, we achieved competitive performance with the state-of-the-art methods on the PASCAL VOC2012 dataset using significantly fewer time costs on pretraining.

Original languageEnglish
Article number3172423
Pages (from-to)132-142
Number of pages11
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume35
Issue number1
Early online date11 May 2022
DOIs
Publication statusPublished - 1 Jan 2024

Fingerprint

Dive into the research topics of 'Fully Convolutional Network-Based Self-Supervised Learning for Semantic Segmentation'. Together they form a unique fingerprint.

Cite this