Discriminative feature learning and region consistency activation for robust scene labeling

Yandong Li, Ferdous Sohel, Mohammed Bennamoun, Hang Lei

Research output: Contribution to journalArticle

Abstract

This paper presents a learned feature based framework for both outdoor and indoor scene labeling. This framework is combined with a discriminative feature learning process to produce the posteriors of every pixel and a novel strategy to improve the global label consistency of a scene. First, we use Convolutional Neural Networks (ConvNets) to learn the most relevant features of a scene at the multi-scale superpixel level. The effect of both trained and general ConvNets features for our scene labeling framework are investigated. Then, based on the predicted posteriors from the learned features, we propose an algorithm called Region Consistency Activation (RCA) to iteratively improve the global label consistency at different levels of the Ultrametric Contour Map (UCM). In addition, we propose a strategy to make the hyper-parameters of RCA adaptive to the test images, which results in a better generalization ability compared with the hyper-parameters tuning based RCA. Our scene labeling framework were rigorously tested on three popular scene labeling datasets: Stanford Background, SIFT Flow and NYU-Depth V2. Experiments show that our proposed method consistently produces better accuracy and visual consistency compared with the state-of-the-art methods for both outdoor and indoor scenes.

Original languageEnglish
Pages (from-to)174-186
Number of pages13
JournalNeurocomputing
Volume243
DOIs
Publication statusPublished - 21 Jun 2017

Fingerprint

Labeling
Chemical activation
Learning
Aptitude
Labels
Neural networks
Tuning
Pixels
Experiments
Datasets

Cite this

@article{75eeade04dc548b0a31a19fb4da5fe79,
title = "Discriminative feature learning and region consistency activation for robust scene labeling",
abstract = "This paper presents a learned feature based framework for both outdoor and indoor scene labeling. This framework is combined with a discriminative feature learning process to produce the posteriors of every pixel and a novel strategy to improve the global label consistency of a scene. First, we use Convolutional Neural Networks (ConvNets) to learn the most relevant features of a scene at the multi-scale superpixel level. The effect of both trained and general ConvNets features for our scene labeling framework are investigated. Then, based on the predicted posteriors from the learned features, we propose an algorithm called Region Consistency Activation (RCA) to iteratively improve the global label consistency at different levels of the Ultrametric Contour Map (UCM). In addition, we propose a strategy to make the hyper-parameters of RCA adaptive to the test images, which results in a better generalization ability compared with the hyper-parameters tuning based RCA. Our scene labeling framework were rigorously tested on three popular scene labeling datasets: Stanford Background, SIFT Flow and NYU-Depth V2. Experiments show that our proposed method consistently produces better accuracy and visual consistency compared with the state-of-the-art methods for both outdoor and indoor scenes.",
keywords = "Convolutional Neural Networks, Feature learning, Scene labelling",
author = "Yandong Li and Ferdous Sohel and Mohammed Bennamoun and Hang Lei",
year = "2017",
month = "6",
day = "21",
doi = "10.1016/j.neucom.2017.03.019",
language = "English",
volume = "243",
pages = "174--186",
journal = "Neurocomputing",
issn = "0925-2312",
publisher = "Pergamon",

}

Discriminative feature learning and region consistency activation for robust scene labeling. / Li, Yandong; Sohel, Ferdous; Bennamoun, Mohammed; Lei, Hang.

In: Neurocomputing, Vol. 243, 21.06.2017, p. 174-186.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Discriminative feature learning and region consistency activation for robust scene labeling

AU - Li, Yandong

AU - Sohel, Ferdous

AU - Bennamoun, Mohammed

AU - Lei, Hang

PY - 2017/6/21

Y1 - 2017/6/21

N2 - This paper presents a learned feature based framework for both outdoor and indoor scene labeling. This framework is combined with a discriminative feature learning process to produce the posteriors of every pixel and a novel strategy to improve the global label consistency of a scene. First, we use Convolutional Neural Networks (ConvNets) to learn the most relevant features of a scene at the multi-scale superpixel level. The effect of both trained and general ConvNets features for our scene labeling framework are investigated. Then, based on the predicted posteriors from the learned features, we propose an algorithm called Region Consistency Activation (RCA) to iteratively improve the global label consistency at different levels of the Ultrametric Contour Map (UCM). In addition, we propose a strategy to make the hyper-parameters of RCA adaptive to the test images, which results in a better generalization ability compared with the hyper-parameters tuning based RCA. Our scene labeling framework were rigorously tested on three popular scene labeling datasets: Stanford Background, SIFT Flow and NYU-Depth V2. Experiments show that our proposed method consistently produces better accuracy and visual consistency compared with the state-of-the-art methods for both outdoor and indoor scenes.

AB - This paper presents a learned feature based framework for both outdoor and indoor scene labeling. This framework is combined with a discriminative feature learning process to produce the posteriors of every pixel and a novel strategy to improve the global label consistency of a scene. First, we use Convolutional Neural Networks (ConvNets) to learn the most relevant features of a scene at the multi-scale superpixel level. The effect of both trained and general ConvNets features for our scene labeling framework are investigated. Then, based on the predicted posteriors from the learned features, we propose an algorithm called Region Consistency Activation (RCA) to iteratively improve the global label consistency at different levels of the Ultrametric Contour Map (UCM). In addition, we propose a strategy to make the hyper-parameters of RCA adaptive to the test images, which results in a better generalization ability compared with the hyper-parameters tuning based RCA. Our scene labeling framework were rigorously tested on three popular scene labeling datasets: Stanford Background, SIFT Flow and NYU-Depth V2. Experiments show that our proposed method consistently produces better accuracy and visual consistency compared with the state-of-the-art methods for both outdoor and indoor scenes.

KW - Convolutional Neural Networks

KW - Feature learning

KW - Scene labelling

UR - http://www.scopus.com/inward/record.url?scp=85015844366&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2017.03.019

DO - 10.1016/j.neucom.2017.03.019

M3 - Article

VL - 243

SP - 174

EP - 186

JO - Neurocomputing

JF - Neurocomputing

SN - 0925-2312

ER -