Discriminative feature learning and region consistency activation for robust scene labeling

This paper presents a learned feature based framework for both outdoor and indoor scene labeling. This framework is combined with a discriminative feature learning process to produce the posteriors of every pixel and a novel strategy to improve the global label consistency of a scene. First, we use Convolutional Neural Networks (ConvNets) to learn the most relevant features of a scene at the multi-scale superpixel level. The effect of both trained and general ConvNets features for our scene labeling framework are investigated. Then, based on the predicted posteriors from the learned features, we propose an algorithm called Region Consistency Activation (RCA) to iteratively improve the global label consistency at different levels of the Ultrametric Contour Map (UCM). In addition, we propose a strategy to make the hyper-parameters of RCA adaptive to the test images, which results in a better generalization ability compared with the hyper-parameters tuning based RCA. Our scene labeling framework were rigorously tested on three popular scene labeling datasets: Stanford Background, SIFT Flow and NYU-Depth V2. Experiments show that our proposed method consistently produces better accuracy and visual consistency compared with the state-of-the-art methods for both outdoor and indoor scenes.