Improving semantic image segmentation with a probabilistic superpixel-based fully connected CRF

Liang Zhang, Huan Li, Guangming Zhu, Peiyi Shen, Syed Afaq Shah, Mohammed Bennamoun

Research output: Contribution to journalConference article

Abstract

Deep convolutional neural networks (DCNNs) have been driving significant advances in image semantic segmentation tasks due to their power-ful feature representation for recognition. However, their performance in pre-serving object boundaries is still not satisfactory. Visual mechanism theory in-dicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we proposed a Probabilistic Superpix-el-based fully connected CRF model (PSP-CRF) to refine label assignments as a post-processing optimization method. Firstly, we employ the SLIC segmented algorithm to obtain superpixels and employ the well-known DCNNs to produce coarse prediction probabilistic maps at each pixel. We present an effective strategy based on entropy to convert the pixel-level color, position and texture features to the normalized probabilistic superpixels. Secondly, we construct a fully connected CRF model with the probabilistic superpixels as vertexes in-stead of pixels. Thirdly, We optimize the PSP-CRF model to obtain the final la-bel assignment results by employing a highly efficient mean field inference al-gorithm. The experimental results on the PASCAL VOC segmentation dataset demonstrate that the proposed method can improve the segmentation perfor-mance to 82% mIoU while increasing the computational efficiency.
Original languageEnglish
Pages (from-to)15297 - 15310
JournalIEEE Access
Volume6
Publication statusPublished - 12 Mar 2018
EventSecond CCF Chinese Conference - Tianjin, China
Duration: 11 Oct 201414 Oct 2017
Conference number: 2

Fingerprint

Image segmentation
Pixels
Semantics
Neural networks
Computational efficiency
Volatile organic compounds
Labels
Entropy
Textures
Color
Processing

Cite this

Zhang, Liang ; Li, Huan ; Zhu, Guangming ; Shen, Peiyi ; Shah, Syed Afaq ; Bennamoun, Mohammed. / Improving semantic image segmentation with a probabilistic superpixel-based fully connected CRF. In: IEEE Access. 2018 ; Vol. 6. pp. 15297 - 15310.
@article{59d549e1636745cf945bb6451c96db1a,
title = "Improving semantic image segmentation with a probabilistic superpixel-based fully connected CRF",
abstract = "Deep convolutional neural networks (DCNNs) have been driving significant advances in image semantic segmentation tasks due to their power-ful feature representation for recognition. However, their performance in pre-serving object boundaries is still not satisfactory. Visual mechanism theory in-dicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we proposed a Probabilistic Superpix-el-based fully connected CRF model (PSP-CRF) to refine label assignments as a post-processing optimization method. Firstly, we employ the SLIC segmented algorithm to obtain superpixels and employ the well-known DCNNs to produce coarse prediction probabilistic maps at each pixel. We present an effective strategy based on entropy to convert the pixel-level color, position and texture features to the normalized probabilistic superpixels. Secondly, we construct a fully connected CRF model with the probabilistic superpixels as vertexes in-stead of pixels. Thirdly, We optimize the PSP-CRF model to obtain the final la-bel assignment results by employing a highly efficient mean field inference al-gorithm. The experimental results on the PASCAL VOC segmentation dataset demonstrate that the proposed method can improve the segmentation perfor-mance to 82{\%} mIoU while increasing the computational efficiency.",
author = "Liang Zhang and Huan Li and Guangming Zhu and Peiyi Shen and Shah, {Syed Afaq} and Mohammed Bennamoun",
year = "2018",
month = "3",
day = "12",
language = "English",
volume = "6",
pages = "15297 -- 15310",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",

}

Improving semantic image segmentation with a probabilistic superpixel-based fully connected CRF. / Zhang, Liang; Li, Huan; Zhu, Guangming; Shen, Peiyi; Shah, Syed Afaq; Bennamoun, Mohammed.

In: IEEE Access, Vol. 6, 12.03.2018, p. 15297 - 15310.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Improving semantic image segmentation with a probabilistic superpixel-based fully connected CRF

AU - Zhang, Liang

AU - Li, Huan

AU - Zhu, Guangming

AU - Shen, Peiyi

AU - Shah, Syed Afaq

AU - Bennamoun, Mohammed

PY - 2018/3/12

Y1 - 2018/3/12

N2 - Deep convolutional neural networks (DCNNs) have been driving significant advances in image semantic segmentation tasks due to their power-ful feature representation for recognition. However, their performance in pre-serving object boundaries is still not satisfactory. Visual mechanism theory in-dicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we proposed a Probabilistic Superpix-el-based fully connected CRF model (PSP-CRF) to refine label assignments as a post-processing optimization method. Firstly, we employ the SLIC segmented algorithm to obtain superpixels and employ the well-known DCNNs to produce coarse prediction probabilistic maps at each pixel. We present an effective strategy based on entropy to convert the pixel-level color, position and texture features to the normalized probabilistic superpixels. Secondly, we construct a fully connected CRF model with the probabilistic superpixels as vertexes in-stead of pixels. Thirdly, We optimize the PSP-CRF model to obtain the final la-bel assignment results by employing a highly efficient mean field inference al-gorithm. The experimental results on the PASCAL VOC segmentation dataset demonstrate that the proposed method can improve the segmentation perfor-mance to 82% mIoU while increasing the computational efficiency.

AB - Deep convolutional neural networks (DCNNs) have been driving significant advances in image semantic segmentation tasks due to their power-ful feature representation for recognition. However, their performance in pre-serving object boundaries is still not satisfactory. Visual mechanism theory in-dicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we proposed a Probabilistic Superpix-el-based fully connected CRF model (PSP-CRF) to refine label assignments as a post-processing optimization method. Firstly, we employ the SLIC segmented algorithm to obtain superpixels and employ the well-known DCNNs to produce coarse prediction probabilistic maps at each pixel. We present an effective strategy based on entropy to convert the pixel-level color, position and texture features to the normalized probabilistic superpixels. Secondly, we construct a fully connected CRF model with the probabilistic superpixels as vertexes in-stead of pixels. Thirdly, We optimize the PSP-CRF model to obtain the final la-bel assignment results by employing a highly efficient mean field inference al-gorithm. The experimental results on the PASCAL VOC segmentation dataset demonstrate that the proposed method can improve the segmentation perfor-mance to 82% mIoU while increasing the computational efficiency.

UR - https://ieeexplore.ieee.org/document/8314143/

M3 - Conference article

VL - 6

SP - 15297

EP - 15310

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

ER -