Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field

Liang Zhang, Huan Li, Peiyi Shen, Guangming Zhu, Juan Song, Syed Afaq Ali Shah, Li Zhang, Mohammed Bennamoun

    Research output: Contribution to journalArticle

    7 Citations (Scopus)

    Abstract

    Deep convolutional neural networks (DCNNs) have been driving significant advances in semantic image segmentation due to their powerful feature representation for recognition. However, their performance in preserving object boundaries is still not satisfactory. Visual mechanism theory indicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we propose a probabilistic superpixel-based dense conditional random field model (PSP-CRF) to refine label assignments as a post-processing optimization method. First, the well-known fully convolutional networks (FCN) and Deeplab-ResNet are employed to produce coarse prediction probabilistic maps at each pixel. Second, we construct a fully connected CRF model based on the PSP generated by the simple linear iterative clustering algorithm. In our approach, an effective refining algorithm with entropy is developed to convert the pixel-level appearance and position features to the normalized PSP, which works well for CRF. Third, our method optimizes the PSP-CRF to obtain the final label assignment results by employing a highly efficient mean field inference algorithm and some quadratic programming relaxation related algorithms. The experiments on the PASCAL VOC segmentation dataset demonstrate the effectiveness of our methods which can improve the segmentation performance of DCNNs to 82% in mIoU while increasing the computational efficiency by 47%.

    Original languageEnglish
    Article number 2169-3536
    Pages (from-to)15297-15310
    Number of pages14
    JournalIEEE Access
    Volume6
    DOIs
    Publication statusPublished - 9 Mar 2018

    Fingerprint

    Image segmentation
    Semantics
    Neural networks
    Labels
    Pixels
    Quadratic programming
    Computational efficiency
    Volatile organic compounds
    Clustering algorithms
    Refining
    Entropy
    Processing
    Experiments

    Cite this

    Zhang, Liang ; Li, Huan ; Shen, Peiyi ; Zhu, Guangming ; Song, Juan ; Shah, Syed Afaq Ali ; Zhang, Li ; Bennamoun, Mohammed. / Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field. In: IEEE Access. 2018 ; Vol. 6. pp. 15297-15310.
    @article{2dd8d293e59f40cdb3e4a7683819b2ac,
    title = "Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field",
    abstract = "Deep convolutional neural networks (DCNNs) have been driving significant advances in semantic image segmentation due to their powerful feature representation for recognition. However, their performance in preserving object boundaries is still not satisfactory. Visual mechanism theory indicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we propose a probabilistic superpixel-based dense conditional random field model (PSP-CRF) to refine label assignments as a post-processing optimization method. First, the well-known fully convolutional networks (FCN) and Deeplab-ResNet are employed to produce coarse prediction probabilistic maps at each pixel. Second, we construct a fully connected CRF model based on the PSP generated by the simple linear iterative clustering algorithm. In our approach, an effective refining algorithm with entropy is developed to convert the pixel-level appearance and position features to the normalized PSP, which works well for CRF. Third, our method optimizes the PSP-CRF to obtain the final label assignment results by employing a highly efficient mean field inference algorithm and some quadratic programming relaxation related algorithms. The experiments on the PASCAL VOC segmentation dataset demonstrate the effectiveness of our methods which can improve the segmentation performance of DCNNs to 82{\%} in mIoU while increasing the computational efficiency by 47{\%}.",
    keywords = "DCNNs, dense conditional random field (CRF), semantic image segmentation, superpixel segmentation",
    author = "Liang Zhang and Huan Li and Peiyi Shen and Guangming Zhu and Juan Song and Shah, {Syed Afaq Ali} and Li Zhang and Mohammed Bennamoun",
    year = "2018",
    month = "3",
    day = "9",
    doi = "10.1109/ACCESS.2018.2814568",
    language = "English",
    volume = "6",
    pages = "15297--15310",
    journal = "IEEE Access",
    issn = "2169-3536",
    publisher = "IEEE, Institute of Electrical and Electronics Engineers",

    }

    Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field. / Zhang, Liang; Li, Huan; Shen, Peiyi; Zhu, Guangming; Song, Juan; Shah, Syed Afaq Ali; Zhang, Li; Bennamoun, Mohammed.

    In: IEEE Access, Vol. 6, 2169-3536, 09.03.2018, p. 15297-15310.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field

    AU - Zhang, Liang

    AU - Li, Huan

    AU - Shen, Peiyi

    AU - Zhu, Guangming

    AU - Song, Juan

    AU - Shah, Syed Afaq Ali

    AU - Zhang, Li

    AU - Bennamoun, Mohammed

    PY - 2018/3/9

    Y1 - 2018/3/9

    N2 - Deep convolutional neural networks (DCNNs) have been driving significant advances in semantic image segmentation due to their powerful feature representation for recognition. However, their performance in preserving object boundaries is still not satisfactory. Visual mechanism theory indicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we propose a probabilistic superpixel-based dense conditional random field model (PSP-CRF) to refine label assignments as a post-processing optimization method. First, the well-known fully convolutional networks (FCN) and Deeplab-ResNet are employed to produce coarse prediction probabilistic maps at each pixel. Second, we construct a fully connected CRF model based on the PSP generated by the simple linear iterative clustering algorithm. In our approach, an effective refining algorithm with entropy is developed to convert the pixel-level appearance and position features to the normalized PSP, which works well for CRF. Third, our method optimizes the PSP-CRF to obtain the final label assignment results by employing a highly efficient mean field inference algorithm and some quadratic programming relaxation related algorithms. The experiments on the PASCAL VOC segmentation dataset demonstrate the effectiveness of our methods which can improve the segmentation performance of DCNNs to 82% in mIoU while increasing the computational efficiency by 47%.

    AB - Deep convolutional neural networks (DCNNs) have been driving significant advances in semantic image segmentation due to their powerful feature representation for recognition. However, their performance in preserving object boundaries is still not satisfactory. Visual mechanism theory indicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we propose a probabilistic superpixel-based dense conditional random field model (PSP-CRF) to refine label assignments as a post-processing optimization method. First, the well-known fully convolutional networks (FCN) and Deeplab-ResNet are employed to produce coarse prediction probabilistic maps at each pixel. Second, we construct a fully connected CRF model based on the PSP generated by the simple linear iterative clustering algorithm. In our approach, an effective refining algorithm with entropy is developed to convert the pixel-level appearance and position features to the normalized PSP, which works well for CRF. Third, our method optimizes the PSP-CRF to obtain the final label assignment results by employing a highly efficient mean field inference algorithm and some quadratic programming relaxation related algorithms. The experiments on the PASCAL VOC segmentation dataset demonstrate the effectiveness of our methods which can improve the segmentation performance of DCNNs to 82% in mIoU while increasing the computational efficiency by 47%.

    KW - DCNNs

    KW - dense conditional random field (CRF)

    KW - semantic image segmentation

    KW - superpixel segmentation

    UR - http://www.scopus.com/inward/record.url?scp=85043468668&partnerID=8YFLogxK

    U2 - 10.1109/ACCESS.2018.2814568

    DO - 10.1109/ACCESS.2018.2814568

    M3 - Article

    VL - 6

    SP - 15297

    EP - 15310

    JO - IEEE Access

    JF - IEEE Access

    SN - 2169-3536

    M1 - 2169-3536

    ER -