Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field

Liang Zhang, Huan Li, Peiyi Shen, Guangming Zhu, Juan Song, Syed Afaq Ali Shah, Li Zhang, Mohammed Bennamoun

    Research output: Contribution to journalArticlepeer-review

    23 Citations (Scopus)


    Deep convolutional neural networks (DCNNs) have been driving significant advances in semantic image segmentation due to their powerful feature representation for recognition. However, their performance in preserving object boundaries is still not satisfactory. Visual mechanism theory indicates that image segmentation tasks require not only recognition, like DCNNs, but also local visual attention capability. Considering that superpixel is good at grasping detailed local structure, we propose a probabilistic superpixel-based dense conditional random field model (PSP-CRF) to refine label assignments as a post-processing optimization method. First, the well-known fully convolutional networks (FCN) and Deeplab-ResNet are employed to produce coarse prediction probabilistic maps at each pixel. Second, we construct a fully connected CRF model based on the PSP generated by the simple linear iterative clustering algorithm. In our approach, an effective refining algorithm with entropy is developed to convert the pixel-level appearance and position features to the normalized PSP, which works well for CRF. Third, our method optimizes the PSP-CRF to obtain the final label assignment results by employing a highly efficient mean field inference algorithm and some quadratic programming relaxation related algorithms. The experiments on the PASCAL VOC segmentation dataset demonstrate the effectiveness of our methods which can improve the segmentation performance of DCNNs to 82% in mIoU while increasing the computational efficiency by 47%.

    Original languageEnglish
    Article number 2169-3536
    Pages (from-to)15297-15310
    Number of pages14
    JournalIEEE Access
    Publication statusPublished - 9 Mar 2018


    Dive into the research topics of 'Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field'. Together they form a unique fingerprint.

    Cite this