Fish species classification in unconstrained underwater environments based on deep learning

A. Salman, A. Jalal, Faisal Shafait, Ajmal Mian, M. Shortis, J. Seager, E. Harvey

    Research output: Contribution to journalArticle

    25 Citations (Scopus)

    Abstract

    © 2016 Association for the Sciences of Limnology and Oceanography. Underwater video and digital still cameras are rapidly being adopted by marine scientists and managers as a tool for non-destructively quantifying and measuring the relative abundance, cover and size of marine fauna and flora. Imagery recorded of fish can be time consuming and costly to process and analyze manually. For this reason, there is great interest in automatic classification, counting, and measurement of fish. Unconstrained underwater scenes are highly variable due to changes in light intensity, changes in fish orientation due to movement, a variety of background habitats which sometimes also move, and most importantly similarity in shape and patterns among fish of different species. This poses a great challenge for image/video processing techniques to accurately differentiate between classes or species of fish to perform automatic classification. We present a machine learning approach, which is suitable for solving this challenge. We demonstrate the use of a convolution neural network model in a hierarchical feature combination setup to learn species-dependent visual features of fish that are unique, yet abstract and robust against environmental and intra-and inter-species variability. This approach avoids the need for explicitly extracting features from raw images of the fish using several fragmented image processing techniques. As a result, we achieve a single and generic trained architecture with favorable performance even for sample images of fish species that have not been used in training. Using the LifeCLEF14 and LifeCLEF15 benchmark fish datasets, we have demonstrated results with a correct classification rate of more than 90%.
    Original languageEnglish
    Pages (from-to)570-585
    JournalLIMNOLOGY AND OCEANOGRAPHY-METHODS
    Volume14
    Issue number9
    Early online date31 May 2016
    DOIs
    Publication statusPublished - Sep 2016

    Fingerprint

    Fish
    Limnology
    Deep learning
    Oceanography
    Digital cameras
    Convolution
    Learning systems
    Image processing
    Managers
    Neural networks
    Processing

    Cite this

    Salman, A. ; Jalal, A. ; Shafait, Faisal ; Mian, Ajmal ; Shortis, M. ; Seager, J. ; Harvey, E. / Fish species classification in unconstrained underwater environments based on deep learning. In: LIMNOLOGY AND OCEANOGRAPHY-METHODS. 2016 ; Vol. 14, No. 9. pp. 570-585.
    @article{232785ba220447a6ac7173216e71712d,
    title = "Fish species classification in unconstrained underwater environments based on deep learning",
    abstract = "{\circledC} 2016 Association for the Sciences of Limnology and Oceanography. Underwater video and digital still cameras are rapidly being adopted by marine scientists and managers as a tool for non-destructively quantifying and measuring the relative abundance, cover and size of marine fauna and flora. Imagery recorded of fish can be time consuming and costly to process and analyze manually. For this reason, there is great interest in automatic classification, counting, and measurement of fish. Unconstrained underwater scenes are highly variable due to changes in light intensity, changes in fish orientation due to movement, a variety of background habitats which sometimes also move, and most importantly similarity in shape and patterns among fish of different species. This poses a great challenge for image/video processing techniques to accurately differentiate between classes or species of fish to perform automatic classification. We present a machine learning approach, which is suitable for solving this challenge. We demonstrate the use of a convolution neural network model in a hierarchical feature combination setup to learn species-dependent visual features of fish that are unique, yet abstract and robust against environmental and intra-and inter-species variability. This approach avoids the need for explicitly extracting features from raw images of the fish using several fragmented image processing techniques. As a result, we achieve a single and generic trained architecture with favorable performance even for sample images of fish species that have not been used in training. Using the LifeCLEF14 and LifeCLEF15 benchmark fish datasets, we have demonstrated results with a correct classification rate of more than 90{\%}.",
    author = "A. Salman and A. Jalal and Faisal Shafait and Ajmal Mian and M. Shortis and J. Seager and E. Harvey",
    year = "2016",
    month = "9",
    doi = "10.1002/lom3.10113",
    language = "English",
    volume = "14",
    pages = "570--585",
    journal = "Limnology and Oceanography : Methods",
    issn = "1541-5856",
    publisher = "American Society of Limnology and Oceanography, Inc.",
    number = "9",

    }

    Fish species classification in unconstrained underwater environments based on deep learning. / Salman, A.; Jalal, A.; Shafait, Faisal; Mian, Ajmal; Shortis, M.; Seager, J.; Harvey, E.

    In: LIMNOLOGY AND OCEANOGRAPHY-METHODS, Vol. 14, No. 9, 09.2016, p. 570-585.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Fish species classification in unconstrained underwater environments based on deep learning

    AU - Salman, A.

    AU - Jalal, A.

    AU - Shafait, Faisal

    AU - Mian, Ajmal

    AU - Shortis, M.

    AU - Seager, J.

    AU - Harvey, E.

    PY - 2016/9

    Y1 - 2016/9

    N2 - © 2016 Association for the Sciences of Limnology and Oceanography. Underwater video and digital still cameras are rapidly being adopted by marine scientists and managers as a tool for non-destructively quantifying and measuring the relative abundance, cover and size of marine fauna and flora. Imagery recorded of fish can be time consuming and costly to process and analyze manually. For this reason, there is great interest in automatic classification, counting, and measurement of fish. Unconstrained underwater scenes are highly variable due to changes in light intensity, changes in fish orientation due to movement, a variety of background habitats which sometimes also move, and most importantly similarity in shape and patterns among fish of different species. This poses a great challenge for image/video processing techniques to accurately differentiate between classes or species of fish to perform automatic classification. We present a machine learning approach, which is suitable for solving this challenge. We demonstrate the use of a convolution neural network model in a hierarchical feature combination setup to learn species-dependent visual features of fish that are unique, yet abstract and robust against environmental and intra-and inter-species variability. This approach avoids the need for explicitly extracting features from raw images of the fish using several fragmented image processing techniques. As a result, we achieve a single and generic trained architecture with favorable performance even for sample images of fish species that have not been used in training. Using the LifeCLEF14 and LifeCLEF15 benchmark fish datasets, we have demonstrated results with a correct classification rate of more than 90%.

    AB - © 2016 Association for the Sciences of Limnology and Oceanography. Underwater video and digital still cameras are rapidly being adopted by marine scientists and managers as a tool for non-destructively quantifying and measuring the relative abundance, cover and size of marine fauna and flora. Imagery recorded of fish can be time consuming and costly to process and analyze manually. For this reason, there is great interest in automatic classification, counting, and measurement of fish. Unconstrained underwater scenes are highly variable due to changes in light intensity, changes in fish orientation due to movement, a variety of background habitats which sometimes also move, and most importantly similarity in shape and patterns among fish of different species. This poses a great challenge for image/video processing techniques to accurately differentiate between classes or species of fish to perform automatic classification. We present a machine learning approach, which is suitable for solving this challenge. We demonstrate the use of a convolution neural network model in a hierarchical feature combination setup to learn species-dependent visual features of fish that are unique, yet abstract and robust against environmental and intra-and inter-species variability. This approach avoids the need for explicitly extracting features from raw images of the fish using several fragmented image processing techniques. As a result, we achieve a single and generic trained architecture with favorable performance even for sample images of fish species that have not been used in training. Using the LifeCLEF14 and LifeCLEF15 benchmark fish datasets, we have demonstrated results with a correct classification rate of more than 90%.

    U2 - 10.1002/lom3.10113

    DO - 10.1002/lom3.10113

    M3 - Article

    VL - 14

    SP - 570

    EP - 585

    JO - Limnology and Oceanography : Methods

    JF - Limnology and Oceanography : Methods

    SN - 1541-5856

    IS - 9

    ER -