Hyperspectral video restoration using optical flow and sparse coding

Ajmal Mian, R. Hartley

    Research output: Contribution to journalArticle

    19 Citations (Scopus)


    Hyperspectral video acquisition is a trade-off between spectral and temporal resolution. We present an algorithm for recovering dense hyperspectral video of dynamic scenes from a few measured multispectral bands per frame using optical flow and sparse coding. Different set of bands are measured in each video frame and optical flow is used to register them. Optical flow errors are corrected by exploiting sparsity in the spectra and the spatial correlation between images of a scene at different wavelengths. A redundant dictionary of atoms is learned that can sparsely approximate training spectra. The restoration of correct spectra is formulated as an ℓ1 convex optimization problem that minimizes a Mahalanobis-like weighted distance between the restored and corrupt signals as well as the restored signal and the median of the eight connected neighbours of the corrupt signal such that the restored signal is a sparse linear combination of the dictionary atoms. Spectral restoration is followed by spatial restoration using a guided dictionary approach where one dictionary is learned for measured bands and another for a band that is to be spatially restored. By constraining the sparse coding coefficients of both dictionaries to be the same, the restoration of corrupt band is guided by the more reliable measured bands. Experiments on real data and comparison with an existing volumetric image denoising technique shows the superiority of our algorithm.
    Original languageEnglish
    Pages (from-to)10658 - 10673
    JournalOptics Express
    Issue number10
    Early online date24 Apr 2012
    Publication statusPublished - 7 May 2012

    Fingerprint Dive into the research topics of 'Hyperspectral video restoration using optical flow and sparse coding'. Together they form a unique fingerprint.

    Cite this