In contrast to most artificial and biological optical sensors, hyperspectral imaging systems allow us to accurately recover the full spectral profile of each point in the observed scene. This rich spectral information has been explored and exploited in many fields such as astronomy, biology, chemistry, agriculture, geology, environmental studies, and in a wide variety of other academic and commercial applications. Here at the ICVL, we are using cutting-edge equipment to explore new avenues of research in the field of hyperspectral imagery. The project described on this page explores hyperspectral acquisition from very low-cost multispectral (and in particular, RGB) sensors.

Hyperspectral from RGB

RGB imaging systems capture a surprising amount of spectral information. By leveraging sparse overcomplete dictionaries, it is possible to recover hyperspectral information from RGB images.

Filter Optimization for Hyperspectral Estimation

While many methods attempt to recover hyperspectral information from RGB images, not all RGB cameras were created equal. Selecting an optimal RGB camera can increase performance by over 20%.

Replacing RGB filters with customized filter sets can improve performance even further, but finding an optimal filter set within large collections is no simple task.

Evolutionary optimization allows us to efficiently explore large filter collections, quickly finding optimal or near-optimal filter sets for hyperspectral estimation.

Software

Sample code demonstrating the hyperspectral recovery methodology described in “Sparse Recovery of Hyperspectral Signal from Natural RGB Images” can be found on the project GitHub repository:

• https://github.com/icvl/shred

BGU ICVL Hyperspectral Dataset

In order to allow rapid access to the dataset described in “Sparse Recovery of Hyperspectral Signal from Natural RGB Images“, we provide a link to the dataset below.

The dataset images were acquired using a Specim PS Kappa DX4 hyperspectral camera and a rotary stage for spatial scanning. At this time it contains 200 images and will continue to grow progressively.

Images were collected at 1392×1300 spatial resolution over 519 spectral bands (400-1,000nm at roughly 1.25nm increments). The .raw files contain raw out-of-camera data in ENVI format and .hdr files contain the headers required to decode them. For your convenience, .mat files are provided, downsampled to 31 spectral channels from 400nm to 700nm at 10nm increments.

BGU ICVL Hyperspectral Dataset

When used, fully or partially, please cite “Arad and Ben-Shahar, Sparse Recovery of Hyperspectral Signal from Natural RGB Images, in the European Conference on Computer Vision, Amsterdam, The Netherlands, October 11–14, 2016”

@inproceedings{arad_and_ben_shahar_2016_ECCV,
        title={Sparse Recovery of Hyperspectral Signal from Natural RGB Images},
        author={Arad, Boaz and Ben-Shahar, Ohad},
        booktitle={European Conference on Computer Vision},
        pages={19--34},
        year={2016},
        organization={Springer}
      }

*RGB renderings are for reference only, and do not depict an expected camera response.

Instructions to Open in Tiles View

To view the dataset in "Tiles" mode on SharePoint:

1. Click the dataset link above to open it in SharePoint.
2. In the top right corner of the document library, click the view options icon (it may look like a grid or list icon).
3. Select "Tiles" from the dropdown menu.

Papers

• B. Arad, and O. Ben-Shahar, Filter Selection for Hyperspectral Estimation, In the Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, October 2017.
• B. Arad and O. Ben-Shahar, Sparse Recovery of Hyperspectral Signal from Natural RGB Images, In the Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October 2016.

Video

Presentation at the 2nd EMVA Forum at AIT.

Acknowledgments

This research was supported in part by the by the Israel Science Foundation (ISF FIRST/BIKURA Grant 281/15) and the European Commission (Horizon 2020 grant SWEEPER GA no 644313). We also thank the Frankel Fund and the Helmsley Charitable Trust through the ABC Robotics Initiative, both at Ben-Gurion University of the Negev.