Why can't hyperspectral or multispectral images be obtained from photos taken by optical cameras through image processing?

The main difference between hyperspectral and multispectral cameras is the number of bands they record and the width of the bands (i.e. spectral resolution).

According to the standard definition, a hyperspectral camera records more than 100 bands, while a multispectral camera records fewer bands. However, this definition does not take into account the width of the spectral range or the sampling rate. This means that if a camera covers the spectral range of 400–600 nm and records 50 bands, it is not a hyperspectral camera, but if it covers 400–800nm and has the same sampling rate (meaning this time it records 100 bands), then it is a hyperspectral camera.

This article prefers to talk about spectral resolution (FWHM, full width at half maximum*), emphasizing the camera's ability to distinguish between two consecutive spectral peaks.

Hyperspectral vs. multispectral data

Hyperspectral imaging involves capturing and analyzing data from a large number of narrow, continuous bands in the electromagnetic spectrum to produce a high-resolution spectrum for each pixel in the image. As a result, hyperspectral cameras provide smooth spectra. Multispectral cameras provide spectra that appear stepped or jagged and cannot accurately depict spectral features.

Spectral imaging provides more detailed data than multispectral imaging, allowing for more specific analysis and more accurate identification of various materials and substances.

Multispectral imaging may not be able to distinguish closely related materials due to limited spectral resolution. For most multispectral cameras on the market, the spectral range is limited to 400 – 1000 nm, with a typical number of bands between 4 – 5. These limitations have important consequences for many applications.

To illustrate the advantages of hyperspectral cameras over multispectral cameras, we studied the process of sorting almond shells from almonds. This is a typical application that requires a high degree of accuracy in identifying different materials that are very similar in appearance.

Advantages of hyperspectral cameras over multispectral cameras in terms of spectral range

The spectrum formed by the reflection, absorption and emission characteristics is closely related to the molecular composition of the material to be tested. Table 1 is very famous. Most of the molecular bonds of substances have spectral fingerprints brought by resonant frequencies (spectral resonance frequencies).

As shown in the table, many applications require a spectral range of 700–2500nm. Especially for those applications related to food quality assessment and plastic sorting, a spectral range of 1100–1700nm is essential. Multispectral cameras limited to the 400–1000nm range are not suitable for these applications.

The article comparing hyperspectral imaging to RGB cameras showed that RGB cameras performed poorly in sorting nuts and pistachios. The CHNSpec FS13 performed better, while the CHNSpec FS17 had the highest sorting accuracy.

The advantage of hyperspectral cameras over multispectral cameras in terms of the number of bands

Taking the previous point into account, almonds and almond shells were compared using CHNSpec FS17 camera data, covering the spectral range of 900–1700 nm and recording 224 bands in the first dataset. In the second dataset, only 28 merged bands (i.e., merging continuous spectral bands) were used to simulate a multispectral camera.

As shown in Figures 1 and 2, the spectra associated with the 224-band dataset are much smoother than those depicted with only 28 bands. In the hyperspectral data, it is also possible to capture small but crucial spectral differences in order to be able to separate almonds from their shells.

In other words, because the number of bands captured is limited to 28, the spectral information required to distinguish almonds from their shells is lost.

In addition, some preprocessing methods are not suitable for multispectral data. For example, derivative or smoothing algorithms such as Savitzky-Golay require continuous spectra to perform well, which multispectral sensors cannot provide.

As mentioned above, by building two models to illustrate these points related to the ability to accurately characterize spectral features, the hyperspectral model covering 224 bands is more accurate than the multispectral model covering only 28 bands. In the hyperspectral data, the edge effects disappear, and even the smallest almond shell fragments can be correctly classified.

In this study, 28 spectral bands were selected to simulate a multispectral camera. However, typical multispectral cameras cover significantly fewer spectral bands, further reducing their ability to depict fine spectral features.

How to choose between hyperspectral imaging and multispectral imaging?

Both hyperspectral imaging and multispectral imaging are widely used to capture and analyze the electromagnetic spectrum in various research, industrial, and remote sensing applications. The two technologies complement each other and the choice depends on the application requirements and the level of available data.

If the application requires more spectral bands to be covered and the required spectral resolution is higher than the capabilities of multispectral imaging technology, then the natural solution is a hyperspectral camera.

If the application does not require the entire spectral range to be covered, a customized multispectral camera covering specific bands can perform just as well as a hyperspectral camera. However, this requires the user to know the selected number of spectral bands that must be covered for inspection or analysis. If the user does not know the spectral requirements of the application or they are very complex, it is better to use a hyperspectral camera to collect the analysis data.

Hyperspectral cameras also provide greater flexibility, allowing users to upgrade the machine at a later time to sort new debris or materials. When using the CHNSpec FS camera, users are free to choose the relevant bands. In fact, the CHNSpec FS hyperspectral camera can be converted to a multispectral camera, while a multispectral camera can never be turned into a hyperspectral camera.

Finally, another aspect to consider when choosing between hyperspectral and multispectral cameras is price. Typically, hyperspectral cameras are more expensive and require more processing power than multispectral cameras, but this is not always the case, especially when a larger number of bands are required for a multispectral camera.