Hyperspectral image compression using a feature extraction model
A computer implemented method for reducing an amount of memory required to store hyperspectral images of an object include: obtaining tensor data representing a hyperspectral image including a first portion depicting an object and a second portion depicting at least a portion of a surrounding environment where the object is located; identifying a portion of the tensor data representing the hyperspectral image that corresponds to the first portion; providing the identified portion of the tensor data representing the hyperspectral image as an input to a feature extraction model; obtaining one or more matrix structures as output by the feature extraction model based on the feature extraction model processing the identified portion of the tensor data, the one or more matrix structures representing a subset of features extracted from the identified portion of the tensor data; and storing the one or more matrix structures in a memory device.
1. A method for reducing an amount of memory required to store hyperspectral images of an object, the method comprising:
obtaining, by one or more computers, tensor data representing a hyperspectral image, wherein the hyperspectral image includes a first portion that depicts an object and a second portion that depicts at least a portion of a surrounding environment where the object is located, wherein the object is a first type, and wherein the hyperspectral image captures data within a wavelength range, and wherein the data within the wavelength range is usable for inferencing conditions of objects of the first type;
identifying, by the one or more computers, a portion of the tensor data representing the hyperspectral image that corresponds to the first portion of the hyperspectral image;
providing, by the one or more computers, the identified portion of the tensor data representing the hyperspectral image as an input to a feature extraction model, wherein the feature extraction model was trained using a training dataset to process portions of tensor data of other hyperspectral images to determine one or more matrix structures associated with the other hyperspectral images;
obtaining, by the one or more computers, the one or more matrix structures as output from the feature extraction model, wherein the one or more matrix structures represent a subset of features extracted from the identified portion of the tensor data representing the hyperspectral image, and wherein the subset of the features extracted from the identified portion comprises distinguishing features usable to inference a current condition of the object; and
storing, by the one or more computers, the one or more matrix structures in a memory device.
2. The method of claim 1 , wherein the object depicted by the hyperspectral image is a food item, the food item being at least one of a citrus fruit, an avocado, a mango, an apple, a berry, a stone fruit, a tomato, meat, or a vegetable.
3. The method of claim 1 , further comprising receiving, by the one or more computers from a hyperspectral sensor, the hyperspectral image, wherein the hyperspectral sensor is configured to detect light waves having the wavelength range, wherein the hyperspectral sensor is configured to automatically select the wavelength range in response to identifying the type of the object, and wherein the wavelength range is between approximately 520 nm to 950 nm.
4. The method of claim 1 , the method further comprising:
normalizing, by the one or more computers, the identified portion of the tensor data representing the hyperspectral image, and
wherein providing, by the one or more computers, the identified portion of the tensor data representing the hyperspectral image as an input to the feature extraction model comprises providing the normalized portion of the tensor data as the input to the feature extraction model.
5. The method of claim 1 , wherein the feature extraction model is an unsupervised learning model.
6. The method of claim 1 , wherein the feature extraction model is a supervised learning model.
7. The method of claim 1 , wherein the feature extraction model is a multilinear principal component analysis model having one or more user defined parameters defining one or more matrix structures to be produced based on processing, by the feature extraction model, the portion of the tensor data representing the hyperspectral image, wherein the one or more user defined parameters include a height and a width of each matrix structure of the one or more matrix structures.
8. The method of claim 1 , wherein at least one of the one or more matrix structures include data that represents a spectral domain of the hyperspectral image.
9. The method of claim 1 , further comprising:
providing, by the one or more computers, (i) the identified portion of the tensor data and (ii) the one or more matrix structures outputted by the feature extraction model as input to a compression engine;
obtaining, by the one or more computers, a core matrix structure as output from the compression engine; and
storing, by the one or more computers, the core matrix structure in the memory device.
10. The method of claim 9 , wherein the core matrix structure is generated by the compression engine based on performing an n-mode dot product on the identified portion of the tensor data that corresponds to the object represented by the one or more matrix structures.
11. The method of claim 9 , further comprising:
performing, by the one or more computers, a first matrix operation on a first set of the one or more matrix structures to generate a second set of one or more matrix structures;
performing, by the one or more computers, a second matrix operation on (i) the second set of one or more matrix structures and (ii) the core matrix structure outputted by the compression engine; and
generating, by the one or more computers, a reconstructed matrix based on performing the second matrix operation, wherein the reconstructed matrix represents features of the identified portion of tensor data.
12. The method of claim 1 , wherein the current condition of the object is a current level of ripeness or a current level of firmness.
13. A system comprising
a processor; and
a memory coupled to the processor and having instructions stored thereon which, when executed by the processor, cause the processor to perform operations, the operations comprising:
obtaining tensor data representing a hyperspectral image, wherein the hyperspectral image includes a first portion that depicts an object and a second portion that depicts at least a portion of a surrounding environment where the object is located, wherein the object is a first type, and wherein the hyperspectral image captures data within a wavelength range, and wherein the data within the wavelength range is usable for inferencing conditions of objects of the first type;
identifying a portion of the tensor data representing the hyperspectral image that corresponds to the first portion of the hyperspectral image;
providing the identified portion of the tensor data representing the hyperspectral image as an input to a feature extraction model, wherein the feature extraction model was trained using a training dataset to process portions of tensor data of other hyperspectral images to determine one or more matrix structures associated with the other hyperspectral images;
obtaining the one or more matrix structures as output from the feature extraction model, wherein the one or more matrix structures represent a subset of features extracted from the identified portion of the tensor data representing the hyperspectral image, and wherein the subset of the features extracted from the identified portion comprises distinguishing features usable to inference a current condition of the object; and
storing the one or more matrix structures in a memory device.
14. The system of claim 13 , wherein the object depicted by the hyperspectral image is a food item, the food item being at least one of a citrus fruit, an avocado, a mango, an apple, a berry, a stone fruit, a tomato, meat, or a vegetable, and wherein the current condition of the object is current level of ripeness.
15. The system of claim 13 , wherein the memory further comprises instructions, which when executed by the processor, cause the processor to perform operations comprising:
receiving, from a hyperspectral sensor, the hyperspectral image, wherein the hyperspectral sensor is configured to detect light waves having the wavelength range, wherein the hyperspectral sensor is configured to automatically select the wavelength range in response to identifying the type of the object, and wherein the wavelength range is between approximately 520 nm to 950 nm.
16. The system of claim 13 , wherein the memory further comprises instructions, which when executed by the processor, cause the processor to perform operations comprising:
normalizing the identified portion of the tensor data representing the hyperspectral image, and
wherein providing the identified portion of the tensor data representing the hyperspectral image as an input to the feature extraction model comprises providing the normalized portion of the tensor data as the input to the feature extraction model.
17. A non-transitory, computer-readable medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining tensor data representing a hyperspectral image, wherein the hyperspectral image includes a first portion that depicts an object and a second portion that depicts at least a portion of a surrounding environment where the object is located, wherein the object is a first type, and wherein the hyperspectral image captures data within a wavelength range, and wherein the data within the wavelength range is usable for inferencing conditions of objects of the first type;
identifying a portion of the tensor data representing the hyperspectral image that corresponds to the first portion of the hyperspectral image;
providing the identified portion of the tensor data representing the hyperspectral image as an input to a feature extraction model, wherein the feature extraction model was trained using a training dataset to process portions of tensor data of other hyperspectral images to determine one or more matrix structures associated with the other hyperspectral images;
obtaining the one or more matrix structures as output from the feature extraction model, wherein the one or more matrix structures represent a subset of features extracted from the identified portion of the tensor data representing the hyperspectral image, and wherein the subset of the features extracted from the identified portion comprises distinguishing features usable to inference a current condition of the object; and
storing the one or more matrix structures in a memory device.
18. The computer-readable medium of claim 17 , wherein the object depicted by the hyperspectral image is a food item, the food item being one of a citrus fruit, a mango, an apple, a berry, a stone fruit, a tomato, meat, or a vegetable, and wherein the current condition of the object is current level of ripeness.
19. The computer-readable medium of claim 17 , further comprising instructions, which when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, from a hyperspectral sensor, the hyperspectral image, wherein the hyperspectral sensor is configured to detect light waves having the wavelength range, wherein the hyperspectral sensor is configured to automatically select the wavelength range in response to identifying the type of the object, and wherein the wavelength range is between approximately 520 nm to 950 nm.
20. The computer-readable medium of claim 17 , further comprising instructions, which when executed by the one or more processors, cause the one or more processors to perform operations comprising:
normalizing the identified portion of the tensor data representing the hyperspectral image, and
wherein providing the identified portion of the tensor data representing the hyperspectral image as an input to the feature extraction model comprises providing the normalized portion of the tensor data as the input to the feature extraction model.