IP Library › Granted Patent US 11,995,842
Granted Patent B2
US 11,995,842 · App. 17/383,293 · Granted May 28, 2024

Segmentation to improve chemical analysis

Inventors: Hongxu Ma (Mountain View, CA); Allen Richard Zhao (Mountain View, CA); Cyrus Behroozi (Menlo Park, CA); Derek Werdenberg (Sunnyvale, CA); Jie Jacquot (San Carlos, CA); Vadim Tschernezki (Schönefeld, DE)
Assignee: X Development LLC
G06T7/174A23L33/10G06N3/04G06N20/20G06V20/13A23V2002/00G06T2207/10036G06T2207/30128G06V20/194
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,995,842
App. No.
17/383,293
Filed
Jul 22, 2021
Granted
May 28, 2024
Kind
B2
Art Unit
2665
USPC
382/173
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for image segmentation and chemical analysis using machine learning. In some implementations, a system obtains a hyperspectral image that includes a representation of an object. The system segments the hyperspectral image to identify regions of a particular type on the object. The system generates a set of feature values derived from image data for different wavelength bands that is located in the hyperspectral image in the identified regions of the particular type. The system generates a prediction of a level of one or more chemicals in the object based on an output produced by a machine learning model in response to the set of feature values being provided as input to the machine learning model. The system provides data indicating the prediction of the level of the one or more chemicals in the object.

Claims (48)

1. A method performed by one or more computers, the method comprising:

obtaining, by the one or more computers, a hyperspectral image that includes a representation of one or more objects, the hyperspectral image comprising image data for each of multiple wavelength bands;

retrieving segmentation profile data from a storage, the segmentation profile data comprises of different predetermined subsets of the wavelength bands, wherein each subset of wavelength bands is defined to include multiple wavelength bands to amplify a region of the hyperspectral image indicative of a particular chemical of one or more chemicals;

generating, for each of the different predetermined subsets of the wavelength bands, a composite image representative of the particular chemical of the one or more chemicals in the hyperspectral image of the one or more objects;

segmenting, based on the segmentation profile data, the composite image to identify regions of the particular chemical of the one or more chemicals;

generating, by the one or more computers, a set of feature values derived from the image data for different wavelength bands that is located in the hyperspectral image in the identified regions of the one or more chemicals;

generating, by one or more machine learning models, a prediction of a level of the one or more chemicals in the hyperspectral image, wherein the prediction is generated by the one or more machine learning models in response to the set of feature values being provided as input to the one or more machine learning models; and

providing, by the one or more computers, data indicating the prediction of the level of the one or more chemicals in the hyperspectral image of the one or more objects.

2. The method of claim 1 , wherein the set of feature values comprises an average intensity value, for each of a plurality of the wavelength bands, for pixels in the identified regions of the particular chemical of the one or more chemicals.

3. The method of claim 1 , wherein generating the set of feature values comprises:

combining the image data for a plurality of the wavelength bands into a composite image according to a predetermined function; and

generating, as one of the feature values in the set of feature values, an average of pixel intensity values for the composite image.

4. The method of claim 1 , wherein the set of feature values includes a feature value for each of a set of wavelength bands, wherein the set of wavelength bands is different from the subset of the wavelength bands that is designated for segmenting the regions of the particular chemical of the one or more chemicals.

5. The method of claim 1 , wherein the one or more machine learning models have been trained to output scores indicative of predicted chemical composition of the one or more objects represented in hyperspectral images in response to receiving feature values derived from image data for multiple different wavelength bands of the hyperspectral image.

6. The method of claim 1 , wherein the hyperspectral image includes the image data for one or more wavelength bands outside a visible wavelength range.

7. The method of claim 1 , wherein the one or more machine learning models have been trained to indicate a predicted level of an additive or contaminant in a plastic, and wherein the prediction of the level of the one or more chemicals is a prediction of the level of the additive or contaminant.

8. The method of claim 7 , wherein the prediction of the level of the additive or contaminant is an estimated mass fraction of the additive or contaminant.

9. The method of claim 1 , comprising sorting plastic objects from a waste stream based on the level of the one or more chemicals predicted for the respective plastic objects.

10. The method of claim 1 , wherein the one or more machine learning models include a decision tree or a neural network.

11. A system comprising:

one or more computers; and

one or more computer-readable media storing instructions that are operable, when executed by the one or more computers, to cause the system to perform operations comprising:

obtain a hyperspectral image that includes a representation of one or more objects, the hyperspectral image comprising image data for each of multiple wavelength bands;

retrieve segmentation profile data from a storage, the segmentation profile data comprises of different predetermined subsets of the wavelength bands, wherein each subset of wavelength bands is defined to include multiple wavelength bands to amplify a region of the hyperspectral image indicative of a particular chemical of one or more chemicals;

generate, for each of the different predetermined subsets of the wavelength bands, a composite image representative of the particular chemical of the one or more chemicals in the hyperspectral image of the one or more objects;

segment, based on the segmentation profile data, the composite image to identify regions of the particular chemical of the one or more chemicals;

generate a set of feature values derived from the image data for different wavelength bands that is located in the hyperspectral image in the identified regions of the one or more chemicals;

generate, by one or more machine learning models, a prediction of a level of the one or more chemicals in the hyperspectral image, wherein the prediction is generated by the one or more machine learning models in response to the set of feature values being provided as input to the one or more machine learning models; and

provide data indicating the prediction of the level of the one or more chemicals in the hyperspectral image of the one or more objects.

12. One or more non-transitory computer-readable media storing instructions that are operable, when executed by one or more computers, to cause a system to perform operations comprising:

obtaining, by the one or more computers, a hyperspectral image that includes a representation of one or more objects, the hyperspectral image comprising image data for each of multiple wavelength bands;

retrieving segmentation profile data from a storage, the segmentation profile data comprises of different predetermined subsets of the wavelength bands, wherein each subset of wavelength bands is defined to include multiple wavelength bands to amplify a region of the hyperspectral image indicative of a particular chemical of one or more chemicals;

generating, for each of the different predetermined subsets of the wavelength bands, a composite image representative of the particular chemical of the one or more chemicals in the hyperspectral image of the one or more objects;

segmenting, based on the segmentation profile data, the composite image to identify regions of the particular chemical of the one or more chemicals;

generating, by the one or more computers, a set of feature values derived from the image data for different wavelength bands that is located in the hyperspectral image in the identified regions of the one or more chemicals;

generating, by the one or more machine learning models, a prediction of a level of the one or more chemicals in the hyperspectral image, wherein the prediction is generated by the one or more machine learning models in response to the set of feature values being provided as input to the one or more machine learning models; and

providing, by the one or more computers, data indicating the prediction of the level of the one or more chemicals in the hyperspectral image of the one or more objects.

13. The method of claim 1 , wherein the one or more objects include plastic bottles made up of different resins.

14. The method of claim 1 , wherein the one or more chemicals include plastic resins, additives, and contaminants,

wherein the resins include polyethylene (PE), polyethylene terephthalate (PET), polyvinyl chloride (PVC), or polypropylene (PP),

wherein the additives include bromides, chlorates, phthalates, or UV-resistant coatings, and

wherein the contaminants include oils or food residue.

15. The method of claim 1 , further comprising:

generating, for each subset of the different predetermined subsets of the wavelength bands, a synthetic band corresponding to the particular chemical of the one or more chemicals.

16. The system of claim 11 , wherein the machine learning model has been trained to predict a level of an additive or contaminant in a plastic.

17. The system of claim 11 , further comprising sorting plastic objects from a waste stream based on the level of the one or more chemicals predicted for the respective plastic objects.

18. The system of claim 11 , wherein the machine learning model has been trained to output scores indicative of predicted chemical composition of the one or more objects represented in hyperspectral images in response to receiving feature values derived from image data for multiple different wavelength bands of the hyperspectral image.

19. The system of claim 11 , wherein the set of feature values comprises an average intensity value, for each of a plurality of the wavelength bands, for pixels in the identified regions of the particular chemical of the one or more chemicals.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2021
From: MA, HONGXU; ZHAO, ALLEN RICHARD; BEHROOZI, CYRUS; WERDENBERG, DEREK; JACQUOT, JIE; TSCHERNEZKI, VADIM
To: X DEVELOPMENT LLC
Reel/Frame 056966/0126 →
Continuity (1)
Related Publication 20230027514A1 · Jan 26, 2023
Cited By (2)
US 12,289,538 US 12,339,890