Granted Patent B2

US 12,277,696 · App. 17/716,590 · Granted Apr 15, 2025

Inventors: Laura Beggel (Stuttgart, DE); Filipe J. Cabrita Condessa (Pittsburgh, PA); Robin Hutmacher (Renningen, DE); Jeremy Kolter (Pittsburgh, PA); Nhung Thi Phuong Ngo (Karlsruhe, DE); Fatemeh Sheikholeslami (Pittsburgh, PA); Devin T. Willmott (Pittsburgh, PA)

Assignee: Robert Bosch GmbH

G06T7/0004G06N20/00G06T7/11G06T7/194G06T2207/10024G06T2207/20081

Abstract

Claims (72)

1. A system for performing at least one task with autonomous control of a part, the system comprising:

an image sensor configured to output an image of the part;

an actuator configured to bin the part based on a detected defect in the part;

a processor; and

memory including instructions that, when executed by the processor, cause the processor to:

utilize an image segmenter on a source image stored in the memory to generate a source image segmentation mask having a foreground region and a background region;

utilize the image segmenter on a target image stored in the memory to generate a target image segmentation mask having a foreground region and a background region;

determine a source image foreground and a source image background of the source image based on the source image segmentation mask;

determine a target image foreground and a target image background of the target image based on the target image segmentation mask;

remove the target image foreground from the target image;

insert the source image foreground into the target image with the removed target image foreground to create an augmented image having the source image foreground and the target image background;

update training data of the machine learning model with the augmented image;

utilize the machine learning model with the updated training data to determine a defect in the part; and

actuate the actuator to bin the part based on the determined defect in the part.

2. The system of claim 1 , wherein the instructions, when executed by the processor, cause the processor to remove the target image foreground from the target image background by:

determining an average color of pixels in the target image background; and

changing the pixels in the target image foreground to assume the average color.

3. A computer-implemented method for generating training data for a machine learning model, the computer-implemented method comprising:

selecting a source image from an image database;

selecting a target image from the image database;

utilizing an image segmenter with the source image to generate a source image segmentation mask having a foreground region and a background region;

utilizing the image segmenter with the target image to generate a target image segmentation mask having a foreground region and a background region;

determining a source image foreground and a source image background of the source image based on the source image segmentation mask;

determining a target image foreground and a target image background of the target image based on the target image segmentation mask;

removing the target image foreground from the target image;

inserting the source image foreground into the target image with the removed target image foreground to create an augmented image having the source image foreground and the target image background; and

updating training data of the machine learning model with the augmented image.

4. The computer-implemented method of claim 3 , wherein the step of removing includes:

changing colors of at least a group of pixels in the target image foreground.

5. The computer-implemented method of claim 3 , wherein the step of removing includes:

determining an average color of pixels in the target image background; and

changing the pixels in the target image foreground to assume the average color.

6. The computer-implemented method of claim 3 , further comprising:

determining a color of a pixels in the source image at a location corresponding to the source image foreground; and

wherein the step of inserting includes assigning the color to pixels in the target image at a location corresponding to the target image foreground.

7. The computer-implemented method of claim 3 , wherein:

the source image segmentation mask and the target image segmentation mask are generated with a first color in the respective foreground regions and a second color in the respective background regions.

8. The computer-implemented method of claim 3 , further comprising outputting a trained machine learning model based on the updated training data.

9. The computer-implemented method of claim 8 , further comprising:

receiving an input image of a part from an image sensor;

using the trained machine learning model and the input image to determine a defect is present in the part; and

binning the part based on the determined defect in the part.

10. The computer-implemented method of claim 3 , wherein the source image is part of a group of source images having a common domain, and wherein the step of selecting the source image is based on a probability such that the probability of selecting a given source image decreases as the number of images with the common domain increases.

11. The computer-implemented method of claim 3 , wherein the image segmenter is an image segmentation machine learning model.

12. A system for training a machine learning model, the system comprising:

a computer-readable storage medium configured to store computer-executable instructions; and

one or more processors configured to execute the computer-executable instructions, the computer-executable instructions comprising:

utilizing an image segmenter with a source image to generate a source image segmentation mask having a foreground region and a background region;

utilizing the image segmenter with a target image to generate a target image segmentation mask having a foreground region and a background region;

determining a source image foreground and a source image background of the source image based on the source image segmentation mask;

determining a target image foreground and a target image background of the target image based on the target image segmentation mask;

removing the target image foreground from the target image;

inserting the source image foreground into the target image with the removed target image foreground to create an augmented image having the source image foreground and the target image background; and

updating training data of the machine learning model with the augmented image.

13. The system of claim 12 , wherein the step of removing includes:

changing colors of at least a group of pixels in the target image foreground.

14. The system of claim 12 , wherein the step of removing includes:

determining an average color of pixels in the target image background; and

changing the pixels in the target image foreground to assume the average color.

15. The system of claim 12 , wherein the computer-executable instructions further comprise:

determining a color of a pixels in the source image at a location corresponding to the source image foreground; and

wherein the step of inserting includes assigning the color to pixels in the target image at a location corresponding to the target image foreground.

16. The system of claim 12 , wherein:

the source image segmentation mask and the target image segmentation mask are generated with a first color in the respective foreground regions and a second color in the respective background regions.

17. The system of claim 12 , wherein the computer-executable instructions further comprise:

outputting a trained machine learning model based on the updated training data.

18. The system of claim 17 , wherein the computer-executable instructions further comprise:

receiving an input image of a part from an image source;

using the trained machine learning model and the input image to determine a defect is present in the part; and

outputting a message indicating the presence of the defect in the part.

19. The system of claim 12 , wherein the source image is part of a group of source images having a common domain, and wherein the step of selecting the source image is based on a probability such that the probability of selecting a given source image decreases as the number of images with the common domain increases.

20. The system of claim 12 , wherein the image segmenter is an image segmentation machine learning model.

Assignments (1)

Continuity (1)

References Cited (20)