IP Library › Granted Patent US 10,984,311
Granted Patent B2
US 10,984,311 · App. 16/287,579 · Granted Apr 20, 2021

Involved generative machine learning models for functional testing

Inventors: Nanxiang Li (San Mateo, CA); Bilal Alsallakh (Sunnyvale, CA); Liu Ren (Cupertino, CA)
Assignee: Robert Bosch GmbH
G06N3/0454G06K9/00664G06K9/2081G06K9/6253G06K9/6256G06K9/6263G06N3/088G06N5/04G06K9/00791
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,984,311
App. No.
16/287,579
Granted
Apr 20, 2021
Kind
B2
Abstract

A system includes a display device, a memory configured to store a visual analysis application and image data including a plurality of images including detectable objects; and a processor, operatively connected to the memory and the display device. The processor is configured to execute the visual analysis application to learn generative factors from objects detected in the plurality of images, visualize the generative factors in a user interface provided to the display device, receive grouped combinations of the generative factors and values to apply to the generative factors to control object features, create generated objects by applying the values of the generative factors to the objects detected in the plurality of images, combine the generated objects into the original images to create generated images, and apply a discriminator to the generated images to reject unrealistic images.

Claims (47)

1. A system comprising:

a display device;

a memory configured to store a visual analysis application and image data including a plurality of images that include detectable objects; and

a processor, operatively connected to the memory and the display device, and configured to execute the visual analysis application to

learn generative factors from objects detected in the plurality of images,

visualize the generative factors in a user interface provided to the display device,

receive grouped combinations of the generative factors and values to apply to the generative factors to control object features,

create generated objects by applying the values of the generative factors to the objects detected in the plurality of images,

combine the generated objects into the plurality of images to create generated images, and

apply a discriminator to the generated images to reject unrealistic images.

2. The system of claim 1 , wherein the processor is further configured to execute the visual analysis application to apply the generated images to an artificial intelligence (AI) model to determine whether the generated objects are correctly detected.

3. The system of claim 2 , wherein the processor is further programmed to include, in the user interface, detection results using the AI model indicative of whether an object is correctly detected in one of the plurality of images and detection results using the AI model indicative of whether a generated object is correctly detected in one of the generated images.

4. The system of claim 1 , wherein the processor is further programmed to utilize disentangled representation learning with a Variational Auto-Encoder (VAE) to learn the generative factors.

5. The system of claim 1 , wherein the processor is further programmed to include, in the user interface, a factor display of the generative factors in which each generative factor is displayed as an average image of the detected objects, perturbed for each of a plurality of values along a scale of values for the respective generative factor.

6. The system of claim 1 , wherein the processor is further programmed to include, in the user interface, for a selected factor, a set of generated images with a value of the selected factor at a first value and a second set of generated images with a value of the selected factor at a second value.

7. The system of claim 1 , wherein the processor is further programmed to include, in the user interface, controls for adjustment of the values of the generative factors as a combination of a plurality of the generative factors.

8. The system of claim 1 , wherein the processor is further programmed to include, in the user interface, an original image from the plurality of images including a detectable object, and a generated image including a generated object replacing the detectable object in the original image.

9. A method comprising:

learning generative factors from objects detected in a plurality of images;

visualizing the generative factors in a user interface provided to a display device;

receiving grouped combinations of the generative factors and values to apply to the generative factors to control object features;

creating generated objects by applying the values of the generative factors to the objects detected in the plurality of images;

combining the generated objects into the plurality of images to create generated images; and

applying a discriminator to the generated images to reject unrealistic images.

10. The method of claim 9 , further comprising applying the generated images to an artificial intelligence (AI) model to determine whether the generated objects are correctly detected.

11. The method of claim 10 , further comprising including, in the user interface, detection results using the AI model indicative of whether an object is correctly detected in one of the plurality of images and detection results using the AI model indicative of whether a generated object is correctly detected in one of the generated images.

12. The method of claim 9 , further comprising utilizing disentangled representation learning with a Variational Auto-Encoder (VAE) to learn the generative factors.

13. The method of claim 9 , further comprising including, in the user interface, a factor display of the generative factors in which each generative factor is displayed as an average image of the detected objects, perturbed for each of a plurality of values along a scale of values for the respective generative factor.

14. The method of claim 9 , further comprising including, in the user interface, for a selected factor, a set of generated images with a value of the selected factor at a first value and a second set of generated images with a value of the selected factor at a second value.

15. The method of claim 9 , further comprising including, in the user interface, controls for adjustment of the values of the generative factors as a combination of a plurality of the generative factors.

16. The method of claim 9 , further comprising including, in the user interface, an original image from the plurality of images including a detectable object, and a generated image including a generated object replacing the detectable object in the original image.

17. A non-transitory computer-readable medium comprising instructions of visual analysis application that when executed by one or more processors, cause the one or more processors to:

perform disentangled representation learning with a Variational Auto-Encoder (VAE) to learn generative factors from objects detected in a plurality of images;

visualize the generative factors in a user interface provided to a display device;

receive grouped combinations of the generative factors and values to apply to the generative factors to control object features;

create generated objects by applying the values of the generative factors to the objects detected in the plurality of images;

combine the generated objects into the plurality of images to create generated images; and

apply a discriminator to the generated images to reject unrealistic images.

18. The medium of claim 17 , further comprising instructions to cause the one or more processors to:

apply the generated images to an artificial intelligence (AI) model to determine whether the generated objects are correctly detected; and

include, in the user interface, detection results using the AI model indicative of whether an object is correctly detected in one of the plurality of images and detection results using the AI model indicative of whether a generated object is a correctly detected in one of the generated images.

19. The medium of claim 17 , further comprising instructions to cause the one or more processors to:

include, in the user interface, a factor display of the generative factors in which each generative factor is displayed as an average image of the detected objects, perturbed for each of a plurality of values along a scale of values for the respective generative factor; and

include, in the user interface, for a selected factor from the factor display, a set of generated images with a value of the selected factor at a first value and a second set of generated images with a value of the selected factor at a second value.

20. The medium of claim 17 , further comprising instructions to cause the one or more processors to:

include, in the user interface, controls for adjustment of the values of the generative factors as a combination of a plurality of the generative factors; and

include, in the user interface, an original image from the plurality of images including a detectable object, and a generated image including a generated object replacing the detectable object in the original image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 27, 2019
From: LI, NANXIANG; ALSALLAKH, BILAL; REN, LIU
To: ROBERT BOSCH GMBH
Reel/Frame 048458/0679 →
Continuity (1)
Related Publication 20200272887A1 · Aug 27, 2020