Granted Patent B2

US 12,233,784 · App. 18/202,120 · Granted Feb 25, 2025

Inventors: Sai Akhil Suggu (Cupertino, CA); Inderjot Singh Saggu (Cupertino, CA)

Assignee: PlusAI, Inc.

B60R1/20G06V10/82G06V20/597

Abstract

Claims (70)

1. A method for augmenting training images for model training, comprising:

at a computer system including one or more processors and memory:

obtaining a driver image and a background image selected from a corpus of background images, where the background image comprises an image of an interior of a first vehicle and the driver image comprises an image of a driver extracted from an image of an interior of a second vehicle;

generating a first image by overlaying the driver image at a first position of the background image;

generating a second image by overlaying the driver image at a second position of the background image; and

adding the first image and the second image to a corpus of training images to generate a model for autonomously monitoring vehicle drivers.

2. The method of claim 1 , further comprising:

obtaining an initial image of a first driver in an interior of a first vehicle; and

separating, from the initial image, the driver image from a driver background image of the interior of the first vehicle.

3. The method of claim 2 , separating the driver image from the driver background image, further comprising:

applying a segmentation model to generate a segmentation mask that (1) associates a plurality of first pixels of the initial image with the driver image or (2) associates a plurality of second pixels of the initial image with the driver background image.

4. The method of claim 3 , wherein the segmentation model includes a U-Net that is based on a fully convolutional network.

5. The method of claim 1 , further comprising:

training the model for autonomously monitoring vehicle drivers to determine whether a vehicle driver is looking forward at a road ahead of a vehicle.

6. The method of claim 1 , further comprising:

training the model for autonomously monitoring vehicle drivers to determine whether a vehicle driver is looking forward at a road, looking to the left, looking to the right, looking down, closing eyes, or talking.

7. The method of claim 1 , further comprising, prior to overlaying the driver image onto the background image, implementing one or more of:

adjusting one or more image properties of at least one of the driver image and the background image to match lighting conditions of the driver image and the background image; and

normalizing at least one of the driver image and the background image to match average brightness levels of the driver image and the background image.

8. The method of claim 1 , further comprising, prior to overlaying the driver image onto the background image,

scaling at least one of the driver image and the background image.

9. The method of claim 8 , wherein the driver image includes a first driver image of a first driver, the method further comprising:

obtaining a second driver image of a second driver distinct from the first driver;

generating a third image by overlaying the second driver image onto the background image; and

adding the third image to the corpus of training images to generate the model for monitoring vehicle drivers, wherein the corpus of training images includes the first image and the second image.

10. The method of claim 1 , further comprising:

collecting a plurality of background images;

mapping each of the background images onto a respective point in a multidimensional space having a distance metric d;

clustering the plurality of background images using the distance metric d to form a plurality of image clusters;

for each of the image clusters, identifying one or more background images in the image cluster that are most distant according to the distance metric d;

forming a set of candidate background images comprising the identified one or more most distant background images in each of the image clusters; and

selecting the background image from the set of candidate background images.

11. The method of claim 10 , wherein clustering the plurality of background images comprises:

selecting a positive integer number K;

selecting K cluster centers; and

for each of the plurality of background images:

determining a distance of the respective background image from each of the cluster centers; and

assigning the respective background image to a respective image cluster associated with a respective cluster center to which the respective background image has a shortest distance.

12. The method of claim 1 , further comprising:

collecting a plurality of background images;

mapping each of the background images onto a respective point in a multidimensional space having a distance metric d;

clustering the plurality of background images using the distance metric d to form a plurality of image clusters; and

for each of the plurality of background images, determining, on a clustering plot, a respective distance between the respective background image and a corresponding cluster center of an image cluster to which the respective background image belongs, wherein the background image is selected from the plurality of background images based on the respective distance between the background image and the corresponding cluster center.

13. The method of claim 1 , further comprising:

training the model for autonomously monitoring vehicle drivers to determine whether a vehicle driver is sitting on a driver seat or a passenger seat, and in accordance with a determination whether the vehicle driver is sitting on the driver eat or the passenger seat, classify the vehicle driver as a distracted driver or a distracted passenger.

14. A computer system, comprising:

one or more processors; and

memory storing one or more programs configured for execution by the one or more processors, the one or more programs comprising instructions for:

obtaining a driver image and a background image selected from a corpus of background images, where the background image comprises an image of an interior of a first vehicle and the driver image comprises an image of a driver extracted from an image of an interior of a second vehicle;

generating a first image by overlaying the driver image at a first position of the background image;

generating a second image by overlaying the driver image at a second position of the background image; and

adding the first image and the second image to a corpus of training images to generate a model for autonomously monitoring vehicle drivers.

15. The computer system of claim 14 , the one or more programs further comprising instructions for:

obtaining an initial image of a first driver in an interior of a first vehicle; and

separating, from the initial image, the driver image from a driver background image of the interior of the first vehicle.

16. The computer system of claim 15 , separating the driver image from the driver background image, further comprising:

applying a segmentation model to generate a segmentation mask that ( 1 ) associates a plurality of first pixels of the initial image with the driver image or ( 2 ) associates a plurality of second pixels of the initial image with the driver background image.

17. The computer system of claim 14 , the one or more programs further comprising instructions for:

training the model for autonomously monitoring vehicle drivers to determine whether a vehicle driver is looking forward at a road ahead of a vehicle.

18. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a computer system, the one or more programs comprising instructions for:

obtaining a driver image and a background image selected from a corpus of background images, where the background image comprises an image of an interior of a first vehicle and the driver image comprises an image of a driver extracted from an image of an interior of a second vehicle;

generating a first image by overlaying the driver image at a first position of the background image;

generating a second image by overlaying the driver image at a second position of the background image; and

adding the first image and the second image to a corpus of training images to generate a model for autonomously monitoring vehicle drivers.

19. The non-transitory computer-readable storage medium of claim 18 , further comprising instructions for:

training the model for autonomously monitoring vehicle drivers to determine whether a vehicle driver is looking forward at a road, looking to the left, looking to the right, looking down, closing eyes, or talking.

20. The non-transitory computer-readable storage medium of claim 18 , further comprising instructions for, prior to overlaying the driver image onto the background image, implementing one or more of:

adjusting one or more image properties of at least one of the driver image and the background image to match lighting conditions of the driver image and the background image;

normalizing at least one of the driver image and the background image to match average brightness levels of the driver image and the background image; and

scaling at least one of the driver image and the background image.

Continuity (2)

References Cited (38)