IP Library › Granted Patent US 12682653
Granted Patent B2
US 12682653 · App. 17/943,576 · Granted Jul 14, 2026

Techniques for identifying occluded objects using a neural network

Inventors: Siva Kumar Sastry Hari (Sunnyvale, CA); Jason Lavar Clemons (Leander, TX); Timothy Kohchih Tsai (Santa Clara, CA)
Assignee: NVIDIA Corporation
G06V20/58G06V10/774G06V10/776G06V10/82G06V2201/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12682653
App. No.
17/943,576
Granted
Jul 14, 2026
Kind
B2
Abstract

In various examples, techniques for detecting occluded objects within an environment are described. For instance, systems and methods may receive training data representing images and ground truth data indicating whether the images are associated with occluded objects or whether the images are not associated with occluded objects. The systems and methods may then train a neural network to detect occluded objects using the training data and the ground truth data. After training, the systems and methods may use the neural network to detect occluded objects within an environment. For instance, while a vehicle is navigating, the vehicle may process sensor data using the neural network. The neural network may then output data indicating whether an object is located within the environment and occluded from view of the vehicle. In some examples, the neural network may further output additional information associated with the occluded object.

Claims (77)

1 . A method comprising:

receiving training data representing an image depicting one or more first objects located within an environment;

receiving ground truth data associated with the training data, the ground truth data indicating a first location within the image at which a second object that is occluded by the one or more first objects would be depicted and a second location within the image that depicts an indicator of the second object that is occluded;

generating, using a neural network and based at least on the training data, output data indicating a third location within the image that is associated with the second object and a fourth location within the image that is associated with the indicator; and

training the neural network to detect occluded objects based at least on a first difference between the first location within the image at which the second object would be depicted and the third location within the image that is associated with the second object and a second difference between the second location within the image that depicts the indicator and the fourth location within the image that is associated with the indicator.

2 . The method of claim 1 , wherein the training of the neural network to detect the occluded objects comprises:

determining one or more losses based at least on the first difference and the second difference; and

updating one or more parameters associated with the neural network based at least on the one or more losses.

3 . The method of claim 1 , wherein:

the indicator includes at least one of:

a shadow created by the second object;

a reflection of the second object;

light being emitted by the second object;

a partial view of the second object through a transparent material.

4 . The method of claim 1 , wherein:

the training data is simulated image data; and

the image is generated by occluding a real-world object depicted by a real-world image with a simulated object.

5 . The method of claim 1 , wherein:

the training data is real-world image data obtained using a sensor of a vehicle; and

the ground truth data is generated using at least one or more inputs or a second neural network that processes the real-world image data.

6 . The method of claim 1 , wherein:

the first location includes a first bounding shape associated with the second object; and

the third location includes a second bounding shape associated with the second object.

7 . The method of claim 1 , wherein:

the first location includes a first bounding shape within the image at which the second object would be depicted;

the second location includes a second bounding shape within the image that depicts the indicator; and

the ground truth data further indicates that the indicator is associated with the second object.

8 . The method of claim 1 , wherein:

the first location includes a first bounding shape that is associated with a first portion of the image; and

the second location includes a second bounding shape that is associated with a second portion of the image that is separate from the first portion of the image.

9 . A system comprising:

one or more processors to:

receive training data representing an image depicting one or more first objects located within an environment;

receive ground truth data associated with the training data, the ground truth data indicating a first location within the image at which a second object that is occluded by the one or more first objects would be depicted and a second location within the image that depicts an indicator of the second object that is occluded;

generate, using a neural network and based at least on the training data, output data indicating a third location within the image that is associated with the second object and a fourth location within the image that is associated with the indicator; and

train the neural network to detect occluded objects based at least on a first difference between the first location within the image at which the second object would be depicted and the third location within the image that is associated with the second object and a second difference between the second location within the image that depicts the indicator and the fourth location within the image that is associated with the indicator.

10 . The system of claim 9 , wherein to train the neural network to detect the occluded objects comprises:

determining one or more losses based at least on the first difference and the second difference; and

updating one or more parameters associated with the neural network based at least on the one or more losses.

11 . The system of claim 9 , wherein the indicator includes at least one of:

a shadow created by the second object;

a reflection of the second object;

light being emitted by the second object;

a partial view of the second object through a transparent material.

12 . The system of claim 9 , wherein:

the training data is simulated image data; and

the image is generated by occluding a real-world object depicted by a real-world image with a simulated object.

13 . The system of claim 9 , wherein:

the training data is real-world image data obtained using a sensor of a vehicle; and

the ground truth data is generated using at least one or more inputs or a second neural network that processes the real-world image data.

14 . The system of claim 9 , wherein:

the first location includes a first bounding shape associated with the second object; and

the third location includes a second bounding shape associated with the second object.

15 . The system of claim 9 , wherein:

the first location includes a first bounding shape within the image at which the second object would be depicted;

the second location includes a second bounding shape within the image that depicts the indicator; and

the ground truth data further indicates that the indicator is associated with the second object.

16 . The system of claim 9 , wherein:

the first location includes a first bounding shape that is associated with a first portion of the image; and

the second location includes a second bounding shape that is associated with a second portion of the image that is separate from the first portion of the image.

17 . One or more processors comprising processing circuitry to:

receive training data representing an image depicting one or more first objects located within an environment;

receive ground truth data associated with the training data, the ground truth data indicating a first location within the image at which a second object that is occluded by the one or more first objects would be depicted and a second location within the image that depicts an indicator of the second object that is occluded;

generate, using a neural network and based at least on the training data, output data indicating a third location within the image that is associated with the second object and a fourth location within the image that is associated with the indicator; and

train the neural network to detect occluded objects based at least on a first difference between the first location within the image at which the second object would be depicted and the third location within the image that is associated with the second object and a second difference between the second location within the image that depicts the indicator and the fourth location within the image that is associated with the indicator.

18 . The one or more processors of claim 17 , wherein to train the neural network to detect the occluded objects comprises:

determining one or more losses based at least on the first difference and the second difference; and

updating one or more parameters associated with the neural network based at least on the one or more losses.

19 . The one or more processors of claim 17 , wherein the indicator includes at least one of:

a shadow created by the second object;

a reflection of the second object;

light being emitted by the second object;

a partial view of the second object through a transparent material.

20 . The one or more processors of claim 17 , wherein:

the first location includes a first bounding shape within the image at which the second object would be depicted;

the second location includes a second bounding shape within the image that depicts the indicator; and

the ground truth data further indicates that the indicator is associated with the second object.