IP Library Granted Patent US 11,610,412
Granted Patent B2
US 11,610,412 · App. 17/024,947 · Granted Mar 21, 2023

Vehicle neural network training

Inventors: Lynn Valerie Keiser (Sunnyvale, CA); Timur Gilmanov (San Jose, CA)
Assignee: Ford Global Technologies, LLC
G06V20/584G06K9/6256G06K9/6277G06N3/0454G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,610,412
App. No.
17/024,947
Granted
Mar 21, 2023
Kind
B2
Abstract

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to determine a vehicle sensor object label, and a vehicle sensor confidence level for the vehicle sensor object label, by processing a vehicle sensor image with a neural network, determine a stationary sensor object label, and a stationary sensor confidence level for the stationary sensor object label, by processing a stationary sensor image with the neural network, and if the stationary sensor confidence level is greater than the vehicle sensor confidence level, operate a vehicle by determining a vehicle path based on the stationary sensor object label.

Claims (28)

1. A computer, comprising:

a processor; and

a memory, the memory including instructions executable by the processor to:

determine a mobile sensor object label, and a mobile sensor confidence level for the mobile sensor object label, by processing a mobile sensor image that includes an object with a neural network;

determine a stationary sensor object label, and a stationary sensor confidence level for the stationary sensor object label, by processing a stationary sensor image that includes the object with the neural network, wherein the mobile sensor object label and the stationary object label are based on different views of the object; and

if the stationary sensor confidence level is greater than the mobile sensor confidence level, operate a vehicle by determining a vehicle path based on the stationary sensor object label.

2. The computer of claim 1 , the instructions including further instructions to retrain the neural network by applying the stationary sensor object label to the vehicle sensor image when the stationary sensor confidence level is greater than the vehicle sensor confidence level.

3. The computer of claim 1 , wherein the vehicle path is based on a polynomial function and the vehicle path avoids contact or near-contact with an object corresponding to the vehicle sensor object label.

4. The computer of claim 1 , wherein the vehicle sensor confidence level and the stationary sensor confidence level are probabilities that the vehicle sensor object label and the stationary sensor object label are correctly identified by the neural network.

5. The computer of claim 1 , wherein the stationary sensor image includes data regarding a location of the stationary sensor object and a time at which the stationary sensor image was acquired.

6. The computer of claim 5 , wherein the location of the stationary sensor object is a real-world location in global coordinates and is based on a location and orientation of the stationary sensor.

7. The computer of claim 6 , the instructions including further instructions to determine that the vehicle sensor object and the stationary sensor object are different views of a same object based on the location of the vehicle sensor object, the location of the stationary sensor object, and the time at which the stationary sensor object was acquired.

8. The computer of claim 1 , wherein the neural network includes convolutional layers and fully connected layers.

9. The computer of claim 1 , wherein the neural network is trained using training datasets that include images of objects and ground truth that includes object labels.

10. The computer of claim 1 , wherein a second neural network included in a traffic infrastructure system is used to determine the stationary sensor object label and the stationary sensor confidence level.

11. The computer of claim 1 , wherein a traffic infrastructure system includes a second computer including a second processor and second memory including instructions executable by the second processor to retrain the neural network by applying the stationary sensor object label to the vehicle sensor image.

12. The computer of claim 11 , wherein the second computer downloads the retrained neural network to the computer.

13. A method, comprising:

determining a mobile sensor object label, and a mobile sensor confidence level for the mobile sensor object label, by processing a mobile sensor that includes an object with a neural network;

determining a stationary sensor object label, and a stationary sensor confidence level for the stationary sensor object label, by processing a stationary sensor image that includes the object with the neural network, wherein the mobile sensor object label and the stationary object label are based on different views of the object; and

if the stationary sensor confidence level is greater than the vehicle sensor confidence level, operating a vehicle by determining a vehicle path based on the stationary sensor object label.

14. The method of claim 13 , further comprising retraining the neural network by applying the stationary sensor object label to the vehicle sensor image when the stationary sensor confidence level is greater than the vehicle sensor confidence level.

15. The method of claim 13 , wherein the vehicle path is based on a polynomial function and the vehicle path avoids contact or near-contact with an object corresponding to the vehicle sensor object label.

16. The method of claim 13 , wherein the vehicle sensor confidence level and the stationary sensor confidence level are probabilities that the vehicle sensor object label and the stationary sensor object label are correctly identified by the neural network.

17. The method of claim 13 , wherein the stationary sensor image includes data regarding a location of the stationary sensor object and a time at which the stationary sensor image was acquired.

18. The method of claim 17 , wherein the location of the stationary sensor object is a real-world location in global coordinates and is based on a location and orientation of the stationary sensor.

19. The method of claim 18 , further comprising determining that the vehicle sensor object and the stationary sensor object are different views of a same object based on the location of the vehicle sensor object, the location of the stationary sensor object, and the time at which the stationary sensor object was acquired.

20. The method of claim 13 , wherein the neural network includes convolutional layers and fully connected layers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 18, 2020
From: KEISER, LYNN VALERIE; GILMANOV, TIMUR
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 053813/0295 →
Continuity (1)
Related Publication 20220092321A1 · Mar 24, 2022