IP Library › Granted Patent US 10,705,525
Granted Patent B2
US 10,705,525 · App. 15/939,116 · Granted Jul 7, 2020

Performing autonomous path navigation using deep neural networks

Inventors: Nikolai Smolyanskiy (Seattle, WA); Alexey Kamenev (Bellevue, WA); Jeffrey David Smith (Duvall, WA); Stanley Thomas Birchfield (Sammamish, WA)
Assignee: NVIDIA Corporation
G05D1/0088B62D6/001B62D15/025G05D1/024G05D1/0221G05D1/0242G05D1/0246G05D1/0255G05D1/0257G05D1/0268G05D1/102G06K9/00G06K9/00791G06K9/00986G06K9/6273G06N3/04G06N3/08G06N3/084G06N7/005G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,705,525
App. No.
15/939,116
Granted
Jul 7, 2020
Kind
B2
Abstract

A method, computer readable medium, and system are disclosed for performing autonomous path navigation using deep neural networks. The method includes the steps of receiving image data at a deep neural network (DNN), determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data, and controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

Claims (30)

1. A method, comprising:

receiving image data at a deep neural network (DNN);

determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data; and

controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

2. The method of claim 1 , wherein the image data is one or more of optical data, infrared data, light detection and ranging (LIDAR) data, radar data, depth data, and sonar data.

3. The method of claim 1 , wherein the DNN includes a supervised classification network.

4. The method of claim 1 , wherein the DNN implements a loss function.

5. The method of claim 1 , wherein the orientation with respect to the path includes a probability that a vehicle is currently facing left with respect to the path, a probability that a vehicle is currently facing right with respect to the path, and a probability that a vehicle is currently facing straight with respect to the path.

6. The method of claim 1 , wherein the lateral position with respect to the path includes a probability that a vehicle is currently shifted left with respect to the path, a probability that a vehicle is currently shifted right with respect to the path, and a probability that a vehicle is centered with respect to the path.

7. The method of claim 1 , wherein the orientation and lateral position are determined in real-time within the vehicle.

8. The method of claim 1 , wherein controlling the location of the vehicle includes converting the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path into steering directions.

9. The method of claim 1 , wherein the DNN controls directional stability by utilizing predictions with a reduced confidence.

10. The method of claim 1 , wherein a second DNN performs object detection within the path.

11. The method of claim 1 , wherein a third DNN performs obstacle detection associated with the path.

12. A system comprising:

a processor that is configured to:

receive image data at a deep neural network (DNN);

determine, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data; and

control a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

13. The system of claim 12 , wherein the image data is one or more of optical data, infrared data, light detection and ranging (LIDAR) data, radar data, depth data, and sonar data.

14. The system of claim 12 , wherein the DNN includes a supervised classification network.

15. The system of claim 12 , wherein the DNN implements a loss function.

16. The system of claim 12 , wherein the orientation with respect to the path includes a probability that a vehicle is currently facing left with respect to the path, a probability that a vehicle is currently facing right with respect to the path, and a probability that a vehicle is currently facing straight with respect to the path.

17. The system of claim 12 , wherein the lateral position with respect to the path includes a probability that a vehicle is currently shifted left with respect to the path, a probability that a vehicle is currently shifted right with respect to the path, and a probability that a vehicle is centered with respect to the path.

18. The system of claim 12 , wherein the orientation and lateral position are determined in real-time within the vehicle.

19. The system of claim 12 , wherein controlling the location of the vehicle includes converting the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path into steering directions.

20. A computer-readable storage medium storing instructions that, when executed by a processor, causes the processor to perform steps comprising:

receiving image data at a deep neural network (DNN);

determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data; and

controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 18, 2018
From: SMOLYANSKIY, NIKOLAI; KAMENEV, ALEXEY; SMITH, JEFFREY DAVID; BIRCHFIELD, STANLEY THOMAS
To: NVIDIA CORPORATION
Reel/Frame 046122/0060 →
Continuity (2)
Provisional Application 62483155 · Apr 7, 2017
Related Publication 20180292825A1 · Oct 11, 2018
Cited By (3)
US 12,198,396 US 12,384,410 US 12,606,209