Granted Patent B2

US 12,728,883 · App. 17/483,019 · Granted Sep 8, 2026

Inventors: Aidin Ehsanibenafati (Santa Clara, CA); Jonas Nilsson (Los Gatos, CA); Amir Akbarzadeh (San Jose, CA); Hae Jong Seo (Campbell, CA)

Assignee: Nvidia Corporation

B60W60/001G06F18/25B60W2420/403B60W2420/408B60W2552/53

Abstract

Claims (69)

1 . A method, comprising

generating, using at least two sources of localization data, a fused localization determination, the at least two sources of localization data fused based in part on weightings determined using relative confidence values associated with respective sources;

comparing the fused localization determination against at least two trusted and independent sources of localization determinations to verify the fused localization determination;

generating a navigation path, based at least on verification of the fused localization determination and using at least two sources of environment perception; comparing the navigation path against at least two trusted and independent sources of navigation path data and the fused localization determination to verify the navigation path;

determining that the navigation path is verified upon receiving two or more votes from at least two comparators, each comparator evaluating the navigation path against data from the at least two trusted and independent sources of navigation path data respectively; and

providing, based at least on the navigation path being verified, the navigation path to a control system to navigate an object within the environment.

2 . The method of claim 1 , wherein the object is a vehicle, and wherein the navigation path comprises a lane graph generated using at least one source of map data and at least one source of environmental perception corresponding to the vehicle.

3 . The method of claim 1 , wherein the sources of localization data include at least one of: a camera, global positioning system (GPS), or radar system.

4 . The method of claim 1 , wherein the sources of environment perception include at least one of: a camera, a radar system, or a LIDAR system, an ultrasonic system, or high definition (HD) map data.

5 . The method of claim 1 , further comprising:

determining a confidence of the fused localization determination before performing the comparing against the at least two trusted and independent localization determinations; and

replacing the fused localization determination with a most confident localization value from the at least two sources of localization determinations if the confidence falls below a confidence threshold.

6 . The method of claim 1 , further comprising:

determining a confidence of the navigation path before performing the comparing against the at least two trusted and independent sources of navigation path data; and

replacing the navigation path with navigation data from the at least two trusted and independent sources of navigation path data corresponding to a highest confidence level if the confidence falls below a confidence threshold.

7 . The method of claim 1 , further comprising:

determining that at least one source of localization determination is temporarily unavailable or unreliable; and

utilizing historical localization data in place of the at least one source of localization determination.

8 . The method of claim 1 , further comprising:

dynamically adjusting a selection of the at least two sources of localization determinations or the at least two sources of environment perception based upon an availability or confidence determination.

9 . The method of claim 1 , further comprising:

generating a second navigation path using a parallel path generator; and

determining, dynamically, whether to provide the navigation path or the second navigation path for navigation of the object.

10 . A system, comprising:

one or more processing units to:

generate, using at least two sources of localization data, a fused localization determination, the at least two sources of localization data fused based in part on weightings determined using relative confidence values associated with respective sources;

compare the fused localization determination against at least two trusted and independent localization values to verify the fused localization determination;

generate a fused navigation path, based at least on verification of the fused localization determination and using at least two sources of environment perception;

compare the fused navigation path against at least two trusted and independent sources of navigation path data and the fused localization determination to verify the fused navigation path;

determine that the fused navigation path is verified upon receiving two or more votes from at least two comparators, each comparator evaluating the fused navigation path against data from the at least two trusted and independent sources of navigation path data respectively; and

provide, based at least on the trust in the fused navigation path being verified, the fused navigation path to a control system to navigate an object within the environment.

11 . The system of claim 10 , wherein the one or more processing units are further to:

determine a confidence of the fused localization determination before performing the comparing against the at least two trusted and independent localization values;

replace the fused localization determination with a most confident localization value from the at least two sources of localization data if the confidence falls below a confidence threshold;

determine a confidence of the fused navigation path before performing the comparing against the at least two trusted and independent sources of navigation path data; and

replace the fused navigation path with most confident navigation data from the at least two trusted and independent sources of navigation data if the confidence falls below a confidence threshold.

12 . The system of claim 10 , wherein one or more processing units are further to:

determine that at least one source of localization data is temporarily unavailable or unreliable; and

utilize historical localization data in place of the at least one source of localization data.

13 . The system of claim 10 , wherein one or more processing units are further to:

generate a second navigation path using a parallel path generator; and

determine, dynamically, whether to provide the fused navigation path or the second navigation path for navigation of the object.

14 . The system of claim 10 , wherein the system comprises at least one of:

a system for performing simulation operations;

a system for performing simulation operations to test or validate autonomous machine applications;

a system for rendering graphical output;

a system for performing deep learning operations;

a system implemented using an edge device;

a system incorporating one or more Virtual Machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

15 . A processor, comprising:

one or more processing units to:

generate a verified localization determination that is fused using at least two sources of localization data and compared against at least two trusted and independent localization values;

generate a verified navigation path for an environment using at least the verified localization determination, wherein the navigation path is verified upon receiving two or more votes from at least two comparators, each comparator evaluating the navigation path against data from at least two trusted and independent sources of navigation path data respectively and the verified localization determination; and

provide the verified navigation path to a control system to navigate an object within the environment.

16 . The processor of claim 15 , wherein at least one processing unit of the one or more processing units is further configured to:

verify the verified localization determination by comparing an unverified localization determination against the at least two trusted and independent localization values;

generate an unverified navigation path using at least two sources of environment perception and the verified localization determination; and

verify the verified navigation path by comparing the unverified navigation path against the at least two trusted and independent sources of navigation path data.

17 . The processor of claim 16 , wherein at least one processing unit of the one or more processing units is further configured to:

determine a confidence of the unverified localization determination before performing the comparing against the at least two trusted and independent localization values; and

replacing the unverified localization determination with a most confident localization value from the at least two sources of localization data if the confidence falls below a confidence threshold.

18 . The processor of claim 16 , wherein at least one processing unit of the one or more processing units is further configured to:

determine a confidence of the unverified navigation path before performing the comparing against the at least two trusted and independent sources of navigation path data; and

replacing the unverified navigation path with navigation data from the at least two trusted and independent sources of navigation path data corresponding to a highest confidence level if the confidence falls below a confidence threshold.

19 . The processor of claim 15 , wherein the object is a vehicle, and wherein the verified navigation path comprises a lane graph generated using at least one source of map data and at least one source of environmental perception corresponding to the vehicle.

20 . The processor of claim 16 , wherein the sources of localization data include at least one of: a camera, global positioning system (GPS), or radar system; and

wherein the sources of environment perception include at least one of: a camera, a radar system, or a LIDAR system, an ultrasonic system, or high definition (HD) map data.

Assignments (1)

Continuity (2)

References Cited (23)