IP Library Granted Patent US 11,594,014
Granted Patent B2
US 11,594,014 · App. 16/908,251 · Granted Feb 28, 2023

Annotating high definition map points with measure of usefulness for localization

Inventors: Di Zeng (Sunnyvale, CA); Mengxi Wu (Mountain View, CA)
Assignee: NVIDIA CORPORATION
G06V10/757G06T7/32G06T7/97G06V20/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,594,014
App. No.
16/908,251
Granted
Feb 28, 2023
Kind
B2
Abstract

According to an aspect of an embodiment, operations may comprise obtaining a first point cloud that includes a first point. The operations also comprises obtaining a second point cloud that is a copy of the first point cloud and that includes a second point that is a copy of the first point. The operations also comprises moving the second point cloud with respect to the first point cloud according to a first vector. The operations also comprises identifying a closest point of the first point cloud that is closest to the second point of the second point cloud. The operations also comprises determining a second vector between the closest point and the second point. The operations also comprises determining a measure of usefulness of the first point based on the first vector and the second vector. The operations also comprises indicating the measure of usefulness of the first point.

Claims (94)

1. A method, comprising:

moving a first point cloud that includes a first point with respect to a second point cloud based at least on a first vector, the second point cloud being a copy of the first point cloud and including a second point that is a copy of the first point;

identifying a closest point of the first point cloud that is closest to the second point of the second point cloud after moving the second point cloud;

determining a second vector corresponding to a locational relationship between the closest point and the second point; and

determining a measure of usefulness corresponding to the first point with respect to mapping other point clouds to the first point cloud based at least on a relationship between the first vector and the second vector,

wherein a machine performs one or more control operations based at least on the measure of usefulness.

2. The method of claim 1 , wherein

the determining the relationship between the first vector and the second vector is based at least on comparing the first vector to the second vector.

3. The method of claim 1 , wherein the first vector is associated with a first direction and wherein the method further comprises:

moving, based at least on a third vector, a third point cloud with respect to the first point cloud in a second direction different from the first direction, the third point cloud being a second copy of the first point cloud and including a third point that is a second copy of the first point; and

wherein the determining the measure of usefulness corresponding to the first point is further based at least on a second relationship between the first vector and the third vector.

4. The method of claim 1 , wherein the first vector is associated with a first direction and wherein the method further comprises:

moving, based at least on a third vector, a third point cloud with respect to the first point cloud in a second direction different from the first direction, the third point cloud being a second copy of the first point cloud and including a third point that is a second copy of the first point;

determining a second measure of usefulness corresponding to the first point based at least on a second relationship between the first vector and the third vector;

moving, based at least on a fourth vector, a fourth point cloud with respect to the first point cloud in a third direction different from the first direction and the second direction, the fourth point cloud being a third copy of the first point cloud and including a fourth point that is a third copy of the first point;

determining a third measure of usefulness corresponding to the first point based at least on a third relationship between the first vector and the fourth vector; and

determining an aggregate measure of usefulness corresponding to the first point based at least on aggregating two or more of: the measure of usefulness as determined based at least on the relationship between the first vector and the second vector, the second measure of usefulness as determined based at least on the second relationship between the first vector and the third vector, or the third measure of usefulness as determined based at least on the third relationship between the first vector and the fourth vector.

5. The method of claim 1 , further comprising:

annotating map data with the measure of usefulness corresponding to the first point, wherein:

localization of the machine is performed based at least on the annotation of the measure of usefulness, and

performance of the one or more control operations by the machine is based at least on the localization.

6. The method of claim 1 , further comprising:

annotating map data with a weight derived from the measure of usefulness corresponding to the first point, wherein;

localization of the machine is performed based at least on the annotation of the weight, and

performance of the one or more control operations by the machine is based at least on the localization.

7. The method of claim 1 , further comprising:

filtering, during a mapping to the first point cloud of a different point cloud, a plurality of points of the first point cloud based at least on respective measures of usefulness corresponding to the plurality of points such that one or more points corresponding to lower measures of usefulness have a higher likelihood of being filtered out compared to one or more points corresponding to higher measures of usefulness.

8. The method of claim 1 , further comprising assigning a usefulness score to the first point based at least on the measure of usefulness corresponding to the first point.

9. The method of claim 1 , further comprising:

sampling, during a mapping to the first point cloud of a different point cloud, a plurality of points of the first point cloud based at least on respective measures of usefulness corresponding to the plurality of points such that one or more points corresponding to higher measures of usefulness have a higher likelihood of being sampled compared to the one or more points corresponding to lower measures of usefulness.

10. The method of claim 1 , wherein the first vector is determined based at least on a potential movement of the machine.

11. A processor comprising processing circuitry to perform operations comprising:

moving a first point cloud that includes a first point with respect to a second point cloud based at least on a first vector, the second point cloud being a copy of the first point cloud and including a second point that is a copy of the first point;

identifying a closest point of the first point cloud that is closest to the second point of the second point cloud after moving the second point cloud;

determining a second vector corresponding to a locational relationship between the closest point and the second point; and

determining a measure of usefulness corresponding to the first point with respect to mapping other point clouds to the first point cloud based at least on a relationship between the first vector and the second vector,

wherein a machine performs one or more control operations based at least on the measure of usefulness.

12. The processor of claim 11 , wherein

the determining the relationship between the first vector and the second vector is based at least on comparing the first vector to the second vector.

13. The processor of claim 11 , wherein the first vector is associated with a first direction and wherein the operations further comprise:

moving, based at least on a third vector, a third point cloud with respect to the first point cloud in a second direction different from the first direction, the third point cloud being a second copy of the first point cloud and including a third point that is a second copy of the first point; and

wherein the determining the measure of usefulness corresponding to the first point is further based at least on a second relationship between the first vector and the third vector.

14. The processor of claim 11 , wherein the first vector is associated with a first direction and wherein the operations further comprise:

moving, based at least on a third vector, a third point cloud with respect to the first point cloud in a second direction different from the first direction, the third point cloud being a second copy of the first point cloud and including a third point that is a second copy of the first point;

determining a second measure of usefulness corresponding to the first point based at least on a second relationship between the first vector and the third vector;

moving, based at least on a fourth vector, a fourth point cloud with respect to the first point cloud in a third direction different from the first direction and the second direction, the fourth point cloud being a third copy of the first point cloud and including a fourth point that is a third copy of the first point;

determining a third measure of usefulness corresponding to the first point based at least on a third relationship between the first vector and the fourth vector; and

determining an aggregate measure of usefulness corresponding to the first point based at least on aggregating two or more of: the measure of usefulness as determined based at least on the relationship between the first vector and the second vector, the second measure of usefulness as determined based at least on the second relationship between the first vector and the third vector, or the third measure of usefulness as determined based at least on the third relationship between the first vector and the fourth vector.

15. The processor of claim 11 , wherein the operations further comprise:

annotating map data with the measure of usefulness corresponding to the first point, wherein:

localization of the machine is performed based at least on the annotation of the measure of usefulness, and

performance of the one or more control operations by the machine is based at least on the localization.

16. The processor of claim 11 , wherein the operations further comprise:

annotating map data with a weight derived from the measure of usefulness corresponding to the first point, wherein:

localization of the machine is performed based at least on the annotation of the weight, and

performance of the one or more control operations by the machine is based at least on the localization.

17. The processor of claim 11 , wherein the operations further comprise:

filtering, during a mapping to the first point cloud of a different point cloud, a plurality of points of the first point cloud based at least on respective measures of usefulness corresponding to the plurality of points such that one or more points corresponding to lower measures of usefulness have a higher likelihood of being filtered out compared to one or more points corresponding to higher measures of usefulness.

18. The processor of claim 11 , wherein the operations further comprise assigning a usefulness score to the first point based at least on the measure of usefulness corresponding to the first point.

19. The processor of claim 11 , wherein the operations further comprise:

sampling, during a mapping to the first point cloud of a different point cloud, a plurality of points of the first point cloud based at least on respective measures of usefulness corresponding to the plurality of points such that one or more points corresponding to higher measures of usefulness have a higher likelihood of being sampled compared to the one or more points corresponding to lower measures of usefulness.

20. The processor of claim 11 , wherein the first vector is determined based at least on a potential movement of the machine.

21. A system comprising:

one or more processors to perform operations comprising:

moving a first point cloud that includes a first point with respect to a second point cloud based at least on a first vector, the second point cloud being a copy of the first point cloud and including a second point that is a copy of the first point;

identifying a closest point of the first point cloud that is closest to the second point of the second point cloud after moving the second point cloud;

determining a second vector corresponding to a locational relationship between the closest point and the second point; and

determining a measure of usefulness corresponding to the first point with respect to mapping other point clouds to the first point cloud based at least on a relationship between the first vector and the second vector,

wherein a machine performs one or more control operations based at least on the measure of usefulness.

22. The system of claim 21 , wherein

the determining the relationship between the first vector and the second vector is based at least on comparing the first vector to the second vector.

23. The system of claim 21 , wherein the first vector is associated with a first direction and wherein the operations further comprise:

moving, based at least on a third vector, a third point cloud with respect to the first point cloud in a second direction different from the first direction, the third point cloud being a second copy of the first point cloud and including a third point that is a second copy of the first point; and

wherein the determining the measure of usefulness corresponding to the first point is further based at least on a second relationship between the first vector and the third vector.

24. The system of claim 21 , wherein the first vector is associated with a first direction and wherein the operations further comprise:

moving, based at least on a third vector, a third point cloud with respect to the first point cloud in a second direction different from the first direction, the third point cloud being a second copy of the first point cloud and including a third point that is a second copy of the first point;

determining a second measure of usefulness corresponding to the first point based at least on a second relationship between the first vector and the third vector;

moving, based at least on a fourth vector, a fourth point cloud with respect to the first point cloud in a third direction different from the first direction and the second direction, the fourth point cloud being a third copy of the first point cloud and including a fourth point that is a third copy of the first point, the third direction differing from the first direction and the second direction according to a third vector;

determining a third measure of usefulness corresponding to the first point further based on based at least on a third relationship between the first vector and the fourth vector; and

determining an aggregate measure of usefulness corresponding to the first point based at least on aggregating two or more of: the measure of usefulness as determined based at least on the relationship between the first vector and the second vector, the second measure of usefulness as determined based at least on the second relationship between the first vector and the third vector, or the third measure of usefulness as determined based at least on the third relationship between the first vector and the fourth vector.

25. The system of claim 21 , wherein the operations further comprise:

annotating map data with the measure of usefulness corresponding to the first point, wherein:

localization of the machine is performed based at least on the annotation of the measure of usefulness, and

performance of the one or more control operations by the machine is based at least on the localization.

26. The system of claim 21 , wherein the operations further comprise:

annotating map data with a weight derived from the measure of usefulness corresponding to the first point, wherein:

localization of the machine is performed based at least on the annotation of the weight, and

performance of the one or more control operations by the machine is based at least on the localization.

27. The system of claim 21 , wherein the operations further comprise:

filtering, during a mapping to the first point cloud of a different point cloud, a plurality of points of the first point cloud based at least on respective measures of usefulness corresponding to the plurality of points such that one or more points corresponding to lower measures of usefulness have a higher likelihood of being filtered out compared to one or more points corresponding to higher measures of usefulness.

28. The system of claim 21 , wherein the operations further comprise assigning a usefulness score to the first point based at least on the measure of usefulness corresponding to the first point.

29. The system of claim 21 , wherein the operations further comprise:

sampling, during a mapping to the first point cloud of a different point cloud, a plurality of points of the first point cloud based at least on respective measures of usefulness corresponding to the plurality of points such that one or more points corresponding to higher measures of usefulness have a higher likelihood of being sampled compared to the one or more points corresponding to lower measures of usefulness.

30. The system of claim 21 , wherein the first vector is determined based at least on a potential movement of the machine.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: DEEPMAP INC.
To: NVIDIA CORPORATION
Reel/Frame 061038/0311 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2020
From: WU, MENGXI; ZENG, DI
To: DEEPMAP INC.
Reel/Frame 053535/0549 →
Continuity (2)
Provisional Application 62865082 · Jun 21, 2019
Related Publication 20200401845A1 · Dec 24, 2020
Cited By (1)
US 12,625,926