IP Library Granted Patent US 11,257,230
Granted Patent B2
US 11,257,230 · App. 16/781,220 · Granted Feb 22, 2022

Adaptive feature map anchor pruning

Inventors: Zachary M. Kendrick (Irvine, CA); Disha Ahuja (San Jose, CA); Ashwath Aithal (Fremont, CA)
Assignee: NIO USA, Inc.
G06T7/50G01C21/26G01S17/89G06T5/002G06T5/20G06T2207/10028G06T2207/20216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,257,230
App. No.
16/781,220
Granted
Feb 22, 2022
Kind
B2
Abstract

Embodiments include a method for pruning anchor points from a feature map generated from a Light Detections And Ranging (LiDAR) point cloud, the method comprising: receiving, by a navigation system, a LiDAR point cloud from a LiDAR sensor, the LiDAR point cloud comprising data representing one or more objects in physical surroundings detected by the LiDAR sensor; extracting, by the navigation system, a feature map from the LiDAR point cloud, the feature map comprising a plurality of anchor points, each anchor point defined by an anchor box; smoothing, by the navigation system, the extracted feature map; determining, by the navigation system, density of pixels within the anchor box of each anchor point; and pruning, by the navigation system, anchor points from the feature map based on a plurality of factors related to the determined density of pixels within the box of each anchor point.

Claims (41)

1. A method for pruning anchor points from a feature map generated from a Light Detections And Ranging (LiDAR) point cloud, the method comprising:

receiving, by a navigation system, a LiDAR point cloud from a LiDAR sensor, the LiDAR point cloud comprising data representing one or more objects in physical surroundings detected by the LiDAR sensor;

extracting, by the navigation system, a feature map from the LiDAR point cloud, the feature map comprising a plurality of anchor points, each anchor point defined by an anchor box;

smoothing, by the navigation system, the extracted feature map;

determining, by the navigation system, a density of pixels within the anchor box of each anchor point; and

pruning, by the navigation system, anchor points from the feature map based on a plurality of factors related to the determined density of pixels within the anchor box of each anchor point.

2. The method of claim 1 , wherein smoothing the extracted feature map comprises applying a Gaussian smoothing to the extracted feature map.

3. The method of claim 1 , wherein determining the density of pixels within the anchor box of each anchor point comprises computing a sum of pixels within the anchor box of each anchor point.

4. The method of claim 3 , wherein determining the density of pixels within the anchor box of each anchor point further comprises computing an average pixel value for the anchor box of each anchor point.

5. The method of claim 4 , wherein determining the density of pixels within the anchor box of each anchor point further comprises computing a center pixel value for the anchor box of each anchor point.

6. The method of claim 5 , wherein pruning anchor points from the feature map based on the plurality of factors related to the determined density of pixels within the anchor box of each anchor point comprises pruning anchor points having an anchor box with an average pixel value below a predetermined threshold for average pixel value.

7. The method of claim 6 , wherein pruning anchor points from the feature map based on the plurality of factors related to the determined density of pixels within the anchor box of each anchor point further comprises pruning anchor points having an anchor box with a center pixel value below a predetermined threshold for center pixel value.

8. A navigation system comprising:

a processor; and

a memory coupled with and readable by the memory and storing therein a set of instructions which, when executed by the processor, causes the processor to prune anchor points from a feature map generated from a Light Detections And Ranging (LiDAR) point cloud by:

receiving, by a navigation system, a LiDAR point cloud from a LiDAR sensor, the LiDAR point cloud comprising data representing one or more objects in physical surroundings detected by the LiDAR sensor;

extracting, by the navigation system, a feature map from the LiDAR point cloud, the feature map comprising a plurality of anchor points, each anchor point defined by an anchor box;

smoothing, by the navigation system, the extracted feature map;

determining, by the navigation system, a density of pixels within the anchor box of each anchor point; and

pruning, by the navigation system, anchor points from the feature map based on a plurality of factors related to the determined density of pixels within the anchor box of each anchor point.

9. The navigation system of claim 8 , wherein smoothing the extracted feature map comprises applying a Gaussian smoothing to the extracted feature map.

10. The navigation system of claim 8 , wherein determining the density of pixels within the anchor box of each anchor point comprises computing a sum of pixels within the anchor box of each anchor point.

11. The navigation system of claim 10 , wherein determining the density of pixels within the anchor box of each anchor point further comprises computing an average pixel value for the anchor box of each anchor point.

12. The navigation system of claim 11 , wherein determining the density of pixels within the anchor box of each anchor point further comprises computing a center pixel value for the anchor box of each anchor point.

13. The navigation system of claim 12 , wherein pruning anchor points from the feature map based on the plurality of factors related to the determined density of pixels within the anchor box of each anchor point comprises pruning anchor points having an anchor box with an average pixel value below a predetermined threshold for average pixel value.

14. The navigation system of claim 13 , wherein pruning anchor points from the feature map based on the plurality of factors related to the determined density of pixels within the anchor box of each anchor point further comprises pruning anchor points having an anchor box with a center pixel value below a predetermined threshold for center pixel value.

15. A vehicle comprising:

a Light Detection And Ranging (LiDAR) sensor;

a navigation system coupled with the LiDAR sensor and comprising a processor and a memory coupled with and readable by the processor and storing therein a set of instructions which, when executed by the processor, causes the processor to prune anchor points from a feature map generated from a Light Detections And Ranging (LiDAR) point cloud by:

receiving, by a navigation system, a LiDAR point cloud from a LiDAR sensor, the LiDAR point cloud comprising data representing one or more objects in physical surroundings detected by the LiDAR sensor;

extracting, by the navigation system, a feature map from the LiDAR point cloud, the feature map comprising a plurality of anchor points, each anchor point defined by an anchor box;

smoothing, by the navigation system, the extracted feature map;

determining, by the navigation system, a density of pixels within the anchor box of each anchor point; and

pruning, by the navigation system, anchor points from the feature map based on a plurality of factors related to the determined density of pixels within the anchor box of each anchor point.

16. The vehicle of claim 15 , wherein smoothing the extracted feature map comprises applying a Gaussian smoothing to the extracted feature map.

17. The vehicle of claim 15 , wherein determining the density of pixels within the anchor box of each anchor point comprises computing a sum of pixels within the anchor box of each anchor point.

18. The vehicle of claim 17 , wherein determining the density of pixels within the anchor box of each anchor point further comprises computing an average pixel value for the anchor box of each anchor point.

19. The vehicle of claim 18 , wherein determining the density of pixels within the anchor box of each anchor point further comprises computing a center pixel value for the anchor box of each anchor point.

20. The vehicle of claim 19 , wherein pruning anchor points from the feature map based on the plurality of factors related to the determined density of pixels within the anchor box of each anchor point comprises:

pruning anchor points having an anchor box with an average pixel value below a predetermined threshold for average pixel value; and

pruning anchor points having an anchor box with a center pixel value below a predetermined threshold for center pixel value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 24, 2022
From: NIO USA, INC.
To: NIO TECHNOLOGY (ANHUI) CO., LTD.
Reel/Frame 060171/0724 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2020
From: KENDRICK, ZACHARY M.; AHUJA, DISHA; AITHAL, ASHWATH
To: NIO USA, INC.
Reel/Frame 051940/0076 →
Continuity (1)
Related Publication 20210241471A1 · Aug 5, 2021