IP Library Granted Patent US 12,044,779
Granted Patent B2
US 12,044,779 · App. 17/115,006 · Granted Jul 23, 2024

Methods and system for analyzing dynamic lidar point cloud data

Inventors: Patrick Zhou (Princeton, NJ); Yijun Tang (Horsham, PA); Guillaume Binet (Pittsburgh, PA)
Assignee: Argo AI, LLC
G01S17/89G01S17/931G05D1/0231G06F16/51G06T1/0007G06T1/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,044,779
App. No.
17/115,006
Granted
Jul 23, 2024
Kind
B2
Abstract

A system includes a computer-readable memory for storage and retrieval of LiDAR data. The memory includes one or more point cloud data structures, each point cloud data structure including a first header block, and a plurality of point blocks. Each of the plurality of point blocks is configured to store at least a portion of LiDAR point cloud data. The memory includes one or more image data structures having a second header block, and a plurality of image column blocks. Each image column block includes a column of pixels of a corresponding image from the LiDAR point cloud data, and the image column blocks collectively contain all pixels of the corresponding image.

Claims (56)

1. A system, comprising:

an electronic device; and

a computer-readable medium comprising one or more programming instructions that, when executed, cause the electronic device to:

receive a stream of one or more frames of LiDAR data from each of a plurality of autonomous vehicles, wherein each said one or more frames of LiDAR data comprises point cloud data representing a respective image of a plurality of images;

translating different data formats of the Lidar data received from the plurality of autonomous vehicles into a common format by performing operations to:

assign at least a portion of the LiDAR data into one or more point cloud data structures, wherein each point cloud data structure comprises:

a first header block, and

a plurality of point blocks that each comprises all LIDAR points contained in a respective frame of the one or more frames; and

assign at least a portion of the LiDAR data into one or more image data structures, wherein the image data structure comprises:

a second header block, and

a plurality of image column blocks, in which each image column block comprises a respective column of pixels of a corresponding image of the plurality of images, and in which the plurality of image column blocks collectively contain all pixels of the corresponding image;

generate a batch comprising one or more of the point cloud data structures and/or one or more of the image data structures; and

provide the batch to a point cloud processing system.

2. The system of claim 1 , wherein the one or more programming instructions that, when executed, cause the electronic device to assign at least a portion of the LiDAR data into one or more point cloud data structures comprises one or more programming instructions that, when executed, cause the electronic device to store part of the at least a portion of the LiDAR data in each of the plurality of point blocks.

3. The system of claim 2 , wherein the first header block comprises a source indicator that identifies a source of the LiDAR data that is stored in the plurality of point blocks.

4. The system of claim 1 , wherein one or more of the point cloud data structures comprises, for each of the point blocks, a size block that immediately precedes the corresponding point block and that includes an indicator of size of the corresponding point block.

5. The system of claim 1 , wherein one or more of the plurality of point blocks comprise an indicator of size of the point block.

6. The system of claim 1 , wherein the first header block comprises an indicator indicating which one of a plurality of different data structure types the first header is associated with.

7. The system of claim 1 , wherein the first header block comprises a follower indicator that identifies which of a plurality of different data structure types is a data structure that follows the point cloud data structure in memory.

8. The system of claim 1 , wherein the one or more programming instructions that, when executed, cause the electronic device to assign at least a portion of the LiDAR data into one or more image data structures comprises one or more programming instructions that, when executed, cause the electronic device to store part of the at least a portion of the LiDAR data in each of the plurality of image column blocks.

9. The system of claim 8 , wherein the second header comprises a source indicator that identifies a source of the LiDAR data that is stored in the plurality of image column blocks.

10. The system of claim 1 , wherein the second header block comprises a data type indicator indicating a data type of the image data structure.

11. The system of claim 1 , wherein the second header block comprises a follower indicator that identifies a data type of a data structure that follows the image data structure in memory.

12. The system of claim 1 , wherein the second header block comprises an indication of a how many image columns are stored by the corresponding image data structure.

13. The system of claim 1 , wherein the second header block comprises an indication of a size of one or more of the image column blocks.

14. A system comprising a computer-readable memory for storage and retrieval of LiDAR data, the memory comprising:

one or more point cloud data structures, wherein each point cloud data structure comprises:

a first header block, and

a plurality of point blocks, wherein each of the plurality of point blocks is configured to store all LiDAR points contained in a respective frame of one or more frames of LIDAR point cloud data, wherein each said one or more frames of LIDAR point cloud data representing a respective image of the plurality of images; and

one or more image data structures, wherein the image data structure comprises:

a second header block, and

a plurality of image column blocks, in which each image column block comprises a respective column of pixels of a corresponding image from the LiDAR point cloud data, and in which the image column blocks collectively contain all pixels of the corresponding image.

15. A method, comprising:

by an electronic device:

receiving a stream of one or more frames of LiDAR data from each of a plurality of autonomous vehicles, wherein each said one or more frames of LiDAR data comprises point cloud data representing a respsective image of a plurality of;

assigning at least a portion of the LiDAR data into one or more point cloud data structures, wherein each point cloud data structure comprises:

a first header block, and

a plurality of point blocks that each comprises all LIDAR point contained in a respective frame of the one or more frames; and

assigning at least a portion of the LiDAR data into one or more image data structures, wherein the image data structure comprises:

a second header block, and

a plurality of image column blocks, in which each image column block comprises a respective column of pixels of a corresponding image of the plurality of images, and in which the image column blocks collectively contain all pixels of the corresponding image;

generating a batch comprising one or more of the point cloud data structures and/or one or more of the image data structures; and

providing the batch to a point cloud processing system.

16. The method of claim 15 , wherein assigning at least a portion of the LiDAR data into one or more point cloud data structures comprises storing part of the at least a portion of the LiDAR data in each of the plurality of point blocks.

17. The method of claim 16 , wherein the first header block comprises a source indicator that identifies a source of the LiDAR data that is stored in the plurality of point blocks.

18. The method of claim 15 , wherein one or more of the point cloud data structures comprises, for each of the point blocks, a size block that immediately precedes the corresponding point block and that includes an indicator of size of the corresponding point block.

19. The method of claim 15 , wherein one or more of the plurality of point blocks comprise an indicator of size of the point block.

20. The method of claim 15 , wherein the first header block comprises a data type indicator indicating a data type of the point cloud data structure.

21. The method of claim 15 , wherein the first header block comprises a follower indicator that identifies a data type of a data structure that follows the point cloud data structure in memory.

22. The method of claim 15 , wherein assigning at least a portion of the LiDAR data into one or more image data structures comprises storing part of the at least a portion of the LiDAR data in each of the plurality of image column blocks.

23. The method of claim 22 , wherein the second header comprises a source indicator that identifies a source of the LiDAR data that is stored in the plurality of image column blocks.

24. The method of claim 15 , wherein the second header block comprises one or more of the following:

a data type indicator indicating a data type of the image data structure;

a follower indicator that identifies a data type of a data structure that follows the image data structure in memory;

an indication of a how many image columns are stored by the corresponding image data structure; or

an indication of a size of one or more of the image column blocks.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069177/0099 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 8, 2020
From: ZHOU, PATRICK; TANG, YIJUN; BINET, GUILLAUME
To: ARGO AI, LLC
Reel/Frame 054577/0787 →
Continuity (1)
Related Publication 20220179082A1 · Jun 9, 2022
Cited By (1)
US 12,352,903