IP Library Granted Patent US 12664732
Granted Patent B2
US 12664732 · App. 18/772,697 · Granted Jun 23, 2026

Methods and systems for generating a three-dimensional representation of a scene

Inventors: Chengjie Huang (Kitchener, CA); Vahdat Abdelzad (Waterloo, CA); Sean Albert Sedwards (Kitchener, CA); Krzysztof Czarnecki (Waterloo, CA); Eduardo R. Corral-Soto (Toronto, CA)
G06T19/00G06T7/20G06T7/62G06T7/70G06V10/25G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12664732
App. No.
18/772,697
Granted
Jun 23, 2026
Kind
B2
Abstract

Systems and methods for generating a three-dimensional (3D) representation of a scene are provided. The method comprises: accessing a series of 3D point clouds; detecting an object in a first 3D point cloud from the series; upon receiving a second 3D point cloud successive to the first 3D point cloud in the series of 3D point clouds: determining an estimated bounding box indicative of an estimated position of the object in the second 3D point cloud; determining a number N of 3D point clouds from the series of 3D point clouds preceding the second 3D point based on a dynamic parameter of the object in the first 3D point cloud; and generating an augmented 3D point cloud by aggregating data points representative of the object from the N 3D point clouds. The present method may help generate 3D points clouds enabling for more accurate determination of driving scenarios.

Claims (61)

1 . A computer-implemented method for generating a three-dimensional (3D) representation of a scene, the method comprising:

accessing a series of 3D point clouds, each 3D point cloud comprising a plurality of data points and being associated with a respective timestamp;

detecting an object in a first 3D point cloud from the series, the detecting comprising:

determining a bounding box indicative of a position of the object in the first 3D point cloud; and

determining a dynamic parameter of the object at the respective timestamp;

upon receiving a second 3D point cloud successive to the first 3D point cloud in the series of 3D point clouds:

determining an estimated bounding box indicative of an estimated position of the object in the second 3D point cloud based on the bounding box and the dynamic parameter of the object in the first 3D point cloud;

determining a number N of 3D point clouds from the series of 3D point clouds preceding the second 3D point based on the dynamic parameter of the object in the first 3D point cloud; and

generating an augmented 3D point cloud by aggregating data points representative of the object from the N 3D point clouds preceding the second 3D point cloud in the series of 3D point clouds with data points of the second 3D point cloud.

2 . The method of claim 1 , wherein the N 3D point clouds are a subseries of the series of 3D point clouds immediately preceding the second 3D point cloud.

3 . The method of claim 1 , wherein, in a given 3D point cloud, each object is associated with a bounding box delimiting data points of the given 3D point cloud representative of the object,

a size of the bounding box on the second 3D point cloud being adjusted based on the dynamic parameter of the object determined using data points of the first 3D point cloud.

4 . The method of claim 3 , further comprising adjusting a position of the bounding box of the object based on the estimated position thereof and the dynamic parameter of the object determined using data points of the first 3D point cloud.

5 . The method of claim 4 , wherein the 3D point cloud are representations of the scene captured at a pre-determined rate f, and wherein the adjusting the position of the bounding box is further based on the pre-determined rate f.

6 . The method of claim 1 , wherein the dynamic parameter of the object is selected from a group of parameters comprising: a velocity of the object; an object type of the object, a size of the object, a location of the object, a number of data points representative of the object on the second 3D point cloud and an occlusion level of the object.

7 . The method of claim 6 , wherein the dynamic parameter of the object is a velocity thereof, and wherein the number N is determined according to a formula:

N=η(|v τ-1 |)=min(1/α|v τ-1 |+n min ; n max ), where α is a pre-determined hyperparameter, v τ-1 is the velocity of the object determined on the first 3D point cloud and n min and n max are respectively a first and a second pre-determined thresholds.

8 . The method of claim 1 , wherein the series of 3D point clouds are representations of the scene captured at a pre-determined rate f, and wherein determining the estimated position of the object is further based on the pre-determined rate f.

9 . The method of claim 1 , further comprising:

generating a two-dimensional (2D) range image based on a projection of the aggregated data points on a 2D plane; and

transmitting the 2D image to an object-detection module.

10 . The method of claim 1 , further comprising:

identifying residual data points among the data points of the second 3D point cloud based on the series of 3D point clouds;

determining a range image based on the data points of the second 3D point cloud;

identifying emerging data points among the data points of the second 3D point cloud using the range image; and

determining an intersection between the emerging data points and the residual data points as being data points representative of the object.

11 . A system for generating a three-dimensional (3D) representation of a scene, the system comprising a controller and a memory storing a plurality of executable instructions which, when executed by the controller, cause the system to:

access a series of 3D point clouds, each 3D point cloud comprising a plurality of data points and being associated with a respective timestamp;

detect an object in a first 3D point cloud from the series, the detecting comprising:

determine a bounding box indicative of a position of the object in the first 3D point cloud; and

determine a dynamic parameter of the object at the respective timestamp;

upon receiving a second 3D point cloud successive to the first 3D point cloud in the series of 3D point clouds:

determine an estimated bounding box indicative of an estimated position of the object in the second 3D point cloud based on the bounding box and the dynamic parameter of the object in the first 3D point cloud;

determine a number N of 3D point clouds from the series of 3D point clouds preceding the second 3D point based on the dynamic parameter of the object in the first 3D point cloud; and

generate an augmented 3D point cloud by aggregating data points representative of the object from the N 3D point clouds preceding the second 3D point cloud in the series of 3D point clouds with data points of the second 3D point cloud.

12 . The system of claim 11 , wherein the N 3D point clouds are a subseries of the series of 3D point clouds immediately preceding the second 3D point cloud.

13 . The system of claim 11 , wherein, in a given 3D point cloud, each object is associated with a bounding box delimiting data points of the given 3D point cloud representative of the object,

a size of the bounding box on the second 3D point cloud being adjusted based on the dynamic parameter of the object determined using data points of the first 3D point cloud.

14 . The system of claim 13 , wherein the system is further configured to adjust a position of the bounding box of the object based on the estimated position thereof and the dynamic parameter of the object determined using data points of the first 3D point cloud,

adjusting the position of the bounding box being based on the pre-determined rate f at which the series of 3D point clouds has been captured.

15 . The system of claim 11 , wherein the dynamic parameter of the object is selected in a group of parameters comprising: a velocity of the object; an object type of the object, a size of the object, a location of the object, a number of data points representative of the object on the second 3D point cloud and an occlusion level of the object.

16 . The system of claim 15 , wherein the dynamic parameter of the object is a velocity thereof, and wherein the number N is determined according to a formula:

N=η(|v τ-1 |)=min(1/α|v τ-1 |+n min ; n max ), where α is a pre-determined hyperparameter, v τ-1 is the velocity of the object determined on the first 3D point cloud and n min and n max are respectively a first and a second pre-determined thresholds.

17 . The system of claim 11 , wherein the series of 3D point clouds are representations of the scene captured at a pre-determined rate f, and wherein determining the estimated position of the object is further based on the pre-determined rate f.

18 . The system of claim 11 , wherein the system is further configured to:

generate a two-dimensional (2D) range image based on a projection of the aggregated data points on a 2D plane; and

transmit the 2D image to an object-detection module.

19 . The system of claim 11 , wherein the system is further configured to:

identify residual data points among the data points of the second 3D point cloud based on the series of 3D point clouds;

determine a range image based on the data points of the second 3D point cloud;

identify emerging data points among the data points of the second 3D point cloud using the range image; and

determine an intersection between the emerging data points and the residual data points as being data points representative of the object.

20 . A non-transitory computer readable medium storing executable instructions for causing one or more computer processors to:

access a series of 3D point clouds, each 3D point cloud comprising a plurality of data points and being associated with a respective timestamp;

detect an object in a first 3D point cloud from the series, the detecting comprising:

determine a bounding box indicative of a position of the object in the first 3D point cloud; and

determine a dynamic parameter of the object at the respective timestamp;

upon receiving a second 3D point cloud successive to the first 3D point cloud in the series of 3D point clouds:

determine an estimated bounding box indicative of an estimated position of the object in the second 3D point cloud based on the bounding box and the dynamic parameter of the object in the first 3D point cloud;

determine a number N of 3D point clouds from the series of 3D point clouds preceding the second 3D point based on the dynamic parameter of the object in the first 3D point cloud; and

generate an augmented 3D point cloud by aggregating data points representative of the object from the N 3D point clouds preceding the second 3D point cloud in the series of 3D point clouds with data points of the second 3D point cloud.