Granted Patent B2

US 12,482,246 · App. 17/931,751 · Granted Nov 25, 2025

Inventor: Robert De Temple (Essen, DE)

Assignee: CRON AI LTD.

G06V10/803G06V10/25G06V10/82G06V2201/07

Abstract

Claims (75)

1 . A method for updating a cartesian representation of a region of interest (ROI), the method comprising:

receiving, by at least one sensor encoding neural network implemented in a computing device A, a sensor point of view (POV) data, wherein the sensor POV data is based on a sensor data generated by at least one sensor for a plurality of view cones;

generating, via the at least one sensor encoding neural network, a polar feature vector based on the sensor POV data, wherein the polar feature vector comprises a plurality of cone vectors corresponding to the plurality of view cones;

mapping, by a projection module implemented in a computing device B, at least one cell from a plurality of cells of the cartesian representation with at least one cone vector from the plurality of cone vectors, wherein the at least one cone vector corresponds to at least one view cone from the plurality of view cones;

generating, via an offset encoding module implemented in the computing device B, at least one offset vector based on one or more offset parameters corresponding to the at least one cell, wherein the at least one offset vector is generated further based on one or more sensor performance parameters, and wherein the one or more sensor performance parameters are indicative of a performance of the at least one sensor;

receiving, by the projection module, the at least one offset vector corresponding to the at least one cell, wherein the at least one offset vector is based on a position of the at least one sensor relative to the at least one cell of the cartesian representation;

concatenating, via the projection module, at least the at least one cone vector and the at least one offset vector to generate at least one transformed tensor corresponding to the at least one cell;

generating, via a convolutional neural network implemented in the computing device B, at least one learned output based on the at least one transformed tensor; and

generating, via an object detection network implemented in the computing device B, an updated cartesian representation based on the at least one learned output.

2 . The method of claim 1 , wherein each of the plurality of view cones is defined by an angular range about the at least one sensor.

3 . The method of claim 1 , wherein the sensor POV data is a spherical projection or a cylindrical projection of the sensor data, and wherein the polar feature vector generated by the at least one sensor encoding neural network is a one dimensional vector of vectors.

4 . The method of claim 1 , wherein:

the at least one cell comprises the plurality of cells;

the at least one cone vector comprises the plurality of cone vectors;

the at least one offset vector comprises a plurality of offset vectors corresponding to the plurality of cells;

the at least one transformed tensor comprises a plurality of transformed tensors corresponding to the plurality of cells;

the at least one learned output comprises a plurality of learned cell feature vectors corresponding to the plurality of cells; and

generating the updated cartesian representation further comprises replacing, via the object detection network, a plurality of previous cell feature vectors corresponding to the plurality of cells with the plurality of learned cell feature vectors to generate the updated cartesian representation; and

the method further comprising providing, via the object detection network, one or more perception outputs for the ROI using the updated cartesian representation of the ROI.

5 . The method of claim 1 , wherein:

the at least one cell comprises one cell;

the at least one offset vector comprises one offset vector corresponding to the one cell;

the at least one transformed tensor comprises one transformed tensor corresponding to the one cell;

the at least one learned output comprises one learned cell feature vector corresponding to the at least one cell; and

generating the updated cartesian representation further comprises replacing, via the object detection network, one previous cell feature vector corresponding to the at least one cell with the one learned cell feature vector to generate the updated cartesian representation; and

the method comprising providing, via the object detection network, one or more perception outputs for the ROI using the updated cartesian representation of the ROI.

6 . The method of claim 1 , wherein the at least one learned output comprises at least one learned weight matrix corresponding to the at least one cell, the method further comprising determining, via the object detection network, at least one learned cell feature vector corresponding to the at least one cell as a dot-product of the at least one transformed tensor and the at least one learned weight matrix.

7 . The method of claim 6 , wherein generating the updated cartesian representation further comprises replacing, via the object detection network, at least one previous cell feature vector corresponding to the at least one cell with the at least one learned cell feature vector to generate the updated cartesian representation; and

the method further comprising providing, via the object detection network, one or more perception outputs for the ROI using the updated cartesian representation of the ROI.

8 . The method of claim 1 further comprising receiving, by the projection module, at least one timing vector corresponding to the at least one cell, wherein the at least one timing vector is indicative of a time instance when the sensor data for the at least one view cone is generated, and wherein the at least one transformed tensor is generated by further concatenating the at least one timing vector with the at least one cone vector and the at least one offset vector.

9 . The method of claim 1 further comprising receiving, by the projection module, at least one previous cell feature vector of a previously updated cartesian representation corresponding to the at least one cell, wherein the at least one transformed tensor is generated by further concatenating the at least one previous cell feature vector with the at least one cone vector and the at least one offset vector.

10 . The method of claim 1 further comprising:

receiving, via a state adaptation module, at least one previous cell feature vector of a previously updated cartesian representation corresponding to the at least one cell; and

adjusting, via the state adaptation module, the at least one previous cell feature vector to generate at least one adapted previous cell feature vector.

11 . The method of claim 10 , wherein the at least one transformed tensor is generated by further concatenating the at least one adapted previous cell feature vector with the at least one cone vector and the at least one offset vector.

12 . The method of claim 10 , wherein the state adaptation module is a convolutional neural network.

13 . The method of claim 10 , wherein adjusting the at least one previous cell feature vector further comprises multiplying a magnitude of the at least one previous cell feature vector by a scalar decay value to generate the at least one adapted previous cell feature vector, and wherein the scalar decay value is based on a time elapsed since a generation of the previously updated cartesian representation.

14 . The method of claim 1 , wherein the one or more offset parameters are further based on one or more sensor parameters indicative of the position of the at least one sensor relative to the at least one cell.

15 . The method of claim 1 , wherein:

the at least one sensor comprises a plurality of sensors; and

the at least one sensor encoding neural network generates a plurality of polar feature vectors based on the sensor POV data received from the plurality of sensors; and

the method further comprising adding the plurality of polar feature vectors received by the projection module to a queue, such that each polar feature vector from the plurality of polar feature vectors in the queue is generated prior to a succeeding polar feature vector from the plurality of polar feature vectors, wherein the cartesian representation is sequentially updated based on each polar feature vector in the queue.

16 . An object detection system for updating a cartesian representation of a region of interest (ROI), the object detection system comprising:

at least one sensor covering at least a portion of the ROI, wherein the at least one sensor is configured to generate a sensor data for a plurality of view cones;

at least one sensor encoding neural network implemented in a computing device A, the at least one sensor encoding neural network configured to:

receive a sensor point of view (POV) data, wherein the sensor POV data is based on the sensor data; and

generate a polar feature vector based on the sensor POV data, wherein the polar feature vector comprises a plurality of cone vectors corresponding to the plurality of view cones;

a projection module implemented in a computing device B communicably coupled to the computing device A, the projection module configured to:

map at least one cell from a plurality of cells of the cartesian representation with at least one cone vector from the plurality of cone vectors, wherein the at least one cone vector corresponds to at least one view cone from the plurality of view cones;

receive at least one offset vector corresponding to the at least one cell, wherein the at least one offset vector is based on a position of the at least one sensor relative to the at least one cell of the cartesian representation;

concatenate at least the at least one cone vector and the at least one offset vector to generate at least one transformed tensor corresponding to the at least one cell;

a convolutional neural network implemented in the computing device B, the convolutional neural network configured to generate at least one learned output based on the at least one transformed tensor; and

an object detection network implemented in the computing device B, the object detection network configured to generate an updated cartesian representation based on the at least one learned output; and

an offset encoding module implemented in the computing device B, the offset encoding module configured to generate the at least one offset vector based on one or more offset parameters corresponding to the at least one cell and one or more sensor performance parameters, wherein the one or more sensor performance parameters are indicative of a performance of the at least one sensor.

17 . The object detection system of claim 16 , wherein:

the at least one cell comprises the plurality of cells;

the at least one cone vector comprises the plurality of cone vectors;

the at least one offset vector comprises a plurality of offset vectors corresponding to the plurality of cells;

the at least one transformed tensor comprises a plurality of transformed tensors corresponding to the plurality of cells;

the at least one learned output comprises a plurality of learned cell feature vectors corresponding to the plurality of cells; and

the object detection network is further configured to:

replace a plurality of previous cell feature vectors corresponding to the plurality of cells with the plurality of learned cell feature vectors to generate the updated cartesian representation; and

provide one or more perception outputs for the ROI using the updated cartesian representation of the ROI.

18 . The object detection system of claim 16 , wherein:

the at least one cell comprises one cell;

the at least one offset vector comprises one offset vector corresponding to the one cell;

the at least one transformed tensor comprises one transformed tensor corresponding to the one cell;

the at least one learned output comprises one learned cell feature vector corresponding to the at least one cell; and

the object detection network is further configured to:

replace one previous cell feature vector corresponding to the one cell with the one learned cell feature vector to generate the updated cartesian representation; and

provide one or more perception outputs for the ROI using the updated cartesian representation of the ROI.

19 . The object detection system of claim 16 , wherein the at least one learned output comprises at least one learned weight matrix corresponding to the at least one cell, and wherein the object detection network is further configured to:

determine at least one learned cell feature vector corresponding to the at least one cell as a dot-product of the at least one transformed tensor and the at least one learned weight matrix;

replace at least one previous cell feature vector corresponding to the at least one cell with the at least one learned cell feature vector to generate the updated cartesian representation; and

provide one or more perception outputs for the ROI using the updated cartesian representation of the ROI.

Assignments (2)

Continuity (1)

References Cited (28)