Method and device for detecting objects

1. A method comprising:

capturing, via a radar sensor, a raw radar point cloud comprising a plurality of data points;

defining at least one object class comprising a predefined object type and a geometrical shape to enclose an object of the predefined object type;

via a first stage of a neural network:

performing, for the plurality of data points, a semantic segmentation regarding the object class and background; and

estimating, for each data point of the plurality of data points, a rough first approximation for a spatial condition of the geometrical shape; and

via a second stage of the neural network:

selecting, based on the first approximation for the spatial condition of the geometrical shape, a subset of the plurality of data points based on the semantic segmentation; and

estimating, for each data point of the subset of the plurality of data points, a second approximation that refines the first approximation for the spatial condition of the geometrical shape and a confidence score for the second approximation for the spatial condition of the geometrical shape;

wherein:

the spatial condition of the geometrical shape comprises a center of the geometrical shape;

estimating the second approximation for the spatial condition of the geometrical shape comprises estimating, for each data point of the subset of the plurality of data points, a respective shifting vector that shifts the respective data point of the subset of the plurality of data points toward the center of the geometrical shape for providing center proposals for the geometrical shape; and

the method further comprises determining a final estimation for the center of the geometrical shape by calculating an average over the center proposals of the geometrical shape.

2. The method of claim 1 , wherein:

the spatial condition of the geometrical shape further comprises dimensions of the geometrical shape; and

estimating the second approximation for the spatial condition of the geometrical shape comprises estimating the dimensions of the geometrical shape.

3. The method of claim 2 , wherein:

the spatial condition of the geometrical shape further comprises an orientation of the geometrical shape; and

estimating the second approximation for the spatial condition of the geometrical shape comprises estimating the orientation of the geometrical shape.

4. The method of claim 3 , wherein estimating the orientation of the geometrical shape is split into a step of regressing angle information for the geometrical shape and a step of classifying the angle information.

5. The method of claim 1 , wherein:

the first stage of the neural network comprises a respective output path for the semantic segmentation and for at least one parameter of the spatial condition of the geometric shape;

the second stage of the neural network comprises a respective output path for the confidence score and for a refinement of the at least one parameter of the spatial condition of the geometric shape; and

each output path of the neural network is trained based on a separate objective function.

6. The method of claim 1 , wherein

performing the semantic segmentation is trained by defining a ground truth of the geometrical shape and determining whether a respective data point of the plurality of data points is located within the ground truth of the geometrical shape.

7. The method of claim 1 , wherein the first stage of the neural network is deactivated while the second stage of the neural network is trained.

8. The method of claim 1 , wherein:

the geometrical shape of the object class comprises a bounding box; and

estimating the second approximation for the spatial condition comprises estimating dimensions and an orientation of the bounding box.

9. The method of claim 1 , wherein:

at least two different object classes are defined; and

the at least two different object classes comprise different object types and different geometrical shapes.

10. The method of claim 1 , wherein

estimating the second approximation for the spatial condition of the geometrical shape is performed by a farthest point sampling on only a selected subset of the plurality of data points.

11. The method of claim 10 , wherein

estimating the second approximation for the spatial condition of the geometrical shape comprises grouping the data points of the selected subset for training the second stage of the neural network regarding local structures.

12. A system comprising:

a radar sensor configured to:

acquire a radar signal being reflected by objects in an environment of the radar sensor; and

provide a raw radar point cloud based on the radar signal, the raw radar point cloud including a plurality of data points;

a neural network including a first stage and a second stage and configured to define at least one object class including a predefined object type and a geometrical shape to enclose an object of the predefined object type;

wherein:

the first stage of the neural network is configured to:

perform a semantic segmentation regarding the object class and background for the plurality of data points; and

estimate, for each data point of the plurality of data points, a first approximation for a spatial condition of the geometrical shape;

the second stage of the neural network is configured to:

select, based on the first approximation for the spatial condition of the geometrical shape, a subset of the plurality of data points based on the semantic segmentation; and

estimate, for each data point of the subset of the plurality of data points, a second approximation that refines the first approximation for the spatial condition of the geometrical shape and a confidence score for the second approximation for the spatial condition of the geometrical shape;

the spatial condition of the geometrical shape comprises a center of the geometrical shape; and

in estimating the second approximation for the spatial condition of the geometrical shape, the neural network is configured to estimate, for each data point of the subset of the plurality of data points, a respective shifting vector that shifts the respective data point of the subset toward the center of the geometrical shape for providing center proposals for the geometrical shape, and to determine a final estimation for the center of the geometrical shape by calculating an average over the center proposals of the geometrical shape.

13. The system of claim 12 , wherein:

the spatial condition of the geometrical shape further comprises dimensions of the geometrical shape; and

in estimating the second approximation for the spatial condition of the geometrical shape, the neural network is configured to estimate the dimensions of the geometrical shape.

14. The system of claim 12 , wherein:

the spatial condition of the geometrical shape further comprises an orientation of the geometrical shape; and

in estimating the second approximation for the spatial condition of the geometrical shape, the neural network is configured to estimate the orientation of the geometrical shape.

15. The system of claim 14 , wherein:

in estimating the orientation of the geometrical shape, the neural network is configured to regress angle information for the geometrical shape and classify the angle information.

16. The system of claim 12 , wherein:

the first stage of the neural network comprises a respective output path for the semantic segmentation and for at least one parameter of the spatial condition of the geometrical shape;

the second stage of the neural network comprises a respective output path for the confidence score and for a refinement of the at least one parameter of the spatial condition of the geometrical shape; and

each output path of the neural network is trained based on a separate objective function.

17. The system of claim 12 , wherein:

performing the semantic segmentation is trained by defining a ground truth of the geometrical shape and determining whether a respective data point of the plurality of data points is located within the ground truth of the geometrical shape.

18. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors of a radar system to:

receive, via a radar sensor, a raw radar point cloud comprising a plurality of data points;

define at least one object class comprising a predefined object type and a geometrical shape to enclose an object of the predefined object type;

via a first stage of a neural network:

perform, for the plurality of data points, a semantic segmentation regarding the object class and background; and

estimate, for each data point of the plurality of data points, a first approximation for a spatial condition of the geometrical shape; and

via a second stage of the neural network:

select, based on the first approximation for the spatial condition of the geometrical shape, a subset of the plurality of data points based on the semantic segmentation; and

estimate, for each data point of the subset of the plurality of data points, a second approximation that refines the first approximation for the spatial condition of the geometrical shape and a confidence score for the second approximation for the spatial condition of the geometrical shape;

wherein:

the spatial condition of the geometrical shape comprises a center of the geometrical shape;

estimating the second approximation for the spatial condition of the geometrical shape comprises estimating, for each data point of the subset of the plurality of data points, a respective shifting vector that shifts the respective data point of the subset of the plurality of data points toward the center of the geometrical shape for providing center proposals for the geometrical shape; and

the instructions further cause the one or more processors of the radar system to determine a final estimation for the center of the geometrical shape by calculating an average over the center proposals of the geometrical shape.