Variational 3D object detection

1. A method for monocular 3D object perception, comprising:

sampling multiple, stochastic latent variables from a learned latent feature distribution of an RGB image for a 2D object detected in the RGB image;

lifting a 3D proposal for each stochastic latent variable sampled for the detected 2D object;

selecting a 3D proposal for the detected 2D object using a proposal selection algorithm to reduce 3D proposal lifting overlap; and

planning a trajectory of an ego vehicle according to a 3D location and pose of the 2D object according to the selected 3D proposal.

2. The method of claim 1 , in which selecting the 3D proposal comprises:

estimating a confidence score for each lifted 3D proposal using a confidence network; and

presenting the 3D proposals with the highest confidence scores as the selected 3D proposal.

3. The method of claim 2 , in which estimating the confidence score for each lifted 3D proposal is based on a respective visual feature, its 2D projection, and its relation to the other 3D proposals.

4. The method of claim 1 , further comprising modeling ambiguity in 2D to 3D projection as uncertainty of a stochastic latent variable sampled from the learned latent distribution, resulting in one-to-many 3D predictions.

5. The method of claim 1 , further comprising:

training a confidence network to estimate confidence scores of multiple 3D detected object proposals; and

selecting a monocular 3D object bounding box to prevent a physical overlap constraint.

6. The method of claim 1 , further comprising:

training both a 2D detection module and a 3D detection module based on a 2D detection backbone.

7. The method of claim 1 , in which selecting comprises constrained optimization leveraging of a 3D collision-free constraint.

8. The method of claim 1 , in which each 3D proposal is represented as a 3D box bound to the detected 2D object.

9. The method of claim 1 , in which sampling comprises generating the multiple stochastic latent variables using a deep convolutional neural network (CNN).

10. A non-transitory computer-readable medium having program code recorded thereon for monocular 3D object perception, the program code being executed by a processor and comprising:

program code to sample multiple, stochastic latent variables from a learned latent feature distribution of an RGB image for a 2D object detected in the RGB image;

program code to lift a 3D proposal for each stochastic latent variable sampled for the detected 2D object;

program code to select a 3D proposal for the detected 2D object using a proposal selection algorithm to reduce 3D proposal lifting overlap; and

program code to plan a trajectory of an ego vehicle according to a 3D location and pose of the 2D object according to the selected 3D proposal.

11. The non-transitory computer-readable medium of claim 10 , in which the program code to select the 3D proposal comprises:

program code to estimate a confidence score for each lifted 3D proposal using a confidence network; and

program code to present the 3D proposal with the highest confidence score as the selected 3D proposal.

12. The non-transitory computer-readable medium of claim 11 , in which the program code to estimate the confidence score for each lifted 3D proposal is based on a respective visual feature, its 2D projection, and its relation to the other 3D proposals.

13. The non-transitory computer-readable medium of claim 10 , further comprising program code to model ambiguity in 2D to 3D projection as uncertainty of a stochastic latent variable sampled from the learned latent distribution, resulting in one-to-many 3D predictions.

14. The non-transitory computer-readable medium of claim 10 , further comprising:

program code to train a confidence network to estimate confidence scores of multiple 3D detected object proposals; and

program code to select a monocular 3D object bounding box to prevent a physical overlap constraint.

15. The non-transitory computer-readable medium of claim 10 , further comprising program code to train both a 2D detection module and a 3D detection module based on a 2D detection backbone.

16. The non-transitory computer-readable medium of claim 10 , in which the program code to select comprises program code to constrain optimization leveraging of a 3D collision-free constraint.

17. The non-transitory computer-readable medium of claim 10 , in which each 3D proposal is represented as a 3D box bound to the detected 2D object.

18. The non-transitory computer-readable medium of claim 10 , in which the program code to sample comprises program code to generate the multiple stochastic latent variables using a deep convolutional neural network (CNN).

19. A system for variational 3D object detection, the system comprising:

a stochastic feature module configured to sample multiple, stochastic latent variables from a learned latent feature distribution of an RGB image for a 2D object detected in the RGB image;

a variational 3D lifting module configured to lift a 3D proposal for each stochastic latent variable sampled for the detected 2D object;

a 3D perception module configured to select a 3D proposal for the detected 2D object using a proposal selection algorithm to reduce 3D proposal lifting overlap; and

a planner module configured to plan a trajectory of an ego vehicle according to a 3D location and pose of the 2D object according to the selected 3D proposal.

20. The system of claim 19 , further comprising confidence network module configured to estimate a confidence score for each lifted 3D proposal, and to present the 3D proposal with the highest confidence score as the selected 3D proposal.