Three-dimensional object estimation using two-dimensional annotations
A method includes obtaining a two-dimensional image, obtaining a two-dimensional image annotation that indicates presence of an object in the two-dimensional image, obtaining three-dimensional sensor information, generating a top-down representation of the three-dimensional sensor information, and obtaining a top-down annotation that indicates presence of the object in the top-down representation. The method also includes determining a bottom surface of a three-dimensional cuboid based on map information, determining a position, a length, a width, and a yaw rotation of the three-dimensional cuboid based on the top-down annotation, and determining a height of the three-dimensional cuboid based on a two-dimensional image annotation, and the position, the length, the width, and the yaw rotation of the three-dimensional cuboid.
1. A non-transitory computer-readable storage device including program instructions executable by one or more processors that, when executed, cause the one or more processors to perform operations, the operations comprising:
obtaining a two-dimensional image;
obtaining a two-dimensional image annotation that indicates presence of an object in the two-dimensional image;
obtaining three-dimensional sensor information;
generating a top-down representation of the three-dimensional sensor information;
obtaining a top-down annotation that indicates presence of the object in the top-down representation;
determining a bottom surface of a three-dimensional cuboid based on map information;
determining a position, a length, a width, and a yaw rotation of the three-dimensional cuboid based on the top-down annotation; and
determining a height of the three-dimensional cuboid based on the two-dimensional image annotation, and the position, the length, the width, and the yaw rotation of the three-dimensional cuboid.
2. The non-transitory computer-readable storage device of claim 1 , wherein the three-dimensional sensor information includes a point cloud.
3. The non-transitory computer-readable storage device of claim 2 , wherein generating the top-down representation of the three-dimensional sensor information is performed by plotting lateral locations of points from the point cloud in a two-dimensional coordinate space.
4. The non-transitory computer-readable storage device of claim 1 , wherein the two-dimensional image annotation is obtained using an automated two-dimensional image annotation process.
5. The non-transitory computer-readable storage device of claim 1 , wherein the top-down annotation is obtained using an automated top-down annotation process.
6. The non-transitory computer-readable storage device of claim 1 , wherein the height of the three-dimensional cuboid is determined based on a geometric projection of the position, the length, the width, and the yaw rotation of the three-dimensional cuboid relative to the two-dimensional image.
7. The non-transitory computer-readable storage device of claim 1 , wherein the position of the three-dimensional cuboid includes a lateral location of the object, and the bottom surface of the three-dimensional cuboid is determined further based on the lateral location of the object.
8. A method, comprising:
obtaining a two-dimensional image;
obtaining a two-dimensional image annotation that indicates presence of an object in the two-dimensional image;
obtaining three-dimensional sensor information;
generating a top-down representation of the three-dimensional sensor information;
obtaining a top-down annotation that indicates presence of the object in the top-down representation;
determining a bottom surface of a three-dimensional cuboid based on map information; and
determining a position, a length, a width, and a yaw rotation of the three-dimensional cuboid based on the top-down annotation; and
determining a height of the three-dimensional cuboid based on the two-dimensional image annotation, and the position, the length, the width, and the yaw rotation of the three-dimensional cuboid.
9. The method of claim 8 , wherein the three-dimensional sensor information includes a point cloud.
10. The method of claim 9 , wherein generating the top-down representation of the three-dimensional sensor information is performed by plotting lateral locations of points from the point cloud in a two-dimensional coordinate space.
11. The method of claim 8 , wherein the two-dimensional image annotation is obtained using an automated two-dimensional image annotation process.
12. The method of claim 8 , wherein the top-down annotation is obtained using an automated top-down annotation process.
13. The method of claim 8 , wherein the height of the three-dimensional cuboid is determined based on a geometric projection of the position, the length, the width, and the yaw rotation of the three-dimensional cuboid relative to the two-dimensional image.
14. The method of claim 8 , wherein the position of the three-dimensional cuboid includes a lateral location of the object, and the bottom surface of the three-dimensional cuboid is determined further based on the lateral location of the object.
15. A system, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
obtain a two-dimensional image;
obtain a two-dimensional image annotation that indicates presence of an object in the two-dimensional image;
obtain three-dimensional sensor information;
generate a top-down representation of the three-dimensional sensor information;
obtain a top-down annotation that indicates presence of the object in the top-down representation;
determine a bottom surface of a three-dimensional cuboid based on map information;
determine a position, a length, a width, and a yaw rotation of the three-dimensional cuboid based on the top-down annotation; and
determine a height of the three-dimensional cuboid based on the two-dimensional image annotation, and the position, the length, the width, and the yaw rotation of the three-dimensional cuboid.
16. The system of claim 15 , wherein the three-dimensional sensor information includes a point cloud.
17. The system of claim 16 , wherein generating the top-down representation of the three-dimensional sensor information is performed by plotting lateral locations of points from the point cloud in a two-dimensional coordinate space.
18. The system of claim 15 , wherein the two-dimensional image annotation is obtained using an automated two-dimensional image annotation process.
19. The system of claim 15 , wherein the top-down annotation is obtained using an automated top-down annotation process.
20. The system of claim 15 , wherein the height of the three-dimensional cuboid is determined based on a geometric projection of the position, the length, the width, and the yaw rotation of the three-dimensional cuboid relative to the two-dimensional image.
21. The system of claim 15 , wherein the position of the three-dimensional cuboid includes a lateral location of the object, and the bottom surface of the three-dimensional cuboid is determined further based on the lateral location of the object.