System and method for self-supervised monocular ground-plane extraction
Systems and methods for extracting ground plane information directly from monocular images using self-supervised depth networks are disclosed. Self-supervised depth networks are used to generate a three-dimensional reconstruction of observed structures. From this reconstruction the system may generate surface normals. The surface normals can be calculated directly from depth maps in a way that is much less computationally expensive and accurate than surface normals extraction from standard LiDAR data. Surface normals facing substantially the same direction and facing upwards may be determined to reflect a ground plane.
1. A system for generating a ground plane of an environment, comprising:
one or more processors;
a memory communicably coupled to the one or more processors and storing:
a depth system including instructions that when executed by the one or more processors cause the one or more processors to generate a plurality of depth maps by:
receiving at least one monocular image;
processing the at least one monocular image according to a depth model and a first and second scale;
output a first depth map at the first scale and a second depth map at the second scale; and
an image module including instructions that when executed by the one or more processors cause the one or more processors to:
define in the first and second depth maps a plurality of polygons;
for each polygon generate a surface normal;
extract the ground plane from the surface normal for each polygon; and
transmit the first depth map to a first vehicle system and the second depth map to a second vehicle system.
2. The system of claim 1 wherein the plurality of polygons comprises a plurality of triangles.
3. The system of claim 2 wherein each surface normal is generated using a cross product of the triangle.
4. The system of claim 1 wherein extracting the ground plane includes identifying a plurality of surface normals facing substantially the same direction.
5. The system of claim 4 wherein substantially the same direction includes an upward direction.
6. The system of claim 1 wherein the depth model comprises a neural network.
7. The system of claim 6 wherein the neural network is self-supervised.
8. A method for generating a ground plane of an environment, comprising:
receiving at least one monocular image;
generating a plurality of depth maps by processing the at least one monocular image according to a depth model and a first and second scale;
outputting a first depth map at the first scale and a second depth map at the second scale;
defining in the first and second depth maps a plurality of polygons;
for each polygon, generating a surface normal;
extracting a ground plane from the surface normal for each polygon; and
transmit the first depth map to a first vehicle system and the second depth map to a second vehicle system.
9. The method of claim 8 wherein the plurality of polygons comprises a plurality of triangles.
10. The method of claim 9 wherein each surface normal is generated using a cross product of the triangle.
11. The method of claim 9 wherein extracting the ground plane includes identifying a plurality of surface normals facing substantially the same direction.
12. The method of claim 11 wherein substantially the same direction includes an upward direction.
13. The method of claim 8 wherein the depth model comprises a neural network.
14. The method of claim 13 wherein the neural network is self-supervised.
15. A non-transitory computer-readable medium for generating a ground plane for an environment and including instructions that when executed by one or more processors cause the one or more processors to:
receive at least one monocular image;
generate a plurality of depth maps by processing the at least one monocular image according to a depth model and a first and second scale;
output a first depth map at the first scale and a second depth map at the second scale;
define in the first and second depth maps a plurality of polygons;
for each polygon, generate a surface normal;
extract a ground plane from the surface normal for each polygon; and
transmit the first depth map to a first vehicle system and the second depth map to a second vehicle system.
16. The non-transitory computer-readable medium of claim 15 wherein the plurality of polygons comprises a plurality of triangles.
17. The non-transitory computer-readable medium of claim 16 wherein each surface normal is generated using a cross product of the triangle.
18. The non-transitory computer-readable medium of claim 16 wherein substantially the same direction includes an upward direction.
19. The non-transitory computer-readable medium of claim 15 wherein extracting the ground plane includes identifying a plurality of surface normals facing substantially the same direction.
20. The non-transitory computer-readable medium of claim 15 wherein the depth model comprises a self-supervised neural network.