System and method for self-supervised monocular depth regularization from surface normals
Systems and methods for generating depth models and depth maps from images obtained from an imaging system are presented. A self-supervised neural network may be capable of regularizing depth information from surface normals. Rather than rely on separate depth and surface normal networks, surface normal information is extracted from the depth information and a smoothness function is applied to the surface normals instead of a depth gradient. Smoothing the surface normal may provide improved representation of environmental structures by both smoothing texture-less areas while preserving sharp boundaries between structures.
1. A system for generating depth estimates of an environment, comprising:
one or more processors;
a memory communicably coupled to the one or more processors and storing:
a depth system including instructions that when executed by the one or more processors cause the one or more processors to:
receive at least one monocular image;
process the at least one monocular image according to a depth model to generate a depth map; and
an image module including instructions that when executed by the one or more processors cause the one or more processors to:
define in the depth map a plurality of polygons;
generate a surface normal for each polygon; and
apply a filter function to the surface normal.
2. The system of claim 1 wherein the plurality of polygons comprises a plurality of triangles.
3. The system of claim 2 wherein each surface normal is generated using a cross product of the triangle.
4. The system of claim 1 wherein the filter function is a smoothing function.
5. The system of claim 1 wherein the depth model comprises a neural network.
6. The system of claim 5 wherein the neural network is self-supervised.
7. The system of claim 1 wherein the filtered surface normal are input into the depth map as training data.
8. A method for generating depth estimates of an environment, comprising:
receiving at least one monocular image;
generating a depth map by processing the at least one monocular image according to a depth model;
defining in the depth map a plurality of polygons;
generating a surface normal for each polygon; and
applying a filter function to the surface normal.
9. The method of claim 8 wherein the plurality of polygons comprises a plurality of triangles.
10. The method of claim 9 wherein each surface normal is generated using a cross product of the triangle.
11. The system of claim 8 wherein the filter function is a smoothing function.
12. The system of claim 8 wherein the depth model comprises a neural network.
13. The system of claim 12 wherein the neural network is self-supervised.
14. The system of claim 8 further comprising inputting the filtered surface normal into the depth map as training data.
15. A non-transitory computer-readable medium for generating depth estimates of an environment and including instructions that when executed by one or more processors cause the one or more processors to:
receive at least one monocular image;
generate a depth map by processing the at least one monocular image according to a depth model;
define in the depth map a plurality of polygons;
generate a surface normal for each polygon; and
apply a filter function to the surface normal.
16. The non-transitory computer-readable medium of claim 15 wherein the plurality of polygons comprises a plurality of triangles.
17. The non-transitory computer-readable medium of claim 16 wherein each surface normal is generated using a cross product of the triangle.
18. The non-transitory computer-readable medium of claim 16 further comprising instructions that when executed by one or more processors cause the one or more processors to input the filtered surface normal into the depth map as training data.
19. The non-transitory computer-readable medium of claim 15 wherein the filter function is a smoothing function.
20. The non-transitory computer-readable medium of claim 15 wherein the depth model comprises a neural network.