Systems and methods for training a machine-learning-based monocular depth estimator
Systems and methods described herein relate to training a machine-learning-based monocular depth estimator. One embodiment selects a virtual image in a virtual dataset, the virtual image having an associated ground-truth depth map; generates a set of ground-truth surface-normal vectors for pixels in the virtual image based on the ground-truth depth map; processes the virtual image using the machine-learning-based monocular depth estimator to generate a predicted depth map; generates a set of calculated surface-normal vectors for the pixels in the virtual image based on the predicted depth map; and supervises training of the machine-learning-based monocular depth estimator by computing a surface-normal loss between the set of calculated surface-normal vectors and the set of ground-truth surface-normal vectors, wherein the surface-normal loss regularizes depth predictions produced by the machine-learning-based monocular depth estimator to improve accuracy of the depth predictions as the machine-learning-based monocular depth estimator is trained.
1. A system for training a machine-learning-based monocular depth estimator, the system comprising:
one or more processors; and
a memory communicably coupled to the one or more processors and storing:
an input module including instructions that when executed by the one or more processors cause the one or more processors to:
select a virtual image in a virtual dataset, the virtual image having an associated ground-truth depth map; and
generate a set of ground-truth surface-normal vectors for pixels in the virtual image based on the ground-truth depth map; and
a training module including instructions that when executed by the one or more processors cause the one or more processors to:
process the virtual image using the machine-learning-based monocular depth estimator to generate a predicted depth map;
generate a set of calculated surface-normal vectors for the pixels in the virtual image based on the predicted depth map; and
supervise training of the machine-learning-based monocular depth estimator by computing a surface-normal loss between the set of calculated surface-normal vectors and the set of ground-truth surface-normal vectors, wherein the surface-normal loss regularizes depth predictions produced by the machine-learning-based monocular depth estimator to improve accuracy of the depth predictions as the machine-learning-based monocular depth estimator is trained.
2. The system of claim 1 , wherein the instructions in the training module to supervise training of the machine-learning-based monocular depth estimator include further instructions to compute a supervised depth loss between the predicted depth map and the ground-truth depth map.
3. The system of claim 1 , wherein the instructions in the training module to supervise training of the machine-learning-based monocular depth estimator include instructions to compute the surface-normal loss as a cosine similarity function.
4. The system of claim 1 , wherein the ground-truth depth map and the predicted depth map represent respective distances, from a camera, of the pixels in the virtual image as grayscale intensities.
5. The system of claim 1 , wherein the training module includes instructions to train the machine-learning-based monocular depth estimator for deployment in one of an autonomous vehicle, a semi-autonomous vehicle, an Advanced Driver-Assistance System, a search and rescue robot, an aerial drone, and an indoor robot.
6. The system of claim 1 , wherein the machine-learning-based monocular depth estimator includes at least one neural network.
7. The system of claim 1 , wherein the virtual image is a Red-Green-Blue (RGB) image.
8. A non-transitory computer-readable medium for training a machine-learning-based monocular depth estimator and storing instructions that when executed by one or more processors cause the one or more processors to:
select a virtual image in a virtual dataset, the virtual image having an associated ground-truth depth map;
generate a set of ground-truth surface-normal vectors for pixels in the virtual image based on the ground-truth depth map;
process the virtual image using the machine-learning-based monocular depth estimator to generate a predicted depth map;
generate a set of calculated surface-normal vectors for the pixels in the virtual image based on the predicted depth map; and
supervise training of the machine-learning-based monocular depth estimator by computing a surface-normal loss between the set of calculated surface-normal vectors and the set of ground-truth surface-normal vectors, wherein the surface-normal loss regularizes depth predictions produced by the machine-learning-based monocular depth estimator to improve accuracy of the depth predictions as the machine-learning-based monocular depth estimator is trained.
9. The non-transitory computer-readable medium of claim 8 , wherein the instructions to supervise training of the machine-learning-based monocular depth estimator include further instructions to compute a supervised depth loss between the predicted depth map and the ground-truth depth map.
10. The non-transitory computer-readable medium of claim 8 , wherein the instructions to supervise training of the machine-learning-based monocular depth estimator include instructions to compute the surface-normal loss as a cosine similarity function.
11. The non-transitory computer-readable medium of claim 8 , wherein the ground-truth depth map and the predicted depth map represent respective distances, from a camera, of the pixels in the virtual image as grayscale intensities.
12. The non-transitory computer-readable medium of claim 8 , wherein the instructions include instructions to train the machine-learning-based monocular depth estimator for deployment in one of an autonomous vehicle, a semi-autonomous vehicle, an Advanced Driver-Assistance System, a search and rescue robot, an aerial drone, and an indoor robot.
13. The non-transitory computer-readable medium of claim 8 , wherein the machine-learning-based monocular depth estimator includes at least one neural network.
14. A method of training a machine-learning-based monocular depth estimator, the method comprising:
selecting a virtual image in a virtual dataset, the virtual image having an associated ground-truth depth map;
generating a set of ground-truth surface-normal vectors for pixels in the virtual image based on the ground-truth depth map;
processing the virtual image using the machine-learning-based monocular depth estimator to generate a predicted depth map;
generating a set of calculated surface-normal vectors for the pixels in the virtual image based on the predicted depth map; and
supervising training of the machine-learning-based monocular depth estimator by computing a surface-normal loss between the set of calculated surface-normal vectors and the set of ground-truth surface-normal vectors, wherein the surface-normal loss regularizes depth predictions produced by the machine-learning-based monocular depth estimator to improve accuracy of the depth predictions as the machine-learning-based monocular depth estimator is trained.
15. The method of claim 14 , wherein supervising training of the machine-learning-based monocular depth estimator further includes computing a supervised depth loss between the predicted depth map and the ground-truth depth map.
16. The method of claim 14 , wherein the surface-normal loss is computed as a cosine similarity function.
17. The method of claim 14 , wherein the ground-truth depth map and the predicted depth map represent respective distances, from a camera, of the pixels in the virtual image as grayscale intensities.
18. The method of claim 14 , wherein the machine-learning-based monocular depth estimator is trained for deployment in one of an autonomous vehicle, a semi-autonomous vehicle, an Advanced Driver-Assistance System, a search and rescue robot, an aerial drone, and an indoor robot.
19. The method of claim 14 , wherein the machine-learning-based monocular depth estimator includes at least one neural network.
20. The method of claim 14 , wherein the virtual image is a Red-Green-Blue (RGB) image.