Camera initialization for lane detection and distance estimation using single-view geometry
Disclosed are methods, devices, and computer-readable media for detecting lanes and objects in image frames of a monocular camera. In one embodiment, a method is disclosed comprising receiving a plurality of images; identifying a horizon in the plurality of images by inputting the plurality of images into a deep learning (DL) model (either stored on a local device or via a network call); determining one or more camera parameters based on the horizon; and storing or using the camera parameters to initialize a camera.
1. A method comprising:
receiving, over a network from a camera device, a video comprising a set of image frames;
identifying one or more lines in the video using a predictive model, the one or more lines including a horizon line;
computing at least one camera parameter based on the one or more lines;
overlaying the one or more lines on the video to generate an overlaid video;
transmitting the overlaid video to an annotator device for manual review;
receiving a confirmation from the annotator device, the confirmation indicating that the one or more lines are accurate; and
transmitting data representing the at least one camera parameter to the camera device.
2. The method of claim 1 , wherein the one or more lines includes at least one lane line of a roadway.
3. The method of claim 1 , wherein computing the at least one camera parameter comprises predicting the at least one camera parameter using the predictive model.
4. The method of claim 1 , wherein computing the at least one camera parameter comprises predicting the at least one camera parameter using a second predictive model.
5. The method of claim 1 , wherein the at least one camera parameter comprises a camera height and road plane normal.
6. The method of claim 1 , further comprising:
receiving, over the network from the camera device, a second video comprising a second set of images;
transmitting the second video to the annotator device for manual review;
adding, by the annotator device, one or more lines to the second video to generate a second overlaid video;
predicting, using the one or more lines in the second overlaid video, at least one camera parameter; and
transmitting the at least one camera parameter to the annotator device.
7. The method of claim 6 , further comprising:
receiving a second confirmation from the annotator device, the second confirmation indicating that the at least one camera parameter are accurate; and
transmitting data representing the one or more lines of the second video to the camera device.
8. A method comprising:
receiving a video, the video comprising one or more image frames;
generating a set of intermediate features representing the video using a backbone network;
inputting the set of intermediate features into a lane or horizon prediction head, the lane or horizon prediction head comprising a neural network;
predicting a horizon line using the lane or horizon prediction head, the horizon line comprising a set of horizon markers; and
using the horizon line to compute at least one camera parameter.
9. The method of claim 8 , the backbone network comprising at least one convolutional layer.
10. The method of claim 8 , wherein the at least one camera parameter comprises one of a camera height, camera angle, and road plane normal.
11. The method of claim 8 , further comprising predicting, using the lane or horizon prediction head, at least one lane line, the at least one lane line comprising a set of lane line markers.
12. The method of claim 11 , wherein generating a plurality of lane key points comprises generating a plurality of lane key points for two lane lines on a roadway.
13. The method of claim 8 , further comprising detecting a movement of a camera that captured the video and re-initializing the at least one camera parameter in response.
14. The method of claim 8 , wherein the lane or horizon prediction head comprises at least one convolutional layer.
15. The method of claim 14 , wherein the lane or horizon prediction head further comprises at least one fully connected layer receiving an output of the convolutional layer.
16. The method of claim 14 , wherein the lane or horizon prediction head further comprises a layer selected from the group consisting of a recurrent neural network layer, long short-term memory layer, and gated recurrent unit layer.
17. A method comprising:
receiving a video, the video comprising one or more image frames;
generating a set of intermediate features representing the video using a backbone network;
inputting the set of intermediate features into a camera parameter estimation head, the camera parameter estimation head comprising a neural network; and
predicting at least one camera parameter using the camera parameter estimation head, the camera parameter estimation head trained using an output of the backbone network and an output of a lane and horizon line prediction head.
18. The method of claim 17 , wherein the backbone network, camera parameter estimation head, lane and horizon line prediction head are trained using a joint loss function, the joint loss function aggregating the losses of the lane and horizon line prediction head and the neural network.
19. The method of claim 17 , further comprising predicting a horizon line using the lane and horizon line prediction head, the lane and horizon line prediction head comprising a neural network configured to receive the set of intermediate features as an input.
20. The method of claim 17 , wherein the camera parameter estimation head comprises at least one convolutional layer.