Advanced path prediction
The present disclosure provides systems and methods for mapping a determined path of travel. The path of travel may be mapped to a camera view of a camera affixed to a vehicle. In some embodiments, the path of travel may be mapped to another view that is based on a camera, such as a bird's eye view anchored to the camera's position at a given time. These systems and methods may determine the path of travel by incorporating data from later points in time.
1. A method, comprising:
determining an actual path of travel of a camera from a first time to a second time, wherein the camera is mounted to a vehicle;
receiving a captured image, wherein the captured image is captured by the camera at the first time;
generating an overhead view from the captured image; wherein the overhead view is anchored to a position of the camera; and
mapping the actual path of travel to the overhead view.
2. The method of claim 1 , wherein the overhead view is generated using an inverse perspective model.
3. The method of claim 1 , wherein the overhead view is a bird's eye view reference frame anchored to the position of the camera.
4. The method of claim 1 , wherein the actual path of travel is determined based at least in part on inertial sensor input data.
5. The method of claim 1 , wherein mapping the actual path comprises:
orienting the actual path of travel to align with the overhead view; and
scaling the actual path of travel based on a scale of the overhead view.
6. The method of claim 1 , further comprising:
detecting an object in the captured image; and
determining a location, in the captured image, of the object.
7. The method of claim 6 , further comprising:
mapping the location of the object in the captured image to a location of the object in the overhead view.
8. The method of claim 7 , further comprising:
determining a distance between the object and the actual path of travel, based on the location of the object in the overhead view and the actual path of travel in the overhead view.
9. The method of claim 6 , further comprising:
determining a perspective mapping from the overhead view to a camera reference frame, wherein the camera reference frame corresponds to the captured image;
mapping the actual path of travel from the overhead view to the camera reference frame based on the perspective mapping; and
determining a distance between the object and the actual path of travel, based on the location of the object in the captured image and the actual path of travel in the camera reference frame.
10. The method of claim 6 , further comprising:
determining that the object and the vehicle are not in the same lane, based at least in part on the actual path of travel.
11. The method of claim 6 , wherein the object is a second vehicle.
12. The method of claim 11 , wherein the second vehicle is parked.
13. The method of claim 6 , wherein the object is a lane boundary.
14. The method of claim 1 , further comprising:
mapping the actual path of travel to a second reference frame.
15. The method of claim 14 , wherein the second reference frame is a predicted camera view of the camera at a third time.
16. The method of claim 14 , wherein the second reference frame is a camera view of a stationary camera.
17. The method of claim 14 , wherein the second reference frame is a satellite image.
18. The method of claim 1 , further comprising:
estimating a lane location based at least in part on the captured image; and
refining the estimated lane location based on the actual path of travel to produce a refined lane location.
19. The method of claim 18 , further comprising:
training a path prediction model based on the estimated lane location and the refined lane location.
20. The method of claim 19 , wherein training the path prediction model is further based on satellite image data for which a person has labeled:
boundaries of a visible lane;
an appropriate future driving path of travel; or
some combination thereof.
21. An apparatus comprising:
at least one memory unit; and
at least one processor coupled to the at least one memory unit, in which the at least one processor is configured to:
determine an actual path of travel of a camera from a first time to a second time;
receive a captured image, wherein the captured image is captured by the camera at the first time, and wherein the camera is mounted to a vehicle;
generate an overhead view from the captured image; wherein the overhead view is anchored to a position of the camera; and
map the actual path of travel to the overhead view.
22. A computer program product, the computer program product comprising:
a non-transitory computer-readable medium having program code recorded thereon, the program code, when executed by a processor, causes the processor to:
determine an actual path of travel of a camera from a first time to a second time;
receive a captured image, wherein the captured image is captured by the camera at the first time, and wherein the camera is mounted to a vehicle;
generate an overhead view from the captured image; wherein the overhead view is anchored to a position of the camera; and
map the actual path of travel to the overhead view.