IP Library Granted Patent US 10,871,783
Granted Patent B2
US 10,871,783 · App. 16/138,059 · Granted Dec 22, 2020

Advanced path prediction

Inventors: David Jonathan Julian (San Diego, CA); Avneesh Agrawal (Bengaluru, IN); Venkata Sreekanta Reddy Annapureddy (San Diego, CA)
Assignee: NETRADYNE, INC.
G05D1/0274G05D1/0246G06K9/00791G06K9/3233G06K9/627G06N20/00G06T7/246G06T7/74G05D1/0278G05D2201/0213G06T2207/10016G06T2207/20081G06T2207/20084G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,871,783
App. No.
16/138,059
Granted
Dec 22, 2020
Kind
B2
Abstract

The present disclosure provides systems and methods for mapping a determined path of travel. The path of travel may be mapped to a camera view of a camera affixed to a vehicle. In some embodiments, the path of travel may be mapped to another view that is based on a camera, such as a bird's eye view anchored to the camera's position at a given time. These systems and methods may determine the path of travel by incorporating data from later points in time.

Claims (53)

1. A method comprising:

determining an actual path of travel of a vehicle from a first time to a second time;

receiving a captured image, wherein the captured image is captured by a camera at the first time, and wherein the camera is mounted to the vehicle;

generating a first reference frame associated with the first time from the captured image using an inverse perspective model, wherein the first reference frame is an image derived from the captured image, and wherein the first reference frame is a bird's eye view reference frame anchored to a position of the camera; and

mapping, at or after the second time, the actual path of travel to the first reference frame.

2. The method of claim 1 , wherein mapping the actual path comprises:

orienting the actual path to align with the bird's eye view reference frame; and

scaling the actual path based on a scale of the bird's eye view reference frame.

3. The method of claim 2 , further comprising:

determining a perspective mapping from the bird's eye view reference frame to a camera reference frame; and

mapping the actual path from the first reference frame to the camera reference frame based on the perspective mapping.

4. The method of claim 1 , further comprising:

receiving visual data from the camera at the first time;

determining a location of an object based on the visual data; and

determining a distance between the object and the actual path of travel in the first reference frame.

5. The method of claim 4 , wherein the object is at least one of a second vehicle, a lane boundary, a pedestrian, a traffic sign, an animal, or a road debris.

6. The method of claim 4 , further comprising:

determining a driver behavior metric based at least in part on the determined distance.

7. The method of claim 6 , wherein the object is a second vehicle, and wherein the driver behavior metric is a tailgating score based on the determined distance.

8. The method of claim 1 , further comprising:

predicting a second path of travel of the vehicle;

determining a difference between the predicted second path and the actual path.

9. The method of claim 8 , further comprising:

determining a driver behavior metric based on the determined difference.

10. The method of claim 8 , further comprising:

training a path prediction model based on the determined difference.

11. The method of claim 1 , further comprising:

receiving visual data from the camera at the first time;

estimating a lane location at the first time based on the visual data; and

refining the estimated lane location based on the actual path of travel to produce a refined lane location.

12. The method of claim 11 , further comprising:

training a lane detection model based on the estimated lane location and the refined lane location.

13. The method of claim 1 , further comprising:

receiving visual data from the camera at the first time;

determining a plurality of regions of interest in the visual data based on the actual path of travel;

processing a first region of interest of the plurality of regions of interest at a first resolution;

processing a second region of interest of the plurality of regions of interest at a second resolution; and

detecting at least one object in at least one the first region of interest or the second region of interest.

14. The method of claim 13 , wherein the first resolution is based on an estimated distance of objects within the first region of interest.

15. The method of claim 14 , wherein the estimated distance of the objects is based on at least one of a typical lane width in the first region of interest, or the actual path of travel to the first region of interest, wherein the typical lane width is 3.7 meters.

16. An apparatus comprising:

at least one memory unit; and

at least one processor coupled to the at least one memory unit, in which the at least one processor is configured to:

determine an actual path of travel of a vehicle from a first time to a second time;

receive a captured image, wherein the captured image is captured by a camera at the first time, and wherein the camera is mounted to the vehicle;

generate a first reference frame associated with the first time from the captured image using an inverse perspective model, wherein the first reference frame is an image derived from the captured image, and wherein the first reference frame is a bird's eye view reference frame anchored to a position of the camera; and

map, at or after the second time, the actual path of travel to the first reference frame.

17. A computer program product, the computer program product comprising:

a non-transitory computer-readable medium having program code recorded thereon, the program code, when executed by a processor, causes the processor to:

determine an actual path of travel of a vehicle from a first time to a second time;

receive a captured image, wherein the captured image is captured by a camera at the first time, and wherein the camera is mounted to the vehicle;

generate a first reference frame associated with the first time from the captured image using an inverse perspective model, wherein the first reference frame is an image derived from the captured image, and wherein the first reference frame is a bird's eye view reference frame anchored to a position of the camera; and

map, at or after the second time, the actual path of travel to the first reference frame.

Assignments (7)
SECURITY INTEREST Recorded Apr 6, 2026
From: NETRADYNE, INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 075359/0194 →
SECURITY INTEREST Recorded Apr 6, 2026
From: NETRADYNE, INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 075435/0670 →
SECURITY INTEREST Recorded Mar 9, 2021
From: NETRADYNE, INC.
To: SILICON VALLEY BANK
Reel/Frame 055530/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2019
From: JULIAN, DAVID JONATHAN; AGRAWAL, AVNEESH; ANNAPUREDDY, VENKATA SREEKANTA REDDY
To: NETRADYNE, INC.
Reel/Frame 049709/0776 →
SECURITY INTEREST Recorded Dec 17, 2018
From: NETRADYNE, INC.
To: SILICON VALLEY BANK
Reel/Frame 047798/0554 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2018
From: JULIAN, DAVID JONATHAN; AGRAWAL, AVNEESH; ANNAPUREDDY, VENKATA SR
To: NETRADYNE INC.
Reel/Frame 047757/0911 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2018
From: JULIAN, DAVID JONATHAN; AGRAWAL, AVNEESH; ANNAPUREDDY, VENKATA S. R.
To: NETRADYNE INC.
Reel/Frame 046939/0217 →
Continuity (3)
Continuation PCTUS2017023781 · Mar 23, 2017
Provisional Application 62312445 · Mar 23, 2016
Related Publication 20190025853A1 · Jan 24, 2019
Cited By (5)
US 12,306,298 US 12,307,788 US 12,332,349 US 12,374,224 US 12,488,235