IP Library › Granted Patent US 10,318,826
Granted Patent B2
US 10,318,826 · App. 15/288,042 · Granted Jun 11, 2019

Rear obstacle detection and distance estimation

Inventors: Yi Zhang (Sunnyvale, CA); Vidya Nariyambut Murali (Sunnyvale, CA); Madeline J Goh (Palo Alto, CA)
Assignee: FORD GLOBAL TECHNOLOGIES, LLC
G06K9/00805G06K9/4671
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,318,826
App. No.
15/288,042
Granted
Jun 11, 2019
Kind
B2
Abstract

The disclosure relates to systems and methods for estimating or determining the motion of a vehicle and/or the distance to objects within view of a rear camera. A method for rear obstacle detection using structure from motion includes identifying image features in a first frame corresponding to features in a second frame, wherein the first frame and the second frame comprise adjacent image frames captured by a rear-facing camera of a vehicle. The method includes determining parameters for a non-planar motion model based on the image features. The method includes determining camera motion based on the parameters for the non-planar motion model.

Claims (38)

1. A method for rear obstacle detection using structure from motion, the method comprising:

identifying image features in a first frame corresponding to features in a second frame, wherein the first frame and the second frame comprise adjacent image frames captured by a rear-facing camera of a vehicle;

determining parameters for a non-planar motion model based on the image features;

generating a three-dimensional reconstruction of feature points of the image features based on the non-planar motion model;

determining a height of one or more objects based on corresponding feature points of the feature points;

determining a distance to the one or more objects having a height above a threshold;

determining camera motion based on the parameters for the non-planar motion model; and

notifying a driver or automated driving system of a presence of the one or more objects having the height above the threshold.

2. The method of claim 1 , further comprising identifying one or more feature points as part of a same object based on one or more of a pixel intensity or two-dimensional location within the first frame or second frame.

3. The method of claim 1 , further comprising determining a scale for location of the feature points in three-dimensional space based on feature points within a predefined ground area in the first frame and the second frame.

4. The method of claim 1 , further comprising classifying features as inliers and outliers, wherein determining the parameters comprises determining based on the inliers.

5. The method of claim 1 , further comprising performing local bundle adjustment on image features for improved accuracy.

6. A system for rear obstacle detection using structure from motion, the system comprising:

a monocular camera of a vehicle; and

a vehicle controller in communication with the rear-facing camera, wherein the vehicle controller is configured to:

obtain a series of image frames captured by the monocular camera during movement of the vehicle;

identify image features in a first frame corresponding to features in a second frame, wherein the first frame and the second frame comprise adjacent image frames in the series of image frames;

determine parameters for a non-planar motion model based on the image features;

generate a three-dimensional reconstruction of feature points of the image features based on the non-planar motion model;

determine a height of one or more objects based on corresponding feature points of the feature points;

determine a distance to the one or more objects having a height above a threshold;

determine camera motion based on the parameters for the non-planar motion model; and

notify a driver or automated driving system of a presence of the one or more objects having the height above the threshold.

7. The system of claim 6 , wherein the vehicle controller is further configured to identify one or more feature points as part of a same object based on one or more of a pixel intensity or two-dimensional location within the first frame or second frame.

8. The system of claim 6 , wherein the vehicle controller is further configured to determine a scale for location of the feature points in three-dimensional space based on feature points within a predefined ground area in the first frame and the second frame.

9. The system of claim 6 , wherein the vehicle controller is further configured to classify features as inliers or outliers, wherein determining the parameters comprises determining based on the inliers.

10. The method of claim 6 , wherein the vehicle controller is further configured to perform local bundle adjustment on image features for improved accuracy.

11. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:

identify image features in a first frame corresponding to features in a second frame, wherein the first frame and the second frame comprise adjacent image frames captured by a rear-facing camera of a vehicle;

determine parameters for a non-planar motion model based on the image features;

generate a three-dimensional reconstruction of feature points of the image features based on the non-planar motion model;

determine a height of one or more objects based on corresponding feature points of the feature points;

determine a distance to the one or more objects having a height above a threshold;

determine camera motion based on the parameters for the non-planar motion model; and

notify a driver or automated driving system of a presence of the one or more objects having the height above the threshold.

12. The non-transitory computer readable storage media of claim 11 , wherein the instructions further cause the one or more processors to identify one or more feature points as part of a same object based on one or more of a pixel intensity or two-dimensional location within the first frame or second frame.

13. The non-transitory computer readable storage media of claim 11 , wherein the instructions further cause the one or more processors to determine a scale for location of the feature points in three-dimensional space based on feature points within a predefined ground area in the first frame and the second frame.

14. The non-transitory computer readable storage media of claim 11 , wherein the instructions further cause the one or more processors to classify features as inliers and outliers, wherein determining the parameters comprises determining based on the inliers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2016
From: ZHANG, YI; NARIYAMBUT MURALI, VIDYA; GOH, MADELINE J
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 039964/0928 →
Continuity (1)
Related Publication 20180101739A1 · Apr 12, 2018
Cited By (3)
US 12,315,265 US 12,354,302 US 12,694,539