IP Library Granted Patent US 9,928,431
Granted Patent B2
US 9,928,431 · App. 15/186,808 · Granted Mar 27, 2018

Verifying a target object with reverse-parallax analysis

Inventors: Bradley Templeton (Sunnyvale, CA); Nathaniel Fairfield (Mountain View, CA); Dave Ferguson (San Francisco, CA)
Assignee: Waymo LLC
G06K9/00805B60W30/14G05D1/0246G06T7/00G06T7/292G06T7/73G06T7/77H04N5/2253H04N5/247G06K9/00825G06T2207/10016G06T2207/30236G06T2207/30241G06T2207/30244G06T2207/30252G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,928,431
App. No.
15/186,808
Granted
Mar 27, 2018
Kind
B2
Abstract

A vehicle configured to operate in an autonomous mode may engage in a reverse-parallax analysis that includes a vehicle system detecting an object, capturing via a camera located at a first location a first image of the detected object, retrieving location data specifying (i) a location of a target object, (ii) the first location, and (iii) a direction of the camera, and based on the location data and the position of the detected object in the first image, predicting where in a second image captured from a second location the detected object would appear if the detected object is the target object.

Claims (56)

1. A method comprising:

a vehicle system detecting an object, the vehicle system being communicatively coupled with a vehicle;

the vehicle system simultaneously (i) capturing, via a first camera located at a first location, a first image of the detected object, the detected object having a position in the first image, and (ii) capturing, via a second camera located at a second location, a second image of the detected object;

the vehicle system retrieving location data specifying (a) a location of a target object, (b) the first location, (c) the second location, (d) a direction of the first camera, and (e) a direction of the second camera; and

based on the location data and the position of the detected object in the first image, the vehicle system predicting where in the second image the detected object would appear if the detected object is the target object.

2. The method of claim 1 , wherein the first location is a location of the first camera mounted on the vehicle, and wherein the second location is a location of the second camera mounted on the vehicle.

3. The method of claim 2 , wherein the first location and the second location are distinct locations.

4. The method of claim 1 , further comprising:

the vehicle system evaluating whether the detected object appears in a predetermined area of the second image, the predetermined area being based on the prediction of where in the second image the detected object would appear if the detected object is the target object.

5. The method of claim 4 , further comprising:

based on the evaluating, the vehicle system determining that the detected object does not appear in the predetermined area of the second image; and

in response to the determining, the vehicle system identifying the detected object as an object other than the target object.

6. The method of claim 5 , wherein the vehicle system predicting where in the second image the detected object would appear if the detected object is the target object comprises the vehicle system designating as a prediction zone a certain number of pixels of the second image, and

wherein evaluating whether the detected object appears in a predetermined area of the second image comprises the vehicle system determining whether a threshold number of pixels of the detected object appears within the prediction zone.

7. The method of claim 6 , further comprising:

based on the location data and a position of the detected object in the second image; the vehicle system predicting where in the first image the detected object would appear if the detected object is the target object.

8. The method of claim 7 , further comprising:

the vehicle system evaluating whether the detected object appears in a predetermined area of the first image, the predetermined area being based on the prediction of where in the first image the detected object would appear if the detected object is the target object.

9. The method of claim 1 , further comprising:

after predicting where in the second image the detected object would appear if the detected object is the target object, the vehicle system capturing, via a third camera located at a third location, a third image of the detected object;

the vehicle system retrieving additional location data specifying (a) the third location and (b) a direction of the third camera; and

based on the additional location data and further based on the predicting of where in the second image the detected object would appear if the detected object is the target object, the vehicle system predicting where in the third image the detected object would appear if the detected object is the target object.

10. The method of claim 9 , wherein the third location is distinct from one or both of the first location and the second location.

11. The method of claim 9 , wherein the third camera is the same camera as either the first camera or the second camera, and

wherein the third location is the same location as either the first location or the second location.

12. The method of claim 1 , further comprising:

after predicting where in the second image the detected object would appear if the detected object is the target object, based on the location data and further based on the predicting of where in the second image the detected object would appear if the detected object is the target object, the vehicle system (i) selecting a third location distinct from the first location and the second location and (ii) predicting where in a third image captured from the third location the detected object would appear if the detected object is the target object, wherein the predicting where in the third image the detected object would appear if the detected object is the target object occurs without having captured the third image from the third location; and

after selecting the third location and predicting where in the third image the detected object would appear if the detected object is the target object, the vehicle system causing the vehicle to move to the third location.

13. The method of claim 12 , wherein the vehicle system selecting a third location distinct from the first location and the second location comprises:

the vehicle system selecting a third location along a predetermined travel pattern of the vehicle.

14. The method of claim 12 , wherein the vehicle system selecting a third location distinct from the first location and the second location comprises:

the vehicle system selecting a third location that is not along a predetermined travel pattern of the vehicle.

15. The method of claim 1 , wherein the first location and the second location are along a predetermined travel pattern of the vehicle.

16. A vehicle comprising:

a vision system;

at least one processor; and

data storage comprising instructions executable by the at least one processor to cause the system to perform operations, the operations comprising:

causing the vision system to detect an object;

causing the vision system to simultaneously (i) capture, via a first camera located at a first location, a first image of the detected object, the detected object having a position in the first image, and (ii) capture, via a second camera located at a second location, a second image of the detected object;

retrieving location data specifying (a) a location of a target object, (b) the first location, (c) the second location, (d) a direction of the first camera, and (e) a direction of the second camera; and

based on the location data and the position of the detected object in the first image, predicting where in the second image the detected object would appear if the detected object is the target object.

17. The vehicle of claim 16 , the operations further comprising:

after predicting where in the second image the detected object would appear if the detected object is the target object, capturing, via a third camera located at a third location, a third image of the detected object;

retrieving additional location data specifying (a) the third location and (b) a direction of the third camera; and

based on the additional location data and further based on the predicting of where in the second image the detected object would appear if the detected object is the target object, predicting where in the third image the detected object would appear if the detected object is the target object.

18. A non-transitory computer readable medium (CRM) having stored thereon instructions executable by a computing device to cause the computing device to perform operations, the operations comprising:

detecting an object in an environment of a vehicle;

simultaneously (i) capturing, via a first camera located at a first location, a first image of the detected object, the detected object having a position in the first image, and (ii) capturing, via a second camera located at a second location, a second image of the detected object;

retrieving location data specifying (a) a location of a target object, (b) the first location, (c) the second location, (d) a direction of the first camera, and (e) a direction of the second camera; and

based on the location data and the position of the detected object in the first image, predicting where in the second image the detected object would appear if the detected object is the target object.

19. The CRM of claim 18 , wherein predicting where in the second image the detected object would appear if the detected object is the target object comprises designating as a prediction zone a certain number of pixels of the second image, and

wherein the operations further comprise:

evaluating whether the detected object appears in a predetermined area of the second image by determining whether a threshold number of pixels of the detected object appears within the prediction zone.

20. The CRM of claim 18 , the operations further comprising:

after predicting where in the second image the detected object would appear if the detected object is the target object, based on the location data and further based on the predicting of where in the second image the detected object would appear if the detected object is the target object, (i) selecting a third location distinct from the first location and the second location and (ii) predicting where in a third image captured from the third location the detected object would appear if the detected object is the target object, wherein the predicting where in the third image the detected object would appear if the detected object is the target object occurs without having captured the third image from the third location; and

after selecting the third location and predicting where in the third image the detected object would appear if the detected object is the target object, causing the vehicle to move to the third location.

Assignments (6)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE REMOVAL OF THE INCORRECTLY RECORDED APPLICATION NUMBERS 14/149802 AND 15/419313 PREVIOUSLY RECORDED AT REEL: 44144 FRAME: 1. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Mar 4, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 068092/0502 →
THE CORRECTION BY DECLARATION OF ERRONEOUS FILED PATENT NUMBER 15176535 RECORDED AT REEL/FRAME 044144/0001 Recorded Apr 14, 2021
From: WAYMO LLC
To: WAYMO LLC
Reel/Frame 056761/0349 →
CHANGE OF NAME Recorded Oct 6, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044144/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2017
From: GOOGLE INC.
To: WAYMO HOLDING INC.
Reel/Frame 042084/0741 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2017
From: WAYMO HOLDING INC.
To: WAYMO LLC
Reel/Frame 042085/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2016
From: TEMPLETON, BRADLEY; FAIRFIELD, NATHANIEL; FERGUSON, DAVE
To: GOOGLE INC.
Reel/Frame 038957/0629 →
Continuity (2)
Continuation 13622915 · Sep 19, 2012
Related Publication 20160292519A1 · Oct 6, 2016