IP Library Granted Patent US 8,630,806
Granted Patent B1
US 8,630,806 · App. 13/277,580 · Granted Jan 14, 2014

Image processing for vehicle control

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,630,806
App. No.
13/277,580
Granted
Jan 14, 2014
Kind
B1
Abstract

A method is provided for processing an image in which only parts of the image that appear above a point on a horizon line are analyzed to identify an object. In one embodiment, the distance between the object and a vehicle is determined, and at least one of the speed and direction of the vehicle is changed when it is determined that the distance is less than the range of a sensor. The method for processing images is not limited to vehicular applications only and it may be used in all applications where computer vision is used to identify objects in an image.

Claims (46)

1. A vehicle comprising:

a sensor configured to detect objects in the vehicle's surroundings, the sensor being associated with a range;

a camera configured to capture a first image of a scene of at least a part of the vehicle's surroundings and a second image overlapping at least a part of the scene of the first image;

a memory; and

a processor coupled to the memory, the sensor, and the camera, the processor being configured to:

determine, based on the vehicle's pitch, a point P on a visual horizon in the first image;

process the first image and the second image to determine distance between the vehicle and an object that is depicted in the first image above and beyond the point P; and

change at least one of a speed of the vehicle and a heading of the vehicle when it is determined that the distance between the vehicle and the object is less than the range of the sensor.

2. The vehicle of claim 1 , wherein

the camera is a stereo camera.

3. The vehicle of claim 1 , wherein the processor changes at least one of the speed and heading of the vehicle based on at least one of:

(i) a grade of a surface on which the vehicle is standing or driving exceeding a threshold, and

(ii) pitch of the vehicle exceeding a threshold.

4. The vehicle of claim 1 , wherein the changing of the heading of the vehicle comprises making a turn onto another roadway.

5. The vehicle of claim 1 , wherein the changing of the heading of the vehicle comprises changing lanes.

6. The vehicle of claim 1 , wherein the changing of the heading of the vehicle comprises selecting a new travel route for the vehicle.

7. A system comprising:

a camera configured to capture a first image and a second image;

a sensor configured to sense the pitch of the camera;

a processor coupled to the camera and to the sensor, the processor being configured to:

select an object that is depicted in the first image based on the object being depicted above and beyond a point P on a visual horizon in the first image, the point P being determined by the processor based on signal from the sensor that senses the pitch of the camera; and

process the first image and the second image to determine a distance between the camera and the reference object;

receive input from a second sensor having a range;

determine a geographic location of the vehicle based on the received input;

change at least one of a speed of a vehicle and a heading of the vehicle when it is determined that the distance between a vehicle and the object is less than the range of the second sensor.

8. The apparatus of claim 7 wherein

the camera is a stereo camera.

9. The apparatus of claim 7 wherein the processor is further configured to select the object further based on the reference object being represented in a database for storing object representations.

10. The apparatus of claim 7 wherein processor is further configured to select the object further based on proximity to the point P.

11. The apparatus of claim 7 wherein the processor is further configured to:

determine a geographic location of the reference object by extracting a template of the object from the first image and comparing the template against a database of object representations; and

determine a geographic location of the apparatus based on the geographic location of the reference object.

12. A method comprising:

controlling operation of a vehicle with a first control strategy;

capturing, by a camera, a first image of a scene at least part of the vehicle's surroundings and a second image overlapping at least a part of the scene of the first image; and

processing the first image and the second image to determine a distance between the vehicle and an object appearing above and beyond a point P on a visual horizon in the first image, wherein:

(i) the point P is determined based on the pitch of the vehicle, and

(ii) the object is selected based on the object being depicted, at least in part, above the point P in the image.

13. The method of claim 12 further comprising switching to controlling the vehicle with a second control strategy based on the distance between the vehicle and the object.

14. The method of claim 13 , wherein controlling the operation of the vehicle with the first control strategy includes maneuvering the vehicle in a first direction, and controlling the operation of the vehicle with the second control strategy includes maneuvering the vehicle in a second direction different from the first direction.

15. The method of claim 13 wherein the switching is performed when the distance is less than a range associated with a sensor.

16. The method of claim 12 comprising

defining a ray R extending from the camera; and

projecting the ray R against a plane of the first image;

wherein the point P is a point where the projected ray R intercepts the plane of the first image.

17. The method of claim 12 , wherein the camera is a stereo camera.

Assignments (7)
SUBMISSION TO CORRECT AN ERROR MADE IN A PREVIOUSLY RECORDED DOCUMENT THAT ERRONEOUSLY AFFECTS THE IDENTIFIED APPLICATIONS Recorded Nov 22, 2019
From: WAYMO LLC
To: WAYMO LLC
Reel/Frame 051093/0861 →
CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECTIVE BY NULLIFICATIONTO CORRECT INCORRECTLY RECORDED APPLICATION NUMBERS PREVIOUSLY RECORDED ON REEL 044142 FRAME 0357. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Nov 14, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 047837/0678 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2018
From: WAYMO HOLDING INC.
To: WAYMO LLC
Reel/Frame 047571/0274 →
CHANGE OF NAME Recorded Oct 6, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044142/0357 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2017
From: WAYMO HOLDING INC.
To: WAYMO LLC
Reel/Frame 042108/0021 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2017
From: GOOGLE INC.
To: WAYMO HOLDING INC.
Reel/Frame 042099/0935 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 17, 2011
From: ZHU, JIAJUN
To: GOOGLE INC.
Reel/Frame 027242/0661 →