IP Library Granted Patent US 9,205,776
Granted Patent B2
US 9,205,776 · App. 14/282,029 · Granted Dec 8, 2015

Vehicle vision system using kinematic model of vehicle motion

Inventor: James Turk (Oakville, CA)
Assignee: MAGNA ELECTRONICS INC.
B60R1/00B60R2300/105B60R2300/30B60R2300/402
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,205,776
App. No.
14/282,029
Granted
Dec 8, 2015
Kind
B2
Abstract

A vision system of a vehicle includes a camera disposed at the vehicle and having a field of view exterior of the vehicle. The camera is operable to capture frames of image data. Responsive to image processing by an image processor of captured image data, a control is operable to determine objects present in the field of view of the camera. Responsive to vehicle data, the control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle. The control determines movement of an object relative to the vehicle via image processing of at least two frames of captured image data during driving of the vehicle by the driver of the vehicle. The control compares the determined relative movement of the object to the determined vehicle motion vector, and responsive to the comparison, the control may determine a misalignment of the camera.

Claims (53)

1. A vision system of a vehicle comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a two dimensional array of a plurality of photosensing elements;

wherein said camera is operable to capture frames of image data;

a control, said control comprising an image processor operable to process image data captured by said camera;

wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the field of view of said camera;

wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;

wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;

wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of captured image data during driving of the vehicle by the driver of the vehicle;

wherein said control compares the determined relative movement of the object to the determined vehicle motion vector; and

wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said camera.

2. The vision system of claim 1 , wherein the vehicle data is communicated to the control via a communication bus of the vehicle.

3. The vision system of claim 1 , wherein a kinematic model comprising a bicycle model is used to represent vehicle motion.

4. The vision system of claim 3 , wherein said kinematic model utilizes at least one of (i) a wheel steering angle, (ii) a wheel pulse, (iii) a heading angle and (iv) translational vehicle motion.

5. The vision system of claim 1 , wherein the vehicle data includes (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle.

6. The vision system of claim 1 , wherein, responsive to a determined misalignment of said camera, said vision system is operable to calibrate said camera.

7. The vision system of claim 1 , wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data and a second position of the object captured in a second frame of image data.

8. The vision system of claim 7 , wherein said control determines a misalignment of said camera responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

9. The vision system of claim 1 , wherein said vision system comprises a plurality of cameras disposed at the vehicle, each having a respective field of view exterior of the vehicle.

10. The vision system of claim 9 , wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle.

11. The vision system of claim 10 , wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side exterior rearview mirror assembly of the vehicle and at least having a generally sideward and rearward field of view.

12. The vision system of claim 1 , wherein said camera comprises a CMOS camera.

13. A vision system of a vehicle comprising:

a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;

wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side of the vehicle and at least having a generally sideward and rearward field of view;

wherein said cameras are operable to capture frames of image data;

a control, said control comprising an image processor operable to process image data captured by said cameras;

wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the fields of view of said cameras;

wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;

wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;

wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of image data captured by at least one of said cameras during driving of the vehicle by the driver of the vehicle;

wherein said control compares the determined relative movement of the object to the determined vehicle motion vector; and

wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said at least one of said cameras.

14. The vision system of claim 13 , wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements.

15. The vision system of claim 13 , wherein the vehicle data is communicated to the control via a communication bus of the vehicle.

16. The vision system of claim 13 , wherein, responsive to a determined misalignment of said at least one of said cameras, said vision system is operable to calibrate said at least one of said cameras.

17. The vision system of claim 13 , wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data by said at least one of said cameras and a second position of the object captured in a second frame of image data by said at least one of said cameras.

18. The vision system of claim 17 , wherein said control determines a misalignment of said at least one of said cameras responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

19. A vision system of a vehicle comprising:

a plurality of cameras disposed at a vehicle, each having a respective field of view exterior of the vehicle;

wherein said plurality of cameras comprises (i) a rear camera disposed at a rear portion of the vehicle and having a rearward field of view, (ii) a driver side camera disposed at a driver side of the vehicle and at least having a generally sideward and rearward field of view and (iii) a passenger side camera disposed at a passenger side of the vehicle and at least having a generally sideward and rearward field of view;

wherein each of said cameras comprises a two dimensional array of a plurality of photosensing elements;

wherein said cameras are operable to capture frames of image data;

a control, said control comprising an image processor operable to process image data captured by said cameras;

wherein, responsive to image processing of captured image data, said control is operable to determine objects present in the fields of view of said cameras;

wherein, responsive to vehicle data, said control determines a vehicle motion vector during driving of the vehicle by a driver of the vehicle;

wherein the vehicle data includes at least two of (i) steering information of the vehicle, (ii) speed of the vehicle and (iii) distance traveled by the vehicle;

wherein the vehicle data is communicated to the control via a communication bus of the vehicle;

wherein said control determines movement of an object relative to the vehicle via image processing of at least two frames of image data captured by at least one of said cameras during driving of the vehicle by the driver of the vehicle;

wherein said control compares the determined relative movement of the object to the determined vehicle motion vector;

wherein, responsive to a difference between the determined relative movement of the object and the determined vehicle motion vector, said control determines a misalignment of said at least one of said cameras; and

wherein, responsive to a determined misalignment of said at least one of said cameras, said vision system is operable to calibrate said at least one of said cameras.

20. The vision system of claim 19 , wherein said control compares the determined vehicle motion vector to an object vector determined between a first position of the object captured in a first frame of image data by said at least one of said cameras and a second position of the object captured in a second frame of image data by said at least one of said cameras, and wherein said control determines a misalignment of said at least one of said cameras responsive to a difference between a direction or magnitude of the determined vehicle motion vector and a direction or magnitude of the determined object vector.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 25, 2014
From: TURK, JAMES
To: MAGNA ELECTRONICS INC.
Reel/Frame 033175/0398 →
Continuity (2)
Provisional Application 61825753 · May 21, 2013
Related Publication 20140350834A1 · Nov 27, 2014