IP Library Granted Patent US 8,908,041
Granted Patent B2
US 8,908,041 · App. 14/156,096 · Granted Dec 9, 2014

Stereo assist with rolling shutters

Inventors: Gideon Stein (Jerusalem, IL); Efim Belman (Jerusalem, IL)
Assignee: Mobileye Vision Technologies Ltd.
H04N13/0282H04N5/232H04N13/0296G06K9/00791
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,908,041
App. No.
14/156,096
Granted
Dec 9, 2014
Kind
B2
Abstract

An imaging system for a vehicle may include a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter. The imaging system may also include a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter. As a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image corresponds with a second overlap portion of the second image. The first image capture device has a first scan rate associated with acquisition of the first series of image scan lines that is different from a second scan rate associated with acquisition of the second series of image scan lines, such that the first image capture device acquires the first overlap portion of the first image over a period of time during which the second overlap portion of the second image is acquired.

Claims (71)

1. An imaging system for a vehicle, the system comprising:

a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;

a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;

wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image corresponds with a second overlap portion of the second image; and

wherein the first image capture device has a first scan rate associated with acquisition of the first series of image scan lines that is different from a second scan rate associated with acquisition of the second series of image scan lines, such that the first image capture device acquires the first overlap portion of the first image over a period of time during which the second overlap portion of the second image is acquired, wherein the period of time is associated with a ratio between the first scan rate and the second scan rate.

2. The system of claim 1 , wherein the second scan rate is greater than the first scan rate.

3. The system of claim 1 , further including an image processing device configured to sample only a subset of scan lines from the second image capture device within the second overlap portion of the second image.

4. The system of claim 1 , wherein:

the first image capture device has a first horizontal blanking period associated with acquisition of the first series of image scan lines;

the second image capture device has a second horizontal blanking period associated with acquisition of the second series of image scan lines;

the second horizontal blanking period is less than the first horizontal blanking period.

5. The system of claim 1 , wherein:

the first image capture device has a first vertical blanking period associated with acquisition of the first series of image scan lines;

the second image capture device has a second vertical blanking period associated with acquisition of the second series of image scan lines;

the first vertical blanking period is less than the second vertical blanking period.

6. The system of claim 1 , wherein:

the first image capture device has a first total frame time associated with acquisition of the first series of image scan lines;

the second image capture device has a second total frame time associated with acquisition of the second series of image scan lines; and wherein

the first total frame time is equal to the second total frame time.

7. The system of claim 1 , wherein the second field of view is narrower than the first field of view.

8. The system of claim 1 , wherein at least a center of the second field of view substantially corresponds to at least a center the first field of view.

9. The system of claim 1 , wherein the ratio of the first field of view to the second field of view ranges between 1.5 to 2.0.

10. The system of claim 1 , wherein the ratio of the first field of view to the second field of view ranges between 1.25 to 2.25.

11. The system of claim 1 , wherein the system further comprises:

at least one processing device configured to:

perform a periodic check to determine whether the first image capture device and the second image capture device are synchronized.

12. The system of claim 11 , wherein the at least one processing device is further configured to perform resynchronization of the first image capture device and the second image capture device if the periodic check indicates that the first image capture device and the second image capture device have fallen out of synch.

13. The system of claim 1 , wherein:

the first image capture device has a first pixel delay associated with acquisition of the first series of image scan lines;

the second image capture device has a second pixel delay associated with acquisition of the second series of image scan lines;

the first pixel delay is more than the second pixel delay.

14. The system of claim 1 , wherein the second overlap portion of the second image corresponds to an entire area of overlap between the first field of view and the second field of view.

15. The system of claim 1 , wherein at least a center of the second field of view falls within a lower half of the first field of view.

16. A vehicle, comprising:

a body;

an imaging system, the system comprising:

a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;

a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;

wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image corresponds with a second overlap portion of the second image; and

wherein the first image capture device has a first scan rate associated with acquisition of the first series of image scan lines that is different from a second scan rate associated with acquisition of the second series of image scan lines, such that the first image capture device acquires the first overlap portion of the first image over a period of time during which the second overlap portion of the second image is acquired, wherein the period of time is associated with a ratio between the first scan rate and the second scan rate.

17. An imaging system for a vehicle, the system comprising:

a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;

a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;

wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image will correspond to a second overlap portion of the second image;

at least one processing device configured to:

receive the first image from the first image capture device;

receive the second image from the second image capture device;

correlate at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image, wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image includes:

selection of a first scan line associated with acquisition of the first image to be synchronized with a second scan line associated with acquisition of the second image; and

synchronizing of the first scan line with the second scan line;

the at least one processing device further configured to:

select a third scan line associated with acquisition of a third image acquired by the first image capture device to be synchronized with a fourth scan line associated with acquisition of a fourth image acquired by the second image capture device, in place of synchronizing the first scan line with the second scan line, wherein the third scan line is different from the first scan line and the third scan line is different from the fourth scan line; and

synchronize the third scan line with the fourth scan line.

18. The system of claim 17 , wherein the second field of view fully overlaps with the first field of view.

19. The system of claim 17 , wherein the second field of view is narrower than the first field of view.

20. The system of claim 17 , wherein the first image capture device and the second image capture device have differing focal lengths.

21. The system of claim 17 , wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image further includes:

performance of a dense depth map computation in a horizontal strip within the first overlap portion of the first image and within a corresponding region of the second overlap portion of the second image.

22. The system of claim 17 , wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image further includes:

computation of dense optical flow between the first overlap portion of the first image and the second overlap portion of the second image; and

use of the dense optical flow to warp at least one of the first or second images.

23. An imaging system for a vehicle, the system comprising:

a first image capture device having a first field of view and configured to acquire a first image relative to a scene associated with the vehicle, the first image being acquired as a first series of image scan lines captured using a rolling shutter;

a second image capture device having a second field of view different from the first field of view and that at least partially overlaps the first field of view, the second image capture device being configured to acquire a second image relative to the scene associated with the vehicle, the second image being acquired as a second series of image scan lines captured using a rolling shutter;

wherein, as a result of overlap between the first field of view and the second field of view, a first overlap portion of the first image will correspond to a second overlap portion of the second image;

at least one processing device configured to:

receive the first image from the first image capture device;

receive the second image from the second image capture device; and

correlate at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image, wherein correlation of the at least a first area of the first overlap portion of the first image with a corresponding second area of the second overlap portion of the second image includes pre-warping at least one of the first image or the second image according to a speed dependent function, the pre-warping being achieved by:

computing a homography matrix associated with motion of the vehicle; and

computing a homography associated with skew of lines of at least one of the first image or the second image.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2017
From: MOBILEYE TECHNOLOGIES LIMITED
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 044369/0134 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2014
From: MOBILEYE TECHNOLOGIES LIMITED
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 033808/0762 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2014
From: STEIN, GIDEON; BELMAN, EFIM
To: MOBILEYE TECHNOLOGIES LIMITED
Reel/Frame 031977/0814 →
Continuity (3)
Provisional Application 61752515 · Jan 15, 2013
Provisional Application 61761724 · Feb 7, 2013
Related Publication 20140198184A1 · Jul 17, 2014