eMirror with 3-in-1 stitching by non-rectilinear warping of camera views
An apparatus includes an interface and a processor. The interface may be configured to receive video frames corresponding to an exterior view of a vehicle generated by a plurality of capture devices. The processor may be configured to perform digital warping on the video frames, generate distorted video frames in response to the digital warping, perform video stitching operations on the distorted video frames and generate panoramic video frames in response to the video stitching operations. The digital warping may be performed to adjust the video frames based on lens characteristics of the capture devices. An amount of the digital warping applied may be selected to provide a size continuity of objects in the distorted video frames at a stitching seam in the panoramic video frames. The panoramic video frames may be generated to fit a size of a display.
1. An apparatus comprising:
an interface configured to receive video frames corresponding to an exterior view of a vehicle generated by a plurality of capture devices; and
a processor configured to (a) perform digital warping on said video frames, (b) generate distorted video frames in response to said digital warping, (c) perform video stitching operations on said distorted video frames and (d) generate panoramic video frames in response to said video stitching operations, wherein (i) said digital warping is performed to adjust said video frames based on lens characteristics of said capture devices, (ii) an amount of said digital warping applied is selected to provide a size continuity of objects in said distorted video frames at a stitching seam in said panoramic video frames and (iii) said panoramic video frames are generated to fit a size of a display.
2. The apparatus according to claim 1 , wherein said distorted video frames are non-rectilinear video frames.
3. The apparatus according to claim 1 , wherein said digital warping is configured to provide a fisheye distortion.
4. The apparatus according to claim 1 , wherein a size of objects in said distorted video frames are larger at a center of said distorted video frames and said size of said objects becomes smaller to the left and to the right of said center.
5. The apparatus according to claim 1 , wherein an amount of said digital warping applied is determined based on a distance from said vehicle captured at said stitching seam.
6. The apparatus according to claim 1 , wherein said size continuity is provided by applying said digital warping to reduce a size of objects in a first of said distorted video frames at a first edge to match said size of said objects in a second of said distorted video frames at a second edge.
7. The apparatus according to claim 1 , wherein said panoramic video frames comprise three video frames stitched together.
8. The apparatus according to claim 7 , wherein a first of said three video frames comprises said exterior view captured by one of said capture devices at a driver side of said vehicle, a second of said three video frames comprises said exterior view captured by one of said capture devices at a rear of said vehicle and a third of said three video frames comprises said exterior view captured by one of said capture devices at a passenger side of said vehicle.
9. The apparatus according to claim 8 , wherein said second of said three video frames has a size similar to said first of said three video frames and said third of said three video frames in said panoramic video frames.
10. The apparatus according to claim 7 , wherein said display receiving said panoramic video frames is a 3-in-1 eMirror.
11. The apparatus according to claim 1 , wherein a portion of said panoramic video frames comprise an invalid area that does not output video data captured by said capture devices.
12. The apparatus according to claim 11 , wherein said processor generates said invalid area to mask stitching artifacts.
13. The apparatus according to claim 11 , wherein said invalid area is a small portion of said panoramic video frames.
14. A method for aligning a stitching seam of video frames, comprising the steps of:
(A) selecting first characteristics of a first lens that implements a first distortion;
(B) selecting second characteristics of a second lens that implements a second distortion;
(C) capturing a first video frame using a first capture device from a first location using said first lens;
(D) capturing a second video frame using a second capture device from a second location using said second lens;
(E) performing video stitching operations using a processor to generate a panoramic video frame comprising said first video frame and said second video frame, wherein said first video frame and said second video frame are joined in said panoramic video frame at said stitching seam; and
(F) presenting said panoramic video frame to a display device, wherein (i) said processor generates said panoramic video to fit a size of said display device, (ii) said first characteristics and said second characteristics are selected to provide a size continuity for objects at said stitching seam for a range of distances.
15. The method according to claim 14 , wherein (i) a location of said stitching seam is selected based on a distance of concern and (ii) said distance of concern is determined based on where other vehicles attempt to pass.
16. The method according to claim 15 , wherein said range of distances is determined with respect to said distance of concern.
17. The method according to claim 14 , wherein said first distortion and said second distortion cause non-rectilinear perspectives in said first video frame and said second video frame.
18. The method according to claim 14 , wherein said first distortion and said second distortion are a barrel distortion.
19. The method according to claim 14 , further comprising a step of:
performing digital warping to said first video frame and said second video frame.