Stitching for a multiple imager video system based on a non-linear seam
View Patent ↗In one embodiment, there is provided a method for stitching left images P from a first imager with right images Q from a second imager thereby to produce stitched video images, the method comprising: for each current frame N: calculating a spatial distortion Spatial(i,j) for each pixel in a region of overlap between an image PN corresponding to the frame N and an image Q N corresponding to the frame N; calculating a temporal distortion Temporal(i,j) for each pixel in said region of overlap; determining a minimal cost path for traversing the region of overlap from top to bottom, wherein said minimal cost path comprises a pixel from each row in said region of overlap; and setting said minimal cost path as a seam for stitching images P N and Q N together to form a composite image C N .
1. A method for stitching left images P from a first imager with right images Q from a second imager thereby to produce stitched video images, the method comprising:
for each current frame N:
calculating a spatial distortion Spatial(i,j) for each pixel in a region of overlap between an image P N corresponding to the frame N and an image Q N corresponding to the frame N, wherein the spatial distortion is calculated based on luminance and chrominance value for each pixel in the region of overlap;
calculating a temporal distortion Temporal(i,j) for each pixel in said region of overlap by sub-sampling the frame and calculating a median pixel value over the region of overlap and then checking a number of times it deviates above a threshold level;
determining a minimal cost path for traversing the region of overlap from top to bottom, wherein said minimal cost path comprises a pixel from each row in said region of overlap; and
setting said minimal cost path as a seam for stitching images P N and Q N together to form a composite image C N .
2. The method of claim 1 , wherein the minimal cost path is non-linear.
3. The method of claim 2 , further comprising calculating a composite distortion Composite(i,j) for each pixel (i,j) in a region of overlap between a composite image C N-1 corresponding to a prior frame N−1 and one of images P and Q corresponding to the frame N based on a location of a seam used for stitching images P and Q of said frame N for each row j.
4. The method of claim 3 , wherein the composite distortion is calculated relative to pixels corresponding to image P the prior frame N−1 for each pixel (i,j) where i is greater than the location of the seam.
5. The method of claim 3 , wherein the composite distortion is calculated relative to pixels corresponding to image Q of the prior frame N−1 for each pixel (i,j) where i is less than the location of the seam.
6. The method of claim 3 , wherein determining the minimal cost path is based on minimizing said spatial, temporal, and composite distortions calculated for pixels in the region of overlap.
7. The method of claim 3 , wherein determining the minimal cost path is based on a weighted sum of the spatial, temporal, and composite distortions.
8. A system, comprising: a processor; and
a memory coupled to the processor, the memory storing instructions which when executed by the processor causes the system to perform a method for stitching left images P from a first imager with right images Q from a second imager thereby to produce stitched video images, the method comprising:
for each current frame N:
calculating a spatial distortion Spatial(i,j) for each pixel in a region of overlap between an image P N corresponding to the frame N and an image Q N corresponding to the frame N, wherein the spatial distortion is calculated based on luminance and chrominance value for each pixel in the region of overlap;
calculating a temporal distortion Temporal(i,j) for each pixel in said region of overlap by sub-sampling the frame and calculating a median pixel value over the region of overlap and then checking a number of times it deviates above a threshold level;
determining a minimal cost path for traversing the region of overlap from top to bottom, wherein said minimal cost path comprises a pixel from each row in said region of overlap; and
setting said minimal cost path as a seam for stitching images P N and Q N together to form a composite image C N .
9. The system of claim 8 , wherein the minimal cost path is non-linear.
10. The system of claim 9 , wherein the method further comprises calculating a composite distortion Composite(i,j) for each pixel (i,j) in a region of overlap between a composite image C N-1 corresponding to a prior frame N−1 and one of images P and Q corresponding to the frame N based on a location of a seam used for stitching images P and Q of said frame N for each row j.
11. The system of claim 10 , wherein the composite distortion is calculated relative to pixels corresponding to image P of the prior frame N−1 for each pixel (i,j) where i is greater than the location of the seam.
12. The system of claim 10 , wherein the composite distortion is calculated relative to pixels corresponding to image Q of the prior frame N−1 for each pixel (i,j) where i is less than the location of the seam.
13. The system of claim 10 , wherein determining the minimal cost path is based on minimizing said spatial, temporal, and composite distortions calculated for pixels in the region of overlap.
14. The system of claim 10 , wherein determining the minimal cost path is based on a weighted sum of the spatial, temporal, and composite distortions.
15. A non-transitory computer-readable medium having stored thereon a sequence of instructions which when executed by a system causes the system to perform a method for stitching left images P from a first imager with right images Q from a second imager thereby to produce stitched video images, the method comprising:
for each current frame N:
calculating a spatial distortion Spatial(i,j) for each pixel in a region of overlap between an image P N corresponding to the frame N and an image Q N corresponding to the frame N, wherein the spatial distortion is calculated based on luminance and chrominance value for each pixel in the region of overlap;
calculating a temporal distortion Temporal(i,j) for each pixel in said region of overlap by sub-sampling the frame and calculating a median pixel value over the region of overlap and then checking a number of times it deviates above a threshold level;
determining a minimal cost path for traversing the region of overlap from top to bottom, wherein said minimal cost path comprises a pixel from each row in said region of overlap; and
setting said minimal cost path as a seam for stitching images P N and Q N together to form a composite image C N .
16. The computer-readable medium of claim 15 , wherein the minimal cost path is non-linear.
17. The computer-readable medium of claim 16 , wherein the method further comprises calculating a composite distortion Composite(i,j) for each pixel (i,j) in a region of overlap between a composite image C N-1 corresponding to a prior frame N−1 and one of images P and Q corresponding to the frame N based on a location of a seam used for stitching images P and Q of said frame N for each row j.
18. The computer-readable medium of claim 17 , wherein the composite distortion is calculated relative to pixels corresponding to image P of the prior frame N−1 for each pixel (i,j) where i is greater than the location of the seam.
19. The computer-readable medium of claim 17 , wherein the composite distortion is calculated relative to pixels corresponding to image Q of the prior frame N−1 for each pixel (i,j) where i is less than the location of the seam.
20. The computer-readable medium of claim 17 , wherein determining the minimal cost path is based on minimizing said spatial, temporal, and composite distortions calculated for pixels in the region of overlap.