Image stitching with electronic rolling shutter correction

1. A system comprising:

a first image sensor configured to capture a first image;

a second image sensor configured to capture a second image; and

a processing apparatus that is configured to:

receive the first image from the first image sensor;

receive the second image from the second image sensor;

determine compensated epipolar lines based on electronic rolling shutter data;

determine a parallax correction mapping based on the first image, the second image, and the compensated epipolar lines;

determine a warp mapping based on the parallax correction mapping and an electronic rolling shutter correction mapping, wherein the warp mapping applies the electronic rolling shutter correction mapping to output of the parallax correction mapping; and

apply the warp mapping to image data based on the first image and the second image to obtain a composite image.

2. The system of claim 1 , in which determining the compensated epipolar lines based on electronic rolling shutter data comprises:

determining a far point and a near point for an initial epipolar line;

determining a compensated near point based on the near point and electronic rolling shutter data for the near point; and

determining one of the compensated epipolar lines based on the far point and the compensated near point.

3. The system of claim 1 , in which the electronic rolling shutter correction mapping is determined at a lower resolution than the parallax correction mapping.

4. The system of claim 1 , comprising a camera housing that contains the first image sensor, the second image sensor, and the processing apparatus.

5. The system of claim 1 , comprising an angular rate sensor and in which the processing apparatus is configured to determine the electronic rolling shutter correction mapping by performing operations comprising:

receiving angular rate measurements from the angular rate sensor for a device including the first image sensor and the second image sensor for times during capture of the first image and the second image; and

determining the electronic rolling shutter correction mapping based on the angular rate measurements and times when portions of the first image and the second image were captured using an electronic rolling shutter.

6. The system of claim 5 , in which the processing apparatus is configured to determine the electronic rolling shutter correction mapping by performing operations comprising:

determining rotations for respective portions of the first image and the second image based on the angular rate measurements corresponding to times when the respective portions were captured; and

interpolating the rotations to determine interpolated rotations for smaller image portions of the first image and the second image; and

determining the electronic rolling shutter correction mapping based on the interpolated rotations.

7. The system of claim 1 , in which the warp mapping includes records that associate image portions of the composite image with corresponding image portions of the first image and the second image.

8. The system of claim 1 , comprising:

a first fish-eye lens attached to the first image sensor; and

a second fish-eye lens attached to the second image sensor.

9. The system of claim 1 , in which the warp mapping is further determined based on an output projection mapping, such that the warp mapping applies the output projection mapping to output of the electronic rolling shutter correction mapping.

10. A method comprising:

receiving a first image from a first image sensor;

receiving a second image from a second image sensor;

determining a parallax correction mapping based on the first image and the second image for stitching the first image and the second image;

determining a warp mapping based on the parallax correction mapping and an electronic rolling shutter correction mapping, wherein the warp mapping applies the electronic rolling shutter correction mapping after the parallax correction mapping;

applying the warp mapping to image data based on the first image and the second image to obtain a composite image; and

storing, displaying, or transmitting an output image that is based on the composite image.

11. The method of claim 10 , in which determining the parallax correction mapping comprises:

determining compensated epipolar lines based on electronic rolling shutter data; and

determining the parallax correction mapping based on the compensated epipolar lines.

12. The method of claim 11 , in which determining the compensated epipolar lines based on electronic rolling shutter data comprises:

determining a far point and a near point for an initial epipolar line;

determining a compensated near point based on the near point and electronic rolling shutter data for the near point; and

determining one of the compensated epipolar lines based on the far point and the compensated near point.

13. The method of claim 11 , in which determining the parallax correction mapping comprises:

performing a one-dimensional search for a parallax translation along one of the compensated epipolar lines.

14. The method of claim 10 , in which determining the warp mapping comprises:

determining the warp mapping based on a lens distortion correction mapping for the first image and the second image, such that the warp mapping applies the parallax correction mapping to output of the lens distortion correction mapping.

15. The method of claim 10 , in which the warp mapping is further determined based on an output projection mapping, such that the warp mapping applies the output projection mapping to output of the electronic rolling shutter correction mapping.

16. The method of claim 10 , comprising:

receiving angular rate measurements from an angular rate sensor for a device including the first image sensor and the second image sensor for times during capture of the first image and the second image; and

determining the electronic rolling shutter correction mapping based on the angular rate measurements and times when portions of the first image and the second image were captured using an electronic rolling shutter.

17. A method comprising:

receiving a first image from a first image sensor;

receiving a second image from a second image sensor;

applying parallax correction for stitching the first image and the second image to obtain a composite image, wherein applying parallax correction comprises:

compensating epipolar lines for electronic rolling shutter distortion, and

determining parallax correction based on a one-dimensional search along the compensated epipolar lines; and

applying electronic rolling shutter correction to the composite image to obtain an electronic rolling shutter corrected image.

18. The method of claim 17 , in which compensating the epipolar lines for electronic rolling shutter distortion comprises:

determining a far point and a near point for an initial epipolar line;

determining a compensated near point based on the near point and electronic rolling shutter data for the near point; and

determining one of the compensated epipolar lines based on the far point and the compensated near point.

19. The method of claim 17 , comprising;

receiving angular rate measurements from an angular rate sensor for a device including the first image sensor and the second image sensor for times during capture of the first image and the second image; and

determining the electronic rolling shutter correction mapping based on the angular rate measurements and times when portions of the first image and the second image were captured using an electronic rolling shutter.

20. The method of claim 17 , comprising;

storing, displaying, or transmitting an output image that is based on the electronic rolling shutter corrected image.