Multi-exposure video
View Patent ↗Certain cameras and systems described herein produce enhanced dynamic range still or video images. The images can also have controlled or reduced motion artifacts. Moreover, the cameras and systems in some cases allow the dynamic range and/or motion artifacts to be tuned to achieve a desired cinematic effect.
1. A method of recording motion video image data using a digital motion video camera, comprising:
for a plurality of successive frames of motion of video image data,
controlling a digital image sensor of a digital motion video camera to begin capturing a first image according to a rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for first duration and read out in successive, overlapping time slots; and
subsequent to a starting group of the groups of pixels completing exposure for the first image, and prior to an ending group of the groups of pixels completing exposure for the first image, controlling the digital image sensor to begin capturing a second image according to the rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for a second duration and read out in successive, overlapping time slots, the second duration being different than the first duration,
wherein the first duration and the second duration remain unchanged over the successive frames of motion video image data,
wherein said controlling the digital image sensor to begin capturing the second image includes allowing each respective group of pixels to complete capturing the light corresponding to the first image prior to controlling that respective group of pixels to begin capturing the light corresponding to the second image, and
wherein the amount of time that elapses from between completion of readout of digital measurements corresponding to the starting group for the first image to the beginning of readout of digital measurements corresponding to the starting group for second image is greater than the amount of time that elapses from between completing readout of the digital measurements corresponding to the starting group for the first image to substantially completing readout of digital measurements for the entire first image.
2. A method according to claim 1 wherein the plurality of successive frames include a first frame and a second frame, wherein the second frame is next in the succession of the plurality of frames following the first frame, and wherein the amount of time that elapses from between completion of readout of digital measurements corresponding to the starting group for the second image of the first frame to the beginning of readout of digital measurements corresponding to the starting group for first image of the second frame is greater than the amount of time that elapses from between completing readout of the digital measurements corresponding to the starting group for the second image of the first frame to substantially completing readout of digital measurements for the entire second image of the first frame.
3. A method according to claim 1 wherein the successive groups of pixels each correspond to one horizontal row of pixels.
4. A method according to claim 1 wherein the successive groups of pixels each correspond to two horizontal rows of pixels, the two horizontal rows being vertically adjacent to one another.
5. A method according to claim 1 wherein the starting group of pixels includes a top-most row of pixels and the ending group of pixels includes a bottom-most row of pixels.
6. A method according to claim 1 further comprising storing the first and second images for a plurality of the successive video frames.
7. A digital motion video camera comprising:
an array of pixels; and
a controller configured to, for a plurality of successive frames of motion video image data:
control the array of pixels to begin to capture a first image according to a rolling shutter technique in which successive groups of the pixels in the array from a starting group of pixels to an ending group of pixels are each exposed for a first duration and read out in successive, overlapping time slots; and
subsequent to the starting group completing exposure for the first image, and prior to the ending group completing exposure for the first image, control the array of pixels to begin capturing a second image according to the rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for a second duration and read out in successive, overlapping time slots, the second duration being different than the first duration,
wherein the first duration and the second duration remain unchanged over the successive frames of motion video image data,
wherein the controller is configured to allow each respective group of pixels to complete capturing the light corresponding to the first image prior to controlling that respective group of pixels to begin capturing the light corresponding to the second image, and
wherein the amount of time that elapses from between completion of readout of digital measurements corresponding to the starting group for the first image to the beginning of readout of digital measurements corresponding to the starting group for second image is greater than the amount of time that elapses from between completing readout of the digital measurements corresponding to the starting group for the first image to substantially completing readout of digital measurements for the entire first image.
8. An imaging device according to claim 7 wherein the plurality of successive frames include a first frame and a second frame, wherein the second frame is next in the succession of the plurality of frames following the first frame, and wherein the amount of time that elapses from between completion of readout of digital measurements corresponding to the starting group for the second image of the first frame to the beginning of readout of digital measurements corresponding to the starting group for first image of the second frame is greater than the amount of time that elapses from between completion of readout of the digital measurements corresponding to the starting group for the second image of the first frame to substantial completion of readout of digital measurements for the entire second image of the first frame.
9. An imaging device according to claim 7 wherein the successive groups of pixels each correspond to one horizontal row of pixels.
10. An imaging device according to claim 7 wherein the successive groups of pixels each correspond to two horizontal rows of pixels, the two horizontal rows being vertically adjacent to one another.
11. A imaging device according to claim 7 wherein the starting group of pixels includes a top-most row of pixels and the ending group of pixels includes a bottom-most row of pixels.
12. A imaging device according to claim 7 further comprising a memory device, the controller further configured to store the first and second images for a plurality of the successive video frames in the memory device.
13. A method of blending motion video image data comprising:
for a plurality of successive frames of motion video image data:
obtaining first image data derived from digital measurements corresponding to a first image captured by a digital image sensor of a motion video camera according to a rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for a first duration and read out in successive, overlapping time slots;
obtaining second image data derived from digital measurements corresponding to a second image captured by the digital image sensor of the motion video camera, wherein the digital image sensor captured the second image according to the rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for a second duration and read out in successive, overlapping time slots, the second duration being different than the first duration,
wherein the first duration and the second duration remain unchanged over the successive frames of motion video image data,
wherein the digital image sensor began capturing the second image subsequent to a starting group of the groups of pixels completing exposure for the first image, and prior to an ending group of the groups of pixels completing exposure for the first image; and
selectively combining the first image data and the second image data to create combined image data corresponding to a combined image,
wherein each respective group of pixels to completed capturing the light corresponding to the first image prior to when that respective group of pixels began capturing the light corresponding to the second image, and
wherein the amount of time that elapsed from between completion of readout of digital measurements corresponding to the starting group for the first image to the beginning of readout of digital measurements corresponding to the starting group for second image is greater than the amount of time that elapsed from between completing readout of the digital measurements corresponding to the starting group for the first image to substantially completing readout of digital measurements for the entire first image.
14. A method according to claim 13 , further comprising:
based on input entered by a user into a graphical user interface, determining at least a first threshold intensity value;
setting one of the first image data and the second image data as reference image data, and
wherein said selectively combining comprises, for each respective pixel of a plurality of pixels in the combined image data:
identifying a reference intensity value of a pixel in the reference image data, wherein the pixel in the reference image data corresponds to the respective pixel;
if the reference intensity value is below the first threshold intensity value, deriving the intensity value of the respective pixel from a first intensity value of a pixel in the first image data which corresponds to the respective pixel; and
if the reference intensity value is above the first threshold intensity value, deriving the intensity value of the respective pixel using either (i) a second intensity value of a pixel in the second image data, where the pixel in the second image data corresponds to the respective pixel, or (ii) a combination of the first intensity value and the second intensity value.
15. A method according to claim 14 , wherein the at least a first threshold pixel intensity value comprises the first threshold intensity value and a second threshold intensity value, wherein said selectively combining comprises, for each respective pixel of the plurality of pixels in the combined image data:
if the reference intensity value is between the first threshold intensity value and the second intensity value, deriving the intensity value of the respective pixel using a combination of the first intensity value and the second intensity value; and
if the reference intensity value is above the second threshold intensity value, deriving the intensity value of the respective pixel using the second intensity value.
16. A method according to claim 13 , further comprising:
based on input entered by a user into a graphical user interface, determining a relative weighting of the first image data and the second image data; and
wherein said selectively combining comprises, for each respective pixel of a plurality of pixels in the combined image data:
identifying a first pixel intensity value corresponding to a pixel in the first image data, where the pixel in the first image data corresponds to the respective pixel;
identifying a second pixel intensity value corresponding to a pixel in the second image data, where the pixel in the second image data corresponds to the respective pixel; and
combining the first pixel intensity value and the second pixel intensity value based on the relative weighting to calculate an intensity value of the respective pixel.
17. A system configured to blend motion video image data comprising:
one or more hardware processors configured to, for a plurality of successive frames of motion video image data:
obtain first image data derived from digital measurements corresponding to a first image captured by a digital image sensor of the motion video camera according to a rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for a first duration and read out in successive, overlapping time slots;
obtain second image data derived from digital measurements corresponding to a second image captured by the digital image sensor, wherein the digital image sensor captured the second image according to the rolling shutter technique in which successive groups of pixels of the image sensor are each exposed for a second duration and read out in successive, overlapping time slots, the second duration being different than the first duration,
wherein the first duration and the second duration remain unchanged over the successive frames of motion video image data,
wherein the digital image sensor began capturing the second image subsequent to a starting group of the groups of pixels completing exposure for the first image, and prior to an ending group of the groups of pixels completing exposure for the first image; and
selectively combine the first image data and the second image data to create combined image data corresponding to a combined image,
wherein each respective group of pixels to completed capturing the light corresponding to the first image prior to when that respective group of pixels began capturing the light corresponding to the second image, and
wherein the amount of time that elapsed from between completion of readout of digital measurements corresponding to the starting group for the first image to the beginning of readout of digital measurements corresponding to the starting group for second image is greater than the amount of time that elapsed from between completing readout of the digital measurements corresponding to the starting group for the first image to substantially completing readout of digital measurements for the entire first image.
18. A system according to claim 17 , wherein the one or more hardware processors are further configured to:
based on input entered by a user into a graphical user interface, determine at least a first threshold intensity value;
set one of the first image and the second image as a reference image, and
for each respective pixel of a plurality of pixels in the combined image data:
identify a reference intensity value of a pixel in the reference image data which corresponds to the respective pixel;
if the reference intensity value is below the first threshold intensity value, derive the intensity value of the respective pixel from a first intensity value of a pixel in the first image data, where the pixel in the first image data corresponds to the respective pixel; and
if the reference intensity value is above the first threshold intensity value, derive the intensity value of the respective pixel using either (i) a second intensity value of a pixel in the second image data, where the pixel in the second image data corresponds to the respective pixel, or (ii) a combination of the first intensity value and the second intensity value.
19. A system according to claim 18 , wherein the at least a first threshold pixel intensity value comprises the first threshold intensity value and a second threshold intensity value, and wherein the one or more hardware processors are further configured to, for each respective pixel of the plurality of pixels in the combined image data:
if the reference intensity value is between the first threshold intensity value and the second intensity value, derive the intensity value of the respective pixel using a combination of the first intensity value and the second intensity value; and
if the reference intensity value is above the second threshold intensity value, derive the intensity value of the respective pixel using the second intensity value.
20. A system according to claim 17 , wherein the one or more hardware processors are further configured to:
based on input entered by a user into a graphical user interface, determine a relative weighting of the first image data and the second image data; and
for each respective pixel of a plurality of pixels in the combined image data:
identify a first pixel intensity value corresponding to a pixel in the first image data, where the pixel in the first image data corresponds to the respective pixel;
identify a second pixel intensity value corresponding to a pixel in the second image data, where the pixel in the second image data corresponds to the respective pixel; and
combine the first pixel intensity value and the second pixel intensity value based on the relative weighting to calculate an intensity value of the respective pixel.