Method and adaptively deriving exposure time and frame rate from image motion
In a method for determining the frame rate and exposure time for each frame of a video collection, an image capture system acquires at least two successive frames of a scene, separated in time. The two images are compared to determine if objects in the scene are in motion. If motion is detected, then the speed and displacement of the objects that are moving is determined. If the speed of the fastest moving object creates an unacceptable amount of image displacement, then the frame rate for the next frame is changed to one that produces an acceptable amount of image displacement. Also, if the speed of the fastest moving object creates an unacceptable amount of motion blur, then the exposure time for the next frame is changed to one that produces an acceptable amount of motion blur.
1. A method of adaptively determining the exposure time and frame rate for each frame captured by an image capture system of the type having an image sensor, optics for forming an image of a scene on the image sensor, and an image processor for determining an exposure time for the image sensor, said method comprising the steps of:
a) acquiring two time-separated image frames during an initial exposure time set for each of the image frames, wherein each image frame includes one or more moving objects from the scene;
b) spatially registering the two image frames;
c) measuring spatial displacements between the two image frames for the same moving objects;
d) determining a fastest moving object in the scene from the measured spatial displacements;
e) calculating a speed of the fastest moving object;
f) calculating object smear for the fastest moving object based on the initial exposure time;
g) if the object smear exceeds a maximum desired smear, calculating a new exposure time;
h) setting the exposure time for a subsequent image frame to the new exposure time;
i) if the measured spatial displacement of the fastest moving object exceeds a maximum desired displacement, calculating a new frame rate; and
j) setting the frame rate for a subsequent image frame to the new frame rate.
2. The method as claimed in claim 1 wherein the step b) of spatially registering the two image frames comprises generating a correlation matrix from pixels in the two image frames in order to remove translation differences between the two image frames.
3. The method as claimed in claim 1 wherein the step c) of measuring spatial displacements comprises the step of dividing one of the image frames into blocks and correlating each block with the other frame to detect the displacement of moving objects between the frames.
4. The method as claimed in claim 1 wherein the step h) of setting an exposure time comprises selecting one of a predetermined set of exposure times from a database.
5. The method as claimed in claim 1 wherein a mechanical shutter or an electronic shutter is used to control the exposure time.
6. A computer program product for adaptively determining the exposure time and frame rate for each frame captured by an image capture system of the type that acquires two time-separated image frames during an initial exposure time set for each of the image frames, wherein each image frame includes one or more moving objects from a scene, said computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of:
a) spatially registering the two image frames;
b) measuring spatial displacements between the two image frames for the same moving objects;
c) determining a fastest moving object in the scene from the measured spatial displacements;
d) calculating a speed of the fastest moving object;
e) calculating object smear for the fastest moving object based on the initial exposure time;
f) if the object smear exceeds a maximum desired smear, calculating a new exposure time;
g) if the measured spatial displacement of the fastest moving object exceeds a maximum desired displacement, calculating a new frame rate; and
h) setting the frame rate for a subsequent image frame to the new frame rate.
7. A method of adaptively determining the frame rate for each frame captured by an image capture system of the type having an image sensor, optics for forming an image of a scene on the image sensor, and an image processor for determining a frame rate for the image sensor, said method comprising the steps of:
a) acquiring two image frames separated in time by an initial frame rate, wherein each image frame includes one or more moving objects from the scene;
b) spatially registering the two image frames;
c) measuring spatial displacements between the two image frames for the same moving objects;
d) determining a fastest moving object in the scene from the measured spatial displacements;
e) calculating a speed of the fastest moving object;
f) calculating the object displacement based on the initial frame rate;
g) if the object displacement exceeds a maximum desired displacement, calculating a new frame rate; and
h) setting the frame rate for a subsequent image frame to the new frame rate.
8. The method as claimed in claim 7 wherein the step b) of spatially registering the two image frames comprises generating a correlation matrix from pixels in the two image frames in order to remove translation differences between the two image frames.
9. The method as claimed in claim 7 wherein the step c) of measuring spatial displacements comprises the step of dividing one of the image frames into blocks and correlating each block with the other frame to detect the displacement of moving objects between the frames.
10. The method as claimed in claim 7 wherein the step h) of setting the frame rate comprises selecting one of a predetermined set of frame rates from a database.
11. The method as claimed in claim 7 wherein a mechanical shutter or electronic shutter is used to control the frame rate.
12. A computer program product for adaptively determining the frame rate for each frame captured by an image capture system of the type that acquires two image frames separated in time by an initial frame rate, wherein each image frame includes one or more moving objects from a scene, said computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of:
a) spatially registering the two image frames;
b) measuring spatial displacements between the two image frames for the same moving objects;
c) determining a fastest moving object in the scene from the measured spatial displacements;
d) calculating a speed of the fastest moving object;
e) calculating the object displacement based on the initial frame rate; and
f) if the object displacement exceeds a maximum desired displacement, calculating a new frame rate.
13. A method of adaptively determining the exposure time and frame rate for each frame of an image capture system of the type having an image sensor, optics for forming an image of a scene on the image sensor, and an image processor for determining an exposure time and frame rate for the image sensor, said method comprising the steps of:
a) acquiring two time-separated image frames according to an initial exposure time and an initial frame rate, wherein each image frame includes one or more moving objects from the scene;
b) spatially registering the two image frames;
c) measuring spatial displacements between the two image frames for the same moving objects;
d) determining a fastest moving object in the scene from the measured spatial displacements;
e) calculating a speed of the fastest moving object;
f) calculating object smear for the fastest moving object based on the initial exposure time;
g) if the object smear exceeds a maximum desired smear, calculating a new exposure time;
h) calculating the object displacement based on the initial frame rate;
i) if the object displacement exceeds a maximum desired displacement, calculating a new frame rate; and
j) setting the exposure time and frame rate for the next image frame to the new exposure time and new frame rate.
14. The method as claimed in claim 13 wherein step a) involves the acquisition of a continuous sequence of successive image frames and the steps b) through j) are repeated for each pair of successive image frames.
15. A computer program product for adaptively determining the exposure time and frame rate for each frame captured by an image capture system of the type that acquires two time-separated image frames according to an initial exposure time and an initial frame rate, wherein each image frame includes one or more moving objects from a scene, said computer program comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of:
a) measuring spatial displacements between the two image frames for the same moving objects;
b) calculating a speed of a fastest moving object;
c) calculating object smear for the fastest moving object based on the initial exposure time;
d) if the object smear exceeds a maximum desired smear, calculating a new exposure time;
e) calculating the object displacement based on the initial frame rate; and
f) if the object displacement exceeds a maximum desired displacement, calculating a new frame rate.