Image registration system
View Patent ↗Images may be registered using temporal (time-based) and spatial information. In a film implementation, because film is a sequence of frames, using information from neighboring frames may enable a temporally smoother visual experience. In addition, it may be beneficial to take advantage of the fact that consecutive frames are often shifted similarly during the photographic process. Distortion measures may be used that discount candidate transformations that are considered to be too far from one or more preferred transformations, such as, for example, an optimal transformation from another frame or block or a currently-optimal transformation from the same frame/block. Composite color images may be processed to provide registration of underlying components.
1. A computer-implemented method of registering frames comprising:
accessing a first frame that includes at least two component images;
determining a first transformation for registering the at least two component images included in the first frame;
accessing a second frame that includes at least two component images and that is adjacent to the first frame in a series of frames; and
automatically determining, using at least one processor, a second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame.
2. The computer-implemented method of claim 1 wherein accessing the second frame that includes at least two component images and that is adjacent to the first frame in the series of frames comprises accessing a second frame that immediately precedes the first frame in the series of frames and that includes component images representing a first point in time that is prior to a second point in time represented by component images included in the first frame.
3. The computer-implemented method of claim 1 wherein accessing the second frame that includes at least two component images and that is adjacent to the first frame in the series of frames comprises accessing a second frame that immediately follows the first frame in the series of frames and that includes component images representing a first point in time that is after a second point in time represented by component images included in the first frame.
4. The computer-implemented method of claim 1 further comprising:
accessing a third frame that includes at least two component images and that immediately follows the second frame in the series of frames; and
determining a third transformation for registering the at least two component images included in the third frame,
wherein accessing the first frame comprises accessing a first frame that immediately precedes the second frame in the series of frames, and
wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second transformation for registering the at least two component images included in the second frame that accounts for (1) the first transformation determined for registering the at least two component images included in the first frame and (2) the third transformation determined for registering the at least two component images included in the third frame.
5. The computer-implemented method of claim 1 wherein:
determining the first transformation for registering the at least two component images included in the first frame comprises determining a first alignment vector for registering the at least two component images included in the first frame; and
automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second alignment vector for registering the at least two component images included in the second frame as a function of at least the first alignment vector and image data of the at least two component images included in the second frame.
6. The computer-implemented method of claim 1 wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises influencing an extent to which the first transformation is used in the determination of the second transformation based on a similarity between the first and second frames.
7. The computer-implemented method of claim 1 wherein:
accessing the first frame that includes at least two component images comprises accessing a first frame that includes red, green, and blue color component images;
determining the first transformation for registering the at least two component images included in the first frame comprises determining a first transformation for registering the red, green, and blue color component images included in the first frame;
accessing the second frame that includes at least two component images and that is adjacent to the first frame in the series of frames comprises accessing a second frame that includes red, green, and blue color component images; and
automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second transformation for registering the red, green, and blue color component images included in the second frame that accounts for the first transformation determined for registering the red, green, and blue color component images included in the first frame.
8. The computer-implemented method of claim 1 wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises controlling impact of the first transformation on the second transformation based on a distance in time between a first time corresponding to the first frame and a second time corresponding to the second frame.
9. The computer-implemented method of claim 8 wherein controlling impact of the first transformation on the second transformation based on the distance in time between the first time corresponding to the first frame and the second time corresponding to the second frame comprises diminishing impact of the first transformation on the second transformation as the distance in time increases.
10. The computer-implemented method of claim 1 wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically using the first transformation as the second transformation.
11. The computer-implemented method of claim 1 wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises:
determining, without considering the first transformation, an initial alignment vector for registering the at least two component images included in the second frame based on image data of the at least two component images included in the second frame; and
adjusting the initial alignment vector based on the first transformation.
12. The computer-implemented method of claim 1 wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises:
determining whether to bias toward temporal stability or spatial registration based on content included in the at least two component images included in the second frame;
based on the determination of whether to bias toward temporal stability or spatial registration, determining a bias factor; and
applying the bias factor in determining a second transformation that accounts for the first transformation.
13. An apparatus for automatically determining an alignment vector for a region within an image, the apparatus comprising at least one processor programmed to perform operations comprising:
accessing a first frame that includes at least two component images;
determining a first transformation for registering the at least two component images included in the first frame;
accessing a second frame that includes at least two component images and that is adjacent to the first frame in a series of frames; and
automatically determining a second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame.
14. The apparatus of claim 13 wherein the operations further comprise:
accessing a third frame that includes at least two component images and that immediately follows the second frame in the series of frames; and
determining a third transformation for registering the at least two component images included in the third frame,
wherein accessing the first frame comprises accessing a first frame that immediately precedes the second frame in the series of frames, and
wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second transformation for registering the at least two component images included in the second frame that accounts for (1) the first transformation determined for registering the at least two component images included in the first frame and (2) the third transformation determined for registering the at least two component images included in the third frame.
15. The apparatus of claim 13 wherein:
determining the first transformation for registering the at least two component images included in the first frame comprises determining a first alignment vector for registering the at least two component images included in the first frame; and
automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second alignment vector for registering the at least two component images included in the second frame as a function of at least the first alignment vector and image data of the at least two component images included in the second frame.
16. The apparatus of claim 13 wherein:
accessing the first frame that includes at least two component images comprises accessing a first frame that includes red, green, and blue color component images;
determining the first transformation for registering the at least two component images included in the first frame comprises determining a first transformation for registering the red, green, and blue color component images included in the first frame;
accessing the second frame that includes at least two component images and that is adjacent to the first frame in the series of frames comprises accessing a second frame that includes red, green, and blue color component images; and
automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second transformation for registering the red, green, and blue color component images included in the second frame that accounts for the first transformation determined for registering the red, green, and blue color component images included in the first frame.
17. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:
accessing a first frame that includes at least two component images;
determining a first transformation for registering the at least two component images included in the first frame;
accessing a second frame that includes at least two component images and that is adjacent to the first frame in a series of frames; and
automatically determining a second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame.
18. The computer-readable medium of claim 17 wherein the operations further comprise:
accessing a third frame that includes at least two component images and that immediately follows the second frame in the series of frames; and
determining a third transformation for registering the at least two component images included in the third frame,
wherein accessing the first frame comprises accessing a first frame that immediately precedes the second frame in the series of frames, and
wherein automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second transformation for registering the at least two component images included in the second frame that accounts for (1) the first transformation determined for registering the at least two component images included in the first frame and (2) the third transformation determined for registering the at least two component images included in the third frame.
19. The computer-readable medium of claim 17 wherein:
determining the first transformation for registering the at least two component images included in the first frame comprises determining a first alignment vector for registering the at least two component images included in the first frame; and
automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second alignment vector for registering the at least two component images included in the second frame as a function of at least the first alignment vector and image data of the at least two component images included in the second frame.
20. The computer-readable medium of claim 17 wherein:
accessing the first frame that includes at least two component images comprises accessing a first frame that includes red, green, and blue color component images;
determining the first transformation for registering the at least two component images included in the first frame comprises determining a first transformation for registering the red, green, and blue color component images included in the first frame;
accessing the second frame that includes at least two component images and that is adjacent to the first frame in the series of frames comprises accessing a second frame that includes red, green, and blue color component images; and
automatically determining, using at least one processor, the second transformation for registering the at least two component images included in the second frame that accounts for the first transformation determined for registering the at least two component images included in the first frame comprises automatically determining, using at least one processor, a second transformation for registering the red, green, and blue color component images included in the second frame that accounts for the first transformation determined for registering the red, green, and blue color component images included in the first frame.