Video frame pulldown based on frame analysis
The described technology is directed towards generating a new video image sequence (e.g., for playback at 30 frames per second) based on an existing video image sequence (e.g., originated for playback at 24 frames per second). The technology is based on processing frames, e.g., adjacent pairs of frames in a four-frame sequence, to obtain candidate frames for selecting a similar candidate frame to insert into the original sequence to create the new sequence (e.g., a five-frame sequence). Aspects include selecting a repeated frame to insert or creating a new frame from existing frames to insert, to generate the new sequence based on a difference/scoring comparison.
1. A method, comprising:
choosing, by a system comprising a processor, a frame pair of frame pairs in a sequence of frames to insert an additional frame, wherein the sequence of frames comprises at least three frames, each frame pair of the frame pairs comprises adjacent frames in the sequence of frames, and the choosing comprises:
analyzing the frame pairs of the sequence of frames, and the analyzing comprises, for each frame pair:
determining at least one candidate frame for insertion between the adjacent frames of the frame pair, and
determining respective scores for the at least one candidate frame;
selecting a candidate frame based on the respective scores of the candidate frames of the frame pairs; and
selecting the frame pair associated with the candidate frame; and
inserting, by the system, the selected candidate frame as the additional frame between the adjacent frames of the frame pair to generate a new sequence of frames.
2. The method of claim 1 , wherein the sequence of frames has four frames, and wherein the inserting the selected candidate frame comprises generating the new sequence of frames with five frames.
3. The method of claim 1 , wherein the at least one candidate frame for the frame pair comprises a first frame of the adjacent frames of the frame pair according to a sequential order of the sequence of frames.
4. The method of claim 1 , wherein the at least one candidate frame for the frame pair comprises a created frame generating by combining the adjacent frames of the frame pair.
5. The method of claim 4 , wherein the combining the adjacent frames comprises performing motion analysis.
6. The method of claim 4 , wherein the combining the adjacent frames comprises determining moving pixels and static pixels, and combining partially moved pixels representing the moving pixels with overlaid static pixels to form the created frame.
7. The method of claim 4 , wherein the respective score for the created frame for the frame pair is based on an amount of difference between the created frame and a first frame of the adjacent frames according to a sequential order of the sequence of frames.
8. The method of claim 3 , wherein the respective score for the candidate frame for the frame pair is based on an amount of difference between the candidate frame and a second frame of the adjacent frames of the frame pair according to a sequential order of the sequence of frames.
9. A system, comprising:
a processor, and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
selecting a frame pair of frame pairs in a sequence of frames to insert an additional frame, wherein the sequence of frames comprises at least three frames, each frame pair of the frame pairs comprises adjacent frames in the sequence of frames, and the selecting comprises:
evaluating the frame pairs of adjacent frames in the sequence of frames, wherein the evaluating comprises, for each frame pair:
obtaining at least one candidate frame for insertion between the adjacent frames of the pair of adjacent frames, and
determining respective candidate frame scores for the at least one candidate frame, wherein a candidate frame score for a candidate frame is based on an amount of difference between the candidate frame and a frame of the frame pair; and
selecting the frame pair associated with a selected candidate frame of the candidate frames of the frame pairs having a respective candidate frame score indicative of a least amount of difference; and
inserting the selected candidate frame as the additional frame between the adjacent frames of the frame pair of the sequence of frames to obtain a new sequence of frames.
10. The system of claim 9 , wherein the determining the respective candidate frame scores comprises determining respective peak signal to noise ratios.
11. The system of claim 9 , wherein the sequence of frames has four frames, and wherein the new sequence of frames has five frames.
12. The system of claim 9 , wherein the at least one candidate frame for the frame pair comprises a first frame of the adjacent frames of the frame pair according to a sequential order of the sequence of frames.
13. The system of claim 9 , wherein the at least one candidate frame for the frame pair comprises a second frame of the adjacent frames of the frame pair according to the sequential order.
14. The system of claim 9 , wherein the at least one candidate frame for the frame pair comprises a created frame generating by combining the adjacent frames of the frame pair.
15. The system of claim 14 , wherein the at least one candidate frame for the frame pair further comprises a first frame of the adjacent frames of the frame pair according to a sequential order of the sequence of frames.
16. The system of claim 14 , wherein the combining the adjacent frames comprises determining moving pixels and static pixels, and combining partially moved pixels representing the moving pixels with overlaid static pixels to form the created frame.
17. A machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
determining a frame pair of frame pairs in a sequence of frames to insert an additional frame, wherein the sequence of frames comprises at least three frames, each frame pair of the frame pairs comprises adjacent frames in the sequence of frames, and the determining comprises:
processing the frame pairs of the sequence of frames, and the processing comprises, for each frame pair:
determining at least one candidate frame for insertion between the adjacent frames of the frame pair, and
determining respective similarity scores for the at least one candidate frame, wherein a similarity score for a candidate frame is based on an amount of difference between the candidate frame and a frame of the pair of adjacent frames; and
selecting the frame pair associated with a selected candidate frame from the candidate frames of the frame pairs having a respective similarity score indicative of a highest similarity; and
inserting the selected candidate frame as the additional frame between the adjacent frames of the frame pair of the sequence of frames to generate a new sequence of frames.
18. The machine-readable storage medium of claim 17 , wherein the at least one candidate frame for the frame pair comprises a first frame of the adjacent frames of the frame pair according to a sequential order of the sequence of frames, and the frame is a second frame of the adjacent frames of the frame pair according to the sequential order.
19. The machine-readable storage medium of claim 17 , wherein the at least one candidate frame for the frame pair comprises a created frame generating by combining the adjacent frames of the frame pair, and a second frame of the adjacent frames of the frame pair according to a sequential order of the sequence of frames.
20. The machine-readable storage medium of claim 17 , wherein the at least one candidate frame comprises:
a first frame of the adjacent frames of the frame pair, and
a created frame generating by combining the adjacent frames of the frame pair.