Video frame pulldown based on frame analysis
The described technology is directed towards generating a new video image sequence (e.g., for playback at 30 frames per second) based on an existing video image sequence (e.g., originated for playback at 24 frames per second). The technology is based on processing frames, e.g., adjacent pairs of frames in a four-frame sequence, to obtain candidate frames for selecting a similar candidate frame to insert into the original sequence to create the new sequence (e.g., a five-frame sequence). Aspects include selecting a repeated frame to insert or creating a new frame from existing frames to insert, to generate the new sequence based on a difference/scoring comparison.
1. A method, comprising:
choosing, by a system comprising a processor, a frame pair of frame pairs in a sequence of frames to insert an additional frame, wherein the sequence of frames comprises at least three frames, each frame pair of the frame pairs comprises adjacent frames in the sequence of frames, and the choosing comprises:
generating multiple candidate frames for each frame pair of the frame pairs using different frame creation procedures for respective candidate frames of the multiple candidate frames,
determining respective scores for the multiple candidate frames;
choosing a candidate frame of the multiple candidate frames based on the respective scores of the candidate frames; and
choosing the frame pair associated with the candidate frame; and
inserting, by the system, the candidate frame as the additional frame between the adjacent frames of the frame pair to generate a new sequence of frames.
2. The method of claim 1 , further comprising choosing, by the system, a set of frame creation procedures from sets of frame creation procedures based on a type of media content of a media object containing the sequence of frames, and wherein the set of frame creation procedures comprises the different frame creation procedures.
3. The method of claim 1 , further comprising choosing, by the system, a set of frame creation procedures from sets of frame creation procedures based on a type of motion in a video object containing the sequence of frames, and wherein the set of frame creation procedures comprises the different frame creation procedures.
4. The method of claim 1 , wherein a score of the respective scores is indicative of a similarity of an associated candidate frame to a frame of an associated frame pair.
5. The method of claim 4 , wherein the choosing the candidate frame comprises selecting the candidate frame with the score indicative of a greatest similarity to the frame of the associated frame pair.
6. The method of claim 4 , wherein the choosing the candidate frame comprises selecting the candidate frame with the score indicative of a least similarity to the frame of the associated frame pair.
7. The method of claim 1 , wherein the respective scores are based on peak signal to noise ratio data.
8. A system, comprising:
a processor, and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
generating a new sequence of frames from a sequence of frames, wherein the generating comprises:
determining a frame pair of frame pairs in the sequence of frames to insert an additional frame, wherein the sequence of frames comprises at least three frames, each frame pair of the frame pairs comprises adjacent frames in the sequence of frames, and the choosing comprises:
generating multiple candidate frames for each frame pair of the frame pairs using different frame creation procedures for respective candidate frames of the multiple candidate frames,
determining respective scores for the multiple candidate frames;
determining a candidate frame of the multiple candidate frames based on the respective scores of the candidate frames; and
determining the frame pair associated with the candidate frame; and
inserting the candidate frame as the additional frame between the adjacent frames of the frame pair to generate the new sequence of frames.
9. The system of claim 8 , wherein the operations further comprise determining a set of frame creation procedures from sets of frame creation procedures based on a type of media content of a media object containing the sequence of frames, and wherein the set of frame creation procedures comprises the different frame creation procedures.
10. The system of claim 8 , wherein the operations further comprise determining a set of frame creation procedures from sets of frame creation procedures based on a type of motion in a video object containing the sequence of frames, and wherein the set of frame creation procedures comprises the different frame creation procedures.
11. The system of claim 8 , wherein a score of the respective scores is indicative of a similarity of an associated candidate frame to a frame of an associated frame pair.
12. The system of claim 11 , wherein the determining the candidate frame comprises determining the candidate frame with the score indicative of a greatest similarity to the frame of the associated frame pair.
13. The system of claim 11 , wherein the determining the candidate frame comprises determining the candidate frame with the score indicative of a least similarity to the frame of the associated frame pair.
14. The system of claim 8 , wherein the respective scores are based on peak signal to noise ratio determinations.
15. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
performing frame pulldown on a sequence of frames, comprising:
selecting a frame pair of frame pairs in the sequence of frames to insert an additional frame, wherein the sequence of frames comprises at least three frames, each frame pair of the frame pairs comprises adjacent frames in the sequence of frames, and the selecting comprises:
generating multiple candidate frames for each frame pair of the frame pairs using different frame creation procedures for respective candidate frames of the multiple candidate frames,
determining respective scores for the multiple candidate frames;
selecting a candidate frame of the multiple candidate frames based on the respective scores of the candidate frames; and
selecting the frame pair associated with the candidate frame; and
inserting the candidate frame as the additional frame between the adjacent frames of the frame pair to generate a new sequence of frames.
16. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise selecting a set of frame creation procedures from sets of frame creation procedures based on a type of media content of a media object containing the sequence of frames, and wherein the set of frame creation procedures comprises the different frame creation procedures.
17. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise selecting a set of frame creation procedures from sets of frame creation procedures based on a type of motion in a video object containing the sequence of frames, and wherein the set of frame creation procedures comprises the different frame creation procedures.
18. The non-transitory machine-readable storage medium of claim 15 , wherein a score of the respective scores is indicative of a similarity of an associated candidate frame to a frame of an associated frame pair.
19. The non-transitory machine-readable storage medium of claim 18 , wherein the selecting the candidate frame comprises selecting the candidate frame with the score indicative of a greatest similarity to the frame of the associated frame pair.
20. The non-transitory machine-readable storage medium of claim 19 , wherein the selecting the candidate frame comprises selecting the candidate frame with the score indicative of a least similarity to the frame of the associated frame pair.