Method and apparatus for content-adaptive frame duration extension
Systems and methods are described herein for processing video data. An encoder may transmit information indicating a duration that a particular frame should be displayed. The information may comprise an indication of whether the particular frame should be displayed for an increased duration. The encoder may decide during encoding, a whether the particular frame should be displayed for an increased duration (e.g., doubled, tripled, or more). The decision may be based on a similarity analysis performed by the encoder on two or more frames. Information indicating the decision may be inserted into a message, which may then be transmitted in a transport stream comprising encoded video data to a decoder. The decoder may decode the information and cause a display device to display the particular frame for the increased duration.
1. A method comprising:
receiving video content comprising two or more frames;
determining, based on one or more quality metrics, a similarity score, associated with each of the two or more frames, that is indicative of similarity between consecutive frames of the two or more frames;
adjusting, based on one or more criteria, the similarity score, wherein the one or more criteria are associated with at least one of: a scene change in the video content or motion in the video content;
determining that the adjusted similarity score, associated with at least one frame of the two or more frames, exceeds a threshold indicating a minimum similarity score for identification of similar sequences of frames;
generating, based on the threshold being exceeded, information comprising an instruction to display the at least one frame for an increased duration during playback of the video content; and
sending, to a decoder, the information to cause the at least one frame to be displayed for the increased duration.
2. The method of claim 1 , wherein the information is sent in a supplemental enhancement information (SEI) message.
3. The method of claim 1 , wherein the information further comprises an instruction to not encode a second at least one frame.
4. The method of claim 1 , wherein the one or more quality metrics comprise at least one of: a peak signal to noise ratio (PSNR), a video multimethod assessment fusion (VMAF), or a structural similarity (SSIM)+.
5. The method of claim 1 , wherein the adjusting increases the similarity score if a scene cut is detected in the consecutive frames of the two or more frames.
6. The method of claim 1 , wherein the adjusting decreases the similarity score if motion is detected in the consecutive frames of the two or more frames.
7. The method of claim 1 , wherein the adjusting decreases the similarity score if a previous frame, that is a distance shorter than a predetermined distance, was extended.
8. A device, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the device to:
receive video content comprising two or more frames;
determine, based on one or more quality metrics, a similarity score, associated with each of the two or more frames, that is indicative of similarity between consecutive frames of the two or more frames;
adjusting, based on one or more criteria, the similarity score, wherein the one or more criteria are associated with at least one of: a scene change in the video content or motion in the video content;
determine that the adjusted similarity score, associated with at least one frame of the two or more frames, exceeds a threshold indicating a minimum similarity score for identification of similar sequences of frames;
generate, based on the threshold being exceeded, information comprising an instruction to display the at least one frame for an increased duration during playback of the video content; and
send, to a decoder, the information to cause the at least one frame to be displayed for the increased duration.
9. The device of claim 8 , wherein the information is sent in a supplemental enhancement information (SEI) message.
10. The device of claim 8 , wherein the information further comprises an instruction to not encode a second at least one frame.
11. The device of claim 8 , wherein the one or more quality metrics comprise at least one of: a peak signal to noise ratio (PSNR), a video multimethod assessment fusion (VMAF), or a structural similarity (SSIM)+.
12. The device of claim 8 , wherein the adjusting increases the similarity score if a scene cut is detected in the consecutive frames of the two or more frames.
13. The device of claim 8 , wherein the adjusting decreases the similarity score if motion is detected in the consecutive frames of the two or more frames.
14. The device of claim 8 , wherein the adjusting decreases the similarity score if a previous frame, that is a distance shorter than a predetermined distance, was extended.
15. A non-transitory computer-readable medium storing instructions that, when executed, cause:
receiving video content comprising two or more frames;
determining, based on one or more quality metrics, a similarity score, associated with each of the two or more frames, that is indicative of similarity between consecutive frames of the two or more frames;
adjusting, based on one or more criteria, the similarity score, wherein the one or more criteria are associated with at least one of: a scene change in the video content or motion in the video content;
determining that the adjusted similarity score, associated with at least one frame of the two or more frames, exceeds a threshold indicating a minimum similarity score for identification of similar sequences of frames;
generating, based on the threshold being exceeded, information comprising an instruction to display the at least one frame for an increased duration during playback of the video content; and
sending, to a decoder, the information to cause the at least one frame to be displayed for the increased duration.
16. The non-transitory computer-readable medium of claim 15 , wherein the information is sent in a supplemental enhancement information (SEI) message.
17. The non-transitory computer-readable medium of claim 15 , wherein the information further comprises an instruction to not encode a second at least one frame.
18. The non-transitory computer-readable medium of claim 15 , wherein the one or more quality metrics comprise at least one of: a peak signal to noise ratio (PSNR), a video multimethod assessment fusion (VMAF), or a structural similarity (SSIM)+.
19. The non-transitory computer-readable medium of claim 15 , wherein the adjusting increases the similarity score if a scene cut is detected in the consecutive frames of the two or more frames.
20. The non-transitory computer-readable medium of claim 15 , wherein the adjusting decreases the similarity score if at least one of:
motion is detected in the consecutive frames of the two or more frames, or
if a previous frame, that is a distance shorter than a predetermined distance, was extended.