Systems and methods for enhanced video encoding
Systems and methods for enhanced video encoding identify patterns in sequences of raw digital video frames to extract features and identify the type of content represented by the extracted features. The system simulates many outcomes of encoding the sequence of digital video frames by using various different encoding strategies to find the relative best encoding strategy for each sequence of frames. As the encoder processes video, it passes digital video frames to a modeling system which determines whether the video, or video having that same type of content, has been previously observed by the system. The system then selectively applies a saved encoding strategy that had been determined by the system to be particularly suitable for encoding the same sequence of video frames or that same type of content.
1. A method in a digital video encoding system, the method comprising:
receiving, by a digital video encoder of the video encoding system, a sequence of digital video frames;
determining, by the video encoding system, that the sequence of video frames has been previously processed by the video encoding system;
in response to the video encoding system determining that the sequence of video frames has been previously processed by the video encoding system, selecting, by the video encoding system, a video encoding strategy previously used to encode the sequence of video frames and previously saved by the video encoding system for being particularly suitable for encoding the sequence of video frames; and
encoding, by the digital video encoder, the sequence of video frames using the selected video encoding strategy previously saved by the video encoding system for being particularly suitable for encoding the sequence of video frames.
2. The method of claim 1 further comprising:
before encoding the sequence of video frames using the selected video encoding strategy previously saved by the video encoding system for being particularly suitable for encoding the sequence of video frames:
receiving, by a machine learning module of the digital video encoding system, a plurality of different sequences of digital video frames;
detecting, by the machine learning module of the digital video encoding system, a type of content of the plurality of different sequences of digital video frames as an identified type of content shown throughout the sequence of video frames;
encoding, by the machine learning module of the digital video encoding system, the plurality of different sequences of digital video frames using various different encoding strategies;
determining, by the machine learning module of the digital video encoding system, that one of the various different encoding strategies is an encoding strategy particularly suitable for encoding one or more features associated with the identified type of content;
associating, by the machine learning module of the digital video encoding system, with the identified type of content the encoding strategy particularly suitable for encoding the one or more features associated with the identified type of content;
saving in an encoding library, by the machine learning module of the digital video encoding system, the association of the identified type of content with the encoding strategy particularly suitable for encoding the one or more features associated with the identified type of content; and
saving in the encoding model library, by the machine learning module of the digital video encoding system, the encoding strategy particularly suitable for encoding the one or more features associated with the identified type of content.
3. The method of claim 2 wherein the detecting a type of content of the plurality of different sequences of digital video frames as the identified type of content includes:
extracting various features from the plurality of different sequences of digital video frames; and
identifying the extracted various features as being the one or more features associated with the identified type of content.
4. The method of claim 3 wherein the various features are representations of physical objects throughout plurality of different sequences of digital video frames associated with the identified type of content.
5. The method of claim 4 wherein the physical objects include one or more objects associated with sports and the identified type of content is sports.
6. The method of claim 5 wherein the physical objects include one or more of: a baseball, a bat, a basketball, a football, a football helmet, a soccer ball, a stadium, a large crowd of people.
7. The method of claim 2 wherein the determining that one of the various different encoding strategies is the encoding strategy particularly suitable for encoding the one or more features associated with the identified type of content includes:
saving results of encoding the one or more features associated with the identified type of content from the encoding of the plurality of different sequences of digital video frames using the various different encoding strategies;
comparing, using predetermined criteria, the results of encoding the one or more features associated with the identified type of content from the encoding of the plurality of different sequences of digital video frames using the various different encoding strategies; and
selecting, based on the predetermined criteria, one of the various different encoding strategies used to encode the plurality of different sequences of digital video frames as the encoding strategy particularly suitable for encoding the one or more features associated with the identified type of content based on the comparison of, using the predetermined criteria, the results of encoding the one or more features associated with the identified type of content.
8. The method of claim 7 wherein the predetermined criteria are regarding measurements related to one or more of: reduction of color range, uniform or targeted sharpness reduction, frame duplication, proactive bandwidth allocation.
9. The method of claim 2 wherein the various different encoding strategies include two or more of: reducing color range, uniform or targeted sharpness reduction, frame duplication, proactive bandwidth allocation.
10. A digital video encoding system comprising:
at least one processor; and
at least one memory coupled to the at least one processor having computer executable instructions stored thereon that, when executed, cause the at least one processor to:
receive a sequence of digital video frames;
determine that the sequence of digital video frames has been previously processed by the video encoding system;
in response to determining that the sequence of digital video frames has been previously processed by the video encoding system, select a video encoding strategy previously used to encode the sequence of digital video frames, the video encoding strategy previously saved by the video encoding system for being particularly suitable for encoding the sequence of digital video frames;
encode the sequence of video frames using the selected video encoding strategy previously saved by the video encoding system for being particularly suitable for encoding the sequence of digital video frames; and
output the sequence of video frames encoded using the selected video encoding strategy.
11. The system of claim 10 wherein the computer executable instructions, when executed, further cause the at least one processor to:
before encoding the sequence of video frames using the selected video encoding strategy previously saved by the video encoding system for being particularly suitable for encoding the sequence of video frames:
receive a plurality of sequences of digital video frames on multiple different occasions;
detect that each of the plurality of sequences of digital video frames is the same sequence of digital video frames;
encode the plurality of different sequences of digital video frames using various different encoding strategies; and
determine that one of the various different encoding strategies is the encoding strategy particularly suitable for encoding the same sequence of digital video frames;
associate the encoding strategy particularly suitable for encoding the same sequence of digital video frames with the same sequence of digital video frames;
save in an encoding library the association of the encoding strategy particularly suitable for encoding the same sequence of digital video frames with the same sequence of digital video frames; and
save in the encoding model library the encoding strategy particularly suitable for encoding the same sequence of digital video frames.
12. The system of claim 11 wherein the detection that each of the plurality of sequences of digital video frames is the same sequence of digital video frames includes:
comparison of the plurality of different sequences of digital video frames to each other; and
determining that the plurality of sequences of digital video frames is the same sequence of digital video frames based on the comparison of the plurality of different sequences of digital video frames to each other.
13. The system of claim 12 wherein the determining that one of the various different encoding strategies is the encoding strategy particularly suitable for encoding the same sequence of digital video frames includes:
saving results of encoding the plurality of different sequences of digital video frames using various different encoding strategies;
comparing, using predetermined criteria, the results of encoding the plurality of different sequences of digital video frames using various different encoding strategies; and
selecting, based on the predetermined criteria, one of the various different encoding strategies used to encode the plurality of different sequences of digital video frames as the encoding strategy particularly suitable for encoding the same sequence of digital video frames based on the comparison of, using the predetermined criteria, the results of encoding the plurality of different sequences of digital video frames using various different encoding strategies.
14. The system of claim 13 wherein the predetermined criteria are regarding measurements related to one or more of: reduction of color range, uniform or targeted sharpness reduction, frame duplication, proactive bandwidth allocation.
15. The system of claim 11 wherein the various different encoding strategies include two or more of: reducing color range, uniform or targeted sharpness reduction, frame duplication, proactive bandwidth allocation.