Content-based client side video transcoding
Among other things, embodiments of the present disclosure improve the functionality of electronic messaging and imaging software and systems by automating the client-side transcoding of video data based on content. For example, an appropriate transcoding configuration can be selected for video data having complex motion or textures. Accordingly, video quality can be improve when complex motions or textures are present.
1. A system comprising:
a processor;
a camera coupled to the processor; and
memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising:
causing the camera to capture video data use in a messaging system;
encoding the captured video data to generate encoded video data;
determining a complexity within a field of view of the camera based on a comparison between at least a portion of the captured video data and at least a portion of the encoded video data, wherein determining the complexity within the field of view of the camera includes determining an average difference between a plurality of frames in the captured video data and a plurality of frames in the encoded video data;
and
generating, based on the determined complexity and the encoded video, transcoded video data,
wherein generating the transcoded video includes selecting a transcoding configuration that applies a bitrate to the transcoded video,
wherein the bitrate applied to the transcoded video is selected to correspond to a video quality metric, based on predefined mappings between video quality metrics and the transcoding configuration, and
wherein the predefined mappings are stored within a database and are accessible via the database, to provide consistent video transcoding with respect to messaging clients of the messaging system.
2. The system of claim 1 , wherein determining the complexity within the field of view of the camera includes detecting motion within the field of view.
3. The system of claim 2 , wherein determining the complexity within the field of view of the camera includes determining that the detected motion exceeds a predetermined threshold.
4. The system of claim 1 , wherein determining the complexity within the field of view of the camera includes detecting a texture within the field of view.
5. The system of claim 1 , wherein determining the complexity within the field of view of the camera includes determining a square error between a plurality of frames in the captured video data and a plurality of frames in the encoded video data.
6. The system of claim 1 , wherein determining the complexity within the field of view of the camera includes determining a peak signal-to-noise ratio between frames in the captured video data and the encoded video data.
7. The system of claim 1 , wherein the captured video data is uncompressed.
8. The system of claim 1 , wherein the captured video data is compressed.
9. The system of claim 1 , wherein the system receives additional video data from a device in communication with the system.
10. The system of claim 1 , wherein the bitrate applied to the transcoded video is one of:
a fixed bitrate, or a variable bitrate.
11. The system of claim 1 , wherein the bitrate applied to the transcoded video is capped.
12. The system of claim 1 , wherein generating the transcoded video includes transcoding the encoded video according to a predetermined transcoding profile to generate the transcoded video at a predetermined bitrate.
13. The system of claim 1 , further comprising:
a display screen coupled to the processor,
wherein the operations further comprise presenting the video data on the display screen.
14. The system of claim 13 , further comprising:
an input device,
wherein the operations further comprise receiving, from a user of the system via the input device, a user interaction for initiating a recording of the video data.
15. The system of claim 1 , wherein encoding the captured video data is performed using one or more of: a frame by frame encoding, and an encoding of a collection of frames.
16. The system of claim 1 , wherein determining the complexity within the field of view of the camera includes encoding a video frame to a pixel frame and performing one or more subtraction operations with a corresponding frame of the captured video data.
17. A computer-implemented method comprising:
causing, by a computer system, a camera coupled to the computer system to capture video data;
encoding, by the computer system; the captured video data to generate encoded video data;
determining, by the computer system, a complexity within a field of view of the camera based on a comparison between at least a portion of the captured video data and at least a portion of the encoded video data, wherein determining the complexity within the field of view of the camera includes determining an average difference between a plurality of frames in the captured video data and a plurality of frames in the encoded video data;
and
generating, by the computer system and based on the determined complexity and the encoded video, transcoded video data,
wherein generating the transcoded video includes selecting a transcoding configuration that applies a bitrate to the transcoded video,
wherein the bitrate applied to the transcoded video is selected to correspond to a video quality metric, based on predefined mappings between video quality metrics and the transcoding configuration, and
wherein the predefined mappings are stored within a database and are accessible via the database, to provide consistent video transcoding with respect to messaging clients of the messaging system.
18. A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:
causing a camera coupled to the computer system to capture video data;
encoding the captured video data to generate encoded video data;
determining a complexity within a field of view of the camera based on a comparison between at least a portion of the captured video data and at least a portion of the encoded video data, wherein determining the complexity within the field of view of the camera includes determining an average difference between a plurality of frames in the captured video data and a plurality of frames in the encoded video data; and
generating, based on the determined complexity and the encoded video, transcoded video data,
wherein generating the transcoded video includes selecting a transcoding configuration that applies a bitrate to the transcoded video,
wherein the bitrate applied to the transcoded video is selected to correspond to a video quality metric, based on predefined mappings between video quality metrics and the transcoding configuration, and
wherein the predefined mappings are stored within a database and are accessible via the database, to provide consistent video transcoding with respect to messaging clients of the messaging system.