IP Library Granted Patent US 12,069,281
Granted Patent B2
US 12,069,281 · App. 18/452,016 · Granted Aug 20, 2024

Content-based client side video transcoding

Inventors: Liu Liu (Marina Del Rey, CA); Chao Pang (Marina Del Rey, CA); Rong Yan (Marina Del Rey, CA)
Assignee: Snap Inc.
H04N19/40H04L65/70H04L65/762H04N5/77H04N5/9261H04N9/8042H04N19/137H04N19/14H04N19/146H04N19/162H04N19/179H04N21/2662H04N21/4223H04N21/4334H04N21/4402H04W88/181
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,069,281
App. No.
18/452,016
Granted
Aug 20, 2024
Kind
B2
Abstract

Among other things, embodiments of the present disclosure improve the functionality of electronic messaging and imaging software and systems by automating the client-side transcoding of video data based on content. For example, an appropriate transcoding configuration can be selected for video data having complex motion or textures. Accordingly, video quality can be improve when complex motions or textures are present.

Claims (29)

1. A system comprising:

a processor;

a user interface coupled to the processor, the user interface comprising an image capture device; and

memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising:

determining a complexity within a field of view of an image capture device that captured video data; and

generating, based on the determined complexity and the video data, transcoded video data by selecting a transcoding configuration that applies a bitrate to the video data, the transcoding configuration selected from a plurality of transcoding configurations of varying bitrates, wherein the bitrate applied to the transcoded video is selected to correspond to a video quality metric, based on predefined mappings between at least one video quality metric and the transcoding configuration.

2. The system of claim 1 , wherein determining the complexity within the field of view of the image capture device includes detecting motion within the field of view.

3. The system of claim 2 , wherein determining the complexity within the field of view of the image capture device includes determining that the detected motion exceeds a predetermined threshold.

4. The system of claim 1 , wherein determining the complexity within the field of view of the image capture device includes detecting a texture within the field of view.

5. The system of claim 1 , wherein determining the complexity within the field of view of the image capture device includes determining an average difference between a plurality of frames in the captured video data and a plurality of frames in the video data.

6. The system of claim 1 , wherein determining the complexity within the field of view of the image capture device includes determining a square error between a plurality of frames in the captured video data and a plurality of frames in the video data.

7. The system of claim 1 , wherein determining the complexity within the field of view of the image capture device includes determining a peak signal-to-noise ratio between a first frame in the video data and a second frame in the video data.

8. The system of claim 1 , wherein the bitrate applied to the transcoded video is one of: a fixed bitrate, or a variable bitrate.

9. The system of claim 1 , wherein the bitrate applied to the transcoded video is capped.

10. The system of claim 1 , wherein generating the transcoded video includes transcoding the video according to a predetermined transcoding profile to generate the transcoded video at a predetermined bitrate.

11. The system of claim 1 , wherein the user interface includes a display screen, and capturing the video data includes presenting the video data on the display screen.

12. The system of claim 11 , wherein the user interface further includes an input device, and wherein the memory further stores instructions for receiving, from a user of the system via the input device, a user interaction for initiating a recording of the video data.

13. The system of claim 1 , wherein the operations further comprise encoding the captured video data to generate encoded video data, wherein the complexity is determined based on the encoded video data.

14. The system of claim 13 , wherein the complexity is determined based on a comparison between at least a portion of the captured video data and at least a portion of the encoded video data.

15. The system of claim 13 , wherein encoding the captured video data is performed using one or more of: a frame by frame encoding, and an encoding of a collection of frames.

16. The system of claim 13 , wherein the captured video data is compressed, wherein the difference between the captured video data and the encoded video data results in a larger difference for compression artifacts between frames than non-compression artifacts.

17. The system of claim 1 , wherein determining the complexity within the field of view of the image capture device includes encoding a video frame to a pixel frame and performing one or more subtraction operations with a corresponding frame of the captured video data.

18. The system of claim 1 , wherein the predefined mappings are stored within a database and are accessible via the database, to provide consistent video transcoding with respect to messaging clients of the messaging system.

19. A computer-implemented method comprising:

determining a complexity within a field of view of an image capture device that captured video data; and

generating, based on the determined complexity and the video data, transcoded video data by selecting a transcoding configuration that applies a bitrate to the video data, the transcoding configuration selected from a plurality of transcoding configurations of varying bitrates, wherein the bitrate applied to the transcoded video is selected to correspond to a video quality metric, based on predefined mappings between at least one video quality metric and the transcoding configuration.

20. A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:

determining a complexity within a field of view of an image capture device that captured video data; and

generating, based on the determined complexity and the video data, transcoded video data by selecting a transcoding configuration that applies a bitrate to the video data, the transcoding configuration selected from a plurality of transcoding configurations of varying bitrates, wherein the bitrate applied to the transcoded video is selected to correspond to a video quality metric, based on predefined mappings between at least one video quality metric and the transcoding configuration.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2023
From: LIU, LIU; PANG, CHAO; YAN, RONG
To: SNAP INC.
Reel/Frame 064634/0388 →
Continuity (4)
Continuation 17322407 · May 17, 2021
Continuation 15622336 · Jun 14, 2017
Provisional Application 62448465 · Jan 20, 2017
Related Publication 20230396785A1 · Dec 7, 2023