Granted Patent B2

US 12,307,766 · App. 18/078,811 · Granted May 20, 2025

Inventors: Qiuran Tian (Shanghai, CN); Zhaoxin Tan (Shanghai, CN)

Assignee: SHANGHAI BILIBILI TECHNOLOGY CO., LTD.

G06V20/40H04N21/440218

Abstract

Claims (73)

1. A Web-end video playback method, comprising:

receiving on a Web-end computing device FLV-format video data to be played on the Web-end computing device;

decapsulating the FLV-format video data on the Web-end computing device by separating video data from format information in the FLV-format video data using a preset WebAssembly bytecode to obtain effective video data, the effective video data comprising audio data and image data not including packet headers;

storing on the Web-end computing device the effective video data into a preset doubly linked list;

encapsulating the effective video data in the preset doubly linked list using the preset WebAssembly bytecode to obtain FMP4-format video data; and

loading, through media source extensions, the FMP4-format video data to a Web-end video player for playback on the Web-end computing device for viewing by a user of the Web-end computing device.

2. The Web-end video playback method according to claim 1 , wherein the preset WebAssembly bytecode is generated by compiling C language code using an Emscripten tool, and wherein the C language code comprises instruction code configured to decapsulate, at a Web end, the FLV-format video data and encapsulate, at the Web end, the effective video data into the FMP4-format video data.

3. The Web-end video playback method according to claim 1 , wherein the decapsulating the FLV-format video data using a preset WebAssembly bytecode to obtain effective video data further comprises:

identifying, based on an FLV format, an FLV tag in a data encapsulation packet corresponding to the FLV-format video data; and

extracting the effective video data corresponding to the FLV-format video data from the FLV tag.

4. The Web-end video playback method according to claim 1 , wherein the encapsulating the effective video data in the preset doubly linked list using the preset WebAssembly bytecode to obtain FMP4-format video data further comprises:

identifying the effective video data using a head node pointer and a tail node pointer of each data node in the preset doubly linked list; and

encapsulating, based on an FMP4 format, the effective video data into an MP4Box corresponding to the FMP4-format video data.

5. The Web-end video playback method according to claim 4 , wherein the encapsulating, based on an FMP4 format, the effective video data into an MP4Box corresponding to the FMP4-format video data further comprises:

splitting and encapsulating the effective video data into a plurality of boxes based on a preset box size in the FMP4 format; and

arranging the plurality of boxes in a sequence, and splicing the plurality of boxes based on the arrangement sequence to form the MP4Box corresponding to the FMP4-format video data.

6. The Web-end video playback method according to claim 4 , wherein after the identifying the effective video data using a head node pointer and a tail node pointer of each data node in the preset doubly linked list, the Web-end video playback method further comprises:

obtaining each audio frame of audio data in the effective video data and a corresponding timestamp, wherein the corresponding timestamp is a reference timestamp for synchronously playing audio data and image data in video data;

dividing the audio data in the effective video data into a plurality of audio data segments based on a preset time interval;

determining a quantity n of audio frames in a first audio data segment in the plurality of audio data segments and a timestamp difference L between a first audio frame and a last audio frame in the first audio data segment, wherein the first audio data segment is any audio data segment among the plurality of audio data segments;

generating a second audio data segment by inserting at least one blank audio frame into the first audio data segment based on L, n, and a preset rule of blank audio frame insertion; and

replacing the first audio data segment with the second audio data segment.

7. The Web-end video playback method according to claim 6 , wherein the preset rule of blank audio frame insertion comprises:

in response to determining that L is less than a preset threshold M, skipping insertion of the at least one blank audio frame into the first audio data segment; and

in response to determining that L is greater than or equal to the preset threshold M, inserting the at least one blank audio frame into the first audio data segment, wherein a number N of the at least one blank audio frame inserted into the first audio data segment is determined based on a formula: N=(L−n*x)/x, wherein x represents an audio inter-frame spacing, and M>(n+1)*x.

8. A computing device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable by the processor, wherein the processor executes the computer-readable instructions to implement operations comprising:

receiving FLV-format video data to be played on the computing device;

decapsulating the FLV-format video data by separating video data from format information in the FLV-format video data using a preset WebAssembly bytecode to obtain effective video data, the effective video data comprising audio data and image data not including packet headers;

storing on the computing device the effective video data into a preset doubly linked list;

encapsulating the effective video data in the preset doubly linked list using the preset WebAssembly bytecode to obtain FMP4-format video data; and

loading, through media source extensions, the FMP4-format video data to a Web-end video player for playback on the computing device for viewing by a user of the computing device.

9. The computing device according to claim 8 , wherein the decapsulating the FLV-format video data using a preset WebAssembly bytecode to obtain effective video data further comprises:

identifying, based on an FLV format, an FLV tag in a data encapsulation packet corresponding to the FLV-format video data; and

extracting the effective video data corresponding to the FLV-format video data from the FLV tag.

10. The computing device according to claim 8 , wherein the encapsulating the effective video data in the preset doubly linked list using the preset WebAssembly bytecode to obtain FMP4-format video data further comprises:

identifying the effective video data using a head node pointer and a tail node pointer of each data node in the preset doubly linked list; and

encapsulating, based on an FMP4 format, the effective video data into an MP4Box corresponding to the FMP4-format video data.

11. The computing device according to claim 10 , wherein the encapsulating, based on an FMP4 format, the effective video data into an MP4Box corresponding to the FMP4-format video data further comprises:

splitting and encapsulating the effective video data into a plurality of boxes based on a preset box size in the FMP4 format; and

arranging the plurality of boxes in a sequence, and splicing the plurality of boxes based on the arrangement sequence to form the MP4Box corresponding to the FMP4-format video data.

12. The computing device according to claim 10 , wherein after the identifying the effective video data using a head node pointer and a tail node pointer of each data node in the preset doubly linked list, the operations further comprise:

obtaining each audio frame of audio data in the effective video data and a corresponding timestamp, wherein the corresponding timestamp is a reference timestamp for synchronously playing audio data and image data in video data;

dividing the audio data in the effective video data into a plurality of audio data segments based on a preset time interval;

determining a quantity n of audio frames in a first audio data segment in the plurality of audio data segments and a timestamp difference L between a first audio frame and a last audio frame in the first audio data segment, wherein the first audio data segment is any audio data segment among the plurality of audio data segments;

generating a second audio data segment by inserting at least one blank audio frame into the first audio data segment based on L, n, and a preset rule of blank audio frame insertion; and

replacing the first audio data segment with the second audio data segment.

13. The computing device according to claim 12 , wherein the preset rule of blank audio frame insertion comprises:

in response to determining that L is less than a preset threshold M, skipping insertion of the at least one blank audio frame into the first audio data segment; and

in response to determining that L is greater than or equal to the preset threshold M, inserting the at least one blank audio frame into the first audio data segment, wherein a number N of the at least one blank audio frame inserted into the first audio data segment is determined based on a formula: N=(L−n*x)/x, wherein x represents an audio inter-frame spacing, and M>(n+1)*x.

14. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer readable instructions, and the computer readable instructions are executable by at least one processor on a computing device to cause the computing device to perform operations comprising:

obtaining FLV-format video data to be played on the computing device;

decapsulating the FLV-format video data on the computing device by separating video data from format information in the FLV-format video data using a preset WebAssembly bytecode to obtain effective video data, the effective video data comprising audio data and image data not including packet headers;

storing the effective video data on the computing device into a preset doubly linked list;

encapsulating the effective video data in the preset doubly linked list using the preset WebAssembly bytecode to obtain FMP4-format video data; and

loading, through media source extensions, the FMP4-format video data to a Web-end video player for playback on the computing device for viewing by a user of the computing device.

15. The non-transitory computer-readable storage medium according to claim 14 , wherein the decapsulating the FLV-format video data using a preset WebAssembly bytecode to obtain effective video data further comprises:

identifying, based on an FLV format, an FLV tag in a data encapsulation packet corresponding to the FLV-format video data; and

extracting the effective video data corresponding to the FLV-format video data from the FLV tag.

16. The non-transitory computer-readable storage medium according to claim 14 , wherein the encapsulating the effective video data in the preset doubly linked list using the preset WebAssembly bytecode to obtain FMP4-format video data further comprises:

identifying the effective video data using a head node pointer and a tail node pointer of each data node in the preset doubly linked list; and

encapsulating, based on an FMP4 format, the effective video data into an MP4Box corresponding to the FMP4-format video data.

17. The non-transitory computer-readable storage medium according to claim 16 , wherein the encapsulating, based on an FMP4 format, the effective video data into an MP4Box corresponding to the FMP4-format video data further comprises:

splitting and encapsulating the effective video data into a plurality of boxes based on a preset box size in the FMP4 format; and

arranging the plurality of boxes in a sequence, and splicing the plurality of boxes based on the arrangement sequence to form the MP4Box corresponding to the FMP4-format video data.

18. The non-transitory computer-readable storage medium according to claim 16 , wherein after the identifying the effective video data using a head node pointer and a tail node pointer of each data node in the preset doubly linked list, the operations further comprise:

obtaining each audio frame of audio data in the effective video data and a corresponding timestamp, wherein the corresponding timestamp is a reference timestamp for synchronously playing audio data and image data in video data;

dividing the audio data in the effective video data into a plurality of audio data segments based on a preset time interval;

determining a quantity n of audio frames in a first audio data segment in the plurality of audio data segments and a timestamp difference L between a first audio frame and a last audio frame in the first audio data segment, wherein the first audio data segment is any audio data segment among the plurality of audio data segments;

generating a second audio data segment by inserting at least one blank audio frame into the first audio data segment based on L, n, and a preset rule of blank audio frame insertion; and

replacing the first audio data segment with the second audio data segment.

19. The non-transitory computer-readable storage medium according to claim 18 , wherein the preset rule of blank audio frame insertion comprises:

in response to determining that L is less than a preset threshold M, skipping insertion of the at least one blank audio frame into the first audio data segment; and

in response to determining that L is greater than or equal to the preset threshold M, inserting the at least one blank audio frame into the first audio data segment, wherein a number N of the at least one blank audio frame inserted into the first audio data segment is determined based on a formula: N=(L−n*x)/x, wherein x represents an audio inter-frame spacing, and M>(n+1)*x.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (22)