IP Library Granted Patent US 11,025,964
Granted Patent B2
US 11,025,964 · App. 16/569,866 · Granted Jun 1, 2021

Method, apparatus, server, and storage medium for generating live broadcast video of highlight collection

Inventors: Zhengui Zheng (Shanghai, CN); Lei Sun (Shanghai, CN); Xiangxiang Chen (Shanghai, CN)
Assignee: WANGSU SCIENCE & TECHNOLOGY CO., LTD.
H04N21/2187H04N21/2343H04N21/23106
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,025,964
App. No.
16/569,866
Granted
Jun 1, 2021
Kind
B2
Abstract

Some embodiments of the present disclosure relate to the field of video processing, and disclose a method, an apparatus, a server, and a storage medium for generating a live broadcast video of highlight collection. The method for generating a live broadcast video highlight in the embodiments of the present disclosure includes: identifying a live broadcast picture from a live broadcast video and determining if a target image element exists in the live broadcast picture; if the target image element exists in the live broadcast picture, saving, as a highlight segment, a segment of live broadcast video having the live broadcast picture; when a merging condition is satisfied, merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection; and switching an output live broadcast video to the merged video of highlight collection.

Claims (77)

1. A method for generating a live broadcast video of highlight collection, comprising:

identifying a live broadcast picture from a live broadcast video and determining if a target image element exists in the live broadcast picture;

if the target image element exists in the live broadcast picture, saving, as a highlight segment, a segment of the live broadcast video having the live broadcast picture;

when a merging condition is satisfied, merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection; and

switching an output live broadcast video to the merged video of highlight collection;

wherein the identifying a live broadcast picture and determining if a target image element exists in the live broadcast picture specifically comprises:

autonomously identifying the live broadcast picture and autonomously determining if the target image element exists in the live broadcast picture, according to an image identification model established in advance, wherein the image identification model is established by training according to collected image features;

wherein the merging condition is autonomously determined according to the image identification model; and

wherein the merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection comprises:

decoding the highlight segment, and decoding the live broadcast video;

merging each video frame of the decoded highlight segment with each video frame of the

decoded live broadcast video in a preset mode, to obtain merged video data; and

recoding the merged video data to obtain the merged video of highlight collection, such that the merged video of highlight collection displays the live broadcast video and the highlight segment simultaneously.

2. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein the merged data comprises: merged video data and merged audio data; and

the merging the decoded highlight segment into the decoded live broadcast video specifically comprises: merging an audio stream of the decoded highlight segment and an audio stream of the decoded live broadcast video in form of audio mixing, to obtain the merged audio data.

3. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein, the method further comprises performing the following steps before the determining if a target image element exists in the live broadcast picture: caching a data packet of the live broadcast video in real time; and

the saving, as a highlight segment, a segment of live broadcast video having the live broadcast picture specifically comprises: saving a currently cached data packet of the live broadcast video as the highlight segment.

4. The method for generating a live broadcast video of highlight collection according to claim 3 , wherein the method further comprises performing the following step before the saving the currently cached data packet of the live broadcast video as the highlight segment:

determining if a playing duration of the currently cached data packet of the live broadcast video exceeds a preset upper limit; and

if the preset upper limit is exceeded, discarding a part of data in the cached data packet of the live broadcast video.

5. The method for generating a live broadcast video of highlight collection according to claim 4 , wherein the discarding a part of data in the cached data packet live broadcast video specifically comprises:

sequentially discarding, according to a playing order of the data packet of the live broadcast video, video frame sequences in the currently cached data packet of the live broadcast video from front to back, until the playing duration of the cached data packet of the live broadcast video obtained after the discarding does not exceed the preset upper limit.

6. The method for generating a live broadcast video of highlight collection according to claim 3 , wherein the data packet of the live broadcast video saved as the highlight segment is specifically a cached data packet of the live broadcast video in which a first video frame thereof is a key frame.

7. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein, the method further comprises performing the following step before the merging the highlight segment into the live broadcast video based on the merging condition:

if the number of the saved highlight segment is more than one, splicing a plurality of highlight segments into one highlight collection; and

the merging the highlight segment into the live broadcast video is specifically: merging the highlight collection into the live broadcast video.

8. The method for generating a live broadcast video of highlight collection according to claim 7 , wherein the splicing the plurality of highlight segments into one highlight collection specifically comprises:

editing each of the highlight segment;

performing a timestamp recovery on each of the edited highlight segment; and

splicing each of the highlight segment obtained after the timestamp recovery, into one highlight collection according to a timestamp order.

9. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein the switching the output live broadcast video to the merged video of highlight collection specifically comprises:

determining if a switching time point is satisfied, according to a timestamp of the live broadcast video; and

if the switching time point is satisfied, switching the output live broadcast video to the merged video of highlight collection.

10. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein the method further comprises performing the following after the switching the output live broadcast video to the merged video of highlight collection:

determining remaining duration of the merged video of highlight collection; and

when the remaining duration is zero, switching, an output merged video of highlight collection to the live broadcast video.

11. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein the method further comprises performing the following after the switching the output live broadcast video to the merged video of highlight collection:

determining remaining duration of the merged video of highlight collection; and

when the remaining duration of the merged video of highlight collection is zero, determining if a first video frame of the live broadcast video is a key frame; and

if the first video frame of the live broadcast video is the key frame, switching an output merged video of highlight collection to the live broadcast video.

12. The method for generating a live broadcast video of highlight collection according to claim 1 , wherein identifying the live broadcast picture and determining if the target image element exists in the live broadcast picture, according to an image identification model established in advance specifically comprises:

obtaining a specified area in the live broadcast picture;

identifying if the target image element exists in the specified area according to the image identification model established in advance; and

if the target image element exists in the specified area, determining that the target image element exists in the live broadcast picture.

13. The method for generating a live broadcast video of highlight collection according to claim 12 , wherein the identifying if the target image element exists in the specified area specifically comprises:

capturing a target area in the specified area according to a size of the target image element;

inputting a feature value of the target area to the image identification model, and determining if the target image element exists in the target area according to an output result of the image identification model; and

if the target image element does not exist in the target area, shifting a location of the target area according to a preset rule, and identifying if the target image element exists in the shifted target area.

14. The method for generating a live broadcast video of highlight collection according to claim 12 , wherein the method further comprises performing the following step after the obtaining a specified area in the live broadcast picture: obtaining a greyscale map for the specified area; and

identifying if the target image element exists in the specified area according to the image identification model established in advance specifically comprises: identifying if the target image element exists in the greyscale map according to the image identification model established in advance.

15. A server, comprising:

at least one processor; and

a memory communicatively connected to the at least one processor, wherein

the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to implement a method for generating a live broadcast video of highlight collection, wherein, the method comprises:

identifying a live broadcast picture from a live broadcast video and determining if a target image element exists in the live broadcast picture;

if the target image element exists in the live broadcast picture, saving, as a highlight segment, a segment of the live broadcast video having the live broadcast picture;

when a merging condition is satisfied, merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection; and

switching an output live broadcast video to the merged video of highlight collection;

wherein the identifying a live broadcast picture and determining if a target image element exists in the live broadcast picture specifically comprises:

autonomously identifying the live broadcast picture and autonomously determining if the target image element exists in the live broadcast picture, according to an image identification model established in advance, wherein the image identification model is established by training according to collected image features;

wherein the merging condition is autonomously determined according to the image identification model; and

wherein the merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection comprises:

decoding the highlight segment, and decoding the live broadcast video;

merging each video frame of the decoded highlight segment with each video frame of the decoded live broadcast video in a preset mode, to obtain merged video data; and

recoding the merged video data to obtain the merged video of highlight collection, such that the merged video of highlight collection displays the live broadcast video and the highlight segment simultaneously.

16. A non-transitory computer readable storage medium storing a computer program, wherein when being executed by a processor, the computer program implements a method for generating a live broadcast video of highlight collection, wherein, the method comprises:

identifying a live broadcast picture from a live broadcast video and determining if a target image element exists in the live broadcast picture;

if the target image element exists in the live broadcast picture, saving, as a highlight segment, a segment of the live broadcast video having the live broadcast picture;

when a merging condition is satisfied, merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection; and

switching an output live broadcast video to the merged video of highlight collection;

wherein the identifying a live broadcast picture and determining if a target image element exists in the live broadcast picture specifically comprises:

autonomously identifying the live broadcast picture and autonomously determining if the target image element exists in the live broadcast picture, according to an image identification model established in advance, wherein the image identification model is established by training according to collected image features;

wherein the merging condition is autonomously determined according to the image identification model; and

wherein the merging the highlight segment into the live broadcast video based on the merging condition to obtain a merged video of highlight collection comprises:

decoding the highlight segment, and decoding the live broadcast video;

merging each video frame of the decoded highlight segment with each video frame of the decoded live broadcast video in a preset mode, to obtain merged video data; and

recoding the merged video data to obtain the merged video of highlight collection, such that the merged video of highlight collection displays the live broadcast video and the highlight segment simultaneously.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2019
From: ZHENG, ZHENGUI; SUN, LEI; CHEN, XIANGXIANG
To: WANGSU SCIENCE & TECHNOLOGY CO., LTD.
Reel/Frame 050366/0752 →
Priority Claims (1)
CN 201910262673.7 · Apr 2, 2019 · national
Continuity (2)
Continuation PCTCN2019086049 · May 8, 2019
Related Publication 20200322647A1 · Oct 8, 2020
Cited By (1)
US 12,464,205