Granted Patent B2

US 12,518,110 · App. 17/588,025 · Granted Jan 6, 2026

Inventors: Xuming Lin (Hangzhou, CN); Zhongzhou Zhao (Hangzhou, CN); Ji Zhang (Hangzhou, CN); Liming Pu (Hangzhou, CN); Jiashuo Zhang (Hangzhou, CN)

Assignee: Alibaba Group Holding Limited

G06F40/40G06F40/166G06F40/205G10L13/08

Abstract

Claims (61)

1 . A method used in live broadcasting, the method comprising:

acquiring one or more pieces of source data related to an object to be presented in a live broadcast;

analyzing the source data to obtain one or more pieces of material information;

dividing the one or more pieces of material information into two parts according to functions, wherein a first part of the one or more pieces of material information is used for providing text paragraph information, and a second part of the one or more pieces of material information is used for creating a text output frame, the text output frame being used for determining an output sequence of the text paragraph information;

parsing the first part of the one or more pieces of material information to obtain one or more pieces of corresponding text paragraph information;

creating the text output frame according to at least one of respective occurrence frequencies or respective search popularities of attributes included in the second part of the one or more pieces of material information;

generating a text describing the object based at least in part on the text output frame and the second part of the one or more pieces of corresponding text paragraph information using an encoder-decoder model, priorities of the attributes in the text output frame being positively correlated with the at least one of the respective occurrence frequencies or the respective search popularities of the attributes, wherein the attributes include an appearance characteristic, a function characteristic, a user characteristic, and an application scenario characteristic of the object, and generating the text describing the object comprises:

inputting a name of the object, attribute information of the object, and the one or more pieces of corresponding text paragraph information into the encoder-decoder model to obtain the text; and

enabling the text to be broadcast to a user through a voice playback on a user interface of an electronic device of the user when the object is presented in the live broadcast and a playback button presented on the user interface is clicked by the user.

2 . The method according to claim 1 , further comprising:

selecting at least one piece of the one or more pieces of material information and at least one piece of the one or more pieces of corresponding text paragraph information;

converting the at least one piece of text paragraph information into voice information;

combining the at least one piece of material information and the voice information into demo data; and

presenting the demo data on the user interface.

3 . The method according to claim 1 , wherein generating the text describing the object using the second part of the one or more pieces of corresponding text paragraph information further comprises:

determining, based on the text output frame, an output sequence of the second part of the one or more pieces of corresponding text paragraph information to generate the text.

4 . The method according to claim 3 , further comprising:

outputting the text paragraph information according to the output sequence.

5 . The method according to claim 3 , further comprising:

adjusting the text output frame based on an external input.

6 . The method according to claim 1 , wherein analyzing the source data to obtain the one or more pieces of material information comprises:

analyzing the attributes of the source data.

7 . The method according to claim 6 , further comprising:

calculating a similarity level between materials having different attributes; and

combining attributes corresponding to materials having a similarity level greater than a threshold into one attribute.

8 . The method according to claim 1 , further comprising:

adjusting the text output frame based on at least one of an audio material or a video material.

9 . The method according to claim 1 , wherein the source data comprises data in multiple modalities.

10 . The method according to claim 9 , wherein the source data comprises pictures, texts, images, sound, and a combination thereof.

11 . The method according to claim 1 , wherein analyzing the source data to obtain the one or more pieces of material information comprises at least one of:

identifying text information in a picture to obtain first material information comprising the picture and the text information;

obtaining, based on a first text, second material information comprising a second text used for describing the object; or

analyzing audiovisual data to obtain third material information comprising at least one of video data, a speech recognition result of the audiovisual data, or audiovisual analysis data.

12 . The method according to claim 1 , wherein the material information comprises at least one of a text material, an audio material, a picture material, or a video material.

13 . The method according to claim 12 , wherein parsing the part part of the one or more pieces of material information to obtain the one or more pieces of corresponding text paragraph information comprises at least one of:

performing speech recognition on the audio material to obtain text paragraph information corresponding to the audio material; or

performing semantic understanding on at least one of the picture material or the video material to obtain text paragraph information corresponding to the at least one of the picture material or the video material.

14 . The method according to claim 1 , wherein the material information comprises:

unprocessed source data; and

processed data having undergone preset analysis processing.

15 . An apparatus used in live broadcasting, the apparatus comprising:

one or more processors; and

one or more memories storing thereon computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:

acquiring one or more pieces of source data related to an object to be presented in a live broadcast;

analyzing the source data to obtain one or more pieces of material information;

dividing the one or more pieces of material information into two parts according to functions, wherein a first part of the one or more pieces of material information is used for providing text paragraph information, and a second part of the one or more pieces of material information is used for creating a text output frame, the text output frame being used for determining an output sequence of the text paragraph information;

parsing the first part of the one or more pieces of material information to obtain one or more pieces of corresponding text paragraph information;

creating the text output frame according to at least one of respective occurrence frequencies or respective search popularities of attributes included in the second part of the one or more pieces of material information;

generating a text describing the object based at least in part on the text output frame and the second part of the one or more pieces of corresponding text paragraph information using an encoder-decoder model, priorities of the attributes in the text output frame being positively correlated with the at least one of the respective occurrence frequencies or the respective search popularities of the attributes, wherein the attributes include an appearance characteristic, a function characteristic, a user characteristic, and an application scenario characteristic of the object, and generating the text describing the object comprises:

inputting a name of the object, attribute information of the object, and the one or more pieces of corresponding text paragraph information into the encoder-decoder model to obtain the text; and

enabling the text to be broadcast through a voice playback on a user interface when the object is presented in the live broadcast.

16 . One or more memories storing thereon computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

acquiring one or more pieces of source data related to an object to be presented in a live broadcast;

analyzing the source data to obtain one or more pieces of material information;

dividing the one or more pieces of material information into two parts according to functions, wherein a first part of the one or more pieces of material information is used for providing text paragraph information, and a second part of the one or more pieces of material information is used for creating a text output frame, the text output frame being used for determining an output sequence of the text paragraph information;

parsing the first part of the one or more pieces of material information to obtain one or more pieces of corresponding text paragraph information;

creating the text output frame according to at least one of respective occurrence frequencies or respective search popularities of attributes included in the second part of the one or more pieces of material information;

generating a text describing the object based at least in part on the text output frame and the second part of the one or more pieces of corresponding text paragraph information using an encoder-decoder model, priorities of the attributes in the text output frame being positively correlated with the at least one of the respective occurrence frequencies or the respective search popularities of the attributes, wherein the attributes include an appearance characteristic, a function characteristic, a user characteristic, and an application scenario characteristic of the object, and generating the text describing the object comprises:

inputting a name of the object, attribute information of the object, and the one or more pieces of corresponding text paragraph information into the encoder-decoder model to obtain the text; and

enabling the text to be broadcast through a voice playback on a user interface when the object is presented in the live broadcast.

17 . The one or more memories according to claim 16 , wherein the source data comprises association relationships between contents of the live broadcast at different times and respective indicator data at the different times.

Assignments (1)

Continuity (1)

References Cited (28)