Granted Patent B2

US 12,373,486 · App. 17/703,564 · Granted Jul 29, 2025

Inventors: Zhihua Chang (Beijing, CN); Chengzhen Duan (Bejing, CN); Ranran Hou (Bejing, CN); Bin Qin (Bejing, CN)

Assignees: Beijing Xiaomi Mobile Software Co., Ltd.; Beijing Xiaomi Pinecone Electronics Co., Ltd.

G06F16/634G06F16/685G10L15/16

Abstract

Claims (67)

1. An audio recognition method, performed by an electronic device, comprising:

obtaining a query content, wherein the query content comprises segment information representing a to-be-recognized audio;

selecting a preset quantity of candidate audios corresponding to the query content from a preset library, wherein the candidate audio comprises a candidate audio segment matched with the segment information;

obtaining a to-be-detected vector corresponding to the candidate audio according to the segment information and the candidate audio segment;

inputting the to-be-detected vector corresponding to the candidate audio into a trained detection model, so as to obtain a detection result data output by the trained detection model; and

obtaining target segment information comprising the segment information and a target audio where the target segment information is located according to the detection result data;

wherein the detection result data comprise first probability data and second probability data which correspondingly indicate that a morpheme in the candidate audio segment is located at a starting position and an ending position respectively, and wherein:

obtaining the target segment information comprising the segment information and the target audio where the target segment information is located according to the detection result data comprises:

determining, in response to determining that the starting position is smaller than the ending position, a target audio segment from the candidate audio segment based on a product of the first probability data and the second probability data; and

using the target audio segment as the target segment information recognized from the query content, and using an audio where the target segment information is located as the target audio;

wherein determining the target audio segment from the candidate audio segment based on the product of the first probability data and the second probability data comprises:

determining a starting morpheme at the starting position and an ending morpheme at the ending position when the product of the first probability data and the second probability data is the largest; and

determining that all morphemes between the starting morpheme and the ending morpheme constitute the target audio segment.

2. The method according to claim 1 , wherein selecting the preset quantity of the candidate audios corresponding to the query content from the preset library comprises:

determining a similarity between a morpheme of the segment information and text information of an audio in the preset library;

sorting audios in the preset library according to the similarity from large to small so as to obtain a sorting result;

determining a preset quantity of audios with sorting positions at the front as the candidate audios based on the sorting result, the candidate audio comprising at least one audio segment matched with the morphemes of the segment information; and

obtaining an audio segment comprising the longest consecutive matching morpheme from the at least one audio segment of the candidate audio, so as to obtain the candidate audio segment, matched with the segment information, of the candidate audio.

3. The method according to claim 1 , wherein obtaining the to-be-detected vector corresponding to the candidate audio according to the segment information and the candidate audio segment comprises:

splicing the segment information with the candidate audio segment of the candidate audio respectively, so as to obtain the to-be-detected vector corresponding to the candidate audio, and

wherein the to-be-detected vector at least comprises a first identifier and a second identifier, the first identifier is configured to identify a starting position of the to-be-detected vector, and the second identifier is configured to identify a splicing position and an ending position of the to-be-detected vector.

4. The method according to claim 1 , wherein the to-be-recognized audio is a song, and the segment information refers to part of lyrics in the song.

5. An electronic device, comprising:

a processor; and

a memory configured to store computer instructions executable by the processor;

wherein, when the processor executes the instructions the processor is configured to:

obtain a query content, wherein the query content comprises segment information representing a to-be-recognized audio;

select a preset quantity of candidate audios corresponding to the query content from a preset library, wherein the candidate audio comprises a candidate audio segment matched with the segment information;

obtain a to-be-detected vector corresponding to the candidate audio according to the segment information and the candidate audio segment;

input the to-be-detected vector corresponding to the candidate audio into a trained detection model, so as to obtain a detection result data output by the trained detection model; and

obtain target segment information comprising the segment information and a target audio where the target segment information is located according to the detection result data;

wherein the detection result data comprise first probability data and second probability data which correspondingly indicate that a morpheme in the candidate audio segment is located at a starting position and an ending position respectively; and wherein the processor is further configured to:

determine, in response to determining that the starting position is smaller than the ending position, a target audio segment from the candidate audio segment based on a product of the first probability data and the second probability data; and

use the target audio segment as the target segment information recognized from the query content, and using an audio where the target segment information is located as the target audio;

wherein the processor is further configured to:

determine a starting morpheme at the starting position and an ending morpheme at the ending position when the product of the first probability data and the second probability data is the largest; and

determine that all morphemes between the starting morpheme and the ending morpheme constitute the target audio segment.

6. The electronic device according to claim 5 , wherein the processor is further configured to:

determine a similarity between a morpheme of the segment information and text information of an audio in the preset library;

sort audios in the preset library according to the similarity from large to small so as to obtain a sorting result;

determine a preset quantity of audios with sorting positions at the front as the candidate audios based on the sorting result, the candidate audio comprising at least one audio segment matched with the morphemes of the segment information; and

obtain an audio segment comprising the longest consecutive matching morpheme from the at least one audio segment of the candidate audio, so as to obtain the candidate audio segment, matched with the segment information, of the candidate audio.

7. The electronic device according to claim 5 , wherein the processor is further configured to:

splice the segment information with the candidate audio segment of the candidate audio respectively, so as to obtain the to-be-detected vector corresponding to the candidate audio, and

wherein the to-be-detected vector at least comprises a first identifier and a second identifier, and wherein the first identifier is configured to identify a starting position of the to-be-detected vector, and the second identifier is configured to identify a splicing position and an ending position of the to-be-detected vector.

8. The electronic device according to claim 5 , wherein the to-be-recognized audio is a song, and the segment information refers to part of lyrics in the song.

9. A non-transitory computer readable storage medium, wherein when executable computer programs in the storage medium are executed by a processor, the processor is configured to:

obtain a query content, wherein the query content comprises segment information representing a to-be-recognized audio;

select a preset quantity of candidate audios corresponding to the query content from a preset library, wherein the candidate audio comprises a candidate audio segment matched with the segment information;

obtain a to-be-detected vector corresponding to the candidate audio according to the segment information and the candidate audio segment;

input the to-be-detected vector corresponding to the candidate audio into a trained detection model, so as to obtain a detection result data output by the trained detection model; and

obtain target segment information comprising the segment information and a target audio where the target segment information is located according to the detection result data;

wherein the detection result data comprise first probability data and second probability data which correspondingly indicate that a morpheme in the candidate audio segment is located at a starting position and an ending position respectively; and

when executable computer programs in the storage medium are executed by the processor, the processor is configured to:

determine, in response to determining that the starting position is smaller than the ending position, a target audio segment from the candidate audio segment based on a product of the first probability data and the second probability data; and

use the target audio segment as the target segment information recognized from the query content, and using an audio where the target segment information is located as the target audio;

wherein when executable computer programs in the storage medium are executed by the processor, the processor is configured to:

determine a starting morpheme at the starting position and an ending morpheme at the ending position when the product of the first probability data and the second probability data is the largest; and

determine that all morphemes between the starting morpheme and the ending morpheme constitute the target audio segment.

10. The non-transitory computer readable storage medium according to claim 9 , wherein when executable computer programs in the storage medium are executed by the processor, the processor is configured to:

determine a similarity between a morpheme of the segment information and text information of an audio in the preset library;

sort audios in the preset library according to the similarity from large to small so as to obtain a sorting result;

determine a preset quantity of audios with sorting positions at the front as the candidate audios based on the sorting result, the candidate audio comprising at least one audio segment matched with the morphemes of the segment information; and

obtain an audio segment comprising the longest consecutive matching morpheme from the at least one audio segment of the candidate audio, so as to obtain the candidate audio segment, matched with the segment information, of the candidate audio.

11. The non-transitory computer readable storage medium according to claim 9 , wherein when executable computer programs in the storage medium are executed by the processor, the processor is configured to:

splice the segment information with the candidate audio segment of the candidate audio respectively, so as to obtain the to-be-detected vector corresponding to the candidate audio, and

wherein the to-be-detected vector at least comprises a first identifier and a second identifier, the first identifier is configured to identify a starting position of the to-be-detected vector, and the second identifier is configured to identify a splicing position and an ending position of the to-be-detected vector.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (37)