ASR training and adaptation
AM and LM parameters to be used for adapting an ASR model are derived for each audio segment of an audio stream comprising multiple audio programs. A set of identifiers, including a speaker identifier, a speaker domain identifier and a program domain identifier, is obtained for each audio segment. The set of identifiers are used to select most suitable AM and LM parameters for the particular audio segment. The embodiments enable provision of maximum constraints on the AMs and LMs and enable adaptation of the ASR model on the fly for audio streams of multiple audio programs, such as broadcast audio. This means that the embodiments enable selecting AM and LM parameters that are most suitable in terms of ASR performance for each audio segment.
1. An audio processing method for automatic speech recognition, ASR, said method comprising:
for each audio segment of multiple audio segments in an audio stream comprising audio data of multiple audio programs, each audio segment comprising speech of a single speaker:
obtaining a speaker identifier of a speaker of said audio segment;
determining a program domain identifier based on a media description, wherein the media description could be any information or data element comprising information and metadata of the audio program; and
associating said speaker identifier, said program domain identifier and a speaker domain identifier with said audio segment to enable generation of ASR adaptation parameters based on said speaker identifier, said program domain identifier and said speaker domain identifier,
wherein said speaker domain identifier for said audio segment is based on information or metadata associated with an audio program of said multiple audio programs and said audio segment comprises audio data of said audio program.
2. The method according to claim 1 , further comprising retrieving, based on said speaker identifier, acoustic mode (AM) parameters from a database storing speaker specific AM parameters for different speakers with a respective speaker identifier.
3. The method according to claim 2 , wherein retrieving said AM parameters comprises:
retrieving, from said database, speaker specific AM parameters associated with said speaker identifier responsive to when said database comprises speaker specific AM parameters associated with said speaker identifier; and
otherwise selecting generic AM parameters.
4. The method according to claim 3 , further comprises:
retrieving, for each audio segment of said multiple audio segments and from said database, speaker specific AM parameters associated with said speaker identifier responsive to when said database comprises speaker specific AM parameters associated with said speaker identifier; and
selecting, for each audio segment of said multiple audio segments, generic AM parameters responsive to when said database does not comprise said speaker specific AM parameters associated with said speaker identifier.
5. The method according to claim 1 , further comprises:
selecting language model (LM) parameters based on comparison of the speaker domain identifier and the program domain identifier;
adapting, for each audio segment of said multiple audio segments, said ASR model based on said selected LM parameters to form an adapted ASR model; and
transcribing, for each audio segment of said multiple audio segments, said audio segment in said adapted ASR model into a speech transcript.
6. A device configured for audio processing for automatic speech recognition, ASR, the device comprising:
at least one processor; and
at least one memory storing program instructions executable by the at least one processor to operate to:
for each audio segment of multiple audio segments in an audio stream comprising audio data of multiple audio programs, each audio segment comprising speech of a single speaker:
obtaining a speaker identifier of a speaker of said audio segment;
determining a program domain identifier based on a media description, wherein the media description could be any information or data element comprising information and metadata of the audio program; and
associating said speaker identifier, said program domain identifier and a speaker domain identifier with said audio segment to enable generation of ASR adaptation parameters based on said speaker identifier, said program domain identifier and said speaker domain identifier,
wherein said speaker domain identifier for said audio segment is based on information or metadata associated with an audio program of said multiple audio programs and said audio segment comprises audio data of said audio program.
7. The device according to claim 6 , wherein the instructions executable by the at least one processor further operate to:
retrieve, for each audio segment of said multiple audio segments and from said database and responsive to when said speaker domain identifier is equal to said program domain identifier of said audio program, speaker specific language model (LM) parameters associated with said speaker identifier responsive to when a database comprises said speaker specific LM parameters;
retrieve, for each audio segment of said multiple audio segments and responsive to when said speaker domain identifier is equal to said program domain identifier of said audio program, domain specific LM parameters associated with said speaker domain identifier responsive to when said database does not comprise said speaker specific LM parameters; and
select, for each audio segment of said multiple audio segments and responsive to when said speaker domain identifier is not equal to said program domain identifier of said audio program, generic LM parameters.
8. The device according to claim 6 , wherein the instructions executable by the at least one processor further operate to select, for each audio segment of said multiple audio segments, acoustic mode (AM) parameters based on said speaker identifier.
9. The device according to claim 6 , wherein the instructions executable by the at least one processor further operate to retrieve, for each audio segment of said multiple audio segments and based on said speaker identifier, acoustic mode (AM) parameters from a database storing speaker specific AM parameters for different speakers with a respective speaker identifier.
10. The device according to claim 9 , wherein the instructions executable by the at least one processor further operate to:
retrieve, for each audio segment of said multiple audio segments and from said database, speaker specific AM parameters associated with said speaker identifier responsive to when said database comprises speaker specific AM parameters associated with said speaker identifier; and
select, for each audio segment of said multiple audio segments, generic AM parameters responsive to when said database does not comprise said speaker specific AM parameters associated with said speaker identifier.
11. The device according to claim 6 , wherein the instructions executable by the at least one processor further operate to:
select language model (LM) parameters based on comparison of the speaker domain identifier and the program domain identifier;
adapt, for each audio segment of said multiple audio segments, said ASR model based on said selected LM parameters and said selected AM parameters to form an adapted ASR model; and
transcribe, for each audio segment of said multiple audio segments, said audio segment in said adapted ASR model into a speech transcript.
12. A computer program product for automatic speech recognition, ASR, the computer program product comprising a non-transitory computer readable medium storing instructions, which when executed by at least one processor, cause the at least one processor to:
for each audio segment of multiple audio segments in an audio stream comprising audio data of multiple audio programs, each audio segment comprising speech of a single speaker:
obtain a speaker identifier of a speaker of said audio segment;
determine a program domain identifier based on a media description, wherein the media description could be any information or data element comprising information and metadata of the audio program; and
associate said speaker identifier, said program domain identifier and a speaker domain identifier with said audio segment to enable generation of ASR adaptation parameters based on said speaker identifier, said program domain identifier and said speaker domain identifier,
wherein said speaker domain identifier for said audio segment is based on information or metadata associated with an audio program of said multiple audio programs and said audio segment comprises audio data of said audio program.
13. The computer program product according to claim 12 , wherein the instructions executable by the at least one processor further operate to:
retrieve, based on said speaker identifier, acoustic mode (AM) parameters from a database storing speaker specific AM parameters for different speakers with a respective speaker identifier.
14. The computer program product according to claim 13 , wherein the instruction executable by the at least one processor to retrieve said AM parameters comprises:
retrieve, from said database, speaker specific AM parameters associated with said speaker identifier responsive to when said database comprises speaker specific AM parameters associated with said speaker identifier; and
otherwise select generic AM parameters.
15. The computer program product according to claim 14 , wherein the instructions executable by the at least one processor further operate to:
retrieve, for each audio segment of said multiple audio segments and from said database, speaker specific AM parameters associated with said speaker identifier responsive to when said database comprises speaker specific AM parameters associated with said speaker identifier; and
select, for each audio segment of said multiple audio segments, generic AM parameters responsive to when said database does not comprise said speaker specific AM parameters associated with said speaker identifier.
16. The computer program product according to claim 12 , wherein the instructions executable by the at least one processor further operate to:
select language model (LM) parameters based on comparison of the speaker domain identifier and the program domain identifier;
adapt, for each audio segment of said multiple audio segments, said ASR model based on said selected LM parameters to form an adapted ASR model; and
transcribe, for each audio segment of said multiple audio segments, said audio segment in said adapted ASR model into a speech transcript.
17. The computer program product according to claim 12 , wherein the instructions executable by the at least one processor further operate to:
retrieve, for each audio segment of said multiple audio segments and from said database and responsive to when said speaker domain identifier is equal to said program domain identifier of said audio program, speaker specific language model (LM) parameters associated with said speaker identifier responsive to when a database comprises said speaker specific LM parameters;
retrieve, for each audio segment of said multiple audio segments and responsive to when said speaker domain identifier is equal to said program domain identifier of said audio program, domain specific LM parameters associated with said speaker domain identifier responsive to when said database does not comprise said speaker specific LM parameters; and
select, for each audio segment of said multiple audio segments and responsive to when said speaker domain identifier is not equal to said program domain identifier of said audio program, generic LM parameters.
18. The computer program product according to claim 12 , wherein the instructions executable by the at least one processor further operate to select, for each audio segment of said multiple audio segments, acoustic mode (AM) parameters based on said speaker identifier.
19. The computer program product according to claim 12 , wherein the instructions executable by the at least one processor further operate to retrieve, for each audio segment of said multiple audio segments and based on said speaker identifier, acoustic mode (AM) parameters from a database storing speaker specific AM parameters for different speakers with a respective speaker identifier.
20. The computer program product according to claim 19 , wherein the instructions executable by the at least one processor further operate to:
retrieve, for each audio segment of said multiple audio segments and from said database, speaker specific AM parameters associated with said speaker identifier responsive to when said database comprises speaker specific AM parameters associated with said speaker identifier; and
select, for each audio segment of said multiple audio segments, generic AM parameters responsive to when said database does not comprise said speaker specific AM parameters associated with said speaker identifier.
21. The computer program product according to claim 12 , wherein the instructions executable by the at least one processor further operate to:
select language model (LM) parameters based on comparison of the speaker domain identifier and the program domain identifier;
adapt, for each audio segment of said multiple audio segments, said ASR model based on said selected LM parameters and said selected AM parameters to form an adapted ASR model; and
transcribe, for each audio segment of said multiple audio segments, said audio segment in said adapted ASR model into a speech transcript.