Granted Patent B2

US 12,469,501 · App. 18/202,930 · Granted Nov 11, 2025

Inventors: Yuan Gao (Beijing, CN); Shuai Liu (Beijing, CN); Bin Wang (Shenzhen, CN); Zhe Wang (Beijing, CN); Tianshu Qu (Beijing, CN); Jiahao Xu (Beijing, CN)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

G10L19/008

Abstract

Claims (100)

1 . A method of audio encoding performed by an audio encoder, comprising:

selecting a first target virtual speaker from a preset virtual speaker set based on a first scene audio signal;

generating a first virtual speaker signal based on the first scene audio signal and attribute information of the first target virtual speaker;

obtaining a second scene audio signal using the attribute information of the first target virtual speaker and the first virtual speaker signal;

generating a residual signal based on the first scene audio signal and the second scene audio signal; and

encoding the first virtual speaker signal and the residual signal, to produce encoded signals, and writing the encoded signals into a bitstream;

wherein

the first scene audio signal comprises a higher order ambisonics (HOA) signal to be encoded, and the attribute information of the first target virtual speaker comprises location information of the first target virtual speaker; and

generating the first virtual speaker signal comprises:

obtaining an HOA coefficient for the first target virtual speaker based on the location information of the first target virtual speaker; and

performing linear combination on the HOA signal to be encoded and the HOA coefficient for the first target virtual speaker to obtain the first virtual speaker signal.

2 . The method according to claim 1 , wherein

the method further comprises:

obtaining a major sound field component from the first scene audio signal based on the preset virtual speaker set; and

selecting the first target virtual speaker from the preset virtual speaker set comprises:

selecting the first target virtual speaker from the virtual speaker set based on the major sound field component.

3 . The method according to claim 1 , further comprising:

encoding the attribute information of the first target virtual speaker, and writing the encoded attribute information into the bitstream.

4 . The method according to claim 1 , wherein

the method further comprises:

selecting a second target virtual speaker from the preset virtual speaker set based on the first scene audio signal; and

generating a second virtual speaker signal based on the first scene audio signal and attribute information of the second target virtual speaker; and

encoding the first virtual speaker signal and the residual signal comprises:

obtaining a downmixed signal and first side information based on the first virtual speaker signal and the second virtual speaker signal, wherein the first side information indicates a relationship between the first virtual speaker signal and the second virtual speaker signal; and

encoding the downmixed signal, the first side information, and the residual signal.

5 . The method according to claim 1 , wherein

the residual signal comprises residual sub-signals on at least two sound channels, and

the method further comprises:

determining, from the residual sub-signals on the at least two sound channels based on configuration information of an audio encoder or signal class information of the first scene audio signal, a residual sub-signal that needs to be encoded and that is on at least one sound channel; and

encoding the first virtual speaker signal and the residual signal comprises:

encoding the first virtual speaker signal and the residual sub-signal that needs to be encoded and that is on the at least one sound channel.

6 . The method according to claim 5 , further comprising:

when the residual sub-signals on the at least two sound channels comprise a residual sub-signal that does not need to be encoded and that is on at least one sound channel,

obtaining second side information that indicates a relationship between the residual sub-signal that needs to be encoded and that is on the at least one sound channel and the residual sub-signal that does not need to be encoded and that is on the at least one sound channel; and

writing the second side information into the bitstream.

7 . A method of audio decoding performed by an audio decoder, comprising:

receiving a bitstream;

decoding the bitstream to obtain a virtual speaker signal and a residual signal;

decoding the bitstream to obtain attribute information of a target virtual speaker; and

obtaining a reconstructed scene audio signal based on the attribute information of the target virtual speaker, the residual signal, and the virtual speaker signal;

wherein

the attribute information of the target virtual speaker comprises location information of the target virtual speaker; and

obtaining the reconstructed scene audio signal comprises:

determining a higher order ambisonics (HOA) coefficient for the target virtual speaker based on the location information of the target virtual speaker;

performing synthesis processing on the virtual speaker signal and the HOA coefficient for the target virtual speaker to obtain a synthesized scene audio signal; and

adjusting the synthesized scene audio signal using the residual signal to obtain the reconstructed scene audio signal.

8 . The method according to claim 7 , wherein

the virtual speaker signal is a downmixed signal obtained by downmixing a first virtual speaker signal and a second virtual speaker signal;

the method further comprises:

decoding the bitstream to obtain first side information, wherein the first side information indicates a relationship between the first virtual speaker signal and the second virtual speaker signal; and

obtaining the first virtual speaker signal and the second virtual speaker signal based on the first side information and the downmixed signal; and

obtaining the reconstructed scene audio signal comprises:

obtaining the reconstructed scene audio signal based on the attribute information of the target virtual speaker, the residual signal, the first virtual speaker signal, and the second virtual speaker signal.

9 . The method according to claim 7 , wherein

the residual signal comprises a residual sub-signal on a first sound channel;

the method further comprises:

decoding the bitstream to obtain second side information, wherein the second side information indicates a relationship between the residual sub-signal on the first sound channel and a residual sub-signal on a second sound channel; and

obtaining the residual sub-signal on the second sound channel based on the second side information and the residual sub-signal on the first sound channel; and

obtaining the reconstructed scene audio signal comprises:

obtaining the reconstructed scene audio signal based on the attribute information of the target virtual speaker, the residual sub-signal on the first sound channel, the residual sub-signal on the second sound channel, and the virtual speaker signal.

10 . The method according to claim 7 , wherein

the residual signal comprises a residual sub-signal on a first sound channel;

the method further comprises:

decoding the bitstream to obtain second side information, wherein the second side information indicates a relationship between the residual sub-signal on the first sound channel and a residual sub-signal on a third sound channel; and

obtaining the residual sub-signal on the third sound channel and an updated residual sub-signal on the first sound channel based on the second side information and the residual sub-signal on the first sound channel; and

obtaining the reconstructed scene audio signal comprises:

obtaining the reconstructed scene audio signal based on the attribute information of the target virtual speaker, the updated residual sub-signal on the first sound channel, the residual sub-signal on the third sound channel, and the virtual speaker signal.

11 . An audio encoder apparatus, comprising:

at least one processor coupled to a memory storing instructions, which when executed by the at least one processor, cause the audio encoder to perform operations, the operations comprising:

selecting a first target virtual speaker from a preset virtual speaker set based on a first scene audio signal;

generating a first virtual speaker signal based on the first scene audio signal and attribute information of the first target virtual speaker;

obtaining a second scene audio signal using the attribute information of the first target virtual speaker and the first virtual speaker signal;

generating a residual signal based on the first scene audio signal and the second scene audio signal; and

encoding the first virtual speaker signal and the residual signal, to produce encoded signals, and writing the encoded signals into a bitstream;

wherein

the first scene audio signal comprises a higher order ambisonics (HOA) signal to be encoded, and the attribute information of the first target virtual speaker comprises location information of the first target virtual speaker; and

generating the first virtual speaker signal comprises:

obtaining an HOA coefficient for the first target virtual speaker based on the location information of the first target virtual speaker; and

performing linear combination on the HOA signal to be encoded and the HOA coefficient for the first target virtual speaker to obtain the first virtual speaker signal.

12 . The audio encoder according to claim 11 , wherein the operations further comprise:

encoding the attribute information of the first target virtual speaker, and writing the encoded attribute information into the bitstream.

13 . The audio encoder according to claim 11 , wherein

the operations further comprise:

selecting a second target virtual speaker from the preset virtual speaker set based on the first scene audio signal; and

generating a second virtual speaker signal based on the first scene audio signal and attribute information of the second target virtual speaker; and

encoding the first virtual speaker signal and the residual signal comprises:

obtaining a downmixed signal and first side information based on the first virtual speaker signal and the second virtual speaker signal, wherein the first side information indicates a relationship between the first virtual speaker signal and the second virtual speaker signal; and

encoding the downmixed signal, the first side information, and the residual signal.

14 . An audio decoder, comprising:

at least one processor coupled to a memory storing instructions, which when executed by the at least one processor, cause the audio decoder to perform operations, the operations comprising:

receiving a bitstream;

decoding the bitstream to obtain a virtual speaker signal and a residual signal;

decoding the bitstream to obtain attribute information of a target virtual speaker; and

obtaining a reconstructed scene audio signal based on the attribute information of the target virtual speaker, the residual signal, and the virtual speaker signal;

wherein

the attribute information of the target virtual speaker comprises location information of the target virtual speaker; and

obtaining the reconstructed scene audio signal comprises:

determining a higher order ambisonics (HOA) coefficient for the target virtual speaker based on the location information of the target virtual speaker;

performing synthesis processing on the virtual speaker signal and the HOA coefficient for the target virtual speaker to obtain a synthesized scene audio signal; and

adjusting the synthesized scene audio signal using the residual signal to obtain the reconstructed scene audio signal.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (17)