Granted Patent B2

US 12,200,464 · App. 17/728,037 · Granted Jan 14, 2025

Inventors: Tammy Lee (Suwon-si, KR); Sangchul Ko (Suwon-si, KR); Kyungrae Kim (Suwon-si, KR); Sunmin Kim (Suwon-si, KR); Jungkyu Kim (Suwon-si, KR); Woohyun Nam (Suwon-si, KR); Yoonjae Son (Suwon-si, KR); Hyunkwon Chung (Suwon-si, KR); Sunghee Hwang (Suwon-si, KR)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

H04S3/008G10L19/008H04S2400/01H04S2400/03

Abstract

Claims (73)

1. An audio processing method, comprising:

generating a second audio signal by down-mixing at least one first audio signal;

generating first information related to error removal for the at least one first audio signal, using at least one of an original signal power of the at least one first audio signal or a second signal power of the at least one first audio signal after decoding;

generating an audio signal of a low frequency effect (LFE) channel for the first information related to error removal, the first information comprising at least one of speech norm information, information about an error removal factor, and on-screen object information; and

transmitting the down-mixed second audio signal and the audio signal of the LFE channel.

2. The audio processing method of claim 1 , wherein the first information related to the error removal for the at least one first audio signal comprises second information about a factor for the error removal, and

wherein the generating of the first information related to the error removal for the at least one first audio signal comprises, when the original signal power of the at least one first audio signal is less than or equal to a first value, generating the second information about the factor for the error removal, the second information indicating that a value of the factor for the error removal is 0.

3. The audio processing method of claim 1 , wherein the first information related to the error removal for the at least one first audio signal comprises second information about a factor for the error removal, and

wherein the generating of the first information related to the error removal for the at least one first audio signal comprises, when a first ratio of the original signal power of the at least one first audio signal to an original signal power of the second audio signal is less than a second value, generating the second information about the factor for the error removal, based on the original signal power of the at least one first audio signal and the second signal power of the at least one first audio signal after decoding.

4. The audio processing method of claim 3 , wherein the generating of the second information about the factor for the error removal comprises generating the second information about the factor for the error removal, the second information indicating that a value of the factor for the error removal is a second ratio of the original signal power of the at least one first audio signal to the second signal power of the at least one first audio signal after decoding.

5. The audio processing method of claim 4 , wherein the generating of the second information about the factor for the error removal comprises, when the second ratio of the original signal power of the at least one first audio signal to the second signal power of the at least one first audio signal after decoding is greater than 1, generating the second information about the factor for the error removal, the second information indicating that the value of the factor for the error removal is 1.

6. The audio processing method of claim 1 , wherein the first information related to the error removal for the at least one first audio signal comprises second information about a factor for the error removal, and

wherein the generating of the first information related to the error removal for the at least one first audio signal comprises, when a ratio of the original signal power of the at least one first audio signal to the original signal power of the second audio signal is greater than or equal to a second value, generating the second information about the factor for the error removal, the second information indicating that a value of the factor for the error removal is 1.

7. The audio processing method of claim 6 , wherein the generating of the second information about the factor for the error removal comprises:

generating, for each frame of the second audio signal, the first information related to the error removal for the at least one first audio signal.

8. The audio processing method of claim 1 , wherein the down-mixed second audio signal comprises a third audio signal of a base channel group and a fourth audio signal of a dependent channel group,

wherein the fourth audio signal of the dependent channel group comprises a fifth audio signal of a first dependent channel comprising a sixth audio signal of an independent channel included in a first three-dimensional (3D) audio channel in front of a listener, and

wherein a seventh audio signal of a second 3D audio channel on a side and a back of the listener has been obtained by mixing the fifth audio signal of the first dependent channel.

9. The audio processing method of claim 8 , wherein the third audio signal of the base channel group comprises an eighth audio signal of a second channel and a ninth audio signal of a third channel,

wherein the eighth audio signal of the second channel has been generated by mixing a tenth audio signal of a left stereo channel with a decoded audio signal of a center channel in front of the listener, and

wherein the ninth audio signal of the third channel has been generated by mixing an eleventh audio signal of a right stereo channel with the decoded audio signal of the center channel in front of the listener.

10. The audio processing method of claim 1 , wherein the down-mixed second audio signal comprises a third audio signal of a base channel group and a fourth audio signal of a dependent channel group,

wherein the third audio signal of the base channel group comprises a fifth audio signal of a stereo channel,

wherein the transmitting of the first information related to the error removal for the at least one first audio signal and the down-mixed second audio signal comprises:

generating a bitstream including the first information related to the error removal for the at least one first audio signal and second information about the down-mixed second audio signal, and

transmitting the bitstream, and

wherein the generating of the bitstream comprises:

generating a base channel audio stream comprising a compressed fifth audio signal of the stereo channel, and

generating a plurality of dependent channel audio streams comprising a plurality of audio signals of a plurality of dependent channel groups, and

wherein the plurality of dependent channel audio streams comprise a first dependent channel audio stream and a second dependent channel audio stream, and

wherein, when for a first multi-channel audio signal used to generate the base channel audio stream and the first dependent channel audio stream, a first number of surround channels is S n-1 , a second number of subwoofer channels is W n-1 , and a third number of height channels is H n-1 , and for a second multi-channel audio signal used to generate the first dependent channel audio stream and the second dependent channel audio stream, a fourth number of surround channels is S n , a fifth number of subwoofer channels is W n , and a sixth number of height channels is H n ,

S n-1 is less than or equal to S n , W n-1 is less than or equal to W n , and H n-1 is less than or equal to H n , but all of S n-1 , W n-1 , and H n-1 are not equal to S n , W n , and H n , respectively.

11. An audio processing method, comprising:

obtaining, from a bitstream, a second audio signal down-mixed from at least one first audio signal;

obtaining, from the bitstream, an audio signal of a low frequency effect (LFE) channel;

obtaining first information related to error removal for the at least one first audio signal for the obtained audio signal of the LFE channel, the first information comprising at least one of speech norm information, information about an error removal factor, and on-screen object information;

de-mixing the at least one first audio signal from the down-mixed second audio signal; and

reconstructing the at least one first audio signal by mixing the first information related to the error removal for the at least one first audio signal to the de-mixed at least one first audio signal,

wherein the first information related to the error removal for the at least one first audio signal has been generated using at least one of an original signal power of the at least one first audio signal or a second signal power of the at least one first audio signal after decoding.

12. The audio processing method of claim 11 , wherein the first information related to the error removal for the at least one first audio signal comprises second information about a factor for the error removal, and

wherein the factor for the error removal is greater than or equal to 0 and is less than or equal to 1.

13. The audio processing method of claim 11 , wherein the reconstructing of the at least one first audio signal comprises reconstructing the at least one first audio signal to have a third signal power equal to a product of a fourth signal power of the de-mixed at least one first audio signal and a factor for the error removal.

14. The audio processing method of claim 11 , wherein the bitstream comprises second information about a third audio signal of a base channel group and third information about a fourth audio signal of a dependent channel group,

wherein the third audio signal of the base channel group has been obtained by decoding the second information about the third audio signal of the base channel group, included in the bitstream, without being de-mixed with another audio signal of another channel group, and

wherein the audio processing method further comprises reconstructing, using the fourth audio signal of the dependent channel group, a fifth audio signal of an up-mixed channel group comprising at least one up-mixed channel through de-mixing with the third audio signal of the base channel group.

15. The audio processing method of claim 14 , wherein the fourth audio signal of the dependent channel group comprises a first dependent channel audio signal and a second dependent channel audio signal,

wherein the first dependent channel audio signal comprises a sixth audio signal of an independent channel in front of a listener, and

wherein the second dependent channel audio signal comprises a mixed audio signal of audio signals of channels on a side and a back of the listener.

16. The audio processing method of claim 15 , wherein the base channel group comprises a mono channel or a stereo channel, and

wherein the at least one up-mixed channel is a discrete audio channel that is at least one channel except for a channel of the base channel group among a 3D audio channel in front of the listener or a 3D audio channel located omnidirectionally around the listener.

17. The audio processing method of claim 16 , wherein the 3D audio channel in front of the listener is a 3.1.2 channel,

wherein the 3.1.2 channel comprises three surround channels in front of the listener, one subwoofer channel in front of the listener, and two height channels,

wherein the 3D audio channel located omnidirectionally around the listener comprises at least one of a 5.1.2 channel or a 7.1.4 channel,

wherein the 5.1.2 channel comprises three surround channels in front of the listener, two surround channels on a side and a back of the listener, one subwoofer channel in front of the listener, and two height channels in front of the listener, and

wherein the 7.1.4 channel comprises three surround channels in front of the listener, four surround channels on the side and the back of the listener, one subwoofer channel in front of the listener, two height channels in front of the listener, and two height channels on the side and the back of the listener.

18. The audio processing method of claim 14 , wherein the third audio signal of the base channel group comprises sixth audio signal of a first channel and a seventh audio signal of a second channel, and

wherein the sixth audio signal of the first channel has been generated by mixing an eighth audio signal of a left stereo channel and a decoded audio signal of a center channel in front of a listener, and the seventh audio signal of the second channel has been generated by mixing a ninth audio signal of a right stereo channel and a compressed and decompressed audio signal of the center channel in front of the listener.

19. The audio processing method of claim 14 , wherein the de-mixed at least one first audio signal comprises a sixth audio signal of the at least one up-mixed channel and a seventh audio signal of an independent channel, and

wherein the seventh audio signal of the independent channel comprises a first portion of the third audio signal of the base channel group and a second portion of the fourth audio signal of the dependent channel group.

20. An audio processing apparatus, comprising:

a memory storing one or more instructions; and

at least one processor communicatively coupled to the memory, and configured to execute the one or more instructions to:

obtain, from a bitstream, a second audio signal down-mixed from at least one first audio signal,

obtain, from the bitstream, an audio signal of a low frequency effect (LFE) channel,

obtain information related to error removal for the at least one first audio signal for the obtained audio signal of the LFE channel, the first information comprising at least one of speech norm information, information about an error removal factor, and on-screen object information,

de-mix the at least one first audio signal from the down-mixed second audio signal, and

reconstruct the at least one first audio signal by applying the information related to the error removal for the at least one first audio signal to the at least one first audio signal de-mixed from the second audio signal, and

wherein the information related to the error removal for the at least one first audio signal has been generated using at least one of an original signal power of the at least one first audio signal or a second signal power of the at least one first audio signal after decoding.

21. An audio processing method, comprising:

generating a second audio signal by down-mixing at least one first audio signal;

generating first information related to error removal for the at least one first audio signal, using at least one of an original signal power of the at least one first audio signal or a second signal power of the at least one first audio signal after decoding;

generating an output signal based on an audio signal of a low frequency effect (LFE) channel and the first information related to the error removal for the at least one first audio signal, the first information comprising at least one of speech norm information, information about an error removal factor, and on-screen object information; and

transmitting the output signal and the down-mixed second audio signal.

Assignments (1)

Priority Claims (5)

Continuity (2)

References Cited (204)