Electronic device for controlling beamforming and operating method thereof
An electronic device is provided. The electronic device includes, for the purpose of determining a customized beamformer filter, an input module including a plurality of microphones configured to receive an external sound signal, a memory configured to store computer-executable instructions and an initial value of a voice parameter used to perform beamforming on the external sound signal, and a processor configured to execute the instructions by accessing the memory. The instructions may be configured to estimate a feature value of the external sound signal, calculate the initial value of the voice parameter used to perform beamforming based on the external sound signal received by the plurality of microphones, determine whether to store the calculated initial value according to the feature value, determine which one of the calculated initial value or an initial value stored in the memory used according to the feature value, and obtain a target voice parameter.
1. An electronic device, comprising:
a plurality of microphones;
memory storing computer-executable instructions; and
at least one processor including processing circuitry,
wherein the instructions, when executed by the at least one processor, cause the electronic device to:
estimate a feature value of an external sound signal received by the plurality of microphones,
calculate an initial value of a voice parameter used to perform beamforming based on the external sound signal,
in response to the feature value being less than or equal to a first threshold value and exceeding a second threshold value, obtain a first target of the voice parameter based on the calculated initial value,
in response to the feature value being less than or equal to the second threshold value and exceeding a third threshold value, load an initial value of the voice parameter from the memory and obtain a second target of the voice parameter by updating the loaded initial value, and
in response to the feature value being less than or equal to the third threshold value, determine the loaded initial value as a third target of the voice parameter.
2. The electronic device of claim 1 , wherein, when executed by the at least one processor, cause the electronic device to:
determine a filter to be used to perform beamforming on the external sound signal based on the first target, the second target, or the third target.
3. The electronic device of claim 2 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
estimate a magnitude of residual noise with respect to a signal on which beamforming is performed using the filter, and
perform noise processing on the signal on which beamforming is performed according to the estimated magnitude of the residual noise.
4. The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
in response to the feature value exceeding the first threshold value, store the calculated initial value of the voice parameter in the memory; and
obtain a fourth target of the voice parameter by updating the calculated initial value.
5. The electronic device of claim 1 , wherein the instructions, when executed by the at least one processor, cause the electronic device to:
output a guide interface configured to ask a user to perform an utterance to calculate an initial value of the voice parameter to be stored in the memory;
calculate the initial value of the voice parameter based on the user utterance received by the plurality of microphones; and
store the calculated initial value of the voice parameter in the memory.
6. The electronic device of claim 1 ,
wherein an initial value of the voice parameter stored in the memory is classified according to a magnitude of a sound signal, and
wherein the instructions, when executed by the at least one processor, cause the electronic device to:
calculate a magnitude of the external sound signal, and
load the initial value from the memory according to the magnitude of the external sound signal in response to the feature value being less than or equal to the second threshold value.
7. The electronic device of claim 1 ,
wherein an initial value of the voice parameter stored in the memory is classified according to a direction of a sound signal, and
wherein the instructions, when executed by the at least one processor, cause the electronic device to:
determine a direction of the external sound signal, load the initial value from the memory according to the direction of the external sound signal in response to the feature value being less than or equal to the second threshold value.
8. The electronic device of claim 1 ,
wherein the feature value is one of a signal-to-noise ratio (SNR) value or noise power, and
wherein the voice parameter is a voice covariance matrix.
9. The electronic device of claim 1 ,
wherein the plurality of microphones comprises an external microphone placed on one side of the electronic device, and
wherein the electronic device further comprises an in-ear microphone and an accelerator.
10. The electronic device of claim 1 , wherein the electronic device is one of a true-wireless stereo (TWS) earphone, headphones, or a headset.
11. A method performed by an electronic device of obtaining a voice parameter used to perform beamforming, the method comprising:
estimating a feature value of an external sound signal received by a plurality of microphones;
calculating an initial value of a voice parameter used to perform beamforming on the external sound signal;
in response to the feature value being less than or equal to a first threshold value and exceeding a second threshold value, obtaining a first target of the voice parameter based on the calculated initial value,
in response to the feature value being less than or equal to the second threshold value and exceeding a third threshold value, loading an initial value of the voice parameter from a memory of the electronic device and obtaining a second target of the voice parameter by updating the loaded initial value, and
in response to the feature value being less than or equal to the third threshold value, determining the loaded initial value as a third target of the voice parameter.
12. The method of claim 11 , further comprising:
storing the calculated initial value of the voice parameter in response to the feature value exceeding the first threshold value; and
obtaining a fourth target of the voice parameter by updating the calculated initial value.
13. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of an electronic device, cause the electronic device to perform operations, the operations comprising:
estimating a feature value of an external sound signal received by a plurality of microphones;
calculating an initial value of a voice parameter used to perform beamforming on the external sound signal;
in response to the feature value being less than or equal to a first threshold value and exceeding a second threshold value, obtaining a first target of the voice parameter based on the calculated initial value;
in response to the feature value being less than or equal to the second threshold value and exceeding a third threshold value, loading an initial value of the voice parameter from a memory of the electronic device and obtain a second target of the voice parameter by updating the loaded initial value; and
in response to the feature value being less than or equal to the third threshold value, determine the loaded initial value as a third target of the voice parameter.
14. An operating method performed by an electronic device, the operating method comprising:
estimating a feature value of an external sound signal received by a plurality of microphones;
calculating an initial value of a voice parameter used to perform beamforming on the external sound signal;
obtaining a target voice parameter used to perform beamforming on the external sound signal;
determining a filter to be used to perform beamforming on the external sound signal based on the target voice parameter;
estimating a magnitude of residual noise with respect to a signal on which beamforming is performed using the filter; and
performing noise processing on the signal on which beamforming is performed according to the estimated magnitude of the residual noise,
wherein the obtaining of the target voice parameter includes
in response to the feature value being less than or equal to a first threshold value and exceeding a second threshold value, obtaining a first target of the voice parameter based on the calculated initial value;
in response to the feature value being less than or equal to the second threshold value and exceeding a third threshold value, loading an initial value of the voice parameter from a memory of the electronic device and obtaining a second target of the voice parameter by updating the loaded initial value; and
in response to the feature value being less than or equal to the third threshold value, determining the loaded initial value as a third target of the voice parameter.
15. The operating method of claim 14 , wherein the electronic device is one of a true-wireless stereo (TWS) earphone, headphones, or a headset.
16. The operating method of claim 14 , wherein the plurality of microphones comprises a first microphone having a sound hole positioned on a mounting portion of an earphone, the sound hole adjacent to an inner portion of a user ear, the sound hole being exposed to an outside of the user ear.
17. The operating method of claim 16 , wherein the plurality of microphones further comprises a second microphone disposed on an insertion portion of the earphone, a sound hole of the second microphone disposed so that at least a portion of the sound hole of the second microphone is exposed towards an inner wall of the outside of the user ear.
18. The operating method of claim 14 , further comprising:
performing voice activity detection (VAD) using an accelerator and a voice pickup unit (VPU) sensor of the electronic device.