Granted Patent B2

US 12,380,913 · App. 17/710,207 · Granted Aug 5, 2025

Inventors: Cheheung Kim (Yongin-si, KR); Hyunwook Kang (Daejeon, KR)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G10L21/0364G10L21/0216H04R3/005G10L2021/02165

Abstract

Claims (46)

1. A sound signal processing apparatus comprising:

a directional microphone configured to detect a user voice signal comprising a user's voice by arranging the directional microphone to face an utterance point of the user's voice;

a non-directional microphone configured to detect a mixed sound signal comprising the user voice and an external sound;

a processor configured to generate an external sound signal by attenuating the user's voice from the mixed sound signal, by differentially calculating the user voice signal from the mixed sound signal, and process the user voice signal and the external sound signal simultaneously; and

a display configured to display a processing result of the user voice signal and a processing result of the external sound signal in two different regions of the display at a same time, wherein:

the directional microphone comprises vibration structures that vary in length and are configured to sense sound of different frequency bands, and

one side of each of the vibration structures is fixed to a support, and an opposite side of each of the vibration structures extends in a same direction toward a cavity of the support so that extension directions of the vibration structures, from the one side of the support toward the cavity and to the opposite side of the support, are in parallel to one another.

2. The sound signal processing apparatus of claim 1 , wherein the vibration structures are configured to sense sound of different frequency bands, and wherein a vibrator included in each of vibration structures vibrates with a vibration intensity based on an angle between a propagation direction of the mixed sound signal and a plane formed by the vibrator for receiving the mixed sound signal.

3. The sound signal processing apparatus of claim 2 , wherein the vibrator vibrates with a higher vibration intensity as the angle approaches 90°, and vibrates with a lower vibration intensity as the angle approaches 0°.

4. The sound signal processing apparatus of claim 1 , wherein the vibration structures are configured to sense sound of different frequency bands, each of the vibration structures comprises a vibrator that forms a plane for receiving the mixed sound signal, and the directional microphone is arranged so that an angle formed between the plane and a direction from the utterance point of the user's voice is in a range of 60° to 120°.

5. The sound signal processing apparatus of claim 1 , wherein the vibration structures are configured to sense sound of different frequency bands, each of the vibration structures comprises a vibrator that forms a plane for receiving the mixed sound signal, and a vibration detector configured to receive a vibration of the vibrator.

6. The sound signal processing apparatus of claim 1 , wherein the directional microphone is configured to:

obtain a plurality of electrical signals from the vibration structures of the directional microphone; and

for each electrical signal in the plurality of electrical signals, turn on a switch to transmit the electrical signal to the processor when a degree of excess of the electrical signal compared to a threshold exceeds a preset value, and turn off the switch to block the electrical signal when the degree of excess is less than or equal to the preset value.

7. The sound signal processing apparatus of claim 1 , wherein the directional microphone is configured to determine an electrical signal to be attenuated from among electrical signals generated by the vibration structures, and attenuate the determined electrical signal, and

wherein the directional microphone is further configured to determine a threshold value based on an average magnitude of the electrical signals generated by the vibration structures.

8. The sound signal processing apparatus of claim 1 , further comprising an adaptive filter configured to adjust parameters that are applied to the user voice signal before the user voice signal is combined with the mixed sound signal, so that the user's voice is attenuated from the mixed sound signal based on a feedback signal,

wherein the parameters comprise a correlation coefficient between the user voice signal and the mixed sound signal, and amplitudes of the user voice signal and the mixed sound signal, and

wherein the processor is further configured to:

generate the feedback signal by differentially calculating a signal output from the adaptive filter from the mixed sound signal as the user voice signal is input to the adaptive filter; and

control the adaptive filter to adjust the parameters by inputting the feedback signal to the adaptive filter.

9. The sound signal processing apparatus of claim 1 , wherein the sound signal processing apparatus is a glasses-type wearable apparatus, the directional microphone and the non-directional microphone are arranged on a glasses frame of the glasses-type wearable apparatus, and the directional microphone is arranged so that one plane for receiving the mixed sound faces the utterance point of the user's voice.

10. The sound signal processing apparatus of claim 1 , wherein the sound signal processing apparatus is a glasses-type wearable apparatus,

wherein the directional microphone is arranged on a glasses bridge of the glasses-type wearable apparatus, and the non-directional microphone is arranged on a leg of the glasses-type wearable apparatus, and

wherein the display is further configured to indicate a recording state of the external sound signal or a translation result of the external sound signal as the processing result of the external sound signal, and provide a transcript of the user voice signal as the processing result of the user voice signal.

11. The sound signal processing apparatus of claim 1 , wherein

the non-directional microphone is configured to generate a first external sound signal from which the user's voice is attenuated, from the mixed sound signal, by arranging a plane receiving the mixed sound signal in a direction different from a direction in which the direction microphone is arranged, and in a direction corresponding to a point where the external sound is generated; and

the processor is further configured to generate a second external sound signal from which the user's voice is further attenuated than that of the first external sound signal, by differentially calculating the user voice signal from the first external sound signal.

12. A method of processing a sound signal, the method comprising:

detecting a user's voice by a directional microphone;

receiving, by a non-directional microphone, a mixed sound including the user's voice and an external sound generated from outside the user;

generating a mixed sound signal from the mixed sound;

generating a user voice signal from which the external sound is attenuated, from the user's voice;

generating an external sound signal from which the user's voice is attenuated, by differentially calculating the user voice signal from the mixed sound signal;

processing the user voice signal and the external sound signal simultaneously; and

displaying a processing result of the user voice signal and a processing result of the external sound signal in two different regions of a display at a same time, wherein:

the directional microphone comprises vibration structures that vary in length and are configured to sense sound of different frequency bands, and

one side of each of the vibration structures is fixed to a support, and an opposite side of each of the vibration structures extends in a same direction toward a cavity of the support so that extension directions of the vibration structures, from the one side of the support toward the cavity and to the opposite side of the support, are in parallel to one another.

13. The method of claim 12 , wherein the receiving of the mixed sound comprises:

vibrating each of the vibration structures that are configured to sense sound of different frequency bands, in a direction orthogonal to one plane formed to receive the mixed sound, based on a frequency of the received mixed sound.

14. The method of claim 13 , wherein the vibrating each of the plurality of vibration structures comprises vibrating each of the plurality of vibration structures with a vibration intensity based on an angle formed between a propagation direction of the received sound and the one plane.

15. The method of claim 14 , wherein the vibrating each of the plurality of vibration structures further comprises vibrating each of the plurality of vibration structures with a higher vibration intensity as an angle approaches 90°, and vibrating each of the plurality of vibration structures with a lower vibration intensity as an angle approaches 0°.

16. The method of claim 13 , wherein the vibrating each of the plurality of vibration structures comprises generating electrical signals respectively corresponding to vibrations of the plurality of vibration structures.

17. The method of claim 16 , wherein the generating of the user voice signal comprises determining an electrical signal to be attenuated among the electrical signals, based on a threshold value that is set according to an average magnitude of the electrical signals, and attenuating the determined electrical signal.

18. The method of claim 12 , wherein the generating of the external sound signal comprises inputting the user voice signal to an adaptive filter, generating a feedback signal by differentially calculating a signal output from the adaptive filter from the mixed sound signal, and controlling the adaptive filter to adjust parameters that are applied to the user voice signal before the user voice signal is combined with the mixed sound signal by inputting the feedback signal to the adaptive filter,

wherein the parameters comprise a correlation coefficient between the user voice signal and the mixed sound signal, and amplitudes of the user voice signal and the mixed sound signal.

Assignments (1)

Priority Claims (2)

Continuity (1)

References Cited (35)