Gaze-directed audio enhancement
Embodiments relate to a headset that filters sounds according to a direction of a gaze of a user wearing the headset. The user wears the headset including an eye tracking unit and one or more microphones. The eye tracking unit tracks an orientation of an eye of the user to determine the direction of the gaze of the user. The direction of the gaze may be different from a facing direction of the headset. According to the determined direction of the gaze of the user, input sound signals generated by the microphones can be beamformed to amplify or emphasize sound originating from the direction of the gaze.
1. A headset system, comprising:
a frame;
an eye tracking unit coupled to the frame, the eye tracking unit configured to determine a direction of a gaze of a user wearing the headset system;
two or more microphones configured to detect input sounds and generate the input audio signals corresponding to the detected input sounds;
a processor communicating with the two or more microphones to receive the input audio signals, the processor configured to process the input audio signals based on the determined direction of the gaze and a head related transfer function (HRTF) transformation to generate an output audio signal; and
an audio output interface configured to send the output audio signal to two or more speakers to produce an output sound that is perceived by the user as originating from the determined direction of the gaze.
2. The headset system of claim 1 , wherein the two or more microphones are secured to the frame.
3. The headset system of claim 1 , wherein the processing of the input audio signals by the processor comprises performing beamforming based on the determined direction of the gaze.
4. The headset system of claim 1 , wherein the processing of the input audio signals by the processor comprises:
determining a microphone with a primary direction sensitivity corresponding to an input sound originating from the determined direction of the gaze; and
attenuating an input audio signal from at least one microphone other than the determined microphone to generate the output audio signal.
5. The headset system of claim 1 , wherein the processing of the input audio signals by the processor comprises:
determining a microphone with a primary direction of sensitivity corresponding to an input sound originating from the determined direction of the gaze; and
amplifying an input audio signal from the determined microphone to generate the output audio signal.
6. The headset system of claim 1 , wherein the processing of the input audio signals by the processor comprises at least one of transforming, filtering, and mixing the input audio signals with different weights according to the direction of the gaze.
7. The headset system of claim 1 , wherein the processing of the input audio signals by the processor comprises attenuating an input audio signal from a mixed audio signal of two or more input audio signals according to the direction of the gaze, the input audio signal corresponding to an input sound originating from a direction different from the determined direction of the gaze.
8. The headset system of claim 1 , wherein the processing of the input audio signals by the processor comprises applying linear transformations to the input audio signals according to the direction of the gaze.
9. The headset system of claim 1 , wherein the two or more microphones are disposed in an array on the frame.
10. The headset system of claim 1 , wherein the audio output interface is configured to communicate with at least two speakers that generate a stereo sound of an input sound originating from the determined direction of the gaze.
11. A method comprising:
determining, by an eye tracking unit secured to a headset, a direction of a gaze of a user wearing the headset;
generating, by a processor, an output audio signal by processing input audio signals corresponding to input sounds originating from different directions with respect to an orientation of the headset based on the determined direction of the gaze and a head related transfer function (HRTF) transformation; and
sending, by an audio output interface, the output audio signal to two or more speakers to reproduce an output sound that is perceived by the user as originating from the determined direction of the gaze.
12. The method of claim 11 , further comprising:
detecting, by two or more microphones, input sounds and generating input audio signals corresponding to the detected input sounds.
13. The method of claim 11 , wherein the processing of the input audio signals by the processor comprises performing beamforming based on the determined direction of the gaze.
14. The method of claim 11 , wherein the processing of the input audio signals by the processor comprises:
determining a microphone with a primary direction of sensitivity corresponding to an input sound originating from the determined direction of the gaze; and
attenuating an input audio signal from at least one microphone other than the determined microphone to generate the output audio signal.
15. The method of claim 11 , wherein the processing of the input audio signals by the processor comprises:
determining a microphone with a primary direction of sensitivity corresponding to an input sound originating from the determined direction of the gaze; and
amplifying an input audio signal from the determined microphone to generate the output audio signal.
16. The method of claim 11 , wherein the audio output interface is configured to communicate with at least two speakers that generate a stereo sound of an input sound originating from the determined direction of the gaze.
17. A non-transient computer readable medium containing program instructions for causing a computer to perform the method of:
determining, by an eye tracking unit secured to a headset, a direction of a gaze of a user wearing the headset;
generating, by a processor, an output audio signal by processing input audio signals corresponding to input sounds originating from different directions with respect to an orientation of the headset based on the determined direction of the gaze and a head relating transfer function (HRTF) transformation; and
sending, by an audio output interface, the output audio signal to two or more speakers to reproduce an output sound that is perceived by the user as originating from the determined direction of the gaze.
18. The non-transient computer readable medium of claim 17 , further comprising the computer to perform the method of:
detecting, by two or more microphones, input sounds and generating input audio signals corresponding to the detected input sounds.