Granted Patent B2

US 10,659,896 · App. 16/401,174 · Granted May 19, 2020

Inventors: Glen A. Norris (Tokyo, JP); Philip Scott Lyren (Bangkok, TH)

H04S1/002H04M1/605H04M1/6033H04M1/72572H04M3/53366H04M3/568H04M9/085H04R5/027H04R5/033H04R5/04H04S1/007H04S3/008H04S7/303H04S7/304G10L21/028H04R27/00H04R2227/003H04S7/305H04S2400/01H04S2400/11H04S2400/15H04S2420/01

Abstract

Claims (59)

1. A method executed by one or more electronic devices, the method comprising:

selecting, by a computer program that is an intelligent personal assistant, a location for a user where the user hears binaural sound that emanates from a sound localization point (SLP) in empty space away from a head of the user;

adjusting, by a processor in a wearable electronic device (WED) worn on the head of the user, sound with head-related transfer functions (HRTFs) to generate the binaural sound that localizes to the SLP in empty space away from the head of the user;

receiving, from the user and at a microphone in the WED, a voice command that causes the intelligent personal assistant to move the binaural sound from the SLP in empty space to another SLP in empty space; and

moving, in response to the voice command, the binaural sound from the SLP in empty space to the another SLP in empty space.

2. The method of claim 1 further comprising:

determining, with the WED, objects in a room with the user; and

selecting the location where the user hears the binaural sound so a location of the SLP does not conflict with locations of the objects in the room.

3. The method of claim 1 further comprising:

selecting the location where the user hears the binaural sound based on a previous assignment of sound localization where the user heard the binaural sound.

4. The method of claim 1 further comprising:

selecting the location where the user hears the binaural sound based on a location of the user in a room and a head orientation of the user.

5. The method of claim 1 further comprising:

selecting the location where the user hears the binaural sound based on a global positioning system (GPS) location of the user.

6. The method of claim 1 further comprising:

determining, with the WED, objects in a room with the user; and

selecting the location where the user hears the binaural sound based on available space around one of the objects in the room.

7. The method of claim 1 further comprising:

displaying, with the WED, a virtual image of a speaker at the SLP in empty space from where the user hears the binaural sound; and

changing the binaural sound that the user hears from the virtual image at the SLP to coincide with head rotations of a head of the virtual image of the speaker.

8. A non-transitory computer readable storage medium storing instructions that one or more electronic devices execute, the method comprising:

selecting, by an intelligent personal assistant and for a user, a location where the user hears binaural sound that originates from a sound localization point (SLP) in empty space away from a head of the user;

processing, by a processor, sound with head-related transfer functions (HRTFs) to produce the binaural sound that originates at the SLP in empty space away from the head of the user;

receiving, from the user and at a microphone with a wearable electronic device (WED) worn on the head of the user, a voice command that causes the intelligent personal assistant to move the binaural sound that originates at the SLP in empty space; and

adjusting processing of the binaural sound to move the binaural sound according to the voice command.

9. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

displaying, with the WED, a virtual character at the SLP in empty space that provides an origin of the binaural sound; and

changing the binaural sound that the user hears from the virtual character to coincide with head rotations of a head of the virtual character.

10. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

tracking a location of the user as the user moves between two rooms;

calculating a reduction of the binaural sound between the two rooms; and

changing a volume of the binaural sound provided to the user based on the reduction of binaural sound between the two rooms.

11. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

adjusting processing of the binaural sound as the user moves with respect to the SLP in empty space such that an origin of the binaural sound that the user hears continues to originate from the SLP in empty space even as the user changes head orientation and moves with respect to the SLP in empty space.

12. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

detecting, with the WED, eye movement of the user; and

adjusting, in response to the detecting of the eye movement, processing of the binaural sound to move the binaural sound from externally localizing to the SLP in empty space to externally localizing to another SLP in empty space.

13. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

calculating a model of a room where the user is located to determine sound reverberation of the room; and

altering, based on the model of the room, processing of the binaural sound to account for the sound reverberation of the room.

14. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

calculating a model of a room where the user is located to determine sound reflection in the room; and

altering, based on the model of the room, processing of the binaural sound to account for the sound reflection of the room.

15. A method executed by one or more electronic devices, the method comprising:

designating, by an intelligent personal assistant, a sound localization point (SLP) in empty space that is away from a location of a user with the SLP simulating an origin of binaural sound that the user hears;

processing, with one or more processors, sound and head-related transfer functions (HRTFs) to generate the binaural sound that the user hears at the SLP in empty space;

receiving, from the user and at a wearable electronic device (WED), a voice command instructing the intelligent personal assistant to move the SLP where the user hears the binaural sound; and

moving, in response to the voice command, the SLP where the user hears the binaural sound.

16. The method of claim 15 further comprising:

displaying, with the WED, a virtual person during a telephone call between the user and another user represented by the virtual person; and

changing the binaural sound that the user hears from the another person to coincide with head rotations of a head of the virtual person.

17. The method of claim 15 further comprising:

recognizing, by a kinetic user interface in the WED, a head gesture of the user; and

altering, in response to the head gesture, processing of the sound and the HRTFs to move the binaural sound from externally localizing at the SLP in empty space to externally localizing at another SLP in empty space.

18. The method of claim 15 further comprising:

sensing, with a sensor or camera in the WED, a body gesture of the user; and

altering, in response to the body gesture, processing of the sound and the HRTFs to move the binaural sound from externally localizing at the SLP in empty space to externally localizing at another SLP in empty space.

19. The method of claim 15 , wherein the voice command moves the SLP in empty space to a location of a smartphone or a tablet computer in a hand of the user.

20. The method of claim 15 , wherein the voice command moves the SLP to a foreground where the SLP is closer to the user and the binaural sound is louder.

Continuity (15)