IP Library Granted Patent US 10,587,981
Granted Patent B2
US 10,587,981 · App. 15/635,166 · Granted Mar 10, 2020

Providing HRTFs to improve computer performance of electronic devices providing binaural sound for a telephone call

Inventors: Philip Scott Lyren (Hong Kong, CN); Glen A. Norris (Tokyo, JP)
H04S7/304H04R5/033H04S3/008H04S7/303H04S2400/01H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,587,981
App. No.
15/635,166
Granted
Mar 10, 2020
Kind
B2
Abstract

A method improves computer performance to provide binaural sound in a telephone call. The method includes providing the telephone call with a voice of a calling party and head-related transfer functions (HRTFs) providing where a voice of the calling party localizes as binaural sound to the called party. The voice of the calling party is convolved with the HRTFs received so the voice externally localizes as the binaural sound to the called party.

Claims (50)

1. A method that improves computer performance to provide a voice of a calling party as binaural sound to a called party in a Voice over Internet Protocol (VoIP) telephone call, the method comprising:

transmitting, during the VoIP telephone call and to a portable electronic device (PED) of the called party, streaming packets of sound of the voice of the calling party with the streaming packets including head-related transfer functions (HRTFs); and

processing, by a processor in the PED of the called party, the voice of the calling party with the HRTFs received in the streaming packets so the voice of the calling party externally localizes as the binaural sound in empty space to the called party at a sound localization point (SLP) that is at least one meter away from the called party.

2. The method of claim 1 further comprising:

transmitting, with the streaming packets during the VoIP telephone call, an interaural time delay (ITD); and

processing, by the processor in the PED, the voice of the calling party with the ITD.

3. The method of claim 1 further comprising:

transmitting, with the streaming packets during the VoIP telephone call, a coordinate location of the SLP;

determining, by the PED, whether the coordinate location of the SLP conflicts with another binaural sound already externally localizing to the coordinate location of the SLP; and

changing, by the PED, the coordinate location of the SLP to another coordinate location in response to the determining that the coordinate location of the SLP conflicts with the another binaural sound already externally localizing to the coordinate location of the SLP.

4. The method of claim 1 further comprising:

saving processing resources of the PED of the called party by processing, by a processor in a server, the voice of the calling party with the HRTFs to externally localize as the binaural sound in empty space as the voice of the calling party streams to the PED of the called party.

5. The method of claim 1 , wherein the HRTFs are transmitted when the VoIP telephone call commences and along with the streaming packets of sound of the voice of the calling party.

6. The method of claim 1 further comprising:

tracking, during the VoIP telephone call, a head orientation of the called party; and

saving processing resources by automatically switching the voice of the calling party from being provided to the called party as the binaural sound localizing at the SLP to being provided as mono sound or stereo sound when the head orientation of the called party tilts beyond a predetermined elevation angle.

7. The method of claim 1 further comprising:

providing a coordinate location of the SLP to have a spherical coordinate location of (r, θ, φ); and

reducing latency associated with memory access during the VoIP telephone call by prefetching a plurality of HRTFs having spherical coordinate locations (r, θ′, φ) in which θ′ represents different azimuth angles that are compensated by an amount that a head of the called party is predicted to move during the VoIP telephone call.

8. A method that improves computer performance of electronic devices executing a telephone call that provides binaural sound to parties to the telephone call, the method comprising:

receiving, at a portable electronic device (PED) of a called party, the telephone call that includes a voice of a calling party and head related impulse responses (HRIRs); and

convolving, with a processor, the voice of the calling party with the HRIRs received at the PED with the voice of the calling party so the voice of the calling party externally localizes as the binaural sound in empty space away from the called party at a sound localization point (SLP) that is at least one meter away from the called party.

9. The method of claim 8 further comprising:

prefetching a plurality of head related transfer functions (HRTFs) in anticipation of a head of the called party moving during the telephone call, the plurality of HRTFs having spherical coordinates with different azimuth angles than an azimuth angle of the HRIRs received with the voice of the calling party.

10. The method of claim 8 further comprising:

receiving, with the voice of the calling party, a coordinate location in three-dimensional (3D) space for where the voice of the calling party will externally localize as the binaural sound to the called party.

11. The method of claim 8 further comprising:

improving decision-making on which coordinate location to select to externally localize voices during telephone calls by sharing, between intelligent user agents (IUAs), data on where the calling party and the called party previously externally localized voices during telephone calls.

12. The method of claim 8 further comprising:

playing a designated sound to the called party as a reminder that sounds being provided to the called party are electronically generated binaural sounds as opposed to natural sounds in an environment of the called party.

13. The method of claim 8 further comprising:

providing the voice of the calling party in a streaming format that includes a header section, a section that includes the HRIRs, and a sound data section that includes the voice of the calling party.

14. A non-transitory computer-readable storage medium that stores instructions that improve performance of one or more processors in a computer system that provides binaural sound to parties in a telephone call, the one or more processors in the computer system executing the instructions to execute a method comprising:

transmit a telephone call that includes both a voice of a calling party and head-related transfer functions (HRTFs) for where the voice of the calling party will localize as binaural sound to a called party; and

convolve the voice of the calling party with the HRTFs received with the telephone call so the voice of the calling party localizes as binaural sound in an empty space away from the called party at a sound localization point (SLP) with a coordinate location per the HRTFs.

15. The non-transitory computer-readable storage medium of claim 14 in which the one or more processors in the computer system further execute the instructions to execute the method comprising:

transmit, to a portable electronic device (PED) of the called party and at a time when the telephone call commences, the coordinate location that includes an azimuth angle and an elevation angle for the SLP.

16. The non-transitory computer-readable storage medium of claim 14 in which the one or more processors in the computer system further execute the instructions to execute the method comprising:

predict, during the telephone call, head movements of the called party; and

prefetch HRTFs having coordinate changes corresponding with the head movements in order to convolve the voice of the calling party to multiple different locations away from the SLP when a head of the called party moves according to predictions of the head movements.

17. The non-transitory computer-readable storage medium of claim 14 in which the one or more processors in the computer system further execute the instructions to execute the method comprising:

provide the voice of the calling party at the SLP with a head mounted display worn by the called party;

provide, with the head mounted display and to the called party, an image representing the calling party at the empty space, wherein a server convolves the voice of the calling party with the HRTFs before the voice of the calling party is provided to the head mounted display.

18. The non-transitory computer-readable storage medium of claim 14 , wherein the HRTFs and the voice of the calling party are streamed together in packets.

19. The non-transitory computer-readable storage medium of claim 14 in which the one or more processors in the computer system further execute the instructions to execute the method comprising:

provide the voice of the calling party at the SLP with a head mounted display worn by the called party;

provide, with the head mounted display, an image representing the calling party at a location corresponding to a location of the empty space; and

switch the voice of the calling party from being provided as the binaural sound to being provided as either mono sound or stereo sound when a head of the called party moves beyond a predetermined azimuth angle.

20. The non-transitory computer-readable storage medium of claim 14 in which the one or more processors in the computer system further execute the instructions to execute the method comprising:

stream the coordinate locations in packets to a portable electronic device (PED) of the called party when the telephone call is incoming to the PED of the called party.

Assignments (2)
SECURITY INTEREST Recorded Apr 7, 2022
From: EIGHT KHZ, LLC
To: LIT-US CHISUM 22-A LLC
Reel/Frame 059536/0708 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2022
From: NORRIS, GLEN A.; LYREN, PHILIP S.
To: EIGHT KHZ, LLC
Reel/Frame 059293/0646 →
Continuity (4)
Continuation 15429131 · Feb 9, 2017
Continuation 15365880 · Nov 30, 2016
Provisional Application 62348164 · Jun 10, 2016
Related Publication 20170359672A1 · Dec 14, 2017