IP Library Granted Patent US 11,290,836
Granted Patent B2
US 11,290,836 · App. 17/062,637 · Granted Mar 29, 2022

Providing binaural sound behind an image being displayed with an electronic device

Inventors: Glen A. Norris (Tokyo, JP); Philip Scott Lyren (Bangkok, TH)
H04S7/303G06V10/17G06V40/161G06V40/165G06V40/172H04N5/232H04N5/23219H04N7/141H04N7/147H04N7/15H04S7/40H04N2007/145H04S2420/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,290,836
App. No.
17/062,637
Granted
Mar 29, 2022
Kind
B2
Abstract

A electronic device displays an image that has sound. One or more processors process the sound for the image into binaural sound for a user. The binaural sound has a sound localization point (SLP) with a coordinate location that occurs behind the image while the image is located a near-field distance from a head of the user.

Claims (31)

1. A method, comprising:

displaying video on a display of a portable electronic device (PED) held in a hand of a user; and

processing, with one or more processors, sound for the video to generate binaural sound that has a sound localization point (SLP) with a coordinate location that occurs behind the PED held in the hand of the user and along a line-of-sight from the user to the PED.

2. The method of claim 1 further comprising:

tracking, with one or more sensors, movement of a head of the user; and

processing, with the one or more processors, the sound for the video such that the SLP for the sound for the video continues to have the coordinate location that occurs behind the PED held in the hand of the user as the head of the user moves.

3. The method of claim 1 further comprising:

processing, with the one or more processors, the sound for the video with far-field head-related transfer functions (HRTFs) while a location of the PED held in the hand of the user is less than one meter from the user and within a near-field distance.

4. The method of claim 1 further comprising:

processing, with the one or more processors, the sound for the video with head-related transfer functions (HRTFs) having a coordinate location that matches the coordinate location of the SLP that occurs behind the PED held in the hand of the user and along the line-of-sight from the user to the PED.

5. The method of claim 1 , wherein the one or more processors include a digital signal processor (DSP) that processes the sound for the video with far-field head-related transfer functions (HRTFs) while the PED is located at a near-field distance from a face of the user, and the PED is one of a smartphone and a tablet computer.

6. The method of claim 1 , wherein the coordinate location of the SLP is located a distance (d 1 ) from a head of the user in which d 1 >1.0 meter, and the PED is located a distance (d 2 ) from the head of the user in which d 2 <1.0 meter.

7. The method of claim 1 further comprising:

sensing, with one or more sensors in the PED, when the PED rotates in the hand of the user; and

automatically switching, in response to the one or more sensors in the PED sensing rotation of the PED in the hand of the user, the binaural sound to stereo sound.

8. An electronic device, comprising:

a display that displays an image that occurs within a near-field distance from a head of a user; and

one or more processors that process sound for the image to generate binaural sound that has a sound localization point (SLP) with a coordinate location that occurs at a far-field distance located behind the image and along a line-of-sight from the user to the image while the display displays the image to the user at the near-field distance, wherein the one or more processors include a digital signal processor (DSP) that processes the sound for the image with far-field head-related transfer functions (HRTFs) while the electronic device is located at the near-field distance from the head of the user.

9. The electronic device of claim 8 , wherein the image is an augmented reality (AR) image, and the display is included with electronic glasses worn on the head of the user.

10. The electronic device of claim 8 , wherein the image is a virtual reality (VR) image, and the display is included with a head mounted display worn on the head of the user.

11. The electronic device of claim 8 further comprising:

one or more sensors that detect rotation of the electronic device, wherein the binaural sound automatically switches to stereo sound in response to the one or more sensors detecting the rotation of the electronic device.

12. The electronic device of claim 8 , wherein the binaural sound automatically switches to stereo sound in response to the electronic device detecting a change to a head orientation of the user that exceeds a threshold value in an azimuth direction.

13. The electronic device of claim 8 , wherein the image is an augmented reality (AR) image of an intelligent user assistant (IUA) that is located less than one meter away from the head of the user, and a voice of the IUA occurs at a coordinate location behind the AR image of the IUA.

14. The electronic device of claim 8 , wherein the image is a virtual reality (VR) image of an intelligent user assistant (IUA) that is located less than one meter away from the head of the user, and a voice of the IUA occurs at a coordinate location behind the VR image of the IUA.

15. An electronic system, comprising:

a display that displays an image that is located within a near-field distance from a head of a user viewing the image; and

one or more processors that process sound for the image with head-related transfer functions (HRTFs) to generate binaural sound that has a sound localization point (SLP) with a coordinate location that occurs at a far-field distance located behind the image and along a line-of-sight from the user to the image while the display displays the image to the user at the near-field distance, wherein the coordinate location of the binaural sound continues to occur at the far-field distance located behind the image when the head of the user moves away from viewing the image and the image remains within the near-field distance from the head of the user.

16. The electronic system of claim 15 , wherein the display is an augmented reality (AR) display worn on the head of the user viewing the image, and the image is an AR image.

17. The electronic system of claim 15 , wherein the display is a virtual reality (VR) display worn on the head of the user viewing the image, and the image is an VR image.

18. The electronic system of claim 15 , wherein the display is on a smartphone held in a hand of the user.

Assignments (2)
SECURITY INTEREST Recorded Apr 7, 2022
From: EIGHT KHZ, LLC
To: LIT-US CHISUM 22-A LLC
Reel/Frame 059536/0708 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2022
From: NORRIS, GLEN A.; LYREN, PHILIP S.
To: EIGHT KHZ, LLC
Reel/Frame 059293/0646 →
Continuity (4)
Continuation 16354182 · Mar 15, 2019
Continuation 16246501 · Jan 13, 2019
Continuation 15603552 · May 24, 2017
Related Publication 20210021952A1 · Jan 21, 2021