IP Library › Granted Patent US 12,245,021
Granted Patent B2
US 12,245,021 · App. 17/827,745 · Granted Mar 4, 2025

Display a graphical representation to indicate sound will externally localize as binaural sound

Inventor: Philip Scott Lyren (Wadsworth, OH)
Assignee: Pelagic Concepts LLC
H04S7/308G06F3/0481G06T11/60H04M1/6066H04M1/7243H04R5/02H04R5/04H04R2499/11H04S7/40H04S2400/01H04S2400/11H04S2400/15H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,245,021
App. No.
17/827,745
Granted
Mar 4, 2025
Kind
B2
Abstract

A wearable electronic device (WED) worn on a head of a user displays a graphical representation having an appearance in response to sound of the graphical representation playing as mono sound or stereo sound. The appearance of the graphical representation being displayed alters in response to the sound of the graphical representation playing as binaural sound.

Claims (34)

1. A method, comprising:

displaying, with a wearable electronic device (WED) worn on a head of a user, a graphical representation of a second user having a first appearance in response to a sound of the graphical representation of the second user playing as a mono sound or a stereo sound; and

altering an appearance of the graphical representation of the second user from the first appearance to a second appearance, different from the first appearance, being displayed with the WED in response to the sound of the graphical representation of the second user playing as a binaural sound.

2. The method of claim 1 further comprising:

altering the appearance of the graphical representation of the second user in response to the WED determining the user is wearing one of headphones and earphones.

3. The method of claim 1 , wherein the appearance of the graphical representation of the second user includes “3D” that indicates to the user that the sound of the graphical representation of the second user plays as the binaural sound.

4. The method of claim 1 , wherein the appearance of the graphical representation of the second user points to a location where the binaural sound externally localizes in empty space to the user.

5. The method of claim 1 further comprising:

displaying, with a display of the WED, an animated character that moves to indicate where the sound of the graphical representation of the second user externally localizes in empty space as the binaural sound to the user.

6. The method of claim 1 further comprising:

continuing to display the graphical representation of the second user indicating the sound plays as the binaural sound while the user listens to the binaural sound after the user activates the graphical representation of the second user.

7. The method of claim 1 , wherein the WED alters the appearance of the graphical representation of the second user by swapping the graphical representation of the second user being displayed for the mono sound or the stereo sound with the graphical representation of the second user for the binaural sound.

8. A method, comprising:

displaying, with a wearable electronic device (WED) worn on a head of a user, a graphical representation of a second user having a first appearance when a sound for the graphical representation of the second user plays as at least one of a mono sound or a stereo sound; and

altering the first appearance of the graphical representation of the second user being displayed with the WED to a second appearance, different from the first appearance, being displayed with the WED when the sound for the graphical representation of the second user plays as a binaural sound.

9. The method of claim 8 further comprising:

displaying the second appearance as a three-dimensional (3D) emoji in augmented reality (AR) or virtual reality (VR) that indicates where the binaural sound externally localizes to the user.

10. The method of claim 8 further comprising:

removing the second appearance from being displayed in response to the WED no longer providing the binaural sound to the user.

11. The method of claim 8 further comprising:

displaying the second appearance as a talking graphical representation of the second user in augmented reality (AR) or virtual reality (VR) that informs the user where the binaural sound originates to the user.

12. The method of claim 8 further comprising:

changing a facial expression of the graphical representation of the second user to inform the user that the sound plays as the binaural sound.

13. The method of claim 8 further comprising:

displaying, with the WED, the second appearance as an augmented reality (AR) or virtual reality (VR) image at a location where the binaural sound originates to the user in empty space away from the head of the user.

14. A wearable electronic device (WED) worn on a head of a user, the WED comprising:

speakers that play a sound as a mono sound, a stereo sound, or a binaural sound to the user; and

a display that displays a graphical representation of a second user with a first appearance when the speakers play the sound as the mono sound or the stereo sound and changes the first appearance of the graphical representation of the second user being displayed to a second appearance, different from the first appearance, in response to the speakers playing the sound as the binaural sound, wherein the second appearance of the graphical representation visually notifies the user that the speakers play the sound as the binaural sound.

15. The electronic device of claim 14 , wherein the display displays the second appearance of the graphical representation of the second user at a location in empty space where the binaural sound originates to the user.

16. The electronic device of claim 14 , wherein the second appearance of the graphical representation of the second user includes a head of a character with eyes that look where the binaural sound originates to the user.

17. The electronic device of claim 14 , wherein the second appearance of the graphical representation of the second user includes a face that looks where the binaural sound originates to the user.

18. The electronic device of claim 14 , wherein the graphical representation of the second user automatically changes from the first appearance to the second appearance in response to the user donning the WED.

19. The electronic device of claim 14 , wherein the second appearance of the graphical representation of the second user includes a head that rotates to look where the binaural sound originates to the user.

20. The electronic device of claim 14 , wherein the second appearance of the graphical representation of the second user is a three-dimensional (3D) image in augmented reality (AR) or virtual reality (VR) with a face that looks in a direction where the binaural sound externally localizes to the user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2024
From: LYREN, PHILIP SCOTT
To: PELAGIC CONCEPTS LLC
Reel/Frame 069220/0605 →
Continuity (6)
Continuation 17128189 · Dec 20, 2020
Continuation 16695113 · Nov 25, 2019
Continuation 16354185 · Mar 15, 2019
Continuation 16059049 · Aug 9, 2018
Continuation 15898587 · Feb 18, 2018
Related Publication 20220295214A1 · Sep 15, 2022
References Cited (93)
US 5889843A · Singer et al. · 1999 [cited by applicant]
US 6990205B1 · Chen · 2006 [cited by applicant]
US 9584653B1 · Lyren · 2017 [cited by examiner]
US 9584946B1 · Lyren et al. · 2017 [cited by applicant]
US 9591427B1 · Lyren et al. · 2017 [cited by applicant]
US 10063990B1 · Lyren · 2018 [cited by applicant]
US 10088868B1 · Hernandez Garcia et al. · 2018 [cited by applicant]
US 10154364B1 · Lyren · 2018 [cited by applicant]
US 10299065B1 · Lyren · 2019 [cited by applicant]
US 10306395B1 · Lyren · 2019 [cited by applicant]
US 10499179B1 · Lyren · 2019 [cited by applicant]
US 10499184B2 · Lyren · 2019 [cited by applicant]
US 10592072B2 · Yang et al. · 2020 [cited by applicant]
US 10757523B2 · Lyren · 2020 [cited by applicant]
US 10834518B2 · Lyren · 2020 [cited by applicant]
US 10848893B2 · Lyren · 2020 [cited by applicant]
US 10873827B2 · Lyren · 2020 [cited by applicant]
US 11323837B2 · Lyren · 2022 [cited by applicant]
US 11356797B2 · Lyren · 2022 [cited by examiner]
US 11711664B2 · Lyren · 2023 [cited by applicant]
US 20020147586A1 · Belrose · 2002 [cited by applicant]
US 20070061761A1 · Lee et al. · 2007 [cited by applicant]
US 20080187143A1 · Mak-Fan · 2008 [cited by applicant]
US 20080319764A1 · Nagle et al. · 2008 [cited by applicant]
US 20110201362A1 · Bregman-Amitai et al. · 2011 [cited by applicant]
US 20130141605A1 · Kim et al. · 2013 [cited by applicant]
US 20150124975A1 · Pontoppidan · 2015 [cited by applicant]
US 20150172599A1 · Caldwell · 2015 [cited by applicant]
US 20150193197A1 · Nahman et al. · 2015 [cited by applicant]
US 20150296086A1 · Eckert et al. · 2015 [cited by applicant]
US 20150373477A1 · Norris et al. · 2015 [cited by applicant]
US 20160341959A1 · Gibbs et al. · 2016 [cited by applicant]
US 20170046123A1 · Song et al. · 2017 [cited by applicant]
US 20170111741A1 · Pan · 2017 [cited by applicant]
US 20170293461A1 · McCauley et al. · 2017 [cited by applicant]
US 20170347219A1 · McCauley et al. · 2017 [cited by applicant]
US 20170359467A1 · Norris et al. · 2017 [cited by applicant]
US 20170374173A1 · Nassirzadeh · 2017 [cited by applicant]
US 20180026925A1 · Kennedy · 2018 [cited by applicant]
US 20180032237A1 · Richards · 2018 [cited by applicant]
US 20180139565A1 · Norris et al. · 2018 [cited by applicant]
US 20180144524A1 · Lotto et al. · 2018 [cited by applicant]
US 20180336714A1 · Stoyles et al. · 2018 [cited by applicant]
US 20190246233A1 · Lyren · 2019 [cited by applicant]
US 20190261126A1 · Lyren · 2019 [cited by applicant]
US 20190278552A1 · Lyren · 2019 [cited by applicant]
US 20190306451A1 · Wang et al. · 2019 [cited by applicant]
US 20200084561A1 · Lyren · 2020 [cited by applicant]
US 20200097251A1 · Lyren · 2020 [cited by applicant]
US 20200100047A1 · Lyren · 2020 [cited by applicant]
US 20200213799A1 · Lyren · 2020 [cited by applicant]
US 20200312029A1 · Heinen et al. · 2020 [cited by applicant]
US 20200389753A1 · Lyren · 2020 [cited by applicant]
US 20210058732A1 · Lyren · 2021 [cited by applicant]
US 20210105572A1 · Lyren · 2021 [cited by applicant]
US 20210112366A1 · Lyren · 2021 [cited by applicant]
US 20220225052A1 · Lyren · 2022 [cited by applicant]
US 20220248159A1 · Lyren · 2022 [cited by applicant]
US 20240155302A1 · Lyren · 2024 [cited by applicant]
WO 2019082050A1 · 2019 [cited by applicant]
Examiner's Answer to Appeal Brief received for U.S. Appl. No. 17/000,316, mailed on Oct. 20, 2022, 10 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Jul. 12, 2022, 28 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Nov. 21, 2023, 32 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 17/000,316, mailed on Feb. 15, 2022, 10 pages. [cited by applicant]
Non-Final Office Action received for U.S Appl. No. 15/889,185, mailed on Feb. 19, 2019, 6 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 15/913,927, mailed on Sep. 23, 2019, 8 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/210,013, mailed on Mar. 25, 2020, 6 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/237,679, mailed on Sep. 18, 2019, 9 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/354,184, mailed on Apr. 30, 2020, 6 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Mar. 2, 2023, 28 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Nov. 2, 2021, 26 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,119, mailed on Apr. 9, 2020, 9 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/000,316, mailed on Oct. 29, 2021, 11 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/092,201, mailed on Aug. 26, 2021, 8 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/100,893, mailed on Jul. 21, 2021, 16 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/710,088, mailed on Oct. 27, 2022, 6 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 15/889,185, mailed on Apr. 10, 2019, 7 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 15/913,927, mailed on Oct. 11, 2019, 8 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/125,715, mailed on Oct. 29, 2018, 6 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/210,013, mailed on Jul. 7, 2020, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/237,679, mailed on Oct. 4, 2019, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/354,184, mailed on Jul. 13, 2020, 7 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/695,119, mailed on Jul. 27, 2020, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/092,201, mailed on Dec. 3, 2021, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/100,893, mailed on Jan. 4, 2022, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/710,088, mailed on Mar. 9, 2023, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/727,772, mailed on May 30, 2023, 5 pages. [cited by applicant]
Patent Board Decision received for U.S. Appl. No. 17/000,316, mailed on Nov. 7, 2023, 18 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/727,772, mailed on Feb. 3, 2023, 13 pages. [cited by applicant]
Burgess, David, “Techniques for Low Cost Spatial Audio”, UIST'92, Nov. 15-18, 1992, pp. 53-59. [cited by applicant]
Coleman et al., “An Audio-Visual System for Object-Based Audio: From Recording to Listening”, IEEE Transactions on Multimedia, vol. 20, No. 8, Aug. 2018, pp. 1919-1931. [cited by applicant]
Evans, Michael J., “Mapping an Auditory Space onto a Graphical User Interface”, Proceedings IEEE International Conference on Multimedia Computing and Systems, vol. 1, 1999, pp. 807-811. [cited by applicant]
Pugliese et al., “ATSI: Augmented and Tangible Sonic Interaction”, TEI '15: Proceedings of the Ninth International Conference on Tangible, Jan. 15-19, 2015, pp. 97-104. [cited by applicant]