IP Library › Granted Patent US 12,464,308
Granted Patent B2
US 12,464,308 · App. 18/407,005 · Granted Nov 4, 2025

Emoji that indicates a location of binaural sound

Inventor: Philip Scott Lyren (Bangkok, TH)
Assignee: Pelagic Concepts LLC
H04S7/303G06F3/147G06F3/165G06T13/40G09G5/10G09G5/37H04M1/7243H04S3/008G09G5/38G09G2354/00G09G2380/06H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,464,308
App. No.
18/407,005
Filed
Jan 8, 2024
Granted
Nov 4, 2025
Kind
B2
Art Unit
2179
USPC
715/706
Abstract

A smartphone displays an emoji that when activated plays sound that externally localizes in binaural sound to the listener. The emoji includes an indication that shows a location where the sound will externally localize as the binaural sound away from a head of the listener when the listener activates the emoji.

Claims (47)

1 . A method comprising:

at a first electronic device in communication with a display and one or more speakers:

receiving, from a second electronic device that is different from the first electronic device, sound; and

in response to receiving the sound, concurrently:

displaying, via the display, a dynamic graphical representation of a user of the second electronic device at a location on the display indicative of a sound localization point; and

playing, via the one or more speakers, the sound binaurally, such that the sound appears to originate from the sound localization point;

while displaying the dynamical graphical representation of the user of the second electronic device at a first location on the display, receiving an indication of a request to move the sound localization point; and

in response to receiving the indication:

updating the sound localization point in accordance with the request; and

displaying the dynamic graphical representation of the user of the second electronic device at a second location, different from the first location, on the display indicative of the updated sound localization point.

2 . The method of claim 1 , wherein the dynamic graphical representation includes video of the user of the second electronic device.

3 . The method of claim 1 , wherein the dynamic graphical representation includes an animated cartoon representation of the user of the second electronic device.

4 . The method of claim 1 , wherein the dynamic graphical representation of the user of the first electronic device includes video of the user of the first electronic device.

5 . The method of claim 1 , wherein displaying the dynamic graphical representation of the user of the second electronic device at the location on the display indicative of the sound localization point includes displaying the dynamic graphical representation of the user of the second electronic device at a location in a real or virtual environment.

6 . The method of claim 1 , wherein receiving the sound and playing the sound is performed in real-time.

7 . The method of claim 1 , wherein the first electronic device and the second electronic device each includes a head-mounted display.

8 . An electronic device comprising:

one or more processors;

memory; and

one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method comprising:

receiving, from a second electronic device that is different from the electronic device, sound; and

in response to receiving the sound, concurrently:

displaying, via a display, a dynamic graphical representation of a user of the second electronic device at a location on the display indicative of a sound localization point; and

playing, via one or more speakers, the sound binaurally, such that the sound appears to originate from the sound localization point;

while displaying the dynamical graphical representation of the user of the second electronic device at a first location on the display, receiving an indication of a request to move the sound localization point; and

in response to receiving the indication:

updating the sound localization point in accordance with the request; and

displaying the dynamic graphical representation of the user of the second electronic device at a second location, different from the first location, on the display indicative of the updated sound localization point.

9 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:

receiving, from a second electronic device that is different from the electronic device, sound; and

in response to receiving the sound, concurrently:

displaying, via a display, a dynamic graphical representation of a user of the second electronic device at a location on the display indicative of a sound localization point; and

playing, via one or more speakers, the sound binaurally, such that the sound appears to originate from the sound localization point;

while displaying the dynamical graphical representation of the user of the second electronic device at a first location on the display, receiving an indication of a request to move the sound localization point; and

in response to receiving the indication:

updating the sound localization point in accordance with the request; and

displaying the dynamic graphical representation of the user of the second electronic device at a second location, different from the first location, on the display indicative of the updated sound localization point.

10 . The electronic device of claim 8 , wherein the dynamic graphical representation includes video of the user of the second electronic device.

11 . The electronic device of claim 8 , wherein the dynamic graphical representation includes an animated cartoon representation of the user of the second electronic device.

12 . The electronic device of claim 8 , wherein the dynamic graphical representation of the user of the electronic device includes video of the user of the electronic device.

13 . The electronic device of claim 8 , wherein displaying the dynamic graphical representation of the user of the second electronic device at the location on the display indicative of the sound localization point includes displaying the dynamic graphical representation of the user of the second electronic device at a location in a real or virtual environment.

14 . The electronic device of claim 8 , wherein receiving the sound and playing the sound is performed in real-time.

15 . The non-transitory computer readable storage medium of claim 9 , wherein the dynamic graphical representation includes video of the user of the second electronic device.

16 . The non-transitory computer readable storage medium of claim 9 , wherein the dynamic graphical representation includes an animated cartoon representation of the user of the second electronic device.

17 . The non-transitory computer readable storage medium of claim 9 , wherein the dynamic graphical representation of the user of the electronic device includes video of the user of the electronic device.

18 . The non-transitory computer readable storage medium of claim 9 , wherein displaying the dynamic graphical representation of the user of the second electronic device at the location on the display indicative of the sound localization point includes displaying the dynamic graphical representation of the user of the second electronic device at a location in a real or virtual environment.

19 . The non-transitory computer readable storage medium of claim 9 , wherein receiving the sound and playing the sound is performed in real-time.

Continuity (4)
Continuation 17000316 · Aug 22, 2020
Continuation 16354184 · Mar 15, 2019
Continuation 15889185 · Feb 5, 2018
Related Publication 20240155302A1 · May 9, 2024
References Cited (88)
US 5889843A · Singer et al. · 1999 [cited by applicant]
US 6990205B1 · Chen · 2006 [cited by applicant]
US 9584653B1 · Lyren et al. · 2017 [cited by applicant]
US 9584946B1 · Lyren et al. · 2017 [cited by applicant]
US 9591427B1 · Lyren et al. · 2017 [cited by applicant]
US 10284812B1 · Van Os · 2019 [cited by examiner]
US 10299065B1 · Lyren · 2019 [cited by applicant]
US 10499179B1 · Lyren · 2019 [cited by applicant]
US 10499184B2 · Lyren · 2019 [cited by applicant]
US 10592072B2 · Yang et al. · 2020 [cited by applicant]
US 10834518B2 · Lyren · 2020 [cited by applicant]
US 10848893B2 · Lyren · 2020 [cited by applicant]
US 10873827B2 · Lyren · 2020 [cited by applicant]
US 11323837B2 · Lyren · 2022 [cited by applicant]
US 11356797B2 · Lyren · 2022 [cited by applicant]
US 11711664B2 · Lyren · 2023 [cited by applicant]
US 20020147586A1 · Belrose · 2002 [cited by applicant]
US 20080187143A1 · Mak-fan · 2008 [cited by applicant]
US 20080319764A1 · Nagle et al. · 2008 [cited by applicant]
US 20110201362A1 · Bregman-amitai et al. · 2011 [cited by applicant]
US 20130141605A1 · Kim et al. · 2013 [cited by applicant]
US 20150124975A1 · Pontoppidan · 2015 [cited by examiner]
US 20150172599A1 · Caldwell · 2015 [cited by applicant]
US 20160341959A1 · Gibbs et al. · 2016 [cited by applicant]
US 20170046123A1 · Song et al. · 2017 [cited by applicant]
US 20170111741A1 · Pan · 2017 [cited by applicant]
US 20170293461A1 · Mccauley et al. · 2017 [cited by applicant]
US 20170347219A1 · Mccauley et al. · 2017 [cited by applicant]
US 20170359467A1 · Norris et al. · 2017 [cited by applicant]
US 20170374173A1 · Nassirzadeh · 2017 [cited by applicant]
US 20180026925A1 · Kennedy · 2018 [cited by applicant]
US 20180032237A1 · Richards · 2018 [cited by applicant]
US 20180139565A1 · Norris et al. · 2018 [cited by applicant]
US 20180232934A1 · Schmidt · 2018 [cited by examiner]
US 20180336714A1 · Stoyles et al. · 2018 [cited by applicant]
US 20190261126A1 · Lyren · 2019 [cited by applicant]
US 20190278552A1 · Lyren · 2019 [cited by applicant]
US 20190306451A1 · Wang et al. · 2019 [cited by applicant]
US 20200084561A1 · Lyren · 2020 [cited by applicant]
US 20200097251A1 · Lyren · 2020 [cited by applicant]
US 20200100047A1 · Lyren · 2020 [cited by applicant]
US 20200213799A1 · Lyren · 2020 [cited by applicant]
US 20200312029A1 · Heinen · 2020 [cited by examiner]
US 20210058732A1 · Lyren · 2021 [cited by applicant]
US 20210105572A1 · Lyren · 2021 [cited by applicant]
US 20210112366A1 · Lyren · 2021 [cited by applicant]
US 20220225052A1 · Lyren · 2022 [cited by applicant]
US 20220248159A1 · Lyren · 2022 [cited by applicant]
US 20220295214A1 · Lyren · 2022 [cited by applicant]
WO 2019082050A1 · 2019 [cited by applicant]
Examiner Initiated Interview Summary received for U.S. Appl. No. 15/898,587, mailed on Jul. 25, 2018, 1 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Jul. 12, 2022, 28 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Nov. 21, 2023, 32 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 17/827,745, mailed on Aug. 16, 2024, 7 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 15/913,927, mailed on Sep. 23, 2019, 8 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/059,049, mailed on Feb. 14, 2019, 9 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/210,013, mailed on Mar. 25, 2020, 6 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/237,679, mailed on Sep. 18, 2019, 9 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/354,185, mailed on Aug. 30, 2019, 10 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,113, mailed on May 20, 2020, 10 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Mar. 2, 2023, 28 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,115, mailed on Nov. 2, 2021, 26 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/695,119, mailed on Apr. 9, 2020, 9 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/092,201, mailed on Aug. 26, 2021, 8 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/100,893, mailed on Jul. 21, 2021, 16 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/128,189, mailed on Nov. 5, 2021, 9 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/710,088, mailed on Oct. 27, 2022, 6 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/827,745, mailed on Mar. 7, 2024, 11 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 15/898,587, mailed on Jul. 25, 2018, 9 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 15/913,927, mailed on Oct. 11, 2019, 8 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/059,049, mailed on Apr. 5, 2019, 9 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/125,715, mailed on Oct. 29, 2018, 6 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/210,013, mailed on Jul. 7, 2020, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/237,679, mailed on Oct. 4, 2019, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/354,185, mailed on Sep. 24, 2019, 8 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/695,113, mailed on Aug. 24, 2020, 8 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/695,119, mailed on Jul. 27, 2020, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/092,201, mailed on Dec. 3, 2021, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/100,893, mailed on Jan. 4, 2022, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/128,189, mailed on Feb. 16, 2022, 7 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/710,088, mailed on Mar. 9, 2023, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/727,772, mailed on May 30, 2023, 5 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/827,745, mailed on Nov. 14, 2024, 7 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/727,772, mailed on Feb. 3, 2023, 13 pages. [cited by applicant]
Burgess, David, “Techniques for Low Cost Spatial Audio”, UIST'92, Nov. 15-18, 1992, pp. 53-59. [cited by applicant]
Coleman, et al., “An Audio-Visual System for Object-Based Audio: From Recording to Listening”, IEEE Transactions on Multimedia, vol. 20, No. 8, Aug. 2018, pp. 1919-1931. [cited by applicant]
Evans, Michael J., “Mapping an Auditory Space onto a Graphical User Interface”, Proceedings IEEE International Conference on Multimedia Computing and Systems, vol. 1, 1999, pp. 807-811. [cited by applicant]
Pugliese, et al., “ATSI: Augmented and Tangible Sonic Interaction”, TEI '15: Proceedings of the Ninth International Conference on Tangible, Jan. 15-19, 2015, pp. 97-104. [cited by applicant]