IP Library Granted Patent US 10,244,342
Granted Patent B1
US 10,244,342 · App. 15/694,807 · Granted Mar 26, 2019

Spatially representing graphical interface elements as binaural audio content

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,244,342
App. No.
15/694,807
Granted
Mar 26, 2019
Kind
B1
Abstract

Certain embodiments involve spatially representing graphical interface elements as binaural audio content. For example, a system accesses electronic content having various content elements and identifies the various content elements. The system generates a three-dimensional audio representation for a content element by: accessing an audio representation of the content element; identifying a visual location of the content element; and generating a three-dimensional audio representation of the content element by applying, to the audio representation of the content element and using the visual location, a finite impulse response filter using a head-related transfer function, wherein the head-related transfer function comprises a set of filter parameters that model the visual location of the content element and wherein, when played, the three-dimensional audio representation appears to originate from the visual location. The system can then output the three-dimensional audio representation.

Claims (68)

1. A method for spatially representing graphical interface elements as binaural audio content, the method comprising:

accessing, by a processor, electronic content having a plurality of content elements;

identifying, by the processor, the plurality of content elements of the electronic content;

generating, by the processor, a three-dimensional audio representation for a content element of the plurality of content elements, wherein generating the three-dimensional audio representation comprises:

accessing an audio representation of the content element;

identifying a visual location of the content element by;

accessing, by the processor, a content object model specifying visual locations of the plurality of content elements when displayed in a graphical interface; and

identifying, by the processor and from the content object model, the visual location of the content element; and

generating a three-dimensional audio representation of the content element by applying, to the audio representation of the content element and using the visual location, a finite impulse response filter using a head-related transfer function, wherein the head-related transfer function comprises a set of filter parameters that model the visual location of the content element and wherein, when played, the three-dimensional audio representation appears to originate from the visual location; and

outputting, by the processor, the three-dimensional audio representation such that the three-dimensional audio representation is accessible to a user device that displays the electronic content.

2. The method of claim 1 , wherein accessing the audio representation of the content element comprises:

obtaining, by the processor, a textual representation of the content element; and

converting, by the processor, the textual representation to the audio representation.

3. The method of claim 2 , wherein converting the textual representation to the audio representation comprises converting, by the processor, the textual representation to the audio representation by using a text-to-speech engine to generate the audio representation from the textual representation.

4. The method of claim 1 , wherein outputting the three-dimensional audio representation comprises:

receiving, by the processor, user input to interact with the content element; and

outputting, by the processor, the three-dimensional audio representation in response to receiving the user input.

5. The method of claim 1 , wherein identifying the visual location of the content element comprises:

determining, by the processor, an azimuth value of the content element; and

determining, by the processor, an elevation value of the content element.

6. The method of claim 5 , wherein generating the three-dimensional audio representation of the content element comprise:

applying, to the audio representation of the content element and using the azimuth value and the elevation value, the finite impulse response filter using the head-related transfer function, wherein the set of filter parameters are based on the azimuth value and the elevation value.

7. A system comprising:

a processing device; and

a non-transitory computer-readable medium communicatively coupled to the processing device, wherein the processing device is configured to perform operations comprising:

accessing electronic content having a plurality of content elements;

identifying the plurality of content elements of the electronic content;

generating a three-dimensional audio representation for a content element of the plurality of content elements, wherein generating the three-dimensional audio representation comprises:

accessing an audio representation of the content element;

identifying a visual location of the content element by:

accessing a content object model specifying visual locations of the plurality of content elements when displayed in a graphical interface; and

identifying the visual location of the content element from the content object model; and

generating a three-dimensional audio representation of the content element by applying, to the audio representation of the content element and using the visual location, a finite impulse response filter using a head-related transfer function, wherein the head-related transfer function comprises a set of filter parameters that model the visual location of the content element and wherein, when played, the three-dimensional audio representation appears to originate from the visual location; and

outputting the three-dimensional audio representation such that the three-dimensional audio representation is accessible to a user device that displays the electronic content.

8. The system of claim 7 , wherein the processing device is further configured to access the audio representation of the content element by:

obtaining a textual representation of the content element; and

converting the textual representation to the audio representation.

9. The system of claim 8 , wherein the processing device is further configured to convert the textual representation to the audio representation by converting the textual representation to the audio representation by using a text-to-speech engine to generate the audio representation from the textual representation.

10. The system of claim 7 , wherein the processing device is further configured to output the three-dimensional audio representation by:

receiving user input to interact with the content element; and

outputting the three-dimensional audio representation in response to receiving the user input.

11. The system of claim 7 , wherein the processing device is further configured to identify the visual location of the content element by:

determining an azimuth value of the content element; and

determining an elevation value of the content element.

12. The system of claim 11 , wherein the processing device is further configured to generate the three-dimensional audio representation of the content element by applying, to the audio representation of the content element and using the azimuth value and the elevation value, the finite impulse response filter using the head-related transfer function, wherein the set of filter parameters are based on the azimuth value and the elevation value.

13. A system comprising:

a means for accessing electronic content having a plurality of content elements;

a means for identifying the plurality of content elements of the electronic content;

a means for generating a three-dimensional audio representation for a content element of the plurality of content elements, wherein generating the three-dimensional audio representation comprises:

accessing an audio representation of the content element;

identifying a visual location of the content element by:

accessing a content object model specifying visual locations of the plurality of content elements when displayed in a graphical interface; and

identifying the visual location of the content element from the content object model; and

generating a three-dimensional audio representation of the content element by applying, to the audio representation of the content element and using the visual location, a finite impulse response filter using a head-related transfer function, wherein the head-related transfer function comprises a set of filter parameters that model the visual location of the content element and wherein, when played, the three-dimensional audio representation appears to originate from the visual location; and

a means for outputting the three-dimensional audio representation such that the three-dimensional audio representation is accessible to a user device that displays the electronic content.

14. The system of claim 13 , further comprising:

a means for accessing the audio representation of the content element by:

obtaining a textual representation of the content element; and

converting the textual representation to the audio representation.

15. The system of claim 14 , further comprising a means for converting the textual representation to the audio representation by:

converting the textual representation to the audio representation using a text-to-speech engine to generate the audio representation from the textual representation.

16. The system of claim 14 , further comprising a means for outputting the three-dimensional audio representation by:

receiving user input to interact with the content element; and

outputting the three-dimensional audio representation in response to receiving the user input.

17. The system of claim 14 , further comprising a means for generating the three-dimensional audio representation of the content element by:

determining an azimuth value of the content element;

determining an elevation value of the content element; and

applying, to the audio representation of the content element and using the azimuth value and the elevation value, the finite impulse response filter using the head-related transfer function, wherein the set of filter parameters are based on the azimuth value and the elevation value.

Assignments (2)
CHANGE OF NAME Recorded Mar 6, 2019
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 048525/0042 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2017
From: HERUR, PRIYANKA C.; RASHEED, FABIN
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 043653/0931 →