Placement of virtual speakers based on room layout
Virtualizing speakers for a headphone set can include determining a location of a display. Locations of one or more virtual speakers can be assigned based on the location of the display. A first virtual speaker can be located at the display. A position of a head of a user can be tracked. Audio content can be spatialized with a spatial renderer to generate spatialized audio signals, based on the tracked position of the head and the locations of the virtual speakers. Other aspects are also described and claimed.
1. A method of virtualizing speakers, comprising:
determining a location of a display including detecting the display in one or more images generated by a camera;
assigning locations of one or more virtual speakers based on the location of the display, wherein the one or more virtual speakers include a first virtual speaker located at the display;
determining a position of a head of a user; and
spatializing, based on the position of the head and the locations of the one or more virtual speakers, one or more audio signals with a spatial renderer to generate spatialized audio signals that, when used to drive a left speaker and a right speaker of a headphone set, are converted to sound that is perceived by the user to be located at the assigned locations of the one or more virtual speakers.
2. The method of claim 1 , further comprising determining a geometry of a room that the display is located, wherein assigning the locations of the one or more virtual speakers includes comparing dimensions of the display with the geometry of the room to optimize the locations of the one or more virtual speakers.
3. The method of claim 2 , wherein the one or more virtual speakers includes a second speaker having a location at a wall of the room that is at a side, behind, or above an estimated location of the user.
4. The method of claim 1 , wherein the camera is integrated with the headphone set.
5. The method of claim 4 , wherein determining the location of the display includes processing the one or more images with an object detection algorithm to recognize the display in the one or more images.
6. The method of claim 1 , wherein the one or more virtual speakers includes a second virtual speaker located at a right edge of the display, and a third virtual speaker located at a left edge of the display.
7. The method of claim 1 , wherein the first virtual speaker is located at a center of the display, and the one or more virtual speakers includes a second virtual speaker located at a right edge of the display, a third virtual speaker located at a left edge of the display, a fourth virtual speaker located at a right side or behind an estimated location of the user, and a fifth virtual speaker located at a left side or behind the estimated location of the user.
8. The method of claim 1 , wherein determining the position of the head includes generating one or more images with a camera of the headphone set and analyzing the one or more images to determine a location and orientation of the head.
9. The method of claim 8 , wherein the device is a media server that receives the one or more images from the headphone set over a networked connection.
10. The method of claim 8 , wherein determining the position of the head includes recognizing, in the one or more images, the display, and using the display as a reference to calculate a location and orientation of the head.
11. The method of claim 10 , wherein visual information that is displayed on the display is compared to received visual information to help in recognizing the display in the one or more images when determining the position of the head.
12. The method of claim 1 , further comprising
generating a map of a room where the display is located, based on analyzing one or more images, the map including a display location and geometry of the room; and
using the map as a reference when determining the position of the head of the user.
13. The method of claim 1 , further comprising repeating the method for a different user in the same listening area as the user, wherein one of the one or more virtual speakers of the different user has a location that is different from any of the locations of the one or more virtual speakers of the user.
14. The method of claim 1 , wherein
determining the location of the display and assigning the locations of the one or more virtual speakers is performed during an initial process, and
determining the position of the head and spatializing the one or more audio signals is performed repeatedly and concurrent with the playback of the one or more audio signals, to continuously track changing positions of the head and spatialize the one or more audio signals based on the changing positions of the head.
15. The method of claim 1 , further comprising
upmixing the one or more audio signals, wherein spatializing is performed on the upmixed versions of the one or more audio signals.
16. The method of claim 1 , wherein one or more filters of the spatial renderer are modified based on acoustic characteristics of a room that the display is located.
17. The method of claim 1 , wherein placement of the one or more virtual speakers is optimized based on a history of user location or avoiding hallways, windows, or other openings of a room that the display is located.
18. The method of claim 1 , wherein placement of a specified virtual speaker of the one or more virtual speakers is adjusted to be beyond a wall of a room that the display is located, based on the location of the user relative to the specified one of the one or more virtual speakers, to increase a distance between the user and the specified virtual speaker.
19. A system for virtualizing speakers, comprising:
a processor; and
non-transitory memory having stored therein a plurality of instructions, that when executed by the processor, perform the following:
determining a location of a display including detecting the display in one or more images generated by a camera;
assigning locations of one or more virtual speakers based on the location of the display, wherein the one or more virtual speakers include a virtual speaker located at the display;
determining, in real-time, a position of a head of a user; and
spatializing, based on the real-time position of the head relative to the locations of the one or more virtual speakers, one or more audio signals with a spatial renderer to generate spatialized audio signals that, when used to drive a left speaker and a right speaker of a headphone set, are converted to sound that is perceived by the user to be located at the assigned locations of the one or more virtual speakers.
20. An article of manufacture, comprising:
a processor; and
non-transitory memory having stored therein a plurality of instructions, that when executed by the processor, perform the following:
determining a location of a display including detecting the display in one or more images generated by a camera;
assigning locations of one or more virtual speakers based on the location of the display, wherein the one or more virtual speakers include a virtual speaker located at the display;
determining, in real-time, a position of a head of a user, based on a stream of tracking data; and
spatializing, based on the real-time position of the head relative to the locations of the one or more virtual speakers, one or more audio signals with a spatial renderer to generate spatialized audio signals that, when used to drive a left speaker and a right speaker of a headphone set, are converted to sound that is perceived by the user to be located at the assigned locations of the one or more virtual speakers, wherein the camera is separate from the headphone set.