Method and apparatus for rendering an audio source having a modified virtual position
View Patent ↗A method, apparatus and computer program product are provided to cause AN audio source to be modified in a manner consistent with the corresponding video images once the user and/or a display upon which the images are rendered has been tilted. In regards to a method, an initial virtual position of an audio source is determined. The method also includes determining a tilt angle that defines an angle that an apparatus embodying a display for rendering images has been tilted relative to a reference orientation of the apparatus with respect to a user of the apparatus. The method may also include modifying a virtual position of an audio source based upon the tilt angle and an initial virtual position of the audio source. The method may also include causing the audio source to be rendered in accordance with the virtual position as modified.
1. A method comprising:
determining whether an application or content that includes an audio source desires that a virtual position of the audio source is modified in response to tilting; and
selectively modifying the virtual position of the audio source such that the virtual position of the audio source is not modified in response to tilting in an instance in which the application or content does not desire modification but is modified in response to titling in an instance in which the application or content does desire modification, wherein, in an instance in which the application or content does desire modification, modifying the virtual position of the audio source by:
determining the virtual position of the audio source as a function of time based upon information associated with the audio source, wherein the audio source is embodied as one or more audio objects with the one or more audio objects associated with at least one waveform and the virtual position of the audio source as the function of time such that determining the virtual position comprises determining the virtual position from information associated with the one or more audio objects that embody the audio source;
determining a tilt angle that defines an angle that an apparatus embodying a display for rendering images has been tilted relative to a reference orientation of the apparatus, wherein determining the tilt angle comprises determining the tilt angle based upon data provided by one or more sensors that detect an orientation of the display, and wherein the tilt angle includes an angle that the apparatus embodying the display has been physically tilted;
modifying, with a processor, the virtual position of the audio source based upon the tilt angle and the virtual position of the audio source; and
causing the audio source to be rendered in accordance with the virtual position as modified such that the at least one waveform associated with the one or more audio objects that embody the audio source are output with the virtual position as modified.
2. A method according to claim 1 wherein modifying the virtual position comprises combining the tilt angle with the virtual position of the audio source that is based upon information associated with the audio source.
3. A method according to claim 1 wherein determining the tilt angle further comprises:
capturing an image of the user from a vantage point of the display for rendering images; and
determining the tilt angle based upon a predefined feature of the user within the image.
4. A method according to claim 3 wherein the predefined feature comprises eyes of the user.
5. A method according to claim 1 further comprising modifying a gain of one or more audio channels.
6. A method according to claim 1 wherein causing the audio source to be rendered comprises causing the audio source to be rendered by headphones in accordance with the virtual position as modified.
7. A method according to claim 1 wherein the virtual position of the audio source corresponds to a visual object at a position within a corresponding image from which the one or more audio objects that embody the audio source appear to originate, and wherein in an instance in which visual object moves relative to the display, modifying the virtual position of the audio source comprises modifying the virtual position of the audio source such that the audio source appears to move with the visual object.
8. A method according to claim 1 wherein causing the audio source to be rendered comprises causing the audio source to be rendered by a plurality of speakers having height channels, and wherein modifying the virtual position of the audio source comprises modifying respective channel gains of the height channels of the speakers based upon the tilt angle such that different height channels of a respective speaker have different channel gains based upon the tilt angle.
9. A method according to claim 1 wherein modifying the virtual position of the audio source comprises upmixing audio content of the audio source in an audio format that lacks height channels to create audio content of a different audio format having height channels, modifying respective channel gains of the height channels based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle, and combining the virtual position of the audio source with the tilt angle in order to modify the virtual position, and wherein causing the audio source to be rendered comprises causing the audio source to be rendered in accordance with both the virtual position as modified and the height channels having respective gains as modified.
10. A method according to claim 1 wherein modifying the virtual position of the audio source comprises upmixing audio content of the audio source in an audio format that lacks height channels to create audio content of a different audio format having height channels to create audio content of a different audio format having height channels and thereafter modifying respective channel gains of the height channels based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle, and wherein causing the audio source to be rendered comprises causing the audio source to be rendered in accordance with both the virtual position as modified and the height channels having respective gains as modified.
11. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least:
determine whether an application or content that includes an audio source desires that a virtual position of the audio source is modified in response to tilting; and
selectively modify the virtual position of the audio source such that the virtual position of the audio source is not modified in response to tilting in an instance in which the application or content does not desire modification but is modified in response to titling in an instance in which the application or content does desire modification, wherein, in an instance in which the application or content does desire modification, the apparatus is caused to modify the virtual position of the audio source by:
determining the virtual position of the audio source as a function of time based upon information associated with the audio source, wherein the audio source is embodied as one or more audio objects with the one or more audio objects associated with at least one waveform and the virtual position of the audio source as the function of time such that the virtual position is determined from information associated with the one or more audio objects that embody the audio source;
determining a tilt angle that defines an angle that a display for rendering images has been tilted relative to a reference orientation of the display, wherein the tilt angle is determined based upon data provided by one or more sensors that detect an orientation of the display, and wherein the tilt angle includes an angle that the display has been physically tilted;
modifying the virtual position of the audio source based upon the tilt angle and the virtual position of the audio source; and
causing the audio source to be rendered in accordance with the virtual position as modified such that the at least one waveform associated with the one or more audio objects that embody the audio source are output with the virtual position as modified.
12. An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to modify the virtual position by combining the tilt angle with the virtual position of the audio source that is based upon information associated with the audio source.
13. An apparatus according to claim 11 wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to determine the tilt angle by:
capturing an image of the user from a vantage point of the display for rendering images; and
determining the tilt angle based upon a predefined feature of the user within the image.
14. An apparatus according to claim 13 wherein the predefined feature comprises eyes of the user.
15. An apparatus according to claim 11 wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to modify a gain of one or more audio channels.
16. An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the audio source to be rendered by causing the audio source to be rendered by headphones in accordance with the virtual position as modified.
17. An apparatus according to claim 11 wherein the virtual position of the audio source corresponds to a visual object at a position within a corresponding image from which the one or more audio objects that embody the audio source appear to originate, and wherein in an instance in which visual object moves relative to the display, the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to modify the virtual position of the audio source by modifying the virtual position of the audio source such that the audio source appears to move with the visual object.
18. An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the audio source to be rendered by causing the audio source to be rendered by a plurality of speakers having height channels, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to modify the virtual position of the audio source by modifying respective channel gains of the height channels of the speakers based upon the tilt angle such that different height channels of a respective speaker have different channel gains based upon the tilt angle.
19. An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to modify the virtual position of the audio source by upmixing audio content of the audio source in an audio format that lacks height channels to create audio content of a different audio format having height channels, modifying respective channel gains of the height channels based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle, and combining the virtual position of the audio source with the tilt angle in order to modify the virtual position, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the audio source to be rendered by causing the audio source to be rendered in accordance with both the virtual position as modified and the height channels having respective gains as modified.
20. An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to modify the virtual position of the audio source by upmixing audio content of the audio source in an audio format that lacks height channels to create audio content of a different audio format having height channels to create audio content of a different audio format having height channels and thereafter modifying respective channel gains of the height channels based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the audio source to be rendered by causing the audio source to be rendered in accordance with both the virtual position as modified and the height channels having respective gains as modified.
21. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein, the computer-executable program code portions comprising program code instructions to:
determine whether an application or content that includes an audio source desires that a virtual position of the audio source is modified in response to tilting; and
selectively modify the virtual position of the audio source such that the virtual position of the audio source is not modified in response to tilting in an instance in which the application or content does not desire modification but is modified in response to titling in an instance in which the application or content does desire modification, wherein, in an instance in which the application or content does desire modification, the apparatus is caused to modify the virtual position of the audio source by:
determining the virtual position of the audio source as a function of time based upon information associated with the audio source, wherein the audio source is embodied as one or more audio objects with the one or more audio objects associated with at least one waveform and the virtual position of the audio source as the function of time such that determining the virtual position comprises determining the virtual position from information associated with the one or more audio objects that embody the audio source;
determining a tilt angle that defines an angle that an apparatus embodying a display for rendering images has been tilted relative to a reference orientation of the apparatus, wherein determining the tilt angle comprises determining the tilt angle based upon data provided by one or more sensors that detect an orientation of the display, and wherein the tilt angle includes an angle that the apparatus embodying the display has been physically tilted;
modifying the virtual position of the audio source based upon the tilt angle and the virtual position of the audio source; and
causing the audio source to be rendered in accordance with the virtual position as modified such that the at least one waveform associated with the one or more audio objects that embody the audio source are output with the virtual position as modified.
22. A computer program product according to claim 21 wherein the program code instructions for modifying the virtual position comprise program code instructions for combining the tilt angle with the virtual position of the audio source that is based upon information associated with the audio source.
23. A computer program product according to claim 21 wherein the program code instructions for determining the tilt angle further comprise program code instructions for:
capturing an image of the user from a vantage point of the display for rendering images; and
determining the tilt angle based upon a predefined feature of the user within the image.
24. A computer program product according to claim 21 wherein the computer-executable program code portions further comprise program code instructions for modifying a gain of one or more audio channels.
25. A computer program product according to claim 21 wherein the program code instructions for causing the audio source to be rendered comprise program code instructions for causing the audio source to be rendered by headphones in accordance with the virtual position as modified.
26. A computer program code according to claim 21 wherein the program code instructions for causing the audio source to be rendered comprise program code instructions for causing the audio source to be rendered by a plurality of speakers having height channels, and wherein the program code instructions for modifying the virtual position of the audio source comprise program code instructions for modifying respective channel gains of the height channels of the speakers based upon the tilt angle such that different height channels of a respective speaker have different channel gains based upon the tilt angle.
27. A computer program code according to claim 21 wherein the program code instructions for modifying the virtual position of the audio source comprise program code instructions for upmixing audio content of the audio source in an audio format that lacks height channels to create audio content of a different audio format having height channels, modifying respective channel gains of the height channels based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle, and combining the virtual position of the audio source with the tilt angle in order to modify the virtual position, and wherein the program code instructions for causing the audio source to be rendered comprise program code instructions for causing the audio source to be rendered in accordance with both the virtual position as modified and the height channels having respective gains as modified.
28. A computer program code according to claim 21 wherein the program code instructions for modifying the virtual position of the audio source comprise program code instructions for upmixing audio content of the audio source in an audio format that lacks height channels to create audio content of a different audio format having height channels to create audio content of a different audio format having height channels and thereafter modifying respective channel gains of the height channels based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle such that different height channels have different channel gains based upon the tilt angle, and wherein the program code instructions for causing the audio source to be rendered comprise program code instructions for causing the audio source to be rendered in accordance with both the virtual position as modified and the height channels having respective gains as modified.