Sound Based Modification Of A Virtual Environment
A computing device for modifying a virtual environment based on a detected sound, the computing device comprising: a microphone; a display configured to display a virtual environment to a user; and a processor; wherein the processor is configured to: receive, via the microphone, audio data of audio in a monitored environment; use the audio data to determine the occurrence of a non-speech target sound in the monitored environment; determine a modification of the virtual environment associated with the non-speech target sound; and control the display to implement the determined modification.
1 . A computing device for modifying a virtual environment based on a detected sound, the computing device comprising:
a microphone;
a display configured to display a virtual environment to a user;
and
a processor;
wherein the processor is configured to:
receive, via the microphone, audio data of audio in a monitored environment;
use the audio data to determine the occurrence of a non-speech target sound in the monitored environment;
determine a modification of the virtual environment associated with the non-speech target sound; and
control the display to implement the determined modification.
2 . The computing device of claim 1 further comprising a user input device, wherein the virtual environment comprises a virtual scene and a user controlled object positioned within the virtual scene, the user controlled object controllable by the user using the user input device.
3 . The computing device of claim 2 , wherein the determined modification comprises a modification of the virtual scene.
4 . The computing device of claim 3 , wherein the determined modification comprises a modification of a lighting level of the virtual scene.
5 . The computing device of claim 3 , wherein the determined modification comprises a modification of the appearance of scenery of the virtual scene.
6 . The computing device of claim 3 , wherein the determined modification comprises insertion of at least one new virtual object into the virtual scene.
7 . The computing device of claim 1 , wherein the determined modification comprises a modification of the user controlled object.
8 . The computing device of claim 7 , wherein the determined modification comprises at least one of: a modification of an expression of the user controlled object, a modification of a colour of the user controlled object, a visual animation of the user controlled object, and adding a new object to the user controlled object.
9 . The computing device of claim 1 , wherein the user controlled object is an avatar.
10 . The computing device of claim 1 , wherein the processor is further configured to generate metadata relating to the non-speech target sound, and determine the modification based on the metadata.
11 . The computing device of claim 10 , wherein the metadata comprises labels describing at least one of an intensity of the non-speech target sound, a temporal pattern of the non-speech target sound, a count of a number of occurrences of the non-speech target sound, and a duration of the non-speech target sound.
12 . The computing device of claim 10 , wherein the metadata comprises quantitative measurements of an intensity of the non-speech target sound, a temporal pattern of the non-speech target sound, a count of a number of occurrences of the non-speech target sound, and a duration of the non-speech target sound.
13 . The computing device of claim 1 , wherein the at least one target sound conveys semantic information.
14 . The computing device of claim 1 , wherein the virtual environment is an electronic video game environment.
15 . The computing device of claim 1 , wherein the virtual environment is part of a metaverse virtual world.
16 . The computing device of claim 1 , wherein the display is a virtual reality display or an augmented reality device.
17 . The computing device of claim 1 , wherein the computing device further comprises a memory storing one or more sound models, wherein the sound models correspond to one or more non-speech target sounds; and
the processor is configured to determine the occurrence of a non-speech target sound in the monitored environment by comparing the audio data to said one or more sound models to recognize a non-speech target sound of said one or more non-speech target sounds in the monitored environment.
18 . The computing device of claim 17 , wherein each of the one or more sound models correspond to a respective non-speech target sound.
19 . A method of modifying a virtual environment based on a detected sound, the method implemented on a processor contained in a computing device and comprising:
displaying a virtual environment to a user using a display;
receiving audio data of audio in a monitored environment using a microphone;
using the audio data to determine the occurrence of a non-speech target sound in the monitored environment;
determining a modification of the virtual environment associated with the non-speech target sound; and
controlling the display to implement the determined modification.
20 . A non-transitory computer-readable storage medium comprising instructions which, when executed by a processor of a computing device, cause the computing device to perform a method comprising:
displaying a virtual environment to a user using a display;
receiving audio data of audio in a monitored environment using a microphone;
using the audio data to determine the occurrence of a non-speech target sound in the monitored environment;
determining a modification of the virtual environment associated with the non-speech target sound; and
controlling the display to implement the determined modification.