Extraction and classification of audio events in gaming systems
A system that incorporates the subject disclosure may include, for example, receiving an input audio stream from a gaming system, the input audio stream including gaming audio of a video game played by a game player, the input audio stream including a plurality of classes of sounds, providing the input audio stream to a neural network, extracting, by the neural network, sounds of a selected class of sounds of the plurality of classes of sounds, and providing a plurality of output audio streams including providing a first audio stream including the sounds of the selected class of sounds of the input audio stream and a second audio stream including remaining sounds of the input audio stream. Additional embodiments are disclosed.
1. A device, comprising:
a processing system including a processor; and
a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising:
receiving an input audio stream from a gaming system, the input audio stream including gaming audio of a video game played by a game player, the input audio stream including a plurality of classes of sounds;
providing the input audio stream to a neural network;
extracting, by the neural network, sounds of a selected class of sounds of the plurality of classes of sounds;
providing a plurality of output audio streams including providing a first audio stream including the sounds of the selected class of sounds of the input audio stream and a second audio stream including remaining sounds of the input audio stream;
receiving control information from the game player;
modifying one or more of the plurality of output audio streams, forming modified output audio, wherein the modifying includes modifying a gain of sounds of a selected class of sounds according to the control information from the game player to form the modified output audio; and
providing the modified output audio to the game player.
2. The device of claim 1 , wherein the receiving an input audio stream from a gaming system comprises receiving sounds of the gaming audio having an apparent location, and wherein the providing a first audio stream comprises providing the sounds of the selected class of sounds with a same apparent location as the sounds of the gaming audio.
3. The device of claim 2 , wherein the receiving an input audio stream from a gaming system comprises receiving a multichannel audio signal from the gaming system.
4. The device of claim 1 , wherein the receiving an input audio stream from a gaming system comprises receiving surround sound audio from the gaming system and wherein the providing a first audio stream comprises providing the sounds of the selected class of sounds in surround sound audio.
5. The device of claim 2 , wherein the operations further comprise:
receiving footstep sounds having an apparent direction of origin in relation to the game player;
extracting the footstep sounds from the input audio stream; and
providing the first audio stream including the footstep sounds to the game player with same apparent direction of origin in relation to the game player.
6. The device of claim 1 , wherein the modified output audio includes footstep sounds.
7. The device of claim 6 , wherein the modified output audio includes gunshot sounds.
8. The device of claim 6 , wherein the modifying one or more of the plurality of output audio streams comprises:
equalizing sounds of a selected class of sounds according to the control information from the game player to form the modified output audio.
9. The device of claim 6 , wherein the modifying one or more of the plurality of output audio streams comprises:
modifying sounds of a selected class of sounds according to the control information from the game player, forming modified sounds of interest;
modifying sounds of the remaining sounds of the input audio stream according to the control information from the game player, forming modified remaining sounds;
combining the modified sounds of interest and the modified remaining sounds to form the modified output audio, forming combined output audio; and
providing the combined output audio to the game player.
10. The device of claim 9 , wherein the operations further comprise:
providing one of the combined output audio and the input audio stream to the game player according to the control information from the game player.
11. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, the operations comprising:
receiving an input audio stream from a gaming system, the input audio stream including gaming audio of a video game played by a game player, the input audio stream including a plurality of classes of sounds;
extracting, from the input audio stream, sounds of a selected class of sounds of the plurality of classes of sounds;
receiving audio processing control information from the game player;
providing a plurality of output audio streams to the game player according to the audio processing control information, including selectively providing one of a first audio stream including the sounds of the selected class of sounds of the input audio stream and a second audio stream including remaining sounds of the input audio stream;
receiving gain control and equalization control information from the game player;
modifying a gain or a frequency spectrum of the sounds of the selected class of sounds according to the gain control and equalization control information from the game player, forming modified output sounds; and
providing the modified output sounds to the game player.
12. The non-transitory machine-readable medium of claim 11 , wherein the modified output sounds include at least one of footsteps or gunshot sounds.
13. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise:
selecting the selected class of sounds in response to the audio processing control information from the game player.
14. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise:
receiving a multichannel audio stream as the input audio stream from a gaming system, the multichannel audio stream including location information of sounds of the gaming audio of the video game; and
providing the plurality of output audio streams to the game player with same location information of the sounds of the gaming audio of the video game.
15. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise:
selecting one of footstep sounds and gunshot sounds of the gaming audio of the video game as the selected class of sounds, wherein the selecting is responsive to the audio processing control information from the game player.
16. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise:
providing the input audio stream to a neural network; and
receiving, from the neural network, the sounds of a selected class of sounds of the plurality of classes of sounds.
17. A method, comprising:
receiving, by a processing system including a processor, an input audio stream of a gaming system, the input audio stream including gaming audio of a video game played by a game player, the input audio stream including gaming sounds, the gaming sounds organizable as a plurality of classes of sounds;
providing, by the processing system, the input audio stream to a neural network, the neural network trained to recognize gaming sounds of at least one class of the plurality of classes of sounds;
receiving, by the processing system, from the neural network, extracted gaming sounds of the at least one class of the plurality of classes of sounds;
providing, by the processing system, to the game player, a first output audio stream including the extracted gaming sounds and a second output audio stream including remaining sounds of the gaming audio of the video game;
receiving, by the processing system, audio processing control information from the game player; and
modifying, by the processing system, the extracted gaming sounds according to the audio processing control information, wherein the modifying the extracted gaming sounds comprises:
enhancing, by the processing system, the extracted gaming sounds of the gaming audio of the video game; reducing, by the processing system, the remaining sounds of the gaming audio of the video game; or a combination thereof.
18. The method of claim 17 , wherein the extracted gaming sounds include footstep sounds.
19. The method of claim 18 , wherein the modifying the extracted gaming sounds comprises:
modifying, by the processing system, a gain of the extracted gaming sounds;
equalizing, by the processing system, the extracted gaming sounds; or
compressing, by the processing system, the extracted gaming sounds, or a combination of these.
20. The method of claim 18 , wherein the extracted gaming sounds include footstep sounds.