Techniques for providing non-verbal speech recognition in an immersive playtime environment
An immersive play environment platform including techniques describing recognizing non-verbal vocalization gestures from a user is disclosed. A headset device receives audio input from a user. The headset device transmits the audio input to a controller device. The controller device evaluates characteristics of the audio input (e.g., spectral features over a period of time) to determine whether the audio input corresponds to a predefined non-verbal vocalization, such as a humming noise, shouting noise, etc. The controller device may perform an action in response to detecting such non-verbal vocalizations, such as engaging a play object (e.g., an action figure, an action disc) in the play environment.
1. An immersive play experience platform providing an interactive environment, comprising:
an audio device configured to receive audio input; and
a controller device configured to perform an operation for recognizing non-verbal vocalizations, the operation comprising:
receiving audio input transmitted by the audio device,
determining that the audio input matches spectral features of one or more predefined non-verbal vocalizations characterizing gameplay within the interactive environment, and
causing an action corresponding to the one or more predefined non-verbal vocalizations to be performed on an interactive gameplay object within the interactive environment.
2. The platform of claim 1 , further comprising:
a physical device configured to receive instructions from the controller device to perform the corresponding action; and
wherein the interactive gameplay object is connected with the physical device.
3. The platform of claim 2 , wherein performing the corresponding action comprises:
causing the interactive gameplay object to disconnect from the physical device.
4. The platform of claim 1 , wherein the predefined non-verbal vocalizations include at least one of a humming gesture or a shouting gesture.
5. The platform of claim 1 , further comprising:
a sensor device having an accelerometer and a gyroscope, the sensor device configured to receive position and orientation input and to transmit the position and orientation input to the controller device.
6. The platform of claim 5 , wherein the one or more predefined non-verbal vocalizations are associated with a choreographed sequence having one or more gestures corresponding to specified position and orientation input transmitted from the sensor device.
7. The platform of claim 1 , wherein the predefined non-verbal vocalizations are further determined based on amplitude and timing thresholds measured in the audio input.
8. A method for recognizing non-verbal vocalizations, comprising:
receiving, by a controller device executing on an immersive play experience platform providing an interactive environment, audio input;
determining, by the controller device, that the audio input matches spectral features of one or more predefined non-verbal vocalizations characterizing gameplay within the interactive environment; and
causing an action corresponding to the one or more predefined non-verbal vocalizations to be performed on an interactive gameplay object within the interactive environment.
9. The method of claim 8 , further comprising, transmitting instructions to a physical device for performing the corresponding action.
10. The method of claim 9 , wherein performing the corresponding action comprises:
causing the interactive gameplay object to disconnect from the physical device.
11. The method of claim 8 , wherein the predefined non-verbal vocalizations include at least one of a humming gesture or a shouting gesture.
12. The method of claim 8 , further comprising, receiving position and orientation input.
13. The method of claim 12 , wherein the one or more predefined non-verbal vocalizations are associated with a choreographed sequence having one or more gestures corresponding to specified position and orientation input.
14. The method of claim 8 , wherein the predefined non-verbal vocalizations are further determined based on amplitude and timing thresholds measured in the audio input.
15. A non-transitory computer-readable storage medium storing instructions, which, when executed on a processor, perform an operation for recognizing non-verbal vocalizations, the operation comprising:
receiving, by a controller device executing on an immersive play experience platform providing an interactive environment, audio input;
determining, by the controller device, that the audio input matches spectral features of at least a first predefined non-verbal vocalization characterizing gameplay within the interactive environment;
determining that the matched audio input is received as a part of a predefined sequence of gestures including at least a first physical gesture and the first predefined non-verbal vocalization, and
causing an action corresponding to the predefined sequence of gestures to be performed on an interactive gameplay object.
16. The computer-readable storage medium of claim 15 , wherein the operation further comprises, transmitting instructions to a physical device for performing the corresponding action.
17. The computer-readable storage medium of claim 16 , wherein performing the corresponding action comprises:
causing, via the physical device, the interactive gameplay object to disconnect from the physical device.
18. The computer-readable storage medium of claim 15 , wherein the operation further comprises, receiving position and orientation input.
19. The computer-readable storage medium of claim 18 , wherein the at least the first physical gesture matches specified position and orientation input.
20. The computer-readable storage medium of claim 15 , wherein the at least the first predefined non-verbal vocalization is further determined based on amplitude and timing thresholds measured in the audio input.