Surface appropriate collisions
Disclosed herein are systems and methods for presenting an audio signal associated with presentation of a virtual object colliding with a surface. The virtual object and the surface may be associated with a mixed reality environment. Generation of the audio signal may be based on at least one of an audio stream from a microphone and a video stream form a sensor. In some embodiments, the collision between the virtual object and the surface is associated with a footstep on the surface.
1. A system comprising:
a microphone;
a display;
an audio output; and
one or more processors configured to execute a method comprising:
at a first time, detecting a footstep, of a user, on a surface;
in accordance with said detecting the footstep, determining, via a detection of the footstep by the microphone, an acoustic property of the surface;
at a second time, detecting a collision of a virtual object with the surface;
in accordance with said detecting the collision:
presenting to the user, on the display, a view of the collision;
generating an audio signal based on the collision and based further on the acoustic property of the surface; and
presenting to the user, via the audio output, concurrently with presenting the view of the collision, the audio signal.
2. The system of claim 1 , further comprising a sensor, wherein said detecting the footstep is based on movement data of the user captured by the sensor.
3. The system of claim 1 , further comprising an inertial measurement unit, wherein said detecting the footstep is based on inertial data of the user captured by the inertial measurement unit.
4. The system of claim 1 , further comprising a wearable auxiliary device and a wearable head device, wherein:
the wearable auxiliary device comprises the one or more processors, and
the auxiliary device is coupled to the wearable head device.
5. The system of claim 1 , wherein the system further comprises a wearable head device and wherein the method further comprises:
in accordance with said detecting the footstep:
determining a position of the wearable head device of the system; and
determining a position of the footstep based on the position of the wearable head device of the system;
wherein said generating the audio signal is further based on the position of the footstep.
6. The system of claim 5 , wherein:
the view of the collision comprises a view of the virtual object colliding with the surface at the position of the footstep.
7. The system of claim 1 , wherein said generating the audio signal is further based on at least one of a physical model of the surface and a physical model of the virtual object.
8. The system of claim 1 , wherein said generating the audio signal is further based on an analysis of an audio stream associated with the audio signal.
9. The system of claim 1 , wherein the method further comprises associating the audio signal with the surface.
10. The system of claim 1 , wherein the audio signal corresponds to the collision of the virtual object with the surface.
11. The system of claim 1 , wherein:
said generating the audio signal is further based on a stored audio signal.
12. The system of claim 1 , wherein the method further comprises:
detecting a second collision with a second surface;
in accordance with the detection of the second collision:
identifying a portion of an audio stream associated with the audio signal;
generating a second audio signal based on the portion of the audio stream and based further on the acoustic property of the surface; and
presenting to the user, via the audio output, the second audio signal.
13. The system of claim 1 , wherein:
the audio signal is associated with a video stream,
the video stream includes information associated with the surface, and
said generating the audio signal is based further on the information.
14. The system of claim 1 , wherein the virtual object comprises a foot of a virtual character.
15. The system of claim 1 , wherein:
the acoustic property comprises a property of a material of the surface.
16. The system of claim 15 , wherein said determining the acoustic property is based on a coefficient of absorption of the material.
17. A method comprising:
at a first time, detecting a footstep, of a user, on a surface;
in accordance with said detecting the footstep, determining, via a detection of the footstep by a microphone, an acoustic property of the surface;
at a second time, detecting a collision of a virtual object with the surface;
in accordance with said detecting the collision:
presenting to the user, on a display, a view of the collision;
generating an audio signal based on the collision and based further on the acoustic property of the surface; and
presenting to the user, via an audio output, concurrently with presenting the view of the collision, the audio signal.
18. The method of claim 17 , wherein said detecting the footstep is based on movement data of the user captured by a sensor.
19. The method of claim 17 , wherein said detecting the footstep is based on inertial data of the user captured by an inertial measurement unit.
20. A non-transitory computer-readable medium storing one or more instructions, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:
at a first time, detecting a footstep, of a user, on a surface;
in accordance with said detecting the footstep, determining, via a detection of the footstep by a microphone, an acoustic property of the surface;
at a second time, detecting a collision of a virtual object with the surface;
in accordance with said detecting the collision:
presenting to the user, on a display, a view of the collision;
generating an audio signal based on the collision and based further on the acoustic property of the surface; and
presenting to the user, via an audio output, concurrently with presenting the view of the collision, the audio signal.