Surface appropriate collisions
Disclosed herein are systems and methods for presenting an audio signal associated with presentation of a virtual object colliding with a surface. The virtual object and the surface may be associated with a mixed reality environment. Generation of the audio signal may be based on at least one of an audio stream from a microphone and a video stream form a sensor. In some embodiments, the collision between the virtual object and the surface is associated with a footstep on the surface.
1. A system comprising:
a microphone;
a display;
an audio output;
one or more sensors; and
one or more processors configured to perform a method comprising:
detecting, via the microphone, a first audio signal associated with a first contact of a first object with a surface, wherein the first object is associated with a user;
detecting, via the one or more sensors, a movement associated with the first contact;
determining whether the movement associated with the first contact exceeds a movement threshold;
in accordance with a determination that the movement associated with the first contact exceeds the movement threshold:
determining, based on the first audio signal, an acoustic property of the surface;
detecting a second contact of a second object with the surface; and
in accordance with said detecting the second contact:
presenting to the user, on the display, a view of the second object;
generating a second audio signal based on the second contact and based further on the acoustic property of the surface; and
presenting the second audio signal to the user via the audio output concurrently with presenting the view of the second object; and
in accordance with a determination that the movement associated with the first contact does not exceed the movement threshold, forgoing determining, based on the first audio signal, the acoustic property of the surface.
2. The system of claim 1 , wherein the first object comprises a body part of the user.
3. The system of claim 2 , wherein the body part comprises a foot of the user.
4. The system of claim 1 , further comprising an inertial measurement unit, wherein said detecting the first contact is based on inertial data of the user captured by the inertial measurement unit.
5. The system of claim 1 , further comprising a wearable auxiliary device and a wearable head device, wherein:
the wearable auxiliary device comprises the one or more processors, and
the auxiliary device is coupled to the wearable head device.
6. The system of claim 1 , wherein the system further comprises a wearable head device and wherein the method further comprises:
in accordance with said detecting the second contact:
determining a position of the wearable head device of the system; and
determining a position of the second contact based on the position of the wearable head device of the system;
wherein said generating the second audio signal is further based on the position of the second contact.
7. The system of claim 6 , wherein the view of the second object comprises a view of the second object in contact with the surface at the position of the second contact.
8. The system of claim 1 , wherein said generating the second audio signal is further based on at least one of a physical model of the surface and a physical model of the second object.
9. The system of claim 1 , wherein said generating the second audio signal is further based on an analysis of an audio stream associated with the second audio signal.
10. The system of claim 1 , wherein the method further comprises associating the second audio signal with the surface.
11. The system of claim 1 , wherein the second audio signal corresponds to the second contact of the second object with the surface.
12. The system of claim 1 , wherein the system further comprises a memory and wherein said generating the second audio signal is further based on an audio signal stored in the memory.
13. The system of claim 1 , wherein the method further comprises:
detecting a third contact with a second surface;
in accordance with said detection of the third contact:
identifying a portion of an audio stream associated with the second audio signal;
generating a third audio signal based on the portion of the audio stream and based further on the acoustic property of the surface; and
presenting to the user, via the audio output, the third audio signal.
14. The system of claim 1 , wherein:
the second audio signal is associated with a video stream,
the video stream includes information associated with the surface, and
said generating the second audio signal is based further on the information.
15. The system of claim 1 , wherein the second object comprises a virtual object.
16. The system of claim 1 , wherein the acoustic property comprises a property of a material of the surface.
17. The system of claim 16 , wherein said determining the acoustic property is based on a coefficient of absorption of the material.
18. A method comprising:
detecting, via a microphone, a first audio signal associated with a first contact of a first object with a surface, wherein the first object is associated with a user;
detecting, via one or more sensors, a movement associated with the first contact;
determining whether the movement associated with the first contact exceeds a movement threshold;
in accordance with a determination that the movement associated with the first contact exceeds the movement threshold:
determining, based on the first audio signal, an acoustic property of the surface;
detecting a second contact of a second object with the surface; and
in accordance with said detecting the second contact:
presenting to the user, on a display, a view of the second object;
generating a second audio signal based on the second contact and based further on the acoustic property of the surface; and
presenting the second audio signal to the user via an audio output concurrently with presenting the view of the second object; and
in accordance with a determination that the movement associated with the first contact does not exceed the movement threshold, forgoing determining, based on the first audio signal, the acoustic property of the surface.
19. A non-transitory computer-readable medium storing one or more instructions, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:
detecting, via a microphone, a first audio signal associated with a first contact of a first object with a surface, wherein the first object is associated with a user;
detecting, via the one or more sensors, a movement associated with the first contact;
determining whether the movement associated with the first contact exceeds a movement threshold;
in accordance with a determination that the movement associated with the first contact exceeds the movement threshold:
determining, based on the first audio signal, an acoustic property of the surface;
detecting a second contact of a second object with the surface; and
in accordance with said detecting the second contact:
presenting to the user, on a display, a view of the second object;
generating a second audio signal based on the second contact and based further on the acoustic property of the surface; and
presenting the second audio signal to the user via an audio output concurrently with presenting the view of the second object; and
in accordance with a determination that the movement associated with the first contact does not exceed the movement threshold, forgoing determining, based on the first audio signal, the acoustic property of the surface.