SURFACE APPROPRIATE COLLISIONS
Disclosed herein are systems and methods for presenting an audio signal associated with presentation of a virtual object colliding with a surface. The virtual object and the surface may be associated with a mixed reality environment. Generation of the audio signal may be based on at least one of an audio stream from a microphone and a video stream form a sensor. In some embodiments, the collision between the virtual object and the surface is associated with a footstep on the surface.
1 . A system comprising:
a microphone;
one or more sensors; and
one or more processors configured to perform a method comprising:
receiving, via the microphone, a first audio signal associated with a first contact of a first object with a surface, wherein the first object is associated with a user;
determining, via the one or more sensors, a movement associated with the first contact;
determining whether the movement associated with the first contact exceeds a movement threshold;
in accordance with a determination that the movement associated with the first contact exceeds the movement threshold:
determining, based on the first audio signal, an acoustic property of the surface, wherein in response to a second contact of a second object with the surface:
a view of the second object is presented on a display;
a second audio signal is generated based on the second contact and based further on the acoustic property of the surface; and
the second audio signal is presented via an audio output concurrently with the presenting the view of the second object; and
in accordance with a determination that the movement associated with the first contact does not exceed the movement threshold, forgoing determining, based on the first audio signal, the acoustic property of the surface.
2 . The system of claim 1 , wherein the first object comprises a body part of the user.
3 . The system of claim 2 , wherein the body part comprises a foot of the user.
4 . The system of claim 1 , further comprising an inertial measurement unit, wherein the first contact is determined based on inertial data of the user captured by the inertial measurement unit.
5 . The system of claim 1 , further comprising a wearable auxiliary device and a wearable head device, wherein:
the wearable auxiliary device comprises the one or more processors, and
the auxiliary device is coupled to the wearable head device.
6 . The system of claim 1 , wherein the system further comprises a wearable head device and wherein in accordance with determining the second contact:
a position of the wearable head device is determined; and
a position of the second contact based on the position of the wearable head device is determined;
wherein said generating the second audio signal is further based on the position of the second contact.
7 . The system of claim 6 , wherein the view of the second object comprises a view of the second object in contact with the surface at the position of the second contact.
8 . The system of claim 1 , wherein said generating the second audio signal is further based on at least one of a physical model of the surface and a physical model of the second object.
9 . The system of claim 1 , wherein said generating the second audio signal is further based on an analysis of an audio stream associated with the second audio signal.
10 . The system of claim 1 , wherein the method further comprises associating the second audio signal with the surface.
11 . The system of claim 1 , wherein the second audio signal corresponds to the second contact of the second object with the surface.
12 . The system of claim 1 , wherein the system further comprises a memory and wherein said generating the second audio signal is further based on an audio signal stored in the memory.
13 . The system of claim 1 , wherein the method further comprises:
in accordance with determination of a third contact:
a portion of an audio stream associated with the second audio signal is identified;
a third audio signal based on the portion of the audio stream and based further on the acoustic property of the surface is generated; and
the third audio signal is presented via the audio output.
14 . The system of claim 1 , wherein:
the second audio signal is associated with a video stream,
the video stream includes information associated with the surface, and
said generating the second audio signal is based further on the information.
15 . The system of claim 1 , wherein the second object comprises a virtual object.
16 . The system of claim 1 , wherein the acoustic property comprises a property of a material of the surface.
17 . The system of claim 16 , wherein said determining the acoustic property is based on a coefficient of absorption of the material.
18 . The system of claim 1 , wherein the system comprises the display and the audio output.
19 . A method comprising:
receiving, via a microphone, a first audio signal associated with a first contact of a first object with a surface, wherein the first object is associated with a user;
determining, via one or more sensors, a movement associated with the first contact;
determining whether the movement associated with the first contact exceeds a movement threshold;
in accordance with a determination that the movement associated with the first contact exceeds the movement threshold:
determining, based on the first audio signal, an acoustic property of the surface, wherein in response to a second contact of a second object with the surface:
a view of the second object is presented on a display;
a second audio signal is generated based on the second contact and based further on the acoustic property of the surface; and
the second audio signal is presented via an audio output concurrently with the presenting the view of the second object; and
in accordance with a determination that the movement associated with the first contact does not exceed the movement threshold, forgoing determining, based on the first audio signal, the acoustic property of the surface.
20 . A non-transitory computer-readable medium storing one or more instructions, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:
receiving, via a microphone, a first audio signal associated with a first contact of a first object with a surface, wherein the first object is associated with a user;
determining, via one or more sensors, a movement associated with the first contact;
determining whether the movement associated with the first contact exceeds a movement threshold;
in accordance with a determination that the movement associated with the first contact exceeds the movement threshold:
determining, based on the first audio signal, an acoustic property of the surface, wherein in response to a second contact of a second object with the surface:
a view of the second object is presented on a display;
a second audio signal is generated based on the second contact and based further on the acoustic property of the surface; and
the second audio signal is presented via an audio output concurrently with the presenting the view of the second object; and
in accordance with a determination that the movement associated with the first contact does not exceed the movement threshold, forgoing determining, based on the first audio signal, the acoustic property of the surface.