Multi-perspective augmented reality experience
A volumetric video of a three-dimensional space is captured. The volumetric video of the three-dimensional space includes a volumetric representation of a first user within the three-dimensional space. The volumetric video is displayed by a display device worn by a second user, and the second user sees the volumetric representation of the first user within the three-dimensional space. Input indicative of an interaction (e.g., entering or leaving) of the second user with the volumetric representation of the first user is detected. Based on detecting the input indicative of the interaction, the display device switches to a display of a recorded perspective of the first user. Thus, by interacting with a volumetric representation of the first user in a volumetric video, the second user views the first user's perspective of the three-dimensional space.
1 . A method comprising:
accessing a first volumetric video of a three-dimensional space comprising a volumetric representation of a first user within the three-dimensional space;
accessing a second volumetric video of the three-dimensional space, the second volumetric video being recorded at a perspective of the first user within the three-dimensional space;
causing a display device of a second user to present the first volumetric video of the three-dimensional space comprising the volumetric representation of the first user to the second user;
detecting that a real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video, wherein the interacting comprises that the real-life body of the second user at least partially overlaps with the volumetric representation of the first user; and
in response to detecting that the real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video,
causing the display device to switch to present the second volumetric video to the second user at the perspective of the first user.
2 . The method of claim 1 , wherein detecting that a real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video further comprises:
determining a gaze of the second user is directed at the volumetric representation of the first user.
3 . The method of claim 1 , wherein the recorded perspective of the first user further comprises an audio signal comprising audio provided by the first user during capture of the first volumetric video or the second volumetric video, wherein the causing presentation of the second volumetric video at the perspective of the first user comprises causing presentation of the audio signal.
4 . The method of claim 3 , wherein the audio signal comprises a conversation between the first user and the second user during the capture of the first volumetric video or the second volumetric video.
5 . The method of claim 1 , wherein:
the three-dimensional space corresponds to a real-world location;
the display device is worn by the second user at the real-world location; and
the presentation of the first volumetric video comprises a presentation of the volumetric representation of the first user overlaid on the real-world location at a position in the real-world location corresponding to a position of the first user in the real-world location during capture of the first volumetric video or the second volumetric video.
6 . The method of claim 1 , wherein:
the three-dimensional space corresponds to a first real-world location;
the display device is worn by the second user at a second real-world location; and
the presentation of the first volumetric video comprises a presentation of the volumetric representation of the first user at a position related to the second real-world location corresponding to a position of the first user in the real-world location during capture of the volumetric video.
7 . The method of claim 6 , wherein causing presentation of the first volumetric video by the display device comprises causing presentation of one or more AR content items overlaid on one or more elements of the real-world location.
8 . The method of claim 1 , wherein:
the three-dimensional space corresponds to a real-world location;
the display device is worn by the second user at the real-world location; and
causing presentation of the first volumetric video by the display device comprises causing presentation of the volumetric representation of the first user overlaid on one or more elements of the real-world location.
9 . The method of claim 1 , wherein:
the first volumetric video further comprises a volumetric representation of the second user; and
the second volumetric video at the perspective of the first user comprises the volumetric representation of the second user.
10 . The method of claim 1 , wherein the three-dimensional space comprises the second user and the first volumetric video of the three-dimensional space comprises a volumetric representation of the second user within the three-dimensional space, the method further comprising:
accessing a third volumetric video of the three-dimensional space, the third volumetric video being recorded at a perspective of the second user within the three-dimensional space;
detecting that a real-life body of the first user interacts with the volumetric representation of the second user presented in the first volumetric video, wherein the interacting comprises that the real-life body of the first user at least partially overlaps with the volumetric representation of the second user: and
in response to detecting that the real-life body of the first user interacts with the volumetric representation of the second user,
causing the display device of the first user to switch to present the third volumetric video to the first user at the recorded perspective of the second user.
11 . The method of claim 1 , wherein detecting that a real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video comprises:
determining an initial spatial posture of the volumetric representation of the first user and an initial spatial posture of the real-life body of the second user; and
continuously or periodically updating the initial-spatial posture of the real-life body of the second user, until detecting that the real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video.
12 . The method of claim 1 , wherein the display device is an Augmented Reality (AR) display device.
13 . A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to perform operations comprising:
accessing a first volumetric video of a three-dimensional space comprising a volumetric representation of a first user within the three-dimensional space;
accessing a second volumetric video of the three-dimensional space, the second volumetric video being recorded at a perspective of the first user within the three-dimensional space;
causing a display device of a second user to present the first volumetric video of the three-dimensional space comprising the volumetric representation of the first user to the second user;
detecting that a real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video, wherein the interacting comprises that the real-life body of the second user at least partially overlaps with the volumetric representation of the first user; and
in response to detecting that the real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video,
causing the display device to switch to present the second volumetric video to the second user at the perspective of the first user.
14 . The computing apparatus of claim 13 , wherein to detect that a real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video, the instructions further configure the apparatus to perform operations comprising:
determining a gaze of the second user is directed at the volumetric representation of the first user.
15 . The computing apparatus of claim 13 , wherein:
the three-dimensional space corresponds to a real-world location;
the display device is worn by the second user at the real-world location; and
the presentation of the first volumetric video comprises a presentation of the volumetric representation of the first user overlaid on the real-world location at a position in the real-world location corresponding to a position of the first user in the real-world location during capture of the first volumetric video or the second volumetric video.
16 . The computing apparatus of claim 13 , wherein:
the three-dimensional space corresponds to a first real-world location;
the display device is worn by the second user at a second real-world location; and
the presentation of the first volumetric video comprises a presentation of the volumetric representation of the first user at a position related to the second real-world location corresponding to a position of the first user in the real-world location during capture of the volumetric video.
17 . The computing apparatus of claim 16 , wherein cause presentation of the first volumetric video by the display device comprises causing presentation of one or more AR content items overlaid on one or more elements of the real-world location.
18 . The computing apparatus of claim 13 , wherein:
the three-dimensional space corresponds to a real-world location;
the display device is worn by the second user at the real-world location; and
cause presentation of the first volumetric video by the display device comprises causing presentation of the volumetric representation of the first user overlaid on one or more elements of the real-world location.
19 . The computing apparatus of claim 13 , wherein:
the first volumetric video further comprises a volumetric representation of the second user; and
the second volumetric video at the perspective of the first user comprises the volumetric representation of the second user.
20 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
accessing a first volumetric video of a three-dimensional space comprising a volumetric representation of a first user within the three-dimensional space;
accessing a second volumetric video of the three-dimensional space, the second volumetric video being recorded at a perspective of the first user within three dimensional space:
causing a display device of a second user to present the first volumetric video of the three-dimensional space comprising the volumetric representation of the first user to the second user;
detecting that a real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video, wherein the interacting comprises that the real-life body of the second user at least partially overlaps with the volumetric representation of the first user, and
in response to detecting that the real-life body of the second user interacts with the volumetric representation of the first user presented in the first volumetric video,
causing the display device to switch to present the second volumetric video to the second user at the perspective of the first user.