Context-based object viewing within 3D environments
Various implementations disclosed herein include devices, systems, and methods that present views of media objects using different viewing states determined based on context. In some implementations, a view of a 3D environment is presented. Then, a context associated with viewing one or more media objects within the 3D environment is determined, the media objects associated with data for providing an appearance of depth within the one or more media objects. Based on the context, a viewing state is determined for viewing a media object of the one or more media objects within the 3D environment, the viewing state defining whether the media object will be presented as a planar object or with depth within the media object. In accordance with a determination that the viewing state is a first viewing state, the media object is presented within the 3D environment using its associated data for providing the appearance of depth.
1 . A method comprising:
at a processor:
presenting a view of a three-dimensional (3D) environment;
determining a context associated with viewing one or more media objects positioned within the 3D environment, the one or more media objects associated with data for providing an appearance of depth within the one or more media objects, the media object comprising content depicting a media capture environment in which the media object was captured, and the data for providing the appearance of depth corresponding to depths of corresponding elements of the media capture environment, and wherein determining the context comprises determining whether the media object is positioned within a viewing angle threshold in the view of the 3D environment;
based on the context, determining a viewing state for viewing a media object of the one or more media objects within the 3D environment, the viewing state defining whether the media object will be presented within the 3D environment as a planar object or with depth based on depths of corresponding elements of the media capture environment; and
in accordance with a determination that the viewing state is a first viewing state, presenting the view of the 3D environment showing depth for objects of a background setting or physical environment and the media object positioned within the 3D environment, wherein the media object is provided using its associated data for providing an appearance of depth based on depths of corresponding elements of the media capture environment.
2 . The method of claim 1 , the method further comprising in accordance with a determination that the viewing state is a second viewing state, presenting the media object within the 3D environment without using its associated data for providing an appearance of depth.
3 . The method of claim 1 , wherein the second viewing state comprises a monoscopic viewing state in which a left eye view and a right eye view receive a pair of the same view of the media object.
4 . The method of claim 1 , wherein the first viewing state comprises a stereo viewing state in which a left eye view and a right eye view receive a stereoscopic pair of viewing locations within the 3D environment generated using image-pairs captured at different locations within a capturing environment.
5 . The method of claim 4 , wherein the stereo viewing state of the media object is presented for different viewing positions of the media object in the 3D environment.
6 . The method of claim 1 , wherein the first viewing state comprises a stereo viewing state in which a left eye view and a right eye view provide different views of the media object by:
separating a foreground portion and a background portion of the media object using the associated data for providing an appearance of depth, wherein the associated data for providing an appearance of depth comprises depth data associated with the media object; and
adjusting the foreground portion and the background portion to provide the different views for the left eye view and the right eye view.
7 . The method of claim 1 , wherein the associated data for providing an appearance of depth is obtained from a depth sensor.
8 . The method of claim 1 , wherein the associated data for providing an appearance of depth is obtained from stereo pair depth estimation.
9 . The method of claim 1 , wherein the associated data for providing an appearance of depth is obtained from computer vision techniques.
10 . The method of claim 1 , wherein the view depicts depth of objects of the 3D environment using first data corresponding to the 3D positions of the objects within the 3D environment.
11 . The method of claim 1 , further comprising:
based on the context, determining a viewing state for viewing a second media object of the one or more media objects within the 3D environment; and
in accordance with a determination that the viewing state for viewing the second media object is a second viewing state different than the first viewing state, presenting, concurrently with the media object, the second media object within the 3D environment without using its associated data for providing an appearance of depth.
12 . The method of claim 1 , wherein determining the context comprises determining whether a gaze direction is directed towards one of the one or more media objects within the 3D environment.
13 . The method of claim 1 , wherein determining the context comprises determining whether there is an interaction with one of the one or more media objects within the 3D environment.
14 . The method of claim 1 , wherein determining the context comprises determining whether an orientation is directed to within a distance of one of the one or more media objects in the 3D environment.
15 . The method of claim 1 , wherein determining the context comprises determining whether a user interface component overlays one of the one or more media objects in the view of the 3D environment.
16 . The method of claim 1 , wherein the media object comprises a 3D photo.
17 . The method of claim 1 , wherein the media object comprises a 3D video.
18 . The method of claim 1 , wherein the media object comprises a 3D image of a virtual environment.
19 . The method of claim 1 , wherein:
the objects of the background setting or physical environment are shown with depth based on the background setting or physical environment; and
the media object is provided with an appearance of depth based on different sensor data from a media object capture environment.
20 . The method of claim 1 , wherein determining the context comprises determining whether the media object is positioned within a particular distance range of the viewpoint within the 3D environment.
21 . The method of claim 1 , further comprising:
determining an updated context associated with viewing the one or more media objects within the 3D environment;
based on the updated context, determining an updated viewing state for viewing the media object within the 3D environment; and
in accordance with a determination that the viewing state is the first viewing state and the updated viewing state is a second viewing state, transitioning the presentation of the media object from presenting the media object using its associated data for providing an appearance of depth to presenting the media object without using its associated data for providing an appearance of depth in a series of steps over a time period.
22 . The method of claim 21 , further comprising transitioning the viewing state from the first viewing state to a second viewing state by adjusting one eye's view of the media object to match the other eye's view of the media object over a time period.
23 . The method of claim 21 , wherein the transitioning comprises performing a reprojection for each of multiple frames of a transition using depth data or a 3D representation of the 3D environment.
24 . The method of claim 21 , wherein the transitioning is based on the associated data for providing an appearance of depth, wherein the associated data for providing an appearance of depth comprises a 3D geometry of content of the media object.
25 . The method of claim 21 , wherein at least some content of the media object is hidden or obscured during the transitioning.
26 . A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
presenting a view of a three-dimensional (3D) environment;
determining a context associated with viewing one or more media objects positioned within the 3D environment, the one or more media objects associated with data for providing an appearance of depth within the one or more media objects, the media object comprising content depicting a media capture environment in which the media object was captured, and the data for providing the appearance of depth corresponding to depths of corresponding elements of the media capture environment, and wherein determining the context comprises determining whether the media object is positioned within a viewing angle threshold in the view of the 3D environment;
based on the context, determining a viewing state for viewing a media object of the one or more media objects within the 3D environment, the viewing state defining whether the media object will be presented within the 3D environment as a planar object or with depth based on depths of corresponding elements of the media capture environment; and
in accordance with a determination that the viewing state is a first viewing state, presenting the view of the 3D environment showing depth for objects of a background setting or physical environment and the media object positioned within the 3D environment, wherein the media object is provided using its associated data for providing an appearance of depth based on depths of corresponding elements of the media capture environment.
27 . A non-transitory computer-readable storage medium, storing program instructions executable via one or more processors to perform operations comprising:
presenting a view of a three-dimensional (3D) environment;
determining a context associated with viewing one or more media objects positioned within the 3D environment, the one or more media objects associated with data for providing an appearance of depth within the one or more media objects, the media object comprising content depicting a media capture environment in which the media object was captured, and the data for providing the appearance of depth corresponding to depths of corresponding elements of the media capture environment, and wherein determining the context comprises determining whether the media object is positioned within a viewing angle threshold in the view of the 3D environment;
based on the context, determining a viewing state for viewing a media object of the one or more media objects within the 3D environment, the viewing state defining whether the media object will be presented within the 3D environment as a planar object or with depth based on depths of corresponding elements of the media capture environment; and
in accordance with a determination that the viewing state is a first viewing state, presenting the view of the 3D environment showing depth for objects of a background setting or physical environment and the media object positioned within the 3D environment, wherein the media object is provided using its associated data for providing an appearance of depth based on depths of corresponding elements of the media capture environment.