Systems and methods for determining positions for three-dimensional models relative to spatial features
The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.
1. A computer-implemented method comprising:
receiving, from a user device associated with a user, a request for a resource; and
replying to the request, the replying including:
obtaining media comprising an image of a real-world space associated with the user and information identifying three-dimensional (3D) spatial features of the real-world space;
obtaining a 3D model of an object;
determining, based on at least one of the media and the 3D model, a recommended position for the 3D model relative to the 3D spatial features of the real-world space, wherein determining the recommended position for the 3D model relative to the 3D spatial features of the real-world space comprises: providing an input to a trained machine learning (ML) model, the input based on at least one of the media and the 3D model; and obtaining, based on an output of the ML model, the recommended position for the 3D model relative to the 3D spatial features of the real-world space; and
instructing the user device to display the resource, the resource including augmented media in which at least a portion of the image is overlaid with a render of the 3D model defined at the position relative to the 3D spatial features.
2. The computer-implemented method of claim 1 , wherein:
the 3D spatial features of the real-world space comprise a surface within the real-world space;
the recommended position for the 3D model comprises a coordinate on the surface; and
the 3D model is defined relative to the surface such that the augmented media depicts the object resting on the surface.
3. The computer-implemented method of claim 1 , wherein the input to the ML model comprises at least some of the information identifying the 3D spatial features of the real-world space.
4. The computer-implemented method of claim 3 , wherein the ML model is trained using a training data set, the training data set comprising information identifying 3D spatial features of a plurality of real-world spaces and user-defined positions of 3D models in those real-world spaces.
5. The computer-implemented method of claim 4 , wherein the 3D models include one or both of the 3D model and other 3D models similar to the 3D model.
6. The computer-implemented method of claim 3 , wherein the ML model is specific to the 3D model.
7. The computer-implemented method of claim 1 , wherein the input to the ML model comprises product information for the object.
8. The computer-implemented method of claim 7 , wherein the ML model is trained using a training data set, the training data set comprising product information for a plurality of objects and user-defined positions for corresponding 3D models relative to the 3D spatial features of the real-world space.
9. The computer-implemented method of claim 7 , wherein the ML model is specific to the real-world space.
10. The computer-implemented method of claim 1 , wherein the input to the ML model comprises at least some of the information identifying the 3D spatial features of the real-world space associated with the user and product information for the object.
11. The computer-implemented method of claim 10 , wherein the ML model is trained using a training data set, the training data set comprising information identifying 3D spatial features of a plurality of real-world spaces, product information for a plurality of objects, and user-defined positions of corresponding 3D models in those real-world spaces.
12. The computer-implemented method of claim 1 , wherein:
the position for the 3D model overlaps with a particular 3D spatial feature of the real-world space; and
the particular 3D spatial feature is removed in the augmented media.
13. The computer-implemented method of claim 12 , wherein the information further identifies the particular 3D spatial feature as moveable.
14. The computer-implemented method of claim 12 , wherein determining the recommended position for the 3D model relative to the 3D spatial features of the real-world space comprises:
determining that the particular 3D spatial feature of the real-world space relates to the object; and
determining the recommended position for the 3D model based on a position of the particular 3D spatial feature relative to the other 3D spatial features of the real-world space.
15. A system comprising:
memory to store:
media comprising an image of a real-world space associated with a user and information identifying three-dimensional (3D) spatial features of the real-world space; and
a 3D model of an object; and
at least one processor to receive, from a user device associated with the user, a request for a resource, and in response to the request:
determine, based on at least one of the media and the 3D model, a recommended position for the 3D model relative to the 3D spatial features of the real-world space; and
instruct the user device to display the resource, the resource including augmented media in which at least a portion of the image is overlaid with a render of the 3D model defined at the position relative to the 3D spatial features; and
wherein the memory is to store a trained machine learning (ML) model and the at least one processor is to:
provide an input to the ML model, the input based on at least one of the media and the 3D model; and
obtain, based on an output of the ML model, the recommended position for the 3D model relative to the 3D spatial features of the real-world space.
16. The system of claim 15 , wherein:
the 3D spatial features of the real-world space comprise a surface within the real-world space;
the recommended position for the 3D model comprises a coordinate on the surface; and
the 3D model is defined relative to the surface such that the augmented media depicts the object resting on the surface.
17. The system of claim 15 , wherein the input to the ML model comprises at least some of the information identifying the 3D spatial features of the real-world space.
18. The system of claim 17 , wherein the ML model is trained using a training data set, the training data set comprising information identifying 3D spatial features of a plurality of real-world spaces and user-defined positions of 3D models in those real-world spaces.
19. The system of claim 17 , wherein the ML model is specific to the 3D model.
20. The system of claim 15 , wherein the input to the ML model comprises product information for the object.
21. The system of claim 20 , wherein the ML model is trained using a training data set, the training data set comprising product information for a plurality of objects and user-defined positions for corresponding 3D models relative to the 3D spatial features of the real-world space.
22. The system of claim 21 , wherein the ML model is specific to the real-world space.
23. A non-transitory computer readable medium storing computer executable instructions which, when executed by a computer, cause the computer to:
receive, from a user device associated with a user, a request for a resource; and
reply to the request, the replying including:
obtaining media comprising an image of a real-world space associated with the user and information identifying three-dimensional (3D) spatial features of the real-world space;
obtaining a 3D model of an object;
determining, based on at least one of the media and the 3D model, a recommended position for the 3D model relative to the 3D spatial features of the real-world space, wherein determining the recommended position for the 3D model relative to the 3D spatial features of the real-world space comprises: providing an input to a trained machine learning (ML) model, the input based on at least one of the media and the 3D model; and obtaining, based on an output of the ML model, the recommended position for the 3D model relative to the 3D spatial features of the real-world space; and
instructing the user device to display the resource, the resource including augmented media in which at least a portion of the image is overlaid with a render of the 3D model defined at the position relative to the 3D spatial features.
24. The non-transitory computer readable medium of claim 23 , wherein the input to the ML model comprises at least some of the information identifying the 3D spatial features of the real-world space.
25. The non-transitory computer readable medium of claim 23 , wherein the input to the ML model comprises product information for the object.