Systems and methods for modeling real-world objects in virtual scenes
A computer-implemented method includes: obtaining a three-dimensional (3D) representation of a first real-world environment; identifying a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment; determining a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest; and generating an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest in the first position in the AR version of the first real-world environment.
1. A computer-implemented method, comprising:
obtaining a three-dimensional (3D) representation of a first real-world environment;
identifying a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment;
determining a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest based on determining a position of a similar real-world object in the first real-world environment; and
generating an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest at the position of the similar real-world object in the AR version of the first real-world environment so as to replace the similar real-world object.
2. The method of claim 1 , wherein generating the AR version of the first real-world environment comprises removing the similar real-world object from the 3D representation of the first real-world environment.
3. The method of claim 1 , wherein generating the AR version of the first real-world environment comprises:
determining, based on the 3D representation, that the first real-world environment does not contain a similar real-world object; and
in response, positioning the real-world object of interest within an empty space that is sized to fit the real-world object of interest in the AR version of the first real-world environment.
4. The method of claim 3 , wherein determining the first position in the 3D representation of the first real-world environment comprises determining positions of one or more objects in the 3D representation of the first real-world environment.
5. The method of claim 4 , wherein determining the first position in the 3D representation of the first real-world environment comprises determining a position of a second object in the 3D representation of the first real-world environment, the second object satisfying a defined condition with respect to the real-world object of interest.
6. The method of claim 1 , wherein the 3D representation of the first real-world environment comprises metadata indicating at least one of location or boundary associated with at least one object in the 3D representation.
7. The method of claim 1 , further comprising obtaining a first image of the real-world object of interest and wherein generating the AR version of the first real-world environment comprises combining the first image and the 3D representation of the first real-world environment.
8. The method of claim 1 , wherein the AR version of the first real-world environment is generated responsive to determining that a defined trigger condition is satisfied.
9. The method of claim 8 , wherein the defined trigger condition relates to at least one of:
a detected pose of a user relative to the real-world object of interest;
input of the user received via an input interface;
a distance of the user relative to the real-world object of interest; or
detected contact between the user and the real-world object of interest.
10. The method of claim 1 , wherein obtaining the 3D representation of the first real-world environment comprises obtaining 3D scan data including at least one of camera data or LiDAR sensor data.
11. The method of claim 1 , wherein generating the AR version of the first real-world environment comprises identifying a first subregion of a first image containing the real-world object of interest and a second subregion of the first image that does not contain the real-world object of interest.
12. The method of claim 11 , wherein generating the AR version of the first real-world environment comprises combining the 3D representation of the first real-world environment with the first image such that the second subregion of the first image is hidden in the AR version of the first real-world environment.
13. The method of claim 1 , further comprising:
obtaining depth data associated with the second real-world environment; and
partitioning an image of the second real-world environment using the depth data to obtain an image segment containing the real-world object,
wherein generating the AR version of the first real-world environment comprises combining the image segment with the 3D representation of the first real-world environment.
14. The method of claim 13 , wherein the depth data comprises a depth map of the second real-world environment generated using a 3D scanner.
15. The method of claim 14 , further comprising:
obtaining rotation and position data associated with the 3D scanner capturing the depth map; and
matching pixels of the depth map to locations in the image of the second real-world environment based on the rotation and position data.
16. The method of claim 14 , further comprising determining a bounding box representing a spatial extent of the real-world object in the second real-world environment, wherein the bounding box is determined based on the depth data.
17. A computing system, comprising:
a processor; and
a memory coupled to the processor, the memory storing processor-executable instructions that, when executed, are to cause the processor to:
obtain a three-dimensional (3D) representation of a first real-world environment;
identify a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment;
determine a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest based on determining a position of a similar real-world object in the first real-world environment; and
generate an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest at the position of the similar real-world object in the AR version of the first real-world environment so as to replace the similar real-world object.
18. The computing system of claim 17 , wherein the instructions, when executed, are to further cause the processor to:
obtain depth data associated with the second real-world environment; and
partition an image of the second real-world environment using the depth data to obtain an image segment containing the real-world object, wherein generating the AR version of the first real-world environment comprises combining the image segment with the 3D representation of the first real-world environment.
19. The computing system of claim 17 , wherein generating the AR version of the first real-world environment comprises:
determining, based on the 3D representation, that the first real-world environment does not contain a similar real-world object; and
in response, positioning the real-world object of interest within an empty space that is sized to fit the real-world object of interest in the AR version of the first real-world environment.
20. A non-transitory processor-readable medium storing processor-executable instructions that, when executed by a processor, are to cause the processor to:
obtain a three-dimensional (3D) representation of a first real-world environment;
identify a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment;
determine a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest based on determining a position of a similar real-world object in the first real-world environment; and
generate an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest at the position of the similar real-world object in the AR version of the first real-world environment so as to replace the similar real-world object.