SYSTEMS AND METHODS FOR MODELING REAL-WORLD OBJECTS IN VIRTUAL SCENES
A computer-implemented is disclosed. The method includes: obtaining a three-dimensional (3D) representation of a first real-world environment; identifying a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment; determining a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest; and generating an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest in the first position in the AR version of the first real-world environment.
1 . A computer-implemented method, comprising:
obtaining a three-dimensional (3D) representation of a first real-world environment;
identifying an object of interest in a second real-world environment, the second real-world environment being different from the first real-world environment;
determining a target location in the 3D representation of the first real-world environment corresponding to the object of interest; and
generating an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the object of interest at the target location in the AR version of the first real-world environment.
2 . The method of claim 1 , wherein determining the target location in the 3D representation of the first real-world environment comprises determining a location of a similar object in the first real-world environment and wherein generating the AR version of the first real-world environment comprises positioning the object of interest at the location of the similar object in the AR version of the first real-world environment so as to replace the similar object.
3 . The method of claim 2 , wherein generating the AR version of the first real-world environment comprises removing the similar object from the 3D representation of the first real-world environment.
4 . The method of claim 1 , wherein determining the target location in the 3D representation of the first real-world environment comprises identifying an empty space in the 3D representation of the first real-world environment sized to fit the object of interest and wherein generating the AR version of the first real-world environment comprises positioning the object of interest within an empty space in the AR version of the first real-world environment.
5 . The method of claim 4 , wherein identifying the empty space in the 3D representation of the first real-world environment comprises determining positions of one or more objects in the 3D representation of the first real-world environment.
6 . The method of claim 5 , wherein identifying the empty space in the 3D representation of the first real-world environment comprises determining a position of a second object in the 3D representation of the first real-world environment, the second object satisfying a defined condition with respect to the object of interest.
7 . The method of claim 1 , wherein the 3D representation of the first real-world environment comprises metadata indicating at least one of location or boundary associated with at least one object in the 3D representation.
8 . The method of claim 1 , further comprising obtaining a first image of the object of interest and wherein generating the AR version of the first real-world environment comprises combining the first image and the 3D representation of the first real-world environment.
9 . The method of claim 1 , wherein the AR version of the first real-world environment is generated responsive to determining that a defined trigger condition is satisfied.
10 . The method of claim 9 , wherein the defined trigger condition relates to at least one of:
a detected pose of a user relative to the object of interest;
input of the user received via an input interface;
a distance of the user relative to the object of interest; or
detected contact between the user and the object of interest.
11 . The method of claim 1 , wherein obtaining the 3D representation of the first real-world environment comprises obtaining 3D scan data including at least one of camera data or LiDAR sensor data.
12 . The method of claim 1 , wherein generating the AR version of the first real-world environment comprises identifying a first subregion of a first image containing the object of interest and a second subregion of the first image that does not contain the object of interest.
13 . The method of claim 12 , wherein generating the AR version of the first real-world environment comprises combining the 3D representation of the first real-world environment with the first image such that the second subregion of the first image is hidden in the AR version of the first real-world environment.
14 . The method of claim 1 , further comprising:
obtaining depth data associated with the second real-world environment; and
partitioning an image of the second real-world environment using the depth data to obtain an image segment containing the object of interest,
wherein generating the AR version of the first real-world environment comprises combining the image segment with the 3D representation of the first real-world environment.
15 . The method of claim 14 , wherein the depth data comprises a depth map of the second real-world environment generated using a 3D scanner.
16 . The method of claim 15 , further comprising:
obtaining rotation and position data associated with the 3D scanner capturing the depth map; and
matching pixels of the depth map to locations in the image of the second real-world environment based on the rotation and position data.
17 . The method of claim 15 , further comprising determining a bounding box representing a spatial extent of the object of interest in the second real-world environment, wherein the bounding box is determined based on the depth data.
18 . A computing system, comprising:
a processor; and
a memory coupled to the processor, the memory storing processor-executable instructions that, when executed, are to cause the processor to:
obtain a three-dimensional (3D) representation of a first real-world environment;
identify an object of interest in a second real-world environment, the second real-world environment being different from the first real-world environment;
determine a target location in the 3D representation of the first real-world environment corresponding to the object of interest; and
generate an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the object of interest at the target location in the AR version of the first real-world environment.
19 . The computing system of claim 18 , wherein the instructions, when executed, are to further cause the processor to:
obtain depth data associated with the second real-world environment; and
partition an image of the second real-world environment using the depth data to obtain an image segment containing the object of interest, wherein generating the AR version of the first real-world environment comprises combining the image segment with the 3D representation of the first real-world environment.
20 . A non-transitory processor-readable medium storing processor-executable instructions that, when executed by a processor, are to cause the processor to:
obtain a three-dimensional (3D) representation of a first real-world environment;
identify an object of interest in a second real-world environment, the second real-world environment being different from the first real-world environment;
determine a target location in the 3D representation of the first real-world environment corresponding to the object of interest; and
generate an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the object of interest at the target location in the AR version of the first real-world environment.