Methods and systems for scanning objects
There are provided systems and methods for scanning objects in a virtual environment. In particular, the present disclosure pertains to the domain of three-dimensional (3D) object capturing in extended reality (XR) environments and to an optimized system and method for passively capturing 3D objects within Extended Reality (XR) environments. A selection of an object from within an XR environment by a user via a user device is detected. A 3D capture session is initiated based on the detection. Scan data corresponding to the selected object is captured during the 3D capture session, where the capturing is performed as a background process on the user device. A 3D representation or model of the object is generated in the XR environment and provided to the user via the user device.
1 . A method for scanning a physical object, the method comprising:
detecting, by control circuitry, a presence of the physical object to be scanned in an extended reality environment;
initiating, by the control circuitry, a 3D scan of the physical object as a background operation of a user device;
in response to the initiation of the 3D scan, capturing, by the control circuitry, scan data corresponding to the object while the object is in a field of view of the user device;
calculating, by the control circuitry, a confidence score based on the scan data, wherein the confidence score is used to determine whether to continue the 3D scan;
applying, by the control circuitry, spatial segmentation to the scan data to identify regions of interest, and bypass segments of the physical object that have been previously captured and remain unchanged;
performing, by the control circuitry, model quantization on the scan data, wherein a degree of the quantization is adjusted based on the availability of cloud computing resources;
storing, by the control circuitry, the scan data in a database in response to the calculated confidence score exceeding a threshold; and
generating, by the control circuitry, a 3D model of the physical object in the extended reality environment based on the scan data in the database.
2 . The method of claim 1 , further comprising:
initiating a primary 3D scan of the physical object as a background operation of the user device;
based on the primary 3D scan data, determining that a 3D model related to the physical object is incomplete;
retrieving the 3D model related to the physical object from the database; and
updating the retrieved 3D model with the scan data.
3 . The method of claim 1 , wherein, in response to detecting the presence of the object, the method further comprises:
displaying an assistive user interface element around the object on a viewport of the user device to capture a plurality of angles of the object in a 3D space.
4 . The method of claim 1 , further comprising:
determining completion status data for the generation of the 3D model;
generating an overlay comprising the 3D model on the object based on the completion status data; and
displaying the overlay within the viewport of the user device.
5 . The method of claim 1 , wherein:
applying the spatial segmentation to the scan data to identify regions of interest further comprises:
distinguishing the physical object from background elements in the field of view;
extracting a set of data points corresponding to pertinent sections of the physical object to restrict an area of focus for the 3D scan; and
storing a plurality of detailed segmentations of the identified regions of interest in the database for use as a spatial reference during a subsequent encounter with the physical object.
6 . The method of claim 1 , further comprising:
receiving an indication of priority for scanning one or more objects of a plurality of objects; and
capturing the one or more objects of the plurality of objects based on the indication of priority.
7 . The method of claim 1 , further comprising:
tracking a motion of the physical object; and
capturing the scan data corresponding to the physical object based on a speed of the motion of the object, wherein the scan data is captured when the speed of the motion is within a pre-defined threshold.
8 . The method of claim 1 , wherein generating the 3D model comprises:
receiving a set of features of interest corresponding to the object; and
generating the 3D model based on the set of features of interest.
9 . The method of claim 1 , further comprising:
creating at least one classifier algorithm based on the scan data of the physical object; and
using the classifier algorithm to detect a presence of a second physical object.
10 . A system for scanning a physical object, the system comprising control circuitry configured to:
detect a presence of the physical object to be scanned in an extended reality environment;
initiate a 3D scan of the physical object as a background operation of a user device;
in response to the initiation of the 3D scan, capture scan data corresponding to the object while the object is in a field of view of the user device;
calculate a confidence score based on the scan data, wherein the confidence score is used to determine whether to continue the 3D scan;
apply spatial segmentation to the scan data to identify regions of interest and bypass segments of the physical object that have been previously captured and remain unchanged;
perform model quantization on the scan data, wherein a degree of the quantization is adjusted based on the availability of cloud computing resources;
store the scan data in a database in response to the calculated confidence score exceeding a threshold; and
generate a 3D model of the physical object in the extended reality environment based on the scan data in the database.
11 . The system of claim 10 , the control circuitry configured to:
retrieve a prior 3D model related to the physical object from the database;
determine whether the prior 3D model related to the physical object is complete; and
in response to determining the prior 3D model is not complete, update the retrieved prior 3D model with the scan data.
12 . The system of claim 10 , wherein, in response to detecting the presence of the object, the control circuitry configured to:
display an assistive user interface element around the object on a viewport of the user device to capture a plurality of angles of the object in a 3D space.
13 . The system of claim 10 , the control circuitry configured to:
determine completion status data for the generation of the 3D model;
generate an overlay comprising the 3D model on the object based on the completion status data; and
display the overlay on the viewport of the user device.
14 . The system of claim 10 , wherein the control circuitry is configured, when applying spatial segmentation to the scan data to identify regions of interest, to:
distinguish the physical object from background elements in the field of view while applying the spatial segmentation;
extract a set of data points corresponding to pertinent sections of the physical object to restrict an area of focus for the 3D scan; and
store a plurality of detailed segmentations of the identified regions of interest in the database for use as a spatial reference during a subsequent encounter with the physical object.
15 . The system of claim 11 , the control circuitry configured to:
receive an indication of priority for scanning one or more of a plurality of objects; and
capture each of the plurality of objects based on the indication of priority.
16 . The system of claim 10 , the control circuitry configured to:
track a motion of the physical object; and
capture the scan data corresponding to the physical object based on a speed of the motion of the object, wherein the scan data is captured when the speed of the motion is within a pre-defined threshold.
17 . The system of claim 10 , the control circuitry configured to:
receive a set of features of interest corresponding to the object; and
generate the 3D model based on the set of features of interest.
18 . The system of claim 10 , the control circuitry configured to:
create at least one classifier algorithm based on the scan data of the physical object; and
use the classifier algorithm to detect a presence of a second physical object.