Systems and methods for employing invisible fiducials for aligning scans or images to form composite images, models, or datasets
Disclosed is a system and associated methods for generating a composite image from scans or images that are aligned using invisible fiducials. The invisible fiducial is a transparent substance or a projected specific wavelength that is applied to and changes reflectivity of a surface at the specific wavelength without interfering with a capture of positions or visible color characteristics across the surface. The system performs first and second capture of a scene with the surface, and detects a position of the invisible fiducial in each capture based on values measured across the specific wavelength that satisfy a threshold associated with the invisible fiducial. The system aligns the first capture with the second capture based on the detected positions of the invisible fiducial, and generates a composite image by merging or combining the positions or visible color characteristics from the aligned captures.
1. A method comprising:
receiving, from a first device, positional measurements for a plurality of points about one or more surfaces of an object and an intensity with which a particular spectral band reflects off of each point of the plurality of points;
receiving, from a second device, color measurements for the plurality of points and an intensity with which the particular spectral band reflects off of each point of the plurality of points;
determining that the intensity with which the particular spectral band reflects off of a first set of neighboring points and a second set of neighboring points from the plurality of points in the positional measurements and in the color measurements exceeds a threshold;
detecting an invisible fiducial that is represented by the second set of neighboring points in the positional measurements and in the color measurements in response to the second set of neighboring points forming a particular shape of the invisible fiducial and the first set of neighboring points forming a shape that differs from the particular shape of the invisible fiducial;
aligning the particular shape of the invisible fiducial represented by the second set of neighboring points in the color measurements to the particular shape of the invisible fiducial represented by the second set of neighboring points in the positional measurements;
constructing a three-dimensional (“3D”) model of the object by generating primitives in a 3D space according to the positional measurements and by defining colors of the primitives based on the color measurements that are aligned with the positional measurements that are used to generate the primitives;
removing the invisible fiducial from the 3D model by excluding the intensity with which the particular spectral band reflects off of the plurality of points from said generating of the primitives and said defining of the colors; and
presenting the 3D model without the invisible fiducial interfering with the colors or form of a part of the object represented by the second set of neighboring points in response to removing the invisible fiducial from the 3D model.
2. The method of claim 1 further comprising:
applying the invisible fiducial on the second set of neighboring points, wherein applying the invisible fiducial comprises placing a transparent material that is highly reflective of the particular spectral band with the particular shape on the second set of neighboring points or projecting one or more wavelengths associated with the particular spectral band in the particular shape on the second set of neighboring points.
3. The method of claim 1 , wherein receiving the color measurements comprises:
receiving measurements for a first wavelength, a second wavelength, and a third wavelength reflecting off each point of the plurality of points, wherein the first wavelength is associated with a first color and a first spectral band, wherein the second wavelength is associated with a second color and a second spectral band, and wherein the third wavelength is outside a visible spectrum and is used to define the invisible fiducial.
4. The method of claim 1 further comprising:
receiving the particular shape of the invisible fiducial;
determining a first arrangement of the first set of neighboring points and a second arrangement of the second set of neighboring points based on positions associated with each point of the plurality of points; and
wherein detecting the invisible fiducial comprises:
determining that the first arrangement differs from the particular shape of the invisible fiducial; and
determining that the second arrangement matches the particular shape of the invisible fiducial.
5. The method of claim 1 further comprising:
receiving the particular shape of the invisible fiducial;
comparing different shapes formed by the first set of neighboring points and the second set of neighboring points to the particular shape of the invisible fiducial based on the intensity with which the particular spectral band reflects off the first set of neighboring points and the second set of neighboring points exceeding the threshold; and
wherein detecting the invisible fiducial comprises:
determining that the first set of neighboring points corresponds to an invisible fiducial false positive based on an arrangement of the first set of neighboring points differing from the particular shape of the invisible fiducial; and
determining that the second set of neighboring points represent the invisible fiducial based on an arrangement of the second set of neighboring points forming the particular shape of the invisible fiducial.
6. The method of claim 1 , wherein aligning the particular shape of the invisible fiducial comprises:
transforming one or more of the positional measurements or the color measurements received for the plurality of points until the positional measurements or the color measurements received for the second set of neighboring points representing the invisible fiducial have a particular size or orientation.
7. The method of claim 1 , wherein aligning the particular shape of the invisible fiducial comprises:
adjusting one or more of a size or orientation of the plurality of points with the positional measurements until the first set of neighboring points with the positional measurements match a size or orientation of the second set of neighboring points with the color measurements.
8. A system comprising:
one or more hardware processors configured to:
receive, from a first device, measurements for a plurality of points about one or more surfaces of an object and an intensity with which a particular spectral band reflects off of each point of the plurality of points;
receive, from a second device, color measurements for the plurality of points and an intensity with which the particular spectral band reflects off of each point of the plurality of points;
determine that the intensity with which the particular spectral band reflects off of a first set of neighboring points and a second set of neighboring points from the plurality of points in the positional measurements and in the color measurements exceeds a threshold;
detect an invisible fiducial that is represented by the second set of neighboring points in the positional measurements and in the color measurements in response to the second set of neighboring points forming a particular shape of the invisible fiducial and the first set of neighboring points forming a shape that differs from the particular shape of the invisible fiducial;
align the particular shape of the invisible fiducial represented by the second set of neighboring points in the color measurements to the particular shape of the invisible fiducial represented by the second set of neighboring points in the positional measurements;
construct a three-dimensional (“3D”) model of the object by generating primitives in a 3D space according to the positional measurements and by defining colors of the primitives based on the color measurements that are aligned with the positional measurements that are used to generate the primitives;
remove the invisible fiducial from the 3D model by excluding the intensity with which the particular spectral band reflects off of the plurality of points from said generating of the primitives and said defining of the colors; and
present the 3D model without the invisible fiducial interfering with the colors or form of a part of the object represented by the second set of neighboring points in response to removing the invisible fiducial from the 3D model.
9. The system of claim 8 , wherein the one or more hardware processors are further configured to:
apply the invisible fiducial on the second set of neighboring points, wherein applying the invisible fiducial comprises placing a transparent material that is highly reflective of the particular spectral band with the particular shape on the second set of neighboring points or projecting one or more wavelengths associated with the particular spectral band in the particular shape on the second set of neighboring points.
10. The system of claim 8 , wherein receiving the color measurements comprises:
receiving measurements for a first wavelength, a second wavelength, and a third wavelength reflecting off each point of the plurality of points, wherein the first wavelength is associated with a first color and a first spectral band, wherein the second wavelength is associated with a second color and a second spectral band, and wherein the third wavelength is outside a visible spectrum and is used to define the invisible fiducial.
11. The system of claim 8 , wherein the one or more hardware processors are further configured to:
receive the particular shape of the invisible fiducial;
determine a first arrangement of the first set of neighboring points and a second arrangement of the second set of neighboring points based on positions associated with each point of the plurality of points; and
wherein detecting the invisible fiducial comprises:
determining that the first arrangement differs from the particular shape of the invisible fiducial; and
determining that the second arrangement matches the particular shape of the invisible fiducial.
12. The system of claim 8 , wherein the one or more hardware processors are further configured to:
receive the particular shape of the invisible fiducial;
compare different shapes formed by the first set of neighboring points and the second set of neighboring points to the particular shape of the invisible fiducial based on the intensity with which the particular spectral band reflects off the first set of neighboring points and the second set of neighboring points exceeding the threshold; and
wherein detecting the invisible fiducial comprises:
determining that the first set of neighboring points corresponds to an invisible fiducial false positive based on an arrangement of the first set of neighboring points differing from the particular shape of the invisible fiducial; and
determining that the second set of neighboring points represent the invisible fiducial based on an arrangement of the second set of neighboring points forming the particular shape of the invisible fiducial.
13. A non-transitory computer-readable medium storing program instructions that, when executed by one or more hardware processors of an image generation system, cause the image generation system to perform operations comprising:
receiving, from a first device, positional measurements for a plurality of points about one or more surfaces of an object and an intensity with which a particular spectral band reflects off of each point of the plurality of points;
receiving, from a second device, color measurements for the plurality of points and an intensity with which the particular spectral band reflects off of each point of the plurality of points;
determining that the intensity with which the particular spectral band reflects off of a first set of neighboring points and a second set of neighboring points from the plurality of points in the positional measurements and in the color measurements exceeds a threshold;
detecting an invisible fiducial that is represented by the second set of neighboring points in the positional measurements and in the color measurements in response to the second set of neighboring points forming a particular shape of the invisible fiducial and the first set of neighboring points forming a shape that differs from the particular shape of the invisible fiducial;
aligning the particular shape of the invisible fiducial represented by the second set of neighboring points in the color measurements to the particular shape of the invisible fiducial represented by the second set of neighboring points in the positional measurements;
constructing a three-dimensional (“3D”) model of the object by generating primitives in a 3D space according to the positional measurements and by defining colors of the primitives based on the color measurements that are aligned with the positional measurements that are used to generate the primitives;
removing the invisible fiducial from the 3D model by excluding the intensity with which the particular spectral band reflects off of the plurality of points from said generating of the primitives and said defining of the colors; and
presenting the 3D model without the invisible fiducial interfering with the colors or form of a part of the object represented by the second set of neighboring points in response to removing the invisible fiducial from the 3D model.