Systems and methods for surface modeling using polarization cues
A computer-implemented method for surface modeling includes: receiving one or more polarization raw frames of a surface of a physical object, the polarization raw frames being captured with a polarizing filter at different linear polarization angles; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and detecting a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces.
1. A computer-implemented method for surface modeling, the method comprising:
receiving one or more polarization raw frames of a surface of a physical object, the polarization raw frames being captured at different polarizations by a polarization camera comprising a polarizing filter;
extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames, wherein the one or more first tensors in the one or more polarization representation spaces comprise:
a degree of linear polarization (DOLP) image in a DOLP representation space; and
an angle of linear polarization (AOLP) image in an AOLP representation space; and
detecting a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces, the surface characteristic being low contrast or invisible in an intensity representation space.
2. The computer-implemented method of claim 1 , wherein the one or more first tensors further comprise one or more non-polarization tensors in one or more non-polarization representation spaces, and
wherein the one or more non-polarization tensors comprise one or more intensity images in intensity representation space.
3. The computer-implemented method of claim 2 , wherein the one or more intensity images comprise:
a first color intensity image;
a second color intensity image; and
a third color intensity image.
4. The computer-implemented method of claim 1 , wherein the surface characteristic comprises a detection of a defect in the surface of the physical object.
5. The computer-implemented method of claim 4 , wherein the detecting the surface characteristic comprises:
loading a stored model corresponding to a location of the surface of the physical object; and
computing the surface characteristic in accordance with the stored model and the one or more first tensors in the one or more polarization representation spaces.
6. The computer-implemented method of claim 5 , wherein the stored model comprises one or more reference tensors in the one or more polarization representation spaces, and
wherein the computing the surface characteristic comprises computing a difference between the one or more reference tensors and the one or more first tensors in the one or more polarization representation spaces.
7. The computer-implemented method of claim 6 , wherein the difference is computed using a Fresnel distance.
8. The computer-implemented method of claim 5 , wherein the stored model comprises a reference three-dimensional mesh, and
wherein the computing the surface characteristic comprises:
computing a three-dimensional point cloud of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces; and
computing a difference between the three-dimensional point cloud and the reference three-dimensional mesh.
9. The computer-implemented method of claim 5 , wherein the stored model comprises a trained statistical model configured to compute a prediction of the surface characteristic based on the one or more first tensors in the one or more polarization representation spaces.
10. The computer-implemented method of claim 9 , wherein the trained statistical model comprises an anomaly detection model.
11. The computer-implemented method of claim 9 , wherein the trained statistical model comprises a convolutional neural network trained to detect defects in the surface of the physical object.
12. The computer-implemented method of claim 9 , wherein the trained statistical model comprises a trained classifier trained to detect defects.
13. A system for surface modeling, the system comprising:
a polarization camera comprising a polarizing filter, the polarization camera being configured to capture polarization raw frames at different polarizations; and
a processing system comprising a processor and memory storing instructions that, when executed by the processor, cause the processor to:
receive one or more polarization raw frames of a surface of a physical object, the polarization raw frames corresponding to different polarizations of light;
extract one or more first tensors in one or more polarization representation spaces from the polarization raw frames, wherein the one or more first tensors in the one or more polarization representation spaces comprise:
a degree of linear polarization (DOLP) image in a DOLP representation space; and
an angle of linear polarization (AOLP) image in an AOLP representation space; and
detect a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces, the surface characteristic being low contrast or invisible in an intensity representation space.
14. The system claim 13 , wherein the one or more first tensors further comprise one or more non-polarization tensors in one or more non-polarization representation spaces, and
wherein the one or more non-polarization tensors comprise one or more intensity images in intensity representation space.
15. The system of claim 14 , wherein the one or more intensity images comprise:
a first color intensity image;
a second color intensity image; and
a third color intensity image.
16. The system of claim 13 , wherein the surface characteristic comprises a defect in the surface of the physical object.
17. The system of claim 16 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to detect the surface characteristic by:
loading a stored model corresponding to a location of the surface of the physical object; and
computing the surface characteristic in accordance with the stored model and the one or more first tensors in the one or more polarization representation spaces.
18. The system of claim 17 , wherein the stored model comprises one or more reference tensors in the one or more polarization representation spaces, and
wherein the memory further stores instructions that, when executed by the processor, cause the processor to compute the surface characteristic by computing a difference between the one or more reference tensors and the one or more first tensors in the one or more polarization representation spaces.
19. The system of claim 18 , wherein the difference is computed using a Fresnel distance.
20. The system of claim 17 , wherein the stored model comprises a trained statistical model configured to compute a prediction of the surface characteristic based on the one or more first tensors in the one or more polarization representation spaces.
21. The system of claim 20 , wherein the trained statistical model comprises an anomaly detection model.
22. The system of claim 20 , wherein the trained statistical model comprises a convolutional neural network trained to detect defects in the surface of the physical object.
23. The system of claim 20 , wherein the trained statistical model comprises a trained classifier trained to detect defects.
24. A system for surface modeling, the system comprising:
a polarization camera comprising a polarizing filter, the polarization camera being configured to capture polarization raw frames at different polarizations; and
a processing system comprising a processor and memory storing instructions that, when executed by the processor, cause the processor to:
receive one or more polarization raw frames of a surface of a physical object, the polarization raw frames corresponding to different polarizations of light;
extract one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and
detect a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces,
wherein the surface characteristic comprises a defect in the surface of the physical object,
wherein the memory further stores instructions that, when executed by the processor, cause the processor to detect the surface characteristic by:
loading a stored model corresponding to a location of the surface of the physical object; and
computing the surface characteristic in accordance with the stored model and the one or more first tensors in the one or more polarization representation spaces
wherein the stored model comprises a reference three-dimensional mesh, and
wherein the memory further stores instructions that, when executed by the processor, cause the processor to compute the surface characteristic by:
computing a three-dimensional point cloud of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces; and
computing a difference between the three-dimensional point cloud and the reference three-dimensional mesh.