Enhancing artificial intelligence routines using 3D data
In a general aspect, enhancement of artificial intelligence algorithms using 3D data is described. In some aspects, input data of an object is stored in a storage engine of a system. The input data includes first-order primitives and second-order primitives. A plurality of features of the object is determined by operation of an analytics engine of the system, based on the first-order primitives and the second-order primitives. A tensor field is generated by operation of the analytics engine of the system. The tensor field includes an attribute set, which includes one or more attributes selected from the first-order primitives, the second-order primitives, or the plurality of features. The tensor field is processed by operation of the analytics engine of the system according to a series of artificial intelligence algorithms to generate output data representing the object.
1. A method, comprising:
storing input data of an object in a storage engine of a system, the input data comprising first-order primitives and second-order primitives;
determining, by operation of an analytics engine of the system, a plurality of features of the object based on the first-order primitives and the second-order primitives;
generating, by operation of the analytics engine of the system, a tensor field, the tensor field comprising an attribute set, the attribute set comprising one or more attributes selected from the first-order primitives, the second-order primitives, and the plurality of features, the tensor field being organized based on attributes of a single pixel, a single voxel, a super pixel, a super frame, or a cluster of the same, the tensor field comprising subgroups defined based on types of attributes contained in the attribute set, the subgroups comprising:
a first subgroup containing one or more scalar attributes;
a second subgroup containing one or more vector attributes; and
a third subgroup containing one or more attributes containing 3D information; and
processing, by operation of the analytics engine of the system, the tensor field according to an artificial intelligence algorithm to generate output data representing the object, wherein the output data comprises surface normal and curvature data.
2. The method of claim 1 , wherein the first-order primitives comprise pixel-level first-order primitives, and the pixel-level first-order primitives are generated based on electromagnetic (EM) radiation received from the object located in a field-of-view of an image sensor device.
3. The method of claim 2 , wherein the EM radiation received from the object is reflected, refracted, or emitted from the object.
4. The method of claim 2 , wherein the image sensor comprises a single-lens spatial phase imaging (SPI) sensor.
5. The method of claim 2 , wherein the image sensor comprises a plurality of image sensors, and the object comprises a plurality of objects, and the method comprises:
generating in real-time, by operation of the analytics engine of the system, a plurality of tensor fields corresponding to the plurality of objects; and
processing, by operation of the analytics engine of the system, the plurality of tensor fields according to the artificial intelligence algorithm to generate output data representing the plurality of objects.
6. The method of claim 2 , comprising:
after receiving the tensor field, generating, by operation of the analytics engine, synthetic training data by modifying an attribute value of an attribute in the tensor field; and
training the artificial intelligence algorithm using the synthetic training data and the field-of-view of the image sensor and a point-of-view of the image sensor.
7. The method of claim 1 , wherein the second-order primitives are generated based on the first-order primitives.
8. The method of claim 1 , wherein the attribute set of the tensor field represents the object at an orientation within a region of space, the attribute set comprising an attribute that is one of a scaler, a vector, a surface, or a shape.
9. The method of claim 1 , wherein the attribute set of the tensor field comprises respective attributes and respective confidence values for the respective attributes.
10. The method of claim 1 , wherein the tensor field comprises a first tensor field, the first-order primitives comprise pixel-level first-order primitives, the second-order primitives comprise pixel-level second-order primitives, generating the first tensor field comprises:
identifying a pixel cluster by clustering pixels according to one or more attributes in the pixel-level first-order primitives and the pixel-level second-order primitives, each of the one or more attributes of the pixels that are clustered having an attribute value within a predetermined interval to define surfaces of the object; and
generating, by operation of the analytics engine of the system, a second tensor field corresponding to the pixel cluster, the second tensor field having a higher order than the first tensor field.
11. The method of claim 10 , wherein the first tensor field comprises one or more first attributes, the second tensor field comprises one or more second attributes, and each of the one or more second attributes comprises an attribute in surface-level first-order primitives or surface-level second-order primitives.
12. The method of claim 11 , wherein the artificial intelligence algorithm comprises a first artificial intelligence algorithm and a second artificial intelligence algorithm, the first artificial intelligence algorithm is configured to process the one or more first attributes in the first tensor field and the second artificial intelligence algorithm is configured to process the one or more second attributes of the second tensor field.
13. The method of claim 1 , comprising:
after receiving the tensor field, assigning, by operation of the analytics engine, an attribute value to an attribute in the plurality of features in the tensor field.
14. The method of claim 1 , wherein the output data is generated based on the tensor field for one of the following applications: corrosion identification, object tracking, object identification, or object characterization, and wherein the applications are performed on objects comprising one or more of cracks, pits, blisters, skin lesions, or blood vessels.
15. The method of claim 1 , wherein the tensor field comprises a subgroup of attributes, the method comprises:
prior to processing the tensor field, normalizing one or more attributes in the subgroup to generate one or more normalized attributes.
16. A system comprising:
a storage engine configured to store input data of an object, the input data comprising first-order primitives and second-order primitives; and
an analytics engine configured to:
determine a plurality of features of the object based on the first-order primitives and the second-order primitives;
generate a tensor field, the tensor field comprising an attribute set, the attribute set comprising one or more attributes selected from the first-order primitives, the second-order primitives, and the plurality of features, the tensor field being organized based on attributes of a single pixel, a single voxel, a super pixel, a super frame, or a cluster of the same, the tensor field comprising subgroups defined based on types of attributes contained in the attribute set, the subgroups comprising:
a first subgroup containing one or more scalar attributes;
a second subgroup containing one or more vector attributes; and
a third subgroup containing one or more attributes containing 3D information; and
process the tensor field according to an artificial intelligence algorithm to generate output data representing the object, wherein the output data comprises surface normal and curvature data.
17. The system of claim 16 , wherein the first-order primitives comprise pixel-level first-order primitives, and the pixel-level first-order primitives are generated based on electromagnetic (EM) radiation received from the object located in a field-of-view of an image sensor device.
18. The system of claim 17 , wherein the EM radiation received from the object is reflected, refracted, or emitted from the object.
19. The system of claim 17 , wherein the image sensor comprises a single-lens spatial phase imaging (SPI) system.
20. The system of claim 17 , wherein in the image sensor comprises a plurality of image sensors, and the object comprises a plurality of objects, and the analytics engine is configured to:
generate in real-time a plurality of tensor fields corresponding to the plurality of objects; and
process the plurality of tensor fields according to the artificial intelligence algorithm to generate output data representing the plurality of objects.
21. The system of claim 17 , wherein the analytics engine is configured to:
generate synthetic training data by modifying an attribute value of an attribute in the tensor field; and
train the artificial intelligence algorithm using the synthetic training data and the field-of-view of the image sensor and a point-of-view of the image sensor.
22. The system of claim 16 , wherein the second-order primitives are generated based on the first-order primitives.
23. The system of claim 16 , wherein the attribute set of the tensor field represents the object at an orientation within a region of space, the attribute set comprising an attribute that is one of a scaler, a vector, a surface, or a shape.
24. The system of claim 16 , wherein the attribute set of the tensor field comprises respective attributes and respective confidence values for the respective attributes.
25. The system of claim 16 , wherein the tensor field comprise a first tensor field, the first-order primitives comprise pixel-level first-order primitives, the second-order primitives comprise pixel-level second-order primitives, the analytic engine is configured to:
identify a pixel cluster by clustering pixels according to one or more attributes in the pixel-level first-order primitives and the pixel-level second-order primitives, each of the one or more attributes of the pixels that are clustered having an attribute value within a predetermined interval to define surfaces of the object; and
generate a second tensor field corresponding to the pixel cluster, the second tensor field having a higher order than the first tensor field.
26. The system of claim 25 , wherein the first tensor field comprises one or more first attributes, the second tensor field comprises one or more second attributes, and each of the one or more second attributes comprises an attribute in surface-level first-order primitives or surface-level second-order primitives.
27. The system of claim 26 , wherein the artificial intelligence algorithm comprises a first artificial intelligence algorithm and a second artificial intelligence algorithm, the first artificial intelligence algorithm is configured to process the one or more first attributes in the first tensor field and the second artificial intelligence algorithm is configured to process the one or more second attributes of the second tensor field.
28. The system of claim 16 , wherein the analytic engine is configured to:
assign an attribute value to an attribute in the plurality of features in the tensor field.
29. The system of claim 16 , wherein the output data is generated based on the tensor field for one of the following applications: corrosion identification, object tracking, object identification, or object characterization, and wherein the applications are performed on objects comprising one or more of cracks, pits, blisters, skin lesions, or blood vessels.
30. The system of claim 16 , wherein the tensor field comprises a subgroup of attributes, the analytics engine is configured to normalize one or more attributes in the subgroup to generate one or more normalized attributes.