Systems and methods for joint learning of complex visual inspection tasks using computer vision

1. A method for performing automatic visual inspection, comprising:

capturing visual information of an object using a scanning system comprising a plurality of cameras;

extracting, by a computing system comprising a processor and memory, one or more feature maps from the visual information using one or more feature extractors;

classifying, by the computing system, the object by supplying the one or more feature maps to a complex classifier to compute a classification of the object, the complex classifier comprising:

a plurality of simple classifiers, each simple classifier of the plurality of simple classifiers being configured to compute outputs representing a characteristic of the object and one or more of the simple classifiers comprising a K-ary output, where K is greater than two,

a decision tree connecting each simple classifier of the plurality of simple classifiers, wherein:

the K-ary output selects particular branches of the decision tree to take, and

different branches of the decision tree are associated with different trained models, and

one or more logical operators configured to combine the outputs of the simple classifiers to compute the classification of the object; and

outputting, by the computing system, the classification of the object as a result of the automatic visual inspection.

2. The method of claim 1 , wherein the one or more feature extractors comprise one or more convolutional neural networks.

3. The method of claim 1 , wherein the plurality of simple classifiers comprises one or more neural networks.

4. The method of claim 3 , wherein the plurality of simple classifiers comprises one or more support vector machines.

5. The method of claim 4 , wherein at least one logical operation is configured to combine an output of the one or more neural networks and an output of the one or more support vector machines.

6. The method of claim 1 , wherein the plurality of simple classifiers comprises one or more regression model.

7. The method of claim 1 , wherein the plurality of simple classifiers comprises one or more label-based classifier configured to perform on text detection.

8. The method of claim 1 , wherein each simple classifier of the plurality of simple classifiers is configured by a corresponding threshold parameter of a plurality of threshold parameters.

9. The method of claim 8 , wherein the plurality of threshold parameters are jointly trained.

10. The method of claim 1 , wherein the scanning system comprises:

a time-of-flight depth camera;

a structured light depth camera;

a stereo depth camera comprising at least two color cameras;

a stereo depth camera comprising at least two color cameras and a color projector;

a stereo depth camera comprising at least two infrared cameras; or

a stereo depth camera comprising at least two infrared cameras, an infrared projector; and a color camera.

11. A computer system for performing automatic visual inspection, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following:

capture visual information of an object using a scanning system comprising a plurality of cameras;

extract one or more feature maps from the visual information using one or more feature extractors;

classify the object by supplying the one or more feature maps to a complex classifier to compute a classification of the object, the complex classifier comprising:

a plurality of simple classifiers, each simple classifier of the plurality of simple classifiers being configured to compute outputs representing a characteristic of the object and one or more of the simple classifiers comprising a K-ary output, where K is greater than two,

a decision tree connecting each simple classifier of the plurality of simple classifiers, wherein:

the K-ary output selects particular branches of the decision tree to take, and

different branches of the decision tree are associated with different trained models, and

one or more logical operators configured to combine the outputs of the simple classifiers to compute the classification of the object; and

output, by the computing system, the classification of the object as a result of the automatic visual inspection.

12. The computer system of claim 11 , wherein the one or more feature extractors comprise one or more convolutional neural networks.

13. The computer system of claim 11 , wherein the plurality of simple classifiers comprises one or more neural networks.

14. The computer system of claim 13 , wherein the plurality of simple classifiers comprises one or more support vector machines.

15. The computer system of claim 14 , wherein at least one logical operation is configured to combine an output of the one or more neural networks and an output of the one or more support vector machines.

16. The computer system of claim 11 , wherein the plurality of simple classifiers comprises one or more regression model.

17. The computer system of claim 11 , wherein the plurality of simple classifiers comprises one or more label-based classifier configured to perform on text detection.

18. The computer system of claim 11 , wherein each simple classifier of the plurality of simple classifiers is configured by a corresponding threshold parameter of a plurality of threshold parameters.

19. The computer system of claim 18 , wherein the plurality of threshold parameters are jointly trained.

20. The computer system of claim 11 , wherein the scanning system comprises:

a time-of-flight depth camera;

a structured light depth camera;

a stereo depth camera comprising at least two color cameras;

a stereo depth camera comprising at least two color cameras and a color projector;

a stereo depth camera comprising at least two infrared cameras; or

a stereo depth camera comprising at least two infrared cameras, an infrared projector; and a color camera.