Hybrid deep learning for anomaly detection
Hybrid deep learning systems and methods allow for detecting anomalies in objects, such as electrical printed circuit board (PCB) components, based on image data. In one or more embodiments, a hybrid deep learning model comprises a Graph Attention Network (GAT) that uses spatial properties of the PCB components to extract latent semantic information and generate an output set of hidden representations. The GAT treats each of the electrical components as a node and each connection between them as edges in a graph. The hybrid system further comprises a Convolutional Neural Network (CNN) that uses pixel data to obtain its own output set of hidden representations. The hybrid deep learning model concatenates both sets to detect anomalies that may be present on the PCB.
1. A method for detecting an anomaly, the method comprising:
receiving, at a Graph Attention Network (GAT), data associated with objects, the GAT
threating each of the objects as a node, treating connections between objects as edges in a graph, and performing a graph-based convolution using multi-head attention;
obtaining from the GAT a first output set of hidden representations;
receiving, at a Convolutional Neural Network (CNN), at least some of the data;
obtaining from the CNN a second output set of hidden representations;
concatenating the first and second output sets of hidden representations to obtain a concatenated set of representations; and
detecting whether an anomaly exists using the concatenated set of representations and a third neural network, which receives the concatenated set of representations as an input.
2. The method according to claim 1 , wherein the data comprises at least one of a 2D surface image, a 2D X-ray image comprising sub-surface layers, and a 3D tomograph of 2D images generated using computer-aided tomography.
3. The method according to claim 1 , wherein the GAT uses local neighborhood information that is based on proximity between the nodes to extract high level representations of the objects in the graph to obtain spatial properties of the objects.
4. The method according to claim 3 , wherein the GAT establishes neighbor relationships between the nodes by using filters to aggregate the local neighborhood information.
5. The method according to claim 1 , wherein the GAT obtains a link prediction, a node detection, or both.
6. The method according to claim 1 , wherein the CNN uses the data to perform a pixel-based convolution operation.
7. The method according to claim 1 , wherein the CNN has been trained using pixel data from sub-images.
8. A system for detecting an anomaly, the system comprising:
one or more processors; and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
receiving, at a Graph Attention Network (GAT), data associated with objects, the GAT treating each of the objects as a node, treating connections between objects as edges in a graph, and using local neighborhood information that is based on proximity between the nodes to extract high level representations of the objects in the graph to obtain spatial properties of the objects;
obtaining from the GAT a first output set of hidden representations;
receiving, at a Convolutional Neural Network (CNN), at least some of the data;
obtaining from the CNN a second output set of hidden representations;
concatenating the first and second output sets of hidden representations to obtain a concatenated set of representations; and
detecting whether an anomaly exists using the concatenated set of representations and a third neural network, which receives the concatenated set of representations as an input.
9. The system according to claim 8 , wherein the data comprises at least one of a 2D surface image, a 2D X-ray image comprising sub-surface layers, and a 3D tomograph of 2D images generated using computer-aided tomography.
10. The system according to claim 8 , wherein the objects are at least one of components on a printed circuit board or sub-components within components on the printed circuit board, and connections between the components or sub-components are traces.
11. The system according to claim 8 , wherein the CNN performs a pixel-based convolution operation on pixel data obtained from the data.
12. The system according to claim 8 , wherein the GAT objects obtains a link prediction, a node detection, or both.
13. The system according to claim 12 , wherein the GAT establishes neighbor relationships between the nodes by using filters to aggregate the local neighborhood information.
14. A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by at least one processor, causes steps to be performed comprising:
receiving, at a Graph Attention Network (GAT), data associated with objects, the GAT treating each of the objects as a node, treating connections between objects as edges in a graph, and performing a graph-based convolution using multi-head attention;
obtaining from the GAT a first output set of hidden representations;
receiving, at a Convolutional Neural Network (CNN), at least some of the data;
obtaining from the CNN a second output set of hidden representations;
concatenating the first and second output sets of hidden representations to obtain a concatenated set of representations; and
detecting whether an anomaly exists using the concatenated set of representations and a third neural network, which receives the concatenated set of representations as an input.
15. The non-transitory computer-readable medium or media of claim 14 , wherein the objects are at least one of components on a printed circuit board or sub-components within components on the printed circuit board, and connections between the components or sub-components are traces.
16. The non-transitory computer-readable medium or media of claim 15 , wherein the GAT uses local neighborhood information that is based on proximity between the nodes to extract high level representations of the objects in the graph to obtain spatial properties of the objects.
17. The non-transitory computer-readable medium or media of claim 16 , wherein the GAT establishes neighbor relationships between the nodes by using filters to aggregate the local neighborhood information.
18. The non-transitory computer-readable medium or media of claim 14 , wherein the CNN uses the data to perform a pixel-based convolution operation.
19. The non-transitory computer-readable medium or media of claim 14 , wherein the CNN has been trained using pixel data from sub-images.
20. The non-transitory computer-readable medium or media of claim 14 , wherein the data comprises at least one of a 2D surface image, a 2D X-ray image comprising sub-surface layers, and a 3D tomograph of 2D images generated using computer-aided tomography.