Defect detection using one or more neural networks
Apparatuses, systems, and techniques to facilitate feature detection of a manufactured object such as a PCB using combined images of said manufactured object. In at least one embodiment, an automated optical inspection system (AOI) comprising one or more neural networks can infer based, at least in part, on combined images of a PCB the existence of defects on said PCB.
1 . A system, comprising one or more processors to:
cause one or more neural networks to generate;
one or more first feature maps from a first set of one or more images of one or more manufactured objects; and
one or more second feature maps from a second set of one or more other images of the one or more manufactured objects;
generate a combined feature maps that includes features of the one or more first feature maps in addition to features of the one or more second feature maps; and
detect one or more defects in the one or more manufactured objects using the combined feature map.
2 . The system of claim 1 , further comprising two or more cameras to generate two or more images to be combined to obtain the combined feature map.
3 . The system of claim 1 , wherein the combined feature map is from two or more different images of the one or more manufactured objects captured from different locations.
4 . The system of claim 1 , wherein the combined feature map is generated based, at least in part, on multiple images of different lighting conditions.
5 . The system of claim 1 , wherein the combined feature map is generated based, at least in part, on multiple images from different viewing angles.
6 . The system of claim 1 , wherein the one or more processors are to perform one or more convolution operations to generate multiple images to be combined to generate the combined feature map.
7 . The system of claim 1 , wherein the one or more processors are to obtain output of one or more fully connected layers of the one or more neural networks to generate the combined feature map.
8 . The system of claim 1 , wherein the one or more processors are to use a loss function to compare the combined feature map of the one or more manufactured objects with one or more other representations of a sample object.
9 . The system of claim 1 , wherein:
two or more separate images comprise the combined feature map; and
the one or more processors to use the one or more neural networks are further to combine the two or more separate images after a series of filterings and before a comparison of the one or more second feature maps.
10 . One or more processors comprising circuitry to:
cause one or more neural networks to generate:
one or more first feature maps from a first set of one or more images of one or more manufactured objects; and
one or more second feature maps from a second set of one or more other images of the one or more manufactured objects; and
generate combined feature maps that includes features of the one or more first feature maps and features of the one or more second feature maps; and
detect one or more defects in the one or more manufactured objects using the combined feature maps.
11 . The one or more processors of claim 10 , wherein the one or more combined feature maps are generated based, at least in part, on multiple images from multiple video streams.
12 . The one or more processors of claim 10 , wherein the one or more combined feature maps are generated based, at least in part, on multiple images of different wavelengths of electromagnetic radiation.
13 . The one or more processors of claim 10 , wherein the one or more combined feature maps are generated based, at least in part, on multiple images of light directed at different angles.
14 . The one or more processors of claim 10 , wherein the one or more combined feature maps comprise at least a feature map based, at least in part, on a first output of one or more convolution operations and a second output of one or more convolution operations.
15 . The one or more processors of claim 10 , wherein the one or more processors are to further obtain output of one or more fully connected layers of the one or more neural networks to generate the one or more combined feature maps before providing the one or more combined feature maps as input to a loss function.
16 . The one or more processors of claim 10 , wherein the one or more processors are further to use a loss function to compare the one or more combined feature maps of the one or more manufactured objects with one or more other combined feature maps of a sample object.
17 . A method, comprising:
training one or more neural networks to generate a combined feature maps that includes features of one or more first feature maps from a first set of one or more images of one or more manufactured objects in addition to features of one or more second feature maps from a second set of one or more other images of the one or more manufactured objects to detect one or more defects in the one or more manufactured objects using the combined feature map.
18 . The method of claim 17 , wherein the combined feature map is generated based, at least in part, on at least one image of a top of the one or more manufactured objects and at least one image of a bottom of the one or more manufactured objects.
19 . The method of claim 17 , wherein the combined feature map is generated based, at least in part, on X-ray images.
20 . The method of claim 17 , wherein the combined feature map comprises at least a feature map generated based, at least in part, on:
a first output of one or more convolution operations;
a first output of one or more fully connected layers;
a second output of the one or more convolution operations; and
a second output of the one or more fully connected layers.
21 . The method of claim 17 , wherein the combined feature map is from two or more different images of the one or more manufactured objects combined before undergoing convolution operations.
22 . A non-transitory machine-readable medium having stored thereon a set of instructions, which, if performed by one or more processors, cause the one or more processors to at least:
cause one or more neural networks to generate;
one or more first feature maps from a first set of one or more images of one or more manufactured objects; and
one or more second feature maps from a second set of one or more other images of the one or more manufactured objects;
generate one or more combined feature maps that includes features of the one or more first feature maps in addition to features of the one or more second feature maps; and
detect defects in the one or more manufactured objects using the one or more combined feature maps.
23 . The non-transitory machine-readable medium of claim 22 , wherein:
the one or more combined feature maps are from two or more different images of the one or more manufactured objects; and
the two or more different images are combined into the combined feature maps after undergoing one or more convolution operations but before being input into one or more fully connected layers of the one or more neural networks.
24 . The non-transitory machine-readable medium of claim 22 , wherein:
the one or more combined feature maps are from two or more different images of the one or more manufactured objects; and
the two or more different images are combined into the one or more combined feature maps prior to undergoing one or more convolution operations.
25 . The non-transitory machine-readable medium of claim 22 , wherein the combined feature maps are generated based, at least in part, on images of the one or more manufactured objects being illuminated from a side by red, green, and blue (RGB) light and from above by the RGB light.
26 . The non-transitory machine-readable medium of claim 22 , wherein:
the combined feature maps are generated based, at least in part, on the one or more manufactured objects being exposed to red, green, and blue (RGB) light directed in two or more different directions and
the one or more combined feature maps of a sample object are generated based, at least in part, on the sample object being exposed to the RGB light directed in the two or more different directions.
27 . The non-transitory machine-readable medium of claim 22 , wherein:
a loss function of the one or more neural networks compares the one or more combined feature maps of one or more manufactured objects with one or more other combined feature maps of one or more sample objects; and
the one or more combined feature maps of the manufactured objects and the sample objects comprise multiple different images exposed to red, green, and blue (RGB) light.
28 . The system of claim 1 , wherein the combined feature map is a concatenation of the features of the one or more first feature maps and the features of the one or more second feature maps.