Object recognition for security screening and long range video surveillance

1. A method of detecting a weapon in image data, the method comprising:

annotating sections of a plurality of training images to indicate whether each section is one of a plurality of different constituent parts of a weapon;

encoding a pattern grammar describing the weapon using a plurality of first order logic based predicate rules;

training a plurality of distinct component detectors, where each detector is trained to each identify a corresponding one of the weapon parts based on the annotated training images;

processing image data with the component detectors to identify at least one of the weapon parts; and

executing the rules to detect the weapon based on the identified weapon parts.

2. The method of claim 1 , further comprising:

generating an explanation that explains why the weapon is considered a threat from a parse tree of the rules; and

presenting the explanation graphically to a user.

3. The method of claim 1 , wherein the explanation includes features in the weapon that were discovered by at least one of the component detectors, and the method further comprises generating an explanation that explains why the weapon is considered a threat from a parse tree of the rules, where the explanation describes how those features are connected to one another.

4. The method of claim 2 , wherein the explanation includes a graphical virtual assembly of the weapon from the weapon parts.

5. The method of claim 1 , wherein the training of the component detectors is performed using Adaptive Boosting.

6. The method of claim 1 , wherein the pattern grammar is implemented as instructions in a processor, and executing of the rules is performed by the processor executing the instructions.

7. The method of claim 1 , wherein the weapon is a knife and the annotated sections indicate whether each weapon part is one of a handle, a guard, and a blade of the knife.

8. The method of claim 1 , wherein the weapon is a gun and the annotated sections indicate whether each weapon part is one of a lock, a stock, and a barrel of the gun.

9. The method of claim 1 , wherein the weapon is a detonator and the annotated sections indicate whether each weapon part is one of a tube and an explosive material.

10. The method of claim 1 , wherein the weapon is a bomb and the annotated sections indicate whether each component is one of a detonator, explosive material, a cable, and a battery.

11. The method of claim 1 , wherein the image data is X-ray image data.

12. The method of claim 10 , wherein the image data is computed tomography (CT) data.

13. The method of claim 1 , wherein the training comprises:

determining uncertainty values for each of the rules;

converting the rules into a knowledge-based artificial neural network, where each uncertainty value corresponds to a weight of a link in the neural network; and

using a back-propagation algorithm modified to allow local gradients over a bilattice specific inference operation to optimize the link weights.

14. The method of claim 1 , wherein the pattern grammar describes a visual pattern of the weapon by encoding knowledge about contextual clues, scene geometry, and visual pattern constraints.

15. A non-transitory computer readable storage medium embodying instructions executable by a processor to perform method steps for detecting a weapon in image data, the method steps comprising instructions for:

annotating sections of a plurality of training images to indicate whether each section is one of a plurality of different constituent parts of a weapon;

encoding a pattern grammar describing the weapon using a plurality of first order logic based predicate rules;

training a plurality of distinct component detectors, where each detector is trained to each identify a corresponding one of the weapon parts based on the annotated training images;

processing image data with the component detectors to identify at least one of the weapon parts; and

executing the rules to detect the weapon based on the identified weapon parts.

16. A method of training a threat detector to detect an object in image data that is deemed to be a threat, the method comprising:

defining a pattern grammar to describe a visual pattern that is representative of the object;

encoding the pattern grammar using a plurality of first order predicate based logic rules; and

dividing an object into component parts;

training distinct component detectors to each detect a corresponding one of the component parts; and

generating the threat detector from the rules,

wherein the training comprises:

superimposing an artifact with one of the component parts in at least one prior training image annotated to indicate that presence of the one component part to generate a new training image; and

training the component detector for detecting the one component part using the prior training images and the new training image.

17. The method of claim 16 , wherein the pattern grammar is implemented as instructions in a processor.

18. The method of claim 16 , wherein the training comprises: determining uncertainty values for each of the rules; converting the rules into a knowledge-based artificial neural network, where each uncertainty value corresponds to a weight of a link in the neural network; and using a back-propagation algorithm modified to allow local gradients over a bilattice specific inference operation to optimize the link weights.

19. The method of claim 1 further comprises superimposing an artifact with one of the weapon parts in one of the annotated training images to generate a new training image and the training is based on the based on the annotated training images and the new training image.

20. The computer readable storage medium of claim 15 , wherein the instructions further comprise instructions for superimposing an artifact with one of the weapon parts in one of the annotated training images to generate a new training image and the training is based on the based on the annotated training images and the new training image.