Predicate logic based image grammars for complex visual pattern recognition
First order predicate logics are provided, extended with a bilattice based uncertainty handling formalism, as a means of formally encoding pattern grammars, to parse a set of image features, and detect the presence of different patterns of interest implemented on a processor. Information from different sources and uncertainties from detections, are integrated within the bilattice framework. Automated logical rule weight learning in the computer vision domain applies a rule weight optimization method which casts the instantiated inference tree as a knowledge-based neural network, to converge upon a set of rule weights that give optimal performance within the bilattice framework. Applications are in (a) detecting the presence of humans under partial occlusions and (b) detecting large complex man made structures in satellite imagery (c) detection of spatio-temporal human and vehicular activities in video and (c) parsing of Graphical User Interfaces.
1. A method for a detection of a pattern having one or more features from image data of a scene, comprising:
defining a set of rules to detect the pattern based on the one or more features using a plurality of first order logic bilattice predicates;
obtaining the image data related to the scene;
a processor processing the image data with one or more detectors to detect the one or more features;
the processor executing the set of rules to detect the pattern based on a presence or an absence of each of the one or more features;
the processor generating data related to:
a justification that the set of rules detected the pattern;
a location in the scene where the pattern occurs, and
a measure of uncertainty related to the detection of the pattern;
wherein the set of rules is implemented as a knowledge-based artificial neural network;
wherein the measure of uncertainty related to the rule is expressed as a link weight in the knowledge-based artificial neural network; and wherein
the link weight is optimized by applying a change being expressed as:
Δ
w
ji
+
=
ηδ
j
[
∏
l
ϕ
(
o
il
)
]
[
1
-
⊎
k
≠
m
w
jm
+
∏
l
ϕ
(
o
ml
)
]
and
Δ
w
ji
-
=
-
ηδ
j
[
∏
l
ϕ
(
o
il
)
]
[
1
-
⊎
k
≠
m
w
jm
-
∏
l
ϕ
(
o
ml
)
]
with
Δw ji + is a change in link weight related to a propositional rule j for grounded atoms o ji , in a rule body,
Δw ji − is a change in link weight related to the propositional rule j against grounded atoms o ji , in the rule body,
φ(j) is an evidence for component of truth assignment to a propositional rule j corresponding to a node in the knowledge-based artificial neural network,
is a probabilistic sum operator,
η is a constant, and
δ j is a difference between an output of node j and a ground truth of node j.
2. The method as claimed in claim 1 , wherein a bilattice is a set of 2-tuples, quantifying a measure of belief for an assertion and a measure of belief against it.
3. The method as claimed in claim 1 , wherein the one or more rules include a measure of beliefs related to a rule.
4. The method as claimed in claim 1 , further comprising a set of composition rules which encodes the pattern as a hierarchy of parts.
5. The method as claimed in claim 1 , further comprising one or more embodiment rules which encode a geometric layout of the pattern.
6. The method as claimed in claim 1 , further comprising one or more context rules which encode a context of the pattern.
7. The method as claimed in claim 1 , wherein the method is applied to detect an object.
8. The method as claimed in claim 1 , wherein the method is applied to detect a human.