Object detection approach using generative sparse, hierarchical networks with top-down and lateral connections for combining texture/color detection and shape/contour detection
An approach to detecting objects in an image dataset may combine texture/color detection, shape/contour detection, and/or motion detection using sparse, generative, hierarchical models with lateral and top-down connections. A first independent representation of objects in an image dataset may be produced using a color/texture detection algorithm. A second independent representation of objects in the image dataset may be produced using a shape/contour detection algorithm. A third independent representation of objects in the image dataset may be produced using a motion detection algorithm. The first, second, and third independent representations may then be combined into a single coherent output using a combinatorial algorithm.
1. A computer-implemented method, comprising:
producing, by a computing system, a first independent representation of objects in an image dataset using a color/texture detection algorithm;
producing, by the computing system, a second independent representation of objects in the image dataset using a shape/contour detection algorithm; and
combining, by the computing system, the first and second independent representations into a single coherent output using a combinatorial algorithm, wherein the first independent representation of objects and the second independent representation of objects are independent from one another,
wherein the combining of the first and second independent representations into the single coherent output using the combinatorial algorithm comprises:
formatting, by the computing system, the first representation and the second representation as points in a matrix a size of a video frame of the image dataset;
forming, by the computing system, clusters based on hits that are within a predetermined proximity and of a same label; and
finding, by the computing system, a minimum volume enclosing ellipsoid to get an appropriate size and orientation of a bounding box.
2. The computer-implemented method of claim 1 , wherein the color/texture detection algorithm is configured to learn a sparse overcomplete feature dictionary for the image dataset.
3. The computer-implemented method of claim 1 , wherein the shape/contour detection algorithm is configured to take into account lateral interactions based on object-distractor difference (ODD) kernels.
4. The computer-implemented method of claim 1 , wherein the first independent representation comprises rasterized PNG images and pixel values represent different categories.
5. The computer-implemented method of claim 1 , wherein the second independent representation comprises neuron activity files.
6. The computer-implemented method of claim 1 , wherein, for each frame in the image dataset, the color/texture detection algorithm gives a single output and the shape/contour detection algorithm gives an output for each object of interest.
7. The computer-implemented method of claim 1 , further comprising:
producing, by the computing system, a third independent representation of objects in the image dataset using a motion detection algorithm, wherein
the combining further comprises combining the third independent representation with the first and second independent representation into the single coherent output using the combinatorial algorithm.
8. The computer-implemented method of claim 7 , wherein the motion detection algorithm is configured to derive spatio-temporal filter properties for V1 neurons from their synaptic input weights, tuned such that a center frequency of a temporal band)(ω t0 ) divided by a center frequency of a spatial frequency band (ω x0 ) gives a velocity matched by a V1 cell.
9. The computer-implemented method of claim 8 , wherein the spatial filter is implemented using a Gaussian-shaped excitatory connection surrounded by inhibitory Gaussian shaped flanks.
10. The computer-implemented method of claim 7 , wherein an elliptical Gaussian-shaped excitatory connection located at x′=0 and y′=0 is surrounded by inhibitory Gaussian shaped flanks shifted by an amount x 0 and y 0 and centered about an instant in time denoted by t′ is given by:
w
(
x
,
y
,
t
)
=
ⅇ
(
x
′
)
2
2
σ
x
2
ⅇ
-
y
2
2
σ
y
2
ⅇ
(
t
′
)
2
2
σ
t
2
-
ⅇ
(
x
′
-
x
0
)
2
2
σ
x
2
ⅇ
-
y
2
2
σ
y
2
ⅇ
(
t
′
)
2
2
σ
t
2
-
ⅇ
(
x
′
+
x
0
)
2
2
σ
x
2
ⅇ
-
y
2
2
σ
y
2
ⅇ
(
t
′
)
2
2
σ
t
2
11. The computer-implemented method of claim 10 , wherein a center of the elliptical Gaussian is shifted in time and space according to:
x′=x cos θ−t sin θ; t′=t cos θ+x sin θ.
12. The computer-implemented method of claim 7 , further comprising:
using, by the computing system, motion detections to modify confidence values.
13. The computer-implemented method of claim 1 , further comprising:
using, by the computing system, background labels from the color/texture detection algorithm to modify confidence values.
14. A computer program embodied on a non-transitory computer-readable medium, the program configured to cause at least one processor to:
produce a first independent representation of objects in an image dataset using a color/texture detection algorithm;
produce a second independent representation of objects in the image dataset using a shape/contour detection algorithm;
produce a third independent representation of objects in the image dataset using a motion detection algorithm; and
combine the first, second, and third independent representations into a single coherent output using a combinatorial algorithm, wherein the first independent representation of objects, the second independent representation of objects, and the third independent representation of objects are independent from one another,
wherein the combining of the first, second, and third independent representations into the single coherent output using the combinatorial algorithm comprises:
formatting, by the computing system, the first, second, and third representations as points in a matrix a size of a video frame of the image dataset;
forming, by the computing system, clusters based on hits that are within a predetermined proximity and of a same label; and
finding, by the computing system, a minimum volume enclosing ellipsoid to get an appropriate size and orientation of a bounding box.
15. The computer program of claim 14 , wherein, for each frame in the image dataset, the color/texture detection algorithm gives a single output and the shape/contour detection algorithm gives an output for each object of interest.
16. The computer program of claim 14 , wherein the motion detection algorithm is configured to derive spatio-temporal filter properties for V1 neurons from their synaptic input weights, tuned such that a center frequency of a temporal band (ω t0 ) divided by a center frequency of a spatial frequency band (ω x0 ) gives a velocity matched by a V1 cell.
17. An apparatus, comprising: memory storing computer program instructions; and at least one processor configured to execute the computer program instructions, the at least one processor configured to:
combine a first independent representation of objects in an image dataset produced by a color/texture detection algorithm and a second independent representation of objects in the image dataset produced by a shape/contour detection algorithm into a single coherent output using a combinatorial algorithm, wherein the first independent representation of objects and the second independent representation of objects are independent from one another,
wherein the combining of the first and second independent representations into the single coherent output using the combinatorial algorithm comprises:
formatting, by the computing system, the first representation and the second representation as points in a matrix a size of a video frame of the image dataset;
forming, by the computing system, clusters based on hits that are within a predetermined proximity and of a same label; and
finding, by the computing system, a minimum volume enclosing ellipsoid to get an appropriate size and orientation of a bounding box.
18. The apparatus of claim 17 , wherein the at least one processor is further configured to produce a third independent representation of objects in the image dataset using a motion detection algorithm, and the combining further comprises combining the third independent representation with the first and second independent representation into the single coherent output using the combinatorial algorithm.