Regionlets with shift invariant neural patterns for object detection

1. A method for detecting an object in an image, comprising:

determining convolutional neural network responses on the image;

mapping the responses back to their spatial locations in the image;

constructing features densely extract shift invariant activations of a convolutional neural network to produce dense features for the image; and

determining a horizontal location x of a neural pattern feature as:

x

i

=

x

i

-

1

+

(

W

i

-

1

2

-

P

i

)

⁢

S

i

-

1

where i>1,

x

1

=

W

i

-

1

2

,

x i-1 is a top-left location of a previous layer, W i is the window size of a convolutional or pooling layer, P i is a padding of a current layer, S i-1 is actual pixel stride of two adjacent neural patterns output by the previous layer.

2. The method of claim 1 , comprising sharing feature extraction computation for object candidates.

3. The method of claim 1 , comprising augmenting the features with hard-coded features.

4. The method of claim 1 , comprising densely extract shift invariant activations of a convolutional neural network trained on an image database.

5. The method of claim 1 , comprising feeding the features to a Regionlets framework.

6. The method of claim 5 , comprising using a boosting procedure to choose effective features.

7. The method of claim 1 , comprising determining:

S i =S i-1 ×s i ,

where s i is a current stride using neural patterns output by the previous layers as pixel; and

determining pixel locations of neural patterns in different layers recursively by going up the hierarchy.

8. The method of claim 1 , comprising determining a fixed length visual representation for a regionlet of arbitrary resolution.

9. The method of claim 8 , comprising generating a local Dense Neural Patterns (DNPs) histogram, or average pooling of DNPs, inside each regionlet.

10. The method of claim 9 , comprising generating DNPs in a regionlet r as {x i |iδ(1, . . . N r )}, where i indicates the index of the feature point, N r is the total number of feature points in regionlet r and determining a final feature for r as:

x

=

1

N

r

⁢

∑

i

=

1

N

r

⁢

x

i

.

11. The method of claim 9 , wherein each dimension of deep neural patterns corresponds to a histogram bin and their values from different spatial locations are accumulated inside a regionlet, comprising normalizing histograms using L-0 norm.

12. The method of claim 9 , comprising incorporating DNP into a Regionlets detector learning framework.

13. The method of claim 9 , comprising applying a weak learner based on a 1-D feature, and uniformly sample DNP Regionlets configuration space to construct a weak classifier pool.

14. The method of claim 13 , wherein the generated feature pool is easily augmented to the pool of other features including HOG, LBP or Covariance.

15. The method of claim 9 , wherein each configuration specifies a spatial configuration of Regionlets and feature dimension of the DNP.

16. A system for detecting an object in an image, comprising:

a processor for carrying out the following means:

means for determining convolutional neural network responses on the image;

means for mapping the responses back to their spatial locations in the image;

means for constructing features densely extract shift invariant activations of a convolutional neural network to produce dense features for the image; and

determining a horizontal location x of a neural pattern feature as:

x

i

=

x

i

-

1

+

(

W

i

-

1

2

-

P

i

)

⁢

S

i

-

1

where i>1,

x

1

=

W

1

-

1

2

,

x i-1 is a top-left location of a previous layer, W i is the window size of a convolutional or pooling layer, P i is a padding of a current layer, S i-1 is actual pixel stride of two adjacent neural patterns output by the previous lave.

17. The system of claim 16 , comprising means for sharing feature extraction computation for object candidates.

18. The system of claim 16 , comprising means for augmenting the features with hard-coded features.

19. The system of claim 16 , comprising means for densely extract shift invariant activations of a convolutional neural network trained on an image database.