Object-centric spatial pooling for image classification

1. A method for classifying an image, comprising:

inferring location information of an object of interest in an input representation of the image;

determining foreground object features and background object features from the input representation of the image;

pooling the foreground object features separately from the background object features using the location information to form a new representation of the image, the new representation being different than the input representation of the image;

classifying the image based on the new representation of the image;

performing an outer iterative loop, the outer iterative loop comprising:

initializing a background region from the input representation of the image; and

incrementally shrinking at least a smallest bounding box from among a plurality of varying sized bounding boxes applied to the background region to incrementally increase a size of the background region with respect to at least the smallest bounding box;

performing an inner iterative loop that cooperates with the outer iterative loop, the inner iterative loop comprising:

inferring the location information of the object of interest from a current version of the smallest bounding box; and

from among a set of positive bounding boxes from positive images that are known to include the object of interest therein and from among a set of negative bounding boxes from negative images that are known to omit the object of interest therein, training a support vector machine classifier to discriminate the positive bounding boxes from the negative bounding boxes to refine the location information of the object of interest,

wherein said classifying step classifies the image based on discrimination results applied to a binary label of the image.

2. The method of claim 1 , further comprising separating the foreground object features from the background object features using only image-level class labels.

3. The method of claim 2 , wherein the image-level class levels are inferred using one or more bounding boxes.

4. The method of claim 2 , further comprising training an object classification model using the foreground object features, the background object features, and the image-level class labels.

5. The method of claim 1 , wherein said pooling and classifying steps combine the location information with classifying information in a joint model for both localization and classification.

6. The method of claim 1 , wherein at least the smallest bounding box is incrementally shrunk until a predetermined condition is reached.

7. The method of claim 6 , wherein the predetermined condition comprises at least the smallest bounding box entirely encompassing the object of interest including salient and non-salient portions thereof.

8. The method of claim 1 , wherein at least the smallest bounding box is positioned around a most prominent instance of an object class within the background region, the most prominent instant of the object class being equated to the object of interest.

9. The method of claim 1 , wherein the support vector machine classifier uses only features in the set of positive bounding boxes to the exclusion of features from an entirety of the image that are outside the positive bounding boxes.

10. The method of claim 1 , further comprising retraining an object classification model initially used to separate the foreground object features from the background object features based on the new representation of the image.

11. The method of claim 10 , further comprising updating a hypothesis of the location information of the object using the retrained object classification model, and repeating said determining and pooling steps to form another new representation of the Image.