Granted Patent B2

US 10,318,848 · App. 15/247,805 · Granted Jun 11, 2019

Inventors: Daniel Hendricus Franciscus Dijkman (Haarlem, NL); David Jonathan Julian (San Diego, CA)

Assignee: QUALCOMM Incorporated

G06K9/66G06K9/3233G06K9/4628G06K9/623G06K9/627G06K9/628G06K9/6257G06K9/6261

Abstract

Claims (59)

1. A method for image classification, comprising:

labelling, by a first artificial neural network, a crop from an image including an object of interest with an indication of whether the object of interest is framed, partially framed, or not present in the crop;

assigning a fully framed class to the labelled crop, including the object of interest, when the object of interest is framed;

assigning a partially framed class to the labelled crop, including the object of interest, when the object of interest is partially framed;

assigning a background class to the labelled crop when the object of interest is not present in the crop; and

classifying the object of interest in the image based on a class assigned to the labelled crop.

2. The method of claim 1 , further comprising:

weighting a class estimate from a second artificial neural network based on whether the labelled crop is indicated as the background class.

3. The method of claim 1 , further comprising:

determining a proportion of the object of interest that is included in the crop; and

comparing the determined proportion to at least one threshold value, and in which the labelling is based on the comparing.

4. A method of training an artificial neural network for image classification, comprising:

selecting a crop from an image;

determining whether the crop of the image includes an area surrounded by a predetermined bounding box;

discarding, prior to the training, the crop when the crop of the image does not include a portion of the area surrounded by the predetermined bounding box; and

training the artificial neural network to determine a label of the crop of the image including an object of interest with an indication of whether the object of interest is framed, partially framed, or not present in the crop, the artificial network trained for the image classification of the object of interest in the image with a plurality of crops from the image, the classification based on a class assigned to the labelled crop, each crop comprising at least a portion of the area surrounded by the predetermined bounding box.

5. The method of claim 4 , further comprising generating a proposed bounding box for the image if the image lacks the predetermined bounding box, before discarding the crop.

6. The method of claim 5 , further comprising:

selecting a new crop from the image;

determining whether the new crop of the image includes the area surrounded by the proposed bounding box; and

discarding the new crop if the new crop of the image does not include a portion of the area surrounded by the proposed bounding box.

7. The method of claim 5 , in which the generating comprises:

localizing at least one object in the image; and

adding a border around the localized at least one object.

8. The method of claim 7 , further comprising:

selecting a second crop including the proposed bounding box;

determining a proportion of the localized at least one object that is included in the second crop; and

updating the proposed bounding box based on the determined proportion.

9. An apparatus for image classification, comprising:

a memory; and

at least one processor coupled to the memory, the at least one processor configured:

to label, by a first artificial neural network, a crop from an image, including an object of interest with an indication of whether the object of interest is framed, partially framed, or not present in the crop;

to assign a fully framed class to the labelled crop, including the object of interest, when the object of interest is framed;

to assign a partially framed class to the labelled crop, including the object of interest, when the object of interest is partially framed;

to assign a background class to the labelled crop when the object of interest is not present in the crop; and

to classify the object of interest in the image based on a class assigned to the labelled crop.

10. The apparatus of claim 9 , in which the at least one processor is further configured to weight a class estimate from a second artificial neural network, based on whether the labelled crop is indicated as the background class.

11. The apparatus of claim 9 , in which the at least one processor is further configured:

to determine a proportion of the object of interest that is included in the crop; and

to compare the determined proportion to at least one threshold value, and in which the labelling is based on the comparing.

12. An apparatus for training an artificial neural network for image classification, comprising:

a memory; and

at least one processor coupled to the memory, the at least one processor configured:

to select a crop from an image;

to determine whether the crop of the image includes an area surrounded by a predetermined bounding box;

to discard, prior to the training, the crop when the crop of the image does not include a portion of the area surrounded by the predetermined bounding box; and

to train the artificial neural network to determine a label of the crop of the image including an object of interest with an indication of whether the object of interest is framed, partially framed, or not present in the crop, the artificial network trained for the image classification of the object of interest in the image with a plurality of crops from the image, the classification based on a class assigned to the labelled crop, each crop comprising at least a portion of the area surrounded by the predetermined bounding box.

13. The apparatus of claim 12 , in which the at least one processor is further configured to generate a proposed bounding box for the image if the image lacks the predetermined bounding box, before discarding the crop.

14. The apparatus of claim 13 , in which the at least one processor is further configured:

to select a new crop from the image;

to determine whether the new crop of the image includes the area surrounded by the proposed bounding box; and

to discard the new crop if the new crop of the image does not include a portion of the area surrounded by the proposed bounding box.

15. The apparatus of claim 13 in which the at least one processor is further configured to generate the proposed bounding box:

by localizing at least one object in the image; and

by adding a border around the localized at least one object.

16. The apparatus of claim 15 , in which the at least one processor is further configured:

to select a second crop including the proposed bounding box;

to determine a proportion of the localized at least one object that is included in the second crop; and

to update the proposed bounding box based on the determined proportion.

Assignments (1)

Continuity (2)

Cited By (18)

US 12,198,396 US 12,216,610 US 12,223,428 US 12,236,689 US 12,307,350 US 12,346,816 US 12,367,405 US 12,455,739 US 12,462,575 US 12,522,243 US 12,536,131 US 12,554,467 US 12,555,401 US 12,591,240 US 12,618,976 US 12,623,691 US 12,705,870 US 12,709,294