IP Library Granted Patent US 10,474,949
Granted Patent B2
US 10,474,949 · App. 14/528,890 · Granted Nov 12, 2019

Knowledge-graph biased classification for data

Inventors: Somdeb Majumdar (San Diego, CA); Regan Blythe Towal (San Diego, CA); Sachin Subhash Talathi (San Diego, CA); David Jonathan Julian (San Diego, CA); Venkata Sreekanta Reddy Annapureddy (San Diego, CA)
Assignee: Qualcomm Incorporated
G06N3/08G06N3/04G06N3/0427G06N7/005G06N3/049
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,474,949
App. No.
14/528,890
Granted
Nov 12, 2019
Kind
B2
Abstract

A method for classifying an object includes applying multiple confidence values to multiple objects. The method also includes determining a metric based on the multiple confidence values. The method further includes determining a classification of a first object from the multiple objects based on a knowledge-graph when the metric is above a threshold.

Claims (30)

1. A method for computer-implemented tagging of objects in an image, the method comprising:

processing the image by a computer-implemented artificial neural network to identify at least a first and a second object and generate, for at least the first object, at least a first and a second predicted classification and corresponding confidence values, the first object being partially visible in the image;

generating an adjusted first confidence value by adjusting the confidence value corresponding to the first predicted classification based on a predetermined co-existence probability for the first predicted classification and the second object and also at least one of a physical location of an image capturing device or a time when the image was captured;

generating an adjusted second confidence value by adjusting the confidence value corresponding to the second predicted classification based on a predetermined co-existence probability for the second predicted classification and the second object and also at least one of the physical location of the image capturing device or the time when the image was captured;

determining a classification for the first object based on the first and second predicted classifications and the corresponding adjusted first and second confidence values; and

generating a tag for the first object using the determined classification.

2. The method of claim 1 , wherein the predetermined co-existence probabilities are obtained from a knowledge-graph matrix.

3. The method of claim 2 , further comprising updating the knowledge-graph matrix based on the classification of the first object.

4. The method of claim 3 , wherein the updating is based on user input.

5. The method of claim 1 , further comprising calculating a classifier confusion value based on a difference between the confidence values corresponding to the first and second predicted classifications.

6. The method of claim 5 , wherein the steps of generating adjusted confidence values are performed only if the classifier confusion value is greater than a threshold.

7. The method of claim 1 , wherein the image is a frame of a video stream.

8. A computer apparatus comprising a memory and a processor coupled to the memory, wherein the processor is adapted to:

process an image by a computer-implemented artificial neural network to identify at least a first and a second object and generate, for at least the first object, at least a first and a second predicted classification and corresponding confidence values, the first object being partially visible in the image;

generate an adjusted first confidence value by adjusting the confidence value corresponding to the first predicted classification based on a predetermined co-existence probability for the first predicted classification and the second object and also at least one of a physical location of an image capturing device or a time when the image was captured;

generate an adjusted second confidence value by adjusting the confidence value corresponding to the second predicted classification based on a predetermined co-existence probability for the second predicted classification and the second object and also at least one of the physical location of the image capturing device or the time when the image was captured;

determine a classification for the first object based on the first and second predicted classifications and the corresponding adjusted first and second confidence values; and

generate a tag for the first object using the determined classification.

9. The apparatus of claim 8 , wherein the predetermined co-existence probabilities are obtained from a knowledge-graph matrix.

10. The apparatus of claim 9 , wherein the processor is further adapted to update the knowledge-graph matrix based on the classification of the first object.

11. The apparatus of claim 10 , wherein the updating is based on user input.

12. The apparatus of claim 8 , wherein the processor is further adapted to calculate a classifier confusion value based on a difference between the confidence values corresponding to the first and second predicted classifications.

13. The apparatus of claim 12 , wherein the processor is adapted to generate the adjusted confidence values only if the classifier confusion value is greater than a threshold.

14. The apparatus of claim 8 , wherein the image is a frame of a video stream.

15. A non-transitory computer-readable medium having program code recorded thereon for computer-implemented tagging of objects in an image, the program code being executed by a processor and comprising:

program code to process the image by a computer-implemented artificial neural network to identify at least a first and a second object and generate, for at least the first object, at least a first and a second predicted classification and corresponding confidence values, the first object being partially visible in the image;

program code to generate an adjusted first confidence value by adjusting the confidence value corresponding to the first predicted classification based on a predetermined co-existence probability for the first predicted classification and the second object and also at least one of a physical location of an image capturing device or a time when the image was captured;

program code to generate an adjusted second confidence value by adjusting the confidence value corresponding to the second predicted classification based on a predetermined co-existence probability for the second predicted classification and the second object and also at least one of the physical location of the image capturing device or the time when the image was captured;

program code to determine a classification for the first object based on the first and second predicted classifications and the corresponding adjusted first and second confidence values; and

program code to generate a tag for the first object using the determined classification.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2014
From: MAJUMDAR, SOMDEB; TOWAL, REGAN BLYTHE; TALATHI, SACHIN SUBHASH; JULIAN, DAVID JONATHAN; ANNAPUREDDY, VENKATA SREEKANTA REDDY
To: QUALCOMM INCORPORATED
Reel/Frame 034314/0532 →
Continuity (2)
Provisional Application 62039329 · Aug 19, 2014
Related Publication 20160055409A1 · Feb 25, 2016
Cited By (2)
US 12,217,190 US 12,561,400