IP Library Granted Patent US 11,093,800
Granted Patent B2
US 11,093,800 · App. 16/473,007 · Granted Aug 17, 2021

Method and device for identifying object and computer readable storage medium

Inventor: Xiaojun Tang (Beijing, CN)
Assignee: BOE TECHNOLOGY GROUP CO., LTD.
G06K9/627G06K9/628G06K9/6215G06N3/0454
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,093,800
App. No.
16/473,007
Granted
Aug 17, 2021
Kind
B2
Abstract

According to the embodiments of the present disclosure, there are provided a method and device for identifying an object and a computer readable storage medium. The method includes: determining candidate categories of the object using a first neural network; and determining, in response to the candidate categories of the object having been determined, a category of the object using second neural networks respectively corresponding to the candidate categories. The device includes: a processor; a memory having stored thereon instructions which, when executed by the processor, cause the processor to: determine candidate categories of the object using a first neural network; and determine, in response to the candidate categories of the object having been determined, a category of the object using second neural networks respectively corresponding to the candidate categories.

Claims (45)

1. A method for identifying an object, comprising:

determining candidate categories of the object using a first neural network; and

determining, in response to the candidate categories of the object having been determined, a category of the object using second neural networks respectively corresponding to the candidate categories:

wherein the determining candidate categories of the object using a first neural network comprises;

determining a first object feature vector of the object using the first neural network;

calculating first feature similarities between the first object feature vector and first reference feature vectors, respectively, wherein the first reference feature vectors are determined respectively based on the reference data of categories using the first neural network; and

determining categories corresponding to first feature similarities greater than or equal to a predetermined threshold among the first feature similarities as the candidate categories of the object;

wherein the determining a category of the object using second neural networks respectively corresponding to the candidate categories comprises:

determining, for each of the candidate categories, a second object feature vector of the object associated with the candidate category based on the first object feature vector using a corresponding second neural network;

calculating second feature similarities between second object feature vectors of the object respectively associated with the candidate categories and second reference feature vectors, respectively, wherein the second reference features vectors are determined based on the first reference vectors using the second neural networks, respectively; and

determining a category corresponding to the largest one of the second feature similarities as the category of the object.

2. The method according to claim 1 , wherein the predetermined threshold is a threshold which is set uniformly for all the first feature similarities or the predetermined threshold is thresholds which are set respectively for the first feature similarities and can be set independently of each other.

3. The method according to claim 1 , further comprising:

determining, in response to determining that there is only one candidate category of the object, the one candidate category as the category of the object.

4. The method according to claim 1 , further comprising:

outputting, in response to determining that there is no candidate category of the object, a message indicating that the object cannot be identified.

5. The method according to claim 1 , wherein each of the second neural networks is trained by:

using two samples belonging to a category corresponding to the second neural network as a positive sample pair, an expected output value of which is a positive reference value;

using a sample belonging to the category corresponding to the second neural network and a sample not belonging to the category corresponding to the second neural network as a negative sample pair, and an expected output value of which is a negative reference value; and

using a squared error between a calculated value of a corresponding second feature similarity and an expected output value as a loss function.

6. The method according to claim 1 , wherein the first neural network is a convolutional neural network from which a fully connected layer for final classification is removed, and each of the second neural networks is a single layer fully connected neural network.

7. A device for identifying an object, comprising:

a processor;

a memory having stored thereon instructions which, when executed by the processor, cause the processor to:

determine candidate categories of the object using a first neural network; and

determine, in response to the candidate categories of the object having been determined, a category of the object using second neural networks corresponding to the candidate categories, respectively,

wherein the instructions, when executed by the processor, further cause the processor to:

determine a first object feature vector of the object using the first neural network;

calculate first feature similarities between the first object feature vector and first reference feature vectors, respectively, wherein the first reference feature vectors are determined respectively based on the reference data of categories using the first neural network; and

determine categories corresponding to first feature similarities greater than or equal to a predetermined threshold among the first feature similarities as the candidate categories of the object;

wherein the instructions, when executed by the processor, further cause the processor to:

determine, for each of the candidate categories, a second object feature vector of the object associated with the candidate category based on the first object feature vector using a corresponding second neural network;

calculate second feature similarities between second object feature vectors of the object respectively associated with the candidate categories and second reference feature vectors, respectively, wherein the second reference features vectors are determined based on the first reference vectors using the second neural networks, respectively; and

determine a category corresponding to the largest one of the second feature similarities as the category of the object.

8. The device according to claim 7 , wherein the predetermined threshold is a threshold which is set uniformly for all the first feature similarities or the predetermined threshold is thresholds which are set respectively for the first feature similarities and can be set independently of each other.

9. The device according to claim 7 , wherein the instructions, when executed by the processor, further cause the processor to:

determine, in response to determining that there is only one candidate category of the object, the one candidate category as the category of the object.

10. The device according to claim 7 , wherein the instructions, when executed by the processor, further cause the processor to:

output, in response to determining that there is no candidate category of the object, a message indicating that the object cannot be identified.

11. The device according to claim 7 , wherein each of the second neural networks is trained by:

using two samples belonging to a category corresponding to the second neural network as a positive sample pair, an expected output value of which is a positive reference value;

using a sample belonging to the category corresponding to the second neural network and a sample not belonging to the category corresponding to the second neural network as a negative sample pair, an expected output value of which is a negative reference value; and

using a squared error between a calculated value of a corresponding second feature similarity and an expected output value as a loss function.

12. The device according to claim 7 , wherein the first neural network is a convolutional neural network from which a fully connected layer for final classification is removed, and each of the second neural networks is a single layer fully connected neural network.

13. A non-transitory computer readable storage medium having stored thereon instructions which, when executed by a processor, cause the processor to perform the method according to claim 1 .

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2022
From: BOE TECHNOLOGY GROUP CO., LTD.
To: BEIJING BOE TECHNOLOGY DEVELOPMENT CO., LTD.
Reel/Frame 060826/0252 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2019
From: TANG, XIAOJUN
To: BOE TECHNOLOGY GROUP CO., LTD.
Reel/Frame 049564/0916 →
Priority Claims (1)
CN 201810389080.2 · Apr 26, 2018 · national
Continuity (1)
Related Publication 20200380292A1 · Dec 3, 2020
Cited By (1)
US 12,188,823