IP Library Granted Patent US 10,635,948
Granted Patent B2
US 10,635,948 · App. 16/123,773 · Granted Apr 28, 2020

Method for locating one or more candidate digital images being likely candidates for depicting an object

Inventors: Niclas Danielsson (Lund, SE); Simon Molin (Lund, SE); Markus Skans (Lund, SE); Jakob Grundström (Lund, SE)
Assignee: Axis AB
G06K9/6281G06F16/55G06F16/56G06F16/75
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,635,948
App. No.
16/123,773
Granted
Apr 28, 2020
Kind
B2
Abstract

A method for finding one or more candidate digital images being likely candidates for depicting a specific object comprising: receiving an object digital image depicting the specific object; determining, using a classification subnet of a convolutional neural network, a class for the specific object depicted in the object digital image; selecting, based on the determined class for the specific object depicted in the object digital image, a feature vector generating subnet from a plurality of feature vector generating subnets; determining, by the selected feature vector generating subnet, a feature vector of the specific object depicted in the object digital image; locating one or more candidate digital images being likely candidates for depicting the specific object depicted in the object digital image by comparing the determined feature vector and feature vectors registered in a database, wherein each registered feature vector is associated with a digital image.

Claims (26)

1. A method for finding one or more candidate digital images being likely candidates for depicting a specific object, the method comprising:

receiving an object digital image depicting the specific object;

processing the object digital image through convolutional layers in a base neural network of a convolutional neural network to populate activation maps pertaining to the specific object depicted in the object digital image;

determining, using a classification subnet of the convolutional neural network, a class, among a plurality of predetermined classes, for the specific object depicted in the object digital image, wherein the classification subnet is connected to activation maps in the base neural network;

selecting, based on the determined class for the specific object depicted in the object digital image, a feature vector generating subnet from a plurality of feature vector generating subnets of the convolutional neural network, wherein each of the plurality of feature vector generating subnets is linked to at least one of the plurality of predetermined classes;

determining, by the selected feature vector generating subnet, a feature vector of the specific object depicted in the object digital image; and

finding one or more candidate digital images being likely candidates for depicting the specific object depicted in the object digital image by finding one or more matches between the determined feature vector of the specific object depicted in the object digital image and feature vectors registered in a database comprising registered feature vectors of objects, wherein each registered feature vector is associated with a digital image.

2. The method according to claim 1 , wherein the classification subnet comprises one or more fully connected layers connected to the activation maps.

3. The method according to claim 2 , wherein the classification subnet further comprises a soft max layer connected to the one or more fully connected layers.

4. The method according to claim 1 , wherein one or more of the plurality of feature vector generating subnets comprises one or more fully connected layers connected to the activation maps.

5. The method according to claim 4 , wherein one or more of the plurality of feature vector generating subnets further comprises an embedding normalization layer arranged to map data from the activation maps onto a normalized vector structure to generate the determined feature vector.

6. The method according to claim 5 , wherein the determined feature vector is a vector including values from the embedding normalization layer.

7. The method according to claim 1 , wherein the act of finding one or more matches between the determined feature vector of the specific object depicted in the object digital image and the feature vectors registered in the database comprises:

calculating a distance between feature vectors registered in the database and the determined feature vector of the specific object depicted in the object digital image.

8. The method according to claim 7 , wherein the calculated distance is a Euclidian distance.

9. The method according to claim 1 , wherein the finding one or more candidate digital images being likely candidates for depicting the specific object depicted in the object digital image by comparing the determined feature vector of the specific object depicted in the object digital image and feature vectors registered in the database further comprises:

creating a sorted similarity list, wherein each feature vector is sorted according to its corresponding calculated distance.

10. The method according to claim 1 , wherein the one or more specific feature vectors among the feature vectors that are matched with the determined feature vector of the specific object depicted in the object digital images have at least one characteristic from the list of

a calculated distance to the determined feature vector being smaller than the remaining calculated distances,

a calculated distance to the determined feature vector being smaller than a threshold value, and

a fixed number of specific feature vectors having the smallest distance to the determined feature vector among the feature vectors registered in the database.

11. The method according to claim 1 , further comprising storing the determined feature vector of the specific object depicted in the object digital image in the database and associating the determined feature vector with the object digital image.

12. The method according to claim 1 ,

wherein the database is divided into a plurality of class databases, wherein each class database comprises registered feature vectors of objects belonging to the class; and

wherein the method further comprises: selecting, based on the determined class for the specific object depicted in the object digital image, a specific class database from the plurality of class databases.

13. The method according to claim 12 , further comprising storing the determined feature vector of the specific object depicted in the object digital image in the specific class database and associating the determined feature vector with the object digital image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2018
From: DANIELSSON, NICLAS; MOLIN, SIMON; SKANS, MARKUS; GRUNDSTRÖM, JAKOB
To: AXIS AB
Reel/Frame 046806/0180 →
Priority Claims (1)
EP 17191310 · Sep 15, 2017 · regional
Continuity (1)
Related Publication 20190087687A1 · Mar 21, 2019