IP Library Granted Patent US 10,769,496
Granted Patent B2
US 10,769,496 · App. 16/171,129 · Granted Sep 8, 2020

Logo detection

Inventors: Brunno Fidel Maciel Attorre (New York, NY); Nicolas Huynh Thien (San Ramon, CA)
Assignee: Adobe Inc.
G06K9/6257G06K9/6202
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,769,496
App. No.
16/171,129
Granted
Sep 8, 2020
Kind
B2
Abstract

Disclosed herein are techniques for detecting logos in images or video. In one embodiment, a first logo detection model detects, from an image, candidate regions for determining logos in the image. A feature vector is then extracted from each candidate region and is compared with reference feature vectors stored in a database. The logo corresponding to the best matching reference feature vector is determined to be the logo in the candidate region if the best matching meets a certain criterion. In some embodiments, a second logo detection model trained using synthetic training images is used in combination with the first logo detection model to detect logos in a same image.

Claims (41)

1. A method comprising: receiving a source image at one or more computing devices;

detecting, in the source image and using a first logo detection model implemented by the one or more computing devices, a candidate region for determining a logo in the source image; extracting, from the candidate region and by a neural network implemented using the one or more computing devices, a feature vector of the candidate region; determining, for each reference feature vector from a set of reference feature vectors stored in a database, a respective matching score between the reference feature vector and the feature vector of the candidate region, wherein each reference feature vector in the set of reference feature vectors is extracted from a respective image of a respective target logo in a set of target logos; selecting a first reference feature vector associated with a highest matching score among the set of reference feature vectors, the first reference feature vector extracted from a first image of a first target logo in the set of target logos; determining that the candidate region includes an image of the first target logo based on determining that the highest matching score is greater than a threshold value; receiving an image of a new target logo to be added to the set of target logos; extracting, using the neural network implemented using the one or more computing devices, a new reference feature vector from the image of the new target logo; and storing the new reference feature vector in the database.

2. The method of claim 1 , wherein the first logo detection model includes a fast region-based convolutional neural network (R-CNN).

3. The method of claim 1 , wherein the neural network includes a convolutional neural network.

4. The method of claim 1 , further comprising:

detecting, from the source image and using a second logo detection model implemented by the one or more computing devices, a second target logo in the set of target logos,

wherein the second logo detection model is trained using training images including the set of target logos to detect the set of target logos.

5. The method of claim 4 , wherein the second logo detection model includes a Fast R-CNN.

6. The method of claim 4 , wherein: the first logo detection model and the second logo detection model are trained using synthetic training images; a synthetic training image includes a background image and an image of a target logo in the set of target logos; and the image of the target logo is superimposed on the background image.

7. The method of claim 6 , wherein the image of the target logo is transformed before being superimposed on the background image.

8. The method of claim 6 , wherein the image of the target logo is superimposed in a region of the background image, the region of the background image determined to be suitable for logo superposition.

9. The method of claim 8 , wherein determining that the region of the background image is suitable for logo superposition comprises:

classifying each pixel in the background image as suitable or unsuitable for logo superposition; and

grouping neighboring pixels that are classified as suitable for logo superposition to form the region of the background image for logo superposition.

10. The method of claim 1 , wherein the one or more computing devices include one or more cloud computing servers.

11. The method of claim 1 , wherein the source image includes images of two or more logos.

12. A system comprising:

a processing device; and

a non-transitory computer-readable medium communicatively coupled to the processing device, wherein the processing device is configured to execute program code stored in the non-transitory computer-readable medium and thereby perform operations comprising:

receiving a source image at the processing device;

detecting, in the source image and using a first logo detection model implemented by the processing devices, a candidate region for determining a logo in the source image;

extracting, from the candidate region and by a neural network implemented using the processing device, a feature vector of the candidate region;

determining, for each reference feature vector from a set of reference feature vectors stored in a database, a respective matching score between the reference feature vector and the feature vector of the candidate region, wherein each reference feature vector in the set of reference feature vectors is extracted from a respective image of a respective target logo in a set of target logos;

selecting a first reference feature vector associated with a highest matching score among the set of reference feature vectors, the first reference feature vector extracted from a first image of a first target logo in the set of target logos;

determining that the candidate region includes an image of the first target logo based on determining that the highest matching score is greater than a threshold value;

receiving an image of a new target logo to be added to the set of target logos;

extracting, using the neural network implemented using the processing device, a new reference feature vector from the image of the new target logo; and

storing the new reference feature vector in the database.

13. The system of claim 12 , wherein the operations further comprise:

detecting, from the source image and using a second logo detection model implemented by the processing device, a second target logo in the set of target logos,

wherein the second logo detection model is trained using training images including the set of target logos to detect the set of target logos.

14. The system of claim 13 , wherein: the first logo detection model and the second logo detection model are trained using synthetic training images; a synthetic training image includes a background image and an image of a target logo in the set of target logos; and the image of the target logo is superimposed on the background image.

15. The system of claim 14 , wherein:

the image of the target logo is superimposed in a region of the background image; and

the region of the background image is determined by:

classifying each pixel in the background image as suitable or unsuitable for logo superposition; and

grouping neighboring pixels that are classified as suitable for logo superposition to form the region of the background image for logo superposition.

16. A system comprising: means for receiving a source image; means for detecting, in the source image and using a first logo detection model, a candidate region for determining a logo in the source image; means for extracting, from the candidate region and using a neural network, a feature vector of the candidate region; means for determining, for each reference feature vector from a set of reference feature vectors stored in a database, a respective matching score between the reference feature vector and the feature vector of the candidate region, wherein each reference feature vector in the set of reference feature vectors is extracted from a respective image of a respective target logo in a set of target logos; means for selecting a first reference feature vector associated with a highest matching score among the set of reference feature vectors, the first reference feature vector extracted from a first image of a first target logo in the set of target logos; means for determining that the candidate region includes an image of the first target logo based on determining that the highest matching score is greater than a threshold value; means for receiving an image of a new target logo to be added to the set of target logos; means for extracting, using the neural network, a new reference feature vector from the image of the new target logo; and means for storing the new reference feature vector in the database.

17. The system of claim 16 , further comprising:

means for detecting, from the source image and using a second logo detection model, a second target logo in the set of target logos,

wherein the second logo detection model is trained using training images including the set of target logos to detect the set of target logos.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 14, 2020
From: ATTORRE, BRUNNO FIDEL MACIEL; THIEN, NICOLAS HUYNH
To: ADOBE INC.
Reel/Frame 052668/0100 →
CHANGE OF NAME Recorded Mar 6, 2019
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 048525/0042 →