IP Library › Granted Patent US 11,676,161
Granted Patent B2
US 11,676,161 · App. 17/181,735 · Granted Jun 13, 2023

Combined light and heavy models for image filtering

Inventors: Yi Yang (Princeton, NJ); Min Feng (Monmouth, NJ); Srimat Chakradhar (Manalapan, NJ)
Assignee: NEC Corporation
G06Q30/0201G06T7/0014G06V10/764G06V10/809G06V10/82G06V40/173G06T2207/20081G06T2207/20084G06T2207/30201G06V40/179
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,676,161
App. No.
17/181,735
Granted
Jun 13, 2023
Kind
B2
Abstract

Systems and methods for demographic determination using image recognition. The method includes analyzing an image with a pre-trained lightweight neural network model, where the lightweight neural network model generates a confidence value, and comparing the confidence value to a threshold value to determine if the pre-trained lightweight neural network model is sufficiently accurate. The method further includes analyzing the image with a pre-trained heavyweight neural network model for the confidence value below the threshold value, wherein the pre-trained heavyweight neural network model has above about one million trainable parameters and the pre-trained lightweight neural network model has a number of trainable parameters below one tenth the heavyweight model, and displaying demographic data to a user on a user interface, wherein the user modifies store inventory based on the demographic data.

Claims (31)

1. A method for demographic determination using image recognition, comprising:

analyzing an image with a pre-trained lightweight neural network model, where the lightweight neural network model generates a confidence value;

comparing the confidence value to a threshold value to determine if the pre-trained lightweight neural network model is sufficiently accurate;

analyzing the image with a pre-trained heavyweight neural network model for the confidence value below the threshold value, wherein the pre-trained heavyweight neural network model has above about one million trainable parameters and the pre-trained lightweight neural network model has a number of trainable parameters below one tenth the heavyweight model; and

displaying demographic data to a user on a user interface, wherein the user modifies store inventory based on the demographic data.

2. The method as recited in claim 1 , further comprising calculating the demographic data from a plurality of images.

3. The method as recited in claim 1 , wherein the pre-trained lightweight neural network model and the pre-trained heavyweight neural network model are each a convolutional neural network.

4. The method as recited in claim 3 , wherein the pre-trained lightweight neural network model is between 25% and 33% faster than the pre-trained heavyweight neural network model.

5. The method as recited in claim 1 , wherein the threshold value is set equal to the average of confidence values generated for the pre-trained heavyweight neural network model using a training set of images.

6. The method as recited in claim 1 , wherein the threshold value is 0.75.

7. A system for demographic determination using image recognition, comprising:

a memory, wherein a pre-trained lightweight neural network model and a pre-trained heavyweight neural network model are stored in the memory;

one or more processors configured to execute the pre-trained lightweight neural network model and the pre-trained heavyweight neural network model, wherein the pre-trained heavyweight neural network model has above about one million trainable parameters and the pre-trained lightweight neural network model has a number of trainable parameters below one tenth the heavyweight model, and compare a confidence value generated by the pre-trained lightweight neural network model to a threshold value, wherein the one or more processors are configured to execute the pre-trained heavyweight neural network model if the pre-trained lightweight neural network model is sufficiently accurate; and

a user interface configured to display demographic data to a user, wherein the user modifies store inventory based on the demographic data.

8. The system as recited in claim 7 , wherein the pre-trained lightweight neural network model has less than 1/10 th the trainable parameters than the pre-trained heavyweight neural network model.

9. The system as recited in claim 7 , wherein the pre-trained lightweight neural network model and the pre-trained heavyweight neural network model are each a convolutional neural network.

10. The system as recited in claim 7 , wherein the pre-trained lightweight neural network model is selected from the group consisting of Cifar10_quick, NECLA, MobileNet and Squeezenet.

11. The system as recited in claim 7 , wherein the pre-trained lightweight neural network model is selected from the group consisting of NECLA, AlexNet, ResNet, and VGGnet.

12. The system as recited in claim 7 , wherein the pre-trained lightweight neural network model is between 25% and 33% faster than the pre-trained heavyweight neural network model.

13. The system as recited in claim 7 , wherein the threshold value is 0.75.

14. The system as recited in claim 7 , wherein the threshold value is set equal to the average of confidence values generated for the pre-trained heavyweight neural network model using a training set of images.

15. A non-transitory computer readable storage medium comprising a computer readable program for demographic determination using image recognition, wherein the computer readable program when executed on a computer causes the computer to perform:

analyzing an image with a pre-trained lightweight neural network model, where the lightweight neural network model generates a confidence value;

comparing the confidence value to a threshold value to determine if the pre-trained lightweight neural network model is sufficiently accurate;

analyzing the image with a pre-trained heavyweight neural network model for the confidence value below the threshold value, wherein the pre-trained heavyweight neural network model has above about one million trainable parameters and the pre-trained lightweight neural network model has a number of trainable parameters below one tenth the heavyweight model; and

displaying demographic data to a user on a user interface, wherein the user modifies store inventory based on the demographic data.

16. The computer readable storage medium comprising a computer readable program, as recited in claim 15 , further comprising calculating the demographic data from a plurality of images.

17. The computer readable storage medium comprising a computer readable program, as recited in claim 15 , wherein the pre-trained lightweight neural network model and the pre-trained heavyweight neural network model are each a convolutional neural network.

18. The computer readable storage medium comprising a computer readable program, as recited in claim 17 , wherein the pre-trained lightweight neural network model is between 25% and 33% faster than the pre-trained heavyweight neural network model.

19. The computer readable storage medium comprising a computer readable program, as recited in claim 15 , wherein the threshold value is set equal to the average of confidence values generated for the pre-trained heavyweight neural network model using a training set of images.

20. The computer readable storage medium comprising a computer readable program, as recited in claim 15 , wherein the threshold value is 0.75.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2023
From: NEC LABORATORIES AMERICA, INC.
To: NEC CORPORATION
Reel/Frame 063213/0514 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2021
From: YANG, YI; FENG, MIN; CHAKRADHAR, SRIMAT
To: NEC LABORATORIES AMERICA, INC.
Reel/Frame 055667/0882 →
Continuity (2)
Provisional Application 62981054 · Feb 25, 2020
Related Publication 20210264138A1 · Aug 26, 2021