IP Library › Granted Patent US 12,254,688
Granted Patent B2
US 12,254,688 · App. 18/376,204 · Granted Mar 18, 2025

Image recognition system

Inventors: Jason Lee Ertle (Portland, OR); Kevin L. Hofstee (Portland, OR); Shane K. Luke (Portland, OR)
Assignee: NIKE, Inc.
G06V20/10G06F18/2413G06T7/74G06V10/28G06V10/44G06V10/764G06F18/24323G06T2200/24G06V10/141G06V10/143G06V2201/09
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,254,688
App. No.
18/376,204
Granted
Mar 18, 2025
Kind
B2
Abstract

Systems and methods for predicting items within content and using improved, fine-grained image classification techniques to produce images used to identify consumer products in the real-world by allowing for the recognition of a product using an image captured under a variety of conditions and environments, such as angles, lighting, camera settings, and the like.

Claims (62)

1. A method, comprising:

modifying, by a computing device, at least one image frame, associated with image data corresponding to an object, to extract background information;

based on the at least one modified image frame, determining object identification information; and

using the object identification information to identify the object.

2. The method of claim 1 , further comprising:

receiving, by the computing device, input data indicating product information; and

associating the product information with the image data corresponding to the object.

3. The method of claim 1 , further comprising:

sending, to a second computing device, the image data corresponding to the object to verify an identification of the object.

4. The method of claim 3 , wherein sending the image data further comprises:

determining that a processing threshold for the computing device has been exceeded; and

sending, to the second computing device, the image data corresponding to the object.

5. The method of claim 1 , wherein using the object identification information to identify the object further comprises:

determining a product corresponding to the object; and

determining a confidence level associated with the product and the object.

6. The method of claim 1 , further comprising:

determining one or more products corresponding to the object; and

determining inventory data associated with the one or more products corresponding to the object.

7. The method of claim 1 , wherein modifying the at least one image frame to extract background information comprises:

using at least a first image frame to determine a pixel boundary of the object.

8. The method of claim 1 , further comprising:

extracting the at least one image frame from image data corresponding to the object.

9. The method of claim 1 , wherein modifying the at least one image frame to extract background information comprises:

cropping a portion of the at least one image frame associated with image data corresponding to the object.

10. An apparatus comprising:

one or more processors; and

memory storing instructions that, when executed, cause the apparatus to:

modify at least one image frame, associated with image data corresponding to an object, to extract background information;

based on the at least one modified image frame, determine object identification information; and

use the object identification information to identify the object.

11. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to:

receive input data indicating product information; and

associate the product information with the image data corresponding to the object.

12. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to:

determine a product corresponding to the object; and

determine a confidence level associated with the product and the object.

13. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to:

determine one or more products corresponding to the object; and

determine inventory data associated with the one or more products corresponding to the object.

14. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to modify the at least one image frame to extract background information by causing the apparatus to:

use at least a first image frame to determine a pixel boundary of the object.

15. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to:

extract the at least one image frame from image data corresponding to the object.

16. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to modify the at least one image frame to extract background information by causing:

cropping a portion of the at least one image frame associated with image data corresponding to the object.

17. A non-transitory machine readable medium storing instructions that, when executed, cause a computing device to:

modify at least one image frame, associated with image data corresponding to an object, to extract background information;

based on the at least one modified image frame, determine object identification information; and

use the object identification information to identify the object.

18. The non-transitory machine readable medium of claim 17 , wherein the instructions, when executed, further cause the computing device to send the image data by:

determining that a processing threshold for the computing device has been exceeded; and

sending, to a second computing device, the image data corresponding to the object.

19. The non-transitory machine readable medium of claim 17 , wherein the instructions, when executed, further cause the computing device to:

determine a product corresponding to the object; and

determine a confidence level associated with the product and the object.

20. The non-transitory machine readable medium of claim 17 , wherein the instructions, when executed, further cause the computing device to:

determine one or more products corresponding to the object; and

determine inventory data associated with the one or more products corresponding to the object.

21. The non-transitory machine readable medium of claim 17 , wherein the instructions, when executed, further cause the computing device to:

extract the at least one image frame from image data corresponding to the object.

22. The non-transitory machine readable medium of claim 17 , wherein the instructions, when executed, further cause the computing device to modify the at least one image frame to extract background information by causing:

cropping a portion of the at least one image frame associated with image data corresponding to the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2023
From: ERTLE, JASON LEE; HOFSTEE, KEVIN L.; LUKE, SHANE K.
To: NIKE, INC.
Reel/Frame 065129/0941 →
Continuity (4)
Continuation 17401814 · Aug 13, 2021
Continuation 16169774 · Oct 24, 2018
Provisional Application 62576250 · Oct 24, 2017
Related Publication 20240029425A1 · Jan 25, 2024
References Cited (18)
US 9037600B1 · Garrigues et al. · 2015 [cited by applicant]
US 9224070B1 · Sundareswara et al. · 2015 [cited by applicant]
US 9424461B1 · Yuan et al. · 2016 [cited by applicant]
US 9652838B1 · Manmatha et al. · 2017 [cited by applicant]
US 10176550B1 · Baggerman · 2019 [cited by applicant]
US 20100260426A1 · Huang · 2010 [cited by examiner]
US 20100313117A1 · Fukasawa · 2010 [cited by applicant]
US 20150169994A1 · Chinen et al. · 2015 [cited by applicant]
US 20150189188A1 · Penov et al. · 2015 [cited by applicant]
US 20160307028A1 · Fedorov · 2016 [cited by applicant]
US 20160335745A1 · Tajima · 2016 [cited by examiner]
US 20170061197A1 · Brain · 2017 [cited by applicant]
US 20170256040A1 · Grauer · 2017 [cited by applicant]
EP 3218076A1 · 2017 [cited by applicant]
JP 2008205774A · 2008 [cited by applicant]
WO 2016123538A1 · 2016 [cited by applicant]
Apr. 4, 2019—(WO) ISR & WO—App. No. PCT/US18/057370. [cited by applicant]
Jan. 19, 2023—(CN) Office Action—App. No. 201880081419.0. [cited by applicant]