IP Library Granted Patent US 11,790,651
Granted Patent B2
US 11,790,651 · App. 17/455,894 · Granted Oct 17, 2023

System and method for capturing images for training of an item identification model

Inventors: Sumedh Vilas Datar (Grapevine, TX); Tejas Pradip Rode (Coppell, TX); Sailesh Bharathwaaj Krishnamurthy (Irving, TX); Crystal Maung (Dallas, TX)
Assignee: 7-ELEVEN, INC.
G06V20/41G06F18/2148G06T7/55G06T11/20G06T2210/12G06V20/44
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,790,651
App. No.
17/455,894
Granted
Oct 17, 2023
Kind
B2
Abstract

A system for capturing images for training an item identification model obtains an identifier of an item. The system detects a triggering event at a platform, where the triggering event corresponds to a user placing the item on a platform. The system causes the platform to rotate. The system causes at least one camera to capture an image of the item while the platform is rotating. The system extracts a set of features associated with the item from the image. The system associates the item to the identifier and the set of features. The system adds a new entry to a training dataset of the item identification model, where the new entry represents the item labeled with the identifier and the set of features.

Claims (105)

1. A system for capturing images for training an item identification model comprising:

a plurality of cameras, wherein each camera is configured to capture images of at least a portion of a platform;

the platform is configured to rotate;

a memory, operable to store an item identification model, wherein the item identification model is configured to identify items based at least in part upon images of the items; and

a processor, operably coupled with the memory, and configured to:

obtain an identifier associated with an item;

detect a triggering event at the platform, wherein the triggering event corresponds to a user placing the item on the platform;

cause the platform to rotate;

cause at least one camera from among the plurality of cameras to capture an image of the item while the platform is rotating;

extract a set of features associated with the item from the image, wherein each feature corresponds to a physical attribute of the item, and wherein the set of features comprises at least one of:

one or more dominant colors of the item, wherein each of the one or more dominant colors is determined based at least in part upon a set of pixel colors associated with the item from the image; and

a dimension of the item, wherein the dimension comprises a width, a length, and a height of the item;

associate the item to the identifier and the set of features; and

add a new entry to a training dataset of the item identification model, wherein the new entry represents the item labeled with at least one of the identifier and the set of features.

2. The system of claim 1 , wherein:

a first subset of the plurality of cameras is positioned above the platform,

the first subset of plurality of cameras that are positioned above the platform is arranged to form a triangle; and

the first subset of plurality of cameras is configured to capture overhead images of the item placed on the platform.

3. The system of claim 1 , wherein:

a second subset of plurality of cameras is positioned at one or more heights with respect to the platform;

the second subset of plurality of cameras is arranged vertically on a rail;

the rail is on a side of the platform adjacent to the platform; and

the second subset of plurality of cameras is configured to capture perspective images of the item placed on the platform.

4. The system of claim 1 , wherein:

the platform is rotated one degree at a time until the platform is fully rotated once; and

the at least one camera is triggered to capture one image of the item at each of a plurality of degrees of rotation of the platform.

5. The system of claim 1 , further comprising a three-dimensional (3D) sensor positioned above the platform, wherein the 3D sensor is configured to capture overhead depth images of the item placed on the platform, wherein each overhead depth image is configured to capture upward-facing surfaces of the item placed on the platform;

wherein the processor is further configured to:

cause the 3D sensor to capture a depth image of the item while the platform is turning;

determine an orientation of the item with respect to the platform;

determine that the orientation of the item is longitudinal with respect to the platform; and

in response to determining that the orientation of the item is longitudinal with respect to the platform, cause a first subset of cameras from among the plurality of cameras to take overhead images of the item, wherein the first subset of cameras are positioned above the platform.

6. The system of claim 1 , wherein the set of features further comprises at least one of:

a bounding box around the item; and

a mask that defines a contour around the item.

7. The system of claim 1 , further comprising a weight sensor configured to measure weights for items on the platform;

wherein the processor is further configured to:

receive a plurality of weights of multiple instances of the item;

determine a mean of the plurality of weights;

associate the mean of the plurality of weights to the item; and

add the mean of the plurality of weights to the new entry.

8. A method for capturing images for training an item identification model comprising:

obtaining an identifier associated with an item;

detecting a triggering event at a platform, wherein the triggering event corresponds to a user placing the item on the platform;

causing the platform to rotate;

causing at least one camera from among a plurality of cameras to capture an image of the item while the platform is rotating;

extracting a set of features associated with the item from the image, wherein each feature corresponds to a physical attribute of the item, and wherein the set of features comprises at least one of:

one or more dominant colors of the item, wherein each of the one or more dominant colors is determined based at least in part upon a set of pixel colors associated with the item from the image; and

a dimension of the item, wherein the dimension comprises a width, a length, and a height of the item;

associating the item to the identifier and the set of features; and

adding a new entry to a training dataset of the item identification model, wherein the new entry represents the item labeled with at least one of the identifier and the set of features.

9. The method of claim 8 , wherein:

a first subset of the plurality of cameras is positioned above the platform,

the first subset of plurality of cameras that are positioned above the platform is arranged to form a triangle; and

the first subset of plurality of cameras is configured to capture overhead images of the item placed on the platform.

10. The method of claim 8 , wherein:

a second subset of plurality of cameras is positioned at one or more heights with respect to the platform;

the second subset of plurality of cameras is arranged vertically on a rail;

the rail is on a side of the platform adjacent to the platform; and

the second subset of plurality of cameras is configured to capture perspective images of the item placed on the platform.

11. The method of claim 8 , wherein:

the platform is rotated one degree at a time until the platform is fully rotated once; and

the at least one camera is triggered to capture one image of the item at each of a plurality of degrees of rotation of the platform.

12. The method of claim 8 , further comprising:

causing a 3D sensor to capture a depth image of the item while the platform is turning;

determining an orientation of the item with respect to the platform;

determining that the orientation of the item is longitudinal with respect to the platform; and

in response to determining that the orientation of the item is longitudinal with respect to the platform, causing a first subset of cameras from among the plurality of cameras to take overhead images of the item, wherein the first subset of cameras are positioned above the platform.

13. The method of claim 8 , wherein the set of features further comprises at least one of:

a bounding box around the item; and

a mask that defines a contour around the item.

14. The method of claim 8 , further comprising:

receiving a plurality of weights of multiple instances of the item;

determining a mean of the plurality of weights;

associating the mean of the plurality of weights to the item; and

adding the mean of the plurality of weights to the new entry.

15. A non-transitory computer-readable medium storing instructions that when executed by a processor cause the processor to:

obtain an identifier associated with an item;

detect a triggering event at a platform, wherein the triggering event corresponds to a user placing the item on the platform;

cause the platform to rotate;

cause at least one camera from among a plurality of cameras to capture an image of the item while the platform is rotating;

extract a set of features associated with the item from the image, wherein each feature corresponds to a physical attribute of the item, and wherein the set of features comprises at least one of:

one or more dominant colors of the item, wherein each of the one or more dominant colors is determined based at least in part upon a set of pixel colors associated with the item from the image; and

a dimension of the item, wherein the dimension comprises a width, a length, and a height of the item;

associate the item to the identifier and the set of features; and

add a new entry to a training dataset of the item identification model, wherein the new entry represents the item labeled with at least one of the identifier and the set of features.

16. The non-transitory computer-readable medium of claim 15 , wherein:

a first subset of the plurality of cameras is positioned above the platform, the first subset of plurality of cameras that are positioned above the platform is arranged to form a triangle; and

the first subset of plurality of cameras is configured to capture overhead images of the item placed on the platform.

17. The non-transitory computer-readable medium of claim 15 , wherein:

a second subset of plurality of cameras is positioned at one or more heights with respect to the platform;

the second subset of plurality of cameras is arranged vertically on a rail;

the rail is on a side of the platform adjacent to the platform; and

the second subset of plurality of cameras is configured to capture perspective images of the item placed on the platform.

18. The non-transitory computer-readable medium of claim 15 , wherein:

the platform is rotated one degree at a time until the platform is fully rotated once; and

the at least one camera is triggered to capture one image of the item at each of a plurality of degrees of rotation of the platform.

19. The non-transitory computer-readable medium of claim 15 , wherein the instructions when executed by the processor, further cause the processor to:

cause a 3D sensor to capture a depth image of the item while the platform is turning;

determine an orientation of the item with respect to the platform;

determine that the orientation of the item is longitudinal with respect to the platform; and

in response to determining that the orientation of the item is longitudinal with respect to the platform, cause a first subset of cameras from among the plurality of cameras to take overhead images of the item, wherein the first subset of cameras are positioned above the platform.

20. The non-transitory computer-readable medium of claim 15 , wherein the set of features further comprises at least one of:

a bounding box around the item; and

a mask that defines a contour around the item.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2021
From: DATAR, SUMEDH VILAS; RODE, TEJAS PRADIP; KRISHNAMURTHY, SAILESH BHARATHWAAJ; MAUNG, CRYSTAL
To: 7-ELEVEN, INC.
Reel/Frame 058170/0877 →
Continuity (2)
Continuation In Part 17362261 · Jun 29, 2021
Related Publication 20220414378A1 · Dec 29, 2022
Cited By (1)
US 12,340,537