IP Library › Granted Patent US 12,705,570
Granted Patent B2
US 12,705,570 · App. 17/976,219 · Granted Aug 11, 2026

Devices and methods for computer vision guided planogram generation

Inventors: Eunhyang Kim (London, GB); Andrea Mirabile (London, GB)
Assignee: Zebra Technologies Corporation
G06Q10/087G06V20/50
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,705,570
App. No.
17/976,219
Granted
Aug 11, 2026
Kind
B2
Abstract

Devices and methods for planogram generation are disclosed herein. The method detects at least one first item and at least one label present in a captured image and associates the at least one first item with the at least one label based on a boundary between the at least one first item and at least one second item different from the at least one first item. The method identifies the at least one first item based on at least one attribute of the at least one first item and determines an area indicative of a position of the identified at least one first item based on the association. The area can be one or more of an aisle, a module, a shelf, a rack, a bay, and a bin. The method generates a planogram based on the association, the identified at least one first item and the area.

Claims (92)

1 . A method for planogram generation, comprising:

detecting at least one first item and at least one label present in a captured image;

associating the at least one first item with the at least one label based on a boundary between the at least one first item and at least one second item different from the at least one first item;

utilizing a deep neural network to extract at least one attribute of the at least one first item, and identifying the at least one first item based on the extracted at least one attribute of the at least one first item;

determining an area indicative of a position of the identified at least one first item based on the association; and

generating a planogram based on the association, the identified at least one first item and the area,

wherein the boundary is determined based on a probability indicative of a conditional weight, a distance between the at least one first item and the at least one second item, and at least one difference between the at least one first item and the at least one second item.

2 . The method of claim 1 , wherein associating the at least one first item with the at least one label based on the boundary between the at least one first item and the at least one second item different from the at least one first item comprises:

determining the distance between the at least one first item and the at least one second item;

setting the conditional weight based on the determined distance between the at least one first item and the at least one second item;

determining the at least one difference between the at least one first item and the at least one second item;

determining the boundary between the at least one first item and the at least one second item based on the determined distance, the set conditional weight, and the determined at least one difference; and

associating the at least one first item with the at least one label based on the determined boundary.

3 . The method of claim 1 , wherein utilizing the deep neural network to extract the at least one attribute of the at least one first item, and identifying the at least one first item based on the extracted at least one attribute of the at least one first item comprises:

processing the at least one first item by comparing the at least one first item and an item dataset based on the extracted at least one attribute;

retrieving at least one identifier of the at least one first item and a confidence level of the at least one identifier based on the processing; and

identifying the at least one first item based on the confidence level of the at least one identifier.

4 . The method of claim 3 , further comprising:

extracting the at least one attribute of each first item;

processing each first item by comparing each first item with the item dataset based on the extracted at least one attribute;

retrieving the at least one identifier of each first item and the confidence level of the at least one identifier based on the processing;

analyzing the retrieved confidence level of the at least one identifier of each first item by compiling the at least one identifier of each first item based on the confidence level of the at least one identifier; and

identifying the plurality of first items based on the compiled at least one identifier of each first item.

5 . The method of claim 4 , wherein the at least one identifier is one or more of a stock keeping unit (SKU) and a Universal Product Code (UPC).

6 . The method of claim 1 , further comprising

displaying, on a user interface, the generated planogram for a user; and

transmitting the planogram.

7 . The method of claim 1 , wherein

the area is one or more of an aisle, a module, a shelf, a rack, a bay, and a bin, and

the at least one attribute is one or more of a shape, color, pattern, logo, size, width, length, and height of the at least one item.

8 . A device for planogram generation, comprising,

an imaging assembly configured to capture an image featuring a plurality of items; one or more processors; and

a non-transitory computer-readable memory coupled to the imaging assembly and the one or more processors, the memory storing instructions thereon that, when executed by the one or more processors, cause the one or more processors to:

detect at least one first item and at least one label present in a captured image;

associate the at least one first item with the at least one label based on a boundary between the at least one first item and at least one second item different from the at least one first item;

utilize a deep neural network to extract at least one attribute of the at least one first item, and identify the at least one first item based on the extracted at least one attribute of the at least one first item;

determine an area indicative of a position of the identified at least one first item based on the association; and

generate a planogram based on the association, the identified at least one first item and the area,

wherein the boundary is determined based on a probability indicative of a conditional weight, a distance between the at least one first item and the at least one second item, and at least one difference between the at least one first item and the at least one second item.

9 . The device of claim 8 , wherein the instructions, when executed, cause the one or more processors to associate the at least one first item with the at least one label based on the boundary between the at least one first item and the at least one second item different from the at least one first item by:

determining the distance between the at least one first item and the at least one second item;

setting the conditional weight based on the determined distance between the at least one first item and the at least one second item;

determining the at least one difference between the at least one first item and the at least one second item;

determining the boundary between the at least one first item and the at least one second item based on the determined distance, the set conditional weight, and the determined at least one difference; and

associating the at least one first item with the at least one label based on the determined boundary.

10 . The device of claim 8 , wherein the instructions, when executed, cause the one or more processors to utilize the deep neural network to extract the at least one attribute of the at least one first item, and identify the at least one first item based on the extracted at least one attribute of the at least one first item by:

processing the at least one first item by comparing the at least one first item and an item dataset based on the extracted at least one attribute;

retrieving at least one identifier of the at least one first item and a confidence level of the at least one identifier based on the processing; and

identifying the at least one first item based on the confidence level of the at least one identifier.

11 . The device of claim 10 , wherein the instructions, when executed, further cause the one or more processors to:

extract the at least one attribute of each first item;

process each first item by comparing each first item with the item dataset based on the extracted at least one attribute;

retrieve the at least one identifier of each first item and the confidence level of the at least one identifier based on the processing;

analyze the retrieved confidence level of the at least one identifier of each first item by compiling the at least one identifier of each first item based on the confidence level of the at least one identifier; and

identify the plurality of first items based on the compiled at least one identifier of each first item.

12 . The device of claim 11 , wherein the at least one identifier is one or more of a stock keeping unit (SKU) and a Universal Product Code (UPC).

13 . The device of claim 8 , wherein the instructions, when executed, further cause the one or more processors to:

display, on a user interface, the generated planogram for a user; and

transmit the planogram.

14 . The device of claim 8 , wherein

the area is one or more of an aisle, a module, a shelf, a rack, a bay, and a bin, and

the at least one attribute is one or more of a shape, color, pattern, logo, size, width, length, and height of the at least one item.

15 . A non-transitory computer readable storage medium comprising instructions for planogram generation that, when executed, cause a machine to at least:

detect at least one first item and at least one label present in a captured image;

associate the at least one first item with the at least one label based on a boundary between the at least one first item and at least one second item different from the at least one first item;

utilize a deep neural network to extract at least one attribute of the at least one first item, and identify the at least one first item based on the extracted at least one attribute of the at least one first item;

determine an area indicative of a position of the identified at least one first item based on the association; and

generate a planogram based on the association, the identified at least one first item and the area,

wherein the boundary is determined based on a probability indicative of a conditional weight, a distance between the at least one first item and the at least one second item, and at least one difference between the at least one first item and the at least one second item.

16 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, cause the machine to associate the at least one first item with the at least one label based on the boundary between the at least one first item and the at least one second item different from the at least one first item by:

determining the distance between the at least one first item and the at least one second item;

setting the conditional weight based on the determined distance between the at least one first item and the at least one second item;

determining the at least one difference between the at least one first item and the at least one second item;

determining the boundary between the at least one first item and the at least one second item based on the determined distance, the set conditional weight, and the determined at least one difference; and

associating the at least one first item with the at least one label based on the determined boundary.

17 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, cause the machine to utilize the deep neural network to extract the at least one attribute of the at least one first item, and identify the at least one first item based on the extracted at least one attribute of the at least one first item by:

processing the at least one first item by comparing the at least one first item and an item dataset based on the extracted at least one attribute;

retrieving at least one identifier of the at least one first item and a confidence level of the at least one identifier based on the processing; and

identifying the at least one first item based on the confidence level of the at least one identifier.

18 . The non-transitory computer readable storage medium of claim 17 , wherein the instructions, when executed, further cause the machine to:

extract the at least one attribute of each first item;

process each first item by comparing each first item with the item dataset based on the extracted at least one attribute;

retrieve the at least one identifier of each first item and the confidence level of the at least one identifier based on the processing;

analyze the retrieved confidence level of the at least one identifier of each first item by compiling the at least one identifier of each first item based on the confidence level of the at least one identifier; and

identify the plurality of first items based on the compiled at least one identifier of each first item.

19 . The non-transitory computer readable storage medium of claim 18 , wherein the at least one identifier is one or more of a stock keeping unit (SKU) and a Universal Product Code (UPC).

20 . The non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, further cause the machine to:

display, on a user interface, the generated planogram for a user; and

transmit the generated planogram.

21 . The non-transitory computer readable storage medium of claim 15 , wherein

the area is one or more of an aisle, a module, a shelf, a rack, a bay, and a bin, and

the at least one attribute is one or more of a shape, color, pattern, logo, size, width, length, and height of the at least one item.