METHOD FOR CLASSIFYING PRODUCT ORGANIZATION IN AN INVENTORY STRUCTURE

1 . A method comprising:

by a robotic system:

autonomously navigating throughout a facility during a scan cycle; and

via an optical sensor integrated in the robotic system, capturing images of inventory structures in the facility during the scan cycle; and

by a computer system:

accessing a first image depicting an inventory structure in the facility at a first time during the scan cycle, the first image captured by the robotic system;

detecting a first product unit of a first product type in the first image;

locating a first slot boundary in the first image, the first slot boundary containing the first product unit;

detecting a first tag in the first image;

extracting a first product identifier, from the first tag, depicted in the first image; and

in response to the first product identifier corresponding to the first product type:

associating the first slot boundary with the first tag;

extracting a first relative tag-boundary position of the first slot boundary and the first tag from the first image, the first relative tag-boundary position defining a shelf tag location adjacent and below a first lower-left corner of the first slot boundary based on positions of the first slot boundary and the first tag in the first image; and

identifying the first slot boundary as in-stock at the first time.

2 . The method of claim 1 , further comprising, in response to the first product identifier corresponding to a second product type different from the first product type, flagging the first slot boundary as out-of-stock and occupied by an incorrect product type.

3 . The method of claim 1 , wherein locating the first slot boundary in the first image comprises locating the first slot boundary in the first image, the first slot boundary defining a first rectilinear slot boundary.

4 . The method of claim 1 , wherein extracting the first relative tag-boundary position of the first slot boundary and the first tag from the first image comprises extracting the first relative tag-boundary position defining a hang tag location centered above the first slot boundary based on positions of the first slot boundary and the first tag in the first image.

5 . The method of claim 1 , wherein detecting the first product unit of the first product type in the first image comprises:

detecting the first product unit in the first image;

extracting a first set of features, depicting the first product unit, from the first image; and

identifying the first product unit as a first product type based on the first set of features.

6 . The method of claim 5 :

wherein extracting the first set of features from the first image comprises extracting the first set of features comprising color values and text strings from a first region of the first image depicting the first product unit; and

wherein identifying the first product unit as the first product type comprises:

retrieving a set of visual feature templates representing a set of product types;

scanning the set of visual feature templates for features matching color values and text strings in the first set of features; and

identifying the first product unit as the first product type in response to the first set of features approximating a first visual feature template, in the set of visual feature templates, corresponding to the product type.

7 . The method of claim 1 , wherein accessing the first image comprises:

accessing a sequence of photographic images captured by the robotic system while traversing an aisle facing the inventory structure; and

compiling the sequence of photographic images into the first image defining a composite image spanning the inventory structure.

8 . The method of claim 1 , further comprising:

detecting a second tag in the first image;

extracting a second product identifier, from the second tag, depicted in the first image;

defining a second slot boundary assigned to a second product type corresponding to the second product identifier;

locating a second lower-left corner of the second slot boundary adjacent and above the second tag in the first image; and

in response to absence of a second product unit corresponding to the second product identifier within the second slot boundary in the first image, predicting the second slot boundary as out-of-stock at the first time.

9 . The method of claim 8 , further comprising:

storing a first location of the first tag in a realogram of the facility;

storing the first slot boundary, associated with the first tag, in the realogram;

storing a second location of the second tag in the realogram; and

temporarily storing the second slot boundary, associated with the second tag, in the realogram.

10 . The method of claim 9 , further comprising:

accessing a second image depicting the inventory structure at a second time succeeding the first time;

detecting a third product unit in the second image;

extracting a third set of features, depicting the third product unit, from the second image;

identifying the third product unit as the second product type based on the third set of features;

locating a revised second slot boundary, containing the third product unit, in the second image;

detecting the second tag in the second image;

extracting the second product identifier, from the second tag, depicted in the second image;

in response to the second product identifier corresponding to the second product type, associating the revised second slot boundary with the second tag; and

in response to the second slot boundary differing from the revised second slot boundary, replacing the second slot boundary with the revised second slot boundary in the realogram.

11 . The method of claim 1 , further comprising:

detecting a second tag in the first image;

extracting a second product identifier, from the second tag, depicted in the first image;

defining a second slot boundary assigned to a second product type corresponding to the second product identifier;

locating a second lower-left corner of the second slot boundary adjacent and above the second tag in the first image; and

in response to presence of a second product unit corresponding to the second product identifier within the second slot boundary in the first image, identifying the second slot boundary as in-stock at the first time.

12 . The method of claim 1 , further comprising:

detecting a second product unit of a second product type in the first image;

locating a second slot boundary in the first image, the second slot boundary containing the second product unit;

detecting a second tag in the first image;

extracting a second product identifier, from the second tag, depicted in the first image; and

in response to the second product identifier corresponding to the second product type:

associating the second slot boundary with the second tag;

extracting a second relative tag-boundary position of the second slot boundary and the second tag from the first image, the second relative tag-boundary position defining a hang tag location centered above the second slot boundary based on positions of the second slot boundary and the second tag in the first image; and

identifying the second slot boundary as in-stock at the first time.

13 . The method of claim 12 , further comprising:

detecting a third tag in the first image;

extracting a third product identifier, from the third tag, depicted in the first image;

defining a third slot boundary analogous to the second slot boundary;

locating a bottom edge of the third slot boundary adjacent and centered below the third tag in the first image;

scanning the first image for a third product unit corresponding to the third product identifier; and

in response to absence of the third product unit corresponding to the third product identifier within the third slot boundary in the first image, identifying the third slot boundary as out-of-stock at the first time.

14 . A method comprising, by a computer system:

accessing a first image depicting an inventory structure in a facility at a first time;

detecting a first product unit of a first product type in the first image;

locating a first slot boundary, containing the first product unit, in the first image;

detecting a first tag in the first image;

extracting a first product identifier, from the first tag, depicted in the first image;

in response to the first product identifier corresponding to the first product type:

associating the first slot boundary with the first tag; and

extracting a first relative tag-boundary position of the first slot boundary and the first tag from the first image;

detecting a second product unit of a second product type in the first image;

locating a second slot boundary. containing the second product unit, in the first image;

detecting a second tag in the first image;

extracting a second product identifier, from the second tag, depicted in the first image; and

in response to the second product identifier corresponding to the second product type, associating the second slot boundary with the second tag.

15 . The method of claim 14 , further comprising:

detecting a third tag in the first image;

extracting a third product identifier, from the third tag, depicted in the first image;

characterizing a first distance between the first tag and the second tag;

characterizing a second distance between the second tag and the third tag; and

in response to absence of the third product unit corresponding to the third product identifier within the third slot boundary in the first image and in response to the second distance exceeding the first distance, predicting a third slot boundary, assigned to a third product type corresponding to the third product identifier, in the first image based on:

a third position of the third tag in the first image; and

the first relative tag-boundary position.

16 . The method of claim 14 , wherein accessing the first image comprises accessing the first image comprising a photographic image captured by a fixed camera, arranged within the facility and facing the inventory structure, at the first time.

17 . The method of claim 14 :

further comprising deploying a robotic system to autonomously navigate throughout the facility during a scan cycle, the robotic system comprising an optical sensor; and

wherein accessing the first image comprises accessing the first image captured by the robotic system autonomously traversing the facility during a scan cycle.

18 . The method of claim 14 , further comprising:

in response to the first product identifier corresponding to the first product type, identifying the first slot boundary as in-stock at the first time; and

in response to absence of the second product unit corresponding to the second product identifier within the second slot boundary in the first image, identifying the second slot boundary as out-of-stock at the first time.

19 . A method comprising, by a computer system:

accessing a first image depicting an inventory structure in a facility at a first time, the first image captured by a robotic system traversing the facility during a scan cycle;

detecting a first product unit of a first product type in the first image;

locating a first slot boundary, containing the first product unit, in the first image;

detecting a first tag in the first image;

extracting a first product identifier, from the first tag, depicted in the first image;

in response to the first product identifier corresponding to the first product type:

associating the first slot boundary with the first tag;

extracting a first relative tag-boundary position of the first slot boundary and the first tag from the first image, the first relative tag-boundary position defining a hang tag location centered above the first slot boundary based on positions of the first slot boundary and the first tag in the first image; and

identifying the first slot boundary as in-stock at the first time;

detecting a second product unit of a second product type in the first image;

locating a second slot boundary, containing the second product unit, in the first image;

locating a second location of a second tag, corresponding to the second slot boundary adjacent and centered above the second slot boundary in the first image; and

in response to detecting absence of the second tag in the second location in the first image, flagging the second slot boundary as missing the second tag.

20 . The method of claim 19 , further comprising, by the robotic system:

autonomously navigating throughout the facility during the scan cycle; and

via an optical sensor integrated in the robotic system, capturing images of inventory structures in the facility during the scan cycle.