Camera-implemented article layout control method for shelves equipped with electronic shelf labels

1 . A computer-implemented method for checking a layout of articles in a gondola of a sales area, with respect to a realogram database which stores, for each article identified by an article identifier, the article identifier in association with a slot field corresponding to a location of an electronic shelf label arranged in the sales area, the method comprising:

obtaining an image of the gondola acquired by an imaging device;

recognizing in the obtained image an arrangement of electronic shelf labels in rows of ordered electronic shelf labels;

assigning to each electronic shelf label of the recognized arrangement a row number and a label number; and

for an electronic shelf label of the recognized arrangement:

determining a slot field corresponding to the location of the electronic shelf label based on the row number and the label number assigned to the electronic shelf label;

identifying a matching area in the obtained image corresponding to the electronic shelf label;

detecting a zone inside the identified matching area corresponding to a real view of an article in the obtained image;

retrieving, in the realogram database, an article identifier stored in association with the determined slot field;

retrieving, in an article image database which stores, for each article identified by an article identifier, a front view of the article in association with the article identifier, an expected front view of the article identified by the retrieved article identifier;

calculating a similarity rate between the real view of the article in the obtained image and the expected front view of the article identified by the retrieved article identifier;

comparing the calculated similarity rate with a first threshold;

based on the calculated similarity rate being below the first threshold, determining that the article in the obtained image does not match with the article identified by the retrieved article identifier; and

outputting a signal alert based on determining the article in the obtained image does not match with the article identified by the retrieved article identifier.

2 . The method of claim 1 , wherein the front views stored in the article image database are unique front views.

3 . The method of claim 1 , further comprising, for a second electronic shelf label of the recognized arrangement:

determining a second slot field corresponding to the location of the second electronic shelf label;

identifying a second matching area in the obtained image corresponding to the second electronic shelf label;

detecting a zone inside the second matching area corresponding to a real view of a second article in the obtained image;

retrieving, in the realogram database, a second article identifier stored in association with the second slot field;

retrieving, in the article image database, an expected front view of the second article identified by the second article identifier;

calculating a second similarity rate between the real view of the second article in the obtained image and the expected front view of the second article identified by the second article identifier;

comparing the second similarity rate with the first threshold and a second threshold greater than the first threshold; and

based on the second similarity rate being between the first threshold and the second threshold, determining that the real view of the second article in the obtained image does not match with the expected front view of the second article identified by the second article identifier.

4 . The method of claim 3 , further comprising, for the second electronic shelf label of the recognized arrangement, outputting a signal alert based on determining the second article in the obtained image matches with the second article identified by the second article identifier but the real view of the second article in the obtained image does not match with the expected front view of the second article identified by the second article identifier.

5 . The method of claim 1 , wherein calculating the similarity rate comprises:

resizing the expected front view of the article identified by the retrieved article identifier to match a dimension of the zone inside the identified matching area; and

comparing pixels of the zone with pixels of the resized expected front view of the article identified by the retrieved article identifier.

6 . The method of claim 1 , wherein the similarity rate is based on differences in at least one of luminosity or orientation between the real view of the article in the obtained image and the expected front view of the article identified by the retrieved article identifier.

7 . The method of claim 1 , further comprising:

identifying at least one empty area of the gondola in the obtained image by image recognition; and

for the electronic shelf label of the recognized arrangement, determining that the article identified with the retrieved article identifier is out-of-stock in the gondola based on the matching area in the obtained image corresponding to the recognized electronic shelf label matching with the at least one identified empty area.

8 . The method of claim 7 , wherein identifying at least one empty area of the gondola is based on color recognition with respect to a predetermined pattern on at least one of the top surface, the back, or the row of shelves of the gondola.

9 . The method of claim 1 , further comprising, for the electronic shelf label of the recognized arrangement:

retrieving, in the realogram database, expected facing information stored in association with the determined slot field; and

checking, by image recognition, compliance between the expected facing information and real facing information in the identified matching area in the obtained image corresponding to the electronic shelf label.

10 . The method of claim 9 , wherein the expected facing information is a number of consecutive lines of the same article for display in the gondola.

11 . The method of claim 1 , further comprising, for the electronic shelf label of the recognized arrangement, displaying, via a graphical interface, a representation of the gondola along with a visual signal highlighting the matching area corresponding to the recognized electronic shelf label based on determining the article in the obtained image does not match with the article identified by the retrieved article identifier.

12 . The method of claim 1 , wherein recognizing in the obtained image the arrangement of electronic shelf labels in rows of ordered electronic shelf labels is based on pattern recognition with respect to a predetermined set of possible electronic shelf label shapes.

13 . The method of claim 1 , wherein:

recognizing in the obtained image the arrangement of electronic shelf labels in rows of ordered electronic shelf labels comprises detecting rows of electronic shelf labels in the obtained image;

detecting a row of electronic shelf labels comprises detecting an alignment of electronic shelf labels; and

for the electronic shelf label of the recognized arrangement, the matching area in the obtained image corresponding to the location of the electronic shelf label is identified as situated between the electronic shelf label and one consecutive electronic shelf label along the detected row.

14 . A non-transitory computer-readable medium storing code instructions which, when executed by a processor, cause the processor to implement the method according to claim 1 .

15 . A server for checking a layout of articles in a gondola of a sales area with respect to a realogram database which stores, for each article identified by an article identifier, the article identifier in association with a slot field corresponding to a location of an electronic shelf label arranged in the sales area, the server being configured to perform steps:

obtaining an image of the gondola acquired by an imaging device;

recognizing in the obtained image an arrangement of electronic shelf labels in rows of ordered electronic shelf labels;

assigning to each electronic shelf label of the recognized arrangement a row number and a label number; and

for an electronic shelf label of the recognized arrangement:

determining the slot field corresponding to the location of the electronic shelf label based on the row number and the label number assigned to the electronic shelf label;

identifying a matching area in the obtained image corresponding to the electronic shelf label;

detecting a zone inside the identified matching area corresponding to a real view of an article in the obtained image;

retrieving, in the realogram database, an article identifier stored in association with the determined slot field;

retrieving, in an article image database which stores, for each article identified by an article identifier, a front view of the article in association with the article identifier, an expected front view of the article identified by the retrieved article identifier;

calculating a similarity rate between the real view of the article in the obtained image and the expected front view of the article identified by the retrieved article identifier;

comparing the calculated similarity rate with a first threshold;

based on the calculated similarity rate being below the first threshold, determining that the article in the obtained image does not match with the article identified by the retrieved article identifier; and

outputting a signal alert based on determining the article in the obtained image does not match with the article identified by the retrieved article identifier.

16 . The method of claim 1 , further comprising, for a second electronic shelf label of the recognized arrangement:

determining a second slot field corresponding to the location of the second electronic shelf label;

identifying a second matching area in the obtained image corresponding to the second electronic shelf label;

detecting a zone inside the second matching area corresponding to a real view of a second article in the obtained image;

retrieving, in the realogram database, a second article identifier stored in association with the second slot field;

retrieving, in the article image database, an expected front view of the second article identified by the second article identifier;

calculating a second similarity rate between the real view of the second article in the obtained image and the expected front view of the second article identified by the second article identifier;

comparing the second similarity rate with the first threshold; and

based on the second similarity rate being equal to or above the first threshold, determining that the real view of the second article in the obtained image matches with the expected front view of the second article identified by the second article identifier.

17 . The method of claim 1 , further comprising, for a second electronic shelf label of the recognized arrangement:

determining a second slot field corresponding to the location of the second electronic shelf label;

identifying a second matching area in the obtained image corresponding to the second electronic shelf label;

detecting a zone inside the second matching area corresponding to a real view of a second article in the obtained image;

retrieving, in the realogram database, a second article identifier stored in association with the second slot field;

retrieving, in the article image database, an expected front view of the second article identified by the second article identifier;

calculating a second similarity rate between the real view of the second article in the obtained image and the expected front view of the second article identified by the second article identifier;

comparing the second similarity rate with the first threshold and a second threshold greater than the first threshold; and

based on the second similarity rate being equal to or above the second threshold, determining that the real view of the second article in the obtained image matches with the expected front view of the second article identified by the second article identifier.