Method, system and apparatus for detecting product facings

1. A method by an imaging controller of detecting product facings from captured depth and image data, the method comprising:

obtaining at the imaging controller: (i) depth measurements from at least one depth sensor, the depth measurements representing a support structure supporting a plurality of product facings, (ii) image data from at least one image sensor, the image data representing the support structure, and (iii) a set of region of interest (ROI) indicators, each ROI indicator indicating a position of a respective subset of the plurality of product facings on the support structure;

generating, by a depth detector of the imaging controller, from the depth measurements of the at least one depth sensor, a first set of candidate facing edges;

generating, by an image detector of the imaging controller, from the image data of the at least one image sensor, a second set of candidate facing edges from the image data;

generating by a boundary generator of the imaging controller:

(i) a third set of candidate facing edges by combining the first and second sets, and

(ii) a candidate facing boundary for each adjacent pair of candidate facing edges in the third set of candidate facing edges;

verifying, by the imaging controller, that the candidate facing boundaries correspond to positions of products on the support structure by selecting, by the boundary generator of the imaging controller, a subset of output facing boundaries from the candidate facing boundaries that are disposed within the ROI indicators;

detecting, by the boundary generator of the imaging controller, product facings including the selected subset of output facing boundaries.

2. The method of claim 1 , wherein generating the first set of candidate facing edges includes:

generating a two-dimensional depth map from the depth measurements; and

detecting edges in the depth map.

3. The method of claim 2 , wherein detecting edges in the depth map includes:

applying an edge detection operation to the depth map to generate an edge-weighted depth map; and

applying a line detection operation to the edge-weighted depth map.

4. The method of claim 3 , further comprising discarding candidate edges of the first set that are not within an ROI indicator.

5. The method of claim 1 , wherein generating the second set of candidate facing edges includes:

selecting a plurality of windows from the image data; and

classifying each window as one of containing an edge and not containing an edge.

6. The method of claim 5 , further comprising:

determining a position of each candidate facing edge of the second set based on an intensity profile of a corresponding one of the windows.

7. The method of claim 1 , wherein combining the first and second sets of candidate facing edges includes:

for each pair of adjacent candidate facing edges, determining whether a distance separating the pair is below a threshold;

when the distance separating the pair is below the threshold, replacing the pair with a single candidate facing edge.

8. The method of claim 1 , wherein generating the candidate facing boundaries includes:

joining upper ends of the adjacent pair of the third set of candidate facing edges with an upper boundary segment; and

joining lower ends of the adjacent pair of the third set of candidate facing edges with a lower boundary segment.

9. The method of claim 1 , wherein selecting the subset of output facing boundaries includes discarding candidate facing boundaries outside the ROI indicators.

10. A computing device, comprising:

a memory storing: (i) depth measurements from at least one depth sensor, the depth measurements representing a support structure supporting a plurality of product facings, (ii) image data from at least one image sensor, the image data representing the support structure, and (iii) a set of region of interest (ROI) indicators, each ROI indicator indicating a position of a respective subset of the plurality of product facings on the support structure;

an imaging controller having:

a depth detector configured to:

obtain the depth measurements and the ROI indicators; and

generate, from the depth measurements of the at least one depth sensor, a first set of candidate facing edges;

an image detector configured to:

obtain the image data and the ROI indicators; and

generate, from the image data of the at least one image sensor, a second set of candidate facing edges; and

a boundary generator configured to generate:

a third set of candidate facing edges by combining the first and second sets;

a candidate facing boundary for each adjacent pair of candidate facing edges in the third set of candidate facing edges;

verify that the candidate facing boundaries correspond to positions of products on the support structure by selecting a subset of output facing boundaries from the candidate facing boundaries that are disposed within the ROI indicators; and

detect product facings including the selected subset of output facing boundaries.

11. The computing device of claim 10 , wherein the depth detector is configured, in order to generate the first set of candidate facing edges, to:

generate a two-dimensional depth map from the depth measurements; and

detect edges in the depth map.

12. The computing device of claim 11 , wherein the depth detector is configured, in order to detect edges in the depth map, to:

apply an edge detection operation to the depth map to generate an edge-weighted depth map; and

apply a line detection operation to the edge-weighted depth map.

13. The computing device of claim 12 , wherein the depth detector is further configured to discard candidate edges of the first set that are not within an ROI indicator.

14. The computing device of claim 10 , wherein the image detector is configured, in order to generate the second set of candidate facing edges, to:

select a plurality of windows from the image data; and

classify each window as one of containing an edge and not containing an edge.

15. The computing device of claim 14 , wherein the image detector is further configured to determine a position of each candidate facing edge of the second set based on an intensity profile of a corresponding one of the windows.

16. The computing device of claim 10 , wherein the boundary generator is configured, in order to combine the first and second sets of candidate facing edges, to:

for each pair of adjacent candidate facing edges, determine whether a distance separating the pair is below a threshold;

when the distance separating the pair is below the threshold, replace the pair with a single candidate facing edge.

17. The computing device of claim 10 , wherein the boundary generator is configured, in order to generate the candidate facing boundaries, to:

join upper ends of the adjacent pair of the third set of candidate facing edges with an upper boundary segment; and

join lower ends of the adjacent pair of the third set of candidate facing edges with a lower boundary segment.

18. The computing device of claim 10 , wherein the boundary generator is configured, in order to select the subset of output facing boundaries, to discard candidate facing boundaries outside the ROI indicators.

19. A non-transitory computer-readable medium storing instructions executable by an imaging controller to configure the imaging controller to:

obtain (i) depth measurements from at least one depth sensor, the depth measurements representing a support structure supporting a plurality of product facings, (ii) image data from at least one image sensor, the image data representing the support structure, and (iii) a set of region of interest (ROI) indicators, each ROI indicator indicating a position of a respective subset of the plurality of product facings on the support structure;

generate, at a depth detector of the imaging controller, from the depth measurements of the at least one depth sensor, a first set of candidate facing edges;

generate, at an image detector of the imaging controller, from the image data of the at least one image sensor, a second set of candidate facing edges;

generate, at a boundary generator of the imaging controller, a third set of candidate facing edges by combining the first and second sets;

generate, at the boundary generator of the imaging controller, for each adjacent pair of candidate facing edges in the third set of candidate facing edges, a candidate facing boundary;

verify that the candidate facing boundaries correspond to positions of products on the support structure by selecting, at the boundary generator of the imaging controller, a subset of output facing boundaries from the candidate facing boundaries that are disposed within the ROI indicators; and

detect, by the boundary generator of the imaging controller, product facings including the selected subset of output facing boundaries.