Systems and methods for constructing high resolution panoramic imagery for feature identification on robotic devices

1 . A robotic system, comprising:

a memory comprising computer readable instructions stored thereon; and

a processor configured to execute the computer readable instructions to:

receive, via a sensor coupled to the robotic system, a first image of an object and a second image of the object as the robotic system moves along a route, wherein the route comprises one or more scanning segments that extend parallel to surfaces of corresponding objects to be imaged;

determine, via a computer readable map, distance to the object within the first and second images and horizontal translation of the robotic system between the first and second image, wherein the horizontal translation of the robotic system on the computer readable map is determined based on odometry data of the robotic system, wherein the distance to the object is based on a pre-determined distance of the one or more scanning segments to the surfaces of the corresponding objects to be imaged;

determine a bounding box for each price label depicted in the first and second images;

determine an image quality matrix based on a level of contrast detected within the bounding boxes corresponding to the price labels within the first and second images, wherein the contrast is detected based on color transitions between adjacent pixels within the bounding boxes;

align the first and second images to form a panoramic image at least in part based on locations of the bounding boxes and the horizontal translation of the robotic system, such that price labels of the object are neither duplicated nor omitted in the panoramic image, and the panoramic image begins and ends proximate to edges of the object on the computer readable map; and

communicate the panoramic image to a server.

2 . The robotic system of claim 1 ,

wherein, the object comprises a plurality of labels including the price labels, and

wherein each of the plurality of labels correspond to a feature of the object, the plurality of labels comprise at least one of a text or computer readable code element.

3 . The robotic system of claim 2 , wherein the processor is further configured to execute the computer readable instructions to:

determine via optical character recognition (OCR) a bounding box for each label of the plurality of labels depicted in the first and second images; and

perform the alignment at least in part based on locations of the bounding boxes of the plurality of labels.

4 . The robotic system of claim 3 , wherein the processor is further configured to execute the computer readable instructions to,

determine an image quality matrix based on level of contrast detected within bounding boxes of the plurality of labels within a plurality of images; and

during panoramic image construction, bias color values of pixels in the panoramic image towards high-quality pixels denoted in the image quality matrix.

5 . The robotic system of claim 4 , wherein the processor is further configured to execute the computer readable instructions to,

adjust color values of pixels depicting the label within the bounding box of either the first image or the second image based on the color values of the label in the first and second images and the image quality matrix.

6 . The robotic system of claim 1 , wherein,

the computer readable map includes bin-level annotations for the object to be scanned,

wherein the annotations identify a surface of the object, a distance to the surface, and edges of the object defining a start and an end of the one or more scanning segments.

7 . A method for forming an image by a robotic system, comprising:

receiving, via a sensor coupled to the robotic system, a first image of an object and a second image of the object as the robotic system moves along a route, wherein the route comprises one or more scanning segments that extend parallel to surfaces of corresponding objects to be imaged;

determining, via a computer readable map, distance to the object within the first and second images and horizontal translation of the robotic system between the first and second image, wherein the horizontal translation of the robotic system on the computer readable map is determined based on odometry data of the robotic system, wherein the distance to the object is based on a pre-determined distance of the one or more scanning segments to the surfaces of the corresponding objects to be imaged;

determining a bounding box for each price label depicted in the first and second images;

determining an image quality matrix based on a level of contrast detected within the bounding boxes corresponding to the price labels within the first and second images, wherein the contrast is detected based on color transitions between adjacent pixels within the bounding boxes;

aligning the first and second images to form a panoramic image at least in part based on locations of the bounding boxes and the horizontal translation of the robotic system, such that price labels of the object are neither duplicated nor omitted in the panoramic image, and the panoramic image begins and ends proximate to edges of the object on the computer readable map; and

communicating the panoramic image to a server.

8 . The method of claim 7 ,

wherein, the object comprises a plurality of labels including the price labels, and

wherein each of the plurality of labels correspond to a feature of the object, the plurality of labels comprise at least one of a text or computer readable code element.

9 . The method of claim 8 , further comprising:

determining via optical character recognition (OCR) a bounding box for each label of the plurality of labels depicted in the first and second images; and

performing the alignment at least in part based on locations of the bounding boxes of the plurality of labels.

10 . The method of claim 9 , further comprising:

determining an image quality matrix based on level of contrast detected within bounding boxes of the plurality of labels within a plurality of images; and

during panoramic image construction, bias color values of pixels in the panoramic image towards high-quality pixels denoted in the image quality matrix.

11 . The method of claim 10 , further comprising:

adjusting color values of pixels depicting the label within the bounding box of either the first image or the second image based on the color values of the label in the first and second images and the image quality matrix.

12 . The method of claim 7 , wherein,

the computer readable map includes bin-level annotations for the object to be scanned, wherein the annotations identify a surface of the object, a distance to the surface, and edges of the object defining a start and an end of the one or more scanning segments.

13 . A non-transitory computer readable medium comprising computer readable instructions stored there that when executed by at least one processor configure the at least one processor to,

receive, via a sensor coupled to a robotic system, a first image of an object and a second image of the object as the robotic system moves along a route, wherein the route comprises one or more scanning segments that extend parallel to surfaces of corresponding objects to be imaged;

determine, via a computer readable map, a distance to the object within the first and second images and horizontal translation of the robotic system between the first and second image, wherein the horizontal translation of the robotic system on the computer readable map is determined based on odometry data of the robotic system, wherein the distance to the object is based on a pre-determined distance of the one or more scanning segments to the surfaces of the corresponding objects to be imaged;

determine a bounding box for each price label depicted in the first and second images;

determine an image quality matrix based on a level of contrast detected within the bounding boxes corresponding to the price labels within the first and second images, wherein the contrast is detected based on color transitions between adjacent pixels within the bounding boxes:

align the first and second images to form a panoramic image at least in part based on locations of the bounding boxes and the horizontal translation of the robotic system, such that price labels of the object are neither duplicated nor omitted in the panoramic image, and the panoramic image begins and ends proximate to edges of the object on the computer readable map; and

communicate the panoramic image to a server.

14 . The non-transitory computer readable medium of claim 13 ,

wherein the object comprises a plurality of labels including the price labels, and

wherein each of the plurality of labels correspond to a feature of the object, the plurality of labels comprise at least one of a text or computer readable code element.

15 . The non-transitory computer readable medium of claim 14 , wherein the at least one processor is further configured to execute the computer readable instructions to,

determine via optical character recognition (OCR) a bounding box for each label of the plurality of labels depicted in the first and second images; and

perform the alignment at least in part based on locations of the bounding boxes of the plurality of labels.

16 . The non-transitory computer readable medium of claim 15 , wherein the at least one processor is further configured to execute the computer readable instructions to,

determine an image quality matrix based on level of contrast detected within bounding boxes of the plurality of labels within a plurality of images; and

during panoramic image construction, bias color values of pixels in the panoramic image towards high-quality pixels denoted in the image quality matrix.

17 . The non-transitory computer readable medium of claim 16 , wherein the at least one processor is further configured to execute the computer readable instructions to,

adjust color values of pixels depicting the label within the bounding box of tither the first image or the second image based on the color values of the label in the first and second images and the image quality matrix.

18 . The non-transitory computer readable medium of claim 13 , wherein,

the computer readable map includes bin-level annotations for the object to be scanned,

wherein the annotations identify a surface of the object, a distance to the surface, and edges of the object defining a start and an end of the one or more scanning segments.

19 . The robotic system of claim 1 , wherein the odometry data represents physical movement of the robotic system along the route.

20 . The robotic system of claim 1 , wherein the first and second images are aligned to form the panoramic image such that features of the object are neither duplicated nor omitted in the panoramic image.