Method and device for panoramic image processing
The present disclosure provides a method for providing an indication about a distance between a retail unit and an electronic device, comprising: acquiring from an image sensor in a retail establishment, at least one image of a product on a store shelf; accessing in a memory that stores a plurality of templates associated with a plurality of differing products, a template associated with the product on the store shelf in the at least one acquired image; comparing at least one dimension associated with the product in the at least one acquired image with at least one dimension associated with the accessed template; based on the comparison, determining whether a distance between the image sensor and the product on the store shelf is within an acceptable range for additional image processing; and providing a notification to a user of the electronic device to adjust the distance between the image sensor and the product on the shelf when the product is outside of the acceptable range.
1. A non-transitory computer readable medium including instructions that when executed by a processor cause the processor to perform a method for scanning a retail unit including a plurality of store shelves using an electronic device, the method comprising:
acquiring from an image sensor in a retail store environment, a plurality of images of at least one product on a store shelf of the retail unit by moving the electronic device along a scanning direction parallel to a horizontal plane associated with the store shelf;
accessing a memory that stores a plurality of templates associated with a plurality of differing products, wherein each of the plurality of templates is a visual representation of a product;
identifying in the memory a template with a visual representation of a product that matches the at least one product on the store shelf in the plurality of acquired images;
comparing dimensions of the at least one product in the plurality of acquired images with dimensions of the visual representation of the product in the identified template;
based on the dimensions comparison between the at least one product in the plurality of acquired images and the visual representation of the product in the identified template, determining whether a distance between the image sensor and the at least one product on the store shelf is within an acceptable range for additional image processing;
displaying on a display screen configured to display in real time the plurality of images being acquired a notification to adjust the distance between the image sensor and the at least one product on the shelf when the at least one product is outside of the acceptable range;
determining a vertical drift of the electronic device during the acquiring of the plurality of images, wherein the vertical drift is a change in an actual scanning direction relative to a predetermined scanning direction and the change is in a direction perpendicular to the predetermined scanning direction; and
displaying on the display screen a visual indicator indicative of the determined vertical drift.
2. The non-transitory computer readable medium of claim 1 , further comprising generating a panoramic image from the plurality of acquired images, and identifying the at least one product within the panoramic image.
3. The non-transitory computer readable medium of claim 1 , wherein the notification includes a distance indicator indicative of a distance from the image sensor to a front of the store shelf.
4. The non-transitory computer readable medium of claim 3 , wherein the distance indicator is superimposed on and displayed together with an image of the store shelf.
5. The non-transitory computer readable medium of claim 1 , wherein the notification includes an image representation of the template displayed together with an image of the store shelf.
6. The non-transitory computer readable medium of claim 5 , wherein the image representation is a silhouette of the product.
7. The non-transitory computer readable medium of claim 1 , wherein the notification is provided when the at least one product occupies more than about 40% of a field of view of the image sensor or less than about 20% of the field of view of the image sensor.
8. The non-transitory computer readable medium of claim 1 , wherein the notification is provided when the at least one product occupies more than about 35% of a field of view of the image sensor or less than about 25% of the field of view of the image sensor.
9. The non-transitory computer readable medium of claim 1 , wherein the method further comprises:
determining changes in the distance between the image sensor and the at least one product on the store shelf based on data from a position sensor included in the electronic device, wherein the position sensor enables calculation of a rotational change of the electronic device from image to image.
10. The non-transitory computer readable medium of claim 1 , wherein the method further includes:
maintaining a plurality of product-specific templates associated with at least one of products and product categories;
maintaining at least one generic template;
during accessing, when a product-specific template is not identified in memory for the at least one product in the plurality of images, using the at least one generic template to determine whether the distance is within an acceptable range.
11. The non-transitory computer readable medium of claim 1 , wherein providing the notification and acquiring the plurality of images take place substantially simultaneously.
12. The non-transitory computer readable medium of claim 1 , wherein determining the vertical drift of the electronic device includes obtaining data from a position sensor of the electronic device.
13. The non-transitory computer readable medium of claim 1 , wherein determining the vertical drift of the electronic device includes using image processing on the plurality of acquired images.
14. The non-transitory computer readable medium of claim 1 , further comprising displaying on the display screen horizontal lines parallel to shelves in the retail store environment.
15. The non-transitory computer readable medium of claim 1 , further comprising displaying on the display screen the visual indicator superimposed on an image of the store shelf.
16. A handheld electronic device, comprising:
an image sensor for acquiring, in a retail store environment establishment, a plurality of images of at least one product on a store shelf by moving the electronic device along a scanning direction parallel to a horizontal plane associated with the store shelf;
a display screen configured to display in real time the plurality of images being acquired;
a processor configured to:
access a memory that stores a plurality of templates associated with a plurality of differing products, wherein each of the plurality of templates is a visual representation of a product;
identify in the memory a template that matches the at least one product on the store shelf in the plurality of acquired images;
compare dimensions of the at least one product in the plurality of acquired images with dimensions of the visual representation of the product in the identified template;
based on the dimensions comparison between the at least one product in the plurality of acquired images and the visual representation of the product in the identified template, determine whether a distance between the image sensor and the at least one product on the store shelf is within an acceptable range for additional image processing;
display on the display screen a notification to adjust the distance between the image sensor and the at least one product on the shelf when the at least one product is outside of the acceptable range;
determine a vertical drift of the electronic device during acquiring of the plurality of images, wherein the vertical drift is a change in an actual scanning direction relative to a predetermined scanning direction and the change is in a direction perpendicular to the predetermined scanning direction; and
display on the display screen a visual indicator indicative of the determined vertical drift.
17. The handheld electronic device of claim 16 , wherein the processor is further configured to generate a panoramic image from the plurality of acquired images, and to identify the at least one product within the panoramic image.
18. The handheld electronic device of claim 16 , wherein the notification includes a distance indicator indicative of a distance from the image sensor to a front of the store shelf.
19. The handheld electronic device of claim 18 , wherein the distance indicator is superimposed on and displayed together with an image of the store shelf.
20. The handheld electronic device of claim 16 , wherein the notification includes an image representation of the template displayed together with an image of the store shelf.