Comparing virtual and real images in a shopping experience
In one embodiment, a computer-based method includes receiving image data of a real world layout. The image data reflects the real world layout across three dimensions (e.g., vertical, horizontal, and orthogonal). Each of the three dimensions has an image range with a beginning and an end. The real world layout has inventory of products distributed across the three dimensions. The computer-based method further includes comparing a 3D model of a planned layout of the environment to the received image data. The 3D model represents the planned layout across the three dimensions. The computer-based method further includes determining whether results of the comparison reach a first particular threshold, and if so, then output a pertinent indication, alert, or the like.
1. A computer-based method comprising:
receiving image data of a real world layout at a computing device, the image data reflecting the real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions;
receiving audio data of the real world layout, the audio data reflecting at least one of sounds, voice, or music from the real world layout;
comparing a 3D model of a planned layout of the real world layout to the received image data, the 3D model being from a tangible memory operatively coupled to the computing device and representing the planned layout across the three dimensions, wherein the comparing is able to convert the 3D model to a 2D representation and able to convert the received image data to a rendered 3D model;
comparing a sound model of the planned layout of the environment to the received audio data, the sound model representing the planned layout; and
determining, by the computing device, whether results of the comparison reach a first particular threshold and responsively indicating when the first particular threshold is reached.
2. The method of claim 1 , further comprising:
extracting, from the image data, at least one product display grouping;
associating the extracted at least one product display grouping with at least one representation of the product display grouping in the 3D model of the planned layout;
comparing the at least one product display groupings to the at least one representation of the product display grouping; and
determining whether the results meet a second particular threshold.
3. The method of claim 1 , further comprising:
generating an alert if the results of the comparison of the 3D model to the received image data reaches the first particular threshold, wherein the comparison represents similarity or non-similarity and the first particular threshold represents a particular degree of similarity.
4. The method of claim 1 , further comprising generating the 3D model of the planned layout.
5. The method of claim 1 , wherein comparing the 3D model to the received image data includes calculating an image difference.
6. The method of claim 1 , wherein comparing the 3D model to the received image data comprises comparing the 2D representation of the 3D model to the received image data.
7. The method of claim 1 , wherein comparing the 3D model of the planned layout to the received image data comprises comparing the 3D model of the planned layout to the rendered 3D model.
8. The method of claim 1 , wherein comparing a 3D model of a planned layout to the image data comprises:
analyzing the image data to determine a first quantity in an inventory, the first quantity in the inventory represented in the real world layout;
retrieving a second quantity in the inventory, the second quantity in the inventory represented in the 3D model of the planned layout; and
comparing the first quantity and the second quantity.
9. The method of claim 8 , wherein receiving image data comprises capturing image data of products in a customer display location and in a storage location.
10. The method of claim 8 , wherein retrieving a second quantity in the inventory comprises retrieving the second quantity from a transaction accounting system.
11. The method of claim 8 , wherein comparing the first quantity and the second quantity comprises calculating the difference of the first quantity and the second quantity, wherein the difference indicates a difference between a quantity of inventory calculated to be in a store and a quantity of inventory actually in the store.
12. The method of claim 1 , wherein receiving the image data of a real world layout comprises capturing the image data of the real world layout with a plurality of cameras.
13. The method of claim 12 , wherein receiving the image data of the real world layout comprises receiving the image data from the plurality of cameras positioned to be able to generate the 3D model of the real world layout.
14. The method of claim 1 , further comprising:
adjusting the received image data to match properties of the 3D model, wherein the properties include one or more of lighting characteristics, contrast, hue, shadow placement, and shadow removal.
15. The method of claim 1 , further comprising:
generating an instruction to conform the real world layout to the planned layout of the environment.
16. A system comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices and program instructions which are stored on the one or more storage devices for execution by the one or more processors via the one or more memories, the program instructions, when loaded and executed by at least one of the one or more processors, cause the system to implement:
a reception module configured to receive image data of a real world layout, the image data reflecting the real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions;
the reception module further configured to receive audio data of the real world layout, the audio data reflecting at least one of sounds, voice, or music from the real world layout;
a comparison module configured to compare a 3D model of a planned layout of the real world layout to the received image data, the 3D model representing the planned layout across the three dimensions;
the comparison module further configured to compare a sound model of the planned layout of the environment to the received audio data, the sound model representing the planned layout;
a conversion module able to convert the 3D model to a 2D representation and able to convert the received image data to a rendered 3D model; and
a threshold determination module configured to determine whether results of the comparison reach a first particular threshold, and to responsively indicate when the first particular threshold is reached.
17. The system of claim 16 , further comprising:
program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to implement:
a grouping extraction module configured to extract, from the image data, at least one product display grouping; and
an association module configured to associate the extracted at least one product display grouping with at least one representation of the product display grouping in the 3D model of the planned layout, wherein the comparison module is further configured to compare the at least one product display groupings to at least one representation of the product display grouping, and the threshold determination module is further configured to determine whether the results meet a second particular threshold.
18. The system of claim 16 , further comprising program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to implement an alert generation module configured to generate an alert if the comparison of the 3D model to the received image data reaches the first particular threshold, wherein the comparison represents similarity or non-similarity and the first particular threshold represents a particular degree of similarity.
19. The system of claim 16 , further comprising program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to implement a model generation module configured to generate the 3D model of a planned layout.
20. The system of claim 16 , wherein the comparison module is further configured to compare the 3D model to the received image data by calculating an image difference.
21. The system of claim 16 wherein the comparison module is further configured to compare the 2D representation of the 3D model to the received image data.
22. The system of claim 16 wherein the comparison module is further configured to compare the 3D model of the planned layout to the rendered 3D model.
23. The system of claim 16 , wherein the comparison module comprises:
an analysis module configured to analyze the image data to determine a first quantity in an inventory; and
an inventory reception module configured to retrieve a second quantity in an inventory, and wherein the comparison module is further configured to compare the first quantity and the second quantity.
24. The system of claim 23 , wherein the inventory reception module is further configured to receive image data of products in a customer display location and in a storage location.
25. The system of claim 23 , wherein the inventory reception module is further configured to retrieve the second quantity of the products from a transaction accounting system.
26. The system of claim 23 , wherein the comparison module is further configured to calculate the difference of the first and second inventories, wherein the difference indicates a difference between a quantity of inventory calculated to be in a store and a quantity of inventory actually in the store.
27. The system of claim 16 , wherein the reception module is further configured to receive the image data of a real world layout from a plurality of cameras.
28. The system of claim 27 , wherein the reception module is further configured to receive the image data from the plurality of cameras positioned to be able to generate a 3D model of the real world layout.
29. The system of claim 16 , further comprising:
program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to implement:
an image adjustment module configured to adjust the image data to match properties of the 3D model, wherein the properties include one or more of lighting characteristics, contrast, hue, shadow placement, and shadow removal.
30. The system of claim 16 , further comprising:
program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to implement:
an instruction generation module configured to generate an instruction to conform the real world layout to the planned layout of the environment.
31. A computer comprising:
a memory area configured to store image data reflecting a real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions and a 3D model of a planned layout, the 3D model representing the planned layout across the three dimensions in said memory area; and
a processor coupled to said memory area, said processor configured to:
receive the image data of the real world layout;
receive audio data of the real world layout, the audio data reflecting at least one of sounds, voice, or music from the real world layout;
compare the 3D model of the planned layout to the received image data, wherein the comparing is able to convert the 3D model to a 2D representation and able to convert the image data to a rendered 3D model;
compare a sound model of the planned layout of the environment to the received audio data, the sound model representing the planned layout; and
determine whether results of the comparison reach a particular threshold and responsively indicate when the first particular threshold is reached.
32. A computer-based method comprising:
receiving multimedia data of a real world layout at a computing device, the multimedia data including audio data reflecting at least one of sounds, voice, or music from the real world layout and the multimedia data reflecting the real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions;
comparing a 3D model of a planned layout of the real world layout to the received multimedia data, the 3D model, from a tangible memory operatively coupled to the computing device, representing the planned layout across the three dimensions;
comparing a sound model of the planned layout of the environment to the received multimedia data, the sound model representing the planned layout; and
determining, by the computing device, whether results of the comparison reach a first particular threshold and responsively indicating when the first particular threshold has been reached.