Methods and arrangements for identifying objects
In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views—further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modelled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. Logos may be identified and used—or ignored—in product identification. A great variety of other features and arrangements are also detailed.
1. A method comprising the acts:
with a camera in a shopper's mobile device, capturing imagery depicting first and second instances of a particular item on a store shelf;
decoding first digital watermark data depicted with said first instance of the item, and decoding second digital watermark data depicted with said second instance of the item; and
presenting an augmented reality display on a screen of the shopper's mobile device, in which first expiration date information for the first instance of the item, determined from the first watermark data, and second, different, expiration date information for the second instance of the item, determined from the second instance of the item, are displayed as a graphical overlay on the captured imagery;
wherein a physical representation of the digital watermark data on each of said items comprises plural spatial blocks tiled edge-to-edge, each block encoding expiration date information, so that determination of each item's respective expiration date information is not dependent on one particular region of each item being visible to the mobile device.
2. The method of claim 1 in which the graphical overlay comprises a colored heat map, wherein the shopper can identify newest and oldest inventory among the plural packages.
3. The method of claim 1 that includes capturing said imagery using a headworn mobile device.
4. The method of claim 3 that further includes:
sensing direction of a user's gaze, using an eye-tracking system associated with said headworn mobile device;
consulting store layout data, in a memory, to identify a product stocked at a shelf location being viewed by the user; and
taking an action based on said identified product.
5. The method of claim 1 in which the shopper's mobile device further comprises a shopping list, and the method further includes the acts:
sensing the shopper's location in an aisle that stocks an item on the shopping list; and
in response to said sensed shopper location, activating an illumination source at the item to draw attention of the shopper to the location of the item.
6. The method of claim 1 in which the digital watermark data on each of said items spans more than 50% of its surface area.
7. The method of claim 1 in which said decoding and presenting acts are performed by software whose execution is triggered by detection of a shopper location or a shopper audio environment.
8. The method of claim 1 in which said items each comprise plural planar faces, wherein a physical representation of the digital watermark data on each of said items is located on two or more of said faces, so that determination of each item's respective expiration date information is not dependent on one particular face of each item being visible to the mobile device.
9. A non-transitory computer storage medium containing software instructions adapted to configure a processor-based system to interact with first and second instances of a particular item located on a store shelf, by performing acts including:
capturing imagery depicting the first and second instances of the particular item on the store shelf;
decoding first digital watermark data from image data depicting the first instance of the item, and decoding second digital watermark data from image data depicting the second instance of the item; and
presenting an augmented reality display on a screen of the system, in which first expiration date information for the first instance of the item, determined using the first watermark data, and second, different, expiration date information for the second instance of the item, determined using the second instance of the item, are displayed as a graphical overlay on the captured imagery.
10. The computer storage medium of claim 9 in which the graphical overlay comprises a colored heat map.
11. A system including:
a mobile device; and
first and second instances of a particular item located on a store shelf, bearing physical representations of first and second digital watermarks, respectively;
the mobile device comprising:
first means for capturing imagery depicting the first and second instances of the particular item on the store shelf;
a digital watermark decoder; and
a screen, a processor, and a memory, the memory containing instructions configuring the system to perform acts including presenting an augmented reality display on the screen, in which first expiration date information for the first instance of the item, determined from first watermark data decoded by the digital watermark decoder from first imagery that depicts said first instance of said item, and second, different, expiration date information for the second instance of the item, determined from second watermark data decoded by the digital watermark decoder from second imagery that depicts said second instance of said item, are displayed as a graphical overlay on the captured imagery;
wherein a physical representation of the digital watermark data on each of said items comprises plural spatial blocks tiled edge-to-edge, each block encoding expiration date information, so that determination of each item's respective expiration date information is not dependent on one particular region of each item being visible to the mobile device.
12. The system of claim 11 in which the digital watermark decoder comprises a processor and a memory, the memory containing software instructions configuring the processor to perform acts including:
estimating rotation and scale of an orientation signal in the imagery;
estimating translation of the orientation signal in the imagery;
extracting raw message bits from locations in the imagery indicated by the estimated rotation, scale and translation of the orientation signal; and
error-correcting decoding the extracted raw message bits to produce decoded digital watermark data.
13. The system of claim 11 in which the graphical overlay comprises a colored heat map, wherein a user can identify newest and oldest inventory among the plural packages.
14. A headworn system according to claim 11 .
15. The system of claim 11 in which said items each comprise plural planar faces, wherein a physical representation of the digital watermark data on each of said items is located on two or more of said faces so that determination of each item's respective expiration date information is not dependent on one particular face of each item being visible to the mobile device.