Methods and systems for interacting with physical objects
View Patent ↗A user detects data from a physical object (e.g., a poster) using a handheld appliance. The appliance then contacts a remote computer based on the data obtained from the object, and can perform any of a variety of functions, such as purchasing tickets to a concert or movie, or downloading music. The data can be obtained from the object in various ways, such as by optical sensing or by electromagnetic detection.
1. A method comprising:
pointing a portable image sensor device towards a physical object that is free of visible digital encoding;
capturing one or more 2D frames of imagery from the physical object using the sensor device;
providing image information representing the object to a processing device, and receiving first data derived from said provided image information from the processing device;
using the first data to identify corresponding second data in a data structure that stores plural second data, where the data structure associates the plural second data with plural different first data; and
though use of the second data identified from the data structure, initiating an action relating to the physical object from which the imagery was captured;
wherein the method further includes analyzing the imagery for one or more clues, the clue(s) helping the first data to better correspond to a particular object, rather than to another object or to visual clutter within the imagery, the analyzing including one or more of the following:
i. regarding a point at a center of an image frame as corresponding to the particular physical object, and from that point analyzing to discern an associated boundary;
ii. identifying features in the imagery having a relatively greater movement between frames of imagery, as contrasted with other image features;
iii. identifying plural non-parallel straight edge boundaries in the imagery;
iv. identifying a trapezoidal shape in the imagery;
v. identifying an object within the imagery by reference, at least in part, to color; and
vi. identifying an object within the imagery by reference, at least in part, to spatial frequency.
2. The method of claim 1 that includes capturing plural frames of imagery, wherein the first data is dependent on more than one of said frames.
3. The method of claim 2 wherein one of said frames depicts the object from one perspective, and a second of said frames depicts the object from a second, different, perspective.
4. The method of claim 1 wherein the portable image sensor device is attached to a body of a user.
5. The method of claim 4 wherein the portable image sensor device is head-mounted on the user.
6. The method of claim 1 that further includes processing the image information in the processing device, said processing comprising applying a digital watermark decoding algorithm to the image information to produce the first data therefrom.
7. The method of claim 1 wherein said capturing comprises capturing the one or more frames of imagery from a printed poster.
8. The method of claim 1 wherein the action comprises playing a clip of entertainment content.
9. The method of claim 1 that includes discerning orientation data, and mapping image information to another perspective in accordance with the orientation data, wherein the first data is based on the mapped image data.
10. The method of claim 1 wherein the processing device applies an algorithm to the image information that is robust to at least certain changes in scale.
11. The method of claim 1 wherein the processing device applies an algorithm to the image information that is robust to at least certain changes in rotation.
12. The method of claim 1 wherein the sensor forages, operating in a free-running mode to grab plural frames of image data per second, rather than being triggered by a user to capture imagery only when the device is pointed towards an intended object.
13. The method of claim 1 wherein the analyzing includes regarding a point at a center of an image frame as corresponding to the particular physical object, and from that point analyzing to discern an associated boundary.
14. The method of claim 13 wherein the analyzing also includes one of acts (ii) through (vi).
15. The method of claim 1 wherein the analyzing includes identifying features in the imagery associated with the particular physical object by reference to greater movement of certain image features between frames of imagery, as contrasted with other image features.
16. The method of claim 15 wherein the analyzing also includes one of acts (i) or (iii)-(vi).
17. The method of claim 1 wherein the analyzing includes identifying plural non-parallel straight edge boundaries in the imagery.
18. The method of claim 17 wherein the analyzing also includes one of acts (i)-(ii) or (iv)-(vi).
19. The method of claim 1 wherein the analyzing includes identifying a trapezoidal shape in the imagery.
20. The method of claim 19 wherein the analyzing also includes one of acts (i)-(iii) or (v)-(vi).
21. The method of claim 1 wherein the analyzing includes identifying an object within the imagery by reference, at least in part, to color.
22. The method of claim 21 wherein the analyzing also includes one of acts (i)-(iv) or (vi).
23. The method of claim 1 wherein the analyzing includes identifying an object within the imagery by reference, at least in part, to spatial frequency.
24. The method of claim 23 wherein the analyzing also includes one of acts (i)-(v).