IP Library Granted Patent US 9,398,210
Granted Patent B2
US 9,398,210 · App. 13/088,259 · Granted Jul 19, 2016

Methods and systems for dealing with perspective distortion in connection with smartphone cameras

Inventors: John Stach (Tualatin, OR); Ravi K. Sharma (Portland, OR)
Assignee: Digimarc Corporation
H04N5/23222G06T1/0021H04N5/23258H04N5/23293
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,398,210
App. No.
13/088,259
Granted
Jul 19, 2016
Kind
B2
Abstract

Arrangements are detailed to aid a user in positioning a camera-equipped device (e.g., a smartphone) at a particular viewing position relative to an object being imaged. Other embodiments discern the device's viewing position, and then undertake image or other processing that is dependent on such viewpoint. A great number of other features and arrangements are also detailed.

Claims (65)

1. A method of using a camera-equipped mobile telecommunications device, comprising:

capturing first imagery of an object, when the device is at a first position relative to the object;

capturing second imagery of the object, when the device is at a second, different, position relative to the object;

by reference to the first and second imagery, determining pose data that characterizes at least part of a geometrical pose between the device and the object;

through use of said determined pose data, obtaining image data corresponding to the object with reduced perspective distortion, relative to at least one of the first or second captured imagery; and

processing said obtained image data to derive identification data therefrom.

2. The method of claim 1 that further includes:

using a device sensor distinct from the device camera, acquiring first sensor data when the device is in the first position;

using said device sensor, acquiring second sensor data when the device is in the second position;

by reference to the first and second acquired sensor data, estimating a distance between the first and second positions; and

by reference to said estimated distance, estimating scale information about the object depicted in the obtained image data.

3. The method of claim 2 in which said device sensor comprises an accelerometer, magnetometer or gyroscopic sensor.

4. The method of claim 2 in which said device sensor comprises a radio signal-based location determination system.

5. The method of claim 1 in which the object comprises a substrate printed with encoded machine-readable data, and the processing comprises decoding said machine-readable data.

6. The method of claim 5 in which the identification data comprises decoded digital watermark data.

7. The method of claim 1 in which the object comprises a substrate printed with text, and the processing comprises recognizing the text, and outputting information corresponding thereto.

8. The method of claim 1 in which the identification data comprises fingerprint data.

9. The method of claim 8 in which the fingerprint data comprises SIFT or SURF data.

10. The method of claim 1 in which the obtaining act includes providing one or more prompts to a user to guide the user in positioning the device, so as to capture imagery with said reduced perspective distortion.

11. The method of claim 10 in which the providing act includes augmenting an image of the object, displayed on the screen of the device, with a superimposed graphic feature, a position of said superimposed graphic feature being dependent on said determined pose data.

12. The method of claim 1 in which the obtaining act includes warping captured imagery so as to yield imagery with said reduced perspective distortion.

13. The method of claim 1 in which the determining act includes determining a plane in which a surface of the object lies, relative to the device.

14. The method of claim 1 in which the object comprises a printed substrate.

15. A method of using a camera-equipped mobile telecommunications device, comprising:

using a device sensor distinct from the device camera, acquiring first sensor data when the device is in a first position relative to an object;

using said device sensor, acquiring second sensor data when the device is in a second position relative to the object;

by reference to the first and second acquired sensor data, estimating a distance between the first and second positions; and

by reference to said estimated distance, and to imagery captured by the camera, estimating (1) a distance to the object; or (2) scale information about the object as depicted in the imagery.

16. A method of using a camera-equipped mobile telecommunications device, comprising:

sensing first data relating to position of the device when the device is at a first location, in said first location the device touching an object;

sensing second data relating to position of the device when the device is at a second location, removed from the object;

capturing first imagery from the object using the device camera; and

by reference to the first and second data, determining pose data about a geometrical pose of the device relative to the object associated with the captured imagery.

17. The method of claim 16 in which, in said first location, a face of the device is parallel with a face of said object.

18. The method of claim 16 that further includes, through use of said determined pose data, obtaining image data corresponding to the object with reduced perspective distortion, relative to said captured first imagery.

19. The method of claim 18 in which the obtaining act includes providing one or more prompts to a user to guide the user in positioning the device, so as to capture imagery with said reduced perspective distortion.

20. The method of claim 19 in which the providing act includes augmenting an image of the object, displayed on the screen of the device, with a superimposed graphic feature, a position of said superimposed graphic feature being dependent on said determined pose data.

21. The method of claim 18 in which the obtaining act includes warping the captured first imagery so as to yield imagery with said reduced perspective distortion.

22. A method of using a mobile telecommunications device, comprising:

sensing a first orientation of the device when the device is at a first position, in said first position the device touching an object;

presenting a user interface including an indication related to the orientation of the device in said first position; and

updating said user interface to present an indication related to an updated orientation of the device, as the device is repositioned to a second position;

wherein the user interface is adapted to help a user reposition the device to a second position in which the orientation of the device is parallel to the first orientation.

23. The method of claim 22 in which:

the device includes a camera portion;

the object includes steganographically encoded data sensible from said object; and

the method includes capturing imagery of said object, and

decoding the steganographically encoded data from said captured imagery.

24. The method of claim 22 in which, in the first position, a face of the device is parallel with a face of said object.

25. A method of using a camera-equipped mobile telecommunications device, comprising:

capturing image data depicting first and second objects;

by reference to image data related to the second object, determining scale information related to the first object; and

presenting a user interface that aids a user in movement of the device to a different viewing position, based on the determined scale information.

26. The method of claim 25 that includes processing image data from the device to derive identification data corresponding to the first object.

27. The method of claim 26 wherein the identification data comprises fingerprint data, decoded machine readable data, or recognized text data.

28. A method of using a camera-equipped mobile telecommunications device, comprising:

capturing image data depicting first and second objects;

by reference to image data related to the second object, determining scale information related to the first object; and

using the determined scale information in a procedure that processes captured image data to derive fingerprint data or decoded machine readable data corresponding to the first object.

29. A method comprising:

capturing imagery using a camera-equipped mobile telecommunications device; and

presenting with the captured imagery, on a screen of the device, a guide indicia adapted to establish a desired relationship between a feature of the presented imagery, and a physical item placed on said screen.

30. The method of claim 29 in which the desired relationship is a desired scale factor of the imagery.

31. The method of claim 25 , wherein the captured image data simultaneously depicts the first and second objects.

32. The method of claim 28 , wherein the captured image data simultaneously depicts the first and second objects.

Assignments (4)
ARTICLES OF CONVERSION Recorded Jun 19, 2026
From: DIGIMARC CORPORATION
To: DIGIMARC LLC
Reel/Frame 075863/0211 →
ARTICLES OF AMENDMENT OFTHE ARTICLES OF ORGANIZATION OF DIGIMARC LLC Recorded Jun 19, 2026
From: DIGIMARC LLC
To: DMRC LLC
Reel/Frame 075863/0266 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2012
From: STACH, JOHN; SHARMA, RAVI K.
To: DIGIMARC CORPORATION
Reel/Frame 027868/0806 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 2, 2011
From: STACH, JOHN
To: DIGIMARC CORPORATION
Reel/Frame 026688/0756 →
Continuity (2)
Provisional Application 61446464 · Feb 24, 2011
Related Publication 20120218444A1 · Aug 30, 2012