IP Library Granted Patent US 10,977,799
Granted Patent B2
US 10,977,799 · App. 15/934,564 · Granted Apr 13, 2021

Automatically capturing information such as capturing information using a document-aware device

Inventors: Martin T. King (Vashon Island, WA); Redwood Stephens (Seattle, WA); Claes-Fredrik Mannby (Mercer Island, WA); Jesse Peterson (Mercer Island, WA); Mark Sanvitale (Beaverton, OR); Michael J. Smith (Seattle, WA)
Assignee: Google LLC
G06T7/20G01B11/00G01B11/14G06F16/434G06F16/93G06F40/103G06F40/169G06F40/174G06K9/00442G06K9/18G06T7/70H04N1/0036H04N1/00307H04N1/00331H04N1/00355H04N1/00374H04N1/00408H04N1/00469H04N1/00689H04N1/00737H04N1/00758H04N1/00782H04N1/19594H04N1/32144H04N5/23203H04N5/23222G06T2207/30176H04N2201/0084H04N2201/0096H04N2201/3249H04N2201/3266
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,977,799
App. No.
15/934,564
Granted
Apr 13, 2021
Kind
B2
Abstract

A system and method for automatically changing the operation of a mobile device in response to a presence of information is described. In some examples, the system determines an information capture device is proximate to text, automatically changes operation of the capture device to a certain mode, captures the text, and performs an action associated with the captured text.

Claims (38)

1. A method, comprising:

capturing an image of a target object, by a capture component of a mobile device that is being oriented by a user in a direction of the target object;

after the capture component of the mobile device has captured the image of the target object, determining a duration over which the capture component remains oriented by the user in the direction of the target object;

determining, based on the duration, an intent of the user to obtain additional information about the target object; and

performing, by the mobile device, an action based on the determined intent of the user.

2. The method of claim 1 , further comprising determining a proximity of the capture component of the mobile device to the target object, wherein determining the intent of the user to obtain additional information comprises determining the intent of the user based on the proximity and the duration.

3. The method of claim 1 , further comprising identifying an electronic document corresponding to the target object, wherein performing the action comprises presenting information associated with the electronic document at the mobile device.

4. The method of claim 3 , wherein identifying the electronic document corresponding to the target object comprises:

identifying text from the captured image of the target object; and

identifying the electronic document based on the identified text.

5. The method of claim 1 , further comprising identifying address information from the captured image of the target object, wherein performing the action comprises providing a mapping application that displays directions from a current location of the mobile device to a location that corresponds to the identified address information.

6. The method of claim 1 , wherein determining the intent of the user to obtain additional information about the target object comprises determining the intent of the user to obtain additional information at a time of the capturing.

7. The method of claim 1 , wherein determining the intent of the user to obtain additional information about the target object comprises determining the intent of the user to obtain additional information at a future time.

8. The method of claim 7 , wherein the instructions cause the mobile device to perform operations comprising storing the image of the target object in a timeline entry that causes retrieval of the additional information about the target object when the timeline entry is selected by the user.

9. A mobile device, comprising:

a capture component; and

one or more data processors that interact with the capture component and execute instructions that cause the mobile device to perform operations comprising:

capturing an image of a target object, by the capture component of the mobile device that is being oriented by a user in a direction of the target object;

after the capture component of the mobile device has captured the image of the target object, determining a duration over which the capture component remains oriented by the user in the direction of the target object;

determining, based on the duration, an intent of the user to obtain additional information about the target object; and

performing an action based on the determined intent of the user.

10. The mobile device of claim 9 , wherein the instructions cause the mobile device to perform operations further comprising determining a proximity of the capture component of the mobile device to the target object, wherein determining the intent of the user to obtain additional information comprises determining the intent of the user based on the proximity and the duration.

11. The mobile device of claim 9 , wherein the instructions cause the mobile device to perform operations further comprising identifying an electronic document corresponding to the target object, wherein performing the action comprises presenting information associated with the electronic document at the mobile device.

12. The mobile device of claim 11 , wherein identifying the electronic document corresponding to the target object comprises:

identifying text from the captured image of the target object; and

identifying the electronic document based on the identified text.

13. The mobile device of claim 9 , wherein the instructions cause the mobile device to perform operations further comprising identifying address information from the captured image of the target object, wherein performing the action comprises providing a mapping application that displays directions from a current location of the mobile device to a location that corresponds to the identified address information.

14. The mobile device of claim 9 , wherein determining the intent of the user to obtain additional information about the target object comprises determining the intent of the user to obtain additional information at a time of the capturing.

15. The mobile device of claim 9 , wherein determining the intent of the user to obtain additional information about the target object comprises determining the intent of the user to obtain additional information at a future time.

16. The mobile device of claim 15 , wherein the instructions cause the mobile device to perform operations comprising storing the image of the target object in a timeline entry that causes retrieval of the additional information about the target object when the timeline entry is selected by the user.

17. A non-transitory computer-readable medium whose contents, when executed by a mobile device, cause the mobile device to perform a method for performing operations comprising:

capturing an image of a target object, by a capture component of a mobile device that is being oriented by a user in a direction of the target object;

after the capture component of the mobile device has captured the image of the target object, determining a duration over which the capture component remains oriented by the user in the direction of the target object;

determining, based on the duration, an intent of the user to obtain additional information about the target object; and

performing, by the mobile device, an action based on the determined intent of the user.

18. The non-transitory computer-readable medium of claim 17 , the operations further comprising determining a proximity of the capture component of the mobile device to the target object, wherein determining the intent of the user to obtain additional information comprises determining the intent of the user based on the proximity and the duration.

19. The non-transitory computer-readable medium of claim 17 , the operations further comprising identifying an electronic document corresponding to the target object, wherein performing the action comprises presenting information associated with the electronic document at the mobile device.

20. The non-transitory computer-readable medium of claim 17 , the operations further comprising identifying address information from the captured image of the target object, wherein performing the action comprises providing a mapping application that displays directions from a current location of the mobile device to a location that corresponds to the identified address information.

Assignments (4)
NUNC PRO TUNC ASSIGNMENT Recorded Aug 3, 2021
From: GOOGLE LLC
To: KYOCERA CORPORATION
Reel/Frame 057651/0445 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2018
From: KING, MARTIN T.; STEPHENS, REDWOOD; MANNBY, CLAES-FREDRIK; PETERSON, JESSE; SANVITALE, MARK; SMITH, MICHAEL J.
To: EXBIBLIO B.V.
Reel/Frame 045426/0921 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2018
From: EXBIBLIO B.V.
To: GOOGLE INC.
Reel/Frame 045426/0966 →
CERTIFICATE OF CONVERSION Recorded Apr 3, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 045828/0795 →
Continuity (7)
Continuation 14797479 · Jul 13, 2015
Continuation 14161181 · Jan 22, 2014
Continuation 12660146 · Feb 18, 2010
Provisional Application 61184273 · Jun 4, 2009
Provisional Application 61159757 · Mar 12, 2009
Provisional Application 61153614 · Feb 18, 2009
Related Publication 20180211395A1 · Jul 26, 2018
Cited By (1)
US 12,501,225