IP Library Granted Patent US 9,870,633
Granted Patent B2
US 9,870,633 · App. 15/387,219 · Granted Jan 16, 2018

Automated highlighting of identified text

Inventors: Adam Wiggen Kraft (Mountain View, CA); Arnab Sanat Kumar Dhua (Mountain View, CA); Douglas Ryan Gray (Mountain View, CA); Xiaofan Lin (Palo Alto, CA); Yu Lou (Stanford, CA); Sunil Ramesh (San Jose, CA); Colin Jon Taylor (Orinda, CA); David Creighton Mott (Los Altos, CA)
Assignee: A9.COM, INC.
G06T11/60G06K9/00577G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,870,633
App. No.
15/387,219
Granted
Jan 16, 2018
Kind
B2
Abstract

Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Claims (55)

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

receive a user preference, the user preference selected for an individual user;

access a user identified word of a plurality of user identified words, the user identified word associated with the user preference and a color selected for the individual user;

obtain image data that includes a representation of text;

use an object recognition algorithm to determine a recognized word represented in the image data;

associate the user identified word with the recognized word;

display the recognized word emphasized by a visual color attribute, the visual color attribute matching the color associated with the user identified word; and

arrange the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the visual color attribute comprises at least one of a bounding box around the recognized word, a graphical representation of the recognized word in color, or a highlighting of the recognized word with a color.

3. The non-transitory computer-readable storage medium of claim 2 , wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

receive the plurality of user identified words from a user; and

enable the user to assign a first overlay to a first set of words and a second overlay to a second set of words of the plurality of user identified words.

4. The non-transitory computer-readable storage medium of claim 3 , wherein the first overlay is different in appearance relative to the second overlay.

5. The non-transitory computer-readable storage medium of claim 1 , wherein an overlay associated with each occurrence of the recognized word moves in corresponding motion with at least one word of the representation of text on a display element in a live field of view of a camera.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

identify a category associated with the recognized word; and

apply, based at least in part on the identified category, a list of one or more user identified words.

7. A computer-implemented method, comprising:

receiving a user preference, the user preference selected for an individual user;

accessing a user identified word of a plurality of user identified words, the user identified word associated with user preference and a color selected for the individual user;

obtaining image data that includes a representation of text;

using an object recognition algorithm to determine a recognized word represented in the image data;

associating the user identified word with the recognized word;

displaying the recognized word emphasized by a visual color attribute, the visual color attribute matching the color associated with the user identified word; and

arranging the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

8. The computer-implemented method of claim 7 , wherein the image data is obtained by a camera, and wherein the image data is displayed on a display element as at least one of a live field of view of the camera or a real-time video captured by the camera.

9. The computer-implemented method of claim 8 , wherein the visual color attribute comprises displaying at least one of a bounding box around the recognized word, or displaying a graphical representation of the recognized word.

10. The computer-implemented method of claim 8 , wherein the visual color attribute comprises displaying a graphical overlay at least one of above or adjacent the recognized word.

11. The computer-implemented method of claim 10 , wherein the graphical overlay moves in corresponding motion with the recognized word on the display element in the live field of view of the camera.

12. The computer-implemented method of claim 10 , wherein the graphical overlay includes an additional visual attribute operable to cause the recognized word to be emphasized relative to a remainder of words of the representation of text displayed on the display element.

13. The computer-implemented method of claim 10 , wherein the graphical overlay further comprises at least one of a bounding box around the recognized word, a graphical representation of the recognized word in color, or color highlighting.

14. The computer-implemented method of claim 10 , further comprising:

receiving the plurality of user identified words from a user; and

enabling the user to assign an overlay to individual ones of the plurality of user identified words.

15. The computer-implemented method of claim 7 , further comprising:

identifying a category associated with the recognized word; and

applying a list of user identified words based at least in part on the identified category.

16. A computing device, comprising:

a processor;

a camera;

a display screen; and

memory including instructions that, when executed by the processor, cause the computing device to:

receive a user preference associated with behavior of an individual user;

access a user identified word of a plurality of user identified words, the user identified word associated with the user preference and a color selected for the individual user;

obtain image data that includes a representation of text;

use an object recognition algorithm to determine a recognized word represented in the image data;

associated the user identified word with the recognized word;

display associated with the recognized word a visual color attribute, the visual color attribute matching the color associated with the user identified word; and

arrange the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

17. The computing device of claim 16 , wherein the instructions, when executed by the processor, further enable the computing device to:

identify a category associated with the recognized word; and

determine a user identified word list based at least in part on the identified category.

18. The computing device of claim 16 , wherein the visual color attribute comprises displaying at least one of a bounding box around the recognized word, displaying a graphical representation of the recognized word, or color highlighting.

19. The computing device of claim 16 , wherein the visual color attribute comprises displaying a graphical overlay above or adjacent to the recognized word.

20. The computing device of claim 19 , wherein the graphical overlay above or adjacent the recognized word moves in corresponding motion with the recognized word on the display screen in a live field of view of the camera.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2024
From: A9.COM, INC.
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 069167/0493 →
Continuity (2)
Continuation 14037275 · Sep 25, 2013
Related Publication 20170103560A1 · Apr 13, 2017