IP Library › Granted Patent US 10,402,647
Granted Patent B2
US 10,402,647 · App. 15/597,905 · Granted Sep 3, 2019

Adapted user interface for surfacing contextual analysis of content

Inventor: Eric Schuh (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
G06K9/00664G06F16/29G06F16/583H04N5/232G06K2209/27
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,402,647
App. No.
15/597,905
Granted
Sep 3, 2019
Kind
B2
Abstract

Non-limiting examples of the present disclosure describe surfacing of contextual suggestions for image content within an applications/service, a non-limiting example of which is a camera application. Image content is captured through a camera application executing on a computing device. A determination of user intent is generated based on an analysis of a result of execution of image recognition processing on the image content and a result of an analysis of context for data associated with the image content. As an example, processing operations related to determining user intent may be executed by an intelligent personal assistant service that is executing on the computing device. One or more contextual suggestions for the image content are generated based on the determination of the user intent. The contextual suggestions for the image content are surfaced, for example, within the camera application or an accessory device connected with the computing device, among other examples.

Claims (49)

1. A method comprising:

accessing, via an application executing on a computing device, stored image content;

receiving a user selection of the stored image content for display;

displaying, based on the received user selection, the stored image content in a user interface of the application;

determining user intent for the stored image content based on:

executing image recognition processing on the stored image content,

analyzing context for data associated with the stored image content including executing geofencing processing for the stored image content, and

generating a determination of the user intent based on an analysis of a result of the executed image recognition processing and a result of an analysis of the context for the data associated with the stored image content;

generating one or more contextual suggestions for the stored image content based on the determination of the user intent; and

surfacing, as part of the user interface of the application, the one or more contextual suggestions along with the stored image content.

2. The method of claim 1 , wherein the user intent for the stored image content is further determined based on executing processing of an audio signal that is captured, from a microphone of the computing device, proximate with the displaying of the stored image content.

3. The method of claim 1 , wherein the surfacing presents the stored image content and the one or more contextual suggestions using an accessory device connected with the computing device.

4. The method of claim 3 , wherein the accessory device is an augmented reality (AR) hardware component that generates a virtual representation of the one or more contextual suggestions overlaying the stored image content.

5. A system comprising:

at least one processor; and

a memory, operatively connected with the at least one processor, storing computer-executable instructions that, when executed by the at least one processor, causes the at least one processor to execute a method that comprises:

accessing, via an application executing on the system, stored image content;

receiving a user selection of the stored image content for display;

displaying, based on the received user selection, the stored image content in a user interface of the application;

determining user intent for the stored image content based on:

executing image recognition processing on the stored image content,

analyzing context for data associated with the stored image content including executing geofencing processing for the stored image content, and

generating a determination of the user intent based on an analysis of a result of the executed image recognition processing and a result of an analysis of the context for the data associated with the stored image content;

generating one or more contextual suggestions for the stored image content based on the determination of the user intent; and

surfacing, as part of the user interface of the application, the one or more contextual suggestions along with the stored image content.

6. The system of claim 5 , wherein the user intent for the stored image content is further determined based on executing processing of an audio signal that is captured, from a microphone of the computing device, proximate with the displaying of the stored image content.

7. The system of claim 5 , wherein the surfacing presents the stored image content and the one or more contextual suggestions using an accessory device connected with the computing device.

8. The system of claim 7 , wherein the accessory device is an augmented reality (AR) hardware component that generates a virtual representation of the one or more contextual suggestions overlaying the stored image content.

9. A computer-readable medium storing computer-executable instructions that, when executed by at least one processor, causes the at least one processor to execute a method comprising:

accessing, via an application executing on a computing device, stored image content;

receiving a user selection of the stored image content for display;

displaying, based on the received user selection, the stored image content in a user interface of the application;

determining user intent for the stored image content based on:

executing image recognition processing on the stored image content,

analyzing context for data associated with the stored image content including executing geofencing processing for the stored image content, and

generating a determination of the user intent based on an analysis of a result of the executed image recognition processing and a result of an analysis of the context for the data associated with the stored image content;

generating one or more contextual suggestions for the stored image content based on the determination of the user intent; and

surfacing, as part of the user interface of the application, the one or more contextual suggestions along with the stored image content.

10. The method of claim 1 , wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a designation for an object in the stored image content based on a result of the comparing, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the designation of the object.

11. The method of claim 1 , wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a determination that an object in the stored image content has changed locations, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the determination that the object has changed locations.

12. The method of claim 1 , wherein an analysis of the context for data comprises correlating an item in the stored image content with a search query being executed through the computing device, and wherein the one or more contextual suggestions comprise one or more of: a suggested action to execute based on a correlation of the item with the search query, and a search result based on the correlation of the item with the search query.

13. The method of claim 1 , wherein the accessing of the stored image content comprises a selection, through the computing device, of the stored image content from a user-specific distributed storage.

14. The system of claim 5 , wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a designation for an object in the stored image content based on a result of the comparing, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the designation of the object.

15. The system of claim 5 , wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a determination that an object in the stored image content has changed locations, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the determination that the object has changed locations.

16. The system of claim 5 , wherein an analysis of the context for data comprises correlating an item in the stored image content with a search query being executed through the computing device, and wherein the one or more contextual suggestions comprise one or more of: a suggested action to execute based on a correlation of the item with the search query, and a search result based on the correlation of the item with the search query.

17. The system of claim 5 , wherein the accessing of the stored image content comprises a selection, through the computing device, of the stored image content from a user-specific distributed storage.

18. The computer-readable medium of claim 9 , wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a designation for an object in the stored image content based on a result of the comparing, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the designation of the object.

19. The computer-readable medium of claim 9 , wherein an analysis of the context for data comprises comparing a geo-fencing result for the stored image content with a geo-fencing result of other image content, wherein the context comprises a determination that an object in the stored image content has changed locations, and wherein the one or more contextual suggestions comprise a suggested action to execute based on the determination that the object has changed locations.

20. The computer-readable medium of claim 9 , wherein an analysis of the context for data comprises correlating an item in the stored image content with a search query being executed through the computing device, and wherein the one or more contextual suggestions comprise one or more of: a suggested action to execute based on a correlation of the item with the search query, and a search result based on the correlation of the item with the search query.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2017
From: SCHUH, ERIC
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 042415/0061 →
Continuity (1)
Related Publication 20180336411A1 · Nov 22, 2018
Cited By (6)
US 12,548,245 US 12,572,253 US 12,613,938 US 12,633,068 US 12,670,001 US 12,681,561