IP Library Granted Patent US 11,599,807
Granted Patent B2
US 11,599,807 · App. 16/916,855 · Granted Mar 7, 2023

Interactive search training

Inventors: Gerold Hintz (Munich, DE); Eric Crestan (Munich, DE); Andreas Bode (Munich, DE); Tobias Rolf Hassmann (Munich, DE)
Assignee: Microsoft Technology Licensing, LLC
G06N5/04G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,599,807
App. No.
16/916,855
Granted
Mar 7, 2023
Kind
B2
Abstract

Aspects of the present disclosure relate to interactive search training. A training canvas comprises results associated with a search query. The training canvas may be used as part of a training session that occurs during normal use of a search platform. When the search platform is first used, the results may be provided based on an existing model. An irrelevant result may be removed from the training canvas, such that a replacement result is added in its place. Additionally, results may be reordered, thereby indicating a ranking with which results should be displayed. Such interactions with the training canvas may be used to generate training data, such that a new model is trained accordingly. Thus, interactions with the training canvas yield high-quality training data that is usable to generate a model having equal or greater performance than a model that was trained using an equivalent amount of implicit training data.

Claims (57)

1. A system comprising:

at least one processor; and memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising:

generating, as part of a first teaching session, a training canvas comprising a first set of search results associated with a first search query, the training canvas comprising a feedback graphical user interface (GUI) associated with one or more training control elements;

receiving feedback information comprising an explicit indication associated with a search result of the first set of search results;

training a model based at least in part on the explicit indication associated with the search result; and

identifying, based on the model associated with the explicit indication received via the feedback interface of the training canvas, a second set of search results based on a second search query that is different from the first search query.

2. The system of claim 1 , wherein: the training canvas further comprises an irrelevant training control element associated with the search result; and the explicit indication associated with the search result is an indication that the search result is irrelevant to the first search query.

3. The system of claim 1 , wherein: the training canvas further comprises a reorder training control element associated with the search result; and

the explicit indication associated with the search result is an indication relating to a rank of the search result as compared to another search result of the set of search results.

4. The system of claim 1 , wherein the model is further trained based at least in part on training data associated with a different teaching session than the first teaching session.

5. The system of claim 1 , wherein the first set of search results associated with the first search query is generated using an initial model.

6. The system of claim 2 , wherein the set of operations further comprises:

receiving a request for an additional search result associated with the first search query; and

providing, in response to the received request, the additional search result for the first search query.

7. The system of claim 1 , wherein the second search query is associated with a different user than the first search query.

8. A method for generating feedback information via a training canvas, comprising:

receiving, from a search platform, a set of search results associated with a search query;

displaying, at a client device, the training canvas comprising a feedback graphical user interface (GUI) associated with one or more training control elements and the set of search results, wherein a first search results of the set of search results is displayed in association with the one or more training control elements comprising an irrelevant training control element and a reorder training control element;

receiving, via the training canvas, an explicit indication that is:

an indication that the first search result is irrelevant to the search query; or

an indication relating to a first rank of the first search result as compared to a second search result of the set of search results;

generating feedback information based on the explicit indication; and

providing the feedback information to the search platform.

9. The method of claim 8 , further comprising:

when the explicit indication is the indication that the first search result is irrelevant to the search query:

removing the first search result from the training canvas; and

updating the training canvas to comprise a replacement search result.

10. The method of claim 8 , further comprising:

when the explicit indication is the indication relating to the first rank of the first search result, determining a second rank of the second search result as compared to the first rank of the first search result.

11. The method of claim 8 , further comprising:

receiving user input comprising the search query;

providing the search query to the search platform; and

receiving, in response to providing the search query, the set of search results.

12. The method of claim 8 , further comprising:

receiving, from the search platform a set of search queries;

generating a display of a set of search queries;

receiving a selection of the search query from the set of search queries;

providing the selected search query to the search platform; and

receiving, in response to providing the search query, the set of search results.

13. The method of claim 8 , wherein the training canvas is a single page comprising the set of search results.

14. A method for generating an updated model for a search platform, comprising:

generating, as part of a first teaching session, a training canvas comprising a first set of search results associated with a first search query, the training canvas comprising a feedback graphical user interface (GUI) associated with one or more training control elements;

receiving feedback information comprising an explicit indication associated with a search result of the first set of search results;

training a model based at least in part on the explicit indication associated with the search result; and

identifying, based on the model associated with the explicit indication received via the feedback interface of the training canvas, a second set of search results based on a second search query that is different from the first search query.

15. The method of claim 14 , wherein:

the training canvas further comprises an irrelevant training control element associated with the search result; and

the explicit indication associated with the search result is an indication that the search result is irrelevant to the first search query.

16. The method of claim 14 , wherein:

the training canvas further comprises a reorder training control element associated with the search result; and

the explicit indication associated with the search result is an indication relating to a rank of the search result as compared to another search result of the set of search results.

17. The method of claim 14 , wherein the model is further trained based at least in part on training data associated with a different teaching session than the first teaching session.

18. The method of claim 14 , wherein the first set of search results associated with the first search query is generated using an initial model.

19. The method of claim 14 , further comprising:

receiving a request for an additional search result associated with the first search query; and

providing, in response to the received request, the additional search result for the first search query.

20. The method of claim 14 , wherein the second search query is associated with a different user than the first search query.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2020
From: HINTZ, GEROLD; CRESTAN, ERIC; BODE, ANDREAS; HASSMANN, TOBIAS ROLF
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 053088/0801 →
Continuity (1)
Related Publication 20210406723A1 · Dec 30, 2021