IP Library Granted Patent US 11,138,388
Granted Patent B2
US 11,138,388 · App. 15/388,315 · Granted Oct 5, 2021

Method and system for facilitating a user-machine conversation

Inventors: Michael Emery (Scotts Valley, CA); Lavanya Colinjivadi Viswanathan (Sunnyvale, CA)
Assignee: Verizon Media Inc.
G06F40/35G06F16/3329G06F16/3346H04L51/02H04M3/4936
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,138,388
App. No.
15/388,315
Granted
Oct 5, 2021
Kind
B2
Abstract

The present teaching relates to facilitating a user-machine conversation. In one example, a query is obtained from a user. The query is directed to a first conversational bot. A reply in response to the query is obtained from the first conversational bot. A degree of validity of the reply is determined based on the reply and the query. A second conversational bot is determined based on the query and the degree of validity. The conversation is directed to the second conversational bot with the query.

Claims (79)

1. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for facilitating a user-machine conversation, the method comprising:

obtaining a query from a user, wherein the query is directed to a first conversational bot;

obtaining, from the first conversational bot, a reply in response to the query;

determining a degree of validity of the reply based on the reply and the query;

determining, in the event that the degree of validity of the reply meets a certain criterion, using a model for bot recommendation, a second conversational bot other than the first conversational bot, wherein the second conversational bot is selected from a plurality of conversational bots based on (i) one or more features of the query, (ii) a ranked list of confidence scores, each of which is associated with a corresponding one of the plurality of conversational bots, and (iii) one or more thresholds determined based on the reply, wherein the model is trained based on training data comprising historical query-reply pairs, each of which includes (i) a historical query from a conversational bot with an associated confidence score, and (ii) a historical reply corresponding to the historical query, wherein:

one or more of the historical query-reply pairs are selected based on their respective associated confidence scores and a threshold, and

one or more of the plurality of conversational bots handled the historical query in each of the historical query-reply pairs and provided a historical reply corresponding to the historical query, and

the training data is generated based on the historical query-reply pairs; and

directing the user-machine conversation to the second conversational bot with the query.

2. The method of claim 1 , wherein determining the degree of validity comprises:

determining a degree of matching between the reply and the query based on their semantic meanings;

obtaining a first confidence score related to the reply from the first conversational bot; and

determining the degree of validity of the reply based on the degree of matching and the first confidence score.

3. The method of claim 1 , further comprising:

comparing the degree of validity with the threshold, wherein the certain criterion comprises the degree of validity not exceeding the threshold;

providing the reply as a response to the query and providing the query and the reply for training the model for bot recommendation, when the degree of validity exceeds the threshold; and

determining the second conversational bot when the degree of validity does not exceed the threshold, indicating that the certain criterion has been met.

4. The method of claim 1 , wherein determining the second conversational bot comprises:

parsing the query to determine the one or more features when the reply meets the certain criterion; and

obtaining the model for bot recommendation.

5. The method of claim 4 , wherein the plurality of conversational bots are further determined based on the model, the method further comprises:

obtaining, for each of the plurality of conversational bots, the respective confidence score, wherein the ranked list of confidence scores is determined by:

ranking the plurality of conversational bots based on their respective confidence scores.

6. The method of claim 5 , further comprising:

generating a new conversational bot based on the query when none of the confidence scores of the plurality of conversational bots passes the one or more thresholds.

7. The method of claim 1 , further comprising:

providing a recommendation of the second conversational bot to the user as a response to the query.

8. The method of claim 1 , wherein the first conversational bot and the second conversational bot are independent and related to different topics, wherein the topics include at least one of: weather, restaurant, hotel, travel, sports, finance, or news.

9. A system, having at least one processor, storage, and a communication platform connected to a network for facilitating a user-machine conversation, comprising:

a user request analyzer configured for obtaining a query from a user, wherein the query is directed to a first conversational bot;

a bot reply analyzer configured for obtaining, from the first conversational bot, a reply in response to the query and determining a degree of validity of the reply based on the reply and the query;

a conversational bot recommender configured for determining, in the event that the degree of validity of the reply meets a certain criterion, using a model for bot recommendation, a second conversational bot other than the first conversational bot, wherein the second conversational bot is selected from a plurality of conversational bots based on (i) one or more features of the query, (ii) a ranked list of confidence scores, each of which is associated with a corresponding one of the plurality of conversational bots, and (iii) one or more thresholds determined based on the reply, wherein the model is trained based on training data comprising historical query-reply pairs, each of which includes (i) a historical query from a conversational bot with an associated confidence score, and (ii) a historical reply corresponding to the historical query, wherein:

one or more of the historical query-reply pairs are selected based on their respective associated confidence scores and a threshold, and

one or more of the plurality of conversational bots handled the historical query in each of the historical query-reply pairs and provided a historical reply corresponding to the historical query, and

the training data is generated based on the historical query-reply pairs; and

a conversational bot determiner configured for directing the user-machine conversation to the second conversational bot with the query.

10. The system of claim 9 , wherein the bot reply analyzer comprises:

a query/reply semantic matcher configured for determining a degree of matching between the reply and the query based on their semantic meanings;

a reply confidence score determiner configured for obtaining a first confidence score related to the reply from the first conversational bot; and

a bot switch request generator configured for determining the degree of validity of the reply based on the degree of matching and the first confidence score.

11. The system of claim 9 , wherein:

the bot reply analyzer is further configured for:

comparing the degree of validity with the threshold, wherein the certain criterion comprises the degree of validity not exceeding the threshold, and

providing the reply as a response to the query and providing the query and the reply for training the model for bot recommendation, when the degree of validity exceeds the threshold; and

the second conversational bot is determined when the degree of validity does not exceed the threshold, indicating that the certain criterion has been met.

12. The system of claim 9 , wherein the conversational bot recommender comprises:

a user query parser configured for parsing the query to determine the one or more features when the reply meets the certain criterion;

a matching bot determiner configured for obtaining the model for bot recommendation; and

a bot switch instruction generator configured for determining the second conversational bot based on the one or more features, the ranked list of confidence scores associated with the plurality of conversational bots, the one or more thresholds determined based on the reply, and the model.

13. The system of claim 12 , wherein the plurality of conversational bots are further determined based on the model, the conversational bot recommender comprises:

a matching bot confidence score determiner configured for obtaining, for each of the plurality of conversational bots, a respective confidence score;

a matching bot ranker configured for ranking the plurality of conversational bots based on their respective confidence scores to obtain the ranked list of confidence scores; and

an original reply based threshold generator configured for determining the one or more thresholds based on the reply, wherein:

the bot switch instruction generator configured for selecting the second conversational bot from the plurality of conversational bots.

14. The system of claim 13 , further comprising:

a new bot generator configured for generating a new conversational bot based on the query when none of the confidence scores of the plurality of conversational bots passes the one or more thresholds.

15. A non-transitory machine-readable medium having information recorded thereon for facilitating a user-machine conversation, wherein the information, when read by a machine, causes the machine to perform operations comprising:

obtaining a query from a user, wherein the query is directed to a first conversational bot;

obtaining, from the first conversational bot, a reply in response to the query;

determining a degree of validity of the reply based on the reply and the query;

determining, in the event that the degree of validity of the reply meets a certain criterion, using a model for bot recommendation, a second conversational bot other than the first conversational bot, wherein the second conversational bot is selected from a plurality of conversational bots based on (i) one or more features of the query, (ii) a ranked list of confidence scores, each of which is associated with a corresponding one of the plurality of conversational bots, and (iii) one or more thresholds determined based on the reply, wherein the model is trained based on training data comprising historical query-reply pairs, each of which includes (i) a historical query from a conversational bot with an associated confidence score, and (ii) a historical reply corresponding to the historical query, wherein:

one or more of the historical query-reply pairs are selected based on their respective associated confidence scores and a threshold, and

one or more of the plurality of conversational bots handled the historical query in each of the historical query-reply pairs and provided a historical reply corresponding to the historical query, and

the training data is generated based on the historical query-reply pairs; and

directing the user-machine conversation to the second conversational bot with the query.

16. The medium of claim 15 , wherein determining the degree of validity comprises:

determining a degree of matching between the reply and the query;

obtaining a first confidence score related to the reply from the first conversational bot; and

determining the degree of validity of the reply based on the degree of matching and the first confidence score.

17. The medium of claim 15 , wherein the operations further comprise:

comparing the degree of validity with the threshold, wherein the certain criterion comprises the degree of validity not exceeding the threshold;

providing the reply as a response to the query and providing the query and the reply for training the model for bot recommendation, when the degree of validity exceeds the threshold; and

determining the second conversational bot when the degree of validity does not exceed the threshold, indicating that the certain criterion has been met.

18. The medium of claim 15 , wherein determining the second conversational bot comprises:

parsing the query to determine the one or more features when the reply meets the certain criterion; and

obtaining the model for bot recommendation.

19. The medium of claim 18 , wherein the plurality of conversational bots are further determined based on the model, the operations further comprise:

obtaining, for each of the plurality of conversational bots, a respective confidence score, wherein the ranked list of confidence scores is determined by:

ranking the plurality of conversational bots based on their respective confidence scores to obtain the ranked list of confidence scores.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2021
From: VERIZON MEDIA INC.
To: VERIZON PATENT AND LICENSING INC.
Reel/Frame 057453/0431 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: OATH INC.
To: VERIZON MEDIA INC.
Reel/Frame 054258/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2018
From: YAHOO HOLDINGS, INC.
To: OATH INC.
Reel/Frame 045240/0310 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2017
From: YAHOO! INC.
To: YAHOO HOLDINGS, INC.
Reel/Frame 042963/0211 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2016
From: EMERY, MICHAEL; VISWANATHAN, LAVANYA COLINJIVADI
To: YAHOO! INC.
Reel/Frame 040748/0811 →
Continuity (1)
Related Publication 20180181558A1 · Jun 28, 2018