IP Library Granted Patent US 11,430,439
Granted Patent B2
US 11,430,439 · App. 16/935,605 · Granted Aug 30, 2022

System and method for providing assistance in a live conversation

Inventors: Ritesh Shreeshreemal (Noida, IN); Gaurav Chaurasia (Noida, IN)
G10L15/22G06N5/04G06N20/00G10L15/063G10L15/16G10L15/1815G10L15/30G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,430,439
App. No.
16/935,605
Filed
Jul 22, 2020
Granted
Aug 30, 2022
Kind
B2
Art Unit
2655
USPC
704/232
Abstract

Method for providing assistance in conversation including recognizing, by recognition module, conversation between primary user and at least one secondary user, identifying, by recognition module, first and second context data for primary user and at least one secondary user based on conversation; generating, by response generation module, at least one response on behalf of primary user based on at least one of second context data derived from at least one secondary user, and first context data; analyzing, by determining module, at least one action of primary user in at least one response on second context data; determining, by determining module, intervening situation in conversation based on at least one action; selecting, by intervening response module, intervening response from at least one response for determined intervening situation based on at least one action; and delivering, by response delivery module, intervening response to at least one secondary user during determined intervening situation.

Claims (34)

1. A method for operating an electronic device, the method comprising:

recognizing a conversation between a primary user and a secondary user by using at least one of a noise filtering technique, a speech recognition technique, a natural language processing technique, user profiling, and gestures of the primary user and the secondary user;

identifying first context data and second context data for the primary user and the secondary user from the conversation by using at least one of a deep learning recurrent neural network (RNN) technique, an artificial intelligence (AI) technique, and a natural learning processing (NLP) technique;

generating responses to be replied to the second user on behalf of the primary user based on at least one of the second context data derived from the secondary user, and the first context data;

generating scores corresponding to each of the generated responses based on pre-determined parameters with weights by using the AI and NLP techniques, the parameters including the primary user's response history, mood of the primary user, tone of the primary user, body language of the primary user, relationship between the primary user and the secondary user, privacy level of information required, location and environment, common comprehension ability, personalized training data of a virtual assistant communicatively coupled with the electronic device, and a type of the conversation;

analyzing at least one action of the primary user during the conversation;

determining an intervening situation in the conversation based on the analyzed at least one action, the intervening situation including a situation in which the primary user has not spoken within a threshold time, a situation in which the primary user has not completed a sentence, a situation in which the primary user utters neutral vowel sounds, a situation in which the primary user's statement has a significant deviation from a knowledge graph, a situation in which the primary user needs supporting and contradicting facts for his statement, a situation in which the primary user is busy in another task, a situation in which the primary user hesitated while speaking a response, and a situation in which the secondary user is not satisfied from the primary user's response;

selecting a response with a highest score among the generated responses as an intervening response based on the determined intervening situation; and

delivering the intervening response to the secondary user during the determined intervening situation.

2. The method of claim 1 , wherein the first context data includes conversation context related data.

3. The method of claim 1 , wherein the second context data includes purpose context related data.

4. The method of claim 1 , wherein delivering the intervening response comprises delivering the intervening response at a dynamic time interval.

5. The method of claim 1 , further comprises storing, in a database, pre-determined parameters, learning data, generated responses, intervening responses, context related data, primary user information, secondary user information, pre-defined relationship level between the primary user and the at least one secondary users, intimacy categories, and past conversation details.

6. The method of claim 1 , further comprising training the generated responses using learning data, and creating a privacy level for the primary user based on trained responses.

7. The method of claim 1 , wherein generating responses comprises at least one response including at least one of words, sentences, audio, video, environment settings, and augmented/virtual reality content.

8. The method of claim 1 , wherein delivering the intervening response comprises delivering the intervening response at a dynamic time interval to the secondary user.

9. An electronic device comprising:

a memory configured to store first pre-defined rules; and

a processor configured to cooperate with the memory and generate device processing commands based on the first pre-defined rules,

wherein the processor is further configured to:

recognize a conversation between a primary user and a secondary user by using at least one of a noise filtering technique, a speech recognition technique, a natural language processing technique, user profiling, and gestures of the primary user and the secondary user;

identify first context data and second context data for the primary user and the secondary user from the conversation by using at least one of a deep learning recurrent neural network (RNN) technique, an artificial intelligence (AI) technique, and a natural learning processing (NLP) technique;

generate responses to be replied to the second user on behalf of the primary user based on at least one of the second context data derived from the secondary user, and the first context data;

generate scores corresponding to each of the generated responses based on pre-determined parameters with weights by using the AI and NLP techniques, the parameters including the primary user's response history, mood of the primary user, tone of the primary user, body language of the primary user, relationship between the primary user and the secondary user, privacy level of information required, location and environment, common comprehension ability, personalized training data of a virtual assistant communicatively coupled with the electronic device, and a type of the conversation;

analyzing at least one action of the primary user during the conversation;

determine an intervening situation in the conversation based on the analyzed at least one action, the intervening situation including a situation in which the primary user has not spoken within a threshold time, a situation in which the primary user has not completed a sentence, a situation in which the primary user utters neutral vowel sounds, a situation in which the primary user's statement has a significant deviation from a knowledge graph, a situation in which the primary user needs supporting and contradicting facts for his statement, a situation in which the primary user is busy in another task, a situation in which the primary user hesitated while speaking a response, and a situation in which the secondary user is not satisfied from the primary user's response;

select a response with a highest score among the generated responses as an intervening response based on the determined intervening situation; and

deliver the intervening response to the secondary user during the determined intervening situation.

10. The electronic device of claim 9 , wherein the processor is further configured to deliver the intervening response at a dynamic time interval.

11. The electronic device of claim 9 , wherein the processor is further configured to deliver the intervening response at a dynamic time interval determined in real-time.

12. The electronic device of claim 9 , wherein the processor is further configured to store pre-determined parameters, learning data, generated responses, intervening responses, context related data, primary user information, secondary user information, pre-defined relationship level between the primary user and the at least one secondary users, intimacy categories, and past conversation details.

13. The electronic device of claim 12 , wherein the processor is further configured to train the generated responses using the learning data, and create a privacy level for the primary user based on the trained at least one response.

14. The electronic device claim 9 , wherein the processor is configured to deliver the intervening response at a dynamic time interval to the secondary user.

15. The electronic device of claim 9 , further comprising a display for displaying an additional content in a form of the intervening response, and update availability of the additional content to the primary user the and the secondary user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 10, 2020
From: SHREESHREEMAL, RITESH; CHAURASIA, GAURAV
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 053444/0309 →
Priority Claims (1)
IN 201911033883 · Aug 22, 2019 · national
Continuity (1)
Related Publication 20210056968A1 · Feb 25, 2021
Cited By (2)
US 12,430,513 US 12,711,427