IP Library › Granted Patent US 11,468,282

Granted Patent B2

US 11,468,282 · App. 16/595,108 · Granted Oct 11, 2022

Virtual assistant in a communication session

Inventors: Mehul K. Sanghavi (Sunnyvale, CA); Jeffrey P. Schwerdtfeger (Santa Cruz, CA)

Assignee: Apple Inc.

G06N3/006 G06F40/30 G06F40/35 G06Q10/10 H04L51/02 H04L51/212

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,468,282
App. No.: 16/595,108
Granted: Oct 11, 2022
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Disclosed are systems, methods, and non-transitory computer-readable storage media for utilizing a virtual assistant as part of a communication session. One or more of the participant users can select to utilize a virtual assistant to assist the participant users with tasks during the communication session. A user can use a communication application to enter a message directed to the virtual assistant. The virtual assistant can analyze the entered message and determine that the message was directed to the virtual assistant rather than to the other participants of the communication session. As a result, the message will not be transmitted to the other participants of the communication session and the virtual assistant will assist the user with the identified task. A virtual assistant can assist a user with a variety of different tasks. For example, the virtual assistant can gather requested data for the user from one or more web servers, such as weather data, traffic data, etc.

Claims (150)

1. A non-transitory computer-readable storage medium containing instructions that, when executed by a computer processor of a first client device, cause the first client device to:

display, by the first client device, as part of a communication session that takes place within a messaging application between a first user operating the first client device and at least a second user, a first message prepared by the first user;

determine, by a virtual assistant of the first client device, based on a semantic analysis of the first message, that the first message identifies a first task with which the virtual assistant can assist the first user;

in response to determining that the first message identifies the first task, assist, by the first client device, the first user with the first task, wherein assisting the first user with the first task includes:

displaying, as part of the communication session and within the messaging application, a response generated by the virtual assistant based on the first task, wherein the response is not transmitted to at least the second user; and

receive a user selection of the displayed response;

in response to receiving the user selection of the displayed response, present, on the first client device and as part of the communication session, additional content generated by the virtual assistant based on the displayed response.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

display, by the first client device, as part of the communication session, a second message prepared by the first user;

determine, by the virtual assistant of the first client device, based on a semantic analysis of the second message, whether the second message is directed to the virtual assistant or to at least the second user;

determine, by the virtual assistant of the first client device, based on the semantic analysis of the second message, that the second message is directed to at least the second user; and

in response to determining that the second message is directed to at least the second user, transmit, by the first client device, the second message to at least the second user as part of the communication session.

3. The non-transitory computer-readable storage medium of claim 2 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

determine, by the virtual assistant of the first client device, based on the semantic analysis of the second message, that the second message identifies a second task with which the virtual assistant can assist the first user; and

in response to determining that the second message identifies the second task, assist, by the first client device, the first user with the second task.

4. The non-transitory computer-readable storage medium of claim 3 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

present, on the first client device and as part of the communication session, a request message asking the first user whether the first user would like the virtual assistant to assist the first user with the second task; and

receive an input indicating that the first user would like the virtual assistant to assist the first user with the second task.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the displayed response includes one or more words determined based on the first task.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

generate a second message based on the displayed response; and

transmit the second message to at least the second user.

7. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

identify, by the virtual assistant of the first client device, a plurality of applications installed on the first client device that are capable of completing the first task;

present, on the first client device and as part of the communication session, a request message prompting the first user to select an application from the plurality of applications to complete the first task;

receive an input from the first user selecting a first application from the plurality of applications;

in response to receiving the input from the first user, execute the selected first application to perform the first task; and

present, on the first client device and as part of the communication session, a confirmation message indicating the first task was performed.

8. The non-transitory computer-readable storage medium of claim 1 , wherein the first task comprises retrieving information associated with the first message.

9. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

transmit, to a second client device as part of the communication session, a notification message indicating that the virtual assistant is being utilized in relation to the communication session, wherein the second client device is associated with the second user.

10. The non-transitory computer-readable storage medium of claim 1 , wherein the first task comprises performing a financial transaction between the first user and at least the second user.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

identify, by the virtual assistant of the first client device, a set of financial applications installed on both the first client device and a second client device that are capable of completing the financial transaction, wherein the second client device is associated with the second user;

transmit, to the second client device, a second message as part of the communication session, the second message prompting the second user to select a financial application from the set of financial applications to complete the financial transaction;

receive, from the second client device, a first reply message identifying the financial application selected from the set of financial applications selected by the second user to complete the financial transaction; and

execute the financial application.

12. The non-transitory computer-readable storage medium of claim 1 , wherein the first task comprises scheduling a meeting including the first user and at least the second user.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

identify, based on calendar data for the first user and at least the second user, a set of candidate times for the meeting;

place a temporary hold on the set of candidate times, wherein the set of candidate times appear booked in calendars of the first user and at least the second user while the temporary hold is in place;

present the first user and at least the second user with a poll to select from the set of candidate times, the poll presented as part of the communication session; and

schedule the meeting during one of the candidate times based on response data received from the first user and at least the second user to the poll.

14. The non-transitory computer-readable storage medium of claim 1 , wherein the first task comprises determining an estimated time of arrival for at least the second user.

15. The non-transitory computer-readable storage medium of claim 14 , wherein determining the estimated time of arrival for at least the second user comprises:

requesting authorization to access location information of at least the second user; and

in accordance with a determination that the second user selects to authorize access to the location information of at least the second user:

gathering the location information of at least the second user; and

displaying, based on the location information of at least the second user, the estimated time of arrival for at least the second user.

16. A method, comprising:

at a first client device with one or more processors, memory, and a display:

displaying, by the first client device, as part of a communication session that takes place within a messaging application between a first user operating the first client device and at least a second user, a first message prepared by the first user;

determining, by a virtual assistant of the first client device, based on a semantic analysis of the first message, that the first message identifies a first task with which the virtual assistant can assist the first user;

in response to determining that the first message identifies the first task, assisting, by the first client device, the first user with the first task, wherein assisting the first user with the first task includes:

receiving a user selection of the displayed response;

in response to receiving the user selection of the displayed response, presenting, on the first client device and as part of the communication session, additional content generated by the virtual assistant based on the displayed response.

17. The method of claim 16 , further comprising:

displaying, by the first client device, as part of the communication session, a second message prepared by the first user;

determining, by the virtual assistant of the first client device, based on a semantic analysis of the second message, whether the second message is directed to the virtual assistant or to at least the second user;

determining, by the virtual assistant of the first client device, based on the semantic analysis of the second message, that the second message is directed to at least the second user; and

in response to determining that the second message is directed to at least the second user, transmitting, by the first client device, the second message to at least the second user as part of the communication session.

18. The method of claim 17 , further comprising:

determining, by the virtual assistant of the first client device, based on the semantic analysis of the second message, that the second message identifies a second task with which the virtual assistant can assist the first user; and

in response to determining that the second message identifies the second task, assisting, by the first client device, the first user with the second task.

19. The method of claim 18 , further comprising:

presenting, on the first client device and as part of the communication session, a request message asking the first user whether the first user would like the virtual assistant to assist the first user with the second task; and

receiving an input indicating that the first user would like the virtual assistant to assist the first user with the second task.

20. The method of claim 16 , wherein the displayed response includes one or more words determined based on the first task.

21. The method of claim 16 , further comprising:

generating a second message based on the displayed response; and

transmitting the second message to at least the second user.

22. The method of claim 16 , further comprising:

identifying, by the virtual assistant of the first client device, a plurality of applications installed on the first client device that are capable of completing the first task;

presenting, on the first client device and as part of the communication session, a request message prompting the first user to select an application from the plurality of applications to complete the first task;

receiving an input from the first user selecting a first application from the plurality of applications;

in response to receiving the input from the first user, executing the selected first application to perform the first task; and

presenting, on the first client device and as part of the communication session, a confirmation message indicating the first task was performed.

23. The method of claim 16 , wherein the first task comprises retrieving information associated with the first message.

24. The method of claim 16 , further comprising:

transmitting, to a second client device as part of the communication session, a notification message indicating that the virtual assistant is being utilized in relation to the communication session, wherein the second client device is associated with the second user.

25. The method of claim 16 , wherein the first task comprises performing a financial transaction between the first user and at least the second user.

26. The method of claim 25 , further comprising:

identifying, by the virtual assistant of the first client device, a set of financial applications installed on both the first client device and a second client device that are capable of completing the financial transaction, wherein the second client device is associated with the second user;

transmitting, to the second client device, a second message as part of the communication session, the second message prompting the second user to select a financial application from the set of financial applications to complete the financial transaction;

receiving, from the second client device, a first reply message identifying the financial application selected from the set of financial applications selected by the second user to complete the financial transaction; and

executing the financial application.

27. The method of claim 16 , wherein the first task comprises scheduling a meeting including the first user and at least the second user.

28. The method of claim 27 , further comprising:

identifying, based on calendar data for the first user and at least the second user, a set of candidate times for the meeting;

placing a temporary hold on the set of candidate times, wherein the set of candidate times appear booked in calendars of the first user and at least the second user while the temporary hold is in place;

presenting the first user and at least the second user with a poll to select from the set of candidate times, the poll presented as part of the communication session; and

scheduling the meeting during one of the candidate times based on response data received from the first user and at least the second user to the poll.

29. The method of claim 16 , wherein the first task comprises determining an estimated time of arrival for at least the second user.

30. The method of claim 29 , wherein determining the estimated time of arrival for at least the second user comprises:

requesting authorization to access location information of at least the second user; and

in accordance with a determination that the second user selects to authorize access to the location information of at least the second user:

gathering the location information of at least the second user; and

displaying, based on the location information of at least the second user, the estimated time of arrival for at least the second user.

31. A first client device, comprising:

a computer processor; and

a memory containing instructions that, when executed by the computer processor, cause the first client device to:

receive a user selection of the displayed response;

32. The first client device of claim 31 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

display, by the first client device, as part of the communication session, a second message prepared by the first user;

determine, by the virtual assistant of the first client device, based on the semantic analysis of the second message, that the second message is directed to at least the second user; and

33. The first client device of claim 32 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

in response to determining that the second message identifies the second task, assist, by the first client device, the first user with the second task.

34. The first client device of claim 33 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

receive an input indicating that the first user would like the virtual assistant to assist the first user with the second task.

35. The first client device of claim 31 , wherein the displayed response includes one or more words determined based on the first task.

36. The first client device of claim 31 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

generate a second message based on the displayed response; and

transmit the second message to at least the second user.

37. The first client device of claim 31 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

identify, by the virtual assistant of the first client device, a plurality of applications installed on the first client device that are capable of completing the first task;

receive an input from the first user selecting a first application from the plurality of applications;

in response to receiving the input from the first user, execute the selected first application to perform the first task; and

present, on the first client device and as part of the communication session, a confirmation message indicating the first task was performed.

38. The first client device of claim 31 , wherein the first task comprises retrieving information associated with the first message.

39. The first client device of claim 31 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

40. The first client device of claim 31 , wherein the first task comprises performing a financial transaction between the first user and at least the second user.

41. The first client device of claim 40 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

execute the financial application.

42. The first client device of claim 31 , wherein the first task comprises scheduling a meeting including the first user and at least the second user.

43. The first client device of claim 42 , wherein the instructions, when executed by the computer processor, further cause the first client device to:

identify, based on calendar data for the first user and at least the second user, a set of candidate times for the meeting;

present the first user and at least the second user with a poll to select from the set of candidate times, the poll presented as part of the communication session; and

schedule the meeting during one of the candidate times based on response data received from the first user and at least the second user to the poll.

44. The first client device of claim 31 , wherein the first task comprises determining an estimated time of arrival for at least the second user.

45. The first client device of claim 44 , wherein determining the estimated time of arrival for at least the second user comprises:

requesting authorization to access location information of at least the second user; and

in accordance with a determination that the second user selects to authorize access to the location information of at least the second user:

gathering the location information of at least the second user; and

displaying, based on the location information of at least the second user, the estimated time of arrival for at least the second user.

Continuity (2)

Continuation 14713410 · May 15, 2015

Related Publication 20200104668A1 · Apr 2, 2020

Cited By (5)

US 12,321,709 US 12,340,181 US 12,348,465 US 12,563,008 US 12,592,899