IP Library › Granted Patent US 12,223,229
Granted Patent B2
US 12,223,229 · App. 18/403,313 · Granted Feb 11, 2025

Using user input to adapt search results provided for presentation to the user

Inventors: David Kogan (Natick, MA); Bryan Christopher Horling (Belmont, MA)
Assignee: GOOGLE LLC
G06F3/167G06F3/0484G06F16/248G06F16/285G06F16/3326G06F16/3329G06F16/338G06F16/951
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,223,229
App. No.
18/403,313
Granted
Feb 11, 2025
Kind
B2
Abstract

Methods, apparatus, and computer readable media related to interaction between a user and an automated assistant during a dialog between the user and the automated assistant. Some implementations are directed to adapting a graphical and/or audible presentation of search results provided by the automated assistant for presentation to the user. The adaptation may be in response to attribute(s), of one or more of the search results, referenced in spoken and/or typed textual input provided by the user during the dialog. Some of those implementations may enable a user to provide textual input to navigate the search results within the dialog and within resource and/or interface constraints associated with the dialog. Some of those implementations may additionally and/or alternatively enable adapting, based on textual input provided by a user to the automated assistant, when and/or whether search results having certain attributes are provided to the user by the automated assistant.

Claims (79)

1. A method implemented by one or more processors, comprising:

receiving a spoken input of a user, the spoken input being detected, via at least one microphone of a client computing device of the user, as part of a dialog between the user and an automated assistant implemented at least in part by one or more of the processors;

obtaining a search result that is responsive to the spoken input and that has an attribute, the attribute of the search result being one or more of: a name of an entity referenced in the search result, or a name of a source of the search result;

causing the search result to be visually rendered, via a display, that is of a limited size, of the client computing device, for presentation to the user;

in response to causing the search result to be visually rendered for presentation to the user, receiving further spoken input of the user, the further spoken input being detected, via the at least one microphone of the client computing device, as part of the dialog between the user and the automated assistant;

determining, based on processing the further spoken input, that the further spoken input references:

the attribute of the search result, and

a sentiment expressed by the user towards the attribute;

in response to determining that the further spoken input references the attribute of the search result and the sentiment expressed by the user towards the attribute:

determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, an additional search result, that is also responsive to the spoken input and that also has the attribute, to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

2. The method of claim 1 , wherein determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, comprises:

determining that the sentiment expressed by the user towards the attribute is a negative sentiment; and

in response to determining that the sentiment expressed by the user towards the attribute is a negative sentiment:

refraining from causing the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

3. The method of claim 1 , wherein determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, comprises:

determining that the sentiment expressed by the user towards the attribute is a positive sentiment; and

in response to determining that the sentiment expressed by the user towards the attribute is a positive sentiment:

causing the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

4. The method of claim 3 , wherein causing the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, comprises:

causing the additional search result to supplant, via the display, that is of the limited size, of the client computing device, the search result.

5. The method of claim 1 , wherein the additional search result has yet to be provided in the dialog when the further spoken input of the user is received.

6. The method of claim 5 ,

wherein the search result and the additional search result are included in a plurality of search results that are responsive to the spoken input;

wherein causing the search result to be visually rendered, via the display, that is of the limited size, of the client computing device, comprises:

causing the search result to be visually rendered according to an initial order of the plurality of search results that are responsive to the spoken input; and

wherein determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, comprises:

determining to provide the additional search result, in the dialog, in a manner that is counter to the initial order of the search results.

7. The method of claim 1 , wherein the client computing device is a mobile device of the user, and wherein the display, that is of the limited size, is a mobile device display of the mobile device.

8. The method of claim 1 , further comprising:

in response to receiving the spoken input of the user:

performing a voice-to-text conversion to convert the spoken input into a textual input; and

causing the textual input to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user as part of a transcript of the ongoing dialog.

9. The method of claim 8 , wherein causing the search result to be visually rendered for presentation to the user comprises:

causing the search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user as part of the transcript of the ongoing dialog.

10. The method of claim 1 , wherein causing the search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, comprises:

transmitting data to the client computing device that, when received by the client computing device, is operable to cause the client computing device to visually render the search result.

11. The method of claim 1 , wherein causing the search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, comprises:

causing visual rendering of the search result at the client computing device.

12. A system, comprising:

at least one processor; and

memory storing instructions that, when executed, cause the at least one processor to be operable to:

receive a spoken input of a user, the spoken input being detected, via at least one microphone of a client computing device of the user, as part of a dialog between the user and an automated assistant implemented at least in part by the at least one processor;

obtain a search result that is responsive to the spoken input and that has an attribute, the attribute of the search result being one or more of: a name of an entity referenced in the search result, or a name of a source of the search result;

cause the search result to be visually rendered, via a display, that is of a limited size, of the client computing device, for presentation to the user;

in response to causing the search result to be visually rendered for presentation to the user, receive further spoken input of the user, the further spoken input being detected, via the at least one microphone of the client computing device, as part of the dialog between the user and the automated assistant;

determine, based on processing the further spoken input, that the further spoken input references:

the attribute of the search result, and

a sentiment expressed by the user towards the attribute;

in response to determining that the further spoken input references the attribute of the search result and the sentiment expressed by the user towards the attribute:

determine, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, an additional search result, that is also responsive to the spoken input and that also has the attribute, to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

13. The system of claim 12 , wherein, in determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, the at least one processor is operable to:

determine that the sentiment expressed by the user towards the attribute is a negative sentiment; and

in response to determining that the sentiment expressed by the user towards the attribute is a negative sentiment:

refrain from causing the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

14. The system of claim 12 , wherein, in determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, the at least one processor is operable to:

determine that the sentiment expressed by the user towards the attribute is a positive sentiment; and

in response to determining that the sentiment expressed by the user towards the attribute is a positive sentiment:

cause the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

15. The system of claim 14 , wherein, in causing the additional search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, the at least one processor is operable to:

cause the additional search result to supplant, via the display, that is of the limited size, of the client computing device, the search result.

16. The system of claim 12 , wherein the client computing device is a mobile device of the user, and wherein the display, that is of the limited size, is a mobile device display of the mobile device.

17. The system of claim 12 , wherein the at least one processor is further operable to:

in response to receiving the spoken input of the user:

perform a voice-to-text conversion to convert the spoken input into a textual input; and

cause the textual input to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user as part of a transcript of the ongoing dialog.

18. The system of claim 17 , wherein, in causing the search result to be visually rendered for presentation to the user, the at least one processor is operable to:

cause the search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user as part of the transcript of the ongoing dialog.

19. The system of claim 12 , wherein, in causing the search result to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user, the at least one processor is operable to:

transmit data to the client computing device that, when received by the client computing device, is operable to cause the client computing device to visually render the search result.

20. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations, the operations comprising:

receiving a spoken input of a user, the spoken input being detected, via at least one microphone of a client computing device of the user, as part of a dialog between the user and an automated assistant implemented at least in part by one or more of the processors;

obtaining a search result that is responsive to the spoken input and that has an attribute, the attribute of the search result being one or more of: a name of an entity referenced in the search result, or a name of a source of the search result;

causing the search result to be visually rendered, via a display, that is of a limited size, of the client computing device, for presentation to the user;

in response to causing the search result to be visually rendered for presentation to the user, receiving further spoken input of the user, the further spoken input being detected, via the at least one microphone of the client computing device, as part of the dialog between the user and the automated assistant;

determining, based on processing the further spoken input, that the further spoken input references:

the attribute of the search result, and

a sentiment expressed by the user towards the attribute;

in response to determining that the further spoken input references the attribute of the search result and the sentiment expressed by the user towards the attribute:

determining, based on the attribute and the sentiment expressed by the user towards that attribute, whether to cause, as part of the dialog, an additional search result, that is also responsive to the spoken input and that also has the attribute, to be visually rendered, via the display, that is of the limited size, of the client computing device, for presentation to the user.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2024
From: KOGAN, DAVID; HORLING, BRYAN
To: GOOGLE INC.
Reel/Frame 066036/0624 →
CHANGE OF NAME Recorded Jan 5, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 066208/0194 →
Continuity (4)
Continuation 17363350 · Jun 30, 2021
Continuation 16591125 · Oct 2, 2019
Continuation 15252031 · Aug 30, 2016
Related Publication 20240168708A1 · May 23, 2024
References Cited (34)
US 9305064B1 · LeBeau et al. · 2016 [cited by applicant]
US 10481861B2 · Kogan et al. · 2019 [cited by applicant]
US 10546027B1 · Parent et al. · 2020 [cited by applicant]
US 11074038B2 · Kogan et al. · 2021 [cited by applicant]
US 20070112738A1 · Livaditis · 2007 [cited by examiner]
US 20080133488A1 · Bandaru · 2008 [cited by applicant]
US 20120109858A1 · Makadia · 2012 [cited by examiner]
US 20120290950A1 · Rapaport et al. · 2012 [cited by applicant]
US 20130111348A1 · Gruber et al. · 2013 [cited by applicant]
US 20130275164A1 · Gruber · 2013 [cited by examiner]
US 20150189726A1 · Spira · 2015 [cited by applicant]
US 20150286724A1 · Knaapen · 2015 [cited by applicant]
US 20160173578A1 · Sharma · 2016 [cited by applicant]
US 20160224036A1 · Baker · 2016 [cited by applicant]
US 20170068423A1 · Napolitano et al. · 2017 [cited by applicant]
US 20170108236A1 · Guan · 2017 [cited by applicant]
US 20180062871A1 · Jones · 2018 [cited by applicant]
US 20210326105A1 · Kogan et al. · 2021 [cited by applicant]
CN 101216829 · 2008 [cited by applicant]
CN 101847146 · 2010 [cited by applicant]
CN 102867060 · 2013 [cited by applicant]
CN 104641314A · 2015 [cited by applicant]
CN 105183848 · 2015 [cited by applicant]
CN 105247511 · 2016 [cited by applicant]
European Patent Office; Communication pursuant to Article 94(3) EPC issued in Application No. 17755396.3; 6 pages; dated Aug. 25, 2021. [cited by applicant]
China National Intellectual Property Administration; Notification of Notice of Allowance issue in Application No. 201710763373.8; 4 pages; dated Nov. 11, 2020. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action issue in Application No. 201710763373.8; 28 pages; dated Jun. 18, 2020. [cited by applicant]
European Patent Office—International Searching Authority; Notification of Transmittal of the International Search Report and the Written Opinion of PCT Serial No. PCT/US2017/046568; dated Nov. 21, 2017. [cited by applicant]
European Patent Office; Written Opinion of the International Preliminary Examining Authority of PCT Ser. No. PCT/US2017/046568; 7 pages; dated Jul. 10, 2018. [cited by applicant]
European Patent Office; International Preliminary Report on Patentability of PCT Ser. No. PCT/US2017/046568; 10 pages; dated Mar. 14, 2019. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action issued in Application No. 202011637525.8; 10 pages; dated Feb. 5, 2024. [cited by applicant]
European Patent Office; Summons to attend oral proceedings pursuant to Rule 115(1) EPC issued in Application No. 17755396.3; 8 pages; dated Jun. 27, 2024. [cited by applicant]
European Patent Office; Brief Communication—Preliminary Opinion issued in Application No. 17755396.3; 11 pages; dated Oct. 21, 2024. [cited by applicant]
China National Intellectual Property Administration; Notification of Second Office Action issued in Application No. 202011637525.8; 10 pages; dated Sep. 30, 2024. [cited by applicant]