IP Library › Granted Patent US 12,373,487
Granted Patent B2
US 12,373,487 · App. 18/225,560 · Granted Jul 29, 2025

Consolidation of responses from queries to disparate data sources

Inventors: Anuj Thakkar (Mountain View, CA); Chaitrali Amrutkar (Mountain View, CA); Ognjen Nikolic (Mountain View, CA); Jason Douglas (Mountain View, CA); Ilya Firman (Sunnyvale, CA); Steve Chen (Mountain View, CA)
Assignee: GOOGLE LLC
G06F16/634G06F3/165G06F16/9532
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,373,487
App. No.
18/225,560
Granted
Jul 29, 2025
Kind
B2
Abstract

Consolidation of disparate data source queries is provided. An application intake component can receive an application package file, and process the file to identify one or more actions and entities for the application. A natural language processor component can receive an input audio signal to identify an action and an entity on which to perform the action. An application delivery component can identify applications that can perform the action and applications that have access to the entity, and then select an application that is configured to perform the action on the entity. The application delivery component can provide an indication of the application selected by the system to a client device.

Claims (56)

1. A method implemented by one or more processors of a client device, the method comprising:

detecting, via a sensor of the client device, an input audio signal;

transmitting, to a data processing system, data that is based on the input audio signal;

receiving, from the data processing system, an indication of at least one application, an entity, and an action that is performable by the at least one application and with respect to the entity; and

in response to receiving the indication of the at least one application, the entity, and the action that is performable by the at least one application and with respect to the entity:

launching the at least one application to perform the action that is performable with respect to the entity, wherein:

the data processing system parses the input audio signal and queries a plurality of data sources to identify the action and the entity on which to perform the action;

the data processing system selects the at least one application, in response to receiving the data that is based on the input audio signal, based on an action-inventory file that identifies the action as performable by the at least one application and with respect to the entity, and

the at least one application is configured with access to the entity identified by the data processing system and is configured to perform the action identified by the data processing system.

2. The method according to claim 1 , wherein the indication of the at least one application comprises a deep link that causes the application to launch and open or execute the entity within the at least one application.

3. The method according to claim 1 , wherein:

the at least one application is selected based on action-inventory files, including the action-inventory file; and

each of the action-inventory files is processed to identify one or more electronic actions performed by a respective application associated with a respective one of the action-inventory files and corresponding one or more entities on which the respective application performs the one or more electronic actions.

4. The method according to claim 3 , wherein the action-inventory files comprise an extensible markup language file.

5. The method according to claim 3 , wherein the action-inventory files comprise a pointer to a list of entities.

6. The method according to claim 1 , wherein:

a web search is executed via a search engine based on at least one of the action and the entity identified from the input audio signal; and

the at least one application is further identified based on results of the web search output by the search engine.

7. A computer program product comprising one or more non-transitory computer-readable storage media having program instructions collectively stored on the one or more non-transitory computer-readable storage media, the program instructions executable to:

detect, via a sensor of a client device, an input audio signal;

transmit, to a data processing system, data that is based on the input audio signal;

receive, from the data processing system, an indication of at least one application, an entity, and an action that is performable by the at least one application and with respect to the entity; and

in response to receiving the indication of the at least one application, the entity, and the action that is performable by the at least one application and with respect to the entity:

launch the at least one application to perform the action that is performable with respect to the entity, wherein:

the data processing system parses the input audio signal and queries a plurality of data sources to identify the action and the entity on which to perform the action;

the data processing system selects the at least one application, in response to receiving the data that is based on the input audio signal, based on an action-inventory file that identifies the action as performable by the at least one application and with respect to the entity, and

the at least one application is configured with access to the entity identified by the data processing system and is configured to perform the action identified by the data processing system.

8. The computer program product according to claim 7 , wherein the indication of the at least one application comprises a deep link that causes the application to launch and open or execute the entity within the at least one application.

9. The computer program product according to claim 7 , wherein:

the at least one application is selected based on action-inventory files, including the action-inventory file; and

each of the action-inventory files is processed to identify one or more electronic actions performed by a respective application associated with a respective one of the action-inventory files and corresponding one or more entities on which the respective application performs the one or more electronic actions.

10. The computer program product according to claim 9 , wherein the action-inventory files comprise an extensible markup language file.

11. The computer program product according to claim 9 , wherein the action-inventory files comprise a pointer to a list of entities.

12. The computer program product according to claim 7 , wherein:

a web search is executed via a search engine based on at least one of the action and the entity identified from the input audio signal; and

the at least one application is further identified based on results of the web search output by the search engine.

13. A system comprising:

a processor, a computer-readable memory, one or more non-transitory computer-readable storage media, and program instructions collectively stored on the one or more non-transitory computer-readable storage media, the program instructions executable to:

detect, via a sensor of a client device, an input audio signal;

transmit, to a data processing system, data that is based on the input audio signal;

receive, from the data processing system, an indication of at least one application, an entity, and an action that is performable by the at least one application and with respect to the entity; and

in response to receiving the indication of the at least one application, the entity, and the action that is performable by the at least one application and with respect to the entity:

launch the at least one application to perform the action that is performable with respect to the entity,

wherein:

the data processing system parses the input audio signal and queries a plurality of data sources to identify the action and the entity on which to perform the action;

the data processing system selects the at least one application, in response to receiving the data that is based on the input audio signal, based on an action-inventory file that identifies the action as performable by the at least one application and with respect to the entity, and

the at least one application is configured with access to the entity identified by the data processing system and is configured to perform the action identified by the data processing system.

14. The system according to claim 13 , wherein the indication of the at least one application comprises a deep link that causes the application to launch and open or execute the entity within the at least one application.

15. The system according to claim 13 , wherein:

the at least one application is selected based on action-inventory files, including the action-inventory file; and

each of the action-inventory files is processed to identify one or more electronic actions performed by a respective application associated with a respective one of the action-inventory files and corresponding one or more entities on which the respective application performs the one or more electronic actions.

16. The system according to claim 15 , wherein the action-inventory files comprise an extensible markup language file.

17. The system according to claim 15 , wherein the action-inventory files comprise a pointer to a list of entities.

18. The system according to claim 13 , wherein:

a web search is executed via a search engine based on at least one of the action and the entity identified from the input audio signal; and

the at least one application is further identified based on results of the web search output by the search engine.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2023
From: THAKKAR, ANUJ; AMRUTKAR, CHAITRALI; NIKOLIC, OGNJEN; DOUGLAS, JASON; FIRMAN, ILYA; CHEN, STEVE
To: GOOGLE LLC
Reel/Frame 064378/0960 →
Continuity (4)
Continuation 17840264 · Jun 14, 2022
Continuation 16324605
Provisional Application 62770338 · Nov 21, 2018
Related Publication 20230401253A1 · Dec 14, 2023
References Cited (56)
US 8612470B1 · Fushman · 2013 [cited by applicant]
US 11429665B2 · Thakkar et al. · 2022 [cited by applicant]
US 20030084096A1 · Starbuck et al. · 2003 [cited by applicant]
US 20030145317A1 · Chamberlain · 2003 [cited by applicant]
US 20070067272A1 · Flynt · 2007 [cited by examiner]
US 20110276568A1 · Fotev · 2011 [cited by applicant]
US 20120084292A1 · Liang et al. · 2012 [cited by applicant]
US 20120173520A1 · Wu · 2012 [cited by examiner]
US 20120240236A1 · Wyatt et al. · 2012 [cited by applicant]
US 20120284247A1 · Jiang et al. · 2012 [cited by applicant]
US 20130191397A1 · Avadhanam et al. · 2013 [cited by applicant]
US 20130226892A1 · Ehsani · 2013 [cited by applicant]
US 20130331141A1 · Montemurro · 2013 [cited by applicant]
US 20140006374A1 · Forte et al. · 2014 [cited by applicant]
US 20140006418A1 · Forte et al. · 2014 [cited by applicant]
US 20140188482A1 · Chang · 2014 [cited by examiner]
US 20140189572A1 · Martens · 2014 [cited by examiner]
US 20140250106A1 · Shapira et al. · 2014 [cited by applicant]
US 20140278419A1 · Bishop et al. · 2014 [cited by applicant]
US 20140282493A1 · Glover · 2014 [cited by applicant]
US 20140379334A1 · Fry · 2014 [cited by applicant]
US 20150370874A1 · Bansal et al. · 2015 [cited by applicant]
US 20170060966A1 · Glover et al. · 2017 [cited by applicant]
US 20170169506A1 · Wishne · 2017 [cited by applicant]
US 20170352352A1 · Wang et al. · 2017 [cited by applicant]
US 20180018576A1 · Boyer et al. · 2018 [cited by applicant]
US 20180225375A1 · Collins · 2018 [cited by applicant]
US 20180253757A1 · Foladare et al. · 2018 [cited by applicant]
US 20210334305A1 · Thakkar et al. · 2021 [cited by applicant]
CN 105074700 · 2015 [cited by applicant]
CN 105579823 · 2016 [cited by applicant]
CN 108369583 · 2018 [cited by applicant]
CN 108541312 · 2018 [cited by applicant]
JP 2014170185 · 2014 [cited by applicant]
JP 2018511095 · 2018 [cited by applicant]
WO 2018204288 · 2018 [cited by applicant]
WO 2020106314 · 2020 [cited by applicant]
The Korean Intellectual Property Office; Notice of Office Action issued in Application No. 1020227028442; 8 pages; dated Sep. 18, 2023. [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in Application No. 2022137985; 4 pages; dated Oct. 23, 2023. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action issued in Application No. 201980002151.1; 29 pages; dated Mar. 18, 2023. [cited by applicant]
The Korean Intellectual Property Office; Notice of Office Action issued in Application No. 10-2022-7028442; 12 pages; dated Dec. 9, 2022. [cited by applicant]
European Patent Office; Minutes in accordance with Rule 124(4) EPC issued in Application No. 19707184.8, 16 pages, dated Dec. 12, 2022. [cited by applicant]
European Patent Office; Communication under Rule 71(3) EPC issued in Application No. 19707184.8, 21 pages, dated Jan. 2, 2023. [cited by applicant]
European Patent Office; Result of Consultation issued in Application No. 19707184.8, 19 pages, dated Nov. 29, 2022. [cited by applicant]
Japanese Patent Office; Notice of Allowance issued in app. No. 2020-553517, 3 pages, dated Aug. 7, 2022. [cited by applicant]
European Patent Office; Summons to attend oral proceedings pursuant to Rule 115(1) issued in Application No. 19707184.8, 11 pages, dated Jul. 5, 2022. [cited by applicant]
The Korean Intellectual Property Office; Allowance of Patent issued in Application No. 10-2020-7030954; 5 pages; dated May 19, 2022. [cited by applicant]
European Patent Office; Communication Pursuant to Article 94(3) EPC issue in Application No. 19707184.8; 9 pages; dated Feb. 9, 2021. [cited by applicant]
European Patent Office; International Search Report and Written Opinion of the International Searching Aughority on PCT Application No. PCT/US2019016917; 18 pages; dated Apr. 30, 2019. [cited by applicant]
The Korean Intellectual Property Office; Notice of Office Action issued in Application No. 10-2020-7030954; 14 pages; dated Nov. 18, 2021. [cited by applicant]
Intellectual Property India; Examination Report issued in Application No. 202027053613; 7 pages; dated Jul. 14, 2021. [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in Application No. 2020-553517; 8 pages; dated Jan. 11, 2022. [cited by applicant]
Intellectual Property Inda; Office Action issued in Application No. 202027053613; 7 pages; dated Jul. 14, 2021. [cited by applicant]
China National Intellectual Property Administration; Notice of Grant issued in Application No. 201980002151.1; 6 pages; dated Dec. 25, 2023. [cited by applicant]
Intellectual Property India; Hearing Notice issued in Application No. 202027053613; 3 pages; dated May 1, 2024. [cited by applicant]
Intellectual Property India; Examination Report issued in Application No. 202428045056; 8 pages; dated Apr. 11, 2025 . [cited by applicant]