IP Library › Granted Patent US 12,323,556
Granted Patent B2
US 12,323,556 · App. 18/375,857 · Granted Jun 3, 2025

Automatic navigation of an interactive voice response (IVR) tree on behalf of human user(s)

Inventors: Yuval Baror (Mountain View, CA); Michael Andrew Goodman (Oakland, CA); Yoav Tzur (Tel Aviv, IL)
Assignee: GOOGLE LLC
H04M3/4936G06F16/9027H04M3/5166
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,323,556
App. No.
18/375,857
Granted
Jun 3, 2025
Kind
B2
Abstract

Implementations are directed to utilizing an assistant to automatically navigate an interactive voice response (IVR) tree to arrive at a target state during an assisted telephone call. The assistant can receive input to initiate the assisted telephone call, identify an entity to engage with, on behalf of the user, and during the assisted telephone call, based on the input, and identify an IVR tree stored in association with the entity. In various implementations, navigation of the IVR tree can be modified based on interaction(s) detected at a client device subsequent to initiating the assisted telephone call. In various implementations, the assisted telephone call can be initiated from a search interface, and the target state can be associated with a given search result. In various implementations, the IVR tree can be dynamic in that only a subset of candidate state(s) of the IVR tree may be available as the target state.

Claims (64)

1. A method implemented by one or more processors, the method comprising:

receiving, from a given user and via a client device associated with the given user, user input to initiate an assisted telephone call;

identifying, based on the user input, an entity to engage with, on behalf of the given user, during the assisted telephone call;

identifying an interactive voice response (IVR) tree that is associated with the identified entity, the IVR tree including a plurality of candidate states;

initiating performance of the assisted telephone call; and

during performance of the assisted telephone call:

determining a corresponding confidence level associated with the IVR tree;

determining, based on the corresponding confidence level associated with the IVR tree, a set of the candidate states of the IVR tree to be provided for presentation to the user; and

causing the set of the candidate states of the IVR tree to be provided for presentation to the user.

2. The method of claim 1 , wherein the corresponding confidence level associated with the IVR tree satisfies a first threshold confidence level, and wherein determining the set of candidate states of the IVR tree to be provided for presentation to the user based on the corresponding confidence level associated with the IVR tree comprises:

determining the set of the candidate states of the IVR tree to be provided for presentation to the user includes each of the plurality of candidate states of the IVR tree.

3. The method of claim 2 , wherein causing the set of the candidate states of the IVR tree to be provided for presentation to the user comprises:

causing the set of the candidate states of the IVR tree to be provided for presentation to the user prior to an IVR system, that utilizes the IVR tree that is associated with the identified entity, rendering any synthesized speech that identifies any of the candidate states of the IVR tree.

4. The method of claim 1 , wherein the corresponding confidence level associated with a first subset of the plurality of candidate states of the IVR tree satisfies a first threshold confidence level, wherein the corresponding confidence level associated with a second subset of the plurality of candidate states of the IVR tree fails to satisfy the first threshold confidence level but satisfies a second threshold confidence level, and wherein determining the set of the candidate states of the IVR tree to be provided for presentation to the user based on the corresponding confidence level associated with the IVR tree comprises:

determining the set of the candidate states of the IVR tree to be provided for presentation to the user includes the first subset of the plurality of candidate states of the IVR tree, but not the second subset of the plurality of candidate states of the IVR tree.

5. The method of claim 4 , wherein causing the set of the candidate states of the IVR tree to be provided for presentation to the user comprises:

causing the first subset of the candidate states of the IVR tree to be provided for presentation to the user prior to an IVR system, that utilizes the IVR tree that is associated with the identified entity, rendering any synthesized speech that identifies any of the candidate states of the IVR tree.

6. The method of claim 5 , wherein causing the set of the candidate states of the IVR tree to be provided for presentation to the user further comprises:

subsequent to the IVR system rendering synthesized speech that identifies the first subset of the candidate states of the IVR tree:

causing the second subset of the candidate states of the IVR tree to be provided for presentation to the user.

7. The method of claim 4 , wherein the first subset of the candidate states of the IVR tree and the second subset of the candidate states of the IVR tree are mutually exclusive.

8. The method of claim 1 , wherein the corresponding confidence level associated with the IVR tree fails to satisfy a first threshold confidence level and also fails to satisfy a second threshold confidence level, and wherein determining the set of the candidate states of the IVR tree to be provided for presentation to the user based on the corresponding confidence level associated with the IVR tree comprises:

subsequent to an IVR system iteratively rendering corresponding synthesized speech that iteratively identifies each of the candidate states of the IVR tree:

iteratively determining the set of the candidate states of the IVR tree to be provided for presentation to the user includes each of the candidate states of the IVR tree.

9. The method of claim 8 , wherein causing the set of the candidate states of the IVR tree to be provided for presentation to the user further comprises:

iteratively causing the each of the candidate states of the IVR tree to be provided for presentation to the user.

10. The method of claim 1 , wherein the corresponding confidence level associated with the IVR tree fails to satisfy a first threshold confidence level and also fails to satisfy a second threshold confidence level, and wherein determining the set of the candidate states of the IVR tree to be provided for presentation to the user based on the corresponding confidence level associated with the IVR tree comprises:

subsequent to an IVR system, that utilizes the IVR tree that is associated with the identified entity, rendering corresponding synthesized speech that identifies each of the candidate states of the IVR tree:

determining the set of the candidate states of the IVR tree to be provided for presentation to the user includes each of the candidate states of the IVR tree.

11. The method of claim 10 , wherein causing the set of the candidate states of the IVR tree to be provided for presentation to the user further comprises:

causing each of the candidate states of the IVR tree to be provided for presentation to the user.

12. The method of claim 1 , wherein causing the set of the candidate states of the IVR tree to be provided for presentation to the user comprises:

causing a corresponding selectable graphical element associated with a corresponding one of the candidate states, included in the set of the candidate states of the IVR tree, to be provided for presentation to the user.

13. The method of claim 12 , further comprising:

receiving a user selection of a given corresponding selectable graphical element, from among the corresponding selectable graphical elements, the given corresponding selectable graphical element being associated with a given candidate state of the IVR tree, from among the plurality of candidate states of the IVR tree; and

automatically navigating the assisted telephone call to the given candidate state of the IVR tree that is associated with the given corresponding selectable graphical element.

14. The method of claim 12 , wherein each of the corresponding selectable graphical elements include textual content that differs from synthesized speech generated by an IVR system that utilizes the IVR tree that is associated with the identified entity.

15. The method of claim 1 , wherein the corresponding confidence level associated with the IVR tree is based on a quantity of assisted telephone calls with the identified entity that include navigating the IVR tree that is associated with the identified entity, but that differs from a stored IVR tree that is associated with the identified entity.

16. The method of claim 1 , further comprising:

causing a transcript of the assisted telephone call to be provided for presentation to the user.

17. The method of claim 16 , wherein the transcript of the assisted telephone call is provided for presentation to the user at a first portion of a graphical user interface of the client device, and wherein the set of the candidate states of the IVR tree is provided for presentation to the user at a second portion of the graphical user interface of the client device.

18. A system, the system comprising:

one or more processors; and

memory storing instructions that, when executed, the one or more processors are operable to:

receive, from a given user and via a client device associated with the given user, user input to initiate an assisted telephone call;

identify, based on the user input, an entity to engage with, on behalf of the given user, during the assisted telephone call;

identify an interactive voice response (IVR) tree that is associated with the identified entity, the IVR tree including a plurality of candidate states;

initiate performance of the assisted telephone call; and

during performance of the assisted telephone call:

determine a corresponding confidence level associated with the IVR tree;

determine, based on the corresponding confidence level associated with the IVR tree, a set of the candidate states of the IVR tree to be provided for presentation to the user; and

cause the set of the candidate states of the IVR tree to be provided for presentation to the user.

19. The system of claim 18 , wherein the instructions to cause the set of the candidate states of the IVR tree to be provided for presentation to the user comprise instructions to cause a corresponding selectable graphical element associated with a corresponding one of the candidate states, included in the set of the candidate states of the IVR tree, to be provided for presentation to the user, and wherein the one or more processors are further operable to:

receive a user selection of a given corresponding selectable graphical element, from among the corresponding selectable graphical elements, the given corresponding selectable graphical element being associated with a given candidate state of the IVR tree, from among the plurality of candidate states of the IVR tree; and

automatically navigate the assisted telephone call to the given candidate state of the IVR tree that is associated with the given corresponding selectable graphical element.

20. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to be operable to perform operations, the operations comprising:

receiving, from a given user and via a client device associated with the given user, user input to initiate an assisted telephone call;

identifying, based on the user input, an entity to engage with, on behalf of the given user, during the assisted telephone call;

identifying an interactive voice response (IVR) tree that is associated with the identified entity, the IVR tree including a plurality of candidate states;

initiating performance of the assisted telephone call; and

during performance of the assisted telephone call:

determining a corresponding confidence level associated with the IVR tree;

determining, based on the corresponding confidence level associated with the IVR tree, a set of the candidate states of the IVR tree to be provided for presentation to the user; and

causing the set of the candidate states of the IVR tree to be provided for presentation to the user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2023
From: BAROR, YUVAL; GOODMAN, MICHAEL ANDREW; TZUR, YOAV
To: GOOGLE LLC
Reel/Frame 065159/0047 →
Continuity (4)
Continuation 17690201 · Mar 9, 2022
Continuation 17068511 · Oct 12, 2020
Provisional Application 63088178 · Oct 6, 2020
Related Publication 20240031483A1 · Jan 25, 2024
References Cited (163)
US 5815566A · Ramot et al. · 1998 [cited by applicant]
US 6061433A · Polcyn et al. · 2000 [cited by applicant]
US 6304653B1 · O'Neil et al. · 2001 [cited by applicant]
US 6377567B1 · Leonard · 2002 [cited by applicant]
US 6418199B1 · Perrone · 2002 [cited by applicant]
US 6731725B1 · Merwin et al. · 2004 [cited by applicant]
US 6922465B1 · Howe · 2005 [cited by applicant]
US 7084758B1 · Cole · 2006 [cited by applicant]
US 7337158B2 · Fratkina et al. · 2008 [cited by applicant]
US 7539656B2 · Fratkina et al. · 2009 [cited by applicant]
US 7792773B2 · McCord et al. · 2010 [cited by applicant]
US 7920678B2 · Cooper et al. · 2011 [cited by applicant]
US 8345835B1 · Or-Bach et al. · 2013 [cited by applicant]
US 8594308B2 · Soundar · 2013 [cited by applicant]
US 8938058B2 · Soundar · 2015 [cited by applicant]
US 8964963B2 · Soundar · 2015 [cited by applicant]
US 9001819B1 · Or-Bach · 2015 [cited by examiner]
US 9098551B1 · Fryz · 2015 [cited by applicant]
US 9232369B1 · Fujisaki · 2016 [cited by applicant]
US 9318108B2 · Gruber et al. · 2016 [cited by applicant]
US 9467566B2 · Soundar · 2016 [cited by applicant]
US 9473637B1 · Venkatapathy et al. · 2016 [cited by applicant]
US 9721570B1 · Beal et al. · 2017 [cited by applicant]
US 10447860B1 · Hartman et al. · 2019 [cited by applicant]
US 11200893B2 · Kirazci et al. · 2021 [cited by applicant]
US 11303749B1 · Baror et al. · 2022 [cited by applicant]
US 11843718B2 · Baror · 2023 [cited by examiner]
US 20020051522A1 · Merrow et al. · 2002 [cited by applicant]
US 20020055975A1 · Petrovykh · 2002 [cited by applicant]
US 20030009530A1 · Philonenko · 2003 [cited by applicant]
US 20030063732A1 · Mcknight · 2003 [cited by applicant]
US 20040001575A1 · Tang · 2004 [cited by applicant]
US 20040083195A1 · McCord et al. · 2004 [cited by applicant]
US 20040120479A1 · Creamer · 2004 [cited by examiner]
US 20040193603A1 · Ljubicich · 2004 [cited by applicant]
US 20040213384A1 · Alles et al. · 2004 [cited by applicant]
US 20040240642A1 · Crandell et al. · 2004 [cited by applicant]
US 20050047559A1 · Colson · 2005 [cited by examiner]
US 20050147227A1 · Chervirala et al. · 2005 [cited by applicant]
US 20050175168A1 · Summe et al. · 2005 [cited by applicant]
US 20050271250A1 · Vallone et al. · 2005 [cited by applicant]
US 20060039365A1 · Ravikumar et al. · 2006 [cited by applicant]
US 20060056600A1 · Merrow et al. · 2006 [cited by applicant]
US 20060215824A1 · Mitby et al. · 2006 [cited by applicant]
US 20070036320A1 · Mandalia et al. · 2007 [cited by applicant]
US 20070201664A1 · Salafia · 2007 [cited by applicant]
US 20080181371A1 · Merrow et al. · 2008 [cited by applicant]
US 20080209449A1 · Maehira · 2008 [cited by applicant]
US 20080309449A1 · Martin et al. · 2008 [cited by applicant]
US 20090022293A1 · Routt · 2009 [cited by applicant]
US 20090029674A1 · Brezina et al. · 2009 [cited by applicant]
US 20090089096A1 · Schoenberg · 2009 [cited by applicant]
US 20090089100A1 · Nenov et al. · 2009 [cited by applicant]
US 20090137278A1 · Haru et al. · 2009 [cited by applicant]
US 20090154666A1 · Rios · 2009 [cited by examiner]
US 20090232295A1 · Ryskamp · 2009 [cited by applicant]
US 20100088613A1 · DeLuca et al. · 2010 [cited by applicant]
US 20100104087A1 · Byrd et al. · 2010 [cited by applicant]
US 20100124325A1 · Weng et al. · 2010 [cited by applicant]
US 20100228590A1 · Muller et al. · 2010 [cited by applicant]
US 20100272246A1 · Malik et al. · 2010 [cited by applicant]
US 20110092187A1 · Miller · 2011 [cited by applicant]
US 20110103559A1 · Andrews · 2011 [cited by applicant]
US 20110270687A1 · Bazaz · 2011 [cited by applicant]
US 20120016678A1 · Gruber et al. · 2012 [cited by applicant]
US 20120109759A1 · Oren et al. · 2012 [cited by applicant]
US 20120147762A1 · Hancock et al. · 2012 [cited by applicant]
US 20120157067A1 · Turner et al. · 2012 [cited by applicant]
US 20120173243A1 · Anand et al. · 2012 [cited by applicant]
US 20120271676A1 · Aravamudan et al. · 2012 [cited by applicant]
US 20130060587A1 · Bayrak et al. · 2013 [cited by applicant]
US 20130077772A1 · Lichorowic et al. · 2013 [cited by applicant]
US 20130090098A1 · Gidwani · 2013 [cited by applicant]
US 20130136248A1 · Kaiser-Nyman et al. · 2013 [cited by applicant]
US 20130163741A1 · Balasaygun et al. · 2013 [cited by applicant]
US 20130275164A1 · Gruber et al. · 2013 [cited by applicant]
US 20140024362A1 · Kang et al. · 2014 [cited by applicant]
US 20140029734A1 · Kim et al. · 2014 [cited by applicant]
US 20140037084A1 · Dutta · 2014 [cited by applicant]
US 20140107476A1 · Tung et al. · 2014 [cited by applicant]
US 20140122077A1 · Nishikawa et al. · 2014 [cited by applicant]
US 20140122618A1 · Duan · 2014 [cited by applicant]
US 20140200928A1 · Watanabe et al. · 2014 [cited by applicant]
US 20140207882A1 · Joo et al. · 2014 [cited by applicant]
US 20140247933A1 · Soundar · 2014 [cited by applicant]
US 20140279050A1 · Makar et al. · 2014 [cited by applicant]
US 20140280464A1 · De Ding et al. · 2014 [cited by applicant]
US 20140310365A1 · Sample et al. · 2014 [cited by applicant]
US 20150139413A1 · Hewitt et al. · 2015 [cited by applicant]
US 20150142704A1 · London · 2015 [cited by applicant]
US 20150150019A1 · Sheaffer et al. · 2015 [cited by applicant]
US 20150215455A1 · Perotti · 2015 [cited by examiner]
US 20150237203A1 · Siminoff · 2015 [cited by applicant]
US 20150248817A1 · Steir et al. · 2015 [cited by applicant]
US 20150281446A1 · Milstein et al. · 2015 [cited by applicant]
US 20150339707A1 · Harrison et al. · 2015 [cited by applicant]
US 20150347399A1 · Aue et al. · 2015 [cited by applicant]
US 20150350331A1 · Kumar · 2015 [cited by applicant]
US 20150358790A1 · Nasserbakht · 2015 [cited by applicant]
US 20160028891A1 · Pirat et al. · 2016 [cited by applicant]
US 20160105546A1 · Keys et al. · 2016 [cited by applicant]
US 20160139998A1 · Dunn et al. · 2016 [cited by applicant]
US 20160198045A1 · Kulkarni et al. · 2016 [cited by applicant]
US 20160227033A1 · Song · 2016 [cited by applicant]
US 20160227034A1 · Kulkarni et al. · 2016 [cited by applicant]
US 20160277569A1 · Shine et al. · 2016 [cited by applicant]
US 20160379230A1 · Chen · 2016 [cited by applicant]
US 20170037084A1 · Fasan · 2017 [cited by applicant]
US 20170039194A1 · Tschetter · 2017 [cited by applicant]
US 20170061091A1 · McElhinney et al. · 2017 [cited by applicant]
US 20170094052A1 · Zhang et al. · 2017 [cited by applicant]
US 20170177298A1 · Hardee et al. · 2017 [cited by applicant]
US 20170180499A1 · Gelfenbeyn et al. · 2017 [cited by applicant]
US 20170289332A1 · Lavian et al. · 2017 [cited by applicant]
US 20170358296A1 · Segalis et al. · 2017 [cited by applicant]
US 20170359463A1 · Segalis et al. · 2017 [cited by applicant]
US 20170359464A1 · Segalis et al. · 2017 [cited by applicant]
US 20170365277A1 · Park · 2017 [cited by applicant]
US 20180124241A1 · Vijaywargi · 2018 [cited by examiner]
US 20180124255A1 · Kawamura et al. · 2018 [cited by applicant]
US 20180133900A1 · Breazeal et al. · 2018 [cited by applicant]
US 20180220000A1 · Segalis et al. · 2018 [cited by applicant]
US 20180227416A1 · Segalis et al. · 2018 [cited by applicant]
US 20180227417A1 · Segalis et al. · 2018 [cited by applicant]
US 20180227418A1 · Segalis et al. · 2018 [cited by applicant]
US 20180255178A1 · Deng et al. · 2018 [cited by applicant]
US 20180316630A1 · Jacobson · 2018 [cited by applicant]
US 20190082043A1 · Lavian · 2019 [cited by examiner]
US 20190281159A1 · Segalis et al. · 2019 [cited by applicant]
US 20190306314A1 · Segalis et al. · 2019 [cited by applicant]
US 20200042597A1 · Wu · 2020 [cited by applicant]
US 20200314246A1 · Hart · 2020 [cited by applicant]
US 20220201119A1 · Baror et al. · 2022 [cited by applicant]
CN 103795877 · 2014 [cited by applicant]
CN 105592237 · 2016 [cited by applicant]
EP 1679693 · 2006 [cited by applicant]
JP 2004508748 · 2004 [cited by applicant]
JP 2007219385 · 2007 [cited by applicant]
JP 2007524928 · 2007 [cited by applicant]
JP 2008015439 · 2008 [cited by applicant]
JP 2009210703 · 2009 [cited by applicant]
JP 2010258855 · 2010 [cited by applicant]
JP 2015070371 · 2015 [cited by applicant]
JP 2017028346 · 2017 [cited by applicant]
JP 2017188886 · 2017 [cited by applicant]
KR 20130099423 · 2013 [cited by applicant]
KR 20140121105 · 2014 [cited by applicant]
KR 20170013884 · 2017 [cited by applicant]
WO 2007065193 · 2007 [cited by applicant]
WO 2012172416 · 2012 [cited by applicant]
WO 2014197635 · 2017 [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in Application No. 2022570378; 11 pages; dated Feb. 19, 2024. [cited by applicant]
European Patent Office; Communication pursuant to Article 94(3) EPC issued in Application No. 20 839 194.6 ; 10 pages; dated Feb. 29, 2024. [cited by applicant]
European Patent Office; Communication pursuant to Article 94(3) EPC issued in Application No. 20 839 194.6-1213; 8 pages; dated Sep. 5, 2023. [cited by applicant]
Intellectual Property India; First Examination Report issued in Application No. 202227062864; 10 pages; dated Oct. 9, 2023. [cited by applicant]
The Korean Intellectual Property Office; Notice of Office Action issued in Application No. 10-2022-7042494; 6 pages; dated Oct. 20, 2023. [cited by applicant]
European Patent Office; International Search Report and Written Opinion of Serial No. PCT/US2020/064918; 24 pages; dated Jul. 28, 2021. [cited by applicant]
European Patent Office; Invitation to Pay Additional Fees; Serial No. PCT/US2020/064918; 13 pages; dated Jun. 7, 2021. [cited by applicant]
Leviathan, Yaniv et al. Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone; Google AI Blog; 7 pages; dated May 8, 2018. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action issued in Application No. 202080100936.5; 27 pages; dated Jul. 25, 2024. [cited by applicant]
European Patent Office; Intention to Grant issued in Application No. 20 839 194.6 ; 98 pages; dated Nov. 8, 2024. [cited by applicant]
China National Intellectual Property Administration; Notice of Grant issued in Application No. 202080100936.5; 4 pages; dated Dec. 10, 2024. [cited by applicant]
The Korean Intellectual Property Office; Notice of Office Action issued in Application No. 10-2024-7000568; 6 pages; dated Dec. 9, 2024. [cited by applicant]